From dan at codespeak.net Mon Nov 1 00:05:24 2010 From: dan at codespeak.net (dan at codespeak.net) Date: Mon, 1 Nov 2010 00:05:24 +0100 (CET) Subject: [pypy-svn] r78628 - pypy/extradoc/talk/pycon2011 Message-ID: <20101031230524.094F1282B90@codespeak.net> Author: dan Date: Mon Nov 1 00:05:22 2010 New Revision: 78628 Modified: pypy/extradoc/talk/pycon2011/pypy-optimizations.txt Log: Reworded a couple sentences and fixed a typo. Modified: pypy/extradoc/talk/pycon2011/pypy-optimizations.txt ============================================================================== --- pypy/extradoc/talk/pycon2011/pypy-optimizations.txt (original) +++ pypy/extradoc/talk/pycon2011/pypy-optimizations.txt Mon Nov 1 00:05:22 2010 @@ -7,30 +7,30 @@ how PyPy is organized, and what optimizations our JIT can do (and what it can't do) for your code. -This talk will also contain a brief part about status of the PyPy project, +This talk will also contain a brief section discussing status of the PyPy project, it's goals and achievements. Session type: Talk Classification: Discuss in depth Abstract: -The talk will explain a bit in details how a python interpreter works -internally and why some operations are costly. We'll go through several python -features, how they work, why they're slow in CPython and how we're fixing it. +The talk will detail how a python interpreter works internally and why +some operations are costly. We'll go through several python features, +how they work, why they're slow in CPython and how we're fixing it. The list of mentioned features is not exhaustive, however we will try to focus at least on the following: * Dynamic language - In Python code we have no known types, like a statically typed language. Even operations like "a + b" can do anything, unless we know - more about the code, especially associated types. + more about the code, and the types it is operating on. * Frame introspection - Frame objects need to be allocated for every function call, and all local variables are stored on the frame, and must be accessible from further down the call stack. PyPy uses a novel approach called "virtualizables" which makes it possible - to not allocate frames at all in most common cases. + to avoid frame allocation in most common cases. * Object model - All user defined Python objects have a dictionary which stores their attributes, as does every type. When Python does an attribute lookup @@ -39,11 +39,11 @@ In PyPy we use an approach similar to the one used by V8 with hidden classes (except more PyPy specific) called map dictionaries. XXX type versions? -* FFI calls - calling C from Python is costly and hard to optimize. In PyPy we - expse C APIs to Python code via ctypes, this part will explain how are we - going to handle ctypes calls to be fast. +* FFI calls - Calling C from Python is costly and hard to optimize. In PyPy we + expose C APIs to Python code via ctypes. This part explains how we can + optimize ctypes calls. -* `array` module - users of CPython's array module probably know it can save +* `array` module - Users of CPython's array module probably know it can save them quite a bit of memory, however it's also slower than using a list, due to the overhead of boxing and unboxing on every operations. Here we will tie everything together and describe how the ``array`` module is much faster with From fijal at codespeak.net Mon Nov 1 09:22:32 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 1 Nov 2010 09:22:32 +0100 (CET) Subject: [pypy-svn] r78629 - pypy/extradoc/talk/pycon2011 Message-ID: <20101101082232.C36AA282BF4@codespeak.net> Author: fijal Date: Mon Nov 1 09:22:30 2010 New Revision: 78629 Modified: pypy/extradoc/talk/pycon2011/pypy-optimizations.txt Log: Avoid explaining what are type dicts Modified: pypy/extradoc/talk/pycon2011/pypy-optimizations.txt ============================================================================== --- pypy/extradoc/talk/pycon2011/pypy-optimizations.txt (original) +++ pypy/extradoc/talk/pycon2011/pypy-optimizations.txt Mon Nov 1 09:22:30 2010 @@ -37,7 +37,7 @@ this requires at least two dictionary lookups. In PyPy we use an approach similar to the one used by V8 with hidden classes - (except more PyPy specific) called map dictionaries. XXX type versions? + (except more PyPy specific) called map dictionaries and other optimizations. * FFI calls - Calling C from Python is costly and hard to optimize. In PyPy we expose C APIs to Python code via ctypes. This part explains how we can From fijal at codespeak.net Mon Nov 1 10:07:55 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 1 Nov 2010 10:07:55 +0100 (CET) Subject: [pypy-svn] r78632 - pypy/extradoc/talk/pycon2011 Message-ID: <20101101090755.A9495282BF0@codespeak.net> Author: fijal Date: Mon Nov 1 10:07:54 2010 New Revision: 78632 Modified: pypy/extradoc/talk/pycon2011/pypy-optimizations.txt Log: A better title Modified: pypy/extradoc/talk/pycon2011/pypy-optimizations.txt ============================================================================== --- pypy/extradoc/talk/pycon2011/pypy-optimizations.txt (original) +++ pypy/extradoc/talk/pycon2011/pypy-optimizations.txt Mon Nov 1 10:07:54 2010 @@ -1,4 +1,4 @@ -Title: Optimizations in PyPy +Title: Why is Python slow and how PyPy can help? Description: PyPy is a virtual machine for Python, featuring an advanced just in time From david at codespeak.net Mon Nov 1 14:37:36 2010 From: david at codespeak.net (david at codespeak.net) Date: Mon, 1 Nov 2010 14:37:36 +0100 (CET) Subject: [pypy-svn] r78634 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . helper test Message-ID: <20101101133736.1C8C6282BF0@codespeak.net> Author: david Date: Mon Nov 1 14:37:34 2010 New Revision: 78634 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py Log: Better integration of the register allocator Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Mon Nov 1 14:37:34 2010 @@ -113,7 +113,7 @@ def _gen_path_to_exit_path(self, op, args, regalloc, fcond=c.AL): box = Box() - reg = regalloc.try_allocate_reg(box) + reg = regalloc.force_allocate_reg(box) # XXX free this memory mem = lltype.malloc(rffi.CArray(lltype.Char), (len(args)+5)*4, flavor='raw') i = 0 @@ -121,7 +121,7 @@ while(i < len(args)): if args[i]: if not isinstance(args[i], ConstInt): - curreg = regalloc.try_allocate_reg(args[i]) + curreg = regalloc.make_sure_var_in_reg(args[i]) mem[j] = chr(curreg.value) j += 1 else: @@ -147,6 +147,8 @@ # guards going to be patched are allways conditional if fcond != c.AL: op.getdescr()._arm_guard_reg = reg + else: + regalloc.possibly_free_var(reg) return memaddr def align(self): @@ -162,7 +164,7 @@ def gen_bootstrap_code(self, inputargs, regalloc, looptoken): regs = [] for i in range(len(inputargs)): - reg = regalloc.try_allocate_reg(inputargs[i]) + reg = regalloc.force_allocate_reg(inputargs[i]) addr = self.fail_boxes_int.get_addr_for_num(i) self.mc.gen_load_int(reg.value, addr) self.mc.LDR_ri(reg.value, reg.value) @@ -182,6 +184,7 @@ looptoken._arm_loop_code = loop_head fcond=c.AL for op in operations: + # XXX consider merging ops with next one if it is an adecuate guard opnum = op.getopnum() fcond = self.operations[opnum](self, op, regalloc, fcond) self.gen_func_epilog() @@ -226,7 +229,7 @@ # regalloc support def regalloc_mov(self, prev_loc, loc): - if isinstance(prev_loc, ConstInt): + if prev_loc.is_imm(): # XXX check size of imm for current instr self.mc.gen_load_int(loc.value, prev_loc.getint()) else: Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Mon Nov 1 14:37:34 2010 @@ -4,14 +4,12 @@ def gen_emit_op_unary_cmp(true_cond, false_cond): def f(self, op, regalloc, fcond): - arg = op.getarg(0) - reg = self._put_in_reg(arg, regalloc) - res = regalloc.try_allocate_reg(op.result) + reg = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) + res = regalloc.force_allocate_reg(op.result) self.mc.CMP_ri(reg.value, 0) self.mc.MOV_ri(res.value, 1, true_cond) self.mc.MOV_ri(res.value, 0, false_cond) - regalloc.possibly_free_var(reg) - regalloc.possibly_free_var(res) + regalloc.possibly_free_vars_for_op(op) return fcond return f @@ -22,33 +20,32 @@ arg0 = op.getarg(0) arg1 = op.getarg(1) - res = regalloc.try_allocate_reg(op.result) - if (commutative - and self._check_imm_arg(arg0, imm_size) - and not isinstance(arg1, ConstInt)): - reg = regalloc.try_allocate_reg(arg1) - ri_op(res.value, reg.value, imm=arg0.getint(), cond=fcond) - elif self._check_imm_arg(arg1, imm_size) and not isinstance(arg0, ConstInt): - reg = regalloc.try_allocate_reg(arg0) - ri_op(res.value, reg.value, imm=arg1.getint(), cond=fcond) + imm_a0 = self._check_imm_arg(arg0, imm_size) + imm_a1 = self._check_imm_arg(arg1, imm_size) + res = regalloc.force_allocate_reg(op.result) + if commutative and imm_a0: + l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=imm_a0) + l1 = regalloc.make_sure_var_in_reg(arg1) + ri_op(res.value, l1.value, imm=l0.getint(), cond=fcond) + elif imm_a1: + l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(arg1, imm_fine=True) + ri_op(res.value, l0.value, imm=l1.getint(), cond=fcond) else: - reg = self._put_in_reg(arg0, regalloc) - reg2 = self._put_in_reg(arg1, regalloc) - rr_op(res.value, reg.value, reg2.value) - regalloc.possibly_free_var(reg2) - - regalloc.possibly_free_var(res) - regalloc.possibly_free_var(reg) + l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(arg1, imm_fine=False) + rr_op(res.value, l0.value, l1.value) + regalloc.possibly_free_vars_for_op(op) return fcond return f def gen_emit_op_by_helper_call(opname): def f(self, op, regalloc, fcond): - arg1 = regalloc.make_sure_var_in_reg(op.getarg(0), selected_reg=r.r0) - arg2 = regalloc.make_sure_var_in_reg(op.getarg(1), selected_reg=r.r1) + arg1 = regalloc.make_sure_var_in_reg(op.getarg(0), selected_reg=r.r0, imm_fine=False) + arg2 = regalloc.make_sure_var_in_reg(op.getarg(1), selected_reg=r.r1, imm_fine=False) assert arg1 == r.r0 assert arg2 == r.r1 - res = regalloc.try_allocate_reg(op.result) + res = regalloc.force_allocate_reg(op.result) getattr(self.mc, opname)(fcond) self.mc.MOV_rr(res.value, r.r0.value, cond=fcond) regalloc.possibly_free_vars_for_op(op) @@ -64,19 +61,22 @@ else: arg0 = op.getarg(1) arg1 = op.getarg(0) - res = regalloc.try_allocate_reg(op.result) + res = regalloc.force_allocate_reg(op.result) # XXX consider swapping argumentes if arg0 is const - if self._check_imm_arg(arg1) and not isinstance(arg0, ConstInt): - reg = regalloc.try_allocate_reg(arg0) - self.mc.CMP_ri(reg.value, imm=arg1.getint(), cond=fcond) + imm_a0 = self._check_imm_arg(arg0) + imm_a1 = self._check_imm_arg(arg1) + if imm_a1 and not imm_a0: + l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(arg1) + self.mc.CMP_ri(l0.value, imm=l1.getint(), cond=fcond) else: - reg = self._put_in_reg(arg0, regalloc) - reg2 = self._put_in_reg(arg1, regalloc) - self.mc.CMP_rr(reg.value, reg2.value) - regalloc.possibly_free_var(reg2) + l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(arg1, imm_fine=False) + self.mc.CMP_rr(l0.value, l1.value) inv = c.get_opposite_of(condition) self.mc.MOV_ri(res.value, 1, cond=condition) self.mc.MOV_ri(res.value, 0, cond=inv) + regalloc.possibly_free_vars_for_op(op) return condition return f Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py Mon Nov 1 14:37:34 2010 @@ -1,5 +1,7 @@ class AssemblerLocation(object): pass + def is_imm(self): + return False class RegisterLocation(AssemblerLocation): @@ -8,3 +10,17 @@ def __repr__(self): return 'r%d' % self.value + +class ImmLocation(AssemblerLocation): + _immutable_ = True + def __init__(self, value): + self.value = value + + def getint(self): + return self.value + + def __repr__(self): + return "ImmedLoc(%d)" % (self.value) + + def is_imm(self): + return True Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Mon Nov 1 14:37:34 2010 @@ -21,81 +21,69 @@ def emit_op_int_add(self, op, regalloc, fcond): # assuming only one argument is constant - res = regalloc.try_allocate_reg(op.result) + res = regalloc.force_allocate_reg(op.result) a0 = op.getarg(0) a1 = op.getarg(1) imm_a0 = isinstance(a0, ConstInt) and (a0.getint() <= 0xFF or -1 * a0.getint() <= 0xFF) imm_a1 = isinstance(a1, ConstInt) and (a1.getint() <= 0xFF or -1 * a1.getint() <= 0xFF) - if imm_a0 or imm_a1: - if imm_a1: - reg = regalloc.try_allocate_reg(a0) - arg1 = a1 - elif imm_a0: - reg = regalloc.try_allocate_reg(a1) - arg1 = a0 - value = arg1.getint() - if value < 0: - self.mc.SUB_ri(res.value, reg.value, -1 * value) + l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=imm_a0) + l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=imm_a1) + if imm_a0: + imm_a0, imm_a1 = imm_a1, imm_a0 + a0, a1 = a1, a0 + if imm_a1: + l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=True) + if l1.getint() < 0: + self.mc.SUB_ri(res.value, l0.value, -1 * l1.getint()) else: - self.mc.ADD_ri(res.value, reg.value, value) + self.mc.ADD_ri(res.value, l0.value, l1.getint()) else: - r1 = regalloc.try_allocate_reg(a0) - r2 = regalloc.try_allocate_reg(a1) - self.mc.ADD_rr(res.value, r1.value, r2.value) + l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=False) + self.mc.ADD_rr(res.value, l0.value, l1.value) regalloc.possibly_free_vars_for_op(op) return fcond def emit_op_int_sub(self, op, regalloc, fcond): # assuming only one argument is constant - res = regalloc.try_allocate_reg(op.result) + res = regalloc.force_allocate_reg(op.result) a0 = op.getarg(0) a1 = op.getarg(1) imm_a0 = isinstance(a0, ConstInt) and (a0.getint() <= 0xFF or -1 * a0.getint() <= 0xFF) imm_a1 = isinstance(a1, ConstInt) and (a1.getint() <= 0xFF or -1 * a1.getint() <= 0xFF) - if imm_a0 or imm_a1: - if imm_a1: - reg = self._put_in_reg(a0, regalloc) - value = a1.getint() - if value < 0: - self.mc.ADD_ri(res.value, reg.value, -1 * value) - else: - self.mc.SUB_ri(res.value, reg.value, value) - elif imm_a0: - reg = self._put_in_reg(a1, regalloc) - value = a0.getint() - if value < 0: - self.mc.ADD_ri(res.value, reg.value, -1 * value) - self.mc.MVN_rr(res.value, res.value) - else: - # reverse substract ftw - self.mc.RSB_ri(res.value, reg.value, value) + l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=imm_a0) + l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=imm_a1) + if imm_a0: + value = l0.getint() + if value < 0: + # XXX needs a test + self.mc.ADD_ri(res.value, l1.value, -1 * value) + self.mc.MVN_rr(res.value, l1.value) + else: + # reverse substract ftw + self.mc.RSB_ri(res.value, l1.value, value) + elif imm_a1: + value = a1.getint() + if value < 0: + self.mc.ADD_ri(res.value, l0.value, -1 * value) + else: + self.mc.SUB_ri(res.value, l0.value, value) else: - r1 = regalloc.try_allocate_reg(a0) - r2 = regalloc.try_allocate_reg(a1) - self.mc.SUB_rr(res.value, r1.value, r2.value) + self.mc.SUB_rr(res.value, l0.value, l1.value) regalloc.possibly_free_vars_for_op(op) return fcond def emit_op_int_mul(self, op, regalloc, fcond): - res = regalloc.try_allocate_reg(op.result) - reg1 = self._put_in_reg(op.getarg(0), regalloc) - reg2 = self._put_in_reg(op.getarg(1), regalloc) + res = regalloc.force_allocate_reg(op.result) + reg1 = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) + reg2 = regalloc.make_sure_var_in_reg(op.getarg(1), imm_fine=False) self.mc.MUL(res.value, reg1.value, reg2.value) - regalloc.possibly_free_var(reg1) - regalloc.possibly_free_var(reg2) + regalloc.possibly_free_vars_for_op(op) return fcond - def _put_in_reg(self, box, regalloc): - if isinstance(box, ConstInt): - t = Box() - reg = regalloc.try_allocate_reg(t) - self.mc.gen_load_int(reg.value, box.getint()) - else: - reg = regalloc.try_allocate_reg(box) - return reg - emit_op_int_floordiv = gen_emit_op_by_helper_call('DIV') emit_op_int_mod = gen_emit_op_by_helper_call('MOD') emit_op_uint_floordiv = gen_emit_op_by_helper_call('UDIV') @@ -127,25 +115,21 @@ emit_op_int_is_zero = gen_emit_op_unary_cmp(c.EQ, c.NE) def emit_op_int_invert(self, op, regalloc, fcond): - arg = op.getarg(0) - reg = self._put_in_reg(arg, regalloc) - res = regalloc.try_allocate_reg(op.result) + reg = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) + res = regalloc.force_allocate_reg(op.result) self.mc.MVN_rr(res.value, reg.value) - regalloc.possibly_free_var(reg) - regalloc.possibly_free_var(res) + regalloc.possibly_free_vars_for_op(op) return fcond #XXX check for a better way of doing this def emit_op_int_neg(self, op, regalloc, fcond): arg = op.getarg(0) - reg = self._put_in_reg(arg, regalloc) - reg2 = self._put_in_reg(ConstInt(-1), regalloc) - res = regalloc.try_allocate_reg(op.result) + l0 = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(ConstInt(-1), imm_fine=False) + res = regalloc.force_allocate_reg(op.result) self.mc.MUL(res.value, reg.value, reg2.value) - regalloc.possibly_free_var(reg) - regalloc.possibly_free_var(reg2) - regalloc.possibly_free_var(res) + regalloc.possibly_free_vars([l0, l1, res]) return fcond class GuardOpAssembler(object): @@ -175,18 +159,15 @@ _mixin_ = True def emit_op_jump(self, op, regalloc, fcond): - tmp = Box() - tmpreg = regalloc.try_allocate_reg(tmp) registers = op.getdescr()._arm_arglocs for i in range(op.numargs()): - reg = regalloc.try_allocate_reg(op.getarg(i)) + #XXX we are assuming that every value is in a register for now + reg = regalloc.make_sure_var_in_reg(op.getarg(i), imm_fine=False) inpreg = registers[i] - # XXX only if every value is in a register self.mc.MOV_rr(inpreg.value, reg.value) loop_code = op.getdescr()._arm_loop_code - self.mc.gen_load_int(tmpreg.value, loop_code) - self.mc.MOV_rr(r.pc.value, tmpreg.value) - regalloc.possibly_free_var(tmpreg) + self.mc.LDR_ri(r.pc.value, r.pc.value, -4) + self.mc.write32(loop_code) return fcond def emit_op_finish(self, op, regalloc, fcond): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Mon Nov 1 14:37:34 2010 @@ -1,6 +1,7 @@ from pypy.jit.backend.llsupport.regalloc import FrameManager, \ RegisterManager, compute_vars_longevity from pypy.jit.backend.arm import registers as r +from pypy.jit.backend.arm import locations class ARMRegisterManager(RegisterManager): all_regs = r.all_regs @@ -16,30 +17,13 @@ for i in range(len(inputargs)): while enc[j] == '\xFE': j += 1 - self.try_allocate_reg(inputargs[i], r.all_regs[ord(enc[j])]) + self.force_allocate_reg(inputargs[i], selected_reg=r.all_regs[ord(enc[j])]) j += 1 def convert_to_imm(self, c): - return c + return locations.ImmLocation(c.value) class ARMFrameManager(FrameManager): @staticmethod def frame_pos(loc, type): pass - -#class RegAlloc(object): -# def __init__(self, assembler, translate_support_code=False): -# self.assembler = assembler -# self.translate_support_code = translate_support_code -# self.fm = None -# -# def _prepare(self, inputargs, operations): -# longevity = compute_vars_longevity(inputargs, operations) -# self.rm = ARMRegisterManager(longevity, self.fm) -# -# def prepare_loop(self, inputargs, operations, looptoken): -# self._prepare(inputargs, operations) -# -# def force_allocate_reg(self, v, forbidden_vars=[], selected_reg=None, -# need_lower_byte=False): -# return self.rm.force_allocate_reg(v, forbidden_vars, selected_reg, need_lower_byte) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py Mon Nov 1 14:37:34 2010 @@ -109,6 +109,18 @@ self.a.gen_func_epilog() assert run_asm(self.a) == 9 + def test_jump_with_inline_address(self): + self.a.gen_func_prolog() + self.a.mc.MOV_ri(r.r1.value, 1) + loop_head = self.a.mc.curraddr() + self.a.mc.ADD_ri(r.r1.value, r.r1.value, 1) + self.a.mc.CMP_ri(r.r1.value, 9) + self.a.mc.LDR_ri(r.pc.value, r.pc.value, imm=4, cond=c.NE) + self.a.mc.MOV_rr(r.r0.value, r.r1.value) + self.a.gen_func_epilog() + self.a.mc.write32(loop_head) + assert run_asm(self.a) == 9 + def test_call_python_func(self): functype = lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Signed)) From david at codespeak.net Mon Nov 1 15:59:59 2010 From: david at codespeak.net (david at codespeak.net) Date: Mon, 1 Nov 2010 15:59:59 +0100 (CET) Subject: [pypy-svn] r78635 - pypy/branch/arm-backend/pypy/jit/backend/arm/test Message-ID: <20101101145959.1B0CE282B9D@codespeak.net> Author: david Date: Mon Nov 1 15:59:57 2010 New Revision: 78635 Added: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_arch.py Log: Tests for eabi mod function Added: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_arch.py ============================================================================== --- (empty file) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_arch.py Mon Nov 1 15:59:57 2010 @@ -0,0 +1,17 @@ +from pypy.jit.backend.arm import arch + +def test_mod(): + assert arch.arm_int_mod(10, 2) == 0 + assert arch.arm_int_mod(11, 2) == 1 + assert arch.arm_int_mod(11, 3) == 2 + +def test_mod2(): + assert arch.arm_int_mod(-10, 2) == 0 + assert arch.arm_int_mod(-11, 2) == -1 + assert arch.arm_int_mod(-11, 3) == -2 + +def test_mod3(): + assert arch.arm_int_mod(10, -2) == 0 + assert arch.arm_int_mod(11, -2) == 1 + assert arch.arm_int_mod(11, -3) == 2 + From david at codespeak.net Mon Nov 1 16:00:26 2010 From: david at codespeak.net (david at codespeak.net) Date: Mon, 1 Nov 2010 16:00:26 +0100 (CET) Subject: [pypy-svn] r78636 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . test Message-ID: <20101101150026.2BEC6282B9D@codespeak.net> Author: david Date: Mon Nov 1 16:00:24 2010 New Revision: 78636 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py Log: Refactor branch and branch with link patters to methods Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Mon Nov 1 16:00:24 2010 @@ -101,9 +101,7 @@ # parameter to next procedure call self.mc.MOV_rr(r.r1.value, r.sp.value) # pass the current stack pointer as second param - self.mc.ADD_ri(r.lr.value, r.pc.value, 4) - self.mc.LDR_ri(r.pc.value, r.pc.value, -4) - self.mc.write32(rffi.cast(lltype.Signed, decode_registers_addr)) + self.mc.BL(rffi.cast(lltype.Signed, decode_registers_addr)) self.mc.MOV_rr(r.ip.value, r.r0.value) self.mc.LDM(r.sp.value, range(12), w=1) # XXX Replace with POP instr. someday @@ -139,9 +137,8 @@ n = self.cpu.get_fail_descr_number(op.getdescr()) self.encode32(mem, j+1, n) - self.mc.gen_load_int(r.lr.value, memaddr, cond=fcond) - self.mc.gen_load_int(reg.value, self._exit_code_addr, cond=fcond) - self.mc.MOV_rr(r.pc.value, reg.value, cond=fcond) + self.mc.gen_load_int(r.lr.value, memaddr, cond=fcond) # use lr to pass an argument + self.mc.B(self._exit_code_addr, fcond, reg) # This register is used for patching when assembling a bridge # guards going to be patched are allways conditional @@ -183,6 +180,7 @@ looptoken._arm_bootstrap_code = loop_start looptoken._arm_loop_code = loop_head fcond=c.AL + #print inputargs, operations for op in operations: # XXX consider merging ops with next one if it is an adecuate guard opnum = op.getopnum() @@ -222,10 +220,8 @@ def patch_trace(self, faildescr, bridge_addr): # XXX make sure there is enough space at patch target fcond = faildescr._arm_guard_cond - b = ARMv7InMemoryBuilder(faildescr._arm_guard_code, faildescr._arm_guard_code+100) - reg = faildescr._arm_guard_reg - b.gen_load_int(reg.value, bridge_addr, fcond) - b.MOV_rr(r.pc.value, reg.value, cond=fcond) + b = ARMv7InMemoryBuilder(faildescr._arm_guard_code, faildescr._arm_guard_size) + b.B(bridge_addr, fcond, some_reg=faildescr._arm_guard_reg) # regalloc support def regalloc_mov(self, prev_loc, loc): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Mon Nov 1 16:00:24 2010 @@ -12,16 +12,16 @@ def binary_helper_call(name): signature = getattr(arch, 'arm_%s_sign' % name) function = getattr(arch, 'arm_%s' % name) - def f(self, cond=cond.AL): + def f(self, c=cond.AL): """Generates a call to a helper function, takes its arguments in r0 and r1, result is placed in r0""" - self.ensure_can_fit(self.size_of_gen_load_int*2+3*WORD) - self.PUSH(range(2, 4), cond=cond) addr = rffi.cast(lltype.Signed, llhelper(signature, function)) - self.gen_load_int(reg.r2.value, addr, cond=cond) - self.gen_load_int(reg.lr.value, self.curraddr()+self.size_of_gen_load_int+WORD, cond=cond) - self.MOV_rr(reg.pc.value, reg.r2.value, cond=cond) - self.LDM(reg.sp.value, range(2, 4), w=1, cond=cond) # XXX Replace with POP instr. someday + if c == cond.AL: + self.BL(addr) + else: + self.PUSH(range(2, 4), cond=c) + self.BL(addr, cond=c, some_reg=reg.r2) + self.LDM(reg.sp.value, range(2, 4), w=1, cond=c) # XXX Replace with POP instr. someday return f class AbstractARMv7Builder(object): @@ -52,6 +52,30 @@ def BKPT(self, cond=cond.AL): self.write32(cond << 28 | 0x1200070) + def B(self, target, c=cond.AL, some_reg=None): + if c == cond.AL: + self.ensure_can_fit(2*WORD) + self.LDR_ri(reg.pc.value, reg.pc.value, -4) + self.write32(target) + else: + assert some_reg is not None + self.ensure_can_fit(self.size_of_gen_load_int+WORD) + self.gen_load_int(some_reg.value, target, cond=c) + self.MOV_rr(reg.pc.value, some_reg.value, cond=c) + + def BL(self, target, c=cond.AL, some_reg=None): + if c == cond.AL: + self.ensure_can_fit(3*WORD) + self.ADD_ri(reg.lr.value, reg.pc.value, 4) + self.LDR_ri(reg.pc.value, reg.pc.value, imm=-4) + self.write32(target) + else: + assert some_reg is not None + self.ensure_can_fit(self.size_of_gen_load_int*2+WORD) + self.gen_load_int(some_reg.value, target, cond=c) + self.gen_load_int(reg.lr.value, self.curraddr()+self.size_of_gen_load_int+WORD, cond=c) + self.MOV_rr(reg.pc.value, some_reg.value, cond=c) + DIV = binary_helper_call('int_div') MOD = binary_helper_call('int_mod') UDIV = binary_helper_call('uint_div') @@ -104,6 +128,11 @@ data = rffi.cast(PTR, start) self._init(data, map_size) + def ensure_can_fit(self, n): + """ensure after this call there is enough space for n instructions + in a contiguous memory chunk or raise an exception""" + if not self._pos + n < self._size: + raise ValueError class ARMv7Builder(AbstractARMv7Builder): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Mon Nov 1 16:00:24 2010 @@ -128,7 +128,7 @@ l0 = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) l1 = regalloc.make_sure_var_in_reg(ConstInt(-1), imm_fine=False) res = regalloc.force_allocate_reg(op.result) - self.mc.MUL(res.value, reg.value, reg2.value) + self.mc.MUL(res.value, l0.value, l1.value) regalloc.possibly_free_vars([l0, l1, res]) return fcond @@ -142,6 +142,7 @@ memaddr = self._gen_path_to_exit_path(op, op.getfailargs(), regalloc, fcond) descr._failure_recovery_code = memaddr descr._arm_guard_cond = fcond + descr._arm_guard_size = self.mc.curraddr() - descr._arm_guard_code def emit_op_guard_true(self, op, regalloc, fcond): assert fcond == c.LE @@ -166,8 +167,7 @@ inpreg = registers[i] self.mc.MOV_rr(inpreg.value, reg.value) loop_code = op.getdescr()._arm_loop_code - self.mc.LDR_ri(r.pc.value, r.pc.value, -4) - self.mc.write32(loop_code) + self.mc.B(loop_code, fcond) return fcond def emit_op_finish(self, op, regalloc, fcond): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py Mon Nov 1 16:00:24 2010 @@ -91,8 +91,7 @@ self.a.mc.CMP_ri(r.r1.value, 0) # z=0, z=1 self.a.mc.MOV_ri(r.r1.value, 0, cond=c.NE) self.a.mc.MOV_ri(r.r1.value, 7, cond=c.EQ) - self.a.mc.gen_load_int(r.r4.value, loop_head, cond=c.NE) - self.a.mc.MOV_rr(r.pc.value, r.r4.value, cond=c.NE) + self.a.mc.B(loop_head, c.NE, some_reg = r.r4) self.a.mc.MOV_rr(r.r0.value, r.r1.value) self.a.gen_func_epilog() assert run_asm(self.a) == 7 @@ -103,8 +102,7 @@ loop_head = self.a.mc.curraddr() self.a.mc.ADD_ri(r.r1.value, r.r1.value, 1) self.a.mc.CMP_ri(r.r1.value, 9) - self.a.mc.gen_load_int(r.r4.value, loop_head, cond=c.NE) - self.a.mc.MOV_rr(r.pc.value, r.r4.value, cond=c.NE) + self.a.mc.B(loop_head, c.NE, some_reg = r.r4) self.a.mc.MOV_rr(r.r0.value, r.r1.value) self.a.gen_func_epilog() assert run_asm(self.a) == 9 @@ -127,9 +125,7 @@ call_addr = rffi.cast(lltype.Signed, llhelper(functype, callme)) self.a.gen_func_prolog() self.a.mc.MOV_ri(r.r0.value, 123) - self.a.mc.gen_load_int(r.r1.value, call_addr) - self.a.mc.gen_load_int(r.lr.value, self.a.mc.curraddr()+self.a.mc.size_of_gen_load_int+WORD) - self.a.mc.MOV_rr(r.pc.value, r.r1.value) + self.a.mc.BL(call_addr) self.a.gen_func_epilog() assert run_asm(self.a) == 133 @@ -141,9 +137,7 @@ # call to div self.a.mc.PUSH(range(2, 12)) div_addr = rffi.cast(lltype.Signed, llhelper(arm_int_div_sign, arm_int_div)) - self.a.mc.gen_load_int(r.r10.value, div_addr) - self.a.mc.gen_load_int(r.lr.value, self.a.mc.curraddr()+self.a.mc.size_of_gen_load_int+WORD) - self.a.mc.MOV_rr(r.pc.value, r.r10.value) + self.a.mc.BL(div_addr, some_reg=r.r2) self.a.mc.LDM(r.sp.value, range(2, 12), w=1) # XXX Replace with POP instr. someday self.a.gen_func_epilog() assert run_asm(self.a) == 61 @@ -174,6 +168,17 @@ self.a.gen_func_epilog() assert run_asm(self.a) == -36 + + def test_bl_with_conditional_exec(self): + functype = lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Signed)) + call_addr = rffi.cast(lltype.Signed, llhelper(functype, callme)) + self.a.gen_func_prolog() + self.a.mc.MOV_ri(r.r0.value, 123) + self.a.mc.CMP_ri(r.r0.value, 1) + self.a.mc.BL(call_addr, c.NE, some_reg=r.r1) + self.a.gen_func_epilog() + assert run_asm(self.a) == 133 + def callme(inp): i = inp + 10 return i From hpk at codespeak.net Mon Nov 1 17:00:02 2010 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 1 Nov 2010 17:00:02 +0100 (CET) Subject: [pypy-svn] r78637 - pypy/trunk/pypy/doc Message-ID: <20101101160002.A0B4B282BF1@codespeak.net> Author: hpk Date: Mon Nov 1 17:00:00 2010 New Revision: 78637 Modified: pypy/trunk/pypy/doc/getting-started.txt Log: FWIW insert a link to a Belorussian translation from Paul Bukhovko Modified: pypy/trunk/pypy/doc/getting-started.txt ============================================================================== --- pypy/trunk/pypy/doc/getting-started.txt (original) +++ pypy/trunk/pypy/doc/getting-started.txt Mon Nov 1 17:00:00 2010 @@ -5,6 +5,11 @@ .. contents:: .. sectnum:: +translations: Belorussian_ (Nov 2010) by Movavi_ + +.. _Belorussian: http://www.movavi.com/opensource/getting-started-be +.. _Movavi: http://www.movavi.com/ + .. _howtopypy: What is PyPy ? @@ -18,6 +23,7 @@ translation process - as opposed to encoding low level details into the language implementation itself. `more...`_ + .. _Python: http://docs.python.org/ref .. _`more...`: architecture.html From hpk at codespeak.net Mon Nov 1 17:12:33 2010 From: hpk at codespeak.net (hpk at codespeak.net) Date: Mon, 1 Nov 2010 17:12:33 +0100 (CET) Subject: [pypy-svn] r78638 - pypy/trunk/pypy/doc Message-ID: <20101101161233.5827C282BFD@codespeak.net> Author: hpk Date: Mon Nov 1 17:12:31 2010 New Revision: 78638 Modified: pypy/trunk/pypy/doc/getting-started.txt Log: remove the link again, it's not as useful and linking to an unrelated commercial site from here feels a bit strange. Modified: pypy/trunk/pypy/doc/getting-started.txt ============================================================================== --- pypy/trunk/pypy/doc/getting-started.txt (original) +++ pypy/trunk/pypy/doc/getting-started.txt Mon Nov 1 17:12:31 2010 @@ -5,11 +5,6 @@ .. contents:: .. sectnum:: -translations: Belorussian_ (Nov 2010) by Movavi_ - -.. _Belorussian: http://www.movavi.com/opensource/getting-started-be -.. _Movavi: http://www.movavi.com/ - .. _howtopypy: What is PyPy ? From afa at codespeak.net Mon Nov 1 21:45:22 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 1 Nov 2010 21:45:22 +0100 (CET) Subject: [pypy-svn] r78639 - in pypy/branch/fast-forward/pypy/module/_hashlib: . test Message-ID: <20101101204522.8B9FA282C04@codespeak.net> Author: afa Date: Mon Nov 1 21:45:18 2010 New Revision: 78639 Modified: pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py Log: Fix hashlib module, when method name is given as uppercase. Modified: pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py (original) +++ pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py Mon Nov 1 21:45:18 2010 @@ -82,12 +82,12 @@ # XXX significantly harder to implement in another way. # Values are digest sizes in bytes return { - 'md5': 16, - 'sha1': 20, - 'sha224': 28, - 'sha256': 32, - 'sha384': 48, - 'sha512': 64, + 'md5': 16, 'MD5': 16, + 'sha1': 20, 'SHA1': 20, + 'sha224': 28, 'SHA224': 28, + 'sha256': 32, 'SHA256': 32, + 'sha384': 48, 'SHA384': 48, + 'sha512': 64, 'SHA512': 64, }.get(self.name, 0) def _block_size(self): @@ -96,12 +96,12 @@ # XXX and would be significantly harder to implement in # XXX another way. return { - 'md5': 64, - 'sha1': 64, - 'sha224': 64, - 'sha256': 64, - 'sha384': 128, - 'sha512': 128, + 'md5': 64, 'MD5': 64, + 'sha1': 64, 'SHA1': 64, + 'sha224': 64, 'SHA224': 64, + 'sha256': 64, 'SHA256': 64, + 'sha384': 128, 'SHA384': 128, + 'sha512': 128, 'SHA512': 128, }.get(self.name, 0) W_Hash.typedef = TypeDef( Modified: pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py (original) +++ pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py Mon Nov 1 21:45:18 2010 @@ -65,3 +65,9 @@ import _hashlib assert _hashlib.new('sha1', u'xxx').__class__.__name__ == 'HASH' + def test_uppercase(self): + import _hashlib + h = _hashlib.new('MD5') + assert h.digest_size == 16 + assert len(h.hexdigest()) == 32 + From afa at codespeak.net Tue Nov 2 11:18:31 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 2 Nov 2010 11:18:31 +0100 (CET) Subject: [pypy-svn] r78640 - in pypy/branch/fast-forward: lib_pypy/_ctypes pypy/module/test_lib_pypy/ctypes_tests Message-ID: <20101102101831.2FEA3282BFD@codespeak.net> Author: afa Date: Tue Nov 2 11:18:27 2010 New Revision: 78640 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_array.py pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_numbers.py Log: ctypes objects should not reallocate the buffer when __init__ is called again. Issue found in multiprocessing tests Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/array.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/array.py Tue Nov 2 11:18:27 2010 @@ -145,8 +145,12 @@ __metaclass__ = ArrayMeta _ffiargshape = 'P' - def __init__(self, *args): + def __new__(cls, *args): + self = _CData.__new__(cls, *args) self._buffer = self._ffiarray(self._length_, autofree=True) + return self + + def __init__(self, *args): for i, arg in enumerate(args): self[i] = arg Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py Tue Nov 2 11:18:27 2010 @@ -288,8 +288,12 @@ __metaclass__ = SimpleType _type_ = 'i' - def __init__(self, value=DEFAULT_VALUE): + def __new__(cls, *args, **kwds): + self = _CData.__new__(cls, *args, **kwds) self._buffer = self._ffiarray(1, autofree=True) + return self + + def __init__(self, value=DEFAULT_VALUE): if value is not DEFAULT_VALUE: self.value = value Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_array.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_array.py (original) +++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_array.py Tue Nov 2 11:18:27 2010 @@ -109,6 +109,13 @@ assert sz[:] == "foo" assert sz.value == "foo" + def test_init_again(self): + sz = (c_char * 3)() + addr1 = addressof(sz) + sz.__init__(*"foo") + addr2 = addressof(sz) + assert addr1 == addr2 + try: create_unicode_buffer except NameError: Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_numbers.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_numbers.py (original) +++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_numbers.py Tue Nov 2 11:18:27 2010 @@ -89,6 +89,13 @@ parm = byref(t()) assert ArgType == type(parm) + def test_init_again(self): + for t in signed_types + unsigned_types + float_types: + parm = t() + addr1 = addressof(parm) + parm.__init__(0) + addr2 = addressof(parm) + assert addr1 == addr2 def test_floats(self): # c_float and c_double can be created from From afa at codespeak.net Tue Nov 2 17:16:14 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 2 Nov 2010 17:16:14 +0100 (CET) Subject: [pypy-svn] r78641 - in pypy/branch/fast-forward/pypy: config module/_multiprocessing module/rctime rlib Message-ID: <20101102161614.A182B282C02@codespeak.net> Author: afa Date: Tue Nov 2 17:16:11 2010 New Revision: 78641 Modified: pypy/branch/fast-forward/pypy/config/pypyoption.py pypy/branch/fast-forward/pypy/module/_multiprocessing/__init__.py pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_semaphore.py pypy/branch/fast-forward/pypy/module/rctime/__init__.py pypy/branch/fast-forward/pypy/module/rctime/interp_time.py pypy/branch/fast-forward/pypy/rlib/rwin32.py Log: Implement interruptible time.sleep() on Windows. + _multiprocessing uses the same ConsoleCtrlHandler handler Modified: pypy/branch/fast-forward/pypy/config/pypyoption.py ============================================================================== --- pypy/branch/fast-forward/pypy/config/pypyoption.py (original) +++ pypy/branch/fast-forward/pypy/config/pypyoption.py Tue Nov 2 17:16:11 2010 @@ -62,7 +62,10 @@ -module_dependencies = {} +module_dependencies = { + '_multiprocessing': [('objspace.usemodules.rctime', True), + ('objspace.usemodules.thread', True)], + } module_suggests = { # the reason you want _rawffi is for ctypes, which # itself needs the interp-level struct module Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/__init__.py Tue Nov 2 17:16:11 2010 @@ -16,7 +16,3 @@ if sys.platform == 'win32': interpleveldefs['win32'] = 'interp_win32.win32_namespace(space)' - - def startup(self, space): - from pypy.module._multiprocessing.interp_semaphore import CounterState - space.fromcache(CounterState).startup(space) Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_semaphore.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_semaphore.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_semaphore.py Tue Nov 2 17:16:11 2010 @@ -27,67 +27,6 @@ 'ReleaseSemaphore', [rwin32.HANDLE, rffi.LONG, rffi.LONGP], rwin32.BOOL) - CtrlHandler_type = lltype.Ptr(lltype.FuncType([], rwin32.BOOL)) - _CreateEvent = rwin32.winexternal( - 'CreateEventA', [rffi.VOIDP, rwin32.BOOL, rwin32.BOOL, rwin32.LPCSTR], - rwin32.HANDLE) - _SetEvent = rwin32.winexternal( - 'SetEvent', [rwin32.HANDLE], rwin32.BOOL) - _ResetEvent = rwin32.winexternal( - 'ResetEvent', [rwin32.HANDLE], rwin32.BOOL) - - # This is needed because the handler function must have the "WINAPI" - # calling convention, which is not supported by lltype.Ptr. - eci = ExternalCompilationInfo( - separate_module_sources=[''' - #include - - static BOOL (*CtrlHandlerRoutine)( - DWORD dwCtrlType); - - static BOOL WINAPI winapi_CtrlHandlerRoutine( - DWORD dwCtrlType) - { - return CtrlHandlerRoutine(dwCtrlType); - } - - BOOL pypy_multiprocessing_setCtrlHandlerRoutine(BOOL (*f)(DWORD)) - { - CtrlHandlerRoutine = f; - SetConsoleCtrlHandler(winapi_CtrlHandlerRoutine, TRUE); - } - - '''], - export_symbols=['pypy_multiprocessing_setCtrlHandlerRoutine'], - ) - _setCtrlHandlerRoutine = rffi.llexternal( - 'pypy_multiprocessing_setCtrlHandlerRoutine', - [CtrlHandler_type], rwin32.BOOL, - compilation_info=eci) - - def ProcessingCtrlHandler(): - _SetEvent(globalState.sigint_event) - return 0 - - class GlobalState: - def __init__(self): - self.init() - - def init(self): - self.sigint_event = rwin32.NULL_HANDLE - - def startup(self, space): - # Initialize the event handle used to signal Ctrl-C - globalState.sigint_event = _CreateEvent( - rffi.NULL, True, False, rffi.NULL) - if globalState.sigint_event == rwin32.NULL_HANDLE: - raise wrap_windowserror( - space, rwin32.lastWindowsError("CreateEvent")) - if not _setCtrlHandlerRoutine(ProcessingCtrlHandler): - raise wrap_windowserror( - space, rwin32.lastWindowsError("SetConsoleCtrlHandler")) - - else: from pypy.rlib import rposix @@ -187,25 +126,12 @@ def handle_w(space, w_handle): return rffi.cast(SEM_T, space.uint_w(w_handle)) - class GlobalState: - def init(self): - pass - - def startup(self, space): - pass - -globalState = GlobalState() - class CounterState: def __init__(self, space): self.counter = 0 def _freeze_(self): self.counter = 0 - globalState.init() - - def startup(self, space): - globalState.startup(space) def getCount(self): value = self.counter @@ -250,10 +176,12 @@ start = _GetTickCount() while True: - handles = [self.handle, globalState.sigint_event] + from pypy.module.rctime.interp_time import State + interrupt_event = space.fromcache(State).get_interrupt_event() + handles = [self.handle, interrupt_event] # do the wait - _ResetEvent(globalState.sigint_event) + rwin32.ResetEvent(interrupt_event) res = rwin32.WaitForMultipleObjects(handles, timeout=msecs) if res != rwin32.WAIT_OBJECT_0 + 1: Modified: pypy/branch/fast-forward/pypy/module/rctime/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/rctime/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/rctime/__init__.py Tue Nov 2 17:16:11 2010 @@ -2,6 +2,8 @@ from pypy.interpreter.mixedmodule import MixedModule import os +_WIN = os.name == "nt" + class Module(MixedModule): applevel_name = 'time' @@ -27,6 +29,10 @@ } def startup(self, space): + if _WIN: + from pypy.module.rctime.interp_time import State + space.fromcache(State).startup(space) + # this machinery is needed to expose constants # that have to be initialized one time only from pypy.module.rctime import interp_time Modified: pypy/branch/fast-forward/pypy/module/rctime/interp_time.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/rctime/interp_time.py (original) +++ pypy/branch/fast-forward/pypy/module/rctime/interp_time.py Tue Nov 2 17:16:11 2010 @@ -14,6 +14,89 @@ _POSIX = os.name == "posix" _WIN = os.name == "nt" +if _WIN: + # Interruptible sleeps on Windows: + # We install a specific Console Ctrl Handler which sets an 'event'. + # time.sleep() will actually call WaitForSingleObject with the desired + # timeout. On Ctrl-C, the signal handler is called, the event is set, + # and the wait function exits. + from pypy.rlib import rwin32 + from pypy.interpreter.error import wrap_windowserror, wrap_oserror + from pypy.module.thread import ll_thread as thread + + # This is needed because the handler function must have the "WINAPI" + # calling convention, which is not supported by lltype.Ptr. + CtrlHandler_type = lltype.Ptr(lltype.FuncType([], rwin32.BOOL)) + eci = ExternalCompilationInfo( + separate_module_sources=[''' + #include + + static BOOL (*CtrlHandlerRoutine)( + DWORD dwCtrlType); + + static BOOL WINAPI winapi_CtrlHandlerRoutine( + DWORD dwCtrlType) + { + return CtrlHandlerRoutine(dwCtrlType); + } + + BOOL pypy_timemodule_setCtrlHandlerRoutine(BOOL (*f)(DWORD)) + { + CtrlHandlerRoutine = f; + SetConsoleCtrlHandler(winapi_CtrlHandlerRoutine, TRUE); + } + + '''], + export_symbols=['pypy_timemodule_setCtrlHandlerRoutine'], + ) + _setCtrlHandlerRoutine = rffi.llexternal( + 'pypy_timemodule_setCtrlHandlerRoutine', + [CtrlHandler_type], rwin32.BOOL, + compilation_info=eci) + + def ProcessingCtrlHandler(): + rwin32.SetEvent(globalState.interrupt_event) + # allow other default handlers to be called. + # Default Python handler will setup the + # KeyboardInterrupt exception. + return 0 + + class GlobalState: + def __init__(self): + self.init() + + def init(self): + self.interrupt_event = rwin32.NULL_HANDLE + + def startup(self, space): + # Initialize the event handle used to signal Ctrl-C + try: + globalState.interrupt_event = rwin32.CreateEvent( + rffi.NULL, True, False, rffi.NULL) + except WindowsError, e: + raise wrap_windowserror(space, e) + if not _setCtrlHandlerRoutine(ProcessingCtrlHandler): + raise wrap_windowserror( + space, rwin32.lastWindowsError("SetConsoleCtrlHandler")) + + globalState = GlobalState() + + class State: + def __init__(self, space): + self.main_thread = 0 + + def _freeze_(self): + self.main_thread = 0 + globalState.init() + + def startup(self, space): + self.main_thread = thread.get_ident() + globalState.startup(space) + + def get_interrupt_event(self): + return globalState.interrupt_event + + _includes = ["time.h"] if _POSIX: _includes.append('sys/time.h') @@ -166,9 +249,37 @@ errno = rposix.get_errno() return os.strerror(errno) -def sleep(secs): - pytime.sleep(secs) -sleep.unwrap_spec = [float] +if sys.platform != 'win32': + def sleep(space, secs): + pytime.sleep(secs) +else: + from pypy.rlib import rwin32 + from errno import EINTR + def _simple_sleep(space, secs, interruptible): + if secs == 0.0 or not interruptible: + pytime.sleep(secs) + else: + millisecs = int(secs * 1000) + interrupt_event = space.fromcache(State).get_interrupt_event() + rwin32.ResetEvent(interrupt_event) + rc = rwin32.WaitForSingleObject(interrupt_event, millisecs) + if rc == rwin32.WAIT_OBJECT_0: + # Yield to make sure real Python signal handler + # called. + pytime.sleep(0.001) + raise wrap_oserror(space, + OSError(EINTR, "sleep() interrupted")) + def sleep(space, secs): + # as decreed by Guido, only the main thread can be + # interrupted. + main_thread = space.fromcache(State).main_thread + interruptible = (main_thread == thread.get_ident()) + MAX = sys.maxint / 1000.0 # > 24 days + while secs > MAX: + _simple_sleep(space, MAX, interruptible) + secs -= MAX + _simple_sleep(space, secs, interruptible) +sleep.unwrap_spec = [ObjSpace, float] def _get_module_object(space, obj_name): w_module = space.getbuiltinmodule('time') Modified: pypy/branch/fast-forward/pypy/rlib/rwin32.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rwin32.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rwin32.py Tue Nov 2 17:16:11 2010 @@ -275,3 +275,16 @@ return res finally: lltype.free(handle_array, flavor='raw') + + _CreateEvent = winexternal( + 'CreateEventA', [rffi.VOIDP, BOOL, BOOL, LPCSTR], HANDLE) + def CreateEvent(*args): + handle = _CreateEvent(*args) + if handle == NULL_HANDLE: + raise lastWindowsError("CreateEvent") + return handle + SetEvent = winexternal( + 'SetEvent', [HANDLE], BOOL) + ResetEvent = winexternal( + 'ResetEvent', [HANDLE], BOOL) + From afa at codespeak.net Tue Nov 2 17:18:08 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 2 Nov 2010 17:18:08 +0100 (CET) Subject: [pypy-svn] r78642 - pypy/branch/fast-forward/pypy/module/_multiprocessing Message-ID: <20101102161808.4630D282C03@codespeak.net> Author: afa Date: Tue Nov 2 17:18:06 2010 New Revision: 78642 Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Log: Simplify error handling and use real exceptions Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Tue Nov 2 17:18:06 2010 @@ -13,17 +13,9 @@ PY_SSIZE_T_MAX = sys.maxint PY_SSIZE_T_MIN = -sys.maxint - 1 -MP_END_OF_FILE = -1002 -MP_EARLY_END_OF_FILE = -1003 -MP_BAD_MESSAGE_LENGTH = -1004 - -def mp_error(space, res): - return OperationError(space.w_ValueError, - space.wrap("MULTIPROCESSING")) - -def BufferTooShort(space): +def BufferTooShort(space, w_data): return OperationError(space.w_ValueError, - space.wrap("BUFFERTOOSHORT")) + space.wrap("BufferTooShort")) def w_handle(space, handle): return space.wrap(rffi.cast(rffi.INTPTR_T, handle)) @@ -101,13 +93,6 @@ res, newbuf = self.do_recv_string(space, maxlength) res = intmask(res) # XXX why? try: - if res < 0: - if res == MP_BAD_MESSAGE_LENGTH: - self.flags &= ~READABLE - if self.flags == 0: - self.close() - raise mp_error(space, res) - if newbuf: return space.wrap(rffi.charpsize2str(newbuf, res)) else: @@ -124,19 +109,9 @@ res, newbuf = self.do_recv_string(space, length - offset) res = intmask(res) # XXX why? try: - if res < 0: - if res == MP_BAD_MESSAGE_LENGTH: - self.flags &= ~READABLE - if self.flags == 0: - self.close() - raise mp_error(space, res) - - if res > length - offset: - raise BufferTooShort(space) if newbuf: - rwbuffer.setslice(offset, rffi.charpsize2str(newbuf, res)) - else: - rwbuffer.setslice(offset, rffi.charpsize2str(self.buffer, res)) + raise BufferTooShort(space, rffi.charpsize2str(newbuf, res)) + rwbuffer.setslice(offset, rffi.charpsize2str(self.buffer, res)) finally: if newbuf: rffi.free_charp(newbuf) @@ -165,12 +140,6 @@ res, newbuf = self.do_recv_string(space, PY_SSIZE_T_MAX) res = intmask(res) # XXX why? try: - if res < 0: - if res == MP_BAD_MESSAGE_LENGTH: - self.flags &= ~READABLE - if self.flags == 0: - self.close() - raise mp_error(space, res) if newbuf: w_received = space.wrap(rffi.charpsize2str(newbuf, res)) else: @@ -258,8 +227,12 @@ flavor='raw') self._recvall(space, rffi.cast(rffi.CCHARP, length_ptr), 4) length = intmask(length_ptr[0]) - if length > maxlength: - return MP_BAD_MESSAGE_LENGTH, lltype.nullptr(rffi.CCHARP.TO) + if length > maxlength: # bad message, close connection + self.flags &= ~READABLE + if self.flags == 0: + self.close() + raise OperationError(space.w_IOError, space.wrap( + "bad message length")) if length <= self.BUFFER_SIZE: self._recvall(space, self.buffer, length) @@ -291,9 +264,10 @@ count = len(data) if count == 0: if remaining == length: - raise mp_error(space, MP_END_OF_FILE) + raise OperationError(space.w_EOFError, space.w_None) else: - raise mp_error(space, MP_EARLY_END_OF_FILE) + raise OperationError(space.w_IOError, space.wrap( + "got end of file during message")) # XXX inefficient for i in range(count): buffer[i] = data[i] @@ -400,7 +374,7 @@ err = rwin32.GetLastError() if err == ERROR_BROKEN_PIPE: - return MP_END_OF_FILE, lltype.nullptr(rffi.CCHARP.TO) + raise OperationError(space.w_EOFError, space.w_None) elif err != ERROR_MORE_DATA: raise wrap_windowserror(space, WindowsError(err, "_ReadFile")) @@ -412,8 +386,12 @@ raise wrap_windowserror(space, rwin32.lastWindowsError()) length = intmask(read_ptr[0] + left_ptr[0]) - if length > maxlength: - return MP_BAD_MESSAGE_LENGTH, lltype.nullptr(rffi.CCHARP.TO) + if length > maxlength: # bad message, close connection + self.flags &= ~READABLE + if self.flags == 0: + self.close() + raise OperationError(space.w_IOError, space.wrap( + "bad message length")) newbuf = lltype.malloc(rffi.CCHARP.TO, length + 1, flavor='raw') for i in range(read_ptr[0]): From afa at codespeak.net Tue Nov 2 17:54:45 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 2 Nov 2010 17:54:45 +0100 (CET) Subject: [pypy-svn] r78643 - in pypy/branch/fast-forward/pypy/module/_multiprocessing: . test Message-ID: <20101102165445.A0B70282C05@codespeak.net> Author: afa Date: Tue Nov 2 17:54:43 2010 New Revision: 78643 Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_semaphore.py pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_win32.py Log: - Attach received data to multiprocessing.BufferTooShort exception - rewrite tests so they pass with -A Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Tue Nov 2 17:54:43 2010 @@ -14,8 +14,11 @@ PY_SSIZE_T_MIN = -sys.maxint - 1 def BufferTooShort(space, w_data): - return OperationError(space.w_ValueError, - space.wrap("BufferTooShort")) + w_builtins = space.getbuiltinmodule('__builtin__') + w_module = space.call_method( + w_builtins, '__import__', space.wrap("multiprocessing")) + w_BufferTooShort = space.getattr(w_module, space.wrap("BufferTooShort")) + return OperationError(w_BufferTooShort, w_data) def w_handle(space, handle): return space.wrap(rffi.cast(rffi.INTPTR_T, handle)) Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py Tue Nov 2 17:54:43 2010 @@ -1,12 +1,38 @@ import py import sys -from pypy.conftest import gettestobjspace +from pypy.conftest import gettestobjspace, option +from pypy.interpreter.gateway import interp2app class TestImport: def test_simple(self): from pypy.module._multiprocessing import interp_connection from pypy.module._multiprocessing import interp_semaphore +class AppTestBufferTooShort: + def setup_class(cls): + space = gettestobjspace(usemodules=('_multiprocessing', 'thread')) + cls.space = space + + if option.runappdirect: + def raiseBufferTooShort(data): + import multiprocessing + raise multiprocessing.BufferTooShort(data) + cls.w_raiseBufferTooShort = raiseBufferTooShort + else: + from pypy.module._multiprocessing import interp_connection + def raiseBufferTooShort(space, w_data): + raise interp_connection.BufferTooShort(space, w_data) + cls.w_raiseBufferTooShort = space.wrap( + interp2app(raiseBufferTooShort)) + + def test_exception(self): + import multiprocessing + try: + self.raiseBufferTooShort("data") + except multiprocessing.BufferTooShort, e: + assert isinstance(e, multiprocessing.ProcessError) + assert e.args == ("data",) + class BaseConnectionTest(object): def test_connection(self): rhandle, whandle = self.make_pair() @@ -32,16 +58,19 @@ if sys.platform != "win32": py.test.skip("win32 only") - space = gettestobjspace(usemodules=('_multiprocessing', 'thread')) - cls.space = space - - # stubs for some modules, - # just for multiprocessing to import correctly on Windows - w_modules = space.sys.get('modules') - space.setitem(w_modules, space.wrap('msvcrt'), space.sys) - space.setitem(w_modules, space.wrap('_subprocess'), space.sys) + if not option.runappdirect: + space = gettestobjspace(usemodules=('_multiprocessing', 'thread')) + cls.space = space + + # stubs for some modules, + # just for multiprocessing to import correctly on Windows + w_modules = space.sys.get('modules') + space.setitem(w_modules, space.wrap('msvcrt'), space.sys) + space.setitem(w_modules, space.wrap('_subprocess'), space.sys) + else: + import _multiprocessing - cls.w_make_pair = space.appexec([], """(): + cls.w_make_pair = cls.space.appexec([], """(): import multiprocessing def make_pair(): rhandle, whandle = multiprocessing.Pipe() Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_semaphore.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_semaphore.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_semaphore.py Tue Nov 2 17:54:43 2010 @@ -19,7 +19,7 @@ sem = SemLock(kind, value, maxvalue) assert sem.kind == kind assert sem.maxvalue == maxvalue - assert isinstance(sem.handle, int) + assert isinstance(sem.handle, (int, long)) assert sem._count() == 0 assert sem._get_value() == 1 Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_win32.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_win32.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_win32.py Tue Nov 2 17:54:43 2010 @@ -43,7 +43,8 @@ if e.args[0] != win32.ERROR_PIPE_CONNECTED: raise - exc = raises(WindowsError, win32.WaitNamedPipe, address, timeout=100) + timeout = 100 + exc = raises(WindowsError, win32.WaitNamedPipe, address, timeout) assert exc.value.winerror == 121 # ERROR_SEM_TIMEOUT win32.CloseHandle(readhandle) From afa at codespeak.net Tue Nov 2 18:23:52 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 2 Nov 2010 18:23:52 +0100 (CET) Subject: [pypy-svn] r78644 - pypy/branch/fast-forward/pypy/module/_multiprocessing Message-ID: <20101102172352.80A66282C09@codespeak.net> Author: afa Date: Tue Nov 2 18:23:50 2010 New Revision: 78644 Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Log: Fix translation Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Tue Nov 2 18:23:50 2010 @@ -113,7 +113,8 @@ res = intmask(res) # XXX why? try: if newbuf: - raise BufferTooShort(space, rffi.charpsize2str(newbuf, res)) + raise BufferTooShort(space, space.wrap( + rffi.charpsize2str(newbuf, res))) rwbuffer.setslice(offset, rffi.charpsize2str(self.buffer, res)) finally: if newbuf: From hakanardo at codespeak.net Tue Nov 2 19:04:50 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Tue, 2 Nov 2010 19:04:50 +0100 (CET) Subject: [pypy-svn] r78645 - in pypy/branch/jit-unroll-loops/pypy: jit/tl/spli/test module/array/benchmark Message-ID: <20101102180450.090AA282C0C@codespeak.net> Author: hakanardo Date: Tue Nov 2 19:04:49 2010 New Revision: 78645 Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/spli/test/test_jit.py pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py Log: We currently cant virtualize across bridges Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/spli/test/test_jit.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/tl/spli/test/test_jit.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/tl/spli/test/test_jit.py Tue Nov 2 19:04:49 2010 @@ -39,6 +39,7 @@ self.check_loops(new_with_vtable=0) def test_bridge(self): + py.test.skip('We currently cant virtualize across bridges') def f(a, b): total = 0 i = 0 @@ -54,6 +55,7 @@ self.check_loops(new_with_vtable=0) def test_bridge_bad_case(self): + py.test.skip('We currently cant virtualize across bridges') def f(a, b): i = 0 while i < 100: Modified: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile (original) +++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile Tue Nov 2 19:04:49 2010 @@ -19,6 +19,6 @@ @/usr/bin/time -f '%e s' ../../../translator/goal/pypy-c $^tst.py 2>&1 | tail -1 @echo -n 'pypy-trunk: ' @/usr/bin/time -f '%e s' ../../../../../trunk/pypy/translator/goal/pypy-c $^tst.py 2>&1 | tail -1 - @echo -n ' cpython: ' - @/usr/bin/time -f '%e s' python $^tst.py 2>&1 | tail -1 - @echo + #@echo -n ' cpython: ' + #@/usr/bin/time -f '%e s' python $^tst.py 2>&1 | tail -1 + #@echo Modified: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py Tue Nov 2 19:04:49 2010 @@ -7,12 +7,15 @@ assert len(self) == 65536 return array.__getitem__(self, i & 65535) +import sys def main(): buf = Circular() i = 10 sa = 0 - while i < 200000000: + # 1048576 + while i < 949999: sa += buf[i-2] + buf[i-1] + buf[i] + buf[i+1] + buf[i+2] + if i%100 == 0: sys.stderr.write('%d\n'%i) i += 1 return sa From hakanardo at codespeak.net Tue Nov 2 19:08:26 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Tue, 2 Nov 2010 19:08:26 +0100 (CET) Subject: [pypy-svn] r78646 - pypy/branch/jit-unroll-loops/pypy/module/array/benchmark Message-ID: <20101102180826.7AC96282C0D@codespeak.net> Author: hakanardo Date: Tue Nov 2 19:08:25 2010 New Revision: 78646 Modified: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py Log: Adjusted benchmark to test array Modified: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile (original) +++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile Tue Nov 2 19:08:25 2010 @@ -19,6 +19,6 @@ @/usr/bin/time -f '%e s' ../../../translator/goal/pypy-c $^tst.py 2>&1 | tail -1 @echo -n 'pypy-trunk: ' @/usr/bin/time -f '%e s' ../../../../../trunk/pypy/translator/goal/pypy-c $^tst.py 2>&1 | tail -1 - #@echo -n ' cpython: ' - #@/usr/bin/time -f '%e s' python $^tst.py 2>&1 | tail -1 - #@echo + @echo -n ' cpython: ' + @/usr/bin/time -f '%e s' python $^tst.py 2>&1 | tail -1 + @echo Modified: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py Tue Nov 2 19:08:25 2010 @@ -1,3 +1,6 @@ +import sys +sys.setcheckinterval(1<<20) + from array import array class Circular(array): def __new__(cls): @@ -12,8 +15,7 @@ buf = Circular() i = 10 sa = 0 - # 1048576 - while i < 949999: + while i < 200000000: sa += buf[i-2] + buf[i-1] + buf[i] + buf[i+1] + buf[i+2] if i%100 == 0: sys.stderr.write('%d\n'%i) i += 1 From afa at codespeak.net Tue Nov 2 19:15:10 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 2 Nov 2010 19:15:10 +0100 (CET) Subject: [pypy-svn] r78647 - in pypy/branch/fast-forward/pypy/module/_multiprocessing: . test Message-ID: <20101102181510.87509282C0A@codespeak.net> Author: afa Date: Tue Nov 2 19:15:09 2010 New Revision: 78647 Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py Log: Test and fix for the size of receiving buffer. Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Tue Nov 2 19:15:09 2010 @@ -42,7 +42,7 @@ return False def do_send_string(self, space, buffer, offset, size): raise NotImplementedError - def do_recv_string(self, space, maxlength): + def do_recv_string(self, space, buflength, maxlength): raise NotImplementedError def close(self): @@ -93,7 +93,8 @@ raise OperationError(space.w_ValueError, space.wrap("maxlength < 0")) - res, newbuf = self.do_recv_string(space, maxlength) + res, newbuf = self.do_recv_string( + space, self.BUFFER_SIZE, maxlength) res = intmask(res) # XXX why? try: if newbuf: @@ -109,7 +110,8 @@ rwbuffer = space.rwbuffer_w(w_buffer) length = rwbuffer.getlength() - res, newbuf = self.do_recv_string(space, length - offset) + res, newbuf = self.do_recv_string( + space, length - offset, PY_SSIZE_T_MAX) res = intmask(res) # XXX why? try: if newbuf: @@ -141,7 +143,8 @@ def recv(self, space): self._check_readable(space) - res, newbuf = self.do_recv_string(space, PY_SSIZE_T_MAX) + res, newbuf = self.do_recv_string( + space, self.BUFFER_SIZE, PY_SSIZE_T_MAX) res = intmask(res) # XXX why? try: if newbuf: @@ -226,7 +229,7 @@ finally: lltype.free(message, flavor='raw') - def do_recv_string(self, space, maxlength): + def do_recv_string(self, space, buflength, maxlength): length_ptr = lltype.malloc(rffi.CArrayPtr(rffi.UINT).TO, 1, flavor='raw') self._recvall(space, rffi.cast(rffi.CCHARP, length_ptr), 4) @@ -238,7 +241,7 @@ raise OperationError(space.w_IOError, space.wrap( "bad message length")) - if length <= self.BUFFER_SIZE: + if length <= buflength: self._recvall(space, self.buffer, length) return length, lltype.nullptr(rffi.CCHARP.TO) else: @@ -359,7 +362,7 @@ rffi.free_charp(charp) lltype.free(written_ptr, flavor='raw') - def do_recv_string(self, space, maxlength): + def do_recv_string(self, space, buflength, maxlength): from pypy.module._multiprocessing.interp_win32 import ( _ReadFile, _PeekNamedPipe, ERROR_BROKEN_PIPE, ERROR_MORE_DATA) from pypy.rlib import rwin32 @@ -371,7 +374,7 @@ flavor='raw') try: result = _ReadFile(self.handle, - self.buffer, min(self.BUFFER_SIZE, maxlength), + self.buffer, min(self.BUFFER_SIZE, buflength), read_ptr, rffi.NULL) if result: return read_ptr[0], lltype.nullptr(rffi.CCHARP.TO) @@ -394,8 +397,8 @@ self.flags &= ~READABLE if self.flags == 0: self.close() - raise OperationError(space.w_IOError, space.wrap( - "bad message length")) + raise OperationError(space.w_IOError, space.wrap( + "bad message length")) newbuf = lltype.malloc(rffi.CCHARP.TO, length + 1, flavor='raw') for i in range(read_ptr[0]): Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py Tue Nov 2 19:15:09 2010 @@ -53,6 +53,16 @@ assert rhandle.recv() == 1 assert rhandle.poll() == False + def test_read_into(self): + import array, multiprocessing + rhandle, whandle = self.make_pair() + + obj = [1, 2.0, "hello"] + whandle.send(obj) + buffer = array.array('b', [0]*10) + raises(multiprocessing.BufferTooShort, rhandle.recv_bytes_into, buffer) + assert rhandle.readable + class AppTestWinpipeConnection(BaseConnectionTest): def setup_class(cls): if sys.platform != "win32": From hakanardo at codespeak.net Tue Nov 2 19:38:26 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Tue, 2 Nov 2010 19:38:26 +0100 (CET) Subject: [pypy-svn] r78648 - in pypy/branch/jit-unroll-loops/pypy/module: array/benchmark pypyjit/test Message-ID: <20101102183826.87B09282C09@codespeak.net> Author: hakanardo Date: Tue Nov 2 19:38:24 2010 New Revision: 78648 Modified: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py Log: fixed test to look for the profiling guard in the preamble Modified: pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/circulartst.py Tue Nov 2 19:38:24 2010 @@ -17,7 +17,6 @@ sa = 0 while i < 200000000: sa += buf[i-2] + buf[i-1] + buf[i] + buf[i+1] + buf[i+2] - if i%100 == 0: sys.stderr.write('%d\n'%i) i += 1 return sa Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py Tue Nov 2 19:38:24 2010 @@ -131,27 +131,35 @@ return result def parse_loops(self, opslogfile): - from pypy.jit.tool.oparser import parse from pypy.tool import logparser assert opslogfile.check() log = logparser.parse_log_file(str(opslogfile)) parts = logparser.extract_category(log, 'jit-log-opt-') self.rawloops = [part for part in parts if not from_entry_bridge(part, parts)] - # skip entry bridges, they can contain random things - self.loops = [parse(part, no_namespace=True) for part in self.rawloops] - self.sliced_loops = [] # contains all bytecodes of all loops - self.total_ops = 0 - for loop in self.loops: - self.total_ops += len(loop.operations) + self.loops, self.sliced_loops, self.total_ops = \ + self.parse_rawloops(self.rawloops) + self.check_0_op_bytecodes() + self.rawentrybridges = [part for part in parts + if from_entry_bridge(part, parts)] + _, self.sliced_entrybridge, _ = \ + self.parse_rawloops(self.rawentrybridges) + + def parse_rawloops(self, rawloops): + from pypy.jit.tool.oparser import parse + loops = [parse(part, no_namespace=True) for part in rawloops] + sliced_loops = [] # contains all bytecodes of all loops + total_ops = 0 + for loop in loops: + total_ops += len(loop.operations) for op in loop.operations: if op.getopname() == "debug_merge_point": sliced_loop = BytecodeTrace() sliced_loop.bytecode = op.getarg(0)._get_str().rsplit(" ", 1)[1] - self.sliced_loops.append(sliced_loop) + sliced_loops.append(sliced_loop) else: sliced_loop.append(op) - self.check_0_op_bytecodes() + return loops, sliced_loops, total_ops def check_0_op_bytecodes(self): for bytecodetrace in self.sliced_loops: @@ -159,8 +167,12 @@ continue assert not bytecodetrace - def get_by_bytecode(self, name): - return [ops for ops in self.sliced_loops if ops.bytecode == name] + def get_by_bytecode(self, name, from_entry_bridge=False): + if from_entry_bridge: + sliced_loops = self.sliced_entrybridge + else: + sliced_loops = self.sliced_loops + return [ops for ops in sliced_loops if ops.bytecode == name] def print_loops(self): for rawloop in self.rawloops: @@ -471,6 +483,8 @@ bytecode, = self.get_by_bytecode("CALL_METHOD") assert len(bytecode.get_opnames("new_with_vtable")) == 1 # the forcing of the int assert len(bytecode.get_opnames("call")) == 1 # the call to append + assert len(bytecode.get_opnames("guard")) == 1 # guard for guard_no_exception after the call + bytecode, = self.get_by_bytecode("CALL_METHOD", True) assert len(bytecode.get_opnames("guard")) == 2 # guard for profiling disabledness + guard_no_exception after the call def test_range_iter(self): From afa at codespeak.net Tue Nov 2 21:17:16 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 2 Nov 2010 21:17:16 +0100 (CET) Subject: [pypy-svn] r78649 - pypy/trunk/lib_pypy/pypy_test Message-ID: <20101102201716.C7CDC282C0C@codespeak.net> Author: afa Date: Tue Nov 2 21:17:03 2010 New Revision: 78649 Modified: pypy/trunk/lib_pypy/pypy_test/test_ctypes_support.py Log: Fix test on Windows Modified: pypy/trunk/lib_pypy/pypy_test/test_ctypes_support.py ============================================================================== --- pypy/trunk/lib_pypy/pypy_test/test_ctypes_support.py (original) +++ pypy/trunk/lib_pypy/pypy_test/test_ctypes_support.py Tue Nov 2 21:17:03 2010 @@ -22,12 +22,11 @@ assert get_errno() == 0 def test_argument_conversion_and_checks(): - import ctypes - libc = ctypes.cdll.LoadLibrary("libc.so.6") - libc.strlen.argtypes = ctypes.c_char_p, - libc.strlen.restype = ctypes.c_size_t - assert libc.strlen("eggs") == 4 - + strlen = standard_c_lib.strlen + strlen.argtypes = [c_char_p] + strlen.restype = c_size_t + assert strlen("eggs") == 4 + # Should raise ArgumentError, not segfault - py.test.raises(ctypes.ArgumentError, libc.strlen, False) + py.test.raises(ArgumentError, strlen, False) From hakanardo at codespeak.net Tue Nov 2 21:36:13 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Tue, 2 Nov 2010 21:36:13 +0100 (CET) Subject: [pypy-svn] r78650 - pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test Message-ID: <20101102203613.B73F5282C05@codespeak.net> Author: hakanardo Date: Tue Nov 2 21:36:11 2010 New Revision: 78650 Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py Log: fixed test Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py Tue Nov 2 21:36:11 2010 @@ -358,6 +358,14 @@ ops = self.get_by_bytecode("CALL_FUNCTION") assert len(ops) == 2 for i, bytecode in enumerate(ops): + assert not bytecode.get_opnames("call") + assert not bytecode.get_opnames("new") + assert len(ops[0].get_opnames("guard")) <= 14 + assert len(ops[1].get_opnames("guard")) <= 3 + + ops = self.get_by_bytecode("CALL_FUNCTION", True) + assert len(ops) == 2 + for i, bytecode in enumerate(ops): if i == 0: assert "call(getexecutioncontext)" in str(bytecode) else: From afa at codespeak.net Tue Nov 2 22:11:32 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 2 Nov 2010 22:11:32 +0100 (CET) Subject: [pypy-svn] r78651 - in pypy/trunk/pypy/interpreter/pyparser: . test Message-ID: <20101102211132.DAC0A282BFF@codespeak.net> Author: afa Date: Tue Nov 2 22:11:26 2010 New Revision: 78651 Modified: pypy/trunk/pypy/interpreter/pyparser/genpytokenize.py pypy/trunk/pypy/interpreter/pyparser/pytokenize.py pypy/trunk/pypy/interpreter/pyparser/test/test_pyparse.py Log: Merge r78320 from the fast-foward branch: """ Allow \r as a newline; also tokens that go to the end of the line should stop on either \r or \n. """ It should fix some line ending on windows, like the extra blank lines in the test_gzip output for http://buildbot.pypy.org/summary?category=windows Modified: pypy/trunk/pypy/interpreter/pyparser/genpytokenize.py ============================================================================== --- pypy/trunk/pypy/interpreter/pyparser/genpytokenize.py (original) +++ pypy/trunk/pypy/interpreter/pyparser/genpytokenize.py Tue Nov 2 22:11:26 2010 @@ -17,12 +17,17 @@ def makePyPseudoDFA (): import string states = [] + def makeEOL(): + return group(states, + newArcPair(states, "\n"), + chain(states, + newArcPair(states, "\r"), + maybe(states, newArcPair(states, "\n")))) # ____________________________________________________________ def makeLineCont (): return chain(states, newArcPair(states, "\\"), - maybe(states, newArcPair(states, "\r")), - newArcPair(states, "\n")) + makeEOL()) # ____________________________________________________________ # Ignore stuff def makeWhitespace (): @@ -124,9 +129,7 @@ newArcPair(states, "~")) bracket = groupStr(states, "[](){}") special = group(states, - chain(states, - maybe(states, newArcPair(states, "\r")), - newArcPair(states, "\n")), + makeEOL(), groupStr(states, "@:;.,`")) funny = group(states, operator, bracket, special) # ____________________________________________________________ @@ -140,13 +143,13 @@ makeStrPrefix(), newArcPair(states, "'"), any(states, - notGroupStr(states, "\n'\\")), + notGroupStr(states, "\r\n'\\")), any(states, chain(states, newArcPair(states, "\\"), newArcPair(states, DEFAULT), any(states, - notGroupStr(states, "\n'\\")))), + notGroupStr(states, "\r\n'\\")))), group(states, newArcPair(states, "'"), makeLineCont())), @@ -154,13 +157,13 @@ makeStrPrefix(), newArcPair(states, '"'), any(states, - notGroupStr(states, '\n"\\')), + notGroupStr(states, '\r\n"\\')), any(states, chain(states, newArcPair(states, "\\"), newArcPair(states, DEFAULT), any(states, - notGroupStr(states, '\n"\\')))), + notGroupStr(states, '\r\n"\\')))), group(states, newArcPair(states, '"'), makeLineCont()))) Modified: pypy/trunk/pypy/interpreter/pyparser/pytokenize.py ============================================================================== --- pypy/trunk/pypy/interpreter/pyparser/pytokenize.py (original) +++ pypy/trunk/pypy/interpreter/pyparser/pytokenize.py Tue Nov 2 22:11:26 2010 @@ -24,7 +24,7 @@ # Automatically generated DFA's accepts = [True, True, True, True, True, True, True, True, - True, True, False, True, True, True, False, False, + True, True, False, True, True, True, True, False, False, False, True, True, True, False, True, False, True, False, True, False, False, True, False, False, False, False, True, False, False, @@ -142,9 +142,11 @@ # 14 {'\n': 13}, # 15 - {automata.DEFAULT: 28, '\n': 25, "'": 26, '\\': 27}, + {automata.DEFAULT: 28, '\n': 25, + '\r': 25, "'": 26, '\\': 27}, # 16 - {automata.DEFAULT: 31, '\n': 25, '"': 29, '\\': 30}, + {automata.DEFAULT: 31, '\n': 25, + '\r': 25, '"': 29, '\\': 30}, # 17 {'\n': 13, '\r': 14}, # 18 @@ -188,13 +190,15 @@ # 27 {automata.DEFAULT: 35, '\n': 13, '\r': 14}, # 28 - {automata.DEFAULT: 28, '\n': 25, "'": 13, '\\': 27}, + {automata.DEFAULT: 28, '\n': 25, + '\r': 25, "'": 13, '\\': 27}, # 29 {'"': 13}, # 30 {automata.DEFAULT: 36, '\n': 13, '\r': 14}, # 31 - {automata.DEFAULT: 31, '\n': 25, '"': 13, '\\': 30}, + {automata.DEFAULT: 31, '\n': 25, + '\r': 25, '"': 13, '\\': 30}, # 32 {'+': 37, '-': 37, '0': 38, '1': 38, '2': 38, '3': 38, '4': 38, '5': 38, @@ -208,9 +212,11 @@ '4': 34, '5': 34, '6': 34, '7': 34, '8': 34, '9': 34, 'J': 13, 'j': 13}, # 35 - {automata.DEFAULT: 35, '\n': 25, "'": 13, '\\': 27}, + {automata.DEFAULT: 35, '\n': 25, + '\r': 25, "'": 13, '\\': 27}, # 36 - {automata.DEFAULT: 36, '\n': 25, '"': 13, '\\': 30}, + {automata.DEFAULT: 36, '\n': 25, + '\r': 25, '"': 13, '\\': 30}, # 37 {'0': 38, '1': 38, '2': 38, '3': 38, '4': 38, '5': 38, '6': 38, '7': 38, @@ -282,7 +288,6 @@ ] doubleDFA = automata.DFA(states, accepts) - #_______________________________________________________________________ # End of automatically generated DFA's Modified: pypy/trunk/pypy/interpreter/pyparser/test/test_pyparse.py ============================================================================== --- pypy/trunk/pypy/interpreter/pyparser/test/test_pyparse.py (original) +++ pypy/trunk/pypy/interpreter/pyparser/test/test_pyparse.py Tue Nov 2 22:11:26 2010 @@ -92,6 +92,9 @@ exc = py.test.raises(IndentationError, parse, input).value assert exc.msg == "unindent does not match any outer indentation level" + def test_mac_newline(self): + self.parse("this_is\ra_mac\rfile") + def test_mode(self): assert self.parse("x = 43*54").type == syms.file_input tree = self.parse("43**54", "eval") From afa at codespeak.net Tue Nov 2 22:26:32 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 2 Nov 2010 22:26:32 +0100 (CET) Subject: [pypy-svn] r78652 - pypy/branch/fast-forward/lib-python Message-ID: <20101102212632.7B4B5282C05@codespeak.net> Author: afa Date: Tue Nov 2 22:26:30 2010 New Revision: 78652 Modified: pypy/branch/fast-forward/lib-python/conftest.py Log: Restore standard test_ctypes in this branch: - our own tests are obviously incomplete (nothing for c_longdouble) - pypy is mature enough to try to be fully compatible with cpython - we are in a branch anyway Modified: pypy/branch/fast-forward/lib-python/conftest.py ============================================================================== --- pypy/branch/fast-forward/lib-python/conftest.py (original) +++ pypy/branch/fast-forward/lib-python/conftest.py Tue Nov 2 22:26:30 2010 @@ -496,12 +496,7 @@ RegrTest('test_coding.py'), RegrTest('test_complex_args.py'), RegrTest('test_contextlib.py', usemodules="thread"), - # we skip test ctypes, since we adapted it massively in order - # to test what we want to support. There are real failures, - # but it's about missing features that we don't want to support - # now - RegrTest('test_ctypes.py', usemodules="_rawffi", - skip="missing features that we don't want to support now"), + RegrTest('test_ctypes.py', usemodules="_rawffi") RegrTest('test_defaultdict.py'), RegrTest('test_email_renamed.py'), RegrTest('test_exception_variations.py'), From afa at codespeak.net Tue Nov 2 22:28:28 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 2 Nov 2010 22:28:28 +0100 (CET) Subject: [pypy-svn] r78653 - pypy/branch/fast-forward/lib-python Message-ID: <20101102212828.0FDC3282C0F@codespeak.net> Author: afa Date: Tue Nov 2 22:28:26 2010 New Revision: 78653 Modified: pypy/branch/fast-forward/lib-python/conftest.py Log: oops Modified: pypy/branch/fast-forward/lib-python/conftest.py ============================================================================== --- pypy/branch/fast-forward/lib-python/conftest.py (original) +++ pypy/branch/fast-forward/lib-python/conftest.py Tue Nov 2 22:28:26 2010 @@ -496,7 +496,7 @@ RegrTest('test_coding.py'), RegrTest('test_complex_args.py'), RegrTest('test_contextlib.py', usemodules="thread"), - RegrTest('test_ctypes.py', usemodules="_rawffi") + RegrTest('test_ctypes.py', usemodules="_rawffi"), RegrTest('test_defaultdict.py'), RegrTest('test_email_renamed.py'), RegrTest('test_exception_variations.py'), From afa at codespeak.net Tue Nov 2 22:38:26 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 2 Nov 2010 22:38:26 +0100 (CET) Subject: [pypy-svn] r78654 - in pypy/branch/fast-forward/pypy/module/_rawffi: . test Message-ID: <20101102213826.B85EF282C11@codespeak.net> Author: afa Date: Tue Nov 2 22:38:25 2010 New Revision: 78654 Modified: pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py Log: Fix c_longdouble._setvalue() Modified: pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py (original) +++ pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py Tue Nov 2 22:38:25 2010 @@ -316,7 +316,7 @@ else: ptr = unwrap_truncate_int(rffi.VOIDP, space, w_arg) push_func(add_arg, argdesc, ptr) - elif letter == "d": + elif letter == "d" or letter == "g": push_func(add_arg, argdesc, space.float_w(w_arg)) elif letter == "f": push_func(add_arg, argdesc, rffi.cast(rffi.FLOAT, Modified: pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py (original) +++ pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py Tue Nov 2 22:38:25 2010 @@ -52,9 +52,9 @@ } const char *static_str = "xxxxxx"; - const long static_int = 42; - const double static_double = 42.42; - const long double static_longdouble = 42.42; + long static_int = 42; + double static_double = 42.42; + long double static_longdouble = 42.42; unsigned short add_shorts(short one, short two) { @@ -817,10 +817,16 @@ return _rawffi.Array(typecode).fromaddress(addr, 1) a = getprimitive("l", "static_int") assert a[0] == 42 + a[0] = 43 + assert a[0] == 43 a = getprimitive("d", "static_double") assert a[0] == 42.42 + a[0] = 43.43 + assert a[0] == 43.43 a = getprimitive("g", "static_longdouble") assert a[0] == 42.42 + a[0] = 43.43 + assert a[0] == 43.43 raises(ValueError, getprimitive, 'z', 'ddddddd') raises(ValueError, getprimitive, 'zzz', 'static_int') From afa at codespeak.net Tue Nov 2 22:55:02 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 2 Nov 2010 22:55:02 +0100 (CET) Subject: [pypy-svn] r78655 - in pypy/branch/fast-forward/pypy/module/_hashlib: . test Message-ID: <20101102215502.0CDE4282C09@codespeak.net> Author: afa Date: Tue Nov 2 22:54:57 2010 New Revision: 78655 Modified: pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py Log: _haslib module now accept any buffer object Modified: pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py (original) +++ pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py Tue Nov 2 22:54:57 2010 @@ -26,13 +26,10 @@ return space.wrap("<%s HASH object at 0x%s>" % ( self.name, addrstring)) - @unwrap_spec('self', ObjSpace, str) - def update(self, space, buffer): - buf = rffi.str2charp(buffer) - try: - ropenssl.EVP_DigestUpdate(self.ctx, buf, len(buffer)) - finally: - rffi.free_charp(buf) + @unwrap_spec('self', ObjSpace, 'bufferstr') + def update(self, space, string): + with rffi.scoped_nonmovingbuffer(string) as buf: + ropenssl.EVP_DigestUpdate(self.ctx, buf, len(string)) @unwrap_spec('self', ObjSpace) def copy(self, space): @@ -117,7 +114,7 @@ block_size=GetSetProperty(W_Hash.get_block_size), ) - at unwrap_spec(ObjSpace, str, str) + at unwrap_spec(ObjSpace, str, 'bufferstr') def new(space, name, string=''): w_hash = W_Hash(space, name) w_hash.update(space, string) Modified: pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py (original) +++ pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py Tue Nov 2 22:54:57 2010 @@ -71,3 +71,10 @@ assert h.digest_size == 16 assert len(h.hexdigest()) == 32 + def test_buffer(self): + import _hashlib, array + b = array.array('b', 'x' * 10) + h = _hashlib.new('md5', b) + h.update(b) + assert h.digest() == _hashlib.openssl_md5('x' * 20).digest() + From afa at codespeak.net Wed Nov 3 01:13:53 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 3 Nov 2010 01:13:53 +0100 (CET) Subject: [pypy-svn] r78656 - in pypy/branch/fast-forward/pypy/module/_file: . test Message-ID: <20101103001353.096B6282BAD@codespeak.net> Author: afa Date: Wed Nov 3 01:13:51 2010 New Revision: 78656 Modified: pypy/branch/fast-forward/pypy/module/_file/interp_file.py pypy/branch/fast-forward/pypy/module/_file/test/test_file.py Log: file.newlines should be accessible even after the file has been closed. Modified: pypy/branch/fast-forward/pypy/module/_file/interp_file.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_file/interp_file.py (original) +++ pypy/branch/fast-forward/pypy/module/_file/interp_file.py Wed Nov 3 01:13:51 2010 @@ -31,6 +31,8 @@ encoding = None # This is not used internally by file objects fd = -1 + newlines = 0 # Updated when the stream is closed + def __init__(self, space): self.space = space @@ -113,6 +115,7 @@ space = self.space stream = self.stream if stream is not None: + self.newlines = self.stream.getnewlines() self.stream = None self.fd = -1 openstreams = getopenstreams(self.space) @@ -439,7 +442,10 @@ return space.wrap(file.stream is None) def descr_file_newlines(space, file): - newlines = file.getstream().getnewlines() + if file.stream: + newlines = file.stream.getnewlines() + else: + newlines = file.newlines if newlines == 0: return space.w_None elif newlines == 1: Modified: pypy/branch/fast-forward/pypy/module/_file/test/test_file.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_file/test/test_file.py (original) +++ pypy/branch/fast-forward/pypy/module/_file/test/test_file.py Wed Nov 3 01:13:51 2010 @@ -108,10 +108,13 @@ f.write("\r\n") assert f.newlines is None f.close() + assert f.newlines is None f = self.file(self.temppath, "rU") res = f.read() assert res == "\n" assert f.newlines == "\r\n" + f.close() + assert f.newlines == "\r\n" def test_unicode(self): import os From afa at codespeak.net Wed Nov 3 09:25:57 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 3 Nov 2010 09:25:57 +0100 (CET) Subject: [pypy-svn] r78657 - in pypy/branch/fast-forward/pypy/module/_multiprocessing: . test Message-ID: <20101103082557.EF9E6282BAD@codespeak.net> Author: afa Date: Wed Nov 3 09:25:55 2010 New Revision: 78657 Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py Log: poll() should not work on write-only connections Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Wed Nov 3 09:25:55 2010 @@ -165,6 +165,7 @@ @unwrap_spec('self', ObjSpace, W_Root) def poll(self, space, w_timeout=0.0): + self._check_readable(space) if space.is_w(w_timeout, space.w_None): timeout = -1.0 # block forever else: Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py Wed Nov 3 09:25:55 2010 @@ -52,6 +52,7 @@ assert rhandle.poll(None) == True assert rhandle.recv() == 1 assert rhandle.poll() == False + raises(IOError, whandle.poll) def test_read_into(self): import array, multiprocessing @@ -83,7 +84,7 @@ cls.w_make_pair = cls.space.appexec([], """(): import multiprocessing def make_pair(): - rhandle, whandle = multiprocessing.Pipe() + rhandle, whandle = multiprocessing.Pipe(duplex=False) return rhandle, whandle return make_pair """) From antocuni at codespeak.net Wed Nov 3 10:29:24 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 3 Nov 2010 10:29:24 +0100 (CET) Subject: [pypy-svn] r78658 - pypy/extradoc/planning/hg-migration Message-ID: <20101103092924.773EF282B9D@codespeak.net> Author: antocuni Date: Wed Nov 3 10:29:22 2010 New Revision: 78658 Added: pypy/extradoc/planning/hg-migration/ pypy/extradoc/planning/hg-migration/plan.txt (contents, props changed) pypy/extradoc/planning/hg-migration/usermap.txt (contents, props changed) Log: mercurial migration plan Added: pypy/extradoc/planning/hg-migration/plan.txt ============================================================================== --- (empty file) +++ pypy/extradoc/planning/hg-migration/plan.txt Wed Nov 3 10:29:22 2010 @@ -0,0 +1,53 @@ +.. -*- mode: rst -*- + +Mercurial migration plan +======================== + +- Convert repo from SVN to Mercurial, including all the branches that we can + find. IN-PROGRESS (ronny) + +- Put the repo on bitbucket/google code: + https://bitbucket.org/pypy/pypy-tentative (note that it is a bit outdated) + +- Adapt buildbot to pull from hg instead of svn (DONE): + + * https://bitbucket.org/pypy/bot2 + + * live version at: http://wyvern.cs.uni-duesseldorf.de:8099/ (which pulls + from the ``pypy-tenative`` above) + +- Map codespeak usernames to real name/emails: see usermap.txt: + + * by default, we map each username to just its real name using the info in + /etc/passwd on codespeak: this should not be an issue for the privacy, + as the real names are listed in contributor.txt anyway. + + * If people want to put their emails as well, they are welcome to edit + usernames.txt and put it. I **strongly** suggest everyone to do it, as + it is the "standard" way to keep track of authors in the repositories. + **NOTE**: it is not possible to add the email *after* the conversion has + been completed. + + * It is important to use the same email address that it is used also for + e.g. the bitbucket account, so that it is easier for the system to link + the two correctly. + + * I left "anna", "gintas" e "ignas" out of usermap.txt, because they + explicitly chose not to be in the contributor list. Their hg username + will be simply left untouched (i.e., without real name). + +- Email diff: bitbucket has a builtin service to send email diff. It is not + perfect because the sender of the email is always + ``commits-noreply at bitbucket.org``. Personally, I prefer the codespeak way + to put the author of the commit as the sender, as it simplifies searching in + the emails. I think that to start with, the builtin service is enough, but + in the future we might want to write our own hook to send emails in the + "codespeak way". + +- I tentatively tried to send the commits of the bot2 repo to + pypy-svn at codespeak.net, but they did not arrive. Probably because the + mailing list blocks mails if the author is not subscribed? + +- IRC bot: bitbucket has a builtin hook for CIA.vc; alternatively, we can try + to adapt kenaan: who owns it? + Added: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- (empty file) +++ pypy/extradoc/planning/hg-migration/usermap.txt Wed Nov 3 10:29:22 2010 @@ -0,0 +1,106 @@ +# map codespeak usernames to mercurial usernames +# +# If you want your email address to be in the "official" PyPy repo, please add +# it below. + +arigo=Armin Rigo +fijal=Maciej Fijalkowski +cfbolz=Carl Friedrich Bolz +pedronis=Samuele Pedroni +antocuni=Antonio Cuni +mwh=Michael Hudson +tismer=Christian Tismer +hpk=Holger Krekel +afa=Amaury Forgeot d Arc +ericvrp=Eric van Riet Paap +rxe=Richard Emslie +ac=Anders Chrigstrom +auc=Aurelien Campeas +benjamin=Benjamin Peterson +ale=Anders Lehmann +nik=Niklaus Haldimann +sanxiyn=Seo Sanghyeon +santagada=Leonardo Santagada +rhymes=Lawrence Oluyede +xoraxax=Alexander Schremmer +jlg=Jakub Gustak +guido=Guido Wesdorp +niko=Niko Matsakis +ludal=Ludovic Aubry +alex=Alex Martelli +tverwaes=Toon Verwaest +stephan=Stephan Diehl +adim=Adrien Di Mascio +sschwarzer=Stefan Schwarzer +tomek=Tomek Meka +pmaupin=Patrick Maupin +jacob=Jacob Hallen +lac=Laura Creighton +bob=Bob Ippolito +cami=Camillo Bruni +simonb=Simon Burton +bgola=Bruno Gola +afayolle=Alexandre Fayolle +agaynor=Alex Gaynor +mgedmin=Marius Gedminas +getxsick=Bartosz Skowron,,, +gvanrossum=Guido van Rossum +dialtone=Valentino Volonghi +akuhn=Adrian Kuhn +pdg=Paul deGrandis +gromit=Gerald Klix +wanja=Wanja Saatkamp +iko=Anders Hammarquist +oscar=Oscar Nierstrasz +jandem=Jan de Mooij +goden=Eugene Oden +exarkun=Jean-Paul Calderone +lukas=Lukas Renggli +hakanardo=Hakan Ardo +guenter=Guenter Jantzen +dinu=Dinu Gherman +gbrandl=Georg Brandl +benyoung=Ben Young +nico=Nicolas Chauvat +rocco=Rocco Moretti +micktwomey=Michael Twomey +boria=boria +jared.grubb Jared Grubb +odie=Olivier Dormond +stuart=Stuart Williams +jum=Jens-Uwe Mager +jcreigh=Jason Creighton +justas=Justas Sadzevicius +wildchild=Gabriel Lavoie +micke=Mikael Schnenberg +briandorsey=Brian Dorsey +lucian=Lucian Branescu Mihaila +jriehl=Jonathan David Riehl +bea=Beatrice During +elmom=Elmo Mntynen +ancestor=Andreas Friedge +dan=Daniel Roberts +quest=Anders Qvist +amcintyre=Alan McIntyre +bert=Bert Freudenberg +pzieschang=Pieter Zieschang +esmljaos=Jacob Oscarson +lutz_p Lutz Paelike +hruske=Gasper Zejn +bigdog=Michael Schneider +laszlo=Artur Lisiecki +lene=Lene Wagner +radix=Christopher Armstrong +jacek=Jacek Generowicz +busemann=Stephan Busemann +yusei=Yusei Tahara +gotcha=Godefroid Chappelle +atobe=Toby Watson +aft=Andrew Thompson +jgilbert=Joshua Gilbert +asigfrid=Anders Sigfridsson +david=David Schneider +mcherm=Michael Chermside +tav=tav +blais=Martin Blais +haypo=Victor Stinner From cfbolz at codespeak.net Wed Nov 3 11:39:52 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 3 Nov 2010 11:39:52 +0100 (CET) Subject: [pypy-svn] r78659 - pypy/extradoc/planning/hg-migration Message-ID: <20101103103952.2264C282BAD@codespeak.net> Author: cfbolz Date: Wed Nov 3 11:39:50 2010 New Revision: 78659 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: add user address Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Wed Nov 3 11:39:50 2010 @@ -5,7 +5,7 @@ arigo=Armin Rigo fijal=Maciej Fijalkowski -cfbolz=Carl Friedrich Bolz +cfbolz=Carl Friedrich Bolz pedronis=Samuele Pedroni antocuni=Antonio Cuni mwh=Michael Hudson From afa at codespeak.net Wed Nov 3 11:45:14 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 3 Nov 2010 11:45:14 +0100 (CET) Subject: [pypy-svn] r78660 - in pypy/branch/fast-forward/pypy/module/_multiprocessing: . test Message-ID: <20101103104514.7BD91282BD6@codespeak.net> Author: afa Date: Wed Nov 3 11:45:12 2010 New Revision: 78660 Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py Log: On Windows, _multiprocessing.Connection should be socket-based. Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Wed Nov 3 11:45:12 2010 @@ -5,7 +5,7 @@ OperationError, wrap_oserror, operationerrfmt) from pypy.rpython.lltypesystem import rffi, lltype, llmemory from pypy.rlib.rarithmetic import intmask -import sys, os +import sys READABLE = 1 WRITABLE = 2 @@ -33,7 +33,10 @@ def __del__(self): lltype.free(self.buffer, flavor='raw') - self.do_close() + try: + self.do_close() + except OSError: + pass # Abstract methods def do_close(self): @@ -192,6 +195,35 @@ class W_FileConnection(W_BaseConnection): INVALID_HANDLE_VALUE = -1 + if sys.platform == 'win32': + def WRITE(self, data): + from pypy.rlib._rsocket_rffi import send, geterrno + length = send(self.fd, data, len(data), 0) + if length < 0: + raise WindowsError(geterrno(), "send") + return length + def READ(self, size): + from pypy.rlib._rsocket_rffi import socketrecv, geterrno + with rffi.scoped_alloc_buffer(size) as buf: + length = socketrecv(self.fd, buf.raw, buf.size, 0) + if length < 0: + raise WindowsError(geterrno(), "recv") + return buf.str(length) + def CLOSE(self): + from pypy.rlib._rsocket_rffi import socketclose, geterrno + if socketclose(self.fd) < 0: + raise WindowsError(geterrno(), "close") + else: + def WRITE(self, data): + import os + return os.write(self.fd, data) + def READ(self, length): + import os + return os.read(self.fd, length) + def CLOSE(self): + import os + os.close(self.fd) + def __init__(self, fd, flags): W_BaseConnection.__init__(self, flags) self.fd = fd @@ -213,7 +245,7 @@ def do_close(self): if self.is_valid(): - os.close(self.fd) + self.CLOSE() self.fd = self.INVALID_HANDLE_VALUE def do_send_string(self, space, buffer, offset, size): @@ -255,7 +287,7 @@ # XXX inefficient data = rffi.charpsize2str(message, size) try: - count = os.write(self.fd, data) + count = self.WRITE(data) except OSError, e: raise wrap_oserror(space, e) size -= count @@ -266,7 +298,7 @@ remaining = length while remaining > 0: try: - data = os.read(self.fd, remaining) + data = self.READ(remaining) except OSError, e: raise wrap_oserror(space, e) count = len(data) @@ -282,12 +314,16 @@ remaining -= count buffer = rffi.ptradd(buffer, count) - def do_poll(self, space, timeout): - # XXX Won't work on Windows + if sys.platform == 'win32': + def _check_fd(self): + return self.fd >= 0 + else: + def _check_fd(self): + return self.fd >= 0 and self.fd < rpoll.FD_SETSIZE + def do_poll(self, space, timeout): from pypy.rlib import rpoll - # just verify the fd - if self.fd < 0 or self.fd > rpoll.FD_SETSIZE: + if not self._check_fd(): raise OperationError(space.w_IOError, space.wrap( "handle out of range in select()")) Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py Wed Nov 3 11:45:12 2010 @@ -93,17 +93,42 @@ def setup_class(cls): space = gettestobjspace(usemodules=('_multiprocessing', 'thread')) cls.space = space - cls.w_make_pair = space.appexec([], """(): + + def socketpair(space): + "A socket.socketpair() that works on Windows" + import socket, errno + serverSocket = socket.socket() + serverSocket.bind(('127.0.0.1', 0)) + serverSocket.listen(1) + + client = socket.socket() + client.setblocking(False) + try: + client.connect(('127.0.0.1', serverSocket.getsockname()[1])) + except socket.error, e: + assert e.args[0] in (errno.EINPROGRESS, errno.EWOULDBLOCK) + server, addr = serverSocket.accept() + + # keep sockets alive during the test + cls.connections = server, client + + return space.wrap((server.fileno(), client.fileno())) + w_socketpair = space.wrap(interp2app(socketpair)) + + cls.w_make_pair = space.appexec( + [w_socketpair], """(socketpair): import _multiprocessing import os def make_pair(): - fd1, fd2 = os.pipe() + fd1, fd2 = socketpair() rhandle = _multiprocessing.Connection(fd1, writable=False) whandle = _multiprocessing.Connection(fd2, readable=False) return rhandle, whandle return make_pair """) - if sys.platform == "win32": - def test_poll(self): - skip("poll() does not accept file handles on Windows") + def teardown_class(cls): + try: + del cls.connections + except AttributeError: + pass From afa at codespeak.net Wed Nov 3 11:45:54 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 3 Nov 2010 11:45:54 +0100 (CET) Subject: [pypy-svn] r78661 - pypy/branch/fast-forward/pypy/rpython/lltypesystem Message-ID: <20101103104554.98474282BDB@codespeak.net> Author: afa Date: Wed Nov 3 11:45:53 2010 New Revision: 78661 Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py Log: Belongs to the previous commit. Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py (original) +++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py Wed Nov 3 11:45:53 2010 @@ -943,3 +943,14 @@ return self.buf def __exit__(self, *args): free_charp(self.buf) + +class scoped_alloc_buffer: + def __init__(self, size): + self.size = size + def __enter__(self): + self.raw, self.gc_buf = alloc_buffer(self.size) + return self + def __exit__(self, *args): + keep_buffer_alive_until_here(self.raw, self.gc_buf) + def str(self, length): + return str_from_buffer(self.raw, self.gc_buf, self.size, length) From arigo at codespeak.net Wed Nov 3 12:40:49 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 3 Nov 2010 12:40:49 +0100 (CET) Subject: [pypy-svn] r78662 - in pypy/branch/smalllong/pypy: config module/marshal/test objspace/std objspace/std/test rlib translator/c/src Message-ID: <20101103114049.6C4FB282BD6@codespeak.net> Author: arigo Date: Wed Nov 3 12:40:46 2010 New Revision: 78662 Modified: pypy/branch/smalllong/pypy/config/pypyoption.py pypy/branch/smalllong/pypy/module/marshal/test/test_marshalimpl.py pypy/branch/smalllong/pypy/objspace/std/longobject.py pypy/branch/smalllong/pypy/objspace/std/longtype.py pypy/branch/smalllong/pypy/objspace/std/marshal_impl.py pypy/branch/smalllong/pypy/objspace/std/model.py pypy/branch/smalllong/pypy/objspace/std/objspace.py pypy/branch/smalllong/pypy/objspace/std/smalllongobject.py pypy/branch/smalllong/pypy/objspace/std/strutil.py pypy/branch/smalllong/pypy/objspace/std/test/test_smalllongobject.py pypy/branch/smalllong/pypy/rlib/rarithmetic.py pypy/branch/smalllong/pypy/rlib/rbigint.py pypy/branch/smalllong/pypy/translator/c/src/int.h Log: Mostly finish smalllongobject.py. Add support for translating operations like 'llong_add_ovf'. Modified: pypy/branch/smalllong/pypy/config/pypyoption.py ============================================================================== --- pypy/branch/smalllong/pypy/config/pypyoption.py (original) +++ pypy/branch/smalllong/pypy/config/pypyoption.py Wed Nov 3 12:40:46 2010 @@ -195,7 +195,9 @@ default=100, cmdline="--prebuiltintto"), BoolOption("withsmalllong", "use a version of 'long' in a C long long", - default=False), + default=False, + requires=[("objspace.std.withsmallint", False),]), + # because of missing delegate_xx2yy BoolOption("withstrjoin", "use strings optimized for addition", default=False), Modified: pypy/branch/smalllong/pypy/module/marshal/test/test_marshalimpl.py ============================================================================== --- pypy/branch/smalllong/pypy/module/marshal/test/test_marshalimpl.py (original) +++ pypy/branch/smalllong/pypy/module/marshal/test/test_marshalimpl.py Wed Nov 3 12:40:46 2010 @@ -24,6 +24,10 @@ assert z == 10000000000 z = marshal.loads('I\x00\x1c\xf4\xab\xfd\xff\xff\xff') assert z == -10000000000 + z = marshal.loads('I\x88\x87\x86\x85\x84\x83\x82\x01') + assert z == 108793946209421192 + z = marshal.loads('I\xd8\xd8\xd9\xda\xdb\xdc\xcd\xfe') + assert z == -0x0132232425262728 def test_buffer(self): import marshal Modified: pypy/branch/smalllong/pypy/objspace/std/longobject.py ============================================================================== --- pypy/branch/smalllong/pypy/objspace/std/longobject.py (original) +++ pypy/branch/smalllong/pypy/objspace/std/longobject.py Wed Nov 3 12:40:46 2010 @@ -58,6 +58,9 @@ def get_sign(self): return self.num.sign + def __repr__(self): + return '' % self.num.tolong() + registerimplementation(W_LongObject) def newbigint(space, w_longtype, bigint): @@ -65,6 +68,21 @@ W_LongObject.__init__(w_obj, bigint) return w_obj +def newlong(space, bigint): + """Turn the bigint into a W_LongObject. If withsmalllong is enabled, + check if the bigint would fit in a smalllong, and return a + W_SmallLongObject instead if it does. + """ + if space.config.objspace.std.withsmalllong: + try: + z = bigint.tolonglong() + except OverflowError: + pass + else: + from pypy.objspace.std.smalllongobject import W_SmallLongObject + return W_SmallLongObject(z) + return W_LongObject(bigint) + # bool-to-long def delegate_Bool2Long(space, w_bool): @@ -72,7 +90,7 @@ # int-to-long delegation def delegate_Int2Long(space, w_intobj): - return long__Int(space, w_intobj) + return W_LongObject.fromint(space, w_intobj.intval) # long__Long is supposed to do nothing, unless it has @@ -85,7 +103,7 @@ return W_LongObject(l) def long__Int(space, w_intobj): - return W_LongObject.fromint(space, w_intobj.intval) + return space.newlong(w_intobj.intval) def int__Long(space, w_value): try: @@ -190,38 +208,41 @@ def truediv__Long_Long(space, w_long1, w_long2): try: - return space.newfloat(w_long1.num.truediv(w_long2.num)) + f = w_long1.num.truediv(w_long2.num) except ZeroDivisionError: raise OperationError(space.w_ZeroDivisionError, space.wrap("long division or modulo by zero")) except OverflowError: raise OperationError(space.w_OverflowError, space.wrap("long/long too large for a float")) + return space.newfloat(f) def floordiv__Long_Long(space, w_long1, w_long2): try: - return W_LongObject(w_long1.num.floordiv(w_long2.num)) + z = w_long1.num.floordiv(w_long2.num) except ZeroDivisionError: raise OperationError(space.w_ZeroDivisionError, space.wrap("long division or modulo by zero")) + return newlong(space, z) def div__Long_Long(space, w_long1, w_long2): return floordiv__Long_Long(space, w_long1, w_long2) def mod__Long_Long(space, w_long1, w_long2): try: - return W_LongObject(w_long1.num.mod(w_long2.num)) + z = w_long1.num.mod(w_long2.num) except ZeroDivisionError: raise OperationError(space.w_ZeroDivisionError, space.wrap("long division or modulo by zero")) + return newlong(space, z) def divmod__Long_Long(space, w_long1, w_long2): try: div, mod = w_long1.num.divmod(w_long2.num) - return space.newtuple([W_LongObject(div), W_LongObject(mod)]) except ZeroDivisionError: raise OperationError(space.w_ZeroDivisionError, space.wrap("long division or modulo by zero")) + return space.newtuple([newlong(space, div), newlong(space, mod)]) def pow__Long_Long_Long(space, w_long1, w_long2, w_long3): # XXX need to replicate some of the logic, to get the errors right @@ -282,10 +303,10 @@ except OverflowError: # b too big # XXX maybe just return 0L instead? raise OperationError(space.w_OverflowError, space.wrap("shift count too large")) - return W_LongObject(w_long1.num.rshift(shift)) + return newlong(space, w_long1.num.rshift(shift)) def and__Long_Long(space, w_long1, w_long2): - return W_LongObject(w_long1.num.and_(w_long2.num)) + return newlong(space, w_long1.num.and_(w_long2.num)) def xor__Long_Long(space, w_long1, w_long2): return W_LongObject(w_long1.num.xor(w_long2.num)) @@ -305,11 +326,18 @@ register_all(vars()) # register implementations of ops that recover int op overflows +def recover_with_smalllong(space): + # True if there is a chance that a SmallLong would fit when an Int does not + return (space.config.objspace.std.withsmalllong and + sys.maxint == 2147483647) # binary ops for opname in ['add', 'sub', 'mul', 'div', 'floordiv', 'truediv', 'mod', 'divmod', 'lshift']: exec compile(""" def %(opname)s_ovr__Int_Int(space, w_int1, w_int2): + if recover_with_smalllong(space) and %(opname)r != 'truediv': + from pypy.objspace.std.smalllongobject import %(opname)s_ovr + return %(opname)s_ovr(space, w_int1, w_int2) w_long1 = delegate_Int2Long(space, w_int1) w_long2 = delegate_Int2Long(space, w_int2) return %(opname)s__Long_Long(space, w_long1, w_long2) @@ -322,6 +350,9 @@ for opname in ['neg', 'abs']: exec """ def %(opname)s_ovr__Int(space, w_int1): + if recover_with_smalllong(space): + from pypy.objspace.std.smalllongobject import %(opname)s_ovr + return %(opname)s_ovr(space, w_int1) w_long1 = delegate_Int2Long(space, w_int1) return %(opname)s__Long(space, w_long1) """ % {'opname': opname} @@ -331,6 +362,9 @@ # pow def pow_ovr__Int_Int_None(space, w_int1, w_int2, w_none3): + if recover_with_smalllong(space): + from pypy.objspace.std.smalllongobject import pow_ovr + return pow_ovr(space, w_int1, w_int2) w_long1 = delegate_Int2Long(space, w_int1) w_long2 = delegate_Int2Long(space, w_int2) return pow__Long_Long_None(space, w_long1, w_long2, w_none3) Modified: pypy/branch/smalllong/pypy/objspace/std/longtype.py ============================================================================== --- pypy/branch/smalllong/pypy/objspace/std/longtype.py (original) +++ pypy/branch/smalllong/pypy/objspace/std/longtype.py Wed Nov 3 12:40:46 2010 @@ -35,7 +35,7 @@ return w_obj # the following is all for the 'subclass_of_long(x)' case if W_SmallLongObject and isinstance(w_obj, W_SmallLongObject): - bigint = w_obj.as_bigint() + bigint = w_obj.asbigint() elif isinstance(w_obj, W_LongObject): bigint = w_obj.num elif space.is_true(space.isinstance(w_obj, space.w_int)): @@ -70,14 +70,14 @@ space.wrap(e.msg)) if (space.config.objspace.std.withsmalllong and space.is_w(w_longtype, space.w_long)): + from pypy.objspace.std.smalllongobject import W_SmallLongObject try: - longlong = bigint.tolonglong() + return W_SmallLongObject.frombigint(bigint) except OverflowError: pass - else: - from pypy.objspace.std.smalllongobject import W_SmallLongObject - return W_SmallLongObject(longlong) + from pypy.objspace.std.longobject import newbigint return newbigint(space, w_longtype, bigint) +string_to_w_long._dont_inline_ = True # ____________________________________________________________ Modified: pypy/branch/smalllong/pypy/objspace/std/marshal_impl.py ============================================================================== --- pypy/branch/smalllong/pypy/objspace/std/marshal_impl.py (original) +++ pypy/branch/smalllong/pypy/objspace/std/marshal_impl.py Wed Nov 3 12:40:46 2010 @@ -10,8 +10,8 @@ from pypy.interpreter.error import OperationError from pypy.objspace.std.register_all import register_all -from pypy.rlib.rarithmetic import LONG_BIT -from pypy.objspace.std import longobject, model +from pypy.rlib.rarithmetic import LONG_BIT, r_longlong, r_uint +from pypy.objspace.std import model from pypy.objspace.std.longobject import SHIFT as long_bits from pypy.interpreter.special import Ellipsis from pypy.interpreter.pycode import PyCode @@ -144,27 +144,13 @@ register(TYPE_INT, unmarshal_Int) def unmarshal_Int64(space, u, tc): + lo = u.get_int() + hi = u.get_int() if LONG_BIT >= 64: - lo = u.get_int() & (2**32-1) - hi = u.get_int() - return space.newint((hi << 32) | lo) - else: - # fall back to a long - # XXX at some point, we need to extend longobject - # by _PyLong_FromByteArray and _PyLong_AsByteArray. - # I will do that when implementing cPickle. - # for now, this rare case is solved the simple way. - lshift = longobject.lshift__Long_Long - longor = longobject.or__Long_Long - lo1 = space.newlong(u.get_short() & 0xffff) - lo2 = space.newlong(u.get_short() & 0xffff) - res = space.newlong(u.get_int()) - nbits = space.newlong(16) - res = lshift(space, res, nbits) - res = longor(space, res, lo2) - res = lshift(space, res, nbits) - res = longor(space, res, lo1) - return res + x = (hi << 32) | (lo & (2**32-1)) # result fits in an int + else: + x = (r_longlong(hi) << 32) | r_longlong(r_uint(lo)) # get a r_longlong + return space.wrap(x) register(TYPE_INT64, unmarshal_Int64) def pack_float(f): Modified: pypy/branch/smalllong/pypy/objspace/std/model.py ============================================================================== --- pypy/branch/smalllong/pypy/objspace/std/model.py (original) +++ pypy/branch/smalllong/pypy/objspace/std/model.py Wed Nov 3 12:40:46 2010 @@ -16,6 +16,7 @@ option_to_typename = { "withsmallint" : ["smallintobject.W_SmallIntObject"], + "withsmalllong" : ["smalllongobject.W_SmallLongObject"], "withstrslice" : ["strsliceobject.W_StringSliceObject"], "withstrjoin" : ["strjoinobject.W_StringJoinObject"], "withstrbuf" : ["strbufobject.W_StringBufferObject"], @@ -68,6 +69,7 @@ from pypy.objspace.std import complexobject from pypy.objspace.std import setobject from pypy.objspace.std import smallintobject + from pypy.objspace.std import smalllongobject from pypy.objspace.std import tupleobject from pypy.objspace.std import listobject from pypy.objspace.std import dictmultiobject @@ -180,6 +182,18 @@ (longobject.W_LongObject, longobject.delegate_Int2Long), (complexobject.W_ComplexObject, complexobject.delegate_Int2Complex), ] + if config.objspace.std.withsmalllong: + self.typeorder[boolobject.W_BoolObject] += [ + (smalllongobject.W_SmallLongObject, smalllongobject.delegate_Bool2SmallLong), + ] + self.typeorder[intobject.W_IntObject] += [ + (smalllongobject.W_SmallLongObject, smalllongobject.delegate_Int2SmallLong), + ] + self.typeorder[smalllongobject.W_SmallLongObject] += [ + (floatobject.W_FloatObject, smalllongobject.delegate_SmallLong2Float), + (longobject.W_LongObject, smalllongobject.delegate_SmallLong2Long), + (complexobject.W_ComplexObject, smalllongobject.delegate_SmallLong2Complex), + ] self.typeorder[longobject.W_LongObject] += [ (floatobject.W_FloatObject, floatobject.delegate_Long2Float), (complexobject.W_ComplexObject, Modified: pypy/branch/smalllong/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/smalllong/pypy/objspace/std/objspace.py (original) +++ pypy/branch/smalllong/pypy/objspace/std/objspace.py Wed Nov 3 12:40:46 2010 @@ -176,7 +176,13 @@ #print 'wrapping', x, '->', w_result return w_result if isinstance(x, base_int): - xxxxxxxxx + if self.config.objspace.std.withsmalllong: + from pypy.objspace.std.smalllongobject import W_SmallLongObject + from pypy.rlib.rarithmetic import r_longlong, r_ulonglong + from pypy.rlib.rarithmetic import longlongmax + if (not isinstance(x, r_ulonglong) + or x <= r_ulonglong(longlongmax)): + return W_SmallLongObject(r_longlong(x)) return W_LongObject.fromrarith_int(x) # _____ below here is where the annotator should not get _____ @@ -274,7 +280,7 @@ def newlong(self, val): # val is an int if self.config.objspace.std.withsmalllong: from pypy.objspace.std.smalllongobject import W_SmallLongObject - return W_SmallLongObject.fromint(self, val) + return W_SmallLongObject.fromint(val) return W_LongObject.fromint(self, val) def newtuple(self, list_w): Modified: pypy/branch/smalllong/pypy/objspace/std/smalllongobject.py ============================================================================== --- pypy/branch/smalllong/pypy/objspace/std/smalllongobject.py (original) +++ pypy/branch/smalllong/pypy/objspace/std/smalllongobject.py Wed Nov 3 12:40:46 2010 @@ -1,19 +1,428 @@ """ -Implementation of 'small' longs, stored as a C 'long long' value. -Useful for 32-bit applications manipulating 64-bit values. +Implementation of 'small' longs, stored as a r_longlong. +Useful for 32-bit applications manipulating values a bit larger than +fits in an 'int'. """ from pypy.objspace.std.model import registerimplementation, W_Object -from pypy.rlib.rarithmetic import r_longlong +from pypy.objspace.std.register_all import register_all +from pypy.objspace.std.multimethod import FailedToImplementArgs +from pypy.rlib.rarithmetic import r_longlong, r_int, r_uint +from pypy.rlib.rarithmetic import intmask, ovfcheck, LONGLONG_BIT +from pypy.rlib.rbigint import rbigint +from pypy.objspace.std.longobject import W_LongObject +from pypy.objspace.std.intobject import W_IntObject +from pypy.objspace.std.noneobject import W_NoneObject +from pypy.interpreter.error import OperationError class W_SmallLongObject(W_Object): from pypy.objspace.std.longtype import long_typedef as typedef + _immutable_ = True def __init__(w_self, value): assert isinstance(value, r_longlong) - w_self._longlong = value + w_self.longlong = value - def as_bigint(w_self): - xxx + @staticmethod + def fromint(value): + return W_SmallLongObject(r_longlong(value)) + + @staticmethod + def frombigint(bigint): + return W_SmallLongObject(bigint.tolonglong()) + + def asbigint(w_self): + return rbigint.fromrarith_int(w_self.longlong) + + def __repr__(w_self): + return '' % w_self.longlong registerimplementation(W_SmallLongObject) + + +def delegate_Bool2SmallLong(space, w_bool): + return W_SmallLongObject(r_longlong(space.is_true(w_bool))) + +def delegate_Int2SmallLong(space, w_int): + return W_SmallLongObject(r_longlong(w_int.intval)) + +def delegate_SmallLong2Long(space, w_small): + return W_LongObject(w_small.asbigint()) + +def delegate_SmallLong2Float(space, w_small): + return space.newfloat(float(w_small.longlong)) + +def delegate_SmallLong2Complex(space, w_small): + return space.newcomplex(float(w_small.longlong), 0.0) + + +def long__SmallLong(space, w_value): + return w_value + +def int__SmallLong(space, w_value): + a = w_value.longlong + b = intmask(a) + if b == a: + return space.newint(b) + else: + return w_value + +def index__SmallLong(space, w_value): + return w_value + +def float__SmallLong(space, w_value): + return space.newfloat(float(w_value.longlong)) + +def int_w__SmallLong(space, w_value): + a = w_value.longlong + b = intmask(a) + if b == a: + return b + else: + raise OperationError(space.w_OverflowError, space.wrap( + "long int too large to convert to int")) + +def uint_w__SmallLong(space, w_value): + a = w_value.longlong + if a < 0: + raise OperationError(space.w_ValueError, space.wrap( + "cannot convert negative integer to unsigned int")) + b = r_uint(a) + if r_longlong(b) == a: + return b + else: + raise OperationError(space.w_OverflowError, space.wrap( + "long int too large to convert to unsigned int")) + +def bigint_w__SmallLong(space, w_value): + return w_value.asbigint() + +def lt__SmallLong_SmallLong(space, w_small1, w_small2): + return space.newbool(w_small1.longlong < w_small2.longlong) +def le__SmallLong_SmallLong(space, w_small1, w_small2): + return space.newbool(w_small1.longlong <= w_small2.longlong) +def eq__SmallLong_SmallLong(space, w_small1, w_small2): + return space.newbool(w_small1.longlong == w_small2.longlong) +def ne__SmallLong_SmallLong(space, w_small1, w_small2): + return space.newbool(w_small1.longlong != w_small2.longlong) +def gt__SmallLong_SmallLong(space, w_small1, w_small2): + return space.newbool(w_small1.longlong > w_small2.longlong) +def ge__SmallLong_SmallLong(space, w_small1, w_small2): + return space.newbool(w_small1.longlong >= w_small2.longlong) + +def lt__SmallLong_Long(space, w_small1, w_long2): + return space.newbool(w_small1.asbigint().lt(w_long2.num)) +def le__SmallLong_Long(space, w_small1, w_long2): + return space.newbool(w_small1.asbigint().le(w_long2.num)) +def eq__SmallLong_Long(space, w_small1, w_long2): + return space.newbool(w_small1.asbigint().eq(w_long2.num)) +def ne__SmallLong_Long(space, w_small1, w_long2): + return space.newbool(w_small1.asbigint().ne(w_long2.num)) +def gt__SmallLong_Long(space, w_small1, w_long2): + return space.newbool(w_small1.asbigint().gt(w_long2.num)) +def ge__SmallLong_Long(space, w_small1, w_long2): + return space.newbool(w_small1.asbigint().ge(w_long2.num)) + +def lt__Long_SmallLong(space, w_long1, w_small2): + return space.newbool(w_long1.num.lt(w_small2.asbigint())) +def le__Long_SmallLong(space, w_long1, w_small2): + return space.newbool(w_long1.num.le(w_small2.asbigint())) +def eq__Long_SmallLong(space, w_long1, w_small2): + return space.newbool(w_long1.num.eq(w_small2.asbigint())) +def ne__Long_SmallLong(space, w_long1, w_small2): + return space.newbool(w_long1.num.ne(w_small2.asbigint())) +def gt__Long_SmallLong(space, w_long1, w_small2): + return space.newbool(w_long1.num.gt(w_small2.asbigint())) +def ge__Long_SmallLong(space, w_long1, w_small2): + return space.newbool(w_long1.num.ge(w_small2.asbigint())) + +def lt__SmallLong_Int(space, w_small1, w_int2): + return space.newbool(w_small1.longlong < w_int2.intval) +def le__SmallLong_Int(space, w_small1, w_int2): + return space.newbool(w_small1.longlong <= w_int2.intval) +def eq__SmallLong_Int(space, w_small1, w_int2): + return space.newbool(w_small1.longlong == w_int2.intval) +def ne__SmallLong_Int(space, w_small1, w_int2): + return space.newbool(w_small1.longlong != w_int2.intval) +def gt__SmallLong_Int(space, w_small1, w_int2): + return space.newbool(w_small1.longlong > w_int2.intval) +def ge__SmallLong_Int(space, w_small1, w_int2): + return space.newbool(w_small1.longlong >= w_int2.intval) + +def lt__Int_SmallLong(space, w_int1, w_small2): + return space.newbool(w_int1.intval < w_small2.longlong) +def le__Int_SmallLong(space, w_int1, w_small2): + return space.newbool(w_int1.intval <= w_small2.longlong) +def eq__Int_SmallLong(space, w_int1, w_small2): + return space.newbool(w_int1.intval == w_small2.longlong) +def ne__Int_SmallLong(space, w_int1, w_small2): + return space.newbool(w_int1.intval != w_small2.longlong) +def gt__Int_SmallLong(space, w_int1, w_small2): + return space.newbool(w_int1.intval > w_small2.longlong) +def ge__Int_SmallLong(space, w_int1, w_small2): + return space.newbool(w_int1.intval >= w_small2.longlong) + + +#hash: default implementation via Longs (a bit messy) + +def add__SmallLong_SmallLong(space, w_small1, w_small2): + x = w_small1.longlong + y = w_small2.longlong + try: + z = ovfcheck(x + y) + except OverflowError: + raise FailedToImplementArgs(space.w_OverflowError, + space.wrap("integer addition")) + return W_SmallLongObject(z) + +def add_ovr(space, w_int1, w_int2): + x = r_longlong(w_int1.intval) + y = r_longlong(w_int2.intval) + return W_SmallLongObject(x + y) + +def sub__SmallLong_SmallLong(space, w_small1, w_small2): + x = w_small1.longlong + y = w_small2.longlong + try: + z = ovfcheck(x - y) + except OverflowError: + raise FailedToImplementArgs(space.w_OverflowError, + space.wrap("integer subtraction")) + return W_SmallLongObject(z) + +def sub_ovr(space, w_int1, w_int2): + x = r_longlong(w_int1.intval) + y = r_longlong(w_int2.intval) + return W_SmallLongObject(x - y) + +def mul__SmallLong_SmallLong(space, w_small1, w_small2): + x = w_small1.longlong + y = w_small2.longlong + try: + z = ovfcheck(x * y) + except OverflowError: + raise FailedToImplementArgs(space.w_OverflowError, + space.wrap("integer multiplication")) + return W_SmallLongObject(z) + +def mul_ovr(space, w_int1, w_int2): + x = r_longlong(w_int1.intval) + y = r_longlong(w_int2.intval) + return W_SmallLongObject(x * y) + +#truediv: default implementation via Longs + +def floordiv__SmallLong_SmallLong(space, w_small1, w_small2): + x = w_small1.longlong + y = w_small2.longlong + try: + z = ovfcheck(x // y) + except ZeroDivisionError: + raise OperationError(space.w_ZeroDivisionError, + space.wrap("integer division by zero")) + except OverflowError: + raise FailedToImplementArgs(space.w_OverflowError, + space.wrap("integer division")) + return W_SmallLongObject(z) +div__SmallLong_SmallLong = floordiv__SmallLong_SmallLong + +def floordiv_ovr(space, w_int1, w_int2): + x = r_longlong(w_int1.intval) + y = r_longlong(w_int2.intval) + return W_SmallLongObject(x // y) +div_ovr = floordiv_ovr + +def mod__SmallLong_SmallLong(space, w_small1, w_small2): + x = w_small1.longlong + y = w_small2.longlong + try: + z = ovfcheck(x % y) + except ZeroDivisionError: + raise OperationError(space.w_ZeroDivisionError, + space.wrap("integer modulo by zero")) + except OverflowError: + raise FailedToImplementArgs(space.w_OverflowError, + space.wrap("integer modulo")) + return W_SmallLongObject(z) + +def mod_ovr(space, w_int1, w_int2): + x = r_longlong(w_int1.intval) + y = r_longlong(w_int2.intval) + return W_SmallLongObject(x % y) + +def divmod__SmallLong_SmallLong(space, w_small1, w_small2): + x = w_small1.longlong + y = w_small2.longlong + try: + z = ovfcheck(x // y) + except ZeroDivisionError: + raise OperationError(space.w_ZeroDivisionError, + space.wrap("integer divmod by zero")) + except OverflowError: + raise FailedToImplementArgs(space.w_OverflowError, + space.wrap("integer modulo")) + # no overflow possible + m = x % y + return space.newtuple([W_SmallLongObject(z), W_SmallLongObject(m)]) + +def divmod_ovr(space, w_int1, w_int2): + return space.newtuple([div_ovr(space, w_int1, w_int2), + mod_ovr(space, w_int1, w_int2)]) + +def _impl_pow(space, iv, w_int2, iz=r_longlong(0)): + iw = w_int2.intval + if iw < 0: + if iz != 0: + raise OperationError(space.w_TypeError, + space.wrap("pow() 2nd argument " + "cannot be negative when 3rd argument specified")) + ## bounce it, since it always returns float + raise FailedToImplementArgs(space.w_ValueError, + space.wrap("integer exponentiation")) + temp = iv + ix = r_longlong(1) + try: + while iw > 0: + if iw & 1: + ix = ovfcheck(ix*temp) + iw >>= 1 #/* Shift exponent down by 1 bit */ + if iw==0: + break + temp = ovfcheck(temp*temp) #/* Square the value of temp */ + if iz: + #/* If we did a multiplication, perform a modulo */ + ix = ix % iz + temp = temp % iz + if iz: + ix = ix % iz + except OverflowError: + raise FailedToImplementArgs(space.w_OverflowError, + space.wrap("integer exponentiation")) + return W_SmallLongObject(ix) + +def pow__SmallLong_Int_SmallLong(space, w_small1, w_int2, w_small3): + z = w_small3.longlong + if z == 0: + raise OperationError(space.w_ValueError, + space.wrap("pow() 3rd argument cannot be 0")) + return _impl_pow(space, w_small1.longlong, w_int2, z) + +def pow__SmallLong_Int_None(space, w_small1, w_int2, _): + return _impl_pow(space, w_small1.longlong, w_int2) + +def pow_ovr(space, w_int1, w_int2): + try: + return _impl_pow(space, r_longlong(w_int1.intval), w_int2) + except FailedToImplementArgs: + from pypy.objspace.std import longobject + w_a = W_LongObject.fromint(space, w_int1.intval) + w_b = W_LongObject.fromint(space, w_int2.intval) + return longobject.pow__Long_Long_None(space, w_a, w_b, space.w_None) + +def neg__SmallLong(space, w_small): + a = w_small.longlong + try: + x = ovfcheck(-a) + except OverflowError: + raise FailedToImplementArgs(space.w_OverflowError, + space.wrap("integer negation")) + return W_SmallLongObject(x) +get_negint = neg__SmallLong + +def neg_ovr(space, w_int): + a = r_longlong(w_int.intval) + return W_SmallLongObject(-a) + + +def pos__SmallLong(space, w_small): + return w_small + +def abs__SmallLong(space, w_small): + if w_small.longlong >= 0: + return w_small + else: + return get_negint(space, w_small) + +def abs_ovr(space, w_int): + a = r_longlong(w_int.intval) + if a < 0: a = -a + return W_SmallLongObject(a) + +def nonzero__SmallLong(space, w_small): + return space.newbool(bool(w_small.longlong)) + +def invert__SmallLong(space, w_small): + x = w_small.longlong + a = ~x + return W_SmallLongObject(a) + +def lshift__SmallLong_Int(space, w_small1, w_int2): + a = w_small1.longlong + b = w_int2.intval + if r_uint(b) < LONGLONG_BIT: # 0 <= b < LONGLONG_BIT + try: + c = ovfcheck(a << b) + except OverflowError: + raise FailedToImplementArgs(space.w_OverflowError, + space.wrap("integer left shift")) + return W_SmallLongObject(c) + if b < 0: + raise OperationError(space.w_ValueError, + space.wrap("negative shift count")) + else: #b >= LONGLONG_BIT + if a == 0: + return w_small1 + raise FailedToImplementArgs(space.w_OverflowError, + space.wrap("integer left shift")) + +def lshift_ovr(space, w_int1, w_int2): + a = r_longlong(w_int1.intval) + try: + return lshift__SmallLong_Int(space, W_SmallLongObject(a), w_int2) + except FailedToImplementArgs: + from pypy.objspace.std import longobject + w_a = W_LongObject.fromint(space, w_int1.intval) + w_b = W_LongObject.fromint(space, w_int2.intval) + return longobject.lshift__Long_Long(space, w_a, w_b) + +def rshift__SmallLong_Int(space, w_small1, w_int2): + a = w_small1.longlong + b = w_int2.intval + if r_uint(b) >= LONGLONG_BIT: # not (0 <= b < LONGLONG_BIT) + if b < 0: + raise OperationError(space.w_ValueError, + space.wrap("negative shift count")) + else: # b >= LONGLONG_BIT + if a == 0: + return w_small1 + if a < 0: + a = -1 + else: + a = 0 + else: + a = a >> b + return W_SmallLongObject(a) + +def and__SmallLong_SmallLong(space, w_small1, w_small2): + a = w_small1.longlong + b = w_small2.longlong + res = a & b + return W_SmallLongObject(res) + +def xor__SmallLong_SmallLong(space, w_small1, w_small2): + a = w_small1.longlong + b = w_small2.longlong + res = a ^ b + return W_SmallLongObject(res) + +def or__SmallLong_SmallLong(space, w_small1, w_small2): + a = w_small1.longlong + b = w_small2.longlong + res = a | b + return W_SmallLongObject(res) + +#oct: default implementation via Longs +#hex: default implementation via Longs +#getnewargs: default implementation via Longs + +register_all(vars()) Modified: pypy/branch/smalllong/pypy/objspace/std/strutil.py ============================================================================== --- pypy/branch/smalllong/pypy/objspace/std/strutil.py (original) +++ pypy/branch/smalllong/pypy/objspace/std/strutil.py Wed Nov 3 12:40:46 2010 @@ -205,67 +205,66 @@ digits = '0' # a few abbreviations - from pypy.objspace.std import longobject - mklong = longobject.W_LongObject.fromint - d2long = longobject.W_LongObject.fromdecimalstr - adlong = longobject.add__Long_Long - longup = longobject.pow__Long_Long_None - multip = longobject.mul__Long_Long - divide = longobject.div__Long_Long - lshift = longobject.lshift__Long_Long - rshift = longobject.rshift__Long_Long + from pypy.rlib.rbigint import rbigint + mklong = rbigint.fromint + d2long = rbigint.fromdecimalstr + adlong = rbigint.static_add + longup = rbigint.static_pow + multip = rbigint.static_mul + divide = rbigint.static_floordiv + lshift = rbigint.static_lshift + rshift = rbigint.static_rshift # 4) compute the exponent and truncate to +-400 if not exponent: exponent = '0' - w_le = d2long(exponent) - w_le = adlong(space, w_le, mklong(space, dexp)) + b_le = d2long(exponent) + b_le = adlong(b_le, mklong(dexp)) try: - e = w_le.toint() + e = b_le.toint() except OverflowError: - # XXX poking at internals - e = w_le.num.sign * 400 + e = b_le.getsign() * 400 if e >= 400: e = 400 elif e <= -400: e = -400 # 5) compute the value using long math and proper rounding. - w_lr = d2long(digits) - w_10 = mklong(space, 10) - w_1 = mklong(space, 1) + b_lr = d2long(digits) + b_10 = mklong(10) + b_1 = mklong(1) if e >= 0: bits = 0 - w_pten = longup(space, w_10, mklong(space, e), space.w_None) - w_m = multip(space, w_lr, w_pten) + b_pten = longup(b_10, mklong(e)) + b_m = multip(b_lr, b_pten) else: # compute a sufficiently large scale prec = MANTISSA_DIGITS * 2 + 22 # 128, maybe bits = - (int(math.ceil(-e / math.log10(2.0) - 1e-10)) + prec) - w_scale = lshift(space, w_1, mklong(space, -bits)) - w_pten = longup(space, w_10, mklong(space, -e), None) - w_tmp = multip(space, w_lr, w_scale) - w_m = divide(space, w_tmp, w_pten) + b_scale = lshift(b_1, -bits) + b_pten = longup(b_10, mklong(-e)) + b_tmp = multip(b_lr, b_scale) + b_m = divide(b_tmp, b_pten) # we now have a fairly large mantissa. # Shift it and round the last bit. # first estimate the bits and do a big shift - mbits = w_m._count_bits() + mbits = b_m._count_bits() needed = MANTISSA_BITS if mbits > needed: if mbits > needed+1: shifted = mbits - (needed+1) - w_m = rshift(space, w_m, mklong(space, shifted)) + b_m = rshift(b_m, shifted) bits += shifted # do the rounding bits += 1 - round = w_m.is_odd() - w_m = rshift(space, w_m, w_1) - w_m = adlong(space, w_m, mklong(space, round)) + round = b_m.is_odd() + b_m = rshift(b_m, 1) + b_m = adlong(b_m, mklong(round)) try: - r = math.ldexp(w_m.tofloat(), bits) + r = math.ldexp(b_m.tofloat(), bits) # XXX I guess we do not check for overflow in ldexp as we agreed to! if r == 2*r and r != 0.0: raise OverflowError Modified: pypy/branch/smalllong/pypy/objspace/std/test/test_smalllongobject.py ============================================================================== --- pypy/branch/smalllong/pypy/objspace/std/test/test_smalllongobject.py (original) +++ pypy/branch/smalllong/pypy/objspace/std/test/test_smalllongobject.py Wed Nov 3 12:40:46 2010 @@ -1,5 +1,39 @@ +import py +import sys +from pypy.objspace.std.smalllongobject import W_SmallLongObject from pypy.objspace.std.test import test_longobject from pypy.conftest import gettestobjspace +from pypy.rlib.rarithmetic import r_longlong +from pypy.interpreter.error import OperationError + + +def test_direct(): + space = gettestobjspace(**{"objspace.std.withsmalllong": True}) + w5 = space.wrap(r_longlong(5)) + assert isinstance(w5, W_SmallLongObject) + wlarge = space.wrap(r_longlong(0x123456789ABCDEFL)) + # + assert space.int_w(w5) == 5 + if sys.maxint < 0x123456789ABCDEFL: + py.test.raises(OperationError, space.int_w, wlarge) + else: + assert space.int_w(wlarge) == 0x123456789ABCDEF + # + assert space.pos(w5) is w5 + assert space.abs(w5) is w5 + wm5 = space.wrap(r_longlong(-5)) + assert space.int_w(space.abs(wm5)) == 5 + assert space.int_w(space.neg(w5)) == -5 + assert space.is_true(w5) is True + assert space.is_true(wm5) is True + w0 = space.wrap(r_longlong(0)) + assert space.is_true(w0) is False + # + w14000000000000 = space.wrap(r_longlong(0x14000000000000L)) + assert space.is_true(space.eq( + space.lshift(w5, space.wrap(49)), w14000000000000)) is False + assert space.is_true(space.eq( + space.lshift(w5, space.wrap(50)), w14000000000000)) is True class AppTestSmallLong(test_longobject.AppTestLong): @@ -10,3 +44,75 @@ import __pypy__ s = __pypy__.internal_repr(5L) assert 'SmallLong' in s + + def test_sl_hash(self): + import __pypy__ + x = 5L + assert 'SmallLong' in __pypy__.internal_repr(x) + assert hash(5) == hash(x) + biglong = 5L + biglong ^= 2**100 # hack based on the fact that xor__Long_Long + biglong ^= 2**100 # does not call newlong() + assert biglong == 5L + assert 'SmallLong' not in __pypy__.internal_repr(biglong) + assert hash(5) == hash(biglong) + # + x = 0x123456789ABCDEFL + assert 'SmallLong' in __pypy__.internal_repr(x) + biglong = x + biglong ^= 2**100 + biglong ^= 2**100 + assert biglong == x + assert 'SmallLong' not in __pypy__.internal_repr(biglong) + assert hash(biglong) == hash(x) + + def test_sl_int(self): + x = 0x123456789ABCDEFL + two = 2 + assert int(x) == x + assert type(int(x)) == type(0x1234567 ** two) + y = x >> 32 + assert int(y) == y + assert type(int(y)) is int + + def test_sl_long(self): + import __pypy__ + x = long(0) + assert 'SmallLong' in __pypy__.internal_repr(x) + + def test_sl_add(self): + import __pypy__ + x = 0x123456789ABCDEFL + assert x + x == 0x2468ACF13579BDEL + assert 'SmallLong' in __pypy__.internal_repr(x + x) + x = -0x123456789ABCDEFL + assert x + x == -0x2468ACF13579BDEL + assert 'SmallLong' in __pypy__.internal_repr(x + x) + x = 0x723456789ABCDEF0L + assert x + x == 0xE468ACF13579BDE0L + assert 'SmallLong' not in __pypy__.internal_repr(x + x) + x = -0x723456789ABCDEF0L + assert x + x == -0xE468ACF13579BDE0L + assert 'SmallLong' not in __pypy__.internal_repr(x + x) + + def test_sl_add_32(self): + import sys, __pypy__ + if sys.maxint == 2147483647: + x = 2147483647 + assert x + x == 4294967294L + assert 'SmallLong' in __pypy__.internal_repr(x + x) + y = -1 + assert x - y == 2147483648L + assert 'SmallLong' in __pypy__.internal_repr(x - y) + + def test_sl_lshift(self): + for x in [1, 1L]: + assert x << 1 == 2L + assert x << 30 == 1073741824L + assert x << 31 == 2147483648L + assert x << 32 == 4294967296L + assert x << 62 == 4611686018427387904L + assert x << 63 == 9223372036854775808L + assert x << 64 == 18446744073709551616L + assert (x << 31) << 31 == 4611686018427387904L + assert (x << 32) << 32 == 18446744073709551616L Modified: pypy/branch/smalllong/pypy/rlib/rarithmetic.py ============================================================================== --- pypy/branch/smalllong/pypy/rlib/rarithmetic.py (original) +++ pypy/branch/smalllong/pypy/rlib/rarithmetic.py Wed Nov 3 12:40:46 2010 @@ -49,6 +49,9 @@ LONG_BIT = _bits+1 LONG_MASK = _Ltest*2-1 LONG_TEST = _Ltest +LONGLONG_BIT = 64 +LONGLONG_MASK = (2**LONGLONG_BIT)-1 +LONGLONG_TEST = 2**(LONGLONG_BIT-1) LONG_BIT_SHIFT = 0 while (1 << LONG_BIT_SHIFT) != LONG_BIT: @@ -76,6 +79,15 @@ n -= 2*LONG_TEST return int(n) +def longlongmask(n): + if isinstance(n, int): + n = long(n) + assert isinstance(n, long) + n &= LONGLONG_MASK + if n >= LONGLONG_TEST: + n -= 2*LONGLONG_TEST + return r_longlong(n) + def widen(n): from pypy.rpython.lltypesystem import lltype if _should_widen_type(lltype.typeOf(n)): @@ -389,6 +401,8 @@ r_longlong = build_int('r_longlong', True, 64) r_ulonglong = build_int('r_ulonglong', False, 64) +longlongmax = r_longlong(LONGLONG_TEST - 1) + # float as string -> sign, beforept, afterpt, exponent Modified: pypy/branch/smalllong/pypy/rlib/rbigint.py ============================================================================== --- pypy/branch/smalllong/pypy/rlib/rbigint.py (original) +++ pypy/branch/smalllong/pypy/rlib/rbigint.py Wed Nov 3 12:40:46 2010 @@ -558,6 +558,34 @@ if self._numdigits() == 1 and self.digits[0] == 0: self.sign = 0 + def getsign(self): + return self.sign + + + @staticmethod + def static_add(a, b): + return a.add(b) + + @staticmethod + def static_mul(a, b): + return a.mul(b) + + @staticmethod + def static_floordiv(a, b): + return a.floordiv(b) + + @staticmethod + def static_lshift(a, b): + return a.lshift(b) + + @staticmethod + def static_rshift(a, b): + return a.rshift(b) + + @staticmethod + def static_pow(a, b): + return a.pow(b) + def __repr__(self): return "" % (self.digits, self.sign, self.str()) Modified: pypy/branch/smalllong/pypy/translator/c/src/int.h ============================================================================== --- pypy/branch/smalllong/pypy/translator/c/src/int.h (original) +++ pypy/branch/smalllong/pypy/translator/c/src/int.h Wed Nov 3 12:40:46 2010 @@ -64,6 +64,11 @@ if ((r^(x)) >= 0 || (r^(y)) >= 0); \ else FAIL_OVF("integer addition") +#define OP_LLONG_ADD_OVF(x,y,r) \ + OP_LLONG_ADD(x,y,r); \ + if ((r^(x)) >= 0 || (r^(y)) >= 0); \ + else FAIL_OVF("integer addition") + #define OP_INT_ADD_NONNEG_OVF(x,y,r) /* y can be assumed >= 0 */ \ r = (long)((unsigned long)x + (unsigned long)y); \ if (r >= (x)); \ @@ -78,22 +83,17 @@ if ((r^(x)) >= 0 || (r^~(y)) >= 0); \ else FAIL_OVF("integer subtraction") -#define OP_INT_MUL(x,y,r) r = (x) * (y) - -#if defined(HAVE_LONG_LONG) && SIZE_OF_LONG_LONG < SIZE_OF_LONG -# define OP_INT_MUL_OVF_LL 1 -#lse -# define OP_INT_MUL_OVF_LL 0 -#endif +#define OP_LLONG_SUB_OVF(x,y,r) \ + OP_LLONG_SUB(x,y,r); \ + if ((r^(x)) >= 0 || (r^~(y)) >= 0); \ + else FAIL_OVF("integer subtraction") -#if !OP_INT_MUL_OVF_LL +#define OP_INT_MUL(x,y,r) r = (x) * (y) +#if SIZE_OF_LONG == SIZE_OF_LONG_LONG #define OP_INT_MUL_OVF(x,y,r) \ - if (op_int_mul_ovf(x,y,&r)); \ - else FAIL_OVF("integer multiplication") - + r = op_llong_mul_ovf(x, y) #else - #define OP_INT_MUL_OVF(x,y,r) \ { \ PY_LONG_LONG lr = (PY_LONG_LONG)(x) * (PY_LONG_LONG)(y); \ @@ -103,6 +103,9 @@ } #endif +#define OP_LLONG_MUL_OVF(x,y,r) \ + r = op_llong_mul_ovf(x, y) + /* shifting */ /* NB. shifting has same limitations as C: the shift count must be @@ -121,6 +124,10 @@ OP_INT_LSHIFT(x,y,r); \ if ((x) != Py_ARITHMETIC_RIGHT_SHIFT(long, r, (y))) \ FAIL_OVF("x< Author: arigo Date: Wed Nov 3 12:52:49 2010 New Revision: 78663 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: My e-mail. Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Wed Nov 3 12:52:49 2010 @@ -3,7 +3,7 @@ # If you want your email address to be in the "official" PyPy repo, please add # it below. -arigo=Armin Rigo +arigo=Armin Rigo fijal=Maciej Fijalkowski cfbolz=Carl Friedrich Bolz pedronis=Samuele Pedroni From fijal at codespeak.net Wed Nov 3 13:18:03 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 3 Nov 2010 13:18:03 +0100 (CET) Subject: [pypy-svn] r78664 - pypy/extradoc/planning/hg-migration Message-ID: <20101103121803.A3469282B9E@codespeak.net> Author: fijal Date: Wed Nov 3 13:18:01 2010 New Revision: 78664 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: put my mail Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Wed Nov 3 13:18:01 2010 @@ -4,7 +4,7 @@ # it below. arigo=Armin Rigo -fijal=Maciej Fijalkowski +fijal=Maciej Fijalkowski cfbolz=Carl Friedrich Bolz pedronis=Samuele Pedroni antocuni=Antonio Cuni From arigo at codespeak.net Wed Nov 3 13:29:05 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 3 Nov 2010 13:29:05 +0100 (CET) Subject: [pypy-svn] r78665 - pypy/trunk/pypy/translator/c Message-ID: <20101103122905.C6670282BDC@codespeak.net> Author: arigo Date: Wed Nov 3 13:29:04 2010 New Revision: 78665 Modified: pypy/trunk/pypy/translator/c/genc.py Log: Add a special rule to the Makefile preventing it to delete .s files whenever we press Ctrl-C. Modified: pypy/trunk/pypy/translator/c/genc.py ============================================================================== --- pypy/trunk/pypy/translator/c/genc.py (original) +++ pypy/trunk/pypy/translator/c/genc.py Wed Nov 3 13:29:04 2010 @@ -626,6 +626,7 @@ python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py -m$(PYPY_MAIN_FUNCTION) -t $< > $*.gcmap') mk.rule('gcmaptable.s', '$(GCMAPFILES)', python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py $(GCMAPFILES) > $@') + mk.rule('.PRECIOUS', '%.s', "# don't remove .s files if Ctrl-C'ed") else: mk.definition('DEBUGFLAGS', '-O1 -g') From cfbolz at codespeak.net Wed Nov 3 13:34:43 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 3 Nov 2010 13:34:43 +0100 (CET) Subject: [pypy-svn] r78666 - pypy/extradoc/planning/hg-migration Message-ID: <20101103123443.CC74C282BE0@codespeak.net> Author: cfbolz Date: Wed Nov 3 13:34:42 2010 New Revision: 78666 Modified: pypy/extradoc/planning/hg-migration/plan.txt Log: typo Modified: pypy/extradoc/planning/hg-migration/plan.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/plan.txt (original) +++ pypy/extradoc/planning/hg-migration/plan.txt Wed Nov 3 13:34:42 2010 @@ -32,7 +32,7 @@ e.g. the bitbucket account, so that it is easier for the system to link the two correctly. - * I left "anna", "gintas" e "ignas" out of usermap.txt, because they + * I left "anna", "gintas" and "ignas" out of usermap.txt, because they explicitly chose not to be in the contributor list. Their hg username will be simply left untouched (i.e., without real name). From agaynor at codespeak.net Wed Nov 3 14:43:23 2010 From: agaynor at codespeak.net (agaynor at codespeak.net) Date: Wed, 3 Nov 2010 14:43:23 +0100 (CET) Subject: [pypy-svn] r78667 - pypy/extradoc/planning/hg-migration Message-ID: <20101103134323.7177C282BDB@codespeak.net> Author: agaynor Date: Wed Nov 3 14:43:21 2010 New Revision: 78667 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: Updated myself in the usermap. Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Wed Nov 3 14:43:21 2010 @@ -41,7 +41,7 @@ simonb=Simon Burton bgola=Bruno Gola afayolle=Alexandre Fayolle -agaynor=Alex Gaynor +agaynor=Alex Gaynor mgedmin=Marius Gedminas getxsick=Bartosz Skowron,,, gvanrossum=Guido van Rossum From arigo at codespeak.net Wed Nov 3 14:49:18 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 3 Nov 2010 14:49:18 +0100 (CET) Subject: [pypy-svn] r78668 - pypy/branch/smalllong/pypy/objspace/std/test Message-ID: <20101103134918.D7517282BE0@codespeak.net> Author: arigo Date: Wed Nov 3 14:49:17 2010 New Revision: 78668 Modified: pypy/branch/smalllong/pypy/objspace/std/test/test_smalllongobject.py Log: Add a (passing) test. Modified: pypy/branch/smalllong/pypy/objspace/std/test/test_smalllongobject.py ============================================================================== --- pypy/branch/smalllong/pypy/objspace/std/test/test_smalllongobject.py (original) +++ pypy/branch/smalllong/pypy/objspace/std/test/test_smalllongobject.py Wed Nov 3 14:49:17 2010 @@ -34,6 +34,10 @@ space.lshift(w5, space.wrap(49)), w14000000000000)) is False assert space.is_true(space.eq( space.lshift(w5, space.wrap(50)), w14000000000000)) is True + # + w_huge = space.sub(space.lshift(w5, space.wrap(150)), space.wrap(1)) + wx = space.and_(w14000000000000, w_huge) + assert space.is_true(space.eq(wx, w14000000000000)) class AppTestSmallLong(test_longobject.AppTestLong): From arigo at codespeak.net Wed Nov 3 14:49:41 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 3 Nov 2010 14:49:41 +0100 (CET) Subject: [pypy-svn] r78669 - in pypy/branch/smalllong/pypy: module/marshal/test objspace/std Message-ID: <20101103134941.D8B4C282BE7@codespeak.net> Author: arigo Date: Wed Nov 3 14:49:40 2010 New Revision: 78669 Modified: pypy/branch/smalllong/pypy/module/marshal/test/test_marshalimpl.py pypy/branch/smalllong/pypy/objspace/std/model.py Log: Multimethods are evil. The order of delegates must be carefully done right. Modified: pypy/branch/smalllong/pypy/module/marshal/test/test_marshalimpl.py ============================================================================== --- pypy/branch/smalllong/pypy/module/marshal/test/test_marshalimpl.py (original) +++ pypy/branch/smalllong/pypy/module/marshal/test/test_marshalimpl.py Wed Nov 3 14:49:40 2010 @@ -14,6 +14,7 @@ z = 0L z1 = marshal.loads(marshal.dumps(z)) assert z == z1 + assert type(z1) is long def test_unmarshal_int64(self): # test that we can unmarshal 64-bit ints on 32-bit platforms @@ -53,3 +54,10 @@ assert z == 10000000000 z = marshal.loads('I\x00\x1c\xf4\xab\xfd\xff\xff\xff') assert z == -10000000000 + + +class AppTestMarshalSmallLong(AppTestMarshalMore): + def setup_class(cls): + space = gettestobjspace(usemodules=('array',), + **{"objspace.std.withsmalllong": True}) + cls.space = space Modified: pypy/branch/smalllong/pypy/objspace/std/model.py ============================================================================== --- pypy/branch/smalllong/pypy/objspace/std/model.py (original) +++ pypy/branch/smalllong/pypy/objspace/std/model.py Wed Nov 3 14:49:40 2010 @@ -190,8 +190,8 @@ (smalllongobject.W_SmallLongObject, smalllongobject.delegate_Int2SmallLong), ] self.typeorder[smalllongobject.W_SmallLongObject] += [ - (floatobject.W_FloatObject, smalllongobject.delegate_SmallLong2Float), (longobject.W_LongObject, smalllongobject.delegate_SmallLong2Long), + (floatobject.W_FloatObject, smalllongobject.delegate_SmallLong2Float), (complexobject.W_ComplexObject, smalllongobject.delegate_SmallLong2Complex), ] self.typeorder[longobject.W_LongObject] += [ From ac at codespeak.net Wed Nov 3 15:01:22 2010 From: ac at codespeak.net (ac at codespeak.net) Date: Wed, 3 Nov 2010 15:01:22 +0100 (CET) Subject: [pypy-svn] r78670 - pypy/extradoc/planning/hg-migration Message-ID: <20101103140122.32A88282BDD@codespeak.net> Author: ac Date: Wed Nov 3 15:01:20 2010 New Revision: 78670 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: Update myself Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Wed Nov 3 15:01:20 2010 @@ -14,7 +14,7 @@ afa=Amaury Forgeot d Arc ericvrp=Eric van Riet Paap rxe=Richard Emslie -ac=Anders Chrigstrom +ac=Anders Chrigstrom auc=Aurelien Campeas benjamin=Benjamin Peterson ale=Anders Lehmann From david at codespeak.net Wed Nov 3 15:10:39 2010 From: david at codespeak.net (david at codespeak.net) Date: Wed, 3 Nov 2010 15:10:39 +0100 (CET) Subject: [pypy-svn] r78671 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . helper test Message-ID: <20101103141039.76EA1282BE8@codespeak.net> Author: david Date: Wed Nov 3 15:10:37 2010 New Revision: 78671 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py Log: Implement register spilling on the stack. Introduce a frame pointer to manage the stack frame For now with a fixed frame size of 10 words for spilling Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Wed Nov 3 15:10:37 2010 @@ -57,16 +57,21 @@ while(True): i += 1 fail_index += 1 - r = enc[i] - if r == '\xFE': + res = enc[i] + if res == '\xFE': continue - if r == '\xFF': + if res == '\xFF': break - if r == '\xFD': + if res == '\xFD': # imm value value = self.decode32(enc, i+1) i += 4 - else: + elif res == '\xFC': # stack location + stack_loc = self.decode32(enc, i+1) + #XXX ffuu use propper calculation here + value = self.decode32(stack, len(r.all_regs)*WORD+40-stack_loc*WORD) + i += 4 + else: # an int for now reg = ord(enc[i]) value = self.decode32(stack, reg*WORD) @@ -96,20 +101,24 @@ self.setup_failure_recovery() functype = lltype.Ptr(lltype.FuncType([lltype.Signed, lltype.Signed], lltype.Signed)) decode_registers_addr = llhelper(functype, self.failure_recovery_func) - self.mc.PUSH(range(12)) # registers r0 .. r11 + self.mc.PUSH([reg.value for reg in r.all_regs]) # registers r0 .. r10 self.mc.MOV_rr(r.r0.value, r.lr.value) # move mem block address, to r0 to pass as # parameter to next procedure call self.mc.MOV_rr(r.r1.value, r.sp.value) # pass the current stack pointer as second param self.mc.BL(rffi.cast(lltype.Signed, decode_registers_addr)) self.mc.MOV_rr(r.ip.value, r.r0.value) - self.mc.LDM(r.sp.value, range(12), w=1) # XXX Replace with POP instr. someday - + self.mc.LDM(r.sp.value, [reg.value for reg in r.all_regs], w=1) # XXX Replace with POP instr. someday self.mc.MOV_rr(r.r0.value, r.ip.value) - self.gen_func_epilog() def _gen_path_to_exit_path(self, op, args, regalloc, fcond=c.AL): + """ + \xFC = stack location + \xFD = imm location + \xFE = Empty arg + """ + box = Box() reg = regalloc.force_allocate_reg(box) # XXX free this memory @@ -118,13 +127,18 @@ j = 0 while(i < len(args)): if args[i]: - if not isinstance(args[i], ConstInt): - curreg = regalloc.make_sure_var_in_reg(args[i]) - mem[j] = chr(curreg.value) + loc = regalloc.loc(args[i]) + if loc.is_reg(): + mem[j] = chr(loc.value) j += 1 - else: + elif loc.is_imm(): mem[j] = '\xFD' - self.encode32(mem, j+1, args[i].getint()) + self.encode32(mem, j+1, loc.getint()) + j += 5 + else: + #print 'Encoding a stack location' + mem[j] = '\xFC' + self.encode32(mem, j+1, loc.position) j += 5 else: mem[j] = '\xFE' @@ -152,11 +166,14 @@ while(self.mc.curraddr() % FUNC_ALIGN != 0): self.mc.writechar(chr(0)) + epilog_size = 2*WORD def gen_func_epilog(self,cond=c.AL): + self.mc.MOV_rr(r.sp.value, r.fp.value) self.mc.LDM(r.sp.value, [reg.value for reg in r.callee_restored_registers], cond=cond, w=1) def gen_func_prolog(self): self.mc.PUSH([reg.value for reg in r.callee_saved_registers]) + self.mc.MOV_rr(r.fp.value, r.sp.value) def gen_bootstrap_code(self, inputargs, regalloc, looptoken): regs = [] @@ -175,6 +192,10 @@ self.align() loop_start=self.mc.curraddr() self.gen_func_prolog() + # XXX + #Patch the sp with a correct value + self.mc.SUB_ri(r.sp.value, r.sp.value, 10*WORD) + # END self.gen_bootstrap_code(inputargs, regalloc, looptoken) loop_head=self.mc.curraddr() looptoken._arm_bootstrap_code = loop_start @@ -185,7 +206,7 @@ # XXX consider merging ops with next one if it is an adecuate guard opnum = op.getopnum() fcond = self.operations[opnum](self, op, regalloc, fcond) - self.gen_func_epilog() + #self.gen_func_epilog() if self._debug_asm: self._dump_trace('loop.asm') print 'Done assembling' @@ -213,9 +234,13 @@ def _dump_trace(self, name): self.mc._dump_trace(name) - def _check_imm_arg(self, arg, size=0xFF): + def _check_imm_arg(self, arg, size=0xFF, allow_zero=True): + if allow_zero: + lower_bound = arg.getint() >= 0 + else: + lower_bound = arg.getint() > 0 #XXX check ranges for different operations - return isinstance(arg, ConstInt) and arg.getint() <= size and arg.getint() > 0 + return isinstance(arg, ConstInt) and arg.getint() <= size and lower_bound def patch_trace(self, faildescr, bridge_addr): # XXX make sure there is enough space at patch target @@ -228,8 +253,13 @@ if prev_loc.is_imm(): # XXX check size of imm for current instr self.mc.gen_load_int(loc.value, prev_loc.getint()) + elif loc.is_stack(): + self.mc.STR_ri(prev_loc.value, r.fp.value, loc.position*-WORD) + elif prev_loc.is_stack(): + self.mc.LDR_ri(loc.value, r.fp.value, prev_loc.position*-WORD) else: self.mc.MOV_rr(loc.value, prev_loc.value) + mov_loc_loc = regalloc_mov def make_operation_list(): def notimplemented(self, op, regalloc, fcond): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Wed Nov 3 15:10:37 2010 @@ -13,27 +13,29 @@ return fcond return f -def gen_emit_op_ri(opname, imm_size=0xFF, commutative=True): +def gen_emit_op_ri(opname, imm_size=0xFF, commutative=True, allow_zero=True): def f(self, op, regalloc, fcond): ri_op = getattr(self.mc, '%s_ri' % opname) rr_op = getattr(self.mc, '%s_rr' % opname) arg0 = op.getarg(0) arg1 = op.getarg(1) - imm_a0 = self._check_imm_arg(arg0, imm_size) - imm_a1 = self._check_imm_arg(arg1, imm_size) - res = regalloc.force_allocate_reg(op.result) + imm_a0 = self._check_imm_arg(arg0, imm_size, allow_zero=allow_zero) + imm_a1 = self._check_imm_arg(arg1, imm_size, allow_zero=allow_zero) if commutative and imm_a0: l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=imm_a0) l1 = regalloc.make_sure_var_in_reg(arg1) + res = regalloc.force_allocate_reg(op.result) ri_op(res.value, l1.value, imm=l0.getint(), cond=fcond) elif imm_a1: l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) l1 = regalloc.make_sure_var_in_reg(arg1, imm_fine=True) + res = regalloc.force_allocate_reg(op.result) ri_op(res.value, l0.value, imm=l1.getint(), cond=fcond) else: l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) l1 = regalloc.make_sure_var_in_reg(arg1, imm_fine=False) + res = regalloc.force_allocate_reg(op.result) rr_op(res.value, l0.value, l1.value) regalloc.possibly_free_vars_for_op(op) return fcond @@ -61,17 +63,18 @@ else: arg0 = op.getarg(1) arg1 = op.getarg(0) - res = regalloc.force_allocate_reg(op.result) # XXX consider swapping argumentes if arg0 is const imm_a0 = self._check_imm_arg(arg0) imm_a1 = self._check_imm_arg(arg1) if imm_a1 and not imm_a0: l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) - l1 = regalloc.make_sure_var_in_reg(arg1) + l1 = regalloc.make_sure_var_in_reg(arg1, [l0]) + res = regalloc.force_allocate_reg(op.result) self.mc.CMP_ri(l0.value, imm=l1.getint(), cond=fcond) else: l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) - l1 = regalloc.make_sure_var_in_reg(arg1, imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(arg1, [l0], imm_fine=False) + res = regalloc.force_allocate_reg(op.result) self.mc.CMP_rr(l0.value, l1.value) inv = c.get_opposite_of(condition) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py Wed Nov 3 15:10:37 2010 @@ -1,8 +1,16 @@ +from pypy.jit.metainterp.history import INT +from pypy.jit.backend.arm.arch import WORD class AssemblerLocation(object): pass def is_imm(self): return False + def is_stack(self): + return False + + def is_reg(self): + return False + class RegisterLocation(AssemblerLocation): def __init__(self, value): @@ -11,6 +19,9 @@ def __repr__(self): return 'r%d' % self.value + def is_reg(self): + return True + class ImmLocation(AssemblerLocation): _immutable_ = True def __init__(self, value): @@ -20,7 +31,32 @@ return self.value def __repr__(self): - return "ImmedLoc(%d)" % (self.value) + return "imm(%d)" % (self.value) def is_imm(self): return True + +class StackLocation(AssemblerLocation): + _immutable_ = True + def __init__(self, position, num_words=1, type=INT): + self.position = position + self.width = num_words * WORD + # One of INT, REF, FLOAT + assert num_words == 1 + assert type == INT + #self.type = type + + def frame_size(self): + return self.width // WORD + + def __repr__(self): + return 'SP+%d' % (self.position,) + + def location_code(self): + return 'b' + + def assembler(self): + return repr(self) + + def is_stack(self): + return True Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Wed Nov 3 15:10:37 2010 @@ -21,19 +21,17 @@ def emit_op_int_add(self, op, regalloc, fcond): # assuming only one argument is constant - res = regalloc.force_allocate_reg(op.result) a0 = op.getarg(0) a1 = op.getarg(1) imm_a0 = isinstance(a0, ConstInt) and (a0.getint() <= 0xFF or -1 * a0.getint() <= 0xFF) imm_a1 = isinstance(a1, ConstInt) and (a1.getint() <= 0xFF or -1 * a1.getint() <= 0xFF) - l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=imm_a0) - l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=imm_a1) if imm_a0: imm_a0, imm_a1 = imm_a1, imm_a0 a0, a1 = a1, a0 if imm_a1: l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=True) + res = regalloc.force_allocate_reg(op.result) if l1.getint() < 0: self.mc.SUB_ri(res.value, l0.value, -1 * l1.getint()) else: @@ -41,6 +39,7 @@ else: l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=False) + res = regalloc.force_allocate_reg(op.result) self.mc.ADD_rr(res.value, l0.value, l1.value) regalloc.possibly_free_vars_for_op(op) @@ -48,13 +47,13 @@ def emit_op_int_sub(self, op, regalloc, fcond): # assuming only one argument is constant - res = regalloc.force_allocate_reg(op.result) a0 = op.getarg(0) a1 = op.getarg(1) imm_a0 = isinstance(a0, ConstInt) and (a0.getint() <= 0xFF or -1 * a0.getint() <= 0xFF) imm_a1 = isinstance(a1, ConstInt) and (a1.getint() <= 0xFF or -1 * a1.getint() <= 0xFF) l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=imm_a0) l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=imm_a1) + res = regalloc.force_allocate_reg(op.result) if imm_a0: value = l0.getint() if value < 0: @@ -65,7 +64,7 @@ # reverse substract ftw self.mc.RSB_ri(res.value, l1.value, value) elif imm_a1: - value = a1.getint() + value = l1.getint() if value < 0: self.mc.ADD_ri(res.value, l0.value, -1 * value) else: @@ -77,9 +76,9 @@ return fcond def emit_op_int_mul(self, op, regalloc, fcond): - res = regalloc.force_allocate_reg(op.result) reg1 = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) reg2 = regalloc.make_sure_var_in_reg(op.getarg(1), imm_fine=False) + res = regalloc.force_allocate_reg(op.result) self.mc.MUL(res.value, reg1.value, reg2.value) regalloc.possibly_free_vars_for_op(op) return fcond @@ -91,7 +90,7 @@ emit_op_int_and = gen_emit_op_ri('AND') emit_op_int_or = gen_emit_op_ri('ORR') emit_op_int_xor = gen_emit_op_ri('EOR') - emit_op_int_lshift = gen_emit_op_ri('LSL', imm_size=0x1F, commutative=False) + emit_op_int_lshift = gen_emit_op_ri('LSL', imm_size=0x1F, allow_zero=False, commutative=False) emit_op_int_rshift = gen_emit_op_ri('ASR', imm_size=0x1F, commutative=False) emit_op_uint_rshift = gen_emit_op_ri('LSR', imm_size=0x1F, commutative=False) @@ -143,6 +142,7 @@ descr._failure_recovery_code = memaddr descr._arm_guard_cond = fcond descr._arm_guard_size = self.mc.curraddr() - descr._arm_guard_code + regalloc.possibly_free_vars_for_op(op) def emit_op_guard_true(self, op, regalloc, fcond): assert fcond == c.LE @@ -162,10 +162,11 @@ def emit_op_jump(self, op, regalloc, fcond): registers = op.getdescr()._arm_arglocs for i in range(op.numargs()): - #XXX we are assuming that every value is in a register for now - reg = regalloc.make_sure_var_in_reg(op.getarg(i), imm_fine=False) - inpreg = registers[i] - self.mc.MOV_rr(inpreg.value, reg.value) + # avoid moving stuff twice + loc = registers[i] + prev_loc = regalloc.loc(op.getarg(i)) + self.mov_loc_loc(prev_loc, loc) + loop_code = op.getdescr()._arm_loop_code self.mc.B(loop_code, fcond) return fcond Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Wed Nov 3 15:10:37 2010 @@ -15,6 +15,7 @@ def update_bindings(self, enc, inputargs): j = 0 for i in range(len(inputargs)): + # XXX decode imm and and stack locs while enc[j] == '\xFE': j += 1 self.force_allocate_reg(inputargs[i], selected_reg=r.all_regs[ord(enc[j])]) @@ -24,6 +25,11 @@ return locations.ImmLocation(c.value) class ARMFrameManager(FrameManager): + def __init__(self): + FrameManager.__init__(self) + self.frame_depth = 1 + @staticmethod def frame_pos(loc, type): - pass + # XXX for now we only have one word stack locs + return locations.StackLocation(loc) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py Wed Nov 3 15:10:37 2010 @@ -10,7 +10,7 @@ lr = r14 pc = r15 -all_regs = registers[:12] +all_regs = registers[:11] callee_resp = [r4, r5, r6, r7, r8, r9, r10, r11] callee_saved_registers = callee_resp+[lr] Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py Wed Nov 3 15:10:37 2010 @@ -113,7 +113,8 @@ loop_head = self.a.mc.curraddr() self.a.mc.ADD_ri(r.r1.value, r.r1.value, 1) self.a.mc.CMP_ri(r.r1.value, 9) - self.a.mc.LDR_ri(r.pc.value, r.pc.value, imm=4, cond=c.NE) + self.a.mc.LDR_ri(r.pc.value, r.pc.value, + imm=self.a.epilog_size, cond=c.NE) # we want to read after the last instr. of gen_func_prolog self.a.mc.MOV_rr(r.r0.value, r.r1.value) self.a.gen_func_epilog() self.a.mc.write32(loop_head) From afa at codespeak.net Wed Nov 3 15:18:06 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 3 Nov 2010 15:18:06 +0100 (CET) Subject: [pypy-svn] r78672 - pypy/extradoc/planning/hg-migration Message-ID: <20101103141806.276D1282BE8@codespeak.net> Author: afa Date: Wed Nov 3 15:18:04 2010 New Revision: 78672 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: Do like everybody. Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Wed Nov 3 15:18:04 2010 @@ -11,7 +11,7 @@ mwh=Michael Hudson tismer=Christian Tismer hpk=Holger Krekel -afa=Amaury Forgeot d Arc +afa=Amaury Forgeot d'Arc ericvrp=Eric van Riet Paap rxe=Richard Emslie ac=Anders Chrigstrom From david at codespeak.net Wed Nov 3 15:41:46 2010 From: david at codespeak.net (david at codespeak.net) Date: Wed, 3 Nov 2010 15:41:46 +0100 (CET) Subject: [pypy-svn] r78673 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101103144146.EEC3B282BEF@codespeak.net> Author: david Date: Wed Nov 3 15:41:44 2010 New Revision: 78673 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Log: Patch generated code to update the stack pointer by the correct amount of words to fit all spilled registers Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Wed Nov 3 15:41:44 2010 @@ -192,25 +192,44 @@ self.align() loop_start=self.mc.curraddr() self.gen_func_prolog() - # XXX - #Patch the sp with a correct value - self.mc.SUB_ri(r.sp.value, r.sp.value, 10*WORD) + # Generate NOP as placeholder to patch the instruction(s) to update the + # sp according to the number of spilled variables + sp_patch_location = self.mc.curraddr() + for _ in range((self.mc.size_of_gen_load_int+WORD)//WORD): + self.mc.MOV_rr(r.r0.value, r.r0.value) # END self.gen_bootstrap_code(inputargs, regalloc, looptoken) loop_head=self.mc.curraddr() looptoken._arm_bootstrap_code = loop_start looptoken._arm_loop_code = loop_head fcond=c.AL - #print inputargs, operations + print inputargs, operations for op in operations: # XXX consider merging ops with next one if it is an adecuate guard opnum = op.getopnum() fcond = self.operations[opnum](self, op, regalloc, fcond) + + self._patch_sp_offset(sp_patch_location, regalloc) #self.gen_func_epilog() if self._debug_asm: self._dump_trace('loop.asm') print 'Done assembling' + def _patch_sp_offset(self, addr, regalloc): + cb = ARMv7InMemoryBuilder(addr, ARMv7InMemoryBuilder.size_of_gen_load_int) + n = regalloc.frame_manager.frame_depth*WORD + if n == 1: + return + if n <= 0xFF: + cb.SUB_ri(r.sp.value, r.sp.value, n) + else: + b = Box() + reg = regalloc.force_allocate_reg(b) + cb.gen_load_int(reg.value, n) + cb.SUB_rr(r.sp.value, r.sp.value, reg.value) + regalloc.possibly_free_var(reg) + + def assemble_bridge(self, faildescr, inputargs, operations): enc = rffi.cast(rffi.CCHARP, faildescr._failure_recovery_code) longevity = compute_vars_longevity(inputargs, operations) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Wed Nov 3 15:41:44 2010 @@ -4,7 +4,7 @@ from pypy.jit.backend.arm import locations class ARMRegisterManager(RegisterManager): - all_regs = r.all_regs + all_regs = r.all_regs[:3] box_types = None # or a list of acceptable types no_lower_byte_regs = all_regs save_around_call_regs = all_regs From david at codespeak.net Wed Nov 3 15:44:56 2010 From: david at codespeak.net (david at codespeak.net) Date: Wed, 3 Nov 2010 15:44:56 +0100 (CET) Subject: [pypy-svn] r78674 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101103144456.3ACC0282BEA@codespeak.net> Author: david Date: Wed Nov 3 15:44:54 2010 New Revision: 78674 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Log: Extract patch preparation to a function Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Wed Nov 3 15:44:54 2010 @@ -192,12 +192,9 @@ self.align() loop_start=self.mc.curraddr() self.gen_func_prolog() - # Generate NOP as placeholder to patch the instruction(s) to update the - # sp according to the number of spilled variables - sp_patch_location = self.mc.curraddr() - for _ in range((self.mc.size_of_gen_load_int+WORD)//WORD): - self.mc.MOV_rr(r.r0.value, r.r0.value) - # END + + sp_patch_location = self._prepare_sp_patch_location() + self.gen_bootstrap_code(inputargs, regalloc, looptoken) loop_head=self.mc.curraddr() looptoken._arm_bootstrap_code = loop_start @@ -210,11 +207,19 @@ fcond = self.operations[opnum](self, op, regalloc, fcond) self._patch_sp_offset(sp_patch_location, regalloc) - #self.gen_func_epilog() + if self._debug_asm: self._dump_trace('loop.asm') print 'Done assembling' + def _prepare_sp_patch_location(self): + """Generate NOPs as placeholder to patch the instruction(s) to update the + sp according to the number of spilled variables""" + l = self.mc.curraddr() + for _ in range((self.mc.size_of_gen_load_int+WORD)//WORD): + self.mc.MOV_rr(r.r0.value, r.r0.value) + return l + def _patch_sp_offset(self, addr, regalloc): cb = ARMv7InMemoryBuilder(addr, ARMv7InMemoryBuilder.size_of_gen_load_int) n = regalloc.frame_manager.frame_depth*WORD From afa at codespeak.net Wed Nov 3 15:55:49 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 3 Nov 2010 15:55:49 +0100 (CET) Subject: [pypy-svn] r78675 - pypy/branch/leak-finder-more/pypy/module/cpyext/test Message-ID: <20101103145549.272A6282BF0@codespeak.net> Author: afa Date: Wed Nov 3 15:55:48 2010 New Revision: 78675 Modified: pypy/branch/leak-finder-more/pypy/module/cpyext/test/test_borrow.py Log: Fix mixed test output. Modified: pypy/branch/leak-finder-more/pypy/module/cpyext/test/test_borrow.py ============================================================================== --- pypy/branch/leak-finder-more/pypy/module/cpyext/test/test_borrow.py (original) +++ pypy/branch/leak-finder-more/pypy/module/cpyext/test/test_borrow.py Wed Nov 3 15:55:48 2010 @@ -31,6 +31,7 @@ g = PyTuple_GetItem(t, 0); // borrows reference again printf("Refcnt4: %i\\n", f->ob_refcnt); printf("COMPARE: %i\\n", f == g); + fflush(stdout); Py_DECREF(t); Py_RETURN_TRUE; """), From david at codespeak.net Wed Nov 3 16:22:16 2010 From: david at codespeak.net (david at codespeak.net) Date: Wed, 3 Nov 2010 16:22:16 +0100 (CET) Subject: [pypy-svn] r78676 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . test Message-ID: <20101103152216.0C9A3282BAD@codespeak.net> Author: david Date: Wed Nov 3 16:22:15 2010 New Revision: 78676 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Log: Fix issue with the update flags bit not getting set Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Wed Nov 3 16:22:15 2010 @@ -184,7 +184,7 @@ def reg_operation(rt, rn, rm, imm, s, shifttype): # XXX encode shiftype correctly - return ((s & 0x1 << 20) + return ((s & 0x1) << 20 | (rn & 0xFF) << 16 | (rt & 0xFF) << 12 | (imm & 0x1F) << 7 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Wed Nov 3 16:22:15 2010 @@ -209,22 +209,29 @@ return f def gen_test_data_reg_func(name, table): + op_name = name[:name.index('_')] if name[-2:] == 'ri': def f(self): func = getattr(self.cb, name) func(r.r3.value, r.r7.value, 12) - self.assert_equal('%s r3, r7, #12' % name[:name.index('_')]) + func(r.r3.value, r.r7.value, 12, s=1) + self.assert_equal(""" +%(name)s r3, r7, #12 +%(name)sS r3, r7, #12""" % {'name': op_name}) elif table['base'] and table['result']: def f(self): func = getattr(self.cb, name) func(r.r3.value, r.r7.value, r.r12.value) - self.assert_equal('%s r3, r7, r12' % name[:name.index('_')]) + func(r.r3.value, r.r7.value, r.r12.value, s=1) + self.assert_equal("""%(name)s r3, r7, r12 +%(name)sS r3, r7, r12 +""" % {'name':op_name}) else: def f(self): func = getattr(self.cb, name) func(r.r3.value, r.r7.value) - self.assert_equal('%s r3, r7' % name[:name.index('_')]) + self.assert_equal("""%(name)s r3, r7""" % {'name':op_name}) return f From david at codespeak.net Wed Nov 3 16:30:23 2010 From: david at codespeak.net (david at codespeak.net) Date: Wed, 3 Nov 2010 16:30:23 +0100 (CET) Subject: [pypy-svn] r78677 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101103153023.B721E282BEC@codespeak.net> Author: david Date: Wed Nov 3 16:30:22 2010 New Revision: 78677 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Log: Implement int_add_ovf and int_sub_ovf Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Wed Nov 3 16:30:22 2010 @@ -222,9 +222,9 @@ def _patch_sp_offset(self, addr, regalloc): cb = ARMv7InMemoryBuilder(addr, ARMv7InMemoryBuilder.size_of_gen_load_int) - n = regalloc.frame_manager.frame_depth*WORD - if n == 1: + if regalloc.frame_manager.frame_depth == 1: return + n = regalloc.frame_manager.frame_depth*WORD if n <= 0xFF: cb.SUB_ri(r.sp.value, r.sp.value, n) else: Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Wed Nov 3 16:30:22 2010 @@ -33,14 +33,14 @@ l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=True) res = regalloc.force_allocate_reg(op.result) if l1.getint() < 0: - self.mc.SUB_ri(res.value, l0.value, -1 * l1.getint()) + self.mc.SUB_ri(res.value, l0.value, -1 * l1.getint(), s=1) else: - self.mc.ADD_ri(res.value, l0.value, l1.getint()) + self.mc.ADD_ri(res.value, l0.value, l1.getint(), s=1) else: l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=False) res = regalloc.force_allocate_reg(op.result) - self.mc.ADD_rr(res.value, l0.value, l1.value) + self.mc.ADD_rr(res.value, l0.value, l1.value, s=1) regalloc.possibly_free_vars_for_op(op) return fcond @@ -58,19 +58,19 @@ value = l0.getint() if value < 0: # XXX needs a test - self.mc.ADD_ri(res.value, l1.value, -1 * value) - self.mc.MVN_rr(res.value, l1.value) + self.mc.ADD_ri(res.value, l1.value, -1 * value, s=1) + self.mc.MVN_rr(res.value, l1.value, s=1) else: # reverse substract ftw - self.mc.RSB_ri(res.value, l1.value, value) + self.mc.RSB_ri(res.value, l1.value, value, s=1) elif imm_a1: value = l1.getint() if value < 0: - self.mc.ADD_ri(res.value, l0.value, -1 * value) + self.mc.ADD_ri(res.value, l0.value, -1 * value, s=1) else: - self.mc.SUB_ri(res.value, l0.value, value) + self.mc.SUB_ri(res.value, l0.value, value, s=1) else: - self.mc.SUB_rr(res.value, l0.value, l1.value) + self.mc.SUB_rr(res.value, l0.value, l1.value, s=1) regalloc.possibly_free_vars_for_op(op) return fcond @@ -107,6 +107,8 @@ emit_op_uint_lt = gen_emit_cmp_op(c.HI, inverse=True) emit_op_uint_ge = gen_emit_cmp_op(c.LS, inverse=True) + emit_op_int_add_ovf = emit_op_int_add + emit_op_int_sub_ovf = emit_op_int_sub class UnaryIntOpAssembler(object): @@ -143,18 +145,20 @@ descr._arm_guard_cond = fcond descr._arm_guard_size = self.mc.curraddr() - descr._arm_guard_code regalloc.possibly_free_vars_for_op(op) + return c.AL def emit_op_guard_true(self, op, regalloc, fcond): assert fcond == c.LE cond = c.get_opposite_of(fcond) assert cond == c.GT - self._emit_guard(op, regalloc, cond) - return c.AL + return self._emit_guard(op, regalloc, cond) def emit_op_guard_false(self, op, regalloc, fcond): assert fcond == c.EQ - self._emit_guard(op, regalloc, fcond) - return c.AL + return self._emit_guard(op, regalloc, fcond) + + def emit_op_guard_no_overflow(self, op, regalloc, fcond): + return self._emit_guard(op, regalloc, c.VS) class OpAssembler(object): _mixin_ = True Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Wed Nov 3 16:30:22 2010 @@ -4,7 +4,7 @@ from pypy.jit.backend.arm import locations class ARMRegisterManager(RegisterManager): - all_regs = r.all_regs[:3] + all_regs = r.all_regs box_types = None # or a list of acceptable types no_lower_byte_regs = all_regs save_around_call_regs = all_regs From david at codespeak.net Wed Nov 3 16:53:24 2010 From: david at codespeak.net (david at codespeak.net) Date: Wed, 3 Nov 2010 16:53:24 +0100 (CET) Subject: [pypy-svn] r78678 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . test Message-ID: <20101103155324.48873282B9E@codespeak.net> Author: david Date: Wed Nov 3 16:53:22 2010 New Revision: 78678 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Log: Correctly set the update flags bit on multiply operations Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Wed Nov 3 16:53:22 2010 @@ -146,20 +146,29 @@ def define_multiply_instructions(name, table): n = (table['op'] & 0xF) << 20 | 0x9 << 4 if 'acc' in table and table['acc']: - def f(self, rd, rn, rm, ra, cond=cond.AL, s=0): - self.write32(n - | cond << 28 - | (s & 0x1) - | (rd & 0xF) << 16 - | (ra & 0xF) << 12 - | (rm & 0xF) << 8 - | (rn & 0xF)) + if 'update_flags' in table and table['update_flags']: + def f(self, rd, rn, rm, ra, cond=cond.AL, s=0): + self.write32(n + | cond << 28 + | (s & 0x1) << 20 + | (rd & 0xF) << 16 + | (ra & 0xF) << 12 + | (rm & 0xF) << 8 + | (rn & 0xF)) + else: + def f(self, rd, rn, rm, ra, cond=cond.AL): + self.write32(n + | cond << 28 + | (rd & 0xF) << 16 + | (ra & 0xF) << 12 + | (rm & 0xF) << 8 + | (rn & 0xF)) + elif 'long' in table and table['long']: - def f(self, rdhi, rdlo, rn, rm, cond=cond.AL, s=0): + def f(self, rdhi, rdlo, rn, rm, cond=cond.AL): assert rdhi != rdlo self.write32(n | cond << 28 - | (s & 0x1) | (rdhi & 0xF) << 16 | (rdlo & 0xF) << 12 | (rm & 0xF) << 8 @@ -168,7 +177,7 @@ def f(self, rd, rn, rm, cond=cond.AL, s=0): self.write32(n | cond << 28 - | (s & 0x1) + | (s & 0x1) << 20 | (rd & 0xF) << 16 | (rm & 0xF) << 8 | (rn & 0xF)) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py Wed Nov 3 16:53:22 2010 @@ -99,7 +99,7 @@ multiply = { 'MUL': {'op':0x0}, - 'MLA': {'op':0x2, 'acc': True}, + 'MLA': {'op':0x2, 'acc': True, 'update_flags':True}, 'UMAAL': {'op':0x4, 'long': True}, 'MLS': {'op':0x6, 'acc': True}, 'UMULL': {'op':0x8, 'long': True}, Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Wed Nov 3 16:53:22 2010 @@ -167,20 +167,30 @@ def gen_test_mul_func(name, table): if 'acc' in table and table['acc']: - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, r.r7.value, r.r12.value, r.r13.value) - self.assert_equal('%s r3, r7, r12, r13' % name) + if 'update_flags' in table and table['update_flags']: + def f(self): + func = getattr(self.cb, name) + func(r.r3.value, r.r7.value, r.r12.value, r.r13.value) + func(r.r3.value, r.r7.value, r.r12.value, r.r13.value, s=1) + self.assert_equal(""" + %(name)s r3, r7, r12, r13 + %(name)sS r3, r7, r12, r13 + """ % {'name':name}) + else: + def f(self): + func = getattr(self.cb, name) + func(r.r3.value, r.r7.value, r.r12.value, r.r13.value) + self.assert_equal("""%(name)s r3, r7, r12, r13 """ % {'name':name}) elif 'long' in table and table['long']: def f(self): func = getattr(self.cb, name) func(r.r3.value, r.r13.value, r.r7.value, r.r12.value) - self.assert_equal('%s r13, r3, r7, r12' % name) + self.assert_equal("""%(name)s r13, r3, r7, r12 """ % {'name':name}) else: def f(self): func = getattr(self.cb, name) func(r.r3.value, r.r7.value, r.r12.value) - self.assert_equal('%s r3, r7, r12' % name) + self.assert_equal("""%(name)s r3, r7, r12 """ % {'name':name}) return f def gen_test_data_reg_shift_reg_func(name, table): From afa at codespeak.net Wed Nov 3 17:22:14 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 3 Nov 2010 17:22:14 +0100 (CET) Subject: [pypy-svn] r78679 - pypy/branch/leak-finder-more/pypy/module/cpyext Message-ID: <20101103162214.35D98282B9E@codespeak.net> Author: afa Date: Wed Nov 3 17:22:12 2010 New Revision: 78679 Modified: pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py Log: "Fix" most leaks in cpyext tests Modified: pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py ============================================================================== --- pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py (original) +++ pypy/branch/leak-finder-more/pypy/module/cpyext/typeobject.py Wed Nov 3 17:22:12 2010 @@ -182,10 +182,10 @@ subtype = rffi.cast(PyTypeObjectPtr, make_ref(space, w_subtype)) try: - obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds) + w_obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds) finally: Py_DecRef(space, w_subtype) - return obj + return w_obj @specialize.memo() def get_new_method_def(space): @@ -433,6 +433,12 @@ finish_type_1(space, pto) finish_type_2(space, pto, w_type) + if space.type(w_type).is_cpytype(): + # XXX Types with a C metatype are never freed, try to see why... + render_immortal(pto, w_type) + lltype.render_immortal(pto) + lltype.render_immortal(pto.c_tp_name) + pto.c_tp_basicsize = rffi.sizeof(typedescr.basestruct) if pto.c_tp_base: if pto.c_tp_base.c_tp_basicsize > pto.c_tp_basicsize: @@ -554,6 +560,8 @@ lltype.render_immortal(py_type.c_tp_dict) else: lltype.render_immortal(py_type.c_tp_name) + if not w_obj.is_cpytype() and w_obj.is_heaptype(): + lltype.render_immortal(py_type) def finish_type_1(space, pto): """ From hakanardo at codespeak.net Wed Nov 3 20:22:21 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Wed, 3 Nov 2010 20:22:21 +0100 (CET) Subject: [pypy-svn] r78680 - in pypy/branch/jit-unroll-loops/pypy/jit: metainterp/optimizeopt metainterp/test tl Message-ID: <20101103192221.5CF07282BDB@codespeak.net> Author: hakanardo Date: Wed Nov 3 20:22:19 2010 New Revision: 78680 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit.py pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py Log: Problematic getfield case where the heap optimizer fails in the loop but not in the preamble Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py Wed Nov 3 20:22:19 2010 @@ -97,7 +97,7 @@ self.emitting_operation(op) self.next_optimization.propagate_forward(op) - def emitting_operation(self, op): + def emitting_operation(self, op): if op.has_no_side_effect(): return if op.is_ovf(): Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Wed Nov 3 20:22:19 2010 @@ -3518,6 +3518,41 @@ self.node.value = 5 self.optimize_loop(ops, expected) + def test_getfield_guard_const(self): + ops = """ + [p0] + p20 = getfield_gc(p0, descr=nextdescr) + guard_nonnull(p20) [] + guard_class(p20, ConstClass(node_vtable)) [] + guard_class(p20, ConstClass(node_vtable)) [] + p23 = getfield_gc(p20, descr=valuedescr) + guard_isnull(p23) [] + guard_class(p20, ConstClass(node_vtable)) [] + guard_value(p20, ConstPtr(myptr)) [] + + p37 = getfield_gc(p0, descr=nextdescr) + guard_nonnull(p37) [] + guard_class(p37, ConstClass(node_vtable)) [] + guard_class(p37, ConstClass(node_vtable)) [] + p40 = getfield_gc(p37, descr=valuedescr) + guard_isnull(p40) [] + guard_class(p37, ConstClass(node_vtable)) [] + guard_value(p37, ConstPtr(myptr)) [] + + p64 = call_may_force(p23, p40, descr=plaincalldescr) + jump(p0) + """ + expected = """ + [p0] + p20 = getfield_gc(p0, descr=nextdescr) + guard_value(p20, ConstPtr(myptr)) [] + p23 = getfield_gc(p20, descr=valuedescr) + guard_isnull(p23) [] + p64 = call_may_force(NULL, NULL, descr=plaincalldescr) + jump(p0) + """ + self.optimize_loop(ops, expected, expected) + def test_addsub_ovf(self): ops = """ [i0] @@ -4506,6 +4541,7 @@ # can be raised by ll_str2unicode() + ##class TestOOtype(OptimizeOptTest, OOtypeMixin): ## def test_instanceof(self): Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit.py Wed Nov 3 20:22:19 2010 @@ -38,7 +38,7 @@ config.objspace.allworkingmodules = False config.objspace.usemodules.pypyjit = True config.objspace.usemodules.array = True -config.objspace.usemodules._weakref = False +config.objspace.usemodules._weakref = True config.objspace.usemodules._sre = False # config.objspace.usemodules._ffi = True Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_child.py Wed Nov 3 20:22:19 2010 @@ -35,5 +35,5 @@ warmspot.jittify_and_run(interp, graph, [], policy=policy, listops=True, CPUClass=CPUClass, backendopt=True, inline=False, - optimizer=OPTIMIZER_NO_UNROLL) + optimizer=OPTIMIZER_FULL) Modified: pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/tl/pypyjit_demo.py Wed Nov 3 20:22:19 2010 @@ -58,27 +58,38 @@ ## #img=array('h',(1,2,3,4)) ## print f(3) - from array import array - class Circular(array): - def __new__(cls): - self = array.__new__(cls, 'i', range(16)) - return self - def __getitem__(self, i): - #assert self.__len__() == 16 - return array.__getitem__(self, i & 15) +## from array import array +## class Circular(array): +## def __new__(cls): +## self = array.__new__(cls, 'i', range(16)) +## return self +## def __getitem__(self, i): +## #assert self.__len__() == 16 +## return array.__getitem__(self, i & 15) - def main(): - buf = Circular() - i = 10 - sa = 0 - while i < 20: - #sa += buf[i-2] + buf[i-1] + buf[i] + buf[i+1] + buf[i+2] - sa += buf[i] - i += 1 - return sa +## def main(): +## buf = Circular() +## i = 10 +## sa = 0 +## while i < 20: +## #sa += buf[i-2] + buf[i-1] + buf[i] + buf[i+1] + buf[i+2] +## sa += buf[i] +## i += 1 +## return sa import pypyjit pypyjit.set_param(threshold=3, inlining=True) +## print main() + + def main(): + i=2 + sa=0 + while i < 10: + #sa+=max(range(i)) + a = range + b = max([i]) + i+=1 + return sa print main() except Exception, e: From hakanardo at codespeak.net Wed Nov 3 21:48:46 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Wed, 3 Nov 2010 21:48:46 +0100 (CET) Subject: [pypy-svn] r78681 - in pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test Message-ID: <20101103204846.99C36282BF0@codespeak.net> Author: hakanardo Date: Wed Nov 3 21:48:43 2010 New Revision: 78681 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Log: This test should fail on trunk aswell. Problem is that getvalue() uses getinterned(), while calling make_constat() on a value already in cached_fields does not. Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Wed Nov 3 21:48:43 2010 @@ -320,7 +320,6 @@ self.i = 0 while self.i < len(self.loop.operations): op = self.loop.operations[self.i] - #print "OP: %s" % op self.first_optimization.propagate_forward(op) self.i += 1 self.loop.operations = self.newoperations Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Wed Nov 3 21:48:43 2010 @@ -3553,6 +3553,41 @@ """ self.optimize_loop(ops, expected, expected) + def test_getfield_guard_const_preamble(self): + ops = """ + [p0] + p01 = getfield_gc(p0, descr=nextdescr) + p02 = getfield_gc(p01, descr=valuedescr) + guard_value(p01, ConstPtr(myptr)) [] + p11 = getfield_gc(p0, descr=nextdescr) + p12 = getfield_gc(p11, descr=valuedescr) + guard_value(p11, ConstPtr(myptr)) [] + p64 = call_may_force(p02, p12, descr=plaincalldescr) + + p21 = getfield_gc(p0, descr=nextdescr) + p22 = getfield_gc(p21, descr=valuedescr) + guard_value(p21, ConstPtr(myptr)) [] + p31 = getfield_gc(p0, descr=nextdescr) + p32 = getfield_gc(p31, descr=valuedescr) + guard_value(p31, ConstPtr(myptr)) [] + p65 = call_may_force(p22, p32, descr=plaincalldescr) + jump(p0) + """ + expected = """ + [p0] + p01 = getfield_gc(p0, descr=nextdescr) + p02 = getfield_gc(p01, descr=valuedescr) + guard_value(p01, ConstPtr(myptr)) [] + p64 = call_may_force(p02, p02, descr=plaincalldescr) + + p21 = getfield_gc(p0, descr=nextdescr) + p22 = getfield_gc(p21, descr=valuedescr) + guard_value(p21, ConstPtr(myptr)) [] + p65 = call_may_force(p22, p22, descr=plaincalldescr) + jump(p0) + """ + self.optimize_loop(ops, expected, expected) + def test_addsub_ovf(self): ops = """ [i0] From afa at codespeak.net Wed Nov 3 22:51:21 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 3 Nov 2010 22:51:21 +0100 (CET) Subject: [pypy-svn] r78682 - pypy/branch/fast-forward/pypy/rpython/lltypesystem Message-ID: <20101103215121.C461D282BD4@codespeak.net> Author: afa Date: Wed Nov 3 22:51:14 2010 New Revision: 78682 Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py Log: Forgot to add this helper class Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py (original) +++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py Wed Nov 3 22:51:14 2010 @@ -944,6 +944,17 @@ def __exit__(self, *args): free_charp(self.buf) + +class scoped_nonmovingbuffer: + def __init__(self, data): + self.data = data + def __enter__(self): + self.buf = get_nonmovingbuffer(self.data) + return self.buf + def __exit__(self, *args): + free_nonmovingbuffer(self.data, self.buf) + + class scoped_alloc_buffer: def __init__(self, size): self.size = size From afa at codespeak.net Wed Nov 3 23:32:45 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 3 Nov 2010 23:32:45 +0100 (CET) Subject: [pypy-svn] r78684 - pypy/branch/fast-forward/pypy/module/_multiprocessing Message-ID: <20101103223245.ABBF0282B9C@codespeak.net> Author: afa Date: Wed Nov 3 23:32:43 2010 New Revision: 78684 Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Log: One translation fix Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Wed Nov 3 23:32:43 2010 @@ -5,6 +5,7 @@ OperationError, wrap_oserror, operationerrfmt) from pypy.rpython.lltypesystem import rffi, lltype, llmemory from pypy.rlib.rarithmetic import intmask +from pypy.rlib import rpoll import sys READABLE = 1 @@ -322,7 +323,6 @@ return self.fd >= 0 and self.fd < rpoll.FD_SETSIZE def do_poll(self, space, timeout): - from pypy.rlib import rpoll if not self._check_fd(): raise OperationError(space.w_IOError, space.wrap( "handle out of range in select()")) From afa at codespeak.net Thu Nov 4 01:09:26 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 4 Nov 2010 01:09:26 +0100 (CET) Subject: [pypy-svn] r78685 - in pypy/branch/fast-forward/pypy/module: _warnings _warnings/test sys Message-ID: <20101104000926.14A4A282B9C@codespeak.net> Author: afa Date: Thu Nov 4 01:09:23 2010 New Revision: 78685 Added: pypy/branch/fast-forward/pypy/module/_warnings/ (props changed) pypy/branch/fast-forward/pypy/module/_warnings/__init__.py (contents, props changed) pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py (contents, props changed) pypy/branch/fast-forward/pypy/module/_warnings/test/ (props changed) pypy/branch/fast-forward/pypy/module/_warnings/test/test_warnings.py (contents, props changed) Modified: pypy/branch/fast-forward/pypy/module/sys/__init__.py Log: Start implementing the _warnings module Added: pypy/branch/fast-forward/pypy/module/_warnings/__init__.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/pypy/module/_warnings/__init__.py Thu Nov 4 01:09:23 2010 @@ -0,0 +1,21 @@ +from pypy.interpreter.mixedmodule import MixedModule + +class Module(MixedModule): + """provides basic warning filtering support. + It is a helper module to speed up interpreter start-up.""" + + interpleveldefs = { + 'warn' : 'interp_warnings.warn', + 'warn_explicit': 'interp_warnings.warn_explicit', + } + + appleveldefs = { + } + + def setup_after_space_initialization(self): + from pypy.module._warnings.interp_warnings import State + state = self.space.fromcache(State) + self.setdictvalue(self.space, "filters", state.w_filters) + self.setdictvalue(self.space, "once_registry", state.w_once_registry) + self.setdictvalue(self.space, "default_action", state.w_default_action) + Added: pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py Thu Nov 4 01:09:23 2010 @@ -0,0 +1,341 @@ +from pypy.interpreter.gateway import unwrap_spec +from pypy.interpreter.baseobjspace import W_Root, ObjSpace +from pypy.interpreter.error import OperationError + +def create_filter(space, w_category, action): + return space.newtuple([ + space.wrap(action), space.w_None, w_category, + space.w_None, space.wrap(0)]) + +class State: + def __init__(self, space): + self.init_filters(space) + self.w_once_registry = space.newdict() + self.w_default_action = space.wrap("default") + + def init_filters(self, space): + filters_w = [] + + if (not space.sys.get_flag('py3k_warning') and + not space.sys.get_flag('division_warning')): + filters_w.append(create_filter( + space, space.w_DeprecationWarning, "ignore")) + + filters_w.append(create_filter( + space, space.w_PendingDeprecationWarning, "ignore")) + filters_w.append(create_filter( + space, space.w_ImportWarning, "ignore")) + + bytes_warning = space.sys.get_flag('bytes_warning') + if bytes_warning > 1: + action = "error" + elif bytes_warning == 0: + action = "ignore" + else: + action = "default" + filters_w.append(create_filter( + space, space.w_BytesWarning, action)) + + self.w_filters = space.newlist(filters_w) + +def get_warnings_attr(space, name): + try: + w_module = space.getitem(space.sys.get('modules'), + space.wrap('warnings')) + except OperationError, e: + if not e.match(space, space.w_KeyError): + raise + return None + + try: + return space.getattr(w_module, space.wrap(name)) + except OperationError, e: + if not e.match(space, space.w_AttributeError): + raise + return None + +def get_category(space, w_message, w_category): + # Get category + if space.isinstance_w(w_message, space.w_Warning): + w_category = space.type(w_message) + elif space.is_w(w_category, space.w_None): + w_category = space.w_UserWarning + + # Validate category + if not space.abstract_issubclass_w(w_category, space.w_Warning): + raise OperationError(space.w_ValueError, space.wrap( + "category is not a subclass of Warning")) + + return w_category + +def setup_context(space, stacklevel): + # Setup globals and lineno + ec = space.getexecutioncontext() + frame = ec.gettopframe_nohidden() + while frame and stacklevel: + frame = ec.getnextframe_nohidden(frame) + stacklevel -= 1 + if frame: + w_globals = frame.w_globals + lineno = frame.f_lineno + else: + w_globals = space.sys.w_dict + lineno = 1 + + # setup registry + try: + w_registry = space.getitem(w_globals, space.wrap("__warningregistry__")) + except OperationError, e: + if not e.match(space, space.w_KeyError): + raise + w_registry = space.newdict() + space.setitem(w_globals, space.wrap("__warningregistry__"), w_registry) + + # setup module + try: + w_module = space.getitem(w_globals, space.wrap("__name__")) + except OperationError, e: + if not e.match(space, space.w_KeyError): + raise + w_module = space.wrap("") + + # setup filename + try: + w_filename = space.getitem(w_globals, space.wrap("__file__")) + except OperationError, e: + if not e.match(space, space.w_KeyError): + raise + if space.str_w(w_module) == '__main__': + w_argv = space.sys.get('argv') + if space.int_w(space.len(w_argv)) > 0: + w_filename = space.getitem(w_argv, space.wrap(0)) + if not space.is_true(w_filename): + w_filename = space.wrap('__main__') + else: + # embedded interpreters don't have sys.argv + w_filename = space.wrap('__main__') + else: + w_filename = w_module + else: + # if filename.lower().endswith((".pyc", ".pyo")) + if space.is_true(space.call_method( + w_filename, "endswith", + space.newtuple([space.wrap(".pyc"), space.wrap(".pyo")]))): + # strip last character + w_filename = space.wrap(space.str_w(w_filename)[:-1]) + + return (w_filename, lineno, w_module, w_registry) + +def check_matched(space, w_obj, w_arg): + if space.is_w(w_obj, space.w_None): + return True + return space.is_true(space.call_method(w_obj, "match", w_arg)) + +def get_filter(space, w_category, w_text, lineno, w_module): + w_filters = get_warnings_attr(space, "filters") + if w_filters: + space.fromcache(State).w_filters = w_filters + else: + w_filters = space.fromcache(State).w_filters + + # filters could change while we are iterating over it + for w_item in space.fixedview(w_filters): + w_action, w_msg, w_cat, w_mod, w_lineno = space.fixedview( + w_item, 5) + ln = space.int_w(w_lineno) + + if (check_matched(space, w_msg, w_text) and + check_matched(space, w_mod, w_module) and + space.abstract_issubclass_w(w_category, w_cat) and + (ln == 0 or ln == lineno)): + return space.str_w(w_action), w_item + + action = get_default_action(space) + if not action: + raise OperationError(space.w_ValueError, space.wrap( + "warnings.defaultaction not found")) + return action, None + +def get_default_action(space): + w_action = get_warnings_attr(space, "default_action"); + if w_action is None: + return space.str_w(space.fromcache(State).w_default_action) + + space.fromcache(State).w_default_action = w_action + return space.str_w(w_action) + +def get_once_registry(space): + w_registry = get_warnings_attr(space, "onceregistry"); + if w_registry is None: + return space.fromcache(State).w_once_registry + + space.fromcache(State).w_once_registry = w_registry + return w_registry + +def update_registry(space, w_registry, w_text, w_category): + w_key = space.newtuple([w_text, w_category]) + return already_warned(space, w_registry, w_altkey, should_set=True) + +def already_warned(space, w_registry, w_key, should_set=False): + try: + w_warned = space.getitem(w_registry, w_key) + except OperationError, e: + if not e.match(space, space.w_KeyError): + raise + if should_set: + space.setitem(w_registry, w_key, space.w_True) + return False + else: + return space.is_true(w_warned) + +def normalize_module(space, w_filename): + if not space.is_true(w_filename): + return space.wrap("") + + filename = space.str_w(w_filename) + length = len(filename) + if filename.endswith(".py"): + filename = filename[:-3] + return space.wrap(filename) + +def show_warning(space, w_filename, lineno, w_text, w_category, + w_sourceline=None): + w_name = space.getattr(w_category, space.wrap("__name__")) + w_stderr = space.sys.get("stderr") + + # Print "filename:lineno: category: text\n" + message = "%s:%d: %s: %s\n" % (space.str_w(w_filename), lineno, + space.str_w(w_name), space.str_w(w_text)) + space.call_method(w_stderr, "write", space.wrap(message)) + + # Print " source_line\n" + if w_sourceline: + line = space.str_w(w_sourceline) + for i in range(len(line)): + c = line[i] + if c not in ' \t\014': + break + message = " %s\n" % (line[i:],) + space.call_method(w_stderr, "write", space.wrap(message)) + +def do_warn(space, w_message, w_category, stacklevel): + context_w = setup_context(space, stacklevel) + do_warn_explicit(space, w_category, w_message, context_w) + +def do_warn_explicit(space, w_category, w_message, context_w, + w_sourceline=None): + w_filename, lineno, w_module, w_registry = context_w + + # normalize module + if space.is_w(w_module, space.w_None): + w_module = normalize_module(space, w_filename) + + # normalize message + if space.isinstance_w(w_message, space.w_Warning): + w_text = space.str(w_message) + w_category = space.type(w_message) + else: + w_text = w_message + w_message = space.call_function(w_category, w_message) + + w_lineno = space.wrap(lineno) + + # create key + w_key = space.newtuple([w_text, w_category, w_lineno]) + + if not space.is_w(w_registry, space.w_None): + if already_warned(space, w_registry, w_key): + return + # else this warning hasn't been generated before + + action, w_item = get_filter(space, w_category, w_text, lineno, w_module) + + if action == "error": + raise OperationError(w_category, w_message) + + # Store in the registry that we've been here, *except* when the action is + # "always". + warned = False + if action != 'always': + if not space.is_w(w_registry, space.w_None): + space.setitem(w_registry, w_key, space.w_True) + if action == 'ignore': + return + elif action == 'once': + if space.is_w(w_registry, space.w_None): + w_registry = get_once_registry(space) + warned = update_registry(space, w_registry, w_text, w_category) + elif action == 'module': + if not space.is_w(w_registry, space.w_None): + warned = update_registry(space, w_registry, w_text, w_category) + elif action != 'default': + try: + err = space.str_w(space.str(w_item)) + except OperationError: + err = "???" + raise OperationError(space.w_RuntimeError, space.wrap( + "Unrecognized action (%s) in warnings.filters:\n %s" % + (action, err))) + + if warned: + # Already warned for this module + return + w_show_fxn = get_warnings_attr(space, "showwarning") + if w_show_fxn is None: + show_warning(space, w_filename, lineno, w_text, w_category, + w_sourceline) + else: + space.call_function( + w_show_fxn, w_message, w_category, w_filename, w_lineno) + + at unwrap_spec(ObjSpace, W_Root, W_Root, int) +def warn(space, w_message, w_category=None, stacklevel=1): + w_category = get_category(space, w_message, w_category); + do_warn(space, w_message, w_category, stacklevel) + + +def warn_with_loader(space, w_message, w_category, w_filename, lineno, + w_module, w_registry, w_globals): + # Check/get the requisite pieces needed for the loader. + try: + w_loader = space.getitem(w_globals, space.wrap("__loader__")) + w_module_name = space.getitem(w_globals, space.wrap("__name__")) + except OperationError, e: + if not e.match(space, space.w_KeyError): + raise + return # perform standard call + + # Make sure the loader implements the optional get_source() method. + try: + w_get_source = space.getattr(w_loader, space.wrap("get_source")) + except OperationError, e: + if not e.match(space, space.w_AttributeError): + raise + return # perform standard call + + # Call get_source() to get the source code. + w_source = space.call_function(w_get_source, w_module_name) + if space.is_w(w_source, space.w_None): + return # perform standard call + # Split the source into lines. + w_source_list = space.call_method(w_source, "splitlines") + # Get the source line. + w_source_line = space.getitem(w_source_list, space.wrap(lineno - 1)) + + # Handle the warning. + do_warn_explicit(space, w_category, w_message, + (w_filename, lineno, w_module, w_registry), + w_source_line) + return True + + at unwrap_spec(ObjSpace, W_Root, W_Root, W_Root, int, W_Root, W_Root, W_Root) +def warn_explicit(space, w_message, w_category, w_filename, lineno, + w_module=None, w_registry=None, w_module_globals=None): + + if not space.is_w(w_module_globals, space.w_None): + if warn_with_loader(space, w_message, w_category, w_filename, lineno, + w_module, w_registry, w_module_globals): + return + + do_warn_explicit(space, w_category, w_message, + (w_filename, lineno, w_module, w_registry)) Added: pypy/branch/fast-forward/pypy/module/_warnings/test/test_warnings.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/pypy/module/_warnings/test/test_warnings.py Thu Nov 4 01:09:23 2010 @@ -0,0 +1,27 @@ +import py +import sys +from pypy.conftest import gettestobjspace + +class AppTestWarnings: + def setup_class(cls): + space = gettestobjspace(usemodules=('_warnings',)) + cls.space = space + + def test_defaults(self): + import _warnings + assert _warnings.once_registry == {} + assert _warnings.default_action == 'default' + assert "PendingDeprecationWarning" in str(_warnings.filters) + + def test_warn(self): + import _warnings + _warnings.warn("some message", DeprecationWarning) + _warnings.warn("some message", Warning) + + def test_warn_explicit(self): + import _warnings + _warnings.warn_explicit("some message", DeprecationWarning, + "", 1, module_globals=globals()) + _warnings.warn_explicit("some message", Warning, + "", 1, module_globals=globals()) + Modified: pypy/branch/fast-forward/pypy/module/sys/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/sys/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/sys/__init__.py Thu Nov 4 01:09:23 2010 @@ -156,3 +156,7 @@ else: from pypy.module.sys.interp_encoding import get_w_default_encoder return get_w_default_encoder(self.space) + + def get_flag(self, name): + space = self.space + return space.int_w(space.getattr(self.get('flags'), space.wrap(name))) From santagada at codespeak.net Thu Nov 4 04:06:06 2010 From: santagada at codespeak.net (santagada at codespeak.net) Date: Thu, 4 Nov 2010 04:06:06 +0100 (CET) Subject: [pypy-svn] r78686 - pypy/extradoc/planning/hg-migration Message-ID: <20101104030606.14631282B9C@codespeak.net> Author: santagada Date: Thu Nov 4 04:06:02 2010 New Revision: 78686 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: added my email address Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Thu Nov 4 04:06:02 2010 @@ -20,7 +20,7 @@ ale=Anders Lehmann nik=Niklaus Haldimann sanxiyn=Seo Sanghyeon -santagada=Leonardo Santagada +santagada=Leonardo Santagada rhymes=Lawrence Oluyede xoraxax=Alexander Schremmer jlg=Jakub Gustak From hakanardo at codespeak.net Thu Nov 4 07:43:54 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Thu, 4 Nov 2010 07:43:54 +0100 (CET) Subject: [pypy-svn] r78687 - pypy/extradoc/planning/hg-migration Message-ID: <20101104064354.80E47282B9D@codespeak.net> Author: hakanardo Date: Thu Nov 4 07:43:52 2010 New Revision: 78687 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: added email Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Thu Nov 4 07:43:52 2010 @@ -56,7 +56,7 @@ goden=Eugene Oden exarkun=Jean-Paul Calderone lukas=Lukas Renggli -hakanardo=Hakan Ardo +hakanardo=Hakan Ardo guenter=Guenter Jantzen dinu=Dinu Gherman gbrandl=Georg Brandl From hakanardo at codespeak.net Thu Nov 4 08:45:25 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Thu, 4 Nov 2010 08:45:25 +0100 (CET) Subject: [pypy-svn] r78688 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt Message-ID: <20101104074525.BE50C36C228@codespeak.net> Author: hakanardo Date: Thu Nov 4 08:45:23 2010 New Revision: 78688 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py Log: Update heap cache once a value turnes constant Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py Thu Nov 4 08:45:23 2010 @@ -151,6 +151,16 @@ def force_at_end_of_preamble(self): self.force_all_lazy_setfields() + + def turned_constant(self, value): + assert value.is_constant() + newvalue = self.getvalue(value.box) + if value is not newvalue: + for d in self.cached_fields.values(): + if value in d: + d[newvalue] = d[value] + # FIXME: Update the other caches too? + def force_lazy_setfield(self, descr, before_guard=False): try: Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Thu Nov 4 08:45:23 2010 @@ -196,6 +196,9 @@ def force_at_end_of_preamble(self): pass + def turned_constant(self, value): + pass + class Optimizer(Optimization): def __init__(self, metainterp_sd, loop, optimizations=None): @@ -231,6 +234,10 @@ for o in self.optimizations: o.force_at_end_of_preamble() + def turned_constant(self, value): + for o in self.optimizations: + o.turned_constant(value) + def forget_numberings(self, virtualbox): self.metainterp_sd.profiler.count(jitprof.OPT_FORCINGS) self.resumedata_memo.forget_numberings(virtualbox) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py Thu Nov 4 08:45:23 2010 @@ -153,6 +153,7 @@ if emit_operation: self.emit_operation(op) value.make_constant(constbox) + self.optimizer.turned_constant(value) def optimize_GUARD_ISNULL(self, op): value = self.getvalue(op.getarg(0)) From david at codespeak.net Thu Nov 4 11:15:34 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 4 Nov 2010 11:15:34 +0100 (CET) Subject: [pypy-svn] r78689 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101104101534.35900282B9E@codespeak.net> Author: david Date: Thu Nov 4 11:15:32 2010 New Revision: 78689 Added: pypy/branch/arm-backend/pypy/jit/backend/arm/shift.py Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Implement guard_overflow and int_mul_ovf Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Thu Nov 4 11:15:32 2010 @@ -165,7 +165,7 @@ | (rn & 0xF)) elif 'long' in table and table['long']: - def f(self, rdhi, rdlo, rn, rm, cond=cond.AL): + def f(self, rdlo, rdhi, rn, rm, cond=cond.AL): assert rdhi != rdlo self.write32(n | cond << 28 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Thu Nov 4 11:15:32 2010 @@ -1,6 +1,7 @@ from pypy.jit.backend.arm import conditions as c from pypy.jit.backend.arm import locations from pypy.jit.backend.arm import registers as r +from pypy.jit.backend.arm import shift from pypy.jit.backend.arm.arch import (WORD, FUNC_ALIGN, arm_int_div, arm_int_div_sign, arm_int_mod_sign, arm_int_mod) @@ -83,6 +84,16 @@ regalloc.possibly_free_vars_for_op(op) return fcond + #ref: http://blogs.arm.com/software-enablement/detecting-overflow-from-mul/ + def emit_op_int_mul_ovf(self, op, regalloc, fcond): + reg1 = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) + reg2 = regalloc.make_sure_var_in_reg(op.getarg(1), imm_fine=False) + res = regalloc.force_allocate_reg(op.result) + self.mc.SMULL(res.value, r.ip.value, reg1.value, reg2.value, cond=fcond) + self.mc.CMP_rr(r.ip.value, res.value, shifttype=shift.ASR, s=31, cond=fcond) + regalloc.possibly_free_vars_for_op(op) + return fcond + emit_op_int_floordiv = gen_emit_op_by_helper_call('DIV') emit_op_int_mod = gen_emit_op_by_helper_call('MOD') emit_op_uint_floordiv = gen_emit_op_by_helper_call('UDIV') @@ -111,6 +122,7 @@ emit_op_int_sub_ovf = emit_op_int_sub + class UnaryIntOpAssembler(object): emit_op_int_is_true = gen_emit_op_unary_cmp(c.NE, c.EQ) emit_op_int_is_zero = gen_emit_op_unary_cmp(c.EQ, c.NE) @@ -160,6 +172,9 @@ def emit_op_guard_no_overflow(self, op, regalloc, fcond): return self._emit_guard(op, regalloc, c.VS) + def emit_op_guard_overflow(self, op, regalloc, fcond): + return self._emit_guard(op, regalloc, c.VC) + class OpAssembler(object): _mixin_ = True Added: pypy/branch/arm-backend/pypy/jit/backend/arm/shift.py ============================================================================== --- (empty file) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/shift.py Thu Nov 4 11:15:32 2010 @@ -0,0 +1,6 @@ +# According to A8.4 +LSL = 0xB00 +LSR = 0xB01 +ASR = 0xB10 +ROR = 0xB11 +RRX = 0xB11 # with imm = 0 From david at codespeak.net Thu Nov 4 11:19:47 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 4 Nov 2010 11:19:47 +0100 (CET) Subject: [pypy-svn] r78690 - pypy/branch/arm-backend/pypy/jit/backend/arm/test Message-ID: <20101104101947.175E2282B9E@codespeak.net> Author: david Date: Thu Nov 4 11:19:45 2010 New Revision: 78690 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/gen.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Log: Update tests according to changes to long multiply ops Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/gen.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/gen.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/gen.py Thu Nov 4 11:19:45 2010 @@ -6,7 +6,9 @@ asm_opts = '-mcpu=cortex-a8 -march=armv7' body = """.section .text .arm -.global main +_start: .global _start + .global main + b main main: .ascii "START" %s Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Thu Nov 4 11:19:45 2010 @@ -185,7 +185,7 @@ def f(self): func = getattr(self.cb, name) func(r.r3.value, r.r13.value, r.r7.value, r.r12.value) - self.assert_equal("""%(name)s r13, r3, r7, r12 """ % {'name':name}) + self.assert_equal("""%(name)s r3, r13, r7, r12 """ % {'name':name}) else: def f(self): func = getattr(self.cb, name) From afa at codespeak.net Thu Nov 4 13:31:27 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 4 Nov 2010 13:31:27 +0100 (CET) Subject: [pypy-svn] r78691 - in pypy/branch/fast-forward/pypy/module/unicodedata: . test Message-ID: <20101104123127.9E161282B9D@codespeak.net> Author: afa Date: Thu Nov 4 13:31:23 2010 New Revision: 78691 Modified: pypy/branch/fast-forward/pypy/module/unicodedata/test/test_unicodedata.py pypy/branch/fast-forward/pypy/module/unicodedata/unicodedb_5_2_0.py Log: Kill code to fix translation (some constants are larger than a long) and add a failing test to remember it Modified: pypy/branch/fast-forward/pypy/module/unicodedata/test/test_unicodedata.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/unicodedata/test/test_unicodedata.py (original) +++ pypy/branch/fast-forward/pypy/module/unicodedata/test/test_unicodedata.py Thu Nov 4 13:31:23 2010 @@ -82,11 +82,20 @@ import unicodedata raises(TypeError, unicodedata.normalize, 'x') + def test_normalize_wide(self): + import sys, unicodedata + if sys.maxunicode < 0x10ffff: + skip("requires a 'wide' python build.") + # XXX this fails! + # Try to enable the three lines containing "69785 << 17 | 69818" + # in unicodedb_5_2_0.py; then fix translation... + assert unicodedata.normalize('NFC', u'\U000110a5\U000110ba') == u'\U000110ab' + class TestUnicodeData(object): def setup_class(cls): import random, unicodedata - if unicodedata.unidata_version != '4.1.0': - skip('Needs python with unicode 4.1.0 database.') + if unicodedata.unidata_version != '5.2.0': + skip('Needs python with unicode 5.2.0 database.') seed = random.getrandbits(32) print "random seed: ", seed Modified: pypy/branch/fast-forward/pypy/module/unicodedata/unicodedb_5_2_0.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/unicodedata/unicodedb_5_2_0.py (original) +++ pypy/branch/fast-forward/pypy/module/unicodedata/unicodedb_5_2_0.py Thu Nov 4 13:31:23 2010 @@ -147135,9 +147135,9 @@ 12529 << 17 | 12441: 12537, 12530 << 17 | 12441: 12538, 12541 << 17 | 12441: 12542, -69785 << 17 | 69818: 69786, -69787 << 17 | 69818: 69788, -69797 << 17 | 69818: 69803, +## 69785 << 17 | 69818: 69786, +## 69787 << 17 | 69818: 69788, +## 69797 << 17 | 69818: 69803, } _canon_decomposition = { From afa at codespeak.net Thu Nov 4 13:32:27 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 4 Nov 2010 13:32:27 +0100 (CET) Subject: [pypy-svn] r78692 - pypy/branch/fast-forward/pypy/module/_io Message-ID: <20101104123227.63899282B9D@codespeak.net> Author: afa Date: Thu Nov 4 13:32:25 2010 New Revision: 78692 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Log: Silence a translation warning Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Thu Nov 4 13:32:25 2010 @@ -22,6 +22,9 @@ space.wrap('UnsupportedOperation')) raise OperationError(w_exc, space.wrap(message)) + def _check_init(self, space): + raise NotImplementedError + @unwrap_spec('self', ObjSpace, W_Root) def read_w(self, space, w_size=None): self._unsupportedoperation(space, "read") From hpk at codespeak.net Thu Nov 4 13:42:05 2010 From: hpk at codespeak.net (hpk at codespeak.net) Date: Thu, 4 Nov 2010 13:42:05 +0100 (CET) Subject: [pypy-svn] r78693 - pypy/extradoc/planning/hg-migration Message-ID: <20101104124205.844AE282BAD@codespeak.net> Author: hpk Date: Thu Nov 4 13:42:04 2010 New Revision: 78693 Modified: pypy/extradoc/planning/hg-migration/plan.txt pypy/extradoc/planning/hg-migration/usermap.txt Log: my email, some comments Modified: pypy/extradoc/planning/hg-migration/plan.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/plan.txt (original) +++ pypy/extradoc/planning/hg-migration/plan.txt Thu Nov 4 13:42:04 2010 @@ -47,7 +47,8 @@ - I tentatively tried to send the commits of the bot2 repo to pypy-svn at codespeak.net, but they did not arrive. Probably because the mailing list blocks mails if the author is not subscribed? + i added *@bitbucket.org to allowed addresses. - IRC bot: bitbucket has a builtin hook for CIA.vc; alternatively, we can try to adapt kenaan: who owns it? - + exarkun ASFAIK Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Thu Nov 4 13:42:04 2010 @@ -10,7 +10,7 @@ antocuni=Antonio Cuni mwh=Michael Hudson tismer=Christian Tismer -hpk=Holger Krekel +hpk=Holger Krekel afa=Amaury Forgeot d'Arc ericvrp=Eric van Riet Paap rxe=Richard Emslie From commits-noreply at bitbucket.org Thu Nov 4 16:06:25 2010 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Thu, 4 Nov 2010 10:06:25 -0500 (CDT) Subject: [pypy-svn] buildbot commit 7b0fa3515f0d: make the default slave to automatically find its password from the slaveinfo.py which should be inside master/. Very useful when hacking at bot2 Message-ID: <20101104150625.D7BA9240F52@bitbucket01.managed.contegix.com> # HG changeset patch -- Bitbucket.org # Project buildbot # URL http://bitbucket.org/pypy/buildbot/overview # User Antonio Cuni # Date 1288882810 -3600 # Node ID 7b0fa3515f0dda4c5b2b0005672db4c49e1f4311 # Parent 0b7c56df6c16c9efc031800c378e691ba93994e6 make the default slave to automatically find its password from the slaveinfo.py which should be inside master/. Very useful when hacking at bot2 --- a/slave/buildbot.tac +++ b/slave/buildbot.tac @@ -11,10 +11,23 @@ basedir = os.path.abspath(os.path.dirnam # # --------------------------------------------------------------- +def find_passwd(slavename): + masterdir = os.path.join(basedir, '..', 'master') + slaveinfo = os.path.join(masterdir, 'slaveinfo.py') + d = {} + try: + execfile(slaveinfo, d) + return d['passwords'][slavename] + except Exception, e: + print 'error when executing ../master/slaveinfo.py: %s' % repr(e) + print 'using default password for the slave' + return 'default_password' + + buildmaster_host = 'localhost' port = 10407 slavename = 'localhost' -passwd = 'please_change_me' +passwd = find_passwd(slavename) keepalive = 600 usepty = 0 umask = None From commits-noreply at bitbucket.org Thu Nov 4 16:34:49 2010 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Thu, 4 Nov 2010 10:34:49 -0500 (CDT) Subject: [pypy-svn] buildbot commit 3a7c89443fc8: write a readme Message-ID: <20101104153449.BDA346C112C@bitbucket03.managed.contegix.com> # HG changeset patch -- Bitbucket.org # Project buildbot # URL http://bitbucket.org/pypy/buildbot/overview # User Antonio Cuni # Date 1288884345 -3600 # Node ID 3a7c89443fc8703d6aed49589c81a89985e5b8a7 # Parent 7b0fa3515f0dda4c5b2b0005672db4c49e1f4311 write a readme --- /dev/null +++ b/README @@ -0,0 +1,30 @@ +.. -*- mode: rst -*- + +How to hack the PyPy buildbot +============================== + +If you want to run buildbot locally, you need to: + + 1. create a file ``master/slaveinfo.py`` which contains the password for the + local slave you will run:: + + passwords = {"localhost": "an_arbitrary_password" } + + 2. ``cd master; make debug`` + + 3. ``cd slave; make debug`` + + +How to run the PyPy buildbot +============================ + +If you want to run buildbot in production, you need to make sure that the +function ``pypybuildbot.util.we_are_debugging`` returns ``False`` in your +environment. At the moment of writing, debugging is enabled everywhere but on +codespeak.net. + +You still need to fill ``master/slaveinfo.py`` with the passwords of the +various slaves you want to use. + +Then, to start the buildbot master: ``cd master; make start`` + From commits-noreply at bitbucket.org Thu Nov 4 16:34:49 2010 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Thu, 4 Nov 2010 10:34:49 -0500 (CDT) Subject: [pypy-svn] buildbot commit 5cbd6e289c04: ignore irrelevant files Message-ID: <20101104153449.CAAEB6C1414@bitbucket03.managed.contegix.com> # HG changeset patch -- Bitbucket.org # Project buildbot # URL http://bitbucket.org/pypy/buildbot/overview # User Antonio Cuni # Date 1288884871 -3600 # Node ID 5cbd6e289c043c4dd9b6f55b5ec1c8d05711c6ad # Parent 3a7c89443fc8703d6aed49589c81a89985e5b8a7 ignore irrelevant files --- /dev/null +++ b/.hgignore @@ -0,0 +1,25 @@ +syntax: glob +# master/slaveinfo.py contains the passwords, so it should never be tracked +master/slaveinfo.py + +# ignore pidfiles and all the logs +twistd.pid +twistd.log* +httpd.log* + +# ignore all the builders dir (which can be both in master/ and in slave/) +jit-benchmark-linux-x86-32 +jitonly-own-linux-x86-32 +own-linux-x86-32 +own-linux-x86-64 +own-macosx-x86-32 +pypy-c-Ojit-no-jit-linux-x86-32 +pypy-c-app-level-linux-x86-32 +pypy-c-app-level-linux-x86-64 +pypy-c-app-level-win-x86-32 +pypy-c-jit-linux-x86-32 +pypy-c-jit-linux-x86-64 +pypy-c-jit-macosx-x86-64 +pypy-c-jit-win-x86-32 +pypy-c-stackless-app-level-freebsd-7-x86-64 +pypy-c-stackless-app-level-linux-x86-32 From arigo at codespeak.net Thu Nov 4 16:41:28 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 4 Nov 2010 16:41:28 +0100 (CET) Subject: [pypy-svn] r78694 - pypy/trunk/pypy/translator/c/src Message-ID: <20101104154128.933F6282BDA@codespeak.net> Author: arigo Date: Thu Nov 4 16:41:25 2010 New Revision: 78694 Modified: pypy/trunk/pypy/translator/c/src/g_include.h pypy/trunk/pypy/translator/c/src/int.h Log: - a general clean-up of int.h. Fixes an issue with OP_INT_ADD_NONNEG_OVF(), hopefully. - disable the file asm_gcc_x86.h for now. On some simple tests I actually got slower results with it. It seems that correct scheduling is more important on modern processors than simply reducing the number of operations. Modified: pypy/trunk/pypy/translator/c/src/g_include.h ============================================================================== --- pypy/trunk/pypy/translator/c/src/g_include.h (original) +++ pypy/trunk/pypy/translator/c/src/g_include.h Thu Nov 4 16:41:25 2010 @@ -39,9 +39,10 @@ #include "src/instrument.h" /* optional assembler bits */ -#if defined(__GNUC__) && defined(__i386__) -# include "src/asm_gcc_x86.h" -#endif +// disabled: does not give any speed-up +//#if defined(__GNUC__) && defined(__i386__) +//# include "src/asm_gcc_x86.h" +//#endif #if defined(__GNUC__) && defined(__ppc__) # include "src/asm_ppc.h" Modified: pypy/trunk/pypy/translator/c/src/int.h ============================================================================== --- pypy/trunk/pypy/translator/c/src/int.h (original) +++ pypy/trunk/pypy/translator/c/src/int.h Thu Nov 4 16:41:25 2010 @@ -2,40 +2,27 @@ /************************************************************/ /*** C header subsection: operations between ints ***/ -#ifndef LLONG_MAX -# if SIZEOF_LONG_LONG == 8 -# define LLONG_MAX 0X7FFFFFFFFFFFFFFFLL -# else -# error "fix LLONG_MAX" -# endif -#endif - -#ifndef LLONG_MIN -# define LLONG_MIN (-LLONG_MAX-1) -#endif /*** unary operations ***/ -#define OP_INT_IS_TRUE(x,r) OP_INT_NE(x,0,r) - -#define OP_INT_INVERT(x,r) r = ~((x)) - -#define OP_INT_NEG(x,r) r = -(x) +#define OP_INT_IS_TRUE(x,r) r = ((x) != 0) +#define OP_INT_INVERT(x,r) r = ~(x) +#define OP_INT_NEG(x,r) r = -(x) #define OP_INT_NEG_OVF(x,r) \ - if ((x) == LONG_MIN) FAIL_OVF("integer negate"); \ + if ((x) == LONG_MIN) FAIL_OVF("integer negate"); \ OP_INT_NEG(x,r) #define OP_LLONG_NEG_OVF(x,r) \ - if ((x) == LLONG_MIN) FAIL_OVF("integer negate"); \ + if ((x) == LLONG_MIN) FAIL_OVF("integer negate"); \ OP_LLONG_NEG(x,r) #define OP_INT_ABS(x,r) r = (x) >= 0 ? x : -(x) #define OP_INT_ABS_OVF(x,r) \ - if ((x) == LONG_MIN) FAIL_OVF("integer absolute"); \ + if ((x) == LONG_MIN) FAIL_OVF("integer absolute"); \ OP_INT_ABS(x,r) #define OP_LLONG_ABS_OVF(x,r) \ - if ((x) == LLONG_MIN) FAIL_OVF("integer absolute"); \ + if ((x) == LLONG_MIN) FAIL_OVF("integer absolute"); \ OP_LLONG_ABS(x,r) /*** binary operations ***/ @@ -59,50 +46,46 @@ #define OP_INT_ADD(x,y,r) r = (x) + (y) +/* cast to avoid undefined behaviour on overflow */ #define OP_INT_ADD_OVF(x,y,r) \ - OP_INT_ADD(x,y,r); \ - if ((r^(x)) >= 0 || (r^(y)) >= 0); \ - else FAIL_OVF("integer addition") + r = (long)((unsigned long)x + y); \ + if ((r^x) < 0 && (r^y) < 0) FAIL_OVF("integer addition") + +#define OP_LLONG_ADD_OVF(x,y,r) \ + r = (long long)((unsigned long long)x + y); \ + if ((r^x) < 0 && (r^y) < 0) FAIL_OVF("integer addition") #define OP_INT_ADD_NONNEG_OVF(x,y,r) /* y can be assumed >= 0 */ \ - r = (long)((unsigned long)x + (unsigned long)y); \ - if (r >= (x)); \ - else FAIL_OVF("integer addition") -/* Can a C compiler be too clever and think it can "prove" that - * r >= x always holds above? Yes. Hence the casting. */ + r = (long)((unsigned long)x + y); \ + if ((r&~x) < 0) FAIL_OVF("integer addition") #define OP_INT_SUB(x,y,r) r = (x) - (y) #define OP_INT_SUB_OVF(x,y,r) \ - OP_INT_SUB(x,y,r); \ - if ((r^(x)) >= 0 || (r^~(y)) >= 0); \ - else FAIL_OVF("integer subtraction") - -#define OP_INT_MUL(x,y,r) r = (x) * (y) - -#if defined(HAVE_LONG_LONG) && SIZE_OF_LONG_LONG < SIZE_OF_LONG -# define OP_INT_MUL_OVF_LL 1 -#lse -# define OP_INT_MUL_OVF_LL 0 -#endif + r = (long)((unsigned long)x - y); \ + if ((r^x) < 0 && (r^~y) < 0) FAIL_OVF("integer subtraction") -#if !OP_INT_MUL_OVF_LL +#define OP_LLONG_SUB_OVF(x,y,r) \ + r = (long long)((unsigned long long)x - y); \ + if ((r^x) < 0 && (r^~y) < 0) FAIL_OVF("integer subtraction") -#define OP_INT_MUL_OVF(x,y,r) \ - if (op_int_mul_ovf(x,y,&r)); \ - else FAIL_OVF("integer multiplication") - -#else +#define OP_INT_MUL(x,y,r) r = (x) * (y) +#if SIZEOF_LONG * 2 <= SIZEOF_LONG_LONG #define OP_INT_MUL_OVF(x,y,r) \ { \ - PY_LONG_LONG lr = (PY_LONG_LONG)(x) * (PY_LONG_LONG)(y); \ - r = (long)lr; \ - if ((PY_LONG_LONG)r == lr); \ - else FAIL_OVF("integer multiplication"); \ + long long _lr = (long long)x * y; \ + r = (long)_lr; \ + if (_lr != (long long)r) FAIL_OVF("integer multiplication"); \ } +#else +#define OP_INT_MUL_OVF(x,y,r) \ + r = op_llong_mul_ovf(x, y) /* long == long long */ #endif +#define OP_LLONG_MUL_OVF(x,y,r) \ + r = op_llong_mul_ovf(x, y) + /* shifting */ /* NB. shifting has same limitations as C: the shift count must be @@ -121,6 +104,10 @@ OP_INT_LSHIFT(x,y,r); \ if ((x) != Py_ARITHMETIC_RIGHT_SHIFT(long, r, (y))) \ FAIL_OVF("x< # HG changeset patch -- Bitbucket.org # Project buildbot # URL http://bitbucket.org/pypy/buildbot/overview # User Antonio Cuni # Date 1288885830 -3600 # Node ID e4dfbfe4da0b34ae6dc4b394959e3f9f64a34eb9 # Parent 5cbd6e289c043c4dd9b6f55b5ec1c8d05711c6ad specify which version of buildbot you should run --- a/README +++ b/README @@ -1,5 +1,8 @@ .. -*- mode: rst -*- +Everything has been tested with builbot 0.7.12. Not sure what happens with +other versions :-) + How to hack the PyPy buildbot ============================== From hakanardo at codespeak.net Thu Nov 4 16:54:48 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Thu, 4 Nov 2010 16:54:48 +0100 (CET) Subject: [pypy-svn] r78695 - pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test Message-ID: <20101104155448.8B4CD282BDD@codespeak.net> Author: hakanardo Date: Thu Nov 4 16:54:42 2010 New Revision: 78695 Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py Log: fixed test Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py Thu Nov 4 16:54:42 2010 @@ -286,14 +286,18 @@ ''', 93, ([20], 20), ([31], 32)) - ops = self.get_by_bytecode("LOOKUP_METHOD") + ops = self.get_by_bytecode("LOOKUP_METHOD", True) assert len(ops) == 2 assert not ops[0].get_opnames("call") assert not ops[0].get_opnames("new") - assert len(ops[0].get_opnames("guard")) <= 2 + assert len(ops[0].get_opnames("guard")) <= 3 assert not ops[1] # second LOOKUP_METHOD folded away - ops = self.get_by_bytecode("CALL_METHOD") + ops = self.get_by_bytecode("LOOKUP_METHOD") + assert not ops[0] # first LOOKUP_METHOD folded away + assert not ops[1] # second LOOKUP_METHOD folded away + + ops = self.get_by_bytecode("CALL_METHOD", True) assert len(ops) == 2 for i, bytecode in enumerate(ops): if i == 0: @@ -304,7 +308,12 @@ assert len(bytecode.get_opnames("guard")) <= 6 assert len(ops[1]) < len(ops[0]) - ops = self.get_by_bytecode("LOAD_ATTR") + ops = self.get_by_bytecode("CALL_METHOD") + assert len(ops) == 2 + assert len(ops[0]) <= 1 + assert len(ops[1]) <= 1 + + ops = self.get_by_bytecode("LOAD_ATTR", True) assert len(ops) == 2 # With mapdict, we get fast access to (so far) the 5 first # attributes, which means it is done with only the following @@ -314,6 +323,10 @@ "guard_nonnull_class"] assert not ops[1] # second LOAD_ATTR folded away + ops = self.get_by_bytecode("LOAD_ATTR") + assert not ops[0] # first LOAD_ATTR folded away + assert not ops[1] # second LOAD_ATTR folded away + def test_static_classmethod_call(self): self.run_source(''' class A(object): From arigo at codespeak.net Thu Nov 4 17:26:45 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 4 Nov 2010 17:26:45 +0100 (CET) Subject: [pypy-svn] r78696 - pypy/trunk/pypy/jit/backend/x86/test Message-ID: <20101104162645.9BC41282BDC@codespeak.net> Author: arigo Date: Thu Nov 4 17:26:42 2010 New Revision: 78696 Modified: pypy/trunk/pypy/jit/backend/x86/test/test_runner.py Log: A test reliably reproducing the occasional crash of test_zll_random. Modified: pypy/trunk/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/trunk/pypy/jit/backend/x86/test/test_runner.py Thu Nov 4 17:26:42 2010 @@ -478,6 +478,10 @@ # whether the test segfaults. assert self.cpu.get_latest_value_int(0) == finished.value + def test_overflow_guard_exception(self): + for i in range(50): + self.test_exceptions() + class TestDebuggingAssembler(object): def setup_method(self, meth): From david at codespeak.net Thu Nov 4 17:33:22 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 4 Nov 2010 17:33:22 +0100 (CET) Subject: [pypy-svn] r78697 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . helper Message-ID: <20101104163322.9731B282BDC@codespeak.net> Author: david Date: Thu Nov 4 17:33:20 2010 New Revision: 78697 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Log: Implement CALL operation Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Thu Nov 4 17:33:20 2010 @@ -225,16 +225,20 @@ if regalloc.frame_manager.frame_depth == 1: return n = regalloc.frame_manager.frame_depth*WORD - if n <= 0xFF: + self._adjust_sp(n, cb) + + def _adjust_sp(self, n, cb=None, fcond=c.AL): + if cb is None: + cb = self.mc + if n <= 0xFF and fcond == c.AL: cb.SUB_ri(r.sp.value, r.sp.value, n) else: b = Box() reg = regalloc.force_allocate_reg(b) - cb.gen_load_int(reg.value, n) - cb.SUB_rr(r.sp.value, r.sp.value, reg.value) + cb.gen_load_int(reg.value, n, cond=fcond) + cb.SUB_rr(r.sp.value, r.sp.value, reg.value, cond=fcond) regalloc.possibly_free_var(reg) - def assemble_bridge(self, faildescr, inputargs, operations): enc = rffi.cast(rffi.CCHARP, faildescr._failure_recovery_code) longevity = compute_vars_longevity(inputargs, operations) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Thu Nov 4 17:33:20 2010 @@ -146,27 +146,27 @@ _space_for_jump = 2 * WORD def writechar(self, char): - if self.checks and not self._pos < self._size - self._space_for_jump: - self.checks = False + if self.checks and not self._pos < self._size - self._space_for_jump - WORD: self._add_more_mem() - self.checks = True - assert self._pos < self._size + assert self._pos < self._size - 1 AbstractARMv7Builder.writechar(self, char) def _add_more_mem(self): + self.checks = False new_mem = alloc(self._size) new_mem_addr = rffi.cast(lltype.Signed, new_mem) self.LDR_ri(reg.pc.value, reg.pc.value, -4) self.write32(new_mem_addr) - self._dump_trace('data%d.asm' % self.n_data) + self._dump_trace('data%04d.asm' % self.n_data) self.n_data += 1 self._data = new_mem self._pos = 0 + self.checks = True def ensure_can_fit(self, n): """ensure after this call there is enough space for n instructions in a contiguous memory chunk""" - if not self._pos + n + self._space_for_jump < self._size: + if not self._pos + n + self._space_for_jump < self._size - WORD: self._add_more_mem() define_instructions(AbstractARMv7Builder) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Thu Nov 4 17:33:20 2010 @@ -47,9 +47,8 @@ arg2 = regalloc.make_sure_var_in_reg(op.getarg(1), selected_reg=r.r1, imm_fine=False) assert arg1 == r.r0 assert arg2 == r.r1 - res = regalloc.force_allocate_reg(op.result) + res = regalloc.force_allocate_reg(op.result, selected_reg=r.r0) getattr(self.mc, opname)(fcond) - self.mc.MOV_rr(res.value, r.r0.value, cond=fcond) regalloc.possibly_free_vars_for_op(op) return fcond return f Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Thu Nov 4 17:33:20 2010 @@ -10,7 +10,7 @@ gen_emit_op_ri, gen_emit_cmp_op) from pypy.jit.backend.arm.codebuilder import ARMv7Builder, ARMv7InMemoryBuilder from pypy.jit.backend.arm.regalloc import ARMRegisterManager -from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity +from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity, TempBox from pypy.jit.metainterp.history import ConstInt, BoxInt, Box, BasicFailDescr from pypy.jit.metainterp.resoperation import rop from pypy.rlib import rgc @@ -193,3 +193,29 @@ def emit_op_finish(self, op, regalloc, fcond): self._gen_path_to_exit_path(op, op.getarglist(), regalloc, c.AL) return fcond + + def emit_op_call(self, op, regalloc, fcond): + locs = [] + # all arguments past the 4th go on the stack + # XXX support types other than int (one word types) + if op.numargs() > 5: + stack_args = op.numargs() - 5 + n = stack_args*WORD + self._adjust_sp(n, fcond=fcond) + for i in range(5, op.numargs()): + reg = regalloc.make_sure_var_in_reg(op.getarg(i)) + self.mc.STR_ri(reg.value, r.sp.value, (i-5)*WORD) + regalloc.possibly_free_var(reg) + + adr = self.cpu.cast_adr_to_int(op.getarg(0).getint()) + regalloc.before_call() + + reg_args = min(op.numargs()-1, 4) + for i in range(1, reg_args+1): + l = regalloc.make_sure_var_in_reg(op.getarg(i), + selected_reg=r.all_regs[i-1]) + locs.append(l) + self.mc.BL(adr) + regalloc.force_allocate_reg(op.result, selected_reg=r.r0) + regalloc.after_call(op.result) + regalloc.possibly_free_vars(locs) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Thu Nov 4 17:33:20 2010 @@ -7,7 +7,7 @@ all_regs = r.all_regs box_types = None # or a list of acceptable types no_lower_byte_regs = all_regs - save_around_call_regs = all_regs + save_around_call_regs = r.caller_resp def __init__(self, longevity, frame_manager=None, assembler=None): RegisterManager.__init__(self, longevity, frame_manager, assembler) @@ -24,6 +24,9 @@ def convert_to_imm(self, c): return locations.ImmLocation(c.value) + def call_result_location(self, v): + return r.r0 + class ARMFrameManager(FrameManager): def __init__(self): FrameManager.__init__(self) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py Thu Nov 4 17:33:20 2010 @@ -10,8 +10,9 @@ lr = r14 pc = r15 -all_regs = registers[:11] +all_regs = [r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10] +caller_resp = [r0, r1, r2, r3] callee_resp = [r4, r5, r6, r7, r8, r9, r10, r11] callee_saved_registers = callee_resp+[lr] callee_restored_registers = callee_resp+[pc] Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Thu Nov 4 17:33:20 2010 @@ -1,7 +1,7 @@ from pypy.jit.backend.arm.assembler import AssemblerARM from pypy.jit.backend.llsupport.llmodel import AbstractLLCPU from pypy.rpython.llinterp import LLInterpreter -from pypy.rpython.lltypesystem import lltype, rffi +from pypy.rpython.lltypesystem import lltype, rffi, llmemory class ArmCPU(AbstractLLCPU): @@ -51,3 +51,7 @@ # LLInterpreter.current_interpreter = prev_interpreter return res + @staticmethod + def cast_ptr_to_int(x): + adr = llmemory.cast_ptr_to_adr(x) + return self.cast_adr_to_int(adr) From arigo at codespeak.net Thu Nov 4 17:42:11 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 4 Nov 2010 17:42:11 +0100 (CET) Subject: [pypy-svn] r78698 - pypy/trunk/pypy/jit/backend/x86 Message-ID: <20101104164211.CA56C282B9E@codespeak.net> Author: arigo Date: Thu Nov 4 17:42:10 2010 New Revision: 78698 Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py Log: Fix for the test of r78696. Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/assembler.py (original) +++ pypy/trunk/pypy/jit/backend/x86/assembler.py Thu Nov 4 17:42:10 2010 @@ -712,9 +712,11 @@ dispatch_opnum = guard_opnum else: dispatch_opnum = op.getopnum() - res = genop_guard_list[dispatch_opnum](self, op, guard_op, guard_token, - arglocs, resloc) - faildescr._x86_adr_jump_offset = res + genop_guard_list[dispatch_opnum](self, op, guard_op, guard_token, + arglocs, resloc) + if not we_are_translated(): + # must be added by the genop_guard_list[]() + assert hasattr(faildescr, '_x86_adr_jump_offset') def regalloc_perform_guard(self, guard_op, faillocs, arglocs, resloc, current_depths): @@ -771,15 +773,15 @@ if isinstance(op.getarg(0), Const): self.mc.CMP(arglocs[1], arglocs[0]) if guard_opnum == rop.GUARD_FALSE: - return self.implement_guard(guard_token, rev_cond) + self.implement_guard(guard_token, rev_cond) else: - return self.implement_guard(guard_token, false_rev_cond) + self.implement_guard(guard_token, false_rev_cond) else: self.mc.CMP(arglocs[0], arglocs[1]) if guard_opnum == rop.GUARD_FALSE: - return self.implement_guard(guard_token, cond) + self.implement_guard(guard_token, cond) else: - return self.implement_guard(guard_token, false_cond) + self.implement_guard(guard_token, false_cond) return genop_cmp_guard def _cmpop_guard_float(cond, false_cond, need_jp): @@ -793,13 +795,14 @@ if guard_opnum == rop.GUARD_FALSE: if need_jp: self.mc.J_il8(rx86.Conditions['P'], 6) - return self.implement_guard(guard_token, cond) + self.implement_guard(guard_token, cond) else: if need_jp: self.mc.J_il8(rx86.Conditions['P'], 2) self.mc.J_il8(rx86.Conditions[cond], 5) - return self.implement_guard(guard_token) - return self.implement_guard(guard_token, false_cond) + self.implement_guard(guard_token) + else: + self.implement_guard(guard_token, false_cond) return genop_cmp_guard_float def _emit_call(self, x, arglocs, start=0, tmp=eax): @@ -961,11 +964,11 @@ self.mc.ensure_bytes_available(16 + guard_token.recovery_stub_size()) if guard_opnum == rop.GUARD_TRUE: self.mc.J_il8(rx86.Conditions['P'], 6) - return self.implement_guard(guard_token, 'E') + self.implement_guard(guard_token, 'E') else: self.mc.J_il8(rx86.Conditions['P'], 2) self.mc.J_il8(rx86.Conditions['E'], 5) - return self.implement_guard(guard_token) + self.implement_guard(guard_token) def genop_float_neg(self, op, arglocs, resloc): # Following what gcc does: res = x ^ 0x8000000000000000 @@ -985,9 +988,9 @@ guard_opnum = guard_op.getopnum() self.mc.CMP(arglocs[0], imm0) if guard_opnum == rop.GUARD_TRUE: - return self.implement_guard(guard_token, 'Z') + self.implement_guard(guard_token, 'Z') else: - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') def genop_int_is_true(self, op, arglocs, resloc): self.mc.CMP(arglocs[0], imm0) @@ -999,9 +1002,9 @@ guard_opnum = guard_op.getopnum() self.mc.CMP(arglocs[0], imm0) if guard_opnum == rop.GUARD_TRUE: - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') else: - return self.implement_guard(guard_token, 'Z') + self.implement_guard(guard_token, 'Z') def genop_int_is_zero(self, op, arglocs, resloc): self.mc.CMP(arglocs[0], imm0) @@ -1198,13 +1201,13 @@ def genop_guard_guard_true(self, ign_1, guard_op, guard_token, locs, ign_2): loc = locs[0] self.mc.TEST(loc, loc) - return self.implement_guard(guard_token, 'Z') + self.implement_guard(guard_token, 'Z') genop_guard_guard_nonnull = genop_guard_guard_true def genop_guard_guard_no_exception(self, ign_1, guard_op, guard_token, locs, ign_2): self.mc.CMP(heap(self.cpu.pos_exception()), imm0) - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') def genop_guard_guard_exception(self, ign_1, guard_op, guard_token, locs, resloc): @@ -1212,19 +1215,18 @@ loc1 = locs[1] self.mc.MOV(loc1, heap(self.cpu.pos_exception())) self.mc.CMP(loc1, loc) - addr = self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') if resloc is not None: self.mc.MOV(resloc, heap(self.cpu.pos_exc_value())) self.mc.MOV(heap(self.cpu.pos_exception()), imm0) self.mc.MOV(heap(self.cpu.pos_exc_value()), imm0) - return addr def _gen_guard_overflow(self, guard_op, guard_token): guard_opnum = guard_op.getopnum() if guard_opnum == rop.GUARD_NO_OVERFLOW: - return self.implement_guard(guard_token, 'O') + self.implement_guard(guard_token, 'O') elif guard_opnum == rop.GUARD_OVERFLOW: - return self.implement_guard(guard_token, 'NO') + self.implement_guard(guard_token, 'NO') else: not_implemented("int_xxx_ovf followed by %s" % guard_op.getopname()) @@ -1244,7 +1246,7 @@ def genop_guard_guard_false(self, ign_1, guard_op, guard_token, locs, ign_2): loc = locs[0] self.mc.TEST(loc, loc) - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') genop_guard_guard_isnull = genop_guard_guard_false def genop_guard_guard_value(self, ign_1, guard_op, guard_token, locs, ign_2): @@ -1253,7 +1255,7 @@ self.mc.UCOMISD(locs[0], locs[1]) else: self.mc.CMP(locs[0], locs[1]) - return self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') def _cmp_guard_class(self, locs): offset = self.cpu.vtable_offset @@ -1285,7 +1287,7 @@ def genop_guard_guard_class(self, ign_1, guard_op, guard_token, locs, ign_2): self.mc.ensure_bytes_available(256) self._cmp_guard_class(locs) - return self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') def genop_guard_guard_nonnull_class(self, ign_1, guard_op, guard_token, locs, ign_2): @@ -1300,7 +1302,7 @@ assert 0 < offset <= 127 self.mc.overwrite(jb_location-1, [chr(offset)]) # - return self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') def implement_guard_recovery(self, guard_opnum, faildescr, failargs, fail_locs): @@ -1627,13 +1629,15 @@ def implement_guard(self, guard_token, condition=None): self.mc.reserve_bytes(guard_token.recovery_stub_size()) self.pending_guard_tokens.append(guard_token) - # XXX: These jumps are patched later, the self.mc.tell() are just - # dummy values + # These jumps are patched later, the mc.tell() are just + # dummy values. Also, use self.mc._mc to avoid triggering a + # "buffer full" exactly here. + mc = self.mc._mc if condition: - self.mc.J_il(rx86.Conditions[condition], self.mc.tell()) + mc.J_il(rx86.Conditions[condition], mc.tell()) else: - self.mc.JMP_l(self.mc.tell()) - return self.mc.tell() - 4 + mc.JMP_l(mc.tell()) + guard_token.faildescr._x86_adr_jump_offset = mc.tell() - 4 def genop_call(self, op, arglocs, resloc): sizeloc = arglocs[0] @@ -1674,7 +1678,7 @@ self.mc.MOV_bi(FORCE_INDEX_OFS, fail_index) self.genop_call(op, arglocs, result_loc) self.mc.CMP_bi(FORCE_INDEX_OFS, 0) - return self.implement_guard(guard_token, 'L') + self.implement_guard(guard_token, 'L') def genop_guard_call_assembler(self, op, guard_op, guard_token, arglocs, result_loc): @@ -1761,7 +1765,7 @@ assert 0 <= offset <= 127 self.mc.overwrite(jmp_location - 1, [chr(offset)]) self.mc.CMP_bi(FORCE_INDEX_OFS, 0) - return self.implement_guard(guard_token, 'L') + self.implement_guard(guard_token, 'L') def genop_discard_cond_call_gc_wb(self, op, arglocs): # use 'mc._mc' directly instead of 'mc', to avoid From afa at codespeak.net Thu Nov 4 18:07:13 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 4 Nov 2010 18:07:13 +0100 (CET) Subject: [pypy-svn] r78699 - pypy/branch/fast-forward/pypy/module/_warnings Message-ID: <20101104170713.12292282BDD@codespeak.net> Author: afa Date: Thu Nov 4 18:07:12 2010 New Revision: 78699 Modified: pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py Log: Fix translation of the _warnings module Modified: pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py (original) +++ pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py Thu Nov 4 18:07:12 2010 @@ -174,7 +174,7 @@ def update_registry(space, w_registry, w_text, w_category): w_key = space.newtuple([w_text, w_category]) - return already_warned(space, w_registry, w_altkey, should_set=True) + return already_warned(space, w_registry, w_key, should_set=True) def already_warned(space, w_registry, w_key, should_set=False): try: @@ -195,7 +195,9 @@ filename = space.str_w(w_filename) length = len(filename) if filename.endswith(".py"): - filename = filename[:-3] + n = len(filename) - 3 + assert n >= 0 + filename = filename[:n] return space.wrap(filename) def show_warning(space, w_filename, lineno, w_text, w_category, @@ -211,11 +213,12 @@ # Print " source_line\n" if w_sourceline: line = space.str_w(w_sourceline) + message = "\n" for i in range(len(line)): c = line[i] if c not in ' \t\014': + message = " %s\n" % (line[i:],) break - message = " %s\n" % (line[i:],) space.call_method(w_stderr, "write", space.wrap(message)) def do_warn(space, w_message, w_category, stacklevel): @@ -294,8 +297,10 @@ do_warn(space, w_message, w_category, stacklevel) -def warn_with_loader(space, w_message, w_category, w_filename, lineno, - w_module, w_registry, w_globals): +def get_source_line(space, w_globals, lineno): + if space.is_w(w_globals, space.w_None): + return None + # Check/get the requisite pieces needed for the loader. try: w_loader = space.getitem(w_globals, space.wrap("__loader__")) @@ -303,7 +308,7 @@ except OperationError, e: if not e.match(space, space.w_KeyError): raise - return # perform standard call + return None # Make sure the loader implements the optional get_source() method. try: @@ -311,31 +316,26 @@ except OperationError, e: if not e.match(space, space.w_AttributeError): raise - return # perform standard call + return None # Call get_source() to get the source code. w_source = space.call_function(w_get_source, w_module_name) if space.is_w(w_source, space.w_None): - return # perform standard call + return None + # Split the source into lines. w_source_list = space.call_method(w_source, "splitlines") + # Get the source line. w_source_line = space.getitem(w_source_list, space.wrap(lineno - 1)) - - # Handle the warning. - do_warn_explicit(space, w_category, w_message, - (w_filename, lineno, w_module, w_registry), - w_source_line) - return True + return w_source_line @unwrap_spec(ObjSpace, W_Root, W_Root, W_Root, int, W_Root, W_Root, W_Root) def warn_explicit(space, w_message, w_category, w_filename, lineno, w_module=None, w_registry=None, w_module_globals=None): - if not space.is_w(w_module_globals, space.w_None): - if warn_with_loader(space, w_message, w_category, w_filename, lineno, - w_module, w_registry, w_module_globals): - return + w_source_line = get_source_line(space, w_module_globals, lineno) do_warn_explicit(space, w_category, w_message, - (w_filename, lineno, w_module, w_registry)) + (w_filename, lineno, w_module, w_registry), + w_source_line) From arigo at codespeak.net Thu Nov 4 18:23:20 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 4 Nov 2010 18:23:20 +0100 (CET) Subject: [pypy-svn] r78700 - pypy/trunk/pypy/jit/tool Message-ID: <20101104172320.F0492282B9E@codespeak.net> Author: arigo Date: Thu Nov 4 18:23:18 2010 New Revision: 78700 Modified: pypy/trunk/pypy/jit/tool/pypytrace.vim Log: Comment out making the opnames bold. Seems to be more annoying than useful. Modified: pypy/trunk/pypy/jit/tool/pypytrace.vim ============================================================================== --- pypy/trunk/pypy/jit/tool/pypytrace.vim (original) +++ pypy/trunk/pypy/jit/tool/pypytrace.vim Thu Nov 4 18:23:18 2010 @@ -22,7 +22,7 @@ hi def link pypyLoopStart Structure "hi def link pypyLoopArgs PreProc hi def link pypyFailArgs String -hi def link pypyOpName Statement +"hi def link pypyOpName Statement hi def link pypyDebugMergePoint Comment hi def link pypyConstPtr Constant hi def link pypyNumber Number From arigo at codespeak.net Thu Nov 4 18:25:08 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 4 Nov 2010 18:25:08 +0100 (CET) Subject: [pypy-svn] r78701 - pypy/trunk/pypy/interpreter Message-ID: <20101104172508.70B18282BAD@codespeak.net> Author: arigo Date: Thu Nov 4 18:25:06 2010 New Revision: 78701 Modified: pypy/trunk/pypy/interpreter/baseobjspace.py Log: Force the tick counter to have a valid value at start-up. Without this, the counter would start at 0 but still increase until it reaches 2**20, meaning that the action_dispatcher() function would not be called for the first 1'000'000 iterations of a pypy-c run. Modified: pypy/trunk/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/trunk/pypy/interpreter/baseobjspace.py (original) +++ pypy/trunk/pypy/interpreter/baseobjspace.py Thu Nov 4 18:25:06 2010 @@ -299,6 +299,8 @@ self.timer.start("startup " + modname) mod.init(self) self.timer.stop("startup " + modname) + # Force the tick counter to have a valid value + self.actionflag.force_tick_counter() def finish(self): self.wait_for_thread_shutdown() From hakanardo at codespeak.net Thu Nov 4 18:40:16 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Thu, 4 Nov 2010 18:40:16 +0100 (CET) Subject: [pypy-svn] r78702 - pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test Message-ID: <20101104174016.4798D282BAD@codespeak.net> Author: hakanardo Date: Thu Nov 4 18:40:14 2010 New Revision: 78702 Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py Log: fixed test Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py Thu Nov 4 18:40:14 2010 @@ -519,19 +519,30 @@ s += g(n)[i] return s ''', 143, ([1000], 1000 * 999 / 2)) - bytecode, = self.get_by_bytecode("BINARY_SUBSCR") + bytecode, = self.get_by_bytecode("BINARY_SUBSCR", True) assert bytecode.get_opnames("guard") == [ "guard_false", # check that the index is >= 0 "guard_false", # check that the index is lower than the current length ] - bytecode, _ = self.get_by_bytecode("FOR_ITER") # second bytecode is the end of the loop + bytecode, _ = self.get_by_bytecode("FOR_ITER", True) # second bytecode is the end of the loop assert bytecode.get_opnames("guard") == [ + "guard_value", "guard_class", # check the class of the iterator "guard_nonnull", # check that the iterator is not finished "guard_isnull", # check that the range list is not forced "guard_false", # check that the index is lower than the current length ] - + + bytecode, = self.get_by_bytecode("BINARY_SUBSCR") + assert bytecode.get_opnames("guard") == [ + "guard_false", # check that the index is >= 0 + "guard_false", # check that the index is lower than the current length + ] + bytecode, _ = self.get_by_bytecode("FOR_ITER") # second bytecode is the end of the loop + assert bytecode.get_opnames("guard") == [ + "guard_false", # check that the index is lower than the current length + ] + def test_exception_inside_loop_1(self): py.test.skip("exceptions: in-progress") self.run_source(''' From afa at codespeak.net Thu Nov 4 21:21:32 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 4 Nov 2010 21:21:32 +0100 (CET) Subject: [pypy-svn] r78703 - in pypy/branch/fast-forward/pypy/module/_warnings: . test Message-ID: <20101104202132.329DB282B9D@codespeak.net> Author: afa Date: Thu Nov 4 21:21:18 2010 New Revision: 78703 Modified: pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py pypy/branch/fast-forward/pypy/module/_warnings/test/test_warnings.py Log: Fixes in the _warnings module Modified: pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py (original) +++ pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py Thu Nov 4 21:21:18 2010 @@ -72,7 +72,7 @@ # Setup globals and lineno ec = space.getexecutioncontext() frame = ec.gettopframe_nohidden() - while frame and stacklevel: + while frame and stacklevel > 1: frame = ec.getnextframe_nohidden(frame) stacklevel -= 1 if frame: @@ -106,8 +106,8 @@ if not e.match(space, space.w_KeyError): raise if space.str_w(w_module) == '__main__': - w_argv = space.sys.get('argv') - if space.int_w(space.len(w_argv)) > 0: + w_argv = space.sys.getdictvalue(space, 'argv') + if w_argv and space.int_w(space.len(w_argv)) > 0: w_filename = space.getitem(w_argv, space.wrap(0)) if not space.is_true(w_filename): w_filename = space.wrap('__main__') @@ -157,7 +157,7 @@ return action, None def get_default_action(space): - w_action = get_warnings_attr(space, "default_action"); + w_action = get_warnings_attr(space, "defaultaction"); if w_action is None: return space.str_w(space.fromcache(State).w_default_action) Modified: pypy/branch/fast-forward/pypy/module/_warnings/test/test_warnings.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_warnings/test/test_warnings.py (original) +++ pypy/branch/fast-forward/pypy/module/_warnings/test/test_warnings.py Thu Nov 4 21:21:18 2010 @@ -25,3 +25,16 @@ _warnings.warn_explicit("some message", Warning, "", 1, module_globals=globals()) + def test_default_action(self): + import warnings, _warnings + warnings.defaultaction = 'ignore' + warnings.resetwarnings() + with warnings.catch_warnings(record=True) as w: + __warningregistry__ = {} + _warnings.warn_explicit("message", UserWarning, "", 44, + registry={}) + assert len(w) == 0 + warnings.defaultaction = 'default' + + + From hakanardo at codespeak.net Thu Nov 4 21:39:22 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Thu, 4 Nov 2010 21:39:22 +0100 (CET) Subject: [pypy-svn] r78704 - pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test Message-ID: <20101104203922.31875282BDA@codespeak.net> Author: hakanardo Date: Thu Nov 4 21:39:19 2010 New Revision: 78704 Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py Log: fixed test Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py Thu Nov 4 21:39:19 2010 @@ -245,9 +245,10 @@ ''', 98, ([20], 20), ([31], 32)) - ops = self.get_by_bytecode("LOAD_GLOBAL") + ops = self.get_by_bytecode("LOAD_GLOBAL", True) assert len(ops) == 5 - assert ops[0].get_opnames() == ["getfield_gc", "guard_value", + assert ops[0].get_opnames() == ["guard_value", + "getfield_gc", "guard_value", "getfield_gc", "guard_isnull", "getfield_gc", "guard_nonnull_class"] # the second getfield on the same globals is quicker @@ -259,7 +260,7 @@ assert ops[3].get_opnames() == ["guard_value", "getfield_gc", "guard_isnull"] assert not ops[4] - ops = self.get_by_bytecode("CALL_FUNCTION") + ops = self.get_by_bytecode("CALL_FUNCTION", True) assert len(ops) == 2 for i, bytecode in enumerate(ops): if i == 0: @@ -269,6 +270,17 @@ assert not bytecode.get_opnames("new") assert len(bytecode.get_opnames("guard")) <= 10 + ops = self.get_by_bytecode("LOAD_GLOBAL") + assert len(ops) == 5 + for bytecode in ops: + assert not bytecode + + ops = self.get_by_bytecode("CALL_FUNCTION") + assert len(ops) == 2 + for bytecode in ops: + assert len(bytecode) <= 1 + + def test_method_call(self): self.run_source(''' class A(object): From afa at codespeak.net Thu Nov 4 21:44:50 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 4 Nov 2010 21:44:50 +0100 (CET) Subject: [pypy-svn] r78705 - in pypy/branch/fast-forward/pypy/translator/goal: . test2 Message-ID: <20101104204450.E4806282BD4@codespeak.net> Author: afa Date: Thu Nov 4 21:44:49 2010 New Revision: 78705 Modified: pypy/branch/fast-forward/pypy/translator/goal/app_main.py pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py Log: Allow several -W options and honor the PYTHONWARNINGS env var. Modified: pypy/branch/fast-forward/pypy/translator/goal/app_main.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/goal/app_main.py (original) +++ pypy/branch/fast-forward/pypy/translator/goal/app_main.py Thu Nov 4 21:44:49 2010 @@ -268,6 +268,7 @@ def parse_command_line(argv): options = default_options.copy() + options['warnoptions'] = [] print_sys_flags = False i = 0 while i < len(argv): @@ -335,7 +336,8 @@ options["run_module"] = True break elif arg.startswith('-W'): - options["warnoptions"], i = get_argument('-W', argv, i) + warnoption, i = get_argument('-W', argv, i) + options["warnoptions"].append(warnoption) elif arg.startswith('--jit'): jitparam, i = get_argument('--jit', argv, i) if 'pypyjit' not in sys.builtin_module_names: @@ -401,8 +403,11 @@ # "site.py" file in the script's directory. sys.path.insert(0, '') + pythonwarnings = os.getenv('PYTHONWARNINGS') + if pythonwarnings: + warnoptions.extend(pythonwarnings.split(',')) if warnoptions: - sys.warnoptions.append(warnoptions) + sys.warnoptions[:] = warnoptions from warnings import _processoptions _processoptions(sys.warnoptions) Modified: pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py Thu Nov 4 21:44:49 2010 @@ -431,6 +431,16 @@ finally: os.environ['PYTHONSTARTUP'] = old + def test_pythonwarnings(self): + old = os.environ.get('PYTHONWARNINGS', '') + try: + os.environ['PYTHONWARNINGS'] = "once,error" + data = self.run('-W ignore -W default ' + '-c "import sys; print sys.warnoptions"') + assert "['ignore', 'default', 'once', 'error']" in data + finally: + os.environ['PYTHONWARNINGS'] = old + def test_option_m(self): p = os.path.join(autopath.this_dir, 'mymodule.py') p = os.path.abspath(p) From afa at codespeak.net Thu Nov 4 21:49:57 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 4 Nov 2010 21:49:57 +0100 (CET) Subject: [pypy-svn] r78706 - pypy/branch/fast-forward/pypy/config Message-ID: <20101104204957.55FA7282BDB@codespeak.net> Author: afa Date: Thu Nov 4 21:49:55 2010 New Revision: 78706 Modified: pypy/branch/fast-forward/pypy/config/pypyoption.py Log: Enable the _warnings module by default Modified: pypy/branch/fast-forward/pypy/config/pypyoption.py ============================================================================== --- pypy/branch/fast-forward/pypy/config/pypyoption.py (original) +++ pypy/branch/fast-forward/pypy/config/pypyoption.py Thu Nov 4 21:49:55 2010 @@ -31,7 +31,7 @@ "crypt", "signal", "_rawffi", "termios", "zlib", "bz2", "struct", "_hashlib", "_md5", "_sha", "_minimal_curses", "cStringIO", "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array", - "_multiprocessing"] + "_multiprocessing", '_warnings'] )) working_oo_modules = default_modules.copy() From afa at codespeak.net Thu Nov 4 22:16:52 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 4 Nov 2010 22:16:52 +0100 (CET) Subject: [pypy-svn] r78707 - pypy/branch/fast-forward/lib-python/modified-2.7.0/test Message-ID: <20101104211652.59B0C282BDA@codespeak.net> Author: afa Date: Thu Nov 4 22:16:50 2010 New Revision: 78707 Added: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_marshal.py - copied, changed from r78705, pypy/branch/fast-forward/lib-python/2.7.0/test/test_marshal.py Log: Allow test_marshal to pass, mostly by copying code from py3k, and skip one implementation detail Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_marshal.py (from r78705, pypy/branch/fast-forward/lib-python/2.7.0/test/test_marshal.py) ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_marshal.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_marshal.py Thu Nov 4 22:16:50 2010 @@ -7,20 +7,31 @@ import unittest import os -class IntTestCase(unittest.TestCase): +class HelperMixin: + def helper(self, sample, *extra, **kwargs): + expected = kwargs.get('expected', sample) + new = marshal.loads(marshal.dumps(sample, *extra)) + self.assertEqual(expected, new) + self.assertEqual(type(expected), type(new)) + try: + with open(test_support.TESTFN, "wb") as f: + marshal.dump(sample, f, *extra) + with open(test_support.TESTFN, "rb") as f: + new = marshal.load(f) + self.assertEqual(expected, new) + self.assertEqual(type(expected), type(new)) + finally: + test_support.unlink(test_support.TESTFN) + + +class IntTestCase(unittest.TestCase, HelperMixin): def test_ints(self): # Test the full range of Python ints. n = sys.maxint while n: for expected in (-n, n): - s = marshal.dumps(expected) - got = marshal.loads(s) - self.assertEqual(expected, got) - marshal.dump(expected, file(test_support.TESTFN, "wb")) - got = marshal.load(file(test_support.TESTFN, "rb")) - self.assertEqual(expected, got) + self.helper(expected) n = n >> 1 - os.unlink(test_support.TESTFN) def test_int64(self): # Simulate int marshaling on a 64-bit box. This is most interesting if @@ -48,28 +59,16 @@ def test_bool(self): for b in (True, False): - new = marshal.loads(marshal.dumps(b)) - self.assertEqual(b, new) - self.assertEqual(type(b), type(new)) - marshal.dump(b, file(test_support.TESTFN, "wb")) - new = marshal.load(file(test_support.TESTFN, "rb")) - self.assertEqual(b, new) - self.assertEqual(type(b), type(new)) + self.helper(b) -class FloatTestCase(unittest.TestCase): +class FloatTestCase(unittest.TestCase, HelperMixin): def test_floats(self): # Test a few floats small = 1e-25 n = sys.maxint * 3.7e250 while n > small: for expected in (-n, n): - f = float(expected) - s = marshal.dumps(f) - got = marshal.loads(s) - self.assertEqual(f, got) - marshal.dump(f, file(test_support.TESTFN, "wb")) - got = marshal.load(file(test_support.TESTFN, "rb")) - self.assertEqual(f, got) + self.helper(expected) n /= 123.4567 f = 0.0 @@ -85,59 +84,25 @@ while n < small: for expected in (-n, n): f = float(expected) - - s = marshal.dumps(f) - got = marshal.loads(s) - self.assertEqual(f, got) - - s = marshal.dumps(f, 1) - got = marshal.loads(s) - self.assertEqual(f, got) - - marshal.dump(f, file(test_support.TESTFN, "wb")) - got = marshal.load(file(test_support.TESTFN, "rb")) - self.assertEqual(f, got) - - marshal.dump(f, file(test_support.TESTFN, "wb"), 1) - got = marshal.load(file(test_support.TESTFN, "rb")) - self.assertEqual(f, got) + self.helper(f) + self.helper(f, 1) n *= 123.4567 - os.unlink(test_support.TESTFN) -class StringTestCase(unittest.TestCase): +class StringTestCase(unittest.TestCase, HelperMixin): def test_unicode(self): for s in [u"", u"Andr? Previn", u"abc", u" "*10000]: - new = marshal.loads(marshal.dumps(s)) - self.assertEqual(s, new) - self.assertEqual(type(s), type(new)) - marshal.dump(s, file(test_support.TESTFN, "wb")) - new = marshal.load(file(test_support.TESTFN, "rb")) - self.assertEqual(s, new) - self.assertEqual(type(s), type(new)) - os.unlink(test_support.TESTFN) + self.helper(s) def test_string(self): for s in ["", "Andr? Previn", "abc", " "*10000]: - new = marshal.loads(marshal.dumps(s)) - self.assertEqual(s, new) - self.assertEqual(type(s), type(new)) - marshal.dump(s, file(test_support.TESTFN, "wb")) - new = marshal.load(file(test_support.TESTFN, "rb")) - self.assertEqual(s, new) - self.assertEqual(type(s), type(new)) - os.unlink(test_support.TESTFN) + self.helper(s) def test_buffer(self): for s in ["", "Andr? Previn", "abc", " "*10000]: with test_support.check_py3k_warnings(("buffer.. not supported", DeprecationWarning)): b = buffer(s) - new = marshal.loads(marshal.dumps(b)) - self.assertEqual(s, new) - marshal.dump(b, file(test_support.TESTFN, "wb")) - new = marshal.load(file(test_support.TESTFN, "rb")) - self.assertEqual(s, new) - os.unlink(test_support.TESTFN) + self.helper(b, expected=s) class ExceptionTestCase(unittest.TestCase): def test_exceptions(self): @@ -150,7 +115,7 @@ new = marshal.loads(marshal.dumps(co)) self.assertEqual(co, new) -class ContainerTestCase(unittest.TestCase): +class ContainerTestCase(unittest.TestCase, HelperMixin): d = {'astring': 'foo at bar.baz.spam', 'afloat': 7283.43, 'anint': 2**20, @@ -161,42 +126,20 @@ 'aunicode': u"Andr? Previn" } def test_dict(self): - new = marshal.loads(marshal.dumps(self.d)) - self.assertEqual(self.d, new) - marshal.dump(self.d, file(test_support.TESTFN, "wb")) - new = marshal.load(file(test_support.TESTFN, "rb")) - self.assertEqual(self.d, new) - os.unlink(test_support.TESTFN) + self.helper(self.d) def test_list(self): lst = self.d.items() - new = marshal.loads(marshal.dumps(lst)) - self.assertEqual(lst, new) - marshal.dump(lst, file(test_support.TESTFN, "wb")) - new = marshal.load(file(test_support.TESTFN, "rb")) - self.assertEqual(lst, new) - os.unlink(test_support.TESTFN) + self.helper(lst) def test_tuple(self): t = tuple(self.d.keys()) - new = marshal.loads(marshal.dumps(t)) - self.assertEqual(t, new) - marshal.dump(t, file(test_support.TESTFN, "wb")) - new = marshal.load(file(test_support.TESTFN, "rb")) - self.assertEqual(t, new) - os.unlink(test_support.TESTFN) + self.helper(t) def test_sets(self): for constructor in (set, frozenset): t = constructor(self.d.keys()) - new = marshal.loads(marshal.dumps(t)) - self.assertEqual(t, new) - self.assertTrue(isinstance(new, constructor)) - self.assertNotEqual(id(t), id(new)) - marshal.dump(t, file(test_support.TESTFN, "wb")) - new = marshal.load(file(test_support.TESTFN, "rb")) - self.assertEqual(t, new) - os.unlink(test_support.TESTFN) + self.helper(t) class BugsTestCase(unittest.TestCase): def test_bug_5888452(self): @@ -226,6 +169,7 @@ s = 'c' + ('X' * 4*4) + '{' * 2**20 self.assertRaises(ValueError, marshal.loads, s) + @test_support.impl_detail('specific recursion check') def test_recursion_limit(self): # Create a deeply nested structure. head = last = [] From afa at codespeak.net Thu Nov 4 22:32:02 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 4 Nov 2010 22:32:02 +0100 (CET) Subject: [pypy-svn] r78708 - pypy/branch/fast-forward/lib-python Message-ID: <20101104213202.D627B282B9D@codespeak.net> Author: afa Date: Thu Nov 4 22:32:01 2010 New Revision: 78708 Modified: pypy/branch/fast-forward/lib-python/TODO Log: Update TODO file Modified: pypy/branch/fast-forward/lib-python/TODO ============================================================================== --- pypy/branch/fast-forward/lib-python/TODO (original) +++ pypy/branch/fast-forward/lib-python/TODO Thu Nov 4 22:32:01 2010 @@ -42,11 +42,13 @@ - signal.set_wakeup_fd() +- add 'unicode' in ObjSpace.MethodTable + probably a default implementation that + falls back to space.str(). + Longer tasks ------------ -- Implement the _io module. At least _io.FileIO, and have io.py import - everything from _pyio.py +- Finish the _io module. - Finish _multiprocessing @@ -54,6 +56,10 @@ - Enable -3 option to run test_py3kwarn. +- Change the way the unicodedata module stores its database: in + unicodedb_5_2_0.py, the "_composition" map contains values > 2**32 which + causes the translation to fail. + More difficult issues --------------------- From afa at codespeak.net Thu Nov 4 22:56:14 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 4 Nov 2010 22:56:14 +0100 (CET) Subject: [pypy-svn] r78709 - pypy/branch/fast-forward/lib-python Message-ID: <20101104215614.D72C3282BDC@codespeak.net> Author: afa Date: Thu Nov 4 22:56:13 2010 New Revision: 78709 Modified: pypy/branch/fast-forward/lib-python/TODO Log: One more largish TODO item Modified: pypy/branch/fast-forward/lib-python/TODO ============================================================================== --- pypy/branch/fast-forward/lib-python/TODO (original) +++ pypy/branch/fast-forward/lib-python/TODO Thu Nov 4 22:56:13 2010 @@ -60,6 +60,9 @@ unicodedb_5_2_0.py, the "_composition" map contains values > 2**32 which causes the translation to fail. +- "Shorter float representation": copy dtoa.c from CPython and use it to + format/parse floats. Enable this with a translation option. + More difficult issues --------------------- From afa at codespeak.net Thu Nov 4 23:25:27 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 4 Nov 2010 23:25:27 +0100 (CET) Subject: [pypy-svn] r78710 - in pypy/branch/fast-forward/pypy: module/__builtin__ objspace/std/test Message-ID: <20101104222527.D38AF282BF9@codespeak.net> Author: afa Date: Thu Nov 4 23:25:25 2010 New Revision: 78710 Modified: pypy/branch/fast-forward/pypy/module/__builtin__/operation.py pypy/branch/fast-forward/pypy/objspace/std/test/test_floatobject.py Log: Test and fix extreme cases for round() Modified: pypy/branch/fast-forward/pypy/module/__builtin__/operation.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/operation.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/operation.py Thu Nov 4 23:25:25 2010 @@ -8,6 +8,9 @@ from pypy.interpreter.gateway import interp2app, unwrap_spec from pypy.interpreter.typedef import TypeDef from pypy.rlib.runicode import UNICHR +from pypy.rlib.rarithmetic import isnan, isinf +from pypy.rlib import rfloat +import math import __builtin__ NoneNotWrapped = gateway.NoneNotWrapped @@ -140,38 +143,65 @@ # ____________________________________________________________ -from math import floor as _floor -from math import ceil as _ceil +# Here 0.30103 is an upper bound for log10(2) +NDIGITS_MAX = int((rfloat.DBL_MANT_DIG - rfloat.DBL_MIN_EXP) * 0.30103) +NDIGITS_MIN = -int((rfloat.DBL_MAX_EXP + 1) * 0.30103) -def round(space, number, ndigits=0): +def round(space, number, w_ndigits=0): """round(number[, ndigits]) -> floating point number Round a number to a given precision in decimal digits (default 0 digits). This always returns a floating point number. Precision may be negative.""" - # Algortithm copied directly from CPython - f = 1.0 - if ndigits < 0: - i = -ndigits - else: - i = ndigits - while i > 0: - f = f*10.0 - i -= 1 - if ndigits < 0: - number /= f + # Algorithm copied directly from CPython + + # interpret 2nd argument as a Py_ssize_t; clip on overflow + ndigits = space.getindex_w(w_ndigits, None) + + # nans, infinities and zeros round to themselves + if number == 0 or isinf(number) or isnan(number): + return space.wrap(number) + + # Deal with extreme values for ndigits. For ndigits > NDIGITS_MAX, x + # always rounds to itself. For ndigits < NDIGITS_MIN, x always + # rounds to +-0.0. + if ndigits > NDIGITS_MAX: + return space.wrap(number) + elif ndigits < NDIGITS_MIN: + # return 0.0, but with sign of x + return space.wrap(0.0 * number) + + if ndigits >= 0: + if ndigits > 22: + # pow1 and pow2 are each safe from overflow, but + # pow1*pow2 ~= pow(10.0, ndigits) might overflow + pow1 = math.pow(10.0, ndigits - 22) + pow2 = 1e22 + else: + pow1 = math.pow(10.0, ndigits) + pow2 = 1.0 + + y = (number * pow1) * pow2 + # if y overflows, then rounded value is exactly x + if isinf(y): + return space.wrap(number) + else: - number *= f - if number >= 0.0: - number = _floor(number + 0.5) + pow1 = math.pow(10.0, -ndigits); + pow2 = 1.0 # unused; for translation + y = number / pow1 + + if y >= 0.0: + z = math.floor(y + 0.5) else: - number = _ceil(number - 0.5) - if ndigits < 0: - number *= f + z = math.ceil(y - 0.5) + + if ndigits >= 0: + z = (z / pow2) / pow1 else: - number /= f - return space.wrap(number) + z *= pow1 + return space.wrap(z) # -round.unwrap_spec = [ObjSpace, float, int] +round.unwrap_spec = [ObjSpace, float, W_Root] # ____________________________________________________________ Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_floatobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_floatobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_floatobject.py Thu Nov 4 23:25:25 2010 @@ -147,6 +147,7 @@ assert float(X()) == 42. def test_round(self): + import math assert 1.0 == round(1.0) assert 1.0 == round(1.1) assert 2.0 == round(1.9) @@ -158,6 +159,11 @@ assert 22.2 == round(22.222222, 1) assert 20.0 == round(22.22222, -1) assert 0.0 == round(22.22222, -2) + # + assert round(123.456, -308) == 0.0 + assert round(123.456, -700) == 0.0 + assert round(123.456, -2**100) == 0.0 + assert math.copysign(1., round(-123.456, -700)) == -1. def test_special_float_method(self): class a(object): From afa at codespeak.net Thu Nov 4 23:50:32 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 4 Nov 2010 23:50:32 +0100 (CET) Subject: [pypy-svn] r78711 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101104225032.0B30B282BDC@codespeak.net> Author: afa Date: Thu Nov 4 23:50:30 2010 New Revision: 78711 Modified: pypy/branch/fast-forward/pypy/objspace/std/formatting.py pypy/branch/fast-forward/pypy/objspace/std/test/test_stringformat.py Log: '%F' % nan == 'NAN' Modified: pypy/branch/fast-forward/pypy/objspace/std/formatting.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/formatting.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/formatting.py Thu Nov 4 23:50:30 2010 @@ -117,12 +117,21 @@ space = self.space x = space.float_w(maybe_float(space, w_value)) if isnan(x): - r = 'nan' + if char in 'EFG': + r = 'NAN' + else: + r = 'nan' elif isinf(x): if x < 0: - r = '-inf' + if char in 'EFG': + r = '-INF' + else: + r = '-inf' else: - r = 'inf' + if char in 'EFG': + r = 'INF' + else: + r = 'inf' else: prec = self.prec if prec < 0: Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_stringformat.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_stringformat.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_stringformat.py Thu Nov 4 23:50:30 2010 @@ -253,10 +253,15 @@ def test_subnormal(self): inf = 1e300 * 1e300 assert "%f" % (inf,) == 'inf' + assert "%E" % (inf,) == 'INF' assert "%f" % (-inf,) == '-inf' + assert "%F" % (-inf,) == '-INF' nan = inf / inf assert "%f" % (nan,) == 'nan' assert "%f" % (-nan,) == 'nan' + assert "%E" % (nan,) == 'NAN' + assert "%F" % (nan,) == 'NAN' + assert "%G" % (nan,) == 'NAN' class AppTestUnicodeObject: def test_unicode_convert(self): From afa at codespeak.net Fri Nov 5 01:16:50 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 01:16:50 +0100 (CET) Subject: [pypy-svn] r78712 - in pypy/branch/fast-forward/pypy: . module/_multiprocessing module/_multiprocessing/test Message-ID: <20101105001650.3C3A5282BD4@codespeak.net> Author: afa Date: Fri Nov 5 01:16:47 2010 New Revision: 78712 Modified: pypy/branch/fast-forward/pypy/conftest.py pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py Log: Hard work to let the tests pass with -A. os.dup() could have been a solution, except that it does not work on Windows sockets... Modified: pypy/branch/fast-forward/pypy/conftest.py ============================================================================== --- pypy/branch/fast-forward/pypy/conftest.py (original) +++ pypy/branch/fast-forward/pypy/conftest.py Fri Nov 5 01:16:47 2010 @@ -181,6 +181,9 @@ def getbuiltinmodule(self, name): return __import__(name) + def delslice(self, obj, *args): + obj.__delslice__(*args) + def translation_test_so_skip_if_appdirect(): if option.runappdirect: py.test.skip("translation test, skipped for appdirect") Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Fri Nov 5 01:16:47 2010 @@ -211,9 +211,7 @@ raise WindowsError(geterrno(), "recv") return buf.str(length) def CLOSE(self): - from pypy.rlib._rsocket_rffi import socketclose, geterrno - if socketclose(self.fd) < 0: - raise WindowsError(geterrno(), "close") + socketclose(self.fd) else: def WRITE(self, data): import os @@ -223,7 +221,10 @@ return os.read(self.fd, length) def CLOSE(self): import os - os.close(self.fd) + try: + os.close(self.fd) + except OSError: + pass def __init__(self, fd, flags): W_BaseConnection.__init__(self, flags) Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/test/test_connection.py Fri Nov 5 01:16:47 2010 @@ -1,7 +1,7 @@ import py import sys from pypy.conftest import gettestobjspace, option -from pypy.interpreter.gateway import interp2app +from pypy.interpreter.gateway import interp2app, W_Root class TestImport: def test_simple(self): @@ -93,6 +93,7 @@ def setup_class(cls): space = gettestobjspace(usemodules=('_multiprocessing', 'thread')) cls.space = space + cls.w_connections = space.newlist([]) def socketpair(space): "A socket.socketpair() that works on Windows" @@ -110,25 +111,43 @@ server, addr = serverSocket.accept() # keep sockets alive during the test - cls.connections = server, client + space.call_method(cls.w_connections, "append", space.wrap(server)) + space.call_method(cls.w_connections, "append", space.wrap(client)) return space.wrap((server.fileno(), client.fileno())) - w_socketpair = space.wrap(interp2app(socketpair)) + if option.runappdirect: + w_socketpair = lambda: socketpair(space) + else: + w_socketpair = space.wrap(interp2app(socketpair)) cls.w_make_pair = space.appexec( - [w_socketpair], """(socketpair): + [w_socketpair, cls.w_connections], """(socketpair, connections): import _multiprocessing import os def make_pair(): fd1, fd2 = socketpair() rhandle = _multiprocessing.Connection(fd1, writable=False) whandle = _multiprocessing.Connection(fd2, readable=False) + connections.append(rhandle) + connections.append(whandle) return rhandle, whandle return make_pair """) - def teardown_class(cls): + + def teardown_method(self, func): + # Work hard to close all sockets and connections now! + # since the fd is probably already closed, another unrelated + # part of the program will probably reuse it; + # And any object forgotten here will close it on destruction... try: - del cls.connections + w_connections = self.w_connections except AttributeError: - pass + return + space = self.space + for c in space.unpackiterable(w_connections): + if isinstance(c, W_Root): + space.call_method(c, "close") + else: + c.close() + space.delslice(w_connections, space.wrap(0), space.wrap(100)) From afa at codespeak.net Fri Nov 5 01:35:59 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 01:35:59 +0100 (CET) Subject: [pypy-svn] r78713 - pypy/branch/fast-forward/pypy/module/_lsprof Message-ID: <20101105003559.314F2282BD6@codespeak.net> Author: afa Date: Fri Nov 5 01:35:57 2010 New Revision: 78713 Modified: pypy/branch/fast-forward/pypy/module/_lsprof/interp_lsprof.py Log: The profiler timer function is not allowed to raise. Modified: pypy/branch/fast-forward/pypy/module/_lsprof/interp_lsprof.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_lsprof/interp_lsprof.py (original) +++ pypy/branch/fast-forward/pypy/module/_lsprof/interp_lsprof.py Fri Nov 5 01:35:57 2010 @@ -212,7 +212,12 @@ def timer(self): if self.w_callable: space = self.space - return space.float_w(space.call_function(self.w_callable)) + try: + return space.float_w(space.call_function(self.w_callable)) + except OperationError, e: + e.write_unraisable(space, "timer function ", + self.w_callable) + return 0.0 return time.time() def enable(self, space, w_subcalls=NoneNotWrapped, From fijal at codespeak.net Fri Nov 5 09:50:04 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 5 Nov 2010 09:50:04 +0100 (CET) Subject: [pypy-svn] r78716 - pypy/trunk/pypy/tool Message-ID: <20101105085004.6A5B2282B9D@codespeak.net> Author: fijal Date: Fri Nov 5 09:50:01 2010 New Revision: 78716 Modified: pypy/trunk/pypy/tool/alarm.py pypy/trunk/pypy/tool/readdictinfo.py pypy/trunk/pypy/tool/rundictbenchmarks.py pypy/trunk/pypy/tool/statistic_irc_log.py pypy/trunk/pypy/tool/watchdog.py pypy/trunk/pypy/tool/watchdog_nt.py Log: A bit pointless approach to clean up here. Put if __name__ == '__main__' a bit, most of it should be deleted IMO Modified: pypy/trunk/pypy/tool/alarm.py ============================================================================== --- pypy/trunk/pypy/tool/alarm.py (original) +++ pypy/trunk/pypy/tool/alarm.py Fri Nov 5 09:50:01 2010 @@ -38,8 +38,9 @@ sys.path.insert(0, os.path.dirname(sys.argv[0])) return sys.argv[0] -finished = [] -try: - execfile(_main_with_alarm(finished)) -finally: - finished.append(True) +if __name__ == '__main__': + finished = [] + try: + execfile(_main_with_alarm(finished)) + finally: + finished.append(True) Modified: pypy/trunk/pypy/tool/readdictinfo.py ============================================================================== --- pypy/trunk/pypy/tool/readdictinfo.py (original) +++ pypy/trunk/pypy/tool/readdictinfo.py Fri Nov 5 09:50:01 2010 @@ -7,37 +7,38 @@ import sys -infile = open(sys.argv[1]) +if __name__ == '__main__': + infile = open(sys.argv[1]) -curr = None -slots = [] -for line in infile: - if line == '------------------\n': - if curr: - break - curr = 1 - else: - attr, val = [s.strip() for s in line.split(':')] - slots.append(attr) + curr = None + slots = [] + for line in infile: + if line == '------------------\n': + if curr: + break + curr = 1 + else: + attr, val = [s.strip() for s in line.split(':')] + slots.append(attr) -class DictInfo(object): - __slots__ = slots + class DictInfo(object): + __slots__ = slots -infile = open(sys.argv[1]) + infile = open(sys.argv[1]) -infos = [] + infos = [] -for line in infile: - if line == '------------------\n': - curr = object.__new__(DictInfo) - infos.append(curr) - else: - attr, val = [s.strip() for s in line.split(':')] - if '.' in val: - val = float(val) + for line in infile: + if line == '------------------\n': + curr = object.__new__(DictInfo) + infos.append(curr) else: - val = int(val) - setattr(curr, attr, val) + attr, val = [s.strip() for s in line.split(':')] + if '.' in val: + val = float(val) + else: + val = int(val) + setattr(curr, attr, val) def histogram(infos, keyattr, *attrs): r = {} Modified: pypy/trunk/pypy/tool/rundictbenchmarks.py ============================================================================== --- pypy/trunk/pypy/tool/rundictbenchmarks.py (original) +++ pypy/trunk/pypy/tool/rundictbenchmarks.py Fri Nov 5 09:50:01 2010 @@ -7,20 +7,21 @@ # need to hack a copy of rst2html for yourself (svn docutils # required). -try: - os.unlink("dictinfo.txt") -except os.error: - pass +if __name__ == '__main__': + try: + os.unlink("dictinfo.txt") + except os.error: + pass -progs = [('pystone', ['-c', 'from test import pystone; pystone.main()']), - ('richards', ['richards.py']), - ('docutils', ['rst2html.py', '../../doc/coding-guide.txt', 'foo.html']), - ('translate', ['translate.py', '--backendopt', '--no-compile', '--batch', - 'targetrpystonedalone.py']) - ] + progs = [('pystone', ['-c', 'from test import pystone; pystone.main()']), + ('richards', ['richards.py']), + ('docutils', ['rst2html.py', '../../doc/coding-guide.txt', 'foo.html']), + ('translate', ['translate.py', '--backendopt', '--no-compile', '--batch', + 'targetrpystonedalone.py']) + ] -EXE = sys.argv[1] + EXE = sys.argv[1] -for suffix, args in progs: - os.spawnv(os.P_WAIT, EXE, [EXE] + args) - os.rename('dictinfo.txt', 'dictinfo-%s.txt'%suffix) + for suffix, args in progs: + os.spawnv(os.P_WAIT, EXE, [EXE] + args) + os.rename('dictinfo.txt', 'dictinfo-%s.txt'%suffix) Modified: pypy/trunk/pypy/tool/statistic_irc_log.py ============================================================================== --- pypy/trunk/pypy/tool/statistic_irc_log.py (original) +++ pypy/trunk/pypy/tool/statistic_irc_log.py Fri Nov 5 09:50:01 2010 @@ -2,53 +2,54 @@ from os import system, chdir from urllib import urlopen -log_URL = 'http://tismerysoft.de/pypy/irc-logs/' -archive_FILENAME = 'pypy.tar.gz' - -tempdir = py.test.ensuretemp("irc-log") - -# get compressed archive -chdir( str(tempdir)) -system('wget -q %s%s' % (log_URL, archive_FILENAME)) -system('tar xzf %s' % archive_FILENAME) -chdir('pypy') - -# get more recent daily logs -pypydir = tempdir.join('pypy') -for line in urlopen(log_URL + 'pypy/').readlines(): - i = line.find('%23pypy.log.') - if i == -1: - continue - filename = line[i:].split('"')[0] - system('wget -q %spypy/%s' % (log_URL, filename)) - -# rename to YYYYMMDD -for log_filename in pypydir.listdir('#pypy.log.*'): - rename_to = None - b = log_filename.basename - if '-' in b: - rename_to = log_filename.basename.replace('-', '') - elif len(b) == 19: - months= 'Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec'.split() - day = b[10:12] - month = months.index(b[12:15]) + 1 - year = b[15:20] - rename_to = '#pypy.log.%04s%02d%02s' % (year, month, day) - - if rename_to: - log_filename.rename(rename_to) - #print 'RENAMED', log_filename, 'TO', rename_to - -# print sorted list of filenames of daily logs -print 'irc://irc.freenode.org/pypy' -print 'date, messages, visitors' -for log_filename in pypydir.listdir('#pypy.log.*'): - n_messages, visitors = 0, {} - f = str(log_filename) - for s in file(f): - if '<' in s and '>' in s: - n_messages += 1 - elif ' joined #pypy' in s: - v = s.split()[1] - visitors[v] = True - print '%04s-%02s-%02s, %d, %d' % (f[-8:-4], f[-4:-2], f[-2:], n_messages, len(visitors.keys())) +if __name__ == '__main__': + log_URL = 'http://tismerysoft.de/pypy/irc-logs/' + archive_FILENAME = 'pypy.tar.gz' + + tempdir = py.test.ensuretemp("irc-log") + + # get compressed archive + chdir( str(tempdir)) + system('wget -q %s%s' % (log_URL, archive_FILENAME)) + system('tar xzf %s' % archive_FILENAME) + chdir('pypy') + + # get more recent daily logs + pypydir = tempdir.join('pypy') + for line in urlopen(log_URL + 'pypy/').readlines(): + i = line.find('%23pypy.log.') + if i == -1: + continue + filename = line[i:].split('"')[0] + system('wget -q %spypy/%s' % (log_URL, filename)) + + # rename to YYYYMMDD + for log_filename in pypydir.listdir('#pypy.log.*'): + rename_to = None + b = log_filename.basename + if '-' in b: + rename_to = log_filename.basename.replace('-', '') + elif len(b) == 19: + months= 'Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec'.split() + day = b[10:12] + month = months.index(b[12:15]) + 1 + year = b[15:20] + rename_to = '#pypy.log.%04s%02d%02s' % (year, month, day) + + if rename_to: + log_filename.rename(rename_to) + #print 'RENAMED', log_filename, 'TO', rename_to + + # print sorted list of filenames of daily logs + print 'irc://irc.freenode.org/pypy' + print 'date, messages, visitors' + for log_filename in pypydir.listdir('#pypy.log.*'): + n_messages, visitors = 0, {} + f = str(log_filename) + for s in file(f): + if '<' in s and '>' in s: + n_messages += 1 + elif ' joined #pypy' in s: + v = s.split()[1] + visitors[v] = True + print '%04s-%02s-%02s, %d, %d' % (f[-8:-4], f[-4:-2], f[-2:], n_messages, len(visitors.keys())) Modified: pypy/trunk/pypy/tool/watchdog.py ============================================================================== --- pypy/trunk/pypy/tool/watchdog.py (original) +++ pypy/trunk/pypy/tool/watchdog.py Fri Nov 5 09:50:01 2010 @@ -7,9 +7,6 @@ return name return 'signal %d' % (n,) -timeout = float(sys.argv[1]) -timedout = False - def childkill(): global timedout timedout = True @@ -20,31 +17,35 @@ except OSError: pass -pid = os.fork() -if pid == 0: - os.execvp(sys.argv[2], sys.argv[2:]) -else: # parent - t = threading.Timer(timeout, childkill) - t.start() - while True: - try: - pid, status = os.waitpid(pid, 0) - except KeyboardInterrupt: - continue +if __name__ == '__main__': + timeout = float(sys.argv[1]) + timedout = False + + pid = os.fork() + if pid == 0: + os.execvp(sys.argv[2], sys.argv[2:]) + else: # parent + t = threading.Timer(timeout, childkill) + t.start() + while True: + try: + pid, status = os.waitpid(pid, 0) + except KeyboardInterrupt: + continue + else: + t.cancel() + break + if os.WIFEXITED(status): + sys.exit(os.WEXITSTATUS(status)) else: - t.cancel() - break - if os.WIFEXITED(status): - sys.exit(os.WEXITSTATUS(status)) - else: - assert os.WIFSIGNALED(status) - sign = os.WTERMSIG(status) - if timedout and sign == signal.SIGTERM: + assert os.WIFSIGNALED(status) + sign = os.WTERMSIG(status) + if timedout and sign == signal.SIGTERM: + sys.exit(1) + signame = getsignalname(sign) + sys.stderr.write("="*26 + "timedout" + "="*26 + "\n") + sys.stderr.write("="*25 + " %-08s " % signame + "="*25 + "\n") sys.exit(1) - signame = getsignalname(sign) - sys.stderr.write("="*26 + "timedout" + "="*26 + "\n") - sys.stderr.write("="*25 + " %-08s " % signame + "="*25 + "\n") - sys.exit(1) - - + + Modified: pypy/trunk/pypy/tool/watchdog_nt.py ============================================================================== --- pypy/trunk/pypy/tool/watchdog_nt.py (original) +++ pypy/trunk/pypy/tool/watchdog_nt.py Fri Nov 5 09:50:01 2010 @@ -2,11 +2,6 @@ import threading import ctypes -PROCESS_TERMINATE = 0x1 - -timeout = float(sys.argv[1]) -timedout = False - def childkill(pid): global timedout timedout = True @@ -14,19 +9,25 @@ sys.stderr.write("="*26 + "timedout" + "="*26 + "\n") ctypes.windll.kernel32.TerminateProcess(pid, 1) -pid = os.spawnv(os.P_NOWAIT, sys.argv[2], sys.argv[2:]) +if __name__ == '__main__': + PROCESS_TERMINATE = 0x1 + + timeout = float(sys.argv[1]) + timedout = False + + pid = os.spawnv(os.P_NOWAIT, sys.argv[2], sys.argv[2:]) + + t = threading.Timer(timeout, childkill, (pid,)) + t.start() + while True: + try: + pid, status = os.waitpid(pid, 0) + except KeyboardInterrupt: + continue + else: + t.cancel() + break + + #print 'status ', status >> 8 + sys.exit(status >> 8) -t = threading.Timer(timeout, childkill, (pid,)) -t.start() -while True: - try: - pid, status = os.waitpid(pid, 0) - except KeyboardInterrupt: - continue - else: - t.cancel() - break - -#print 'status ', status >> 8 -sys.exit(status >> 8) - From fijal at codespeak.net Fri Nov 5 09:50:38 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 5 Nov 2010 09:50:38 +0100 (CET) Subject: [pypy-svn] r78717 - pypy/trunk/pypy/jit/tool Message-ID: <20101105085038.467A1282BDC@codespeak.net> Author: fijal Date: Fri Nov 5 09:50:36 2010 New Revision: 78717 Modified: pypy/trunk/pypy/jit/tool/showstats.py Log: minor tweaks Modified: pypy/trunk/pypy/jit/tool/showstats.py ============================================================================== --- pypy/trunk/pypy/jit/tool/showstats.py (original) +++ pypy/trunk/pypy/jit/tool/showstats.py Fri Nov 5 09:50:36 2010 @@ -10,7 +10,8 @@ def main(argv): log = logparser.parse_log_file(argv[0]) - parts = logparser.extract_category(log, "jit-log-noopt-") + log_count_lines = open(argv[0] + '.count').readlines() + parts = logparser.extract_category(log, "jit-log-opt-") for i, oplist in enumerate(parts): loop = parse(oplist, no_namespace=True, nonstrict=True) num_ops = 0 @@ -27,6 +28,7 @@ print "Loop #%d, length: %d, opcodes: %d, guards: %d" % (i, num_ops, num_dmp, num_guards) else: print "Loop #%d, length: %d, opcodes: %d, guards: %d, %f" % (i, num_ops, num_dmp, num_guards, num_ops/num_dmp) - + print loop.comment, "run", log_count_lines[i].split(":")[1].strip(), "times" + if __name__ == '__main__': main(sys.argv[1:]) From fijal at codespeak.net Fri Nov 5 10:24:01 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 5 Nov 2010 10:24:01 +0100 (CET) Subject: [pypy-svn] r78718 - pypy/trunk/pypy/jit/tl Message-ID: <20101105092401.597E7282B9D@codespeak.net> Author: fijal Date: Fri Nov 5 10:23:59 2010 New Revision: 78718 Modified: pypy/trunk/pypy/jit/tl/pypyjit.py Log: enable _weakref module. Otherwise translation is broken Modified: pypy/trunk/pypy/jit/tl/pypyjit.py ============================================================================== --- pypy/trunk/pypy/jit/tl/pypyjit.py (original) +++ pypy/trunk/pypy/jit/tl/pypyjit.py Fri Nov 5 10:23:59 2010 @@ -38,7 +38,7 @@ config.objspace.allworkingmodules = False config.objspace.usemodules.pypyjit = True config.objspace.usemodules.array = True -config.objspace.usemodules._weakref = False +config.objspace.usemodules._weakref = True config.objspace.usemodules._sre = False # config.objspace.usemodules._ffi = True From david at codespeak.net Fri Nov 5 11:01:23 2010 From: david at codespeak.net (david at codespeak.net) Date: Fri, 5 Nov 2010 11:01:23 +0100 (CET) Subject: [pypy-svn] r78719 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . test Message-ID: <20101105100123.62B2F282B9D@codespeak.net> Author: david Date: Fri Nov 5 11:01:20 2010 New Revision: 78719 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Log: Implement POP instruction Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Fri Nov 5 11:01:20 2010 @@ -108,7 +108,7 @@ self.mc.BL(rffi.cast(lltype.Signed, decode_registers_addr)) self.mc.MOV_rr(r.ip.value, r.r0.value) - self.mc.LDM(r.sp.value, [reg.value for reg in r.all_regs], w=1) # XXX Replace with POP instr. someday + self.mc.POP([reg.value for reg in r.all_regs]) self.mc.MOV_rr(r.r0.value, r.ip.value) self.gen_func_epilog() @@ -169,7 +169,7 @@ epilog_size = 2*WORD def gen_func_epilog(self,cond=c.AL): self.mc.MOV_rr(r.sp.value, r.fp.value) - self.mc.LDM(r.sp.value, [reg.value for reg in r.callee_restored_registers], cond=cond, w=1) + self.mc.POP([reg.value for reg in r.callee_restored_registers], cond=cond) def gen_func_prolog(self): self.mc.PUSH([reg.value for reg in r.callee_saved_registers]) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Fri Nov 5 11:01:20 2010 @@ -21,7 +21,7 @@ else: self.PUSH(range(2, 4), cond=c) self.BL(addr, cond=c, some_reg=reg.r2) - self.LDM(reg.sp.value, range(2, 4), w=1, cond=c) # XXX Replace with POP instr. someday + self.POP(range(2,4), cond=c) return f class AbstractARMv7Builder(object): @@ -40,10 +40,15 @@ raise NotImplentedError def PUSH(self, regs, cond=cond.AL): - assert reg.sp not in regs + assert reg.sp.value not in regs instr = self._encode_reg_list(cond << 28 | 0x92D << 16, regs) self.write32(instr) + def POP(self, regs, cond=cond.AL): + assert reg.lr.value not in regs + instr = self._encode_reg_list(cond << 28 | 0x8BD << 16, regs) + self.write32(instr) + def LDM(self, rn, regs, w=0, cond=cond.AL): instr = cond << 28 | 0x89 << 20 | w << 21 | (rn & 0xFF) << 16 instr = self._encode_reg_list(instr, regs) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Fri Nov 5 11:01:20 2010 @@ -208,6 +208,7 @@ regalloc.possibly_free_var(reg) adr = self.cpu.cast_adr_to_int(op.getarg(0).getint()) + # XXX use PUSH here instead of spilling every reg for itself regalloc.before_call() reg_args = min(op.numargs()-1, 4) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_assembler.py Fri Nov 5 11:01:20 2010 @@ -139,7 +139,7 @@ self.a.mc.PUSH(range(2, 12)) div_addr = rffi.cast(lltype.Signed, llhelper(arm_int_div_sign, arm_int_div)) self.a.mc.BL(div_addr, some_reg=r.r2) - self.a.mc.LDM(r.sp.value, range(2, 12), w=1) # XXX Replace with POP instr. someday + self.a.mc.POP(range(2, 12)) self.a.gen_func_epilog() assert run_asm(self.a) == 61 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Fri Nov 5 11:01:20 2010 @@ -1,5 +1,6 @@ from pypy.jit.backend.arm import registers as r from pypy.jit.backend.arm import codebuilder +from pypy.jit.backend.arm import conditions from pypy.jit.backend.arm import instructions from pypy.jit.backend.arm.test.support import requires_arm_as from gen import assemble @@ -121,6 +122,30 @@ self.assert_equal('MCR P15, 0, r1, c7, c10, 0') + def test_push_eq_stmdb(self): + # XXX check other conditions in STMDB + self.cb.PUSH([reg.value for reg in r.caller_resp], cond=conditions.AL) + self.assert_equal('STMDB SP!, {r0, r1, r2, r3}') + + def test_push(self): + self.cb.PUSH([reg.value for reg in r.caller_resp], cond=conditions.AL) + self.assert_equal('PUSH {r0, r1, r2, r3}') + + def test_push_raises_sp(self): + assert py.test.raises(AssertionError, 'self.cb.PUSH([r.sp.value])') + + def test_pop(self): + self.cb.POP([reg.value for reg in r.caller_resp], cond=conditions.AL) + self.assert_equal('POP {r0, r1, r2, r3}') + + def test_pop_eq_ldm(self): + # XXX check other conditions in LDM + self.cb.POP([reg.value for reg in r.caller_resp], cond=conditions.AL) + self.assert_equal('LDM SP!, {r0, r1, r2, r3}') + + def test_pop_raises_on_lr(self): + assert py.test.raises(AssertionError, 'self.cb.POP([r.lr.value])') + class TestInstrCodeBuilderForGeneratedInstr(ASMTest): def setup_method(self, ffuu_method): From fijal at codespeak.net Fri Nov 5 11:10:52 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 5 Nov 2010 11:10:52 +0100 (CET) Subject: [pypy-svn] r78720 - pypy/trunk/pypy/interpreter Message-ID: <20101105101052.2500C282B9E@codespeak.net> Author: fijal Date: Fri Nov 5 11:10:51 2010 New Revision: 78720 Modified: pypy/trunk/pypy/interpreter/executioncontext.py Log: comment about comment which is false Modified: pypy/trunk/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/trunk/pypy/interpreter/executioncontext.py (original) +++ pypy/trunk/pypy/interpreter/executioncontext.py Fri Nov 5 11:10:51 2010 @@ -19,6 +19,9 @@ # XXX self.w_tracefunc, self.profilefunc # XXX frame.is_being_profiled + # XXX [fijal] but they're not. is_being_profiled is guarded a bit all + # over the place as well as w_tracefunc + def __init__(self, space): self.space = space self.topframeref = jit.vref_None From fijal at codespeak.net Fri Nov 5 11:15:18 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 5 Nov 2010 11:15:18 +0100 (CET) Subject: [pypy-svn] r78721 - pypy/trunk/pypy/interpreter Message-ID: <20101105101518.A7933282B9E@codespeak.net> Author: fijal Date: Fri Nov 5 11:15:12 2010 New Revision: 78721 Modified: pypy/trunk/pypy/interpreter/executioncontext.py Log: make AbstractActionFlag a newstyle class Modified: pypy/trunk/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/trunk/pypy/interpreter/executioncontext.py (original) +++ pypy/trunk/pypy/interpreter/executioncontext.py Fri Nov 5 11:15:12 2010 @@ -320,7 +320,7 @@ " traceback and see where this one comes from :-)") -class AbstractActionFlag: +class AbstractActionFlag(object): """This holds the global 'action flag'. It is a single bitfield integer, with bits corresponding to AsyncAction objects that need to be immediately triggered. The correspondance from bits to From hakanardo at codespeak.net Fri Nov 5 11:49:06 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Fri, 5 Nov 2010 11:49:06 +0100 (CET) Subject: [pypy-svn] r78723 - in pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test Message-ID: <20101105104906.C53CF282BD4@codespeak.net> Author: hakanardo Date: Fri Nov 5 11:49:05 2010 New Revision: 78723 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Log: Keep track of boxes created in second iteration insted of trying to figure out all places where boxes were stored during the first. Fixes a problem with boxes proven to be consatant first after 2 iterations. Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Fri Nov 5 11:49:05 2010 @@ -40,15 +40,9 @@ assert len(loop_args) == len(jump_args) for i in range(len(loop_args)): argmap[loop_args[i]] = jump_args[i] - + for v in self.optimizer.values.values(): v.last_guard_index = -1 # FIXME: Are there any more indexes stored? - if not v.is_constant() and v.box: - v.fromstart = True # XXX do we really need to use an attribute? - - for op in self.optimizer.pure_operations.values(): - v = self.getvalue(op.result) - v.fromstart = True self.snapshot_map = {None: None} inputargs = [] @@ -83,29 +77,34 @@ #print 'P: ', str(newop) self.emit_operation(newop) - jmp = self.optimizer.newoperations[-1] + # Remove jump to make sure forced code are placed before it + newoperations = self.optimizer.newoperations + jmp = newoperations[-1] assert jmp.getopnum() == rop.JUMP + self.optimizer.newoperations = newoperations[:-1] + + boxes_created_this_iteration = {} jumpargs = jmp.getarglist() - for op in self.optimizer.newoperations: + + # FIXME: Should also loop over operations added by forcing things in this loop + for op in newoperations: #print 'E: ', str(op) + boxes_created_this_iteration[op.result] = True args = op.getarglist() if op.is_guard(): args = args + op.getfailargs() for a in args: - if not isinstance(a, Const) and a in self.optimizer.values: - v = self.getvalue(a) - if v.fromstart and not v.is_constant(): - b = v.force_box() - if b not in inputargs: - #boxes = [] - #v.enum_forced_boxes(boxes, seen_inputargs) - #for b in boxes: - inputargs.append(b) - newval = self.getvalue(argmap[b]) - jumpargs.append(newval.force_box()) + if not isinstance(a, Const) and not a in boxes_created_this_iteration: + if a not in inputargs: + inputargs.append(a) + b = self.getvalue(a).force_box() + if not isinstance(b, Const): + b = self.getvalue(argmap[b]).force_box() + jumpargs.append(b) jmp.initarglist(jumpargs) + self.optimizer.newoperations.append(jmp) return inputargs def inline_arg(self, arg): Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Fri Nov 5 11:49:05 2010 @@ -3932,6 +3932,69 @@ """ self.optimize_loop(ops, expected, preamble) + def test_value_proven_to_be_constant_after_two_iterations(self): + class FakeDescr(AbstractDescr): + def __init__(self, name): + self.name = name + def sort_key(self): + return id(self) + + + for n in ('inst_w_seq', 'inst_index', 'inst_w_list', 'inst_length', + 'inst_start', 'inst_step'): + self.namespace[n] = FakeDescr(n) + ops = """ + [p0, p1, p2, p3, i4, p5, i6, p7, p8, p9, p14] + guard_value(i4, 3) [] + guard_class(p9, 17278984) [] + guard_class(p9, 17278984) [] + p22 = getfield_gc(p9, descr=inst_w_seq) + guard_nonnull(p22) [] + i23 = getfield_gc(p9, descr=inst_index) + p24 = getfield_gc(p22, descr=inst_w_list) + guard_isnull(p24) [] + i25 = getfield_gc(p22, descr=inst_length) + i26 = int_ge(i23, i25) + guard_true(i26) [] + setfield_gc(p9, ConstPtr(myptr), descr=inst_w_seq) + + guard_nonnull(p14) [] + guard_class(p14, 17273920) [] + guard_class(p14, 17273920) [] + + p75 = new_with_vtable(17278984) + setfield_gc(p75, p14, descr=inst_w_seq) + setfield_gc(p75, 0, descr=inst_index) + guard_class(p75, 17278984) [] + guard_class(p75, 17278984) [] + p79 = getfield_gc(p75, descr=inst_w_seq) + guard_nonnull(p79) [] + i80 = getfield_gc(p75, descr=inst_index) + p81 = getfield_gc(p79, descr=inst_w_list) + guard_isnull(p81) [] + i82 = getfield_gc(p79, descr=inst_length) + i83 = int_ge(i80, i82) + guard_false(i83) [] + i84 = getfield_gc(p79, descr=inst_start) + i85 = getfield_gc(p79, descr=inst_step) + i86 = int_mul(i80, i85) + i87 = int_add(i84, i86) + i91 = int_add(i80, 1) + setfield_gc(p75, i91, descr=inst_index) + + p110 = same_as(ConstPtr(myptr)) + i112 = same_as(3) + i114 = same_as(39) + jump(p0, p1, p110, p3, i112, p5, i114, p7, p8, p75, p14) + """ + expected = """ + [p0, p1, p3, p5, p7, p8, p14, i82] + i115 = int_ge(1, i82) + guard_true(i115) [] + jump(p0, p1, p3, p5, p7, p8, p14, 1) + """ + self.optimize_loop(ops, expected) + # ---------- def optimize_strunicode_loop(self, ops, optops, preamble=None): if not preamble: From arigo at codespeak.net Fri Nov 5 12:41:15 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 5 Nov 2010 12:41:15 +0100 (CET) Subject: [pypy-svn] r78727 - pypy/trunk/pypy/module/pypyjit/test Message-ID: <20101105114115.D7E91282B9D@codespeak.net> Author: arigo Date: Fri Nov 5 12:41:14 2010 New Revision: 78727 Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Log: Most likely, this fixes failures introduced at r78701. Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Fri Nov 5 12:41:14 2010 @@ -93,6 +93,9 @@ # some support code... print >> f, py.code.Source(""" import sys + # we don't want to see the small bridges created + # by the checkinterval reaching the limit + sys.setcheckinterval(10000000) try: # make the file runnable by CPython import pypyjit pypyjit.set_param(threshold=%d) From afa at codespeak.net Fri Nov 5 12:47:45 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 12:47:45 +0100 (CET) Subject: [pypy-svn] r78728 - pypy/branch/fast-forward/lib-python/modified-2.7.0/test Message-ID: <20101105114745.9DD35282BEC@codespeak.net> Author: afa Date: Fri Nov 5 12:47:44 2010 New Revision: 78728 Added: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_generators.py - copied, changed from r78690, pypy/branch/fast-forward/lib-python/2.7.0/test/test_generators.py Log: A few gc.collect() to help generator finalization Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_generators.py (from r78690, pypy/branch/fast-forward/lib-python/2.7.0/test/test_generators.py) ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_generators.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_generators.py Fri Nov 5 12:47:44 2010 @@ -1496,6 +1496,10 @@ """ coroutine_tests = """\ +A helper function to call gc.collect() without printing +>>> import gc +>>> def gc_collect(): gc.collect() + Sending a value into a started generator: >>> def f(): @@ -1697,7 +1701,7 @@ >>> g = f() >>> g.next() ->>> del g +>>> del g; gc_collect() exiting >>> class context(object): @@ -1708,7 +1712,7 @@ ... yield >>> g = f() >>> g.next() ->>> del g +>>> del g; gc_collect() exiting @@ -1721,7 +1725,7 @@ >>> g = f() >>> g.next() ->>> del g +>>> del g; gc_collect() finally From afa at codespeak.net Fri Nov 5 13:00:31 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 13:00:31 +0100 (CET) Subject: [pypy-svn] r78729 - pypy/branch/fast-forward/lib-python/modified-2.7.0/test Message-ID: <20101105120031.51DD7282BF0@codespeak.net> Author: afa Date: Fri Nov 5 13:00:30 2010 New Revision: 78729 Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_generators.py Log: CPython is inconsistent in some exceptions: - some SyntaxError are displayed differently in doctests (in traceback.py) This was fixed in CPython 3.0 with r56853. - An error message in Python/ast.c "assignment to yield expression not possible" is not consistent with "can't assign to yield expression" generated in other cases. CPython should be fixed. - The error message in Object/genobject.c: "exceptions must be classes, or instances, not %s" is not consistent with the same one in Python/ceval.c: "exceptions must be old-style classes or derived from BaseException, not %s" CPython should be fixed. Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_generators.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_generators.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_generators.py Fri Nov 5 13:00:30 2010 @@ -733,14 +733,16 @@ ... yield 1 Traceback (most recent call last): .. -SyntaxError: 'return' with argument inside generator (, line 3) + File "", line 3 +SyntaxError: 'return' with argument inside generator >>> def f(): ... yield 1 ... return 22 Traceback (most recent call last): .. -SyntaxError: 'return' with argument inside generator (, line 3) + File "", line 3 +SyntaxError: 'return' with argument inside generator "return None" is not the same as "return" in a generator: @@ -749,7 +751,8 @@ ... return None Traceback (most recent call last): .. -SyntaxError: 'return' with argument inside generator (, line 3) + File "", line 3 +SyntaxError: 'return' with argument inside generator These are fine: @@ -878,7 +881,9 @@ ... if 0: ... yield 2 # because it's a generator (line 10) Traceback (most recent call last): -SyntaxError: 'return' with argument inside generator (, line 10) + ... + File "", line 10 +SyntaxError: 'return' with argument inside generator This one caused a crash (see SF bug 567538): @@ -1574,13 +1579,14 @@ >>> def f(): return lambda x=(yield): 1 Traceback (most recent call last): ... -SyntaxError: 'return' with argument inside generator (, line 1) + File "", line 1 +SyntaxError: 'return' with argument inside generator >>> def f(): x = yield = y Traceback (most recent call last): ... File "", line 1 -SyntaxError: assignment to yield expression not possible +SyntaxError: can't assign to yield expression >>> def f(): (yield bar) = y Traceback (most recent call last): @@ -1669,7 +1675,7 @@ >>> f().throw("abc") # throw on just-opened generator Traceback (most recent call last): ... -TypeError: exceptions must be classes, or instances, not str +TypeError: exceptions must be old-style classes or derived from BaseException, not str Now let's try closing a generator: From afa at codespeak.net Fri Nov 5 13:13:26 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 13:13:26 +0100 (CET) Subject: [pypy-svn] r78730 - in pypy/branch/fast-forward/pypy/interpreter: . test Message-ID: <20101105121326.A880B282BF5@codespeak.net> Author: afa Date: Fri Nov 5 13:13:21 2010 New Revision: 78730 Modified: pypy/branch/fast-forward/pypy/interpreter/generator.py pypy/branch/fast-forward/pypy/interpreter/test/test_generator.py pypy/branch/fast-forward/pypy/interpreter/typedef.py Log: Add gi_code and __name__ to generator objects Modified: pypy/branch/fast-forward/pypy/interpreter/generator.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/generator.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/generator.py Fri Nov 5 13:13:21 2010 @@ -101,7 +101,7 @@ return self.send_ex(space.w_None, operr) def descr_next(self): - """next() -> the next value, or raise StopIteration""" + """x.next() -> the next value, or raise StopIteration""" return self.send_ex(self.space.w_None) def descr_close(self): @@ -126,6 +126,13 @@ else: return space.w_None + def descr_gi_code(space, self): + return self.frame.pycode + + def descr__name__(space, self): + code_name = self.frame.pycode.co_name + return space.wrap(code_name) + def descr__del__(self): """ applevel __del__, which is called at a safe point after the Modified: pypy/branch/fast-forward/pypy/interpreter/test/test_generator.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/test/test_generator.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/test/test_generator.py Fri Nov 5 13:13:21 2010 @@ -5,6 +5,14 @@ yield 1 assert f().next() == 1 + def test_attributes(self): + def f(): + yield 1 + g = f() + assert g.gi_code is f.func_code + assert g.__name__ == 'f' + assert g.gi_frame is not None + def test_generator2(self): def f(): yield 1 Modified: pypy/branch/fast-forward/pypy/interpreter/typedef.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/typedef.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/typedef.py Fri Nov 5 13:13:21 2010 @@ -903,6 +903,8 @@ descrmismatch='__del__'), gi_running = interp_attrproperty('running', cls=GeneratorIterator), gi_frame = GetSetProperty(GeneratorIterator.descr_gi_frame), + gi_code = GetSetProperty(GeneratorIterator.descr_gi_code), + __name__ = GetSetProperty(GeneratorIterator.descr__name__), __weakref__ = make_weakref_descr(GeneratorIterator), ) GeneratorIterator.typedef.acceptable_as_base_class = False From arigo at codespeak.net Fri Nov 5 13:15:22 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 5 Nov 2010 13:15:22 +0100 (CET) Subject: [pypy-svn] r78731 - pypy/branch/signals Message-ID: <20101105121522.86F46282BF5@codespeak.net> Author: arigo Date: Fri Nov 5 13:15:21 2010 New Revision: 78731 Added: pypy/branch/signals/ - copied from r78730, pypy/trunk/ Log: A branch in which to simplify and make more robust the signal detection. (So far, in a pypy-c-jit, a Ctrl-C to interrupt an empty loop is ignored in about 1 case over 5) From arigo at codespeak.net Fri Nov 5 13:16:59 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 5 Nov 2010 13:16:59 +0100 (CET) Subject: [pypy-svn] r78732 - pypy/branch/signals/pypy/translator/c/src Message-ID: <20101105121659.D1DFC282BF7@codespeak.net> Author: arigo Date: Fri Nov 5 13:16:58 2010 New Revision: 78732 Modified: pypy/branch/signals/pypy/translator/c/src/signals.h Log: Starting from the end, this is the target signals.h. Modified: pypy/branch/signals/pypy/translator/c/src/signals.h ============================================================================== --- pypy/branch/signals/pypy/translator/c/src/signals.h (original) +++ pypy/branch/signals/pypy/translator/c/src/signals.h Fri Nov 5 13:16:58 2010 @@ -47,16 +47,12 @@ /* utility to poll for signals that arrived */ int pypysig_poll(void); /* => signum or -1 */ -/* When a signal is received, the bit 30 of pypysig_occurred is set. - After all signals are processed by pypysig_poll(), the bit 30 is - cleared again. The variable is exposed and RPython code is free to - use the other bits in any way. */ -#define PENDING_SIGNAL_BIT (1 << 30) +/* When a signal is received, pypysig_counter is set to -1. */ /* This is a struct for the JIT. See interp_signal.py. */ struct pypysig_long_struct { long value; }; -extern struct pypysig_long_struct pypysig_occurred; +extern struct pypysig_long_struct pypysig_counter; /* some C tricks to get/set the variable as efficiently as possible: use macros when compiling as a stand-alone program, but still @@ -64,18 +60,20 @@ #undef pypysig_getaddr_occurred void *pypysig_getaddr_occurred(void); #ifndef PYPY_NOT_MAIN_FILE -void *pypysig_getaddr_occurred(void) { return (void *)(&pypysig_occurred); } +void *pypysig_getaddr_occurred(void) { return (void *)(&pypysig_counter); } #endif -#define pypysig_getaddr_occurred() ((void *)(&pypysig_occurred)) +#define pypysig_getaddr_occurred() ((void *)(&pypysig_counter)) /************************************************************/ /* Implementation */ #ifndef PYPY_NOT_MAIN_FILE -struct pypysig_long_struct pypysig_occurred; -static volatile long *pypysig_occurred_v = (volatile long *)&pypysig_occurred.value; -static volatile int pypysig_flags[NSIG]; +struct pypysig_long_struct pypysig_counter = {0}; +static char volatile pypysig_flags[NSIG] = {0}; +static int volatile pypysig_occurred = 0; +/* pypysig_occurred is only an optimization: it tells if any + pypysig_flags could be set. */ void pypysig_ignore(int signum) { @@ -108,10 +106,11 @@ static void signal_setflag_handler(int signum) { if (0 <= signum && signum < NSIG) + { pypysig_flags[signum] = 1; - /* the point of "*pypysig_occurred_v" instead of just "pypysig_occurred" - is the volatile declaration */ - *pypysig_occurred_v |= PENDING_SIGNAL_BIT; + pypysig_occurred = 1; + pypysig_counter.value = -1; + } } void pypysig_setflag(int signum) @@ -130,27 +129,21 @@ int pypysig_poll(void) { - /* the two commented out lines below are useful for performance in - normal usage of pypysig_poll(); however, pypy/module/signal/ is - not normal usage. It only calls pypysig_poll() if the - PENDING_SIGNAL_BIT is set, and it clears that bit first. */ - -/* if (pypysig_occurred & PENDING_SIGNAL_BIT) */ + if (pypysig_occurred) { - int i; -/* pypysig_occurred &= ~PENDING_SIGNAL_BIT; */ - for (i=0; i Author: arigo Date: Fri Nov 5 13:19:00 2010 New Revision: 78733 Modified: pypy/branch/signals/pypy/interpreter/baseobjspace.py pypy/branch/signals/pypy/interpreter/executioncontext.py pypy/branch/signals/pypy/interpreter/test/test_executioncontext.py pypy/branch/signals/pypy/module/signal/__init__.py pypy/branch/signals/pypy/module/signal/interp_signal.py Log: In-progress: simplify the API for registering actions, and adapt interp_signal. Modified: pypy/branch/signals/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/signals/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/signals/pypy/interpreter/baseobjspace.py Fri Nov 5 13:19:00 2010 @@ -260,8 +260,6 @@ self.actionflag = ActionFlag() # changed by the signal module self.user_del_action = UserDelAction(self) self.frame_trace_action = FrameTraceAction(self) - self.actionflag.register_action(self.user_del_action) - self.actionflag.register_action(self.frame_trace_action) from pypy.interpreter.pycode import cpython_magic, default_magic self.our_magic = default_magic @@ -299,8 +297,6 @@ self.timer.start("startup " + modname) mod.init(self) self.timer.stop("startup " + modname) - # Force the tick counter to have a valid value - self.actionflag.force_tick_counter() def finish(self): self.wait_for_thread_shutdown() Modified: pypy/branch/signals/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/branch/signals/pypy/interpreter/executioncontext.py (original) +++ pypy/branch/signals/pypy/interpreter/executioncontext.py Fri Nov 5 13:19:00 2010 @@ -171,19 +171,14 @@ # this is split into a fast path and a slower path that is # not invoked every time bytecode_trace() is. actionflag = self.space.actionflag - ticker = actionflag.get() - if actionflag.has_bytecode_counter: # this "if" is constant-folded - ticker += 1 - actionflag.set(ticker) - if ticker & actionflag.interesting_bits: # fast check + if actionflag.decrement_ticker() < 0: actionflag.action_dispatcher(self, frame) # slow path bytecode_trace._always_inline_ = True def bytecode_trace_after_exception(self, frame): "Like bytecode_trace(), but without increasing the ticker." actionflag = self.space.actionflag - ticker = actionflag.get() - if ticker & actionflag.interesting_bits: # fast check + if actionflag.get_ticker() < 0: actionflag.action_dispatcher(self, frame) # slow path bytecode_trace_after_exception._always_inline_ = True @@ -321,118 +316,86 @@ class AbstractActionFlag(object): - """This holds the global 'action flag'. It is a single bitfield - integer, with bits corresponding to AsyncAction objects that need to - be immediately triggered. The correspondance from bits to - AsyncAction instances is built at translation time. We can quickly - check if there is anything at all to do by checking if any of the - relevant bits is set. If threads are enabled, they consume the 20 - lower bits to hold a counter incremented at each bytecode, to know - when to release the GIL. + """This holds in an integer the 'ticker'. If threads are enabled, + it is decremented at each bytecode; when it reaches zero, we release + the GIL. And whether we have threads or not, it is forced to zero + whenever we fire any of the asynchronous actions. """ def __init__(self): self._periodic_actions = [] self._nonperiodic_actions = [] - self.unused_bits = self.FREE_BITS[:] self.has_bytecode_counter = False - self.interesting_bits = 0 + self.fired_actions = None self._rebuild_action_dispatcher() def fire(self, action): - """Request for the action to be run before the next opcode. - The action must have been registered at space initalization time.""" - ticker = self.get() - self.set(ticker | action.bitmask) - - def register_action(self, action): - "NOT_RPYTHON" - assert isinstance(action, AsyncAction) - if action.bitmask == 0: - while True: - action.bitmask = self.unused_bits.pop(0) - if not (action.bitmask & self.interesting_bits): - break - self.interesting_bits |= action.bitmask - if action.bitmask & self.BYTECODE_COUNTER_OVERFLOW_BIT: - assert action.bitmask == self.BYTECODE_COUNTER_OVERFLOW_BIT - self._periodic_actions.append(action) + """Request for the action to be run before the next opcode.""" + if not action._fired: + action._fired = True + if self.fired_actions is None: + self.fired_actions = [] + self.fired_actions.append(action) + # set the ticker to -1 in order to force action_dispatcher() + # to run at the next possible bytecode + self.reset_ticker(-1) + + def register_periodic_action(self, action, use_bytecode_counter): + """NOT_RPYTHON: + Register the PeriodicAsyncAction action to be called whenever the + tick counter becomes smaller than 0. If 'use_bytecode_counter' is + True, make sure that we decrease the tick counter at every bytecode. + This is needed for threads. Note that 'use_bytecode_counter' can be + False for signal handling, because whenever the process receives a + signal, the tick counter is set to -1 by C code in signals.h. + """ + assert isinstance(action, PeriodicAsyncAction) + self._periodic_actions.append(action) + if use_bytecode_counter: self.has_bytecode_counter = True - self.force_tick_counter() - else: - self._nonperiodic_actions.append((action, action.bitmask)) self._rebuild_action_dispatcher() def setcheckinterval(self, space, interval): - if interval < self.CHECK_INTERVAL_MIN: - interval = self.CHECK_INTERVAL_MIN - elif interval > self.CHECK_INTERVAL_MAX: - interval = self.CHECK_INTERVAL_MAX + if interval < 1: + interval = 1 space.sys.checkinterval = interval - self.force_tick_counter() - - def force_tick_counter(self): - # force the tick counter to a valid value -- this actually forces - # it to reach BYTECODE_COUNTER_OVERFLOW_BIT at the next opcode. - ticker = self.get() - ticker &= ~ self.BYTECODE_COUNTER_OVERFLOW_BIT - ticker |= self.BYTECODE_COUNTER_MASK - self.set(ticker) def _rebuild_action_dispatcher(self): periodic_actions = unrolling_iterable(self._periodic_actions) - nonperiodic_actions = unrolling_iterable(self._nonperiodic_actions) - has_bytecode_counter = self.has_bytecode_counter @jit.dont_look_inside def action_dispatcher(ec, frame): - # periodic actions - if has_bytecode_counter: - ticker = self.get() - if ticker & self.BYTECODE_COUNTER_OVERFLOW_BIT: - # We must run the periodic actions now, but first - # reset the bytecode counter (the following line - # works by assuming that we just overflowed the - # counter, i.e. BYTECODE_COUNTER_OVERFLOW_BIT is - # set but none of the BYTECODE_COUNTER_MASK bits - # are). - ticker -= ec.space.sys.checkinterval - self.set(ticker) - for action in periodic_actions: - action.perform(ec, frame) + # periodic actions (first reset the bytecode counter) + self.reset_ticker(ec.space.sys.checkinterval) + for action in periodic_actions: + action.perform(ec, frame) # nonperiodic actions - for action, bitmask in nonperiodic_actions: - ticker = self.get() - if ticker & bitmask: - self.set(ticker & ~ bitmask) + list = self.fired_actions + if list is not None: + self.fired_actions = None + for action in list: + action._fired = False action.perform(ec, frame) action_dispatcher._dont_inline_ = True self.action_dispatcher = action_dispatcher - # Bits reserved for the bytecode counter, if used - BYTECODE_COUNTER_MASK = (1 << 20) - 1 - BYTECODE_COUNTER_OVERFLOW_BIT = (1 << 20) - - # Free bits - FREE_BITS = [1 << _b for _b in range(21, LONG_BIT-1)] - - # The acceptable range of values for sys.checkinterval, so that - # the bytecode_counter fits in 20 bits - CHECK_INTERVAL_MIN = 1 - CHECK_INTERVAL_MAX = BYTECODE_COUNTER_OVERFLOW_BIT - class ActionFlag(AbstractActionFlag): """The normal class for space.actionflag. The signal module provides a different one.""" - _flags = 0 + _ticker = 0 + + def get_ticker(self): + return self._ticker - def get(self): - return self._flags + def reset_ticker(self, value): + self._ticker = value - def set(self, value): - self._flags = value + def decrement_ticker(self): + if self.has_bytecode_counter: # this 'if' is constant-folded + self._ticker -= 1 + return self._ticker class AsyncAction(object): @@ -440,7 +403,7 @@ asynchronously with regular bytecode execution, but that still need to occur between two opcodes, not at a completely random time. """ - bitmask = 0 # means 'please choose one bit automatically' + _fired = False def __init__(self, space): self.space = space @@ -451,6 +414,7 @@ self.space.actionflag.fire(self) def fire_after_thread_switch(self): + XXX """Bit of a hack: fire() the action but only the next time the GIL is released and re-acquired (i.e. after a potential thread switch). Don't call this if threads are not enabled. @@ -466,7 +430,6 @@ """Abstract base class for actions that occur automatically every sys.checkinterval bytecodes. """ - bitmask = ActionFlag.BYTECODE_COUNTER_OVERFLOW_BIT class UserDelAction(AsyncAction): Modified: pypy/branch/signals/pypy/interpreter/test/test_executioncontext.py ============================================================================== --- pypy/branch/signals/pypy/interpreter/test/test_executioncontext.py (original) +++ pypy/branch/signals/pypy/interpreter/test/test_executioncontext.py Fri Nov 5 13:19:00 2010 @@ -19,7 +19,6 @@ space = self.space a1 = DemoAction(space) - space.actionflag.register_action(a1) for i in range(20): # assert does not raise: space.appexec([], """(): @@ -50,7 +49,7 @@ space = self.space a2 = DemoAction(space) - space.actionflag.register_action(a2) + space.actionflag.register_periodic_action(a2, True) try: for i in range(500): space.appexec([], """(): @@ -59,7 +58,7 @@ """) except Finished: pass - assert space.sys.checkinterval / 10 < i < space.sys.checkinterval * 3 + assert space.sys.checkinterval / 10 < i < space.sys.checkinterval * 1.1 def test_llprofile(self): l = [] Modified: pypy/branch/signals/pypy/module/signal/__init__.py ============================================================================== --- pypy/branch/signals/pypy/module/signal/__init__.py (original) +++ pypy/branch/signals/pypy/module/signal/__init__.py Fri Nov 5 13:19:00 2010 @@ -34,9 +34,7 @@ MixedModule.__init__(self, space, *args) # add the signal-checking callback as an action on the space space.check_signal_action = interp_signal.CheckSignalAction(space) - space.actionflag.register_action(space.check_signal_action) - # use the C-level pypysig_occurred variable as the action flag - # (the result is that the C-level signal handler will directly - # set the flag for the CheckSignalAction) + space.actionflag.register_periodic_action(space.check_signal_action, + use_bytecode_counter=False) space.actionflag.__class__ = interp_signal.SignalActionFlag # xxx yes I know the previous line is a hack Modified: pypy/branch/signals/pypy/module/signal/interp_signal.py ============================================================================== --- pypy/branch/signals/pypy/module/signal/interp_signal.py (original) +++ pypy/branch/signals/pypy/module/signal/interp_signal.py Fri Nov 5 13:19:00 2010 @@ -1,6 +1,7 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.baseobjspace import W_Root, ObjSpace from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag +from pypy.interpreter.executioncontext import PeriodicAsyncAction import signal as cpy_signal from pypy.rpython.lltypesystem import lltype, rffi from pypy.translator.tool.cbuild import ExternalCompilationInfo @@ -52,20 +53,28 @@ class SignalActionFlag(AbstractActionFlag): - def get(self): + # This class uses the C-level pypysig_counter variable as the tick + # counter. The C-level signal handler will reset it to -1 whenever + # a signal is received. + + def get_ticker(self): p = pypysig_getaddr_occurred() return p.c_value - def set(self, value): + + def reset_ticker(self, value): p = pypysig_getaddr_occurred() p.c_value = value + def decrement_ticker(self): + p = pypysig_getaddr_occurred() + if self.has_bytecode_counter: # this 'if' is constant-folded + p.c_value -= 1 + return p.c_value + -class CheckSignalAction(AsyncAction): +class CheckSignalAction(PeriodicAsyncAction): """An action that is automatically invoked when a signal is received.""" - # The C-level signal handler sets the bit 30 of pypysig_occurred: - bitmask = 1 << 30 - def __init__(self, space): AsyncAction.__init__(self, space) self.handlers_w = {} From arigo at codespeak.net Fri Nov 5 13:33:29 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 5 Nov 2010 13:33:29 +0100 (CET) Subject: [pypy-svn] r78734 - in pypy/branch/signals/pypy: interpreter module/signal module/thread module/thread/test Message-ID: <20101105123329.3F4FC282BF7@codespeak.net> Author: arigo Date: Fri Nov 5 13:33:26 2010 New Revision: 78734 Modified: pypy/branch/signals/pypy/interpreter/executioncontext.py pypy/branch/signals/pypy/module/signal/interp_signal.py pypy/branch/signals/pypy/module/thread/gil.py pypy/branch/signals/pypy/module/thread/test/test_gil.py Log: Fix the thread tests. Modified: pypy/branch/signals/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/branch/signals/pypy/interpreter/executioncontext.py (original) +++ pypy/branch/signals/pypy/interpreter/executioncontext.py Fri Nov 5 13:33:26 2010 @@ -414,13 +414,13 @@ self.space.actionflag.fire(self) def fire_after_thread_switch(self): - XXX """Bit of a hack: fire() the action but only the next time the GIL is released and re-acquired (i.e. after a potential thread switch). - Don't call this if threads are not enabled. + Don't call this if threads are not enabled. Currently limited to + one action (i.e. reserved for CheckSignalAction from module/signal). """ from pypy.module.thread.gil import spacestate - spacestate.set_actionflag_bit_after_thread_switch |= self.bitmask + spacestate.action_after_thread_switch = self def perform(self, executioncontext, frame): """To be overridden.""" Modified: pypy/branch/signals/pypy/module/signal/interp_signal.py ============================================================================== --- pypy/branch/signals/pypy/module/signal/interp_signal.py (original) +++ pypy/branch/signals/pypy/module/signal/interp_signal.py Fri Nov 5 13:33:26 2010 @@ -82,7 +82,6 @@ # need a helper action in case signals arrive in a non-main thread self.pending_signals = {} self.reissue_signal_action = ReissueSignalAction(space) - space.actionflag.register_action(self.reissue_signal_action) else: self.reissue_signal_action = None Modified: pypy/branch/signals/pypy/module/thread/gil.py ============================================================================== --- pypy/branch/signals/pypy/module/thread/gil.py (original) +++ pypy/branch/signals/pypy/module/thread/gil.py Fri Nov 5 13:33:26 2010 @@ -20,7 +20,8 @@ def initialize(self, space): # add the GIL-releasing callback as an action on the space - space.actionflag.register_action(GILReleaseAction(space)) + space.actionflag.register_periodic_action(GILReleaseAction(space), + use_bytecode_counter=True) def setup_threads(self, space): """Enable threads in the object space, if they haven't already been.""" @@ -44,7 +45,6 @@ # test_compile_lock. As a workaround, we repatch these global # fields systematically. spacestate.ll_GIL = self.ll_GIL - spacestate.actionflag = space.actionflag invoke_around_extcall(before_external_call, after_external_call) return result @@ -68,18 +68,17 @@ def _freeze_(self): self.ll_GIL = thread.null_ll_lock - self.actionflag = None - self.set_actionflag_bit_after_thread_switch = 0 + self.action_after_thread_switch = None + # ^^^ set by AsyncAction.fire_after_thread_switch() return False def after_thread_switch(self): # this is support logic for the signal module, to help it deliver # signals to the main thread. - actionflag = self.actionflag - if actionflag is not None: - flag = actionflag.get() - flag |= self.set_actionflag_bit_after_thread_switch - actionflag.set(flag) + action = self.action_after_thread_switch + if action is not None: + self.action_after_thread_switch = None + action.fire() spacestate = SpaceState() spacestate._freeze_() Modified: pypy/branch/signals/pypy/module/thread/test/test_gil.py ============================================================================== --- pypy/branch/signals/pypy/module/thread/test/test_gil.py (original) +++ pypy/branch/signals/pypy/module/thread/test/test_gil.py Fri Nov 5 13:33:26 2010 @@ -8,7 +8,7 @@ pass class FakeActionFlag(object): - def register_action(self, action): + def register_periodic_action(self, action, use_bytecode_counter): pass def get(self): return 0 From afa at codespeak.net Fri Nov 5 13:44:43 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 13:44:43 +0100 (CET) Subject: [pypy-svn] r78735 - pypy/branch/fast-forward/pypy/module/_io Message-ID: <20101105124443.758C7282B9D@codespeak.net> Author: afa Date: Fri Nov 5 13:44:42 2010 New Revision: 78735 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Log: Fix exceptions in _io.FileIO: - convert most OSErrors to IOErrors - check whether file is readable or writable Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Fri Nov 5 13:44:42 2010 @@ -191,7 +191,7 @@ try: os.lseek(self.fd, 0, os.SEEK_END) except OSError, e: - raise wrap_oserror(space, e) + raise wrap_oserror(space, e, exception_name='w_IOError') def _mode(self): if self.readable: @@ -214,6 +214,18 @@ if self.fd < 0: raise OperationError(space.w_ValueError, space.wrap(message)) + def _check_readable(self, space): + if not self.readable: + raise OperationError( + space.w_IOError, + space.wrap("file not open for reading")) + + def _check_writable(self, space): + if not self.writable: + raise OperationError( + space.w_IOError, + space.wrap("file not open for writing")) + def _close(self, space): if self.fd < 0: return @@ -305,7 +317,7 @@ try: res = os.isatty(self.fd) except OSError, e: - raise wrap_oserror(space, e) + raise wrap_oserror(space, e, exception_name='w_IOError') return space.wrap(res) @unwrap_spec('self', ObjSpace) @@ -328,7 +340,7 @@ @unwrap_spec('self', ObjSpace, W_Root) def write_w(self, space, w_data): self._check_closed(space) - # XXX self._check_writable(space) + self._check_writable(space) data = space.str_w(w_data) try: @@ -342,7 +354,7 @@ @unwrap_spec('self', ObjSpace, W_Root) def read_w(self, space, w_size=None): self._check_closed(space) - # XXX self._check_readable(space) + self._check_readable(space) size = convert_size(space, w_size) if size < 0: @@ -359,8 +371,7 @@ @unwrap_spec('self', ObjSpace, W_Root) def readinto_w(self, space, w_buffer): self._check_closed(space) - - # XXX check readable + self._check_readable(space) rwbuffer = space.rwbuffer_w(w_buffer) length = rwbuffer.getlength() try: @@ -374,6 +385,7 @@ @unwrap_spec('self', ObjSpace) def readall_w(self, space): self._check_closed(space) + self._check_readable(space) total = 0 builder = StringBuilder() @@ -407,13 +419,15 @@ @unwrap_spec('self', ObjSpace, W_Root) def truncate_w(self, space, w_size=None): + self._check_closed(space) + self._check_writable(space) if space.is_w(w_size, space.w_None): w_size = self.tell_w(space) try: self._truncate(space.r_longlong_w(w_size)) except OSError, e: - raise wrap_oserror(space, e) + raise wrap_oserror(space, e, exception_name='w_IOError') return w_size From arigo at codespeak.net Fri Nov 5 13:49:03 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 5 Nov 2010 13:49:03 +0100 (CET) Subject: [pypy-svn] r78736 - pypy/trunk/pypy/jit/codewriter Message-ID: <20101105124903.A4CAF282BF9@codespeak.net> Author: arigo Date: Fri Nov 5 13:49:02 2010 New Revision: 78736 Modified: pypy/trunk/pypy/jit/codewriter/call.py pypy/trunk/pypy/jit/codewriter/jtransform.py Log: Silence a warning. Modified: pypy/trunk/pypy/jit/codewriter/call.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/call.py (original) +++ pypy/trunk/pypy/jit/codewriter/call.py Fri Nov 5 13:49:02 2010 @@ -237,6 +237,8 @@ effectinfo) def _canraise(self, op): + if op.opname == 'pseudo_call_cannot_raise': + return False try: return self.raise_analyzer.can_raise(op) except lltype.DelayedPointer: Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/jtransform.py (original) +++ pypy/trunk/pypy/jit/codewriter/jtransform.py Fri Nov 5 13:49:02 2010 @@ -1098,7 +1098,7 @@ c_func, TP = support.builtin_func_for_spec(self.cpu.rtyper, oopspec_name, argtypes, resulttype) - op = SpaceOperation('pseudo_call', + op = SpaceOperation('pseudo_call_cannot_raise', [c_func] + [varoftype(T) for T in argtypes], varoftype(resulttype)) calldescr = self.callcontrol.getcalldescr(op, oopspecindex) From afa at codespeak.net Fri Nov 5 14:03:49 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 14:03:49 +0100 (CET) Subject: [pypy-svn] r78737 - in pypy/branch/fast-forward: lib-python/2.7.0/test pypy/module/_io Message-ID: <20101105130349.D78D9282BFC@codespeak.net> Author: afa Date: Fri Nov 5 14:03:48 2010 New Revision: 78737 Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_fileio.py pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Log: This should fix all remaining issues in test_fileio.py Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_fileio.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_fileio.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_fileio.py Fri Nov 5 14:03:48 2010 @@ -12,6 +12,7 @@ from test.test_support import TESTFN, check_warnings, run_unittest, make_bad_fd from test.test_support import py3k_bytes as bytes +from test.test_support import gc_collect from test.script_helper import run_python from _io import FileIO as _FileIO @@ -34,6 +35,7 @@ self.assertEquals(self.f.tell(), p.tell()) self.f.close() self.f = None + gc_collect() self.assertRaises(ReferenceError, getattr, p, 'tell') def testSeekTell(self): @@ -104,8 +106,8 @@ self.assertTrue(f.closed) def testMethods(self): - methods = ['fileno', 'isatty', 'read', 'readinto', - 'seek', 'tell', 'truncate', 'write', 'seekable', + methods = ['fileno', 'isatty', 'read', + 'tell', 'truncate', 'seekable', 'readable', 'writable'] if sys.platform.startswith('atheos'): methods.remove('truncate') @@ -117,6 +119,10 @@ method = getattr(self.f, methodname) # should raise on closed file self.assertRaises(ValueError, method) + # methods with one argument + self.assertRaises(ValueError, self.f.readinto, 0) + self.assertRaises(ValueError, self.f.write, 0) + self.assertRaises(ValueError, self.f.seek, 0) def testOpendir(self): # Issue 3703: opening a directory should fill the errno Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Fri Nov 5 14:03:48 2010 @@ -217,13 +217,13 @@ def _check_readable(self, space): if not self.readable: raise OperationError( - space.w_IOError, + space.w_ValueError, space.wrap("file not open for reading")) def _check_writable(self, space): if not self.writable: raise OperationError( - space.w_IOError, + space.w_ValueError, space.wrap("file not open for writing")) def _close(self, space): From arigo at codespeak.net Fri Nov 5 14:08:25 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 5 Nov 2010 14:08:25 +0100 (CET) Subject: [pypy-svn] r78738 - in pypy/branch/signals/pypy: interpreter module/signal/test Message-ID: <20101105130825.4B4FE282BD4@codespeak.net> Author: arigo Date: Fri Nov 5 14:08:23 2010 New Revision: 78738 Modified: pypy/branch/signals/pypy/interpreter/baseobjspace.py pypy/branch/signals/pypy/interpreter/executioncontext.py pypy/branch/signals/pypy/module/signal/test/test_signal.py Log: Implement space.getexecutioncontext().checksignals(). Modified: pypy/branch/signals/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/signals/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/signals/pypy/interpreter/baseobjspace.py Fri Nov 5 14:08:23 2010 @@ -258,6 +258,7 @@ self.interned_strings = {} self.actionflag = ActionFlag() # changed by the signal module + self.check_signal_action = None # changed by the signal module self.user_del_action = UserDelAction(self) self.frame_trace_action = FrameTraceAction(self) Modified: pypy/branch/signals/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/branch/signals/pypy/interpreter/executioncontext.py (original) +++ pypy/branch/signals/pypy/interpreter/executioncontext.py Fri Nov 5 14:08:23 2010 @@ -309,6 +309,13 @@ frame.last_exception = last_exception self.is_tracing -= 1 + def checksignals(self): + """Similar to PyErr_CheckSignals(). If called in the main thread, + and if signals are pending for the process, deliver them now + (i.e. call the signal handlers).""" + if self.space.check_signal_action is not None: + self.space.check_signal_action.perform(self, None) + def _freeze_(self): raise Exception("ExecutionContext instances should not be seen during" " translation. Now is a good time to inspect the" Modified: pypy/branch/signals/pypy/module/signal/test/test_signal.py ============================================================================== --- pypy/branch/signals/pypy/module/signal/test/test_signal.py (original) +++ pypy/branch/signals/pypy/module/signal/test/test_signal.py Fri Nov 5 14:08:23 2010 @@ -1,6 +1,38 @@ import os, py +import signal as cpy_signal from pypy.conftest import gettestobjspace + +class TestCheckSignals: + + def setup_class(cls): + if not hasattr(os, 'kill') or not hasattr(os, 'getpid'): + py.test.skip("requires os.kill() and os.getpid()") + cls.space = gettestobjspace(usemodules=['signal']) + + def test_checksignals(self): + space = self.space + w_received = space.appexec([], """(): + import signal + received = [] + def myhandler(signum, frame): + received.append(signum) + signal.signal(signal.SIGUSR1, myhandler) + return received""") + # + assert not space.is_true(w_received) + # + # send the signal now + os.kill(os.getpid(), cpy_signal.SIGUSR1) + # + # myhandler() should not be immediately called + assert not space.is_true(w_received) + # + # calling ec.checksignals() should call it + space.getexecutioncontext().checksignals() + assert space.is_true(w_received) + + class AppTestSignal: def setup_class(cls): @@ -24,18 +56,12 @@ signal.signal(signal.SIGUSR1, myhandler) posix.kill(posix.getpid(), signal.SIGUSR1) - for i in range(10000): - # wait a bit for the signal to be delivered to the handler - if received: - break + # the signal should be delivered to the handler immediately assert received == [signal.SIGUSR1] del received[:] posix.kill(posix.getpid(), signal.SIGUSR1) - for i in range(10000): - # wait a bit for the signal to be delivered to the handler - if received: - break + # the signal should be delivered to the handler immediately assert received == [signal.SIGUSR1] del received[:] From afa at codespeak.net Fri Nov 5 14:17:32 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 14:17:32 +0100 (CET) Subject: [pypy-svn] r78739 - in pypy/branch/fast-forward/lib-python: 2.7.0/test modified-2.7.0/test Message-ID: <20101105131732.1779E282BFD@codespeak.net> Author: afa Date: Fri Nov 5 14:17:31 2010 New Revision: 78739 Added: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_fileio.py - copied unchanged from r78737, pypy/branch/fast-forward/lib-python/2.7.0/test/test_fileio.py Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_fileio.py Log: Oops, don't modify the original test suite. Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_fileio.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_fileio.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_fileio.py Fri Nov 5 14:17:31 2010 @@ -12,7 +12,6 @@ from test.test_support import TESTFN, check_warnings, run_unittest, make_bad_fd from test.test_support import py3k_bytes as bytes -from test.test_support import gc_collect from test.script_helper import run_python from _io import FileIO as _FileIO @@ -35,7 +34,6 @@ self.assertEquals(self.f.tell(), p.tell()) self.f.close() self.f = None - gc_collect() self.assertRaises(ReferenceError, getattr, p, 'tell') def testSeekTell(self): @@ -106,8 +104,8 @@ self.assertTrue(f.closed) def testMethods(self): - methods = ['fileno', 'isatty', 'read', - 'tell', 'truncate', 'seekable', + methods = ['fileno', 'isatty', 'read', 'readinto', + 'seek', 'tell', 'truncate', 'write', 'seekable', 'readable', 'writable'] if sys.platform.startswith('atheos'): methods.remove('truncate') @@ -119,10 +117,6 @@ method = getattr(self.f, methodname) # should raise on closed file self.assertRaises(ValueError, method) - # methods with one argument - self.assertRaises(ValueError, self.f.readinto, 0) - self.assertRaises(ValueError, self.f.write, 0) - self.assertRaises(ValueError, self.f.seek, 0) def testOpendir(self): # Issue 3703: opening a directory should fill the errno From arigo at codespeak.net Fri Nov 5 14:28:19 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 5 Nov 2010 14:28:19 +0100 (CET) Subject: [pypy-svn] r78740 - in pypy/branch/signals/pypy: interpreter module/signal Message-ID: <20101105132819.5905F282BFD@codespeak.net> Author: arigo Date: Fri Nov 5 14:28:17 2010 New Revision: 78740 Modified: pypy/branch/signals/pypy/interpreter/executioncontext.py pypy/branch/signals/pypy/module/signal/interp_signal.py Log: No-ops, potentially reduce the JIT complexity of the code. Modified: pypy/branch/signals/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/branch/signals/pypy/interpreter/executioncontext.py (original) +++ pypy/branch/signals/pypy/interpreter/executioncontext.py Fri Nov 5 14:28:17 2010 @@ -400,9 +400,11 @@ self._ticker = value def decrement_ticker(self): + value = self._ticker if self.has_bytecode_counter: # this 'if' is constant-folded - self._ticker -= 1 - return self._ticker + value -= 1 + self._ticker = value + return value class AsyncAction(object): Modified: pypy/branch/signals/pypy/module/signal/interp_signal.py ============================================================================== --- pypy/branch/signals/pypy/module/signal/interp_signal.py (original) +++ pypy/branch/signals/pypy/module/signal/interp_signal.py Fri Nov 5 14:28:17 2010 @@ -67,9 +67,11 @@ def decrement_ticker(self): p = pypysig_getaddr_occurred() + value = p.c_value if self.has_bytecode_counter: # this 'if' is constant-folded - p.c_value -= 1 - return p.c_value + value -= 1 + p.c_value = value + return value class CheckSignalAction(PeriodicAsyncAction): From afa at codespeak.net Fri Nov 5 14:31:06 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 14:31:06 +0100 (CET) Subject: [pypy-svn] r78741 - pypy/branch/fast-forward/lib_pypy Message-ID: <20101105133106.D126A282BFD@codespeak.net> Author: afa Date: Fri Nov 5 14:31:05 2010 New Revision: 78741 Modified: pypy/branch/fast-forward/lib_pypy/dbm.py Log: Implement dbm.__contains__ to help tests that now call "key in db" instead of the outdated "db.has_key(key)" Modified: pypy/branch/fast-forward/lib_pypy/dbm.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/dbm.py (original) +++ pypy/branch/fast-forward/lib_pypy/dbm.py Fri Nov 5 14:31:05 2010 @@ -90,7 +90,7 @@ raise error("cannot add item to database") return default - def has_key(self, key): + def __contains__(self, key): if not self._aobj: raise error('DBM object has already been closed') dat = datum(key) @@ -98,6 +98,7 @@ if k.dptr: return True return False + has_key = __contains__ def __delitem__(self, key): if not self._aobj: From afa at codespeak.net Fri Nov 5 15:07:02 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 15:07:02 +0100 (CET) Subject: [pypy-svn] r78742 - in pypy/branch/fast-forward/pypy/module/sys: . test Message-ID: <20101105140702.4DB69282BFE@codespeak.net> Author: afa Date: Fri Nov 5 15:06:58 2010 New Revision: 78742 Modified: pypy/branch/fast-forward/pypy/module/sys/test/test_version.py pypy/branch/fast-forward/pypy/module/sys/version.py Log: Add compiler info in sys.version. This should help distutils on Windows. Modified: pypy/branch/fast-forward/pypy/module/sys/test/test_version.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/sys/test/test_version.py (original) +++ pypy/branch/fast-forward/pypy/module/sys/test/test_version.py Fri Nov 5 15:06:58 2010 @@ -3,3 +3,9 @@ def test_rev2int(): assert rev2int("71630") == 71630 assert rev2int("") == 0 + +class AppTestVersion: + def test_compiler(self): + import sys + assert ("MSC v." in sys.version or + "GCC " in sys.version) Modified: pypy/branch/fast-forward/pypy/module/sys/version.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/sys/version.py (original) +++ pypy/branch/fast-forward/pypy/module/sys/version.py Fri Nov 5 15:06:58 2010 @@ -2,7 +2,7 @@ Version numbers exposed by PyPy through the 'sys' module. """ import os - +from pypy.translator.platform import platform #XXX # the release serial 42 is not in range(16) CPYTHON_VERSION = (2, 7, 0, "final", 42) #XXX # sync patchlevel.h @@ -16,6 +16,11 @@ REV = """$LastChangedRevision$"""[22:-2] +if platform.name == 'msvc': + COMPILER_INFO = 'MSC v.%d 32 bit' % (platform.version * 10 + 600) +else: + COMPILER_INFO = "" + def rev2int(rev): try: return int(rev) @@ -48,7 +53,7 @@ return space.wrap(CPYTHON_VERSION) def get_version(space): - return space.wrap("%d.%d.%d (%d, %s, %s)\n[PyPy %d.%d.%d]" % ( + return space.wrap("%d.%d.%d (%d, %s, %s)\n[PyPy %d.%d.%d%s]" % ( CPYTHON_VERSION[0], CPYTHON_VERSION[1], CPYTHON_VERSION[2], @@ -57,7 +62,8 @@ time, PYPY_VERSION[0], PYPY_VERSION[1], - PYPY_VERSION[2])) + PYPY_VERSION[2], + compiler_version())) def get_winver(space): return space.wrap("%d.%d" % ( @@ -134,3 +140,8 @@ except (IOError, OSError): pass return rev + +def compiler_version(): + if not COMPILER_INFO: + return "" + return " with %s" % (COMPILER_INFO,) From fijal at codespeak.net Fri Nov 5 15:09:15 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 5 Nov 2010 15:09:15 +0100 (CET) Subject: [pypy-svn] r78743 - pypy/benchmarks/lib/waf/build Message-ID: <20101105140915.AA67C282C00@codespeak.net> Author: fijal Date: Fri Nov 5 15:09:14 2010 New Revision: 78743 Modified: pypy/benchmarks/lib/waf/build/wscript Log: make sure we don't call gcc Modified: pypy/benchmarks/lib/waf/build/wscript ============================================================================== --- pypy/benchmarks/lib/waf/build/wscript (original) +++ pypy/benchmarks/lib/waf/build/wscript Fri Nov 5 15:09:14 2010 @@ -17,3 +17,12 @@ target = 'lib_%d' % i, includes = '.', # include the top-level ) + + # disables the calls to gcc to measure the subprocess performance + from waflib import Task + def run_override(self): + return bld.exec_command('touch %s' % self.outputs[0].abspath()) + Task.classes['cxx'].run = run_override + Task.classes['cxxstlib'].run = run_override + + From hakanardo at codespeak.net Fri Nov 5 15:32:18 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Fri, 5 Nov 2010 15:32:18 +0100 (CET) Subject: [pypy-svn] r78744 - in pypy/branch/jit-unroll-loops: . lib-python/modified-2.5.2/test lib_pypy/pypy_test pypy pypy/doc pypy/interpreter pypy/interpreter/pyparser pypy/interpreter/pyparser/test pypy/jit/backend/x86 pypy/jit/backend/x86/test pypy/jit/codewriter pypy/jit/metainterp/optimizeopt pypy/jit/tool pypy/module/array/benchmark pypy/module/array/test pypy/module/pypyjit/test pypy/rlib pypy/rlib/test pypy/tool pypy/translator/c pypy/translator/c/src Message-ID: <20101105143218.EB678282BF5@codespeak.net> Author: hakanardo Date: Fri Nov 5 15:32:14 2010 New Revision: 78744 Modified: pypy/branch/jit-unroll-loops/ (props changed) pypy/branch/jit-unroll-loops/lib-python/modified-2.5.2/test/mapping_tests.py pypy/branch/jit-unroll-loops/lib_pypy/pypy_test/test_ctypes_support.py pypy/branch/jit-unroll-loops/pypy/ (props changed) pypy/branch/jit-unroll-loops/pypy/doc/getting-started.txt pypy/branch/jit-unroll-loops/pypy/interpreter/baseobjspace.py pypy/branch/jit-unroll-loops/pypy/interpreter/executioncontext.py pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/genpytokenize.py pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/pytokenize.py pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/test/test_pyparse.py pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/assembler.py pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_runner.py pypy/branch/jit-unroll-loops/pypy/jit/codewriter/call.py pypy/branch/jit-unroll-loops/pypy/jit/codewriter/jtransform.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (props changed) pypy/branch/jit-unroll-loops/pypy/jit/tool/pypytrace.vim pypy/branch/jit-unroll-loops/pypy/jit/tool/showstats.py pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/intimg.c (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/loop.c (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/sum.c (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/sumtst.c (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/sumtst.py (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/test/test_array_old.py (props changed) pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py pypy/branch/jit-unroll-loops/pypy/rlib/rerased.py (props changed) pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rerased.py (props changed) pypy/branch/jit-unroll-loops/pypy/tool/alarm.py pypy/branch/jit-unroll-loops/pypy/tool/readdictinfo.py pypy/branch/jit-unroll-loops/pypy/tool/rundictbenchmarks.py pypy/branch/jit-unroll-loops/pypy/tool/statistic_irc_log.py pypy/branch/jit-unroll-loops/pypy/tool/watchdog.py pypy/branch/jit-unroll-loops/pypy/tool/watchdog_nt.py pypy/branch/jit-unroll-loops/pypy/translator/c/genc.py pypy/branch/jit-unroll-loops/pypy/translator/c/src/g_include.h pypy/branch/jit-unroll-loops/pypy/translator/c/src/int.h Log: svn merge -r78530:HEAD svn+ssh://hakanardo at codespeak.net/svn/pypy/trunk Modified: pypy/branch/jit-unroll-loops/lib-python/modified-2.5.2/test/mapping_tests.py ============================================================================== --- pypy/branch/jit-unroll-loops/lib-python/modified-2.5.2/test/mapping_tests.py (original) +++ pypy/branch/jit-unroll-loops/lib-python/modified-2.5.2/test/mapping_tests.py Fri Nov 5 15:32:14 2010 @@ -1,6 +1,7 @@ # tests common to dict and UserDict import unittest import UserDict +from test import test_support class BasicTestMappingProtocol(unittest.TestCase): @@ -525,7 +526,8 @@ self.assertEqual(va, int(ka)) kb, vb = tb = b.popitem() self.assertEqual(vb, int(kb)) - self.assert_(not(copymode < 0 and ta != tb)) + if test_support.check_impl_detail(): + self.assert_(not(copymode < 0 and ta != tb)) self.assert_(not a) self.assert_(not b) Modified: pypy/branch/jit-unroll-loops/lib_pypy/pypy_test/test_ctypes_support.py ============================================================================== --- pypy/branch/jit-unroll-loops/lib_pypy/pypy_test/test_ctypes_support.py (original) +++ pypy/branch/jit-unroll-loops/lib_pypy/pypy_test/test_ctypes_support.py Fri Nov 5 15:32:14 2010 @@ -22,12 +22,11 @@ assert get_errno() == 0 def test_argument_conversion_and_checks(): - import ctypes - libc = ctypes.cdll.LoadLibrary("libc.so.6") - libc.strlen.argtypes = ctypes.c_char_p, - libc.strlen.restype = ctypes.c_size_t - assert libc.strlen("eggs") == 4 - + strlen = standard_c_lib.strlen + strlen.argtypes = [c_char_p] + strlen.restype = c_size_t + assert strlen("eggs") == 4 + # Should raise ArgumentError, not segfault - py.test.raises(ctypes.ArgumentError, libc.strlen, False) + py.test.raises(ArgumentError, strlen, False) Modified: pypy/branch/jit-unroll-loops/pypy/doc/getting-started.txt ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/doc/getting-started.txt (original) +++ pypy/branch/jit-unroll-loops/pypy/doc/getting-started.txt Fri Nov 5 15:32:14 2010 @@ -18,6 +18,7 @@ translation process - as opposed to encoding low level details into the language implementation itself. `more...`_ + .. _Python: http://docs.python.org/ref .. _`more...`: architecture.html Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/jit-unroll-loops/pypy/interpreter/baseobjspace.py Fri Nov 5 15:32:14 2010 @@ -299,6 +299,8 @@ self.timer.start("startup " + modname) mod.init(self) self.timer.stop("startup " + modname) + # Force the tick counter to have a valid value + self.actionflag.force_tick_counter() def finish(self): self.wait_for_thread_shutdown() Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/interpreter/executioncontext.py (original) +++ pypy/branch/jit-unroll-loops/pypy/interpreter/executioncontext.py Fri Nov 5 15:32:14 2010 @@ -19,6 +19,9 @@ # XXX self.w_tracefunc, self.profilefunc # XXX frame.is_being_profiled + # XXX [fijal] but they're not. is_being_profiled is guarded a bit all + # over the place as well as w_tracefunc + def __init__(self, space): self.space = space self.topframeref = jit.vref_None @@ -317,7 +320,7 @@ " traceback and see where this one comes from :-)") -class AbstractActionFlag: +class AbstractActionFlag(object): """This holds the global 'action flag'. It is a single bitfield integer, with bits corresponding to AsyncAction objects that need to be immediately triggered. The correspondance from bits to Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/genpytokenize.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/genpytokenize.py (original) +++ pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/genpytokenize.py Fri Nov 5 15:32:14 2010 @@ -17,12 +17,17 @@ def makePyPseudoDFA (): import string states = [] + def makeEOL(): + return group(states, + newArcPair(states, "\n"), + chain(states, + newArcPair(states, "\r"), + maybe(states, newArcPair(states, "\n")))) # ____________________________________________________________ def makeLineCont (): return chain(states, newArcPair(states, "\\"), - maybe(states, newArcPair(states, "\r")), - newArcPair(states, "\n")) + makeEOL()) # ____________________________________________________________ # Ignore stuff def makeWhitespace (): @@ -124,9 +129,7 @@ newArcPair(states, "~")) bracket = groupStr(states, "[](){}") special = group(states, - chain(states, - maybe(states, newArcPair(states, "\r")), - newArcPair(states, "\n")), + makeEOL(), groupStr(states, "@:;.,`")) funny = group(states, operator, bracket, special) # ____________________________________________________________ @@ -140,13 +143,13 @@ makeStrPrefix(), newArcPair(states, "'"), any(states, - notGroupStr(states, "\n'\\")), + notGroupStr(states, "\r\n'\\")), any(states, chain(states, newArcPair(states, "\\"), newArcPair(states, DEFAULT), any(states, - notGroupStr(states, "\n'\\")))), + notGroupStr(states, "\r\n'\\")))), group(states, newArcPair(states, "'"), makeLineCont())), @@ -154,13 +157,13 @@ makeStrPrefix(), newArcPair(states, '"'), any(states, - notGroupStr(states, '\n"\\')), + notGroupStr(states, '\r\n"\\')), any(states, chain(states, newArcPair(states, "\\"), newArcPair(states, DEFAULT), any(states, - notGroupStr(states, '\n"\\')))), + notGroupStr(states, '\r\n"\\')))), group(states, newArcPair(states, '"'), makeLineCont()))) Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/pytokenize.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/pytokenize.py (original) +++ pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/pytokenize.py Fri Nov 5 15:32:14 2010 @@ -24,7 +24,7 @@ # Automatically generated DFA's accepts = [True, True, True, True, True, True, True, True, - True, True, False, True, True, True, False, False, + True, True, False, True, True, True, True, False, False, False, True, True, True, False, True, False, True, False, True, False, False, True, False, False, False, False, True, False, False, @@ -142,9 +142,11 @@ # 14 {'\n': 13}, # 15 - {automata.DEFAULT: 28, '\n': 25, "'": 26, '\\': 27}, + {automata.DEFAULT: 28, '\n': 25, + '\r': 25, "'": 26, '\\': 27}, # 16 - {automata.DEFAULT: 31, '\n': 25, '"': 29, '\\': 30}, + {automata.DEFAULT: 31, '\n': 25, + '\r': 25, '"': 29, '\\': 30}, # 17 {'\n': 13, '\r': 14}, # 18 @@ -188,13 +190,15 @@ # 27 {automata.DEFAULT: 35, '\n': 13, '\r': 14}, # 28 - {automata.DEFAULT: 28, '\n': 25, "'": 13, '\\': 27}, + {automata.DEFAULT: 28, '\n': 25, + '\r': 25, "'": 13, '\\': 27}, # 29 {'"': 13}, # 30 {automata.DEFAULT: 36, '\n': 13, '\r': 14}, # 31 - {automata.DEFAULT: 31, '\n': 25, '"': 13, '\\': 30}, + {automata.DEFAULT: 31, '\n': 25, + '\r': 25, '"': 13, '\\': 30}, # 32 {'+': 37, '-': 37, '0': 38, '1': 38, '2': 38, '3': 38, '4': 38, '5': 38, @@ -208,9 +212,11 @@ '4': 34, '5': 34, '6': 34, '7': 34, '8': 34, '9': 34, 'J': 13, 'j': 13}, # 35 - {automata.DEFAULT: 35, '\n': 25, "'": 13, '\\': 27}, + {automata.DEFAULT: 35, '\n': 25, + '\r': 25, "'": 13, '\\': 27}, # 36 - {automata.DEFAULT: 36, '\n': 25, '"': 13, '\\': 30}, + {automata.DEFAULT: 36, '\n': 25, + '\r': 25, '"': 13, '\\': 30}, # 37 {'0': 38, '1': 38, '2': 38, '3': 38, '4': 38, '5': 38, '6': 38, '7': 38, @@ -282,7 +288,6 @@ ] doubleDFA = automata.DFA(states, accepts) - #_______________________________________________________________________ # End of automatically generated DFA's Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/test/test_pyparse.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/test/test_pyparse.py (original) +++ pypy/branch/jit-unroll-loops/pypy/interpreter/pyparser/test/test_pyparse.py Fri Nov 5 15:32:14 2010 @@ -92,6 +92,9 @@ exc = py.test.raises(IndentationError, parse, input).value assert exc.msg == "unindent does not match any outer indentation level" + def test_mac_newline(self): + self.parse("this_is\ra_mac\rfile") + def test_mode(self): assert self.parse("x = 43*54").type == syms.file_input tree = self.parse("43**54", "eval") Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/assembler.py Fri Nov 5 15:32:14 2010 @@ -712,9 +712,11 @@ dispatch_opnum = guard_opnum else: dispatch_opnum = op.getopnum() - res = genop_guard_list[dispatch_opnum](self, op, guard_op, guard_token, - arglocs, resloc) - faildescr._x86_adr_jump_offset = res + genop_guard_list[dispatch_opnum](self, op, guard_op, guard_token, + arglocs, resloc) + if not we_are_translated(): + # must be added by the genop_guard_list[]() + assert hasattr(faildescr, '_x86_adr_jump_offset') def regalloc_perform_guard(self, guard_op, faillocs, arglocs, resloc, current_depths): @@ -771,15 +773,15 @@ if isinstance(op.getarg(0), Const): self.mc.CMP(arglocs[1], arglocs[0]) if guard_opnum == rop.GUARD_FALSE: - return self.implement_guard(guard_token, rev_cond) + self.implement_guard(guard_token, rev_cond) else: - return self.implement_guard(guard_token, false_rev_cond) + self.implement_guard(guard_token, false_rev_cond) else: self.mc.CMP(arglocs[0], arglocs[1]) if guard_opnum == rop.GUARD_FALSE: - return self.implement_guard(guard_token, cond) + self.implement_guard(guard_token, cond) else: - return self.implement_guard(guard_token, false_cond) + self.implement_guard(guard_token, false_cond) return genop_cmp_guard def _cmpop_guard_float(cond, false_cond, need_jp): @@ -793,13 +795,14 @@ if guard_opnum == rop.GUARD_FALSE: if need_jp: self.mc.J_il8(rx86.Conditions['P'], 6) - return self.implement_guard(guard_token, cond) + self.implement_guard(guard_token, cond) else: if need_jp: self.mc.J_il8(rx86.Conditions['P'], 2) self.mc.J_il8(rx86.Conditions[cond], 5) - return self.implement_guard(guard_token) - return self.implement_guard(guard_token, false_cond) + self.implement_guard(guard_token) + else: + self.implement_guard(guard_token, false_cond) return genop_cmp_guard_float def _emit_call(self, x, arglocs, start=0, tmp=eax): @@ -961,11 +964,11 @@ self.mc.ensure_bytes_available(16 + guard_token.recovery_stub_size()) if guard_opnum == rop.GUARD_TRUE: self.mc.J_il8(rx86.Conditions['P'], 6) - return self.implement_guard(guard_token, 'E') + self.implement_guard(guard_token, 'E') else: self.mc.J_il8(rx86.Conditions['P'], 2) self.mc.J_il8(rx86.Conditions['E'], 5) - return self.implement_guard(guard_token) + self.implement_guard(guard_token) def genop_float_neg(self, op, arglocs, resloc): # Following what gcc does: res = x ^ 0x8000000000000000 @@ -985,9 +988,9 @@ guard_opnum = guard_op.getopnum() self.mc.CMP(arglocs[0], imm0) if guard_opnum == rop.GUARD_TRUE: - return self.implement_guard(guard_token, 'Z') + self.implement_guard(guard_token, 'Z') else: - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') def genop_int_is_true(self, op, arglocs, resloc): self.mc.CMP(arglocs[0], imm0) @@ -999,9 +1002,9 @@ guard_opnum = guard_op.getopnum() self.mc.CMP(arglocs[0], imm0) if guard_opnum == rop.GUARD_TRUE: - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') else: - return self.implement_guard(guard_token, 'Z') + self.implement_guard(guard_token, 'Z') def genop_int_is_zero(self, op, arglocs, resloc): self.mc.CMP(arglocs[0], imm0) @@ -1198,13 +1201,13 @@ def genop_guard_guard_true(self, ign_1, guard_op, guard_token, locs, ign_2): loc = locs[0] self.mc.TEST(loc, loc) - return self.implement_guard(guard_token, 'Z') + self.implement_guard(guard_token, 'Z') genop_guard_guard_nonnull = genop_guard_guard_true def genop_guard_guard_no_exception(self, ign_1, guard_op, guard_token, locs, ign_2): self.mc.CMP(heap(self.cpu.pos_exception()), imm0) - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') def genop_guard_guard_exception(self, ign_1, guard_op, guard_token, locs, resloc): @@ -1212,19 +1215,18 @@ loc1 = locs[1] self.mc.MOV(loc1, heap(self.cpu.pos_exception())) self.mc.CMP(loc1, loc) - addr = self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') if resloc is not None: self.mc.MOV(resloc, heap(self.cpu.pos_exc_value())) self.mc.MOV(heap(self.cpu.pos_exception()), imm0) self.mc.MOV(heap(self.cpu.pos_exc_value()), imm0) - return addr def _gen_guard_overflow(self, guard_op, guard_token): guard_opnum = guard_op.getopnum() if guard_opnum == rop.GUARD_NO_OVERFLOW: - return self.implement_guard(guard_token, 'O') + self.implement_guard(guard_token, 'O') elif guard_opnum == rop.GUARD_OVERFLOW: - return self.implement_guard(guard_token, 'NO') + self.implement_guard(guard_token, 'NO') else: not_implemented("int_xxx_ovf followed by %s" % guard_op.getopname()) @@ -1244,7 +1246,7 @@ def genop_guard_guard_false(self, ign_1, guard_op, guard_token, locs, ign_2): loc = locs[0] self.mc.TEST(loc, loc) - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') genop_guard_guard_isnull = genop_guard_guard_false def genop_guard_guard_value(self, ign_1, guard_op, guard_token, locs, ign_2): @@ -1253,7 +1255,7 @@ self.mc.UCOMISD(locs[0], locs[1]) else: self.mc.CMP(locs[0], locs[1]) - return self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') def _cmp_guard_class(self, locs): offset = self.cpu.vtable_offset @@ -1285,7 +1287,7 @@ def genop_guard_guard_class(self, ign_1, guard_op, guard_token, locs, ign_2): self.mc.ensure_bytes_available(256) self._cmp_guard_class(locs) - return self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') def genop_guard_guard_nonnull_class(self, ign_1, guard_op, guard_token, locs, ign_2): @@ -1300,7 +1302,7 @@ assert 0 < offset <= 127 self.mc.overwrite(jb_location-1, [chr(offset)]) # - return self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') def implement_guard_recovery(self, guard_opnum, faildescr, failargs, fail_locs): @@ -1627,13 +1629,15 @@ def implement_guard(self, guard_token, condition=None): self.mc.reserve_bytes(guard_token.recovery_stub_size()) self.pending_guard_tokens.append(guard_token) - # XXX: These jumps are patched later, the self.mc.tell() are just - # dummy values + # These jumps are patched later, the mc.tell() are just + # dummy values. Also, use self.mc._mc to avoid triggering a + # "buffer full" exactly here. + mc = self.mc._mc if condition: - self.mc.J_il(rx86.Conditions[condition], self.mc.tell()) + mc.J_il(rx86.Conditions[condition], mc.tell()) else: - self.mc.JMP_l(self.mc.tell()) - return self.mc.tell() - 4 + mc.JMP_l(mc.tell()) + guard_token.faildescr._x86_adr_jump_offset = mc.tell() - 4 def genop_call(self, op, arglocs, resloc): sizeloc = arglocs[0] @@ -1674,7 +1678,7 @@ self.mc.MOV_bi(FORCE_INDEX_OFS, fail_index) self.genop_call(op, arglocs, result_loc) self.mc.CMP_bi(FORCE_INDEX_OFS, 0) - return self.implement_guard(guard_token, 'L') + self.implement_guard(guard_token, 'L') def genop_guard_call_assembler(self, op, guard_op, guard_token, arglocs, result_loc): @@ -1761,7 +1765,7 @@ assert 0 <= offset <= 127 self.mc.overwrite(jmp_location - 1, [chr(offset)]) self.mc.CMP_bi(FORCE_INDEX_OFS, 0) - return self.implement_guard(guard_token, 'L') + self.implement_guard(guard_token, 'L') def genop_discard_cond_call_gc_wb(self, op, arglocs): # use 'mc._mc' directly instead of 'mc', to avoid Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_runner.py Fri Nov 5 15:32:14 2010 @@ -478,6 +478,10 @@ # whether the test segfaults. assert self.cpu.get_latest_value_int(0) == finished.value + def test_overflow_guard_exception(self): + for i in range(50): + self.test_exceptions() + class TestDebuggingAssembler(object): def setup_method(self, meth): Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/call.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/call.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/call.py Fri Nov 5 15:32:14 2010 @@ -237,6 +237,8 @@ effectinfo) def _canraise(self, op): + if op.opname == 'pseudo_call_cannot_raise': + return False try: return self.raise_analyzer.can_raise(op) except lltype.DelayedPointer: Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/jtransform.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/jtransform.py Fri Nov 5 15:32:14 2010 @@ -1098,7 +1098,7 @@ c_func, TP = support.builtin_func_for_spec(self.cpu.rtyper, oopspec_name, argtypes, resulttype) - op = SpaceOperation('pseudo_call', + op = SpaceOperation('pseudo_call_cannot_raise', [c_func] + [varoftype(T) for T in argtypes], varoftype(resulttype)) calldescr = self.callcontrol.getcalldescr(op, oopspecindex) Modified: pypy/branch/jit-unroll-loops/pypy/jit/tool/pypytrace.vim ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/tool/pypytrace.vim (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/tool/pypytrace.vim Fri Nov 5 15:32:14 2010 @@ -22,7 +22,7 @@ hi def link pypyLoopStart Structure "hi def link pypyLoopArgs PreProc hi def link pypyFailArgs String -hi def link pypyOpName Statement +"hi def link pypyOpName Statement hi def link pypyDebugMergePoint Comment hi def link pypyConstPtr Constant hi def link pypyNumber Number Modified: pypy/branch/jit-unroll-loops/pypy/jit/tool/showstats.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/tool/showstats.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/tool/showstats.py Fri Nov 5 15:32:14 2010 @@ -10,7 +10,8 @@ def main(argv): log = logparser.parse_log_file(argv[0]) - parts = logparser.extract_category(log, "jit-log-noopt-") + log_count_lines = open(argv[0] + '.count').readlines() + parts = logparser.extract_category(log, "jit-log-opt-") for i, oplist in enumerate(parts): loop = parse(oplist, no_namespace=True, nonstrict=True) num_ops = 0 @@ -27,6 +28,7 @@ print "Loop #%d, length: %d, opcodes: %d, guards: %d" % (i, num_ops, num_dmp, num_guards) else: print "Loop #%d, length: %d, opcodes: %d, guards: %d, %f" % (i, num_ops, num_dmp, num_guards, num_ops/num_dmp) - + print loop.comment, "run", log_count_lines[i].split(":")[1].strip(), "times" + if __name__ == '__main__': main(sys.argv[1:]) Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py Fri Nov 5 15:32:14 2010 @@ -93,6 +93,9 @@ # some support code... print >> f, py.code.Source(""" import sys + # we don't want to see the small bridges created + # by the checkinterval reaching the limit + sys.setcheckinterval(10000000) try: # make the file runnable by CPython import pypyjit pypyjit.set_param(threshold=%d) Modified: pypy/branch/jit-unroll-loops/pypy/tool/alarm.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/tool/alarm.py (original) +++ pypy/branch/jit-unroll-loops/pypy/tool/alarm.py Fri Nov 5 15:32:14 2010 @@ -38,8 +38,9 @@ sys.path.insert(0, os.path.dirname(sys.argv[0])) return sys.argv[0] -finished = [] -try: - execfile(_main_with_alarm(finished)) -finally: - finished.append(True) +if __name__ == '__main__': + finished = [] + try: + execfile(_main_with_alarm(finished)) + finally: + finished.append(True) Modified: pypy/branch/jit-unroll-loops/pypy/tool/readdictinfo.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/tool/readdictinfo.py (original) +++ pypy/branch/jit-unroll-loops/pypy/tool/readdictinfo.py Fri Nov 5 15:32:14 2010 @@ -7,37 +7,38 @@ import sys -infile = open(sys.argv[1]) +if __name__ == '__main__': + infile = open(sys.argv[1]) -curr = None -slots = [] -for line in infile: - if line == '------------------\n': - if curr: - break - curr = 1 - else: - attr, val = [s.strip() for s in line.split(':')] - slots.append(attr) + curr = None + slots = [] + for line in infile: + if line == '------------------\n': + if curr: + break + curr = 1 + else: + attr, val = [s.strip() for s in line.split(':')] + slots.append(attr) -class DictInfo(object): - __slots__ = slots + class DictInfo(object): + __slots__ = slots -infile = open(sys.argv[1]) + infile = open(sys.argv[1]) -infos = [] + infos = [] -for line in infile: - if line == '------------------\n': - curr = object.__new__(DictInfo) - infos.append(curr) - else: - attr, val = [s.strip() for s in line.split(':')] - if '.' in val: - val = float(val) + for line in infile: + if line == '------------------\n': + curr = object.__new__(DictInfo) + infos.append(curr) else: - val = int(val) - setattr(curr, attr, val) + attr, val = [s.strip() for s in line.split(':')] + if '.' in val: + val = float(val) + else: + val = int(val) + setattr(curr, attr, val) def histogram(infos, keyattr, *attrs): r = {} Modified: pypy/branch/jit-unroll-loops/pypy/tool/rundictbenchmarks.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/tool/rundictbenchmarks.py (original) +++ pypy/branch/jit-unroll-loops/pypy/tool/rundictbenchmarks.py Fri Nov 5 15:32:14 2010 @@ -7,20 +7,21 @@ # need to hack a copy of rst2html for yourself (svn docutils # required). -try: - os.unlink("dictinfo.txt") -except os.error: - pass +if __name__ == '__main__': + try: + os.unlink("dictinfo.txt") + except os.error: + pass -progs = [('pystone', ['-c', 'from test import pystone; pystone.main()']), - ('richards', ['richards.py']), - ('docutils', ['rst2html.py', '../../doc/coding-guide.txt', 'foo.html']), - ('translate', ['translate.py', '--backendopt', '--no-compile', '--batch', - 'targetrpystonedalone.py']) - ] + progs = [('pystone', ['-c', 'from test import pystone; pystone.main()']), + ('richards', ['richards.py']), + ('docutils', ['rst2html.py', '../../doc/coding-guide.txt', 'foo.html']), + ('translate', ['translate.py', '--backendopt', '--no-compile', '--batch', + 'targetrpystonedalone.py']) + ] -EXE = sys.argv[1] + EXE = sys.argv[1] -for suffix, args in progs: - os.spawnv(os.P_WAIT, EXE, [EXE] + args) - os.rename('dictinfo.txt', 'dictinfo-%s.txt'%suffix) + for suffix, args in progs: + os.spawnv(os.P_WAIT, EXE, [EXE] + args) + os.rename('dictinfo.txt', 'dictinfo-%s.txt'%suffix) Modified: pypy/branch/jit-unroll-loops/pypy/tool/statistic_irc_log.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/tool/statistic_irc_log.py (original) +++ pypy/branch/jit-unroll-loops/pypy/tool/statistic_irc_log.py Fri Nov 5 15:32:14 2010 @@ -2,53 +2,54 @@ from os import system, chdir from urllib import urlopen -log_URL = 'http://tismerysoft.de/pypy/irc-logs/' -archive_FILENAME = 'pypy.tar.gz' - -tempdir = py.test.ensuretemp("irc-log") - -# get compressed archive -chdir( str(tempdir)) -system('wget -q %s%s' % (log_URL, archive_FILENAME)) -system('tar xzf %s' % archive_FILENAME) -chdir('pypy') - -# get more recent daily logs -pypydir = tempdir.join('pypy') -for line in urlopen(log_URL + 'pypy/').readlines(): - i = line.find('%23pypy.log.') - if i == -1: - continue - filename = line[i:].split('"')[0] - system('wget -q %spypy/%s' % (log_URL, filename)) - -# rename to YYYYMMDD -for log_filename in pypydir.listdir('#pypy.log.*'): - rename_to = None - b = log_filename.basename - if '-' in b: - rename_to = log_filename.basename.replace('-', '') - elif len(b) == 19: - months= 'Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec'.split() - day = b[10:12] - month = months.index(b[12:15]) + 1 - year = b[15:20] - rename_to = '#pypy.log.%04s%02d%02s' % (year, month, day) - - if rename_to: - log_filename.rename(rename_to) - #print 'RENAMED', log_filename, 'TO', rename_to - -# print sorted list of filenames of daily logs -print 'irc://irc.freenode.org/pypy' -print 'date, messages, visitors' -for log_filename in pypydir.listdir('#pypy.log.*'): - n_messages, visitors = 0, {} - f = str(log_filename) - for s in file(f): - if '<' in s and '>' in s: - n_messages += 1 - elif ' joined #pypy' in s: - v = s.split()[1] - visitors[v] = True - print '%04s-%02s-%02s, %d, %d' % (f[-8:-4], f[-4:-2], f[-2:], n_messages, len(visitors.keys())) +if __name__ == '__main__': + log_URL = 'http://tismerysoft.de/pypy/irc-logs/' + archive_FILENAME = 'pypy.tar.gz' + + tempdir = py.test.ensuretemp("irc-log") + + # get compressed archive + chdir( str(tempdir)) + system('wget -q %s%s' % (log_URL, archive_FILENAME)) + system('tar xzf %s' % archive_FILENAME) + chdir('pypy') + + # get more recent daily logs + pypydir = tempdir.join('pypy') + for line in urlopen(log_URL + 'pypy/').readlines(): + i = line.find('%23pypy.log.') + if i == -1: + continue + filename = line[i:].split('"')[0] + system('wget -q %spypy/%s' % (log_URL, filename)) + + # rename to YYYYMMDD + for log_filename in pypydir.listdir('#pypy.log.*'): + rename_to = None + b = log_filename.basename + if '-' in b: + rename_to = log_filename.basename.replace('-', '') + elif len(b) == 19: + months= 'Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec'.split() + day = b[10:12] + month = months.index(b[12:15]) + 1 + year = b[15:20] + rename_to = '#pypy.log.%04s%02d%02s' % (year, month, day) + + if rename_to: + log_filename.rename(rename_to) + #print 'RENAMED', log_filename, 'TO', rename_to + + # print sorted list of filenames of daily logs + print 'irc://irc.freenode.org/pypy' + print 'date, messages, visitors' + for log_filename in pypydir.listdir('#pypy.log.*'): + n_messages, visitors = 0, {} + f = str(log_filename) + for s in file(f): + if '<' in s and '>' in s: + n_messages += 1 + elif ' joined #pypy' in s: + v = s.split()[1] + visitors[v] = True + print '%04s-%02s-%02s, %d, %d' % (f[-8:-4], f[-4:-2], f[-2:], n_messages, len(visitors.keys())) Modified: pypy/branch/jit-unroll-loops/pypy/tool/watchdog.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/tool/watchdog.py (original) +++ pypy/branch/jit-unroll-loops/pypy/tool/watchdog.py Fri Nov 5 15:32:14 2010 @@ -7,9 +7,6 @@ return name return 'signal %d' % (n,) -timeout = float(sys.argv[1]) -timedout = False - def childkill(): global timedout timedout = True @@ -20,31 +17,35 @@ except OSError: pass -pid = os.fork() -if pid == 0: - os.execvp(sys.argv[2], sys.argv[2:]) -else: # parent - t = threading.Timer(timeout, childkill) - t.start() - while True: - try: - pid, status = os.waitpid(pid, 0) - except KeyboardInterrupt: - continue +if __name__ == '__main__': + timeout = float(sys.argv[1]) + timedout = False + + pid = os.fork() + if pid == 0: + os.execvp(sys.argv[2], sys.argv[2:]) + else: # parent + t = threading.Timer(timeout, childkill) + t.start() + while True: + try: + pid, status = os.waitpid(pid, 0) + except KeyboardInterrupt: + continue + else: + t.cancel() + break + if os.WIFEXITED(status): + sys.exit(os.WEXITSTATUS(status)) else: - t.cancel() - break - if os.WIFEXITED(status): - sys.exit(os.WEXITSTATUS(status)) - else: - assert os.WIFSIGNALED(status) - sign = os.WTERMSIG(status) - if timedout and sign == signal.SIGTERM: + assert os.WIFSIGNALED(status) + sign = os.WTERMSIG(status) + if timedout and sign == signal.SIGTERM: + sys.exit(1) + signame = getsignalname(sign) + sys.stderr.write("="*26 + "timedout" + "="*26 + "\n") + sys.stderr.write("="*25 + " %-08s " % signame + "="*25 + "\n") sys.exit(1) - signame = getsignalname(sign) - sys.stderr.write("="*26 + "timedout" + "="*26 + "\n") - sys.stderr.write("="*25 + " %-08s " % signame + "="*25 + "\n") - sys.exit(1) - - + + Modified: pypy/branch/jit-unroll-loops/pypy/tool/watchdog_nt.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/tool/watchdog_nt.py (original) +++ pypy/branch/jit-unroll-loops/pypy/tool/watchdog_nt.py Fri Nov 5 15:32:14 2010 @@ -2,11 +2,6 @@ import threading import ctypes -PROCESS_TERMINATE = 0x1 - -timeout = float(sys.argv[1]) -timedout = False - def childkill(pid): global timedout timedout = True @@ -14,19 +9,25 @@ sys.stderr.write("="*26 + "timedout" + "="*26 + "\n") ctypes.windll.kernel32.TerminateProcess(pid, 1) -pid = os.spawnv(os.P_NOWAIT, sys.argv[2], sys.argv[2:]) +if __name__ == '__main__': + PROCESS_TERMINATE = 0x1 + + timeout = float(sys.argv[1]) + timedout = False + + pid = os.spawnv(os.P_NOWAIT, sys.argv[2], sys.argv[2:]) + + t = threading.Timer(timeout, childkill, (pid,)) + t.start() + while True: + try: + pid, status = os.waitpid(pid, 0) + except KeyboardInterrupt: + continue + else: + t.cancel() + break + + #print 'status ', status >> 8 + sys.exit(status >> 8) -t = threading.Timer(timeout, childkill, (pid,)) -t.start() -while True: - try: - pid, status = os.waitpid(pid, 0) - except KeyboardInterrupt: - continue - else: - t.cancel() - break - -#print 'status ', status >> 8 -sys.exit(status >> 8) - Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/genc.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/translator/c/genc.py (original) +++ pypy/branch/jit-unroll-loops/pypy/translator/c/genc.py Fri Nov 5 15:32:14 2010 @@ -626,6 +626,7 @@ python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py -m$(PYPY_MAIN_FUNCTION) -t $< > $*.gcmap') mk.rule('gcmaptable.s', '$(GCMAPFILES)', python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py $(GCMAPFILES) > $@') + mk.rule('.PRECIOUS', '%.s', "# don't remove .s files if Ctrl-C'ed") else: mk.definition('DEBUGFLAGS', '-O1 -g') Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/src/g_include.h ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/translator/c/src/g_include.h (original) +++ pypy/branch/jit-unroll-loops/pypy/translator/c/src/g_include.h Fri Nov 5 15:32:14 2010 @@ -39,9 +39,10 @@ #include "src/instrument.h" /* optional assembler bits */ -#if defined(__GNUC__) && defined(__i386__) -# include "src/asm_gcc_x86.h" -#endif +// disabled: does not give any speed-up +//#if defined(__GNUC__) && defined(__i386__) +//# include "src/asm_gcc_x86.h" +//#endif #if defined(__GNUC__) && defined(__ppc__) # include "src/asm_ppc.h" Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/src/int.h ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/translator/c/src/int.h (original) +++ pypy/branch/jit-unroll-loops/pypy/translator/c/src/int.h Fri Nov 5 15:32:14 2010 @@ -2,40 +2,27 @@ /************************************************************/ /*** C header subsection: operations between ints ***/ -#ifndef LLONG_MAX -# if SIZEOF_LONG_LONG == 8 -# define LLONG_MAX 0X7FFFFFFFFFFFFFFFLL -# else -# error "fix LLONG_MAX" -# endif -#endif - -#ifndef LLONG_MIN -# define LLONG_MIN (-LLONG_MAX-1) -#endif /*** unary operations ***/ -#define OP_INT_IS_TRUE(x,r) OP_INT_NE(x,0,r) - -#define OP_INT_INVERT(x,r) r = ~((x)) - -#define OP_INT_NEG(x,r) r = -(x) +#define OP_INT_IS_TRUE(x,r) r = ((x) != 0) +#define OP_INT_INVERT(x,r) r = ~(x) +#define OP_INT_NEG(x,r) r = -(x) #define OP_INT_NEG_OVF(x,r) \ - if ((x) == LONG_MIN) FAIL_OVF("integer negate"); \ + if ((x) == LONG_MIN) FAIL_OVF("integer negate"); \ OP_INT_NEG(x,r) #define OP_LLONG_NEG_OVF(x,r) \ - if ((x) == LLONG_MIN) FAIL_OVF("integer negate"); \ + if ((x) == LLONG_MIN) FAIL_OVF("integer negate"); \ OP_LLONG_NEG(x,r) #define OP_INT_ABS(x,r) r = (x) >= 0 ? x : -(x) #define OP_INT_ABS_OVF(x,r) \ - if ((x) == LONG_MIN) FAIL_OVF("integer absolute"); \ + if ((x) == LONG_MIN) FAIL_OVF("integer absolute"); \ OP_INT_ABS(x,r) #define OP_LLONG_ABS_OVF(x,r) \ - if ((x) == LLONG_MIN) FAIL_OVF("integer absolute"); \ + if ((x) == LLONG_MIN) FAIL_OVF("integer absolute"); \ OP_LLONG_ABS(x,r) /*** binary operations ***/ @@ -59,50 +46,46 @@ #define OP_INT_ADD(x,y,r) r = (x) + (y) +/* cast to avoid undefined behaviour on overflow */ #define OP_INT_ADD_OVF(x,y,r) \ - OP_INT_ADD(x,y,r); \ - if ((r^(x)) >= 0 || (r^(y)) >= 0); \ - else FAIL_OVF("integer addition") + r = (long)((unsigned long)x + y); \ + if ((r^x) < 0 && (r^y) < 0) FAIL_OVF("integer addition") + +#define OP_LLONG_ADD_OVF(x,y,r) \ + r = (long long)((unsigned long long)x + y); \ + if ((r^x) < 0 && (r^y) < 0) FAIL_OVF("integer addition") #define OP_INT_ADD_NONNEG_OVF(x,y,r) /* y can be assumed >= 0 */ \ - r = (long)((unsigned long)x + (unsigned long)y); \ - if (r >= (x)); \ - else FAIL_OVF("integer addition") -/* Can a C compiler be too clever and think it can "prove" that - * r >= x always holds above? Yes. Hence the casting. */ + r = (long)((unsigned long)x + y); \ + if ((r&~x) < 0) FAIL_OVF("integer addition") #define OP_INT_SUB(x,y,r) r = (x) - (y) #define OP_INT_SUB_OVF(x,y,r) \ - OP_INT_SUB(x,y,r); \ - if ((r^(x)) >= 0 || (r^~(y)) >= 0); \ - else FAIL_OVF("integer subtraction") - -#define OP_INT_MUL(x,y,r) r = (x) * (y) - -#if defined(HAVE_LONG_LONG) && SIZE_OF_LONG_LONG < SIZE_OF_LONG -# define OP_INT_MUL_OVF_LL 1 -#lse -# define OP_INT_MUL_OVF_LL 0 -#endif + r = (long)((unsigned long)x - y); \ + if ((r^x) < 0 && (r^~y) < 0) FAIL_OVF("integer subtraction") -#if !OP_INT_MUL_OVF_LL +#define OP_LLONG_SUB_OVF(x,y,r) \ + r = (long long)((unsigned long long)x - y); \ + if ((r^x) < 0 && (r^~y) < 0) FAIL_OVF("integer subtraction") -#define OP_INT_MUL_OVF(x,y,r) \ - if (op_int_mul_ovf(x,y,&r)); \ - else FAIL_OVF("integer multiplication") - -#else +#define OP_INT_MUL(x,y,r) r = (x) * (y) +#if SIZEOF_LONG * 2 <= SIZEOF_LONG_LONG #define OP_INT_MUL_OVF(x,y,r) \ { \ - PY_LONG_LONG lr = (PY_LONG_LONG)(x) * (PY_LONG_LONG)(y); \ - r = (long)lr; \ - if ((PY_LONG_LONG)r == lr); \ - else FAIL_OVF("integer multiplication"); \ + long long _lr = (long long)x * y; \ + r = (long)_lr; \ + if (_lr != (long long)r) FAIL_OVF("integer multiplication"); \ } +#else +#define OP_INT_MUL_OVF(x,y,r) \ + r = op_llong_mul_ovf(x, y) /* long == long long */ #endif +#define OP_LLONG_MUL_OVF(x,y,r) \ + r = op_llong_mul_ovf(x, y) + /* shifting */ /* NB. shifting has same limitations as C: the shift count must be @@ -121,6 +104,10 @@ OP_INT_LSHIFT(x,y,r); \ if ((x) != Py_ARITHMETIC_RIGHT_SHIFT(long, r, (y))) \ FAIL_OVF("x< Author: arigo Date: Fri Nov 5 15:40:08 2010 New Revision: 78745 Modified: pypy/trunk/pypy/interpreter/baseobjspace.py pypy/trunk/pypy/interpreter/executioncontext.py pypy/trunk/pypy/interpreter/test/test_executioncontext.py pypy/trunk/pypy/module/signal/__init__.py pypy/trunk/pypy/module/signal/interp_signal.py pypy/trunk/pypy/module/signal/test/test_signal.py pypy/trunk/pypy/module/thread/gil.py pypy/trunk/pypy/module/thread/test/test_gil.py pypy/trunk/pypy/translator/c/src/signals.h Log: Merge branch/signals. Minor simplification, and fix for the issue of ignored signals (e.g. Ctrl-C) that occurs about 20% of the time in running "while 1: pass" on pypy-c-jit. Modified: pypy/trunk/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/trunk/pypy/interpreter/baseobjspace.py (original) +++ pypy/trunk/pypy/interpreter/baseobjspace.py Fri Nov 5 15:40:08 2010 @@ -258,10 +258,9 @@ self.interned_strings = {} self.actionflag = ActionFlag() # changed by the signal module + self.check_signal_action = None # changed by the signal module self.user_del_action = UserDelAction(self) self.frame_trace_action = FrameTraceAction(self) - self.actionflag.register_action(self.user_del_action) - self.actionflag.register_action(self.frame_trace_action) from pypy.interpreter.pycode import cpython_magic, default_magic self.our_magic = default_magic @@ -299,8 +298,6 @@ self.timer.start("startup " + modname) mod.init(self) self.timer.stop("startup " + modname) - # Force the tick counter to have a valid value - self.actionflag.force_tick_counter() def finish(self): self.wait_for_thread_shutdown() Modified: pypy/trunk/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/trunk/pypy/interpreter/executioncontext.py (original) +++ pypy/trunk/pypy/interpreter/executioncontext.py Fri Nov 5 15:40:08 2010 @@ -171,19 +171,14 @@ # this is split into a fast path and a slower path that is # not invoked every time bytecode_trace() is. actionflag = self.space.actionflag - ticker = actionflag.get() - if actionflag.has_bytecode_counter: # this "if" is constant-folded - ticker += 1 - actionflag.set(ticker) - if ticker & actionflag.interesting_bits: # fast check + if actionflag.decrement_ticker() < 0: actionflag.action_dispatcher(self, frame) # slow path bytecode_trace._always_inline_ = True def bytecode_trace_after_exception(self, frame): "Like bytecode_trace(), but without increasing the ticker." actionflag = self.space.actionflag - ticker = actionflag.get() - if ticker & actionflag.interesting_bits: # fast check + if actionflag.get_ticker() < 0: actionflag.action_dispatcher(self, frame) # slow path bytecode_trace_after_exception._always_inline_ = True @@ -314,6 +309,13 @@ frame.last_exception = last_exception self.is_tracing -= 1 + def checksignals(self): + """Similar to PyErr_CheckSignals(). If called in the main thread, + and if signals are pending for the process, deliver them now + (i.e. call the signal handlers).""" + if self.space.check_signal_action is not None: + self.space.check_signal_action.perform(self, None) + def _freeze_(self): raise Exception("ExecutionContext instances should not be seen during" " translation. Now is a good time to inspect the" @@ -321,118 +323,88 @@ class AbstractActionFlag(object): - """This holds the global 'action flag'. It is a single bitfield - integer, with bits corresponding to AsyncAction objects that need to - be immediately triggered. The correspondance from bits to - AsyncAction instances is built at translation time. We can quickly - check if there is anything at all to do by checking if any of the - relevant bits is set. If threads are enabled, they consume the 20 - lower bits to hold a counter incremented at each bytecode, to know - when to release the GIL. + """This holds in an integer the 'ticker'. If threads are enabled, + it is decremented at each bytecode; when it reaches zero, we release + the GIL. And whether we have threads or not, it is forced to zero + whenever we fire any of the asynchronous actions. """ def __init__(self): self._periodic_actions = [] self._nonperiodic_actions = [] - self.unused_bits = self.FREE_BITS[:] self.has_bytecode_counter = False - self.interesting_bits = 0 + self.fired_actions = None self._rebuild_action_dispatcher() def fire(self, action): - """Request for the action to be run before the next opcode. - The action must have been registered at space initalization time.""" - ticker = self.get() - self.set(ticker | action.bitmask) - - def register_action(self, action): - "NOT_RPYTHON" - assert isinstance(action, AsyncAction) - if action.bitmask == 0: - while True: - action.bitmask = self.unused_bits.pop(0) - if not (action.bitmask & self.interesting_bits): - break - self.interesting_bits |= action.bitmask - if action.bitmask & self.BYTECODE_COUNTER_OVERFLOW_BIT: - assert action.bitmask == self.BYTECODE_COUNTER_OVERFLOW_BIT - self._periodic_actions.append(action) + """Request for the action to be run before the next opcode.""" + if not action._fired: + action._fired = True + if self.fired_actions is None: + self.fired_actions = [] + self.fired_actions.append(action) + # set the ticker to -1 in order to force action_dispatcher() + # to run at the next possible bytecode + self.reset_ticker(-1) + + def register_periodic_action(self, action, use_bytecode_counter): + """NOT_RPYTHON: + Register the PeriodicAsyncAction action to be called whenever the + tick counter becomes smaller than 0. If 'use_bytecode_counter' is + True, make sure that we decrease the tick counter at every bytecode. + This is needed for threads. Note that 'use_bytecode_counter' can be + False for signal handling, because whenever the process receives a + signal, the tick counter is set to -1 by C code in signals.h. + """ + assert isinstance(action, PeriodicAsyncAction) + self._periodic_actions.append(action) + if use_bytecode_counter: self.has_bytecode_counter = True - self.force_tick_counter() - else: - self._nonperiodic_actions.append((action, action.bitmask)) self._rebuild_action_dispatcher() def setcheckinterval(self, space, interval): - if interval < self.CHECK_INTERVAL_MIN: - interval = self.CHECK_INTERVAL_MIN - elif interval > self.CHECK_INTERVAL_MAX: - interval = self.CHECK_INTERVAL_MAX + if interval < 1: + interval = 1 space.sys.checkinterval = interval - self.force_tick_counter() - - def force_tick_counter(self): - # force the tick counter to a valid value -- this actually forces - # it to reach BYTECODE_COUNTER_OVERFLOW_BIT at the next opcode. - ticker = self.get() - ticker &= ~ self.BYTECODE_COUNTER_OVERFLOW_BIT - ticker |= self.BYTECODE_COUNTER_MASK - self.set(ticker) def _rebuild_action_dispatcher(self): periodic_actions = unrolling_iterable(self._periodic_actions) - nonperiodic_actions = unrolling_iterable(self._nonperiodic_actions) - has_bytecode_counter = self.has_bytecode_counter @jit.dont_look_inside def action_dispatcher(ec, frame): - # periodic actions - if has_bytecode_counter: - ticker = self.get() - if ticker & self.BYTECODE_COUNTER_OVERFLOW_BIT: - # We must run the periodic actions now, but first - # reset the bytecode counter (the following line - # works by assuming that we just overflowed the - # counter, i.e. BYTECODE_COUNTER_OVERFLOW_BIT is - # set but none of the BYTECODE_COUNTER_MASK bits - # are). - ticker -= ec.space.sys.checkinterval - self.set(ticker) - for action in periodic_actions: - action.perform(ec, frame) + # periodic actions (first reset the bytecode counter) + self.reset_ticker(ec.space.sys.checkinterval) + for action in periodic_actions: + action.perform(ec, frame) # nonperiodic actions - for action, bitmask in nonperiodic_actions: - ticker = self.get() - if ticker & bitmask: - self.set(ticker & ~ bitmask) + list = self.fired_actions + if list is not None: + self.fired_actions = None + for action in list: + action._fired = False action.perform(ec, frame) action_dispatcher._dont_inline_ = True self.action_dispatcher = action_dispatcher - # Bits reserved for the bytecode counter, if used - BYTECODE_COUNTER_MASK = (1 << 20) - 1 - BYTECODE_COUNTER_OVERFLOW_BIT = (1 << 20) - - # Free bits - FREE_BITS = [1 << _b for _b in range(21, LONG_BIT-1)] - - # The acceptable range of values for sys.checkinterval, so that - # the bytecode_counter fits in 20 bits - CHECK_INTERVAL_MIN = 1 - CHECK_INTERVAL_MAX = BYTECODE_COUNTER_OVERFLOW_BIT - class ActionFlag(AbstractActionFlag): """The normal class for space.actionflag. The signal module provides a different one.""" - _flags = 0 + _ticker = 0 + + def get_ticker(self): + return self._ticker - def get(self): - return self._flags + def reset_ticker(self, value): + self._ticker = value - def set(self, value): - self._flags = value + def decrement_ticker(self): + value = self._ticker + if self.has_bytecode_counter: # this 'if' is constant-folded + value -= 1 + self._ticker = value + return value class AsyncAction(object): @@ -440,7 +412,7 @@ asynchronously with regular bytecode execution, but that still need to occur between two opcodes, not at a completely random time. """ - bitmask = 0 # means 'please choose one bit automatically' + _fired = False def __init__(self, space): self.space = space @@ -453,10 +425,11 @@ def fire_after_thread_switch(self): """Bit of a hack: fire() the action but only the next time the GIL is released and re-acquired (i.e. after a potential thread switch). - Don't call this if threads are not enabled. + Don't call this if threads are not enabled. Currently limited to + one action (i.e. reserved for CheckSignalAction from module/signal). """ from pypy.module.thread.gil import spacestate - spacestate.set_actionflag_bit_after_thread_switch |= self.bitmask + spacestate.action_after_thread_switch = self def perform(self, executioncontext, frame): """To be overridden.""" @@ -466,7 +439,6 @@ """Abstract base class for actions that occur automatically every sys.checkinterval bytecodes. """ - bitmask = ActionFlag.BYTECODE_COUNTER_OVERFLOW_BIT class UserDelAction(AsyncAction): Modified: pypy/trunk/pypy/interpreter/test/test_executioncontext.py ============================================================================== --- pypy/trunk/pypy/interpreter/test/test_executioncontext.py (original) +++ pypy/trunk/pypy/interpreter/test/test_executioncontext.py Fri Nov 5 15:40:08 2010 @@ -19,7 +19,6 @@ space = self.space a1 = DemoAction(space) - space.actionflag.register_action(a1) for i in range(20): # assert does not raise: space.appexec([], """(): @@ -50,7 +49,7 @@ space = self.space a2 = DemoAction(space) - space.actionflag.register_action(a2) + space.actionflag.register_periodic_action(a2, True) try: for i in range(500): space.appexec([], """(): @@ -59,7 +58,7 @@ """) except Finished: pass - assert space.sys.checkinterval / 10 < i < space.sys.checkinterval * 3 + assert space.sys.checkinterval / 10 < i < space.sys.checkinterval * 1.1 def test_llprofile(self): l = [] Modified: pypy/trunk/pypy/module/signal/__init__.py ============================================================================== --- pypy/trunk/pypy/module/signal/__init__.py (original) +++ pypy/trunk/pypy/module/signal/__init__.py Fri Nov 5 15:40:08 2010 @@ -34,9 +34,7 @@ MixedModule.__init__(self, space, *args) # add the signal-checking callback as an action on the space space.check_signal_action = interp_signal.CheckSignalAction(space) - space.actionflag.register_action(space.check_signal_action) - # use the C-level pypysig_occurred variable as the action flag - # (the result is that the C-level signal handler will directly - # set the flag for the CheckSignalAction) + space.actionflag.register_periodic_action(space.check_signal_action, + use_bytecode_counter=False) space.actionflag.__class__ = interp_signal.SignalActionFlag # xxx yes I know the previous line is a hack Modified: pypy/trunk/pypy/module/signal/interp_signal.py ============================================================================== --- pypy/trunk/pypy/module/signal/interp_signal.py (original) +++ pypy/trunk/pypy/module/signal/interp_signal.py Fri Nov 5 15:40:08 2010 @@ -1,6 +1,7 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.baseobjspace import W_Root, ObjSpace from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag +from pypy.interpreter.executioncontext import PeriodicAsyncAction import signal as cpy_signal from pypy.rpython.lltypesystem import lltype, rffi from pypy.translator.tool.cbuild import ExternalCompilationInfo @@ -52,19 +53,29 @@ class SignalActionFlag(AbstractActionFlag): - def get(self): + # This class uses the C-level pypysig_counter variable as the tick + # counter. The C-level signal handler will reset it to -1 whenever + # a signal is received. + + def get_ticker(self): p = pypysig_getaddr_occurred() return p.c_value - def set(self, value): + + def reset_ticker(self, value): p = pypysig_getaddr_occurred() p.c_value = value + def decrement_ticker(self): + p = pypysig_getaddr_occurred() + value = p.c_value + if self.has_bytecode_counter: # this 'if' is constant-folded + value -= 1 + p.c_value = value + return value -class CheckSignalAction(AsyncAction): - """An action that is automatically invoked when a signal is received.""" - # The C-level signal handler sets the bit 30 of pypysig_occurred: - bitmask = 1 << 30 +class CheckSignalAction(PeriodicAsyncAction): + """An action that is automatically invoked when a signal is received.""" def __init__(self, space): AsyncAction.__init__(self, space) @@ -73,7 +84,6 @@ # need a helper action in case signals arrive in a non-main thread self.pending_signals = {} self.reissue_signal_action = ReissueSignalAction(space) - space.actionflag.register_action(self.reissue_signal_action) else: self.reissue_signal_action = None Modified: pypy/trunk/pypy/module/signal/test/test_signal.py ============================================================================== --- pypy/trunk/pypy/module/signal/test/test_signal.py (original) +++ pypy/trunk/pypy/module/signal/test/test_signal.py Fri Nov 5 15:40:08 2010 @@ -1,6 +1,38 @@ import os, py +import signal as cpy_signal from pypy.conftest import gettestobjspace + +class TestCheckSignals: + + def setup_class(cls): + if not hasattr(os, 'kill') or not hasattr(os, 'getpid'): + py.test.skip("requires os.kill() and os.getpid()") + cls.space = gettestobjspace(usemodules=['signal']) + + def test_checksignals(self): + space = self.space + w_received = space.appexec([], """(): + import signal + received = [] + def myhandler(signum, frame): + received.append(signum) + signal.signal(signal.SIGUSR1, myhandler) + return received""") + # + assert not space.is_true(w_received) + # + # send the signal now + os.kill(os.getpid(), cpy_signal.SIGUSR1) + # + # myhandler() should not be immediately called + assert not space.is_true(w_received) + # + # calling ec.checksignals() should call it + space.getexecutioncontext().checksignals() + assert space.is_true(w_received) + + class AppTestSignal: def setup_class(cls): @@ -24,18 +56,12 @@ signal.signal(signal.SIGUSR1, myhandler) posix.kill(posix.getpid(), signal.SIGUSR1) - for i in range(10000): - # wait a bit for the signal to be delivered to the handler - if received: - break + # the signal should be delivered to the handler immediately assert received == [signal.SIGUSR1] del received[:] posix.kill(posix.getpid(), signal.SIGUSR1) - for i in range(10000): - # wait a bit for the signal to be delivered to the handler - if received: - break + # the signal should be delivered to the handler immediately assert received == [signal.SIGUSR1] del received[:] Modified: pypy/trunk/pypy/module/thread/gil.py ============================================================================== --- pypy/trunk/pypy/module/thread/gil.py (original) +++ pypy/trunk/pypy/module/thread/gil.py Fri Nov 5 15:40:08 2010 @@ -20,7 +20,8 @@ def initialize(self, space): # add the GIL-releasing callback as an action on the space - space.actionflag.register_action(GILReleaseAction(space)) + space.actionflag.register_periodic_action(GILReleaseAction(space), + use_bytecode_counter=True) def setup_threads(self, space): """Enable threads in the object space, if they haven't already been.""" @@ -44,7 +45,6 @@ # test_compile_lock. As a workaround, we repatch these global # fields systematically. spacestate.ll_GIL = self.ll_GIL - spacestate.actionflag = space.actionflag invoke_around_extcall(before_external_call, after_external_call) return result @@ -68,18 +68,17 @@ def _freeze_(self): self.ll_GIL = thread.null_ll_lock - self.actionflag = None - self.set_actionflag_bit_after_thread_switch = 0 + self.action_after_thread_switch = None + # ^^^ set by AsyncAction.fire_after_thread_switch() return False def after_thread_switch(self): # this is support logic for the signal module, to help it deliver # signals to the main thread. - actionflag = self.actionflag - if actionflag is not None: - flag = actionflag.get() - flag |= self.set_actionflag_bit_after_thread_switch - actionflag.set(flag) + action = self.action_after_thread_switch + if action is not None: + self.action_after_thread_switch = None + action.fire() spacestate = SpaceState() spacestate._freeze_() Modified: pypy/trunk/pypy/module/thread/test/test_gil.py ============================================================================== --- pypy/trunk/pypy/module/thread/test/test_gil.py (original) +++ pypy/trunk/pypy/module/thread/test/test_gil.py Fri Nov 5 15:40:08 2010 @@ -8,7 +8,7 @@ pass class FakeActionFlag(object): - def register_action(self, action): + def register_periodic_action(self, action, use_bytecode_counter): pass def get(self): return 0 Modified: pypy/trunk/pypy/translator/c/src/signals.h ============================================================================== --- pypy/trunk/pypy/translator/c/src/signals.h (original) +++ pypy/trunk/pypy/translator/c/src/signals.h Fri Nov 5 15:40:08 2010 @@ -47,16 +47,12 @@ /* utility to poll for signals that arrived */ int pypysig_poll(void); /* => signum or -1 */ -/* When a signal is received, the bit 30 of pypysig_occurred is set. - After all signals are processed by pypysig_poll(), the bit 30 is - cleared again. The variable is exposed and RPython code is free to - use the other bits in any way. */ -#define PENDING_SIGNAL_BIT (1 << 30) +/* When a signal is received, pypysig_counter is set to -1. */ /* This is a struct for the JIT. See interp_signal.py. */ struct pypysig_long_struct { long value; }; -extern struct pypysig_long_struct pypysig_occurred; +extern struct pypysig_long_struct pypysig_counter; /* some C tricks to get/set the variable as efficiently as possible: use macros when compiling as a stand-alone program, but still @@ -64,18 +60,20 @@ #undef pypysig_getaddr_occurred void *pypysig_getaddr_occurred(void); #ifndef PYPY_NOT_MAIN_FILE -void *pypysig_getaddr_occurred(void) { return (void *)(&pypysig_occurred); } +void *pypysig_getaddr_occurred(void) { return (void *)(&pypysig_counter); } #endif -#define pypysig_getaddr_occurred() ((void *)(&pypysig_occurred)) +#define pypysig_getaddr_occurred() ((void *)(&pypysig_counter)) /************************************************************/ /* Implementation */ #ifndef PYPY_NOT_MAIN_FILE -struct pypysig_long_struct pypysig_occurred; -static volatile long *pypysig_occurred_v = (volatile long *)&pypysig_occurred.value; -static volatile int pypysig_flags[NSIG]; +struct pypysig_long_struct pypysig_counter = {0}; +static char volatile pypysig_flags[NSIG] = {0}; +static int volatile pypysig_occurred = 0; +/* pypysig_occurred is only an optimization: it tells if any + pypysig_flags could be set. */ void pypysig_ignore(int signum) { @@ -108,10 +106,11 @@ static void signal_setflag_handler(int signum) { if (0 <= signum && signum < NSIG) + { pypysig_flags[signum] = 1; - /* the point of "*pypysig_occurred_v" instead of just "pypysig_occurred" - is the volatile declaration */ - *pypysig_occurred_v |= PENDING_SIGNAL_BIT; + pypysig_occurred = 1; + pypysig_counter.value = -1; + } } void pypysig_setflag(int signum) @@ -130,27 +129,21 @@ int pypysig_poll(void) { - /* the two commented out lines below are useful for performance in - normal usage of pypysig_poll(); however, pypy/module/signal/ is - not normal usage. It only calls pypysig_poll() if the - PENDING_SIGNAL_BIT is set, and it clears that bit first. */ - -/* if (pypysig_occurred & PENDING_SIGNAL_BIT) */ + if (pypysig_occurred) { - int i; -/* pypysig_occurred &= ~PENDING_SIGNAL_BIT; */ - for (i=0; i Author: arigo Date: Fri Nov 5 15:40:23 2010 New Revision: 78746 Removed: pypy/branch/signals/ Log: Remove merged branch. From arigo at codespeak.net Fri Nov 5 15:43:16 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 5 Nov 2010 15:43:16 +0100 (CET) Subject: [pypy-svn] r78747 - pypy/branch/jit-signals Message-ID: <20101105144316.96A21282C05@codespeak.net> Author: arigo Date: Fri Nov 5 15:43:15 2010 New Revision: 78747 Added: pypy/branch/jit-signals/ - copied from r78746, pypy/trunk/ Log: A branch in which to adapt the frequency of calls to action_dispatcher() done in small jitted loops. From arigo at codespeak.net Fri Nov 5 16:05:34 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 5 Nov 2010 16:05:34 +0100 (CET) Subject: [pypy-svn] r78748 - in pypy/branch/jit-signals/pypy: interpreter interpreter/test module/pypyjit module/signal module/sys Message-ID: <20101105150534.BFFB7282BFE@codespeak.net> Author: arigo Date: Fri Nov 5 16:05:32 2010 New Revision: 78748 Modified: pypy/branch/jit-signals/pypy/interpreter/executioncontext.py pypy/branch/jit-signals/pypy/interpreter/test/test_executioncontext.py pypy/branch/jit-signals/pypy/module/pypyjit/interp_jit.py pypy/branch/jit-signals/pypy/module/signal/interp_signal.py pypy/branch/jit-signals/pypy/module/sys/__init__.py pypy/branch/jit-signals/pypy/module/sys/vm.py Log: Allow us to specify the speed at which checkinterval is reached. For now, jitted loops always take 20 times longer to reach the limit. Modified: pypy/branch/jit-signals/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/branch/jit-signals/pypy/interpreter/executioncontext.py (original) +++ pypy/branch/jit-signals/pypy/interpreter/executioncontext.py Fri Nov 5 16:05:32 2010 @@ -5,6 +5,8 @@ from pypy.rlib.unroll import unrolling_iterable from pypy.rlib import jit +TICK_COUNTER_STEP = 100 + def app_profile_call(space, w_callable, frame, event, w_arg): space.call_function(w_callable, space.wrap(frame), @@ -166,12 +168,12 @@ if self.w_tracefunc is not None: self._trace(frame, 'return', w_retval) - def bytecode_trace(self, frame): + def bytecode_trace(self, frame, decr_by=TICK_COUNTER_STEP): "Trace function called before each bytecode." # this is split into a fast path and a slower path that is # not invoked every time bytecode_trace() is. actionflag = self.space.actionflag - if actionflag.decrement_ticker() < 0: + if actionflag.decrement_ticker(decr_by) < 0: actionflag.action_dispatcher(self, frame) # slow path bytecode_trace._always_inline_ = True @@ -333,6 +335,7 @@ self._nonperiodic_actions = [] self.has_bytecode_counter = False self.fired_actions = None + self.checkinterval_scaled = 100 * TICK_COUNTER_STEP self._rebuild_action_dispatcher() def fire(self, action): @@ -361,10 +364,16 @@ self.has_bytecode_counter = True self._rebuild_action_dispatcher() - def setcheckinterval(self, space, interval): + def getcheckinterval(self): + return self.checkinterval_scaled // TICK_COUNTER_STEP + + def setcheckinterval(self, interval): + MAX = sys.maxint // TICK_COUNTER_STEP if interval < 1: interval = 1 - space.sys.checkinterval = interval + elif interval > MAX: + interval = MAX + self.checkinterval_scaled = interval * TICK_COUNTER_STEP def _rebuild_action_dispatcher(self): periodic_actions = unrolling_iterable(self._periodic_actions) @@ -372,7 +381,7 @@ @jit.dont_look_inside def action_dispatcher(ec, frame): # periodic actions (first reset the bytecode counter) - self.reset_ticker(ec.space.sys.checkinterval) + self.reset_ticker(self.checkinterval_scaled) for action in periodic_actions: action.perform(ec, frame) @@ -399,10 +408,10 @@ def reset_ticker(self, value): self._ticker = value - def decrement_ticker(self): + def decrement_ticker(self, by): value = self._ticker if self.has_bytecode_counter: # this 'if' is constant-folded - value -= 1 + value -= by self._ticker = value return value Modified: pypy/branch/jit-signals/pypy/interpreter/test/test_executioncontext.py ============================================================================== --- pypy/branch/jit-signals/pypy/interpreter/test/test_executioncontext.py (original) +++ pypy/branch/jit-signals/pypy/interpreter/test/test_executioncontext.py Fri Nov 5 16:05:32 2010 @@ -58,7 +58,8 @@ """) except Finished: pass - assert space.sys.checkinterval / 10 < i < space.sys.checkinterval * 1.1 + checkinterval = space.actionflag.getcheckinterval() + assert checkinterval / 10 < i < checkinterval * 1.1 def test_llprofile(self): l = [] Modified: pypy/branch/jit-signals/pypy/module/pypyjit/interp_jit.py ============================================================================== --- pypy/branch/jit-signals/pypy/module/pypyjit/interp_jit.py (original) +++ pypy/branch/jit-signals/pypy/module/pypyjit/interp_jit.py Fri Nov 5 16:05:32 2010 @@ -80,8 +80,9 @@ def jump_absolute(self, jumpto, _, ec=None): if we_are_jitted(): + decr_by = 5 # XXX adapt dynamically self.last_instr = intmask(jumpto) - ec.bytecode_trace(self) + ec.bytecode_trace(self, decr_by) jumpto = r_uint(self.last_instr) pypyjitdriver.can_enter_jit(frame=self, ec=ec, next_instr=jumpto, pycode=self.getcode()) Modified: pypy/branch/jit-signals/pypy/module/signal/interp_signal.py ============================================================================== --- pypy/branch/jit-signals/pypy/module/signal/interp_signal.py (original) +++ pypy/branch/jit-signals/pypy/module/signal/interp_signal.py Fri Nov 5 16:05:32 2010 @@ -65,11 +65,11 @@ p = pypysig_getaddr_occurred() p.c_value = value - def decrement_ticker(self): + def decrement_ticker(self, by): p = pypysig_getaddr_occurred() value = p.c_value if self.has_bytecode_counter: # this 'if' is constant-folded - value -= 1 + value -= by p.c_value = value return value Modified: pypy/branch/jit-signals/pypy/module/sys/__init__.py ============================================================================== --- pypy/branch/jit-signals/pypy/module/sys/__init__.py (original) +++ pypy/branch/jit-signals/pypy/module/sys/__init__.py Fri Nov 5 16:05:32 2010 @@ -10,7 +10,6 @@ if space.config.translating: del self.__class__.interpleveldefs['pypy_getudir'] super(Module, self).__init__(space, w_name) - self.checkinterval = 100 self.recursionlimit = 100 self.w_default_encoder = None self.defaultencoding = "ascii" Modified: pypy/branch/jit-signals/pypy/module/sys/vm.py ============================================================================== --- pypy/branch/jit-signals/pypy/module/sys/vm.py (original) +++ pypy/branch/jit-signals/pypy/module/sys/vm.py Fri Nov 5 16:05:32 2010 @@ -67,7 +67,7 @@ def setcheckinterval(space, interval): """Tell the Python interpreter to check for asynchronous events every n instructions. This also affects how often thread switches occur.""" - space.actionflag.setcheckinterval(space, interval) + space.actionflag.setcheckinterval(interval) setcheckinterval.unwrap_spec = [ObjSpace, int] def getcheckinterval(space): @@ -77,7 +77,7 @@ # return 0. The idea is that according to the CPython docs, <= 0 # means "check every virtual instruction, maximizing responsiveness # as well as overhead". - result = space.sys.checkinterval + result = space.actionflag.getcheckinterval() if result <= 1: result = 0 return space.wrap(result) From afa at codespeak.net Fri Nov 5 16:13:33 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 16:13:33 +0100 (CET) Subject: [pypy-svn] r78749 - in pypy/branch/fast-forward/pypy/module: _lsprof _multiprocessing Message-ID: <20101105151333.73D92282B9E@codespeak.net> Author: afa Date: Fri Nov 5 16:13:31 2010 New Revision: 78749 Modified: pypy/branch/fast-forward/pypy/module/_lsprof/interp_lsprof.py pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Log: Fix translation Modified: pypy/branch/fast-forward/pypy/module/_lsprof/interp_lsprof.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_lsprof/interp_lsprof.py (original) +++ pypy/branch/fast-forward/pypy/module/_lsprof/interp_lsprof.py Fri Nov 5 16:13:31 2010 @@ -5,6 +5,7 @@ interp_attrproperty) from pypy.interpreter.gateway import interp2app, NoneNotWrapped from pypy.interpreter.function import Method, Function +from pypy.interpreter.error import OperationError import time, sys class W_StatsEntry(Wrappable): Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Fri Nov 5 16:13:31 2010 @@ -211,6 +211,7 @@ raise WindowsError(geterrno(), "recv") return buf.str(length) def CLOSE(self): + from pypy.rlib._rsocket_rffi import socketclose socketclose(self.fd) else: def WRITE(self, data): From arigo at codespeak.net Fri Nov 5 16:21:41 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 5 Nov 2010 16:21:41 +0100 (CET) Subject: [pypy-svn] r78750 - in pypy/branch/jit-signals/pypy: jit/codewriter jit/metainterp jit/metainterp/test rlib Message-ID: <20101105152141.D92C4282BF5@codespeak.net> Author: arigo Date: Fri Nov 5 16:21:39 2010 New Revision: 78750 Modified: pypy/branch/jit-signals/pypy/jit/codewriter/jtransform.py pypy/branch/jit-signals/pypy/jit/metainterp/blackhole.py pypy/branch/jit-signals/pypy/jit/metainterp/pyjitpl.py pypy/branch/jit-signals/pypy/jit/metainterp/test/test_basic.py pypy/branch/jit-signals/pypy/rlib/jit.py Log: Implement jit.current_trace_length(). See docstring. Modified: pypy/branch/jit-signals/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/branch/jit-signals/pypy/jit/codewriter/jtransform.py (original) +++ pypy/branch/jit-signals/pypy/jit/codewriter/jtransform.py Fri Nov 5 16:21:39 2010 @@ -873,6 +873,8 @@ elif oopspec_name == 'jit.assert_green': kind = getkind(args[0].concretetype) return SpaceOperation('%s_assert_green' % kind, args, None) + elif oopspec_name == 'jit.current_trace_length': + return SpaceOperation('current_trace_length', [], op.result) else: raise AssertionError("missing support for %r" % oopspec_name) Modified: pypy/branch/jit-signals/pypy/jit/metainterp/blackhole.py ============================================================================== --- pypy/branch/jit-signals/pypy/jit/metainterp/blackhole.py (original) +++ pypy/branch/jit-signals/pypy/jit/metainterp/blackhole.py Fri Nov 5 16:21:39 2010 @@ -774,6 +774,10 @@ def bhimpl_float_assert_green(x): pass + @arguments(returns="i") + def bhimpl_current_trace_length(): + return -1 + # ---------- # the main hints and recursive calls Modified: pypy/branch/jit-signals/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jit-signals/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jit-signals/pypy/jit/metainterp/pyjitpl.py Fri Nov 5 16:21:39 2010 @@ -948,6 +948,11 @@ opimpl_ref_assert_green = _opimpl_assert_green opimpl_float_assert_green = _opimpl_assert_green + @arguments() + def opimpl_current_trace_length(self): + trace_length = len(self.metainterp.history.operations) + return ConstInt(trace_length) + @arguments("box") def opimpl_virtual_ref(self, box): # Details on the content of metainterp.virtualref_boxes: Modified: pypy/branch/jit-signals/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/branch/jit-signals/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/branch/jit-signals/pypy/jit/metainterp/test/test_basic.py Fri Nov 5 16:21:39 2010 @@ -3,6 +3,7 @@ from pypy.rlib.jit import JitDriver, we_are_jitted, hint, dont_look_inside from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE, loop_invariant from pypy.rlib.jit import jit_debug, assert_green, AssertGreenFailed +from pypy.rlib.jit import unroll_safe, current_trace_length from pypy.jit.metainterp.warmspot import ll_meta_interp, get_stats from pypy.jit.backend.llgraph import runner from pypy.jit.metainterp import pyjitpl, history @@ -1672,6 +1673,31 @@ assert res == 8 py.test.raises(AssertGreenFailed, self.interp_operations, f, [8, 0]) + def test_current_trace_length(self): + myjitdriver = JitDriver(greens = ['g'], reds = ['x']) + @dont_look_inside + def residual(): + print "hi there" + @unroll_safe + def loop(g): + y = 0 + while y < g: + residual() + y += 1 + def f(x, g): + n = 0 + while x > 0: + myjitdriver.can_enter_jit(x=x, g=g) + myjitdriver.jit_merge_point(x=x, g=g) + loop(g) + x -= 1 + n = current_trace_length() + return n + res = self.meta_interp(f, [5, 8]) + assert 14 < res < 42 + res = self.meta_interp(f, [5, 2]) + assert 4 < res < 14 + class TestOOtype(BasicTests, OOJitMixin): Modified: pypy/branch/jit-signals/pypy/rlib/jit.py ============================================================================== --- pypy/branch/jit-signals/pypy/rlib/jit.py (original) +++ pypy/branch/jit-signals/pypy/rlib/jit.py Fri Nov 5 16:21:39 2010 @@ -4,6 +4,7 @@ from pypy.rlib.objectmodel import CDefinedIntSymbolic from pypy.rlib.objectmodel import keepalive_until_here from pypy.rlib.unroll import unrolling_iterable +from pypy.rlib.nonconst import NonConstant def purefunction(func): """ Decorate a function as pure. Pure means precisely that: @@ -145,6 +146,14 @@ return hop.inputconst(lltype.Signed, _we_are_jitted) +def current_trace_length(): + """During JIT tracing, returns the current trace length (as a constant). + If not tracing, returns -1.""" + if NonConstant(False): + return 73 + return -1 +current_trace_length.oopspec = 'jit.current_trace_length()' + def jit_debug(string, arg1=-sys.maxint-1, arg2=-sys.maxint-1, arg3=-sys.maxint-1, arg4=-sys.maxint-1): """When JITted, cause an extra operation DEBUG_MERGE_POINT to appear in From arigo at codespeak.net Fri Nov 5 16:29:50 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 5 Nov 2010 16:29:50 +0100 (CET) Subject: [pypy-svn] r78751 - pypy/branch/jit-signals/pypy/module/pypyjit Message-ID: <20101105152950.04658282C05@codespeak.net> Author: arigo Date: Fri Nov 5 16:29:49 2010 New Revision: 78751 Modified: pypy/branch/jit-signals/pypy/module/pypyjit/interp_jit.py Log: This would be the goal. Untested so far. Modified: pypy/branch/jit-signals/pypy/module/pypyjit/interp_jit.py ============================================================================== --- pypy/branch/jit-signals/pypy/module/pypyjit/interp_jit.py (original) +++ pypy/branch/jit-signals/pypy/module/pypyjit/interp_jit.py Fri Nov 5 16:29:49 2010 @@ -6,6 +6,7 @@ from pypy.tool.pairtype import extendabletype from pypy.rlib.rarithmetic import r_uint, intmask from pypy.rlib.jit import JitDriver, hint, we_are_jitted, dont_look_inside +from pypy.rlib.jit import current_trace_length import pypy.interpreter.pyopcode # for side-effects from pypy.interpreter.error import OperationError, operationerrfmt from pypy.interpreter.gateway import ObjSpace, Arguments, W_Root @@ -80,10 +81,22 @@ def jump_absolute(self, jumpto, _, ec=None): if we_are_jitted(): - decr_by = 5 # XXX adapt dynamically + # Normally, the tick counter is decremented by 100 for every + # Python opcode. Here, to better support JIT compilation of + # small loops, we decrement it by a possibly smaller constant. + # We get the maximum 100 when the (unoptimized) trace length + # is at least 3200 (a bit randomly). + trace_length = r_uint(current_trace_length()) + decr_by = trace_length // 32 + if decr_by < 1: + decr_by = 1 + elif decr_by > 100: # also if current_trace_length() returned -1 + decr_by = 100 + # self.last_instr = intmask(jumpto) - ec.bytecode_trace(self, decr_by) + ec.bytecode_trace(self, intmask(decr_by)) jumpto = r_uint(self.last_instr) + # pypyjitdriver.can_enter_jit(frame=self, ec=ec, next_instr=jumpto, pycode=self.getcode()) return jumpto From afa at codespeak.net Fri Nov 5 16:41:51 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 16:41:51 +0100 (CET) Subject: [pypy-svn] r78752 - in pypy/branch/fast-forward/pypy/module/posix: . test Message-ID: <20101105154151.872B2282C00@codespeak.net> Author: afa Date: Fri Nov 5 16:41:49 2010 New Revision: 78752 Modified: pypy/branch/fast-forward/pypy/module/posix/__init__.py pypy/branch/fast-forward/pypy/module/posix/interp_posix.py pypy/branch/fast-forward/pypy/module/posix/test/test_posix2.py Log: Expose posix.spawnv, on Windows at least. It was already implemented in ll_os.py, I don't know why the posix module was forgotten... Modified: pypy/branch/fast-forward/pypy/module/posix/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/posix/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/posix/__init__.py Fri Nov 5 16:41:49 2010 @@ -102,6 +102,8 @@ interpleveldefs['execv'] = 'interp_posix.execv' if hasattr(os, 'execve'): interpleveldefs['execve'] = 'interp_posix.execve' + if hasattr(posix, 'spawnv'): + interpleveldefs['spawnv'] = 'interp_posix.spawnv' if hasattr(os, 'uname'): interpleveldefs['uname'] = 'interp_posix.uname' if hasattr(os, 'sysconf'): Modified: pypy/branch/fast-forward/pypy/module/posix/interp_posix.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/posix/interp_posix.py (original) +++ pypy/branch/fast-forward/pypy/module/posix/interp_posix.py Fri Nov 5 16:41:49 2010 @@ -714,6 +714,15 @@ raise wrap_oserror(space, e) execve.unwrap_spec = [ObjSpace, str, W_Root, W_Root] +def spawnv(space, mode, path, w_args): + args = [space.str_w(w_arg) for w_arg in space.unpackiterable(w_args)] + try: + ret = os.spawnv(mode, path, args) + except OSError, e: + raise wrap_oserror(space, e) + return space.wrap(ret) +spawnv.unwrap_spec = [ObjSpace, int, str, W_Root] + def utime(space, w_path, w_tuple): """ utime(path, (atime, mtime)) utime(path, None) Modified: pypy/branch/fast-forward/pypy/module/posix/test/test_posix2.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/posix/test/test_posix2.py (original) +++ pypy/branch/fast-forward/pypy/module/posix/test/test_posix2.py Fri Nov 5 16:41:49 2010 @@ -65,6 +65,7 @@ cls.w_sysconf_value = space.wrap(os.sysconf_names[sysconf_name]) cls.w_sysconf_result = space.wrap(os.sysconf(sysconf_name)) cls.w_SIGABRT = space.wrap(signal.SIGABRT) + cls.w_python = space.wrap(sys.executable) def setup_method(self, meth): if getattr(meth, 'need_sparse_files', False): @@ -375,6 +376,15 @@ os.unlink("onefile") pass # <- please, inspect.getsource(), don't crash + if hasattr(__import__(os.name), "spawnv"): + def test_spawnv(self): + os = self.posix + import sys + print self.python + ret = os.spawnv(os.P_WAIT, self.python, + ['python', '-c', 'raise(SystemExit(42))']) + assert ret == 42 + def test_popen(self): os = self.posix for i in range(5): From antocuni at codespeak.net Fri Nov 5 17:05:21 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 5 Nov 2010 17:05:21 +0100 (CET) Subject: [pypy-svn] r78753 - pypy/extradoc/planning/hg-migration Message-ID: <20101105160521.B719C282C07@codespeak.net> Author: antocuni Date: Fri Nov 5 17:05:20 2010 New Revision: 78753 Modified: pypy/extradoc/planning/hg-migration/plan.txt Log: update Modified: pypy/extradoc/planning/hg-migration/plan.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/plan.txt (original) +++ pypy/extradoc/planning/hg-migration/plan.txt Fri Nov 5 17:05:20 2010 @@ -48,6 +48,7 @@ pypy-svn at codespeak.net, but they did not arrive. Probably because the mailing list blocks mails if the author is not subscribed? i added *@bitbucket.org to allowed addresses. + cool, it works :-) - IRC bot: bitbucket has a builtin hook for CIA.vc; alternatively, we can try to adapt kenaan: who owns it? From odie at codespeak.net Fri Nov 5 17:42:05 2010 From: odie at codespeak.net (odie at codespeak.net) Date: Fri, 5 Nov 2010 17:42:05 +0100 (CET) Subject: [pypy-svn] r78754 - pypy/extradoc/planning/hg-migration Message-ID: <20101105164205.D15D2282B9D@codespeak.net> Author: odie Date: Fri Nov 5 17:42:04 2010 New Revision: 78754 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: my email Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Fri Nov 5 17:42:04 2010 @@ -66,7 +66,7 @@ micktwomey=Michael Twomey boria=boria jared.grubb Jared Grubb -odie=Olivier Dormond +odie=Olivier Dormond stuart=Stuart Williams jum=Jens-Uwe Mager jcreigh=Jason Creighton From mgedmin at codespeak.net Fri Nov 5 17:45:49 2010 From: mgedmin at codespeak.net (mgedmin at codespeak.net) Date: Fri, 5 Nov 2010 17:45:49 +0100 (CET) Subject: [pypy-svn] r78755 - pypy/extradoc/planning/hg-migration Message-ID: <20101105164549.D1892282BEC@codespeak.net> Author: mgedmin Date: Fri Nov 5 17:45:48 2010 New Revision: 78755 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: Fill in my email Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Fri Nov 5 17:45:48 2010 @@ -42,7 +42,7 @@ bgola=Bruno Gola afayolle=Alexandre Fayolle agaynor=Alex Gaynor -mgedmin=Marius Gedminas +mgedmin=Marius Gedminas getxsick=Bartosz Skowron,,, gvanrossum=Guido van Rossum dialtone=Valentino Volonghi From arigo at codespeak.net Fri Nov 5 17:47:43 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 5 Nov 2010 17:47:43 +0100 (CET) Subject: [pypy-svn] r78756 - in pypy/trunk/pypy: interpreter interpreter/test jit/codewriter jit/metainterp jit/metainterp/test module/pypyjit module/signal module/sys rlib Message-ID: <20101105164743.20279282BEC@codespeak.net> Author: arigo Date: Fri Nov 5 17:47:41 2010 New Revision: 78756 Modified: pypy/trunk/pypy/interpreter/executioncontext.py pypy/trunk/pypy/interpreter/test/test_executioncontext.py pypy/trunk/pypy/jit/codewriter/jtransform.py pypy/trunk/pypy/jit/metainterp/blackhole.py pypy/trunk/pypy/jit/metainterp/pyjitpl.py pypy/trunk/pypy/jit/metainterp/test/test_basic.py pypy/trunk/pypy/module/pypyjit/interp_jit.py pypy/trunk/pypy/module/signal/interp_signal.py pypy/trunk/pypy/module/sys/__init__.py pypy/trunk/pypy/module/sys/vm.py pypy/trunk/pypy/rlib/jit.py Log: Merge branch/jit-signals. JITted loops now call the action_dispatcher() less often if they are small. Modified: pypy/trunk/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/trunk/pypy/interpreter/executioncontext.py (original) +++ pypy/trunk/pypy/interpreter/executioncontext.py Fri Nov 5 17:47:41 2010 @@ -5,6 +5,8 @@ from pypy.rlib.unroll import unrolling_iterable from pypy.rlib import jit +TICK_COUNTER_STEP = 100 + def app_profile_call(space, w_callable, frame, event, w_arg): space.call_function(w_callable, space.wrap(frame), @@ -166,12 +168,12 @@ if self.w_tracefunc is not None: self._trace(frame, 'return', w_retval) - def bytecode_trace(self, frame): + def bytecode_trace(self, frame, decr_by=TICK_COUNTER_STEP): "Trace function called before each bytecode." # this is split into a fast path and a slower path that is # not invoked every time bytecode_trace() is. actionflag = self.space.actionflag - if actionflag.decrement_ticker() < 0: + if actionflag.decrement_ticker(decr_by) < 0: actionflag.action_dispatcher(self, frame) # slow path bytecode_trace._always_inline_ = True @@ -333,6 +335,7 @@ self._nonperiodic_actions = [] self.has_bytecode_counter = False self.fired_actions = None + self.checkinterval_scaled = 100 * TICK_COUNTER_STEP self._rebuild_action_dispatcher() def fire(self, action): @@ -361,10 +364,16 @@ self.has_bytecode_counter = True self._rebuild_action_dispatcher() - def setcheckinterval(self, space, interval): + def getcheckinterval(self): + return self.checkinterval_scaled // TICK_COUNTER_STEP + + def setcheckinterval(self, interval): + MAX = sys.maxint // TICK_COUNTER_STEP if interval < 1: interval = 1 - space.sys.checkinterval = interval + elif interval > MAX: + interval = MAX + self.checkinterval_scaled = interval * TICK_COUNTER_STEP def _rebuild_action_dispatcher(self): periodic_actions = unrolling_iterable(self._periodic_actions) @@ -372,7 +381,7 @@ @jit.dont_look_inside def action_dispatcher(ec, frame): # periodic actions (first reset the bytecode counter) - self.reset_ticker(ec.space.sys.checkinterval) + self.reset_ticker(self.checkinterval_scaled) for action in periodic_actions: action.perform(ec, frame) @@ -399,10 +408,10 @@ def reset_ticker(self, value): self._ticker = value - def decrement_ticker(self): + def decrement_ticker(self, by): value = self._ticker if self.has_bytecode_counter: # this 'if' is constant-folded - value -= 1 + value -= by self._ticker = value return value Modified: pypy/trunk/pypy/interpreter/test/test_executioncontext.py ============================================================================== --- pypy/trunk/pypy/interpreter/test/test_executioncontext.py (original) +++ pypy/trunk/pypy/interpreter/test/test_executioncontext.py Fri Nov 5 17:47:41 2010 @@ -58,7 +58,8 @@ """) except Finished: pass - assert space.sys.checkinterval / 10 < i < space.sys.checkinterval * 1.1 + checkinterval = space.actionflag.getcheckinterval() + assert checkinterval / 10 < i < checkinterval * 1.1 def test_llprofile(self): l = [] Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/jtransform.py (original) +++ pypy/trunk/pypy/jit/codewriter/jtransform.py Fri Nov 5 17:47:41 2010 @@ -873,6 +873,8 @@ elif oopspec_name == 'jit.assert_green': kind = getkind(args[0].concretetype) return SpaceOperation('%s_assert_green' % kind, args, None) + elif oopspec_name == 'jit.current_trace_length': + return SpaceOperation('current_trace_length', [], op.result) else: raise AssertionError("missing support for %r" % oopspec_name) Modified: pypy/trunk/pypy/jit/metainterp/blackhole.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/blackhole.py (original) +++ pypy/trunk/pypy/jit/metainterp/blackhole.py Fri Nov 5 17:47:41 2010 @@ -774,6 +774,10 @@ def bhimpl_float_assert_green(x): pass + @arguments(returns="i") + def bhimpl_current_trace_length(): + return -1 + # ---------- # the main hints and recursive calls Modified: pypy/trunk/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/trunk/pypy/jit/metainterp/pyjitpl.py Fri Nov 5 17:47:41 2010 @@ -948,6 +948,11 @@ opimpl_ref_assert_green = _opimpl_assert_green opimpl_float_assert_green = _opimpl_assert_green + @arguments() + def opimpl_current_trace_length(self): + trace_length = len(self.metainterp.history.operations) + return ConstInt(trace_length) + @arguments("box") def opimpl_virtual_ref(self, box): # Details on the content of metainterp.virtualref_boxes: Modified: pypy/trunk/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_basic.py Fri Nov 5 17:47:41 2010 @@ -3,6 +3,7 @@ from pypy.rlib.jit import JitDriver, we_are_jitted, hint, dont_look_inside from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE, loop_invariant from pypy.rlib.jit import jit_debug, assert_green, AssertGreenFailed +from pypy.rlib.jit import unroll_safe, current_trace_length from pypy.jit.metainterp.warmspot import ll_meta_interp, get_stats from pypy.jit.backend.llgraph import runner from pypy.jit.metainterp import pyjitpl, history @@ -1672,6 +1673,31 @@ assert res == 8 py.test.raises(AssertGreenFailed, self.interp_operations, f, [8, 0]) + def test_current_trace_length(self): + myjitdriver = JitDriver(greens = ['g'], reds = ['x']) + @dont_look_inside + def residual(): + print "hi there" + @unroll_safe + def loop(g): + y = 0 + while y < g: + residual() + y += 1 + def f(x, g): + n = 0 + while x > 0: + myjitdriver.can_enter_jit(x=x, g=g) + myjitdriver.jit_merge_point(x=x, g=g) + loop(g) + x -= 1 + n = current_trace_length() + return n + res = self.meta_interp(f, [5, 8]) + assert 14 < res < 42 + res = self.meta_interp(f, [5, 2]) + assert 4 < res < 14 + class TestOOtype(BasicTests, OOJitMixin): Modified: pypy/trunk/pypy/module/pypyjit/interp_jit.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/interp_jit.py (original) +++ pypy/trunk/pypy/module/pypyjit/interp_jit.py Fri Nov 5 17:47:41 2010 @@ -6,6 +6,7 @@ from pypy.tool.pairtype import extendabletype from pypy.rlib.rarithmetic import r_uint, intmask from pypy.rlib.jit import JitDriver, hint, we_are_jitted, dont_look_inside +from pypy.rlib.jit import current_trace_length import pypy.interpreter.pyopcode # for side-effects from pypy.interpreter.error import OperationError, operationerrfmt from pypy.interpreter.gateway import ObjSpace, Arguments, W_Root @@ -80,9 +81,22 @@ def jump_absolute(self, jumpto, _, ec=None): if we_are_jitted(): + # Normally, the tick counter is decremented by 100 for every + # Python opcode. Here, to better support JIT compilation of + # small loops, we decrement it by a possibly smaller constant. + # We get the maximum 100 when the (unoptimized) trace length + # is at least 3200 (a bit randomly). + trace_length = r_uint(current_trace_length()) + decr_by = trace_length // 32 + if decr_by < 1: + decr_by = 1 + elif decr_by > 100: # also if current_trace_length() returned -1 + decr_by = 100 + # self.last_instr = intmask(jumpto) - ec.bytecode_trace(self) + ec.bytecode_trace(self, intmask(decr_by)) jumpto = r_uint(self.last_instr) + # pypyjitdriver.can_enter_jit(frame=self, ec=ec, next_instr=jumpto, pycode=self.getcode()) return jumpto Modified: pypy/trunk/pypy/module/signal/interp_signal.py ============================================================================== --- pypy/trunk/pypy/module/signal/interp_signal.py (original) +++ pypy/trunk/pypy/module/signal/interp_signal.py Fri Nov 5 17:47:41 2010 @@ -65,11 +65,11 @@ p = pypysig_getaddr_occurred() p.c_value = value - def decrement_ticker(self): + def decrement_ticker(self, by): p = pypysig_getaddr_occurred() value = p.c_value if self.has_bytecode_counter: # this 'if' is constant-folded - value -= 1 + value -= by p.c_value = value return value Modified: pypy/trunk/pypy/module/sys/__init__.py ============================================================================== --- pypy/trunk/pypy/module/sys/__init__.py (original) +++ pypy/trunk/pypy/module/sys/__init__.py Fri Nov 5 17:47:41 2010 @@ -10,7 +10,6 @@ if space.config.translating: del self.__class__.interpleveldefs['pypy_getudir'] super(Module, self).__init__(space, w_name) - self.checkinterval = 100 self.recursionlimit = 100 self.w_default_encoder = None self.defaultencoding = "ascii" Modified: pypy/trunk/pypy/module/sys/vm.py ============================================================================== --- pypy/trunk/pypy/module/sys/vm.py (original) +++ pypy/trunk/pypy/module/sys/vm.py Fri Nov 5 17:47:41 2010 @@ -67,7 +67,7 @@ def setcheckinterval(space, interval): """Tell the Python interpreter to check for asynchronous events every n instructions. This also affects how often thread switches occur.""" - space.actionflag.setcheckinterval(space, interval) + space.actionflag.setcheckinterval(interval) setcheckinterval.unwrap_spec = [ObjSpace, int] def getcheckinterval(space): @@ -77,7 +77,7 @@ # return 0. The idea is that according to the CPython docs, <= 0 # means "check every virtual instruction, maximizing responsiveness # as well as overhead". - result = space.sys.checkinterval + result = space.actionflag.getcheckinterval() if result <= 1: result = 0 return space.wrap(result) Modified: pypy/trunk/pypy/rlib/jit.py ============================================================================== --- pypy/trunk/pypy/rlib/jit.py (original) +++ pypy/trunk/pypy/rlib/jit.py Fri Nov 5 17:47:41 2010 @@ -4,6 +4,7 @@ from pypy.rlib.objectmodel import CDefinedIntSymbolic from pypy.rlib.objectmodel import keepalive_until_here from pypy.rlib.unroll import unrolling_iterable +from pypy.rlib.nonconst import NonConstant def purefunction(func): """ Decorate a function as pure. Pure means precisely that: @@ -145,6 +146,14 @@ return hop.inputconst(lltype.Signed, _we_are_jitted) +def current_trace_length(): + """During JIT tracing, returns the current trace length (as a constant). + If not tracing, returns -1.""" + if NonConstant(False): + return 73 + return -1 +current_trace_length.oopspec = 'jit.current_trace_length()' + def jit_debug(string, arg1=-sys.maxint-1, arg2=-sys.maxint-1, arg3=-sys.maxint-1, arg4=-sys.maxint-1): """When JITted, cause an extra operation DEBUG_MERGE_POINT to appear in From arigo at codespeak.net Fri Nov 5 17:47:59 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 5 Nov 2010 17:47:59 +0100 (CET) Subject: [pypy-svn] r78757 - pypy/branch/jit-signals Message-ID: <20101105164759.9EA65282C0B@codespeak.net> Author: arigo Date: Fri Nov 5 17:47:58 2010 New Revision: 78757 Removed: pypy/branch/jit-signals/ Log: Remove merged branch. From exarkun at codespeak.net Fri Nov 5 18:04:59 2010 From: exarkun at codespeak.net (exarkun at codespeak.net) Date: Fri, 5 Nov 2010 18:04:59 +0100 (CET) Subject: [pypy-svn] r78758 - pypy/extradoc/planning/hg-migration Message-ID: <20101105170459.7E6A8282C03@codespeak.net> Author: exarkun Date: Fri Nov 5 18:04:57 2010 New Revision: 78758 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: Specify my email Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Fri Nov 5 18:04:57 2010 @@ -54,7 +54,7 @@ oscar=Oscar Nierstrasz jandem=Jan de Mooij goden=Eugene Oden -exarkun=Jean-Paul Calderone +exarkun=Jean-Paul Calderone lukas=Lukas Renggli hakanardo=Hakan Ardo guenter=Guenter Jantzen From afa at codespeak.net Fri Nov 5 18:38:18 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 18:38:18 +0100 (CET) Subject: [pypy-svn] r78759 - pypy/branch/fast-forward/pypy/module/cpyext/include Message-ID: <20101105173818.B7E46282BFE@codespeak.net> Author: afa Date: Fri Nov 5 18:38:17 2010 New Revision: 78759 Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/Python.h pypy/branch/fast-forward/pypy/module/cpyext/include/pyconfig.h Log: cpyext: Fix definition for old DL_EXPORT macros + unconditionally define HAVE_WCHAR_H Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/Python.h ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/include/Python.h (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/Python.h Fri Nov 5 18:38:17 2010 @@ -32,7 +32,23 @@ # endif # define Py_LOCAL_INLINE(type) static __inline type __fastcall #endif -#define DL_IMPORT(RTYPE) PyAPI_FUNC(RTYPE) + +/* Deprecated DL_IMPORT and DL_EXPORT macros */ +#ifdef _WIN32 +# if defined(Py_BUILD_CORE) +# define DL_IMPORT(RTYPE) __declspec(dllexport) RTYPE +# define DL_EXPORT(RTYPE) __declspec(dllexport) RTYPE +# else +# define DL_IMPORT(RTYPE) __declspec(dllimport) RTYPE +# define DL_EXPORT(RTYPE) __declspec(dllexport) RTYPE +# endif +#endif +#ifndef DL_EXPORT +# define DL_EXPORT(RTYPE) RTYPE +#endif +#ifndef DL_IMPORT +# define DL_IMPORT(RTYPE) RTYPE +#endif #include @@ -55,10 +71,6 @@ #define Py_CHARMASK(c) ((unsigned char)((c) & 0xff)) #endif -#ifndef DL_EXPORT /* declarations for DLL import/export */ -#define DL_EXPORT(RTYPE) RTYPE -#endif - #define statichere static #define Py_MEMCPY memcpy Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/pyconfig.h ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/include/pyconfig.h (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/pyconfig.h Fri Nov 5 18:38:17 2010 @@ -15,6 +15,7 @@ #define WITH_DOC_STRINGS #define HAVE_UNICODE #define WITHOUT_COMPLEX +#define HAVE_WCHAR_H 1 /* PyPy supposes Py_UNICODE == wchar_t */ #define HAVE_USABLE_WCHAR_T 1 From afa at codespeak.net Fri Nov 5 19:04:59 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 19:04:59 +0100 (CET) Subject: [pypy-svn] r78760 - pypy/branch/fast-forward/lib_pypy Message-ID: <20101105180459.BDC62282BF0@codespeak.net> Author: afa Date: Fri Nov 5 19:04:58 2010 New Revision: 78760 Added: pypy/branch/fast-forward/lib_pypy/_ctypes_test.c (contents, props changed) pypy/branch/fast-forward/lib_pypy/_ctypes_test.py (contents, props changed) Log: Be crazy and compile _ctypes_test.c on the fly. It even works! Added: pypy/branch/fast-forward/lib_pypy/_ctypes_test.c ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/lib_pypy/_ctypes_test.c Fri Nov 5 19:04:58 2010 @@ -0,0 +1,603 @@ +/* This is a Verbatim copy of _ctypes_test.c from CPython 2.7 */ +/***************************************************************** + This file should be kept compatible with Python 2.3, see PEP 291. + *****************************************************************/ + + +#include + +/* + Backwards compatibility: + Python2.2 used LONG_LONG instead of PY_LONG_LONG +*/ +#if defined(HAVE_LONG_LONG) && !defined(PY_LONG_LONG) +#define PY_LONG_LONG LONG_LONG +#endif + +#ifdef MS_WIN32 +#include +#endif + +#if defined(MS_WIN32) || defined(__CYGWIN__) +#define EXPORT(x) __declspec(dllexport) x +#else +#define EXPORT(x) x +#endif + +/* some functions handy for testing */ + +EXPORT(void)testfunc_array(int values[4]) +{ + printf("testfunc_array %d %d %d %d\n", + values[0], + values[1], + values[2], + values[3]); +} + +EXPORT(long double)testfunc_Ddd(double a, double b) +{ + long double result = (long double)(a * b); + printf("testfunc_Ddd(%p, %p)\n", &a, &b); + printf("testfunc_Ddd(%g, %g)\n", a, b); + return result; +} + +EXPORT(long double)testfunc_DDD(long double a, long double b) +{ + long double result = a * b; + printf("testfunc_DDD(%p, %p)\n", &a, &b); + printf("testfunc_DDD(%Lg, %Lg)\n", a, b); + return result; +} + +EXPORT(int)testfunc_iii(int a, int b) +{ + int result = a * b; + printf("testfunc_iii(%p, %p)\n", &a, &b); + return result; +} + +EXPORT(int)myprintf(char *fmt, ...) +{ + int result; + va_list argptr; + va_start(argptr, fmt); + result = vprintf(fmt, argptr); + va_end(argptr); + return result; +} + +EXPORT(char *)my_strtok(char *token, const char *delim) +{ + return strtok(token, delim); +} + +EXPORT(char *)my_strchr(const char *s, int c) +{ + return strchr(s, c); +} + + +EXPORT(double) my_sqrt(double a) +{ + return sqrt(a); +} + +EXPORT(void) my_qsort(void *base, size_t num, size_t width, int(*compare)(const void*, const void*)) +{ + qsort(base, num, width, compare); +} + +EXPORT(int *) _testfunc_ai8(int a[8]) +{ + return a; +} + +EXPORT(void) _testfunc_v(int a, int b, int *presult) +{ + *presult = a + b; +} + +EXPORT(int) _testfunc_i_bhilfd(signed char b, short h, int i, long l, float f, double d) +{ +/* printf("_testfunc_i_bhilfd got %d %d %d %ld %f %f\n", + b, h, i, l, f, d); +*/ + return (int)(b + h + i + l + f + d); +} + +EXPORT(float) _testfunc_f_bhilfd(signed char b, short h, int i, long l, float f, double d) +{ +/* printf("_testfunc_f_bhilfd got %d %d %d %ld %f %f\n", + b, h, i, l, f, d); +*/ + return (float)(b + h + i + l + f + d); +} + +EXPORT(double) _testfunc_d_bhilfd(signed char b, short h, int i, long l, float f, double d) +{ +/* printf("_testfunc_d_bhilfd got %d %d %d %ld %f %f\n", + b, h, i, l, f, d); +*/ + return (double)(b + h + i + l + f + d); +} + +EXPORT(long double) _testfunc_D_bhilfD(signed char b, short h, int i, long l, float f, long double d) +{ +/* printf("_testfunc_d_bhilfd got %d %d %d %ld %f %f\n", + b, h, i, l, f, d); +*/ + return (long double)(b + h + i + l + f + d); +} + +EXPORT(char *) _testfunc_p_p(void *s) +{ + return (char *)s; +} + +EXPORT(void *) _testfunc_c_p_p(int *argcp, char **argv) +{ + return argv[(*argcp)-1]; +} + +EXPORT(void *) get_strchr(void) +{ + return (void *)strchr; +} + +EXPORT(char *) my_strdup(char *src) +{ + char *dst = (char *)malloc(strlen(src)+1); + if (!dst) + return NULL; + strcpy(dst, src); + return dst; +} + +EXPORT(void)my_free(void *ptr) +{ + free(ptr); +} + +#ifdef HAVE_WCHAR_H +EXPORT(wchar_t *) my_wcsdup(wchar_t *src) +{ + size_t len = wcslen(src); + wchar_t *ptr = (wchar_t *)malloc((len + 1) * sizeof(wchar_t)); + if (ptr == NULL) + return NULL; + memcpy(ptr, src, (len+1) * sizeof(wchar_t)); + return ptr; +} + +EXPORT(size_t) my_wcslen(wchar_t *src) +{ + return wcslen(src); +} +#endif + +#ifndef MS_WIN32 +# ifndef __stdcall +# define __stdcall /* */ +# endif +#endif + +typedef struct { + int (*c)(int, int); + int (__stdcall *s)(int, int); +} FUNCS; + +EXPORT(int) _testfunc_callfuncp(FUNCS *fp) +{ + fp->c(1, 2); + fp->s(3, 4); + return 0; +} + +EXPORT(int) _testfunc_deref_pointer(int *pi) +{ + return *pi; +} + +#ifdef MS_WIN32 +EXPORT(int) _testfunc_piunk(IUnknown FAR *piunk) +{ + piunk->lpVtbl->AddRef(piunk); + return piunk->lpVtbl->Release(piunk); +} +#endif + +EXPORT(int) _testfunc_callback_with_pointer(int (*func)(int *)) +{ + int table[] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10}; + + return (*func)(table); +} + +#ifdef HAVE_LONG_LONG +EXPORT(PY_LONG_LONG) _testfunc_q_bhilfdq(signed char b, short h, int i, long l, float f, + double d, PY_LONG_LONG q) +{ + return (PY_LONG_LONG)(b + h + i + l + f + d + q); +} + +EXPORT(PY_LONG_LONG) _testfunc_q_bhilfd(signed char b, short h, int i, long l, float f, double d) +{ + return (PY_LONG_LONG)(b + h + i + l + f + d); +} + +EXPORT(int) _testfunc_callback_i_if(int value, int (*func)(int)) +{ + int sum = 0; + while (value != 0) { + sum += func(value); + value /= 2; + } + return sum; +} + +EXPORT(PY_LONG_LONG) _testfunc_callback_q_qf(PY_LONG_LONG value, + PY_LONG_LONG (*func)(PY_LONG_LONG)) +{ + PY_LONG_LONG sum = 0; + + while (value != 0) { + sum += func(value); + value /= 2; + } + return sum; +} + +#endif + +typedef struct { + char *name; + char *value; +} SPAM; + +typedef struct { + char *name; + int num_spams; + SPAM *spams; +} EGG; + +SPAM my_spams[2] = { + { "name1", "value1" }, + { "name2", "value2" }, +}; + +EGG my_eggs[1] = { + { "first egg", 1, my_spams } +}; + +EXPORT(int) getSPAMANDEGGS(EGG **eggs) +{ + *eggs = my_eggs; + return 1; +} + +typedef struct tagpoint { + int x; + int y; +} point; + +EXPORT(int) _testfunc_byval(point in, point *pout) +{ + if (pout) { + pout->x = in.x; + pout->y = in.y; + } + return in.x + in.y; +} + +EXPORT (int) an_integer = 42; + +EXPORT(int) get_an_integer(void) +{ + return an_integer; +} + +EXPORT(double) +integrate(double a, double b, double (*f)(double), long nstep) +{ + double x, sum=0.0, dx=(b-a)/(double)nstep; + for(x=a+0.5*dx; (b-x)*(x-a)>0.0; x+=dx) + sum += f(x); + return sum/(double)nstep; +} + +typedef struct { + void (*initialize)(void *(*)(int), void(*)(void *)); +} xxx_library; + +static void _xxx_init(void *(*Xalloc)(int), void (*Xfree)(void *)) +{ + void *ptr; + + printf("_xxx_init got %p %p\n", Xalloc, Xfree); + printf("calling\n"); + ptr = Xalloc(32); + Xfree(ptr); + printf("calls done, ptr was %p\n", ptr); +} + +xxx_library _xxx_lib = { + _xxx_init +}; + +EXPORT(xxx_library) *library_get(void) +{ + return &_xxx_lib; +} + +#ifdef MS_WIN32 +/* See Don Box (german), pp 79ff. */ +EXPORT(void) GetString(BSTR *pbstr) +{ + *pbstr = SysAllocString(L"Goodbye!"); +} +#endif + +/* + * Some do-nothing functions, for speed tests + */ +PyObject *py_func_si(PyObject *self, PyObject *args) +{ + char *name; + int i; + if (!PyArg_ParseTuple(args, "si", &name, &i)) + return NULL; + Py_INCREF(Py_None); + return Py_None; +} + +EXPORT(void) _py_func_si(char *s, int i) +{ +} + +PyObject *py_func(PyObject *self, PyObject *args) +{ + Py_INCREF(Py_None); + return Py_None; +} + +EXPORT(void) _py_func(void) +{ +} + +EXPORT(PY_LONG_LONG) last_tf_arg_s; +EXPORT(unsigned PY_LONG_LONG) last_tf_arg_u; + +struct BITS { + int A: 1, B:2, C:3, D:4, E: 5, F: 6, G: 7, H: 8, I: 9; + short M: 1, N: 2, O: 3, P: 4, Q: 5, R: 6, S: 7; +}; + +DL_EXPORT(void) set_bitfields(struct BITS *bits, char name, int value) +{ + switch (name) { + case 'A': bits->A = value; break; + case 'B': bits->B = value; break; + case 'C': bits->C = value; break; + case 'D': bits->D = value; break; + case 'E': bits->E = value; break; + case 'F': bits->F = value; break; + case 'G': bits->G = value; break; + case 'H': bits->H = value; break; + case 'I': bits->I = value; break; + + case 'M': bits->M = value; break; + case 'N': bits->N = value; break; + case 'O': bits->O = value; break; + case 'P': bits->P = value; break; + case 'Q': bits->Q = value; break; + case 'R': bits->R = value; break; + case 'S': bits->S = value; break; + } +} + +DL_EXPORT(int) unpack_bitfields(struct BITS *bits, char name) +{ + switch (name) { + case 'A': return bits->A; + case 'B': return bits->B; + case 'C': return bits->C; + case 'D': return bits->D; + case 'E': return bits->E; + case 'F': return bits->F; + case 'G': return bits->G; + case 'H': return bits->H; + case 'I': return bits->I; + + case 'M': return bits->M; + case 'N': return bits->N; + case 'O': return bits->O; + case 'P': return bits->P; + case 'Q': return bits->Q; + case 'R': return bits->R; + case 'S': return bits->S; + } + return 0; +} + +static PyMethodDef module_methods[] = { +/* {"get_last_tf_arg_s", get_last_tf_arg_s, METH_NOARGS}, + {"get_last_tf_arg_u", get_last_tf_arg_u, METH_NOARGS}, +*/ + {"func_si", py_func_si, METH_VARARGS}, + {"func", py_func, METH_NOARGS}, + { NULL, NULL, 0, NULL}, +}; + +#define S last_tf_arg_s = (PY_LONG_LONG)c +#define U last_tf_arg_u = (unsigned PY_LONG_LONG)c + +EXPORT(signed char) tf_b(signed char c) { S; return c/3; } +EXPORT(unsigned char) tf_B(unsigned char c) { U; return c/3; } +EXPORT(short) tf_h(short c) { S; return c/3; } +EXPORT(unsigned short) tf_H(unsigned short c) { U; return c/3; } +EXPORT(int) tf_i(int c) { S; return c/3; } +EXPORT(unsigned int) tf_I(unsigned int c) { U; return c/3; } +EXPORT(long) tf_l(long c) { S; return c/3; } +EXPORT(unsigned long) tf_L(unsigned long c) { U; return c/3; } +EXPORT(PY_LONG_LONG) tf_q(PY_LONG_LONG c) { S; return c/3; } +EXPORT(unsigned PY_LONG_LONG) tf_Q(unsigned PY_LONG_LONG c) { U; return c/3; } +EXPORT(float) tf_f(float c) { S; return c/3; } +EXPORT(double) tf_d(double c) { S; return c/3; } +EXPORT(long double) tf_D(long double c) { S; return c/3; } + +#ifdef MS_WIN32 +EXPORT(signed char) __stdcall s_tf_b(signed char c) { S; return c/3; } +EXPORT(unsigned char) __stdcall s_tf_B(unsigned char c) { U; return c/3; } +EXPORT(short) __stdcall s_tf_h(short c) { S; return c/3; } +EXPORT(unsigned short) __stdcall s_tf_H(unsigned short c) { U; return c/3; } +EXPORT(int) __stdcall s_tf_i(int c) { S; return c/3; } +EXPORT(unsigned int) __stdcall s_tf_I(unsigned int c) { U; return c/3; } +EXPORT(long) __stdcall s_tf_l(long c) { S; return c/3; } +EXPORT(unsigned long) __stdcall s_tf_L(unsigned long c) { U; return c/3; } +EXPORT(PY_LONG_LONG) __stdcall s_tf_q(PY_LONG_LONG c) { S; return c/3; } +EXPORT(unsigned PY_LONG_LONG) __stdcall s_tf_Q(unsigned PY_LONG_LONG c) { U; return c/3; } +EXPORT(float) __stdcall s_tf_f(float c) { S; return c/3; } +EXPORT(double) __stdcall s_tf_d(double c) { S; return c/3; } +EXPORT(long double) __stdcall s_tf_D(long double c) { S; return c/3; } +#endif +/*******/ + +EXPORT(signed char) tf_bb(signed char x, signed char c) { S; return c/3; } +EXPORT(unsigned char) tf_bB(signed char x, unsigned char c) { U; return c/3; } +EXPORT(short) tf_bh(signed char x, short c) { S; return c/3; } +EXPORT(unsigned short) tf_bH(signed char x, unsigned short c) { U; return c/3; } +EXPORT(int) tf_bi(signed char x, int c) { S; return c/3; } +EXPORT(unsigned int) tf_bI(signed char x, unsigned int c) { U; return c/3; } +EXPORT(long) tf_bl(signed char x, long c) { S; return c/3; } +EXPORT(unsigned long) tf_bL(signed char x, unsigned long c) { U; return c/3; } +EXPORT(PY_LONG_LONG) tf_bq(signed char x, PY_LONG_LONG c) { S; return c/3; } +EXPORT(unsigned PY_LONG_LONG) tf_bQ(signed char x, unsigned PY_LONG_LONG c) { U; return c/3; } +EXPORT(float) tf_bf(signed char x, float c) { S; return c/3; } +EXPORT(double) tf_bd(signed char x, double c) { S; return c/3; } +EXPORT(long double) tf_bD(signed char x, long double c) { S; return c/3; } +EXPORT(void) tv_i(int c) { S; return; } + +#ifdef MS_WIN32 +EXPORT(signed char) __stdcall s_tf_bb(signed char x, signed char c) { S; return c/3; } +EXPORT(unsigned char) __stdcall s_tf_bB(signed char x, unsigned char c) { U; return c/3; } +EXPORT(short) __stdcall s_tf_bh(signed char x, short c) { S; return c/3; } +EXPORT(unsigned short) __stdcall s_tf_bH(signed char x, unsigned short c) { U; return c/3; } +EXPORT(int) __stdcall s_tf_bi(signed char x, int c) { S; return c/3; } +EXPORT(unsigned int) __stdcall s_tf_bI(signed char x, unsigned int c) { U; return c/3; } +EXPORT(long) __stdcall s_tf_bl(signed char x, long c) { S; return c/3; } +EXPORT(unsigned long) __stdcall s_tf_bL(signed char x, unsigned long c) { U; return c/3; } +EXPORT(PY_LONG_LONG) __stdcall s_tf_bq(signed char x, PY_LONG_LONG c) { S; return c/3; } +EXPORT(unsigned PY_LONG_LONG) __stdcall s_tf_bQ(signed char x, unsigned PY_LONG_LONG c) { U; return c/3; } +EXPORT(float) __stdcall s_tf_bf(signed char x, float c) { S; return c/3; } +EXPORT(double) __stdcall s_tf_bd(signed char x, double c) { S; return c/3; } +EXPORT(long double) __stdcall s_tf_bD(signed char x, long double c) { S; return c/3; } +EXPORT(void) __stdcall s_tv_i(int c) { S; return; } +#endif + +/********/ + +#ifndef MS_WIN32 + +typedef struct { + long x; + long y; +} POINT; + +typedef struct { + long left; + long top; + long right; + long bottom; +} RECT; + +#endif + +EXPORT(int) PointInRect(RECT *prc, POINT pt) +{ + if (pt.x < prc->left) + return 0; + if (pt.x > prc->right) + return 0; + if (pt.y < prc->top) + return 0; + if (pt.y > prc->bottom) + return 0; + return 1; +} + +typedef struct { + short x; + short y; +} S2H; + +EXPORT(S2H) ret_2h_func(S2H inp) +{ + inp.x *= 2; + inp.y *= 3; + return inp; +} + +typedef struct { + int a, b, c, d, e, f, g, h; +} S8I; + +EXPORT(S8I) ret_8i_func(S8I inp) +{ + inp.a *= 2; + inp.b *= 3; + inp.c *= 4; + inp.d *= 5; + inp.e *= 6; + inp.f *= 7; + inp.g *= 8; + inp.h *= 9; + return inp; +} + +EXPORT(int) GetRectangle(int flag, RECT *prect) +{ + if (flag == 0) + return 0; + prect->left = (int)flag; + prect->top = (int)flag + 1; + prect->right = (int)flag + 2; + prect->bottom = (int)flag + 3; + return 1; +} + +EXPORT(void) TwoOutArgs(int a, int *pi, int b, int *pj) +{ + *pi += a; + *pj += b; +} + +#ifdef MS_WIN32 +EXPORT(S2H) __stdcall s_ret_2h_func(S2H inp) { return ret_2h_func(inp); } +EXPORT(S8I) __stdcall s_ret_8i_func(S8I inp) { return ret_8i_func(inp); } +#endif + +#ifdef MS_WIN32 +/* Should port this */ +#include +#include + +EXPORT (HRESULT) KeepObject(IUnknown *punk) +{ + static IUnknown *pobj; + if (punk) + punk->lpVtbl->AddRef(punk); + if (pobj) + pobj->lpVtbl->Release(pobj); + pobj = punk; + return S_OK; +} + +#endif + +DL_EXPORT(void) +init_ctypes_test(void) +{ + Py_InitModule("_ctypes_test", module_methods); +} Added: pypy/branch/fast-forward/lib_pypy/_ctypes_test.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/lib_pypy/_ctypes_test.py Fri Nov 5 19:04:58 2010 @@ -0,0 +1,44 @@ +import os, sys +import tempfile + +def compile_shared(): + """Compile '_ctypes_test.c' into an extension module, and import it + """ + thisdir = os.path.dirname(__file__) + output_dir = tempfile.mkdtemp() + + from distutils.ccompiler import new_compiler + compiler = new_compiler() + compiler.output_dir = output_dir + + # Compile .c file + include_dir = os.path.join(thisdir, '..', 'include') + res = compiler.compile([os.path.join(thisdir, '_ctypes_test.c')], + include_dirs=[include_dir]) + object_filename = res[0] + + # set link options + if sys.platform == 'win32': + output_filename = '_ctypes_test.pyd' + # XXX libpypy-c.lib is currently not installed automatically + library = os.path.join(thisdir, '..', 'include', 'libpypy-c') + libraries = [library, 'oleaut32'] + extra_ldargs = ['/MANIFEST'] # needed for VC10 + else: + output_filename = '_ctypes_test.so' + libraries = [] + extra_ldargs = [] + + # link the dynamic library + compiler.link_shared_object( + [object_filename], + output_filename, + libraries=libraries, + extra_preargs=extra_ldargs) + + # Now import the newly created library, it will replace our module in sys.modules + import imp + fp, filename, description = imp.find_module('_ctypes_test', path=[output_dir]) + imp.load_module('_ctypes_test', fp, filename, description) + +compile_shared() From afa at codespeak.net Fri Nov 5 19:09:01 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 19:09:01 +0100 (CET) Subject: [pypy-svn] r78761 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101105180901.B8904282BF7@codespeak.net> Author: afa Date: Fri Nov 5 19:09:00 2010 New Revision: 78761 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py Log: ctypes.Structure._fields_ items can now be 3-tuples: name, ctype, and optional bitfield length. At least don't crash when trying to create such a structure. Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py Fri Nov 5 19:09:00 2010 @@ -12,7 +12,9 @@ size = 0 alignment = 1 pos = [] - for fieldname, ctype in fields: + for field in fields: + fieldname = field[0] + ctype = field[1] fieldsize = ctypes.sizeof(ctype) fieldalignment = ctypes.alignment(ctype) alignment = max(alignment, fieldalignment) @@ -28,24 +30,28 @@ def names_and_fields(_fields_, superclass, zero_offset=False, anon=None, is_union=False): + # _fields_: list of (name, ctype, [optional_bitfield]) if isinstance(_fields_, tuple): _fields_ = list(_fields_) - for _, tp in _fields_: + for f in _fields_: + tp = f[1] if not isinstance(tp, _CDataMeta): raise TypeError("Expected CData subclass, got %s" % (tp,)) import ctypes all_fields = _fields_[:] for cls in inspect.getmro(superclass): all_fields += getattr(cls, '_fields_', []) - names = [name for name, ctype in all_fields] - rawfields = [(name, ctype._ffishape) - for name, ctype in all_fields] + names = [f[0] for f in all_fields] + rawfields = [(f[0], f[1]._ffishape) + for f in all_fields] if not zero_offset: _, _, pos = size_alignment_pos(all_fields, is_union) else: pos = [0] * len(all_fields) fields = {} - for i, (name, ctype) in enumerate(all_fields): + for i, field in enumerate(all_fields): + name = field[0] + ctype = field[1] fields[name] = Field(name, pos[i], ctypes.sizeof(ctype), ctype, i) if anon: resnames = [] From afa at codespeak.net Fri Nov 5 19:23:42 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 19:23:42 +0100 (CET) Subject: [pypy-svn] r78762 - pypy/branch/fast-forward/lib_pypy Message-ID: <20101105182342.EE822282C00@codespeak.net> Author: afa Date: Fri Nov 5 19:23:41 2010 New Revision: 78762 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes_test.py Log: Hack to let test_ctypes.py run to the end Modified: pypy/branch/fast-forward/lib_pypy/_ctypes_test.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes_test.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes_test.py Fri Nov 5 19:23:41 2010 @@ -1,6 +1,13 @@ import os, sys import tempfile +# Monkeypatch & hacks to let ctypes.tests import. +# This should be removed at some point. +sys.getrefcount = None +import _ctypes +_ctypes.PyObj_FromPtr = None +del _ctypes + def compile_shared(): """Compile '_ctypes_test.c' into an extension module, and import it """ From afa at codespeak.net Fri Nov 5 19:35:53 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 19:35:53 +0100 (CET) Subject: [pypy-svn] r78763 - pypy/branch/fast-forward/lib_pypy Message-ID: <20101105183553.C278C282BF0@codespeak.net> Author: afa Date: Fri Nov 5 19:35:52 2010 New Revision: 78763 Added: pypy/branch/fast-forward/lib_pypy/_testcapi.py (contents, props changed) pypy/branch/fast-forward/lib_pypy/_testcapimodule.c (contents, props changed) pypy/branch/fast-forward/lib_pypy/testcapi_long.h (contents, props changed) Log: See how much of the _testcapi module we can support on PyPy Added: pypy/branch/fast-forward/lib_pypy/_testcapi.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/lib_pypy/_testcapi.py Fri Nov 5 19:35:52 2010 @@ -0,0 +1,44 @@ +import os, sys +import tempfile + +def compile_shared(): + """Compile '_testcapi.c' into an extension module, and import it + """ + thisdir = os.path.dirname(__file__) + output_dir = tempfile.mkdtemp() + + from distutils.ccompiler import new_compiler + compiler = new_compiler() + compiler.output_dir = output_dir + + # Compile .c file + include_dir = os.path.join(thisdir, '..', 'include') + res = compiler.compile([os.path.join(thisdir, '_testcapimodule.c')], + include_dirs=[include_dir]) + object_filename = res[0] + + # set link options + if sys.platform == 'win32': + output_filename = '_testcapi.pyd' + # XXX libpypy-c.lib is currently not installed automatically + library = os.path.join(thisdir, '..', 'include', 'libpypy-c') + libraries = [library, 'oleaut32'] + extra_ldargs = ['/MANIFEST'] # needed for VC10 + else: + output_filename = '_testcapi.so' + libraries = [] + extra_ldargs = [] + + # link the dynamic library + compiler.link_shared_object( + [object_filename], + output_filename, + libraries=libraries, + extra_preargs=extra_ldargs) + + # Now import the newly created library, it will replace our module in sys.modules + import imp + fp, filename, description = imp.find_module('_testcapi', path=[output_dir]) + imp.load_module('_testcapi', fp, filename, description) + +compile_shared() Added: pypy/branch/fast-forward/lib_pypy/_testcapimodule.c ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/lib_pypy/_testcapimodule.c Fri Nov 5 19:35:52 2010 @@ -0,0 +1,1898 @@ +/* Verbatim copy of Modules/_testcapimodule.c from CPython 2.7 */ +/* + * C Extension module to test Python interpreter C APIs. + * + * The 'test_*' functions exported by this module are run as part of the + * standard Python regression test, via Lib/test/test_capi.py. + */ + +#include "Python.h" +#include +#include "structmember.h" +#include "datetime.h" + +#ifdef WITH_THREAD +#include "pythread.h" +#endif /* WITH_THREAD */ +static PyObject *TestError; /* set to exception object in init */ + +/* Raise TestError with test_name + ": " + msg, and return NULL. */ + +static PyObject * +raiseTestError(const char* test_name, const char* msg) +{ + char buf[2048]; + + if (strlen(test_name) + strlen(msg) > sizeof(buf) - 50) + PyErr_SetString(TestError, "internal error msg too large"); + else { + PyOS_snprintf(buf, sizeof(buf), "%s: %s", test_name, msg); + PyErr_SetString(TestError, buf); + } + return NULL; +} + +/* Test #defines from pyconfig.h (particularly the SIZEOF_* defines). + + The ones derived from autoconf on the UNIX-like OSes can be relied + upon (in the absence of sloppy cross-compiling), but the Windows + platforms have these hardcoded. Better safe than sorry. +*/ +static PyObject* +sizeof_error(const char* fatname, const char* typname, + int expected, int got) +{ + char buf[1024]; + PyOS_snprintf(buf, sizeof(buf), + "%.200s #define == %d but sizeof(%.200s) == %d", + fatname, expected, typname, got); + PyErr_SetString(TestError, buf); + return (PyObject*)NULL; +} + +static PyObject* +test_config(PyObject *self) +{ +#define CHECK_SIZEOF(FATNAME, TYPE) \ + if (FATNAME != sizeof(TYPE)) \ + return sizeof_error(#FATNAME, #TYPE, FATNAME, sizeof(TYPE)) + + CHECK_SIZEOF(SIZEOF_SHORT, short); + CHECK_SIZEOF(SIZEOF_INT, int); + CHECK_SIZEOF(SIZEOF_LONG, long); + CHECK_SIZEOF(SIZEOF_VOID_P, void*); + CHECK_SIZEOF(SIZEOF_TIME_T, time_t); +#ifdef HAVE_LONG_LONG + CHECK_SIZEOF(SIZEOF_LONG_LONG, PY_LONG_LONG); +#endif + +#undef CHECK_SIZEOF + + Py_INCREF(Py_None); + return Py_None; +} + +static PyObject* +test_list_api(PyObject *self) +{ + PyObject* list; + int i; + + /* SF bug 132008: PyList_Reverse segfaults */ +#define NLIST 30 + list = PyList_New(NLIST); + if (list == (PyObject*)NULL) + return (PyObject*)NULL; + /* list = range(NLIST) */ + for (i = 0; i < NLIST; ++i) { + PyObject* anint = PyInt_FromLong(i); + if (anint == (PyObject*)NULL) { + Py_DECREF(list); + return (PyObject*)NULL; + } + PyList_SET_ITEM(list, i, anint); + } + /* list.reverse(), via PyList_Reverse() */ + i = PyList_Reverse(list); /* should not blow up! */ + if (i != 0) { + Py_DECREF(list); + return (PyObject*)NULL; + } + /* Check that list == range(29, -1, -1) now */ + for (i = 0; i < NLIST; ++i) { + PyObject* anint = PyList_GET_ITEM(list, i); + if (PyInt_AS_LONG(anint) != NLIST-1-i) { + PyErr_SetString(TestError, + "test_list_api: reverse screwed up"); + Py_DECREF(list); + return (PyObject*)NULL; + } + } + Py_DECREF(list); +#undef NLIST + + Py_INCREF(Py_None); + return Py_None; +} + +static int +test_dict_inner(int count) +{ + Py_ssize_t pos = 0, iterations = 0; + int i; + PyObject *dict = PyDict_New(); + PyObject *v, *k; + + if (dict == NULL) + return -1; + + for (i = 0; i < count; i++) { + v = PyInt_FromLong(i); + PyDict_SetItem(dict, v, v); + Py_DECREF(v); + } + + while (PyDict_Next(dict, &pos, &k, &v)) { + PyObject *o; + iterations++; + + i = PyInt_AS_LONG(v) + 1; + o = PyInt_FromLong(i); + if (o == NULL) + return -1; + if (PyDict_SetItem(dict, k, o) < 0) { + Py_DECREF(o); + return -1; + } + Py_DECREF(o); + } + + Py_DECREF(dict); + + if (iterations != count) { + PyErr_SetString( + TestError, + "test_dict_iteration: dict iteration went wrong "); + return -1; + } else { + return 0; + } +} + +static PyObject* +test_dict_iteration(PyObject* self) +{ + int i; + + for (i = 0; i < 200; i++) { + if (test_dict_inner(i) < 0) { + return NULL; + } + } + + Py_INCREF(Py_None); + return Py_None; +} + + +/* Issue #4701: Check that PyObject_Hash implicitly calls + * PyType_Ready if it hasn't already been called + */ +static PyTypeObject _HashInheritanceTester_Type = { + PyObject_HEAD_INIT(NULL) + 0, /* Number of items for varobject */ + "hashinheritancetester", /* Name of this type */ + sizeof(PyObject), /* Basic object size */ + 0, /* Item size for varobject */ + (destructor)PyObject_Del, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + PyObject_GenericGetAttr, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT, /* tp_flags */ + 0, /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + 0, /* tp_methods */ + 0, /* tp_members */ + 0, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + 0, /* tp_init */ + 0, /* tp_alloc */ + PyType_GenericNew, /* tp_new */ +}; + +static PyObject* +test_lazy_hash_inheritance(PyObject* self) +{ + PyTypeObject *type; + PyObject *obj; + long hash; + + type = &_HashInheritanceTester_Type; + + if (type->tp_dict != NULL) + /* The type has already been initialized. This probably means + -R is being used. */ + Py_RETURN_NONE; + + + obj = PyObject_New(PyObject, type); + if (obj == NULL) { + PyErr_Clear(); + PyErr_SetString( + TestError, + "test_lazy_hash_inheritance: failed to create object"); + return NULL; + } + + if (type->tp_dict != NULL) { + PyErr_SetString( + TestError, + "test_lazy_hash_inheritance: type initialised too soon"); + Py_DECREF(obj); + return NULL; + } + + hash = PyObject_Hash(obj); + if ((hash == -1) && PyErr_Occurred()) { + PyErr_Clear(); + PyErr_SetString( + TestError, + "test_lazy_hash_inheritance: could not hash object"); + Py_DECREF(obj); + return NULL; + } + + if (type->tp_dict == NULL) { + PyErr_SetString( + TestError, + "test_lazy_hash_inheritance: type not initialised by hash()"); + Py_DECREF(obj); + return NULL; + } + + if (type->tp_hash != PyType_Type.tp_hash) { + PyErr_SetString( + TestError, + "test_lazy_hash_inheritance: unexpected hash function"); + Py_DECREF(obj); + return NULL; + } + + Py_DECREF(obj); + + Py_RETURN_NONE; +} + + +/* Issue #7385: Check that memoryview() does not crash + * when bf_getbuffer returns an error + */ + +static int +broken_buffer_getbuffer(PyObject *self, Py_buffer *view, int flags) +{ + PyErr_SetString( + TestError, + "test_broken_memoryview: expected error in bf_getbuffer"); + return -1; +} + +static PyBufferProcs memoryviewtester_as_buffer = { + 0, /* bf_getreadbuffer */ + 0, /* bf_getwritebuffer */ + 0, /* bf_getsegcount */ + 0, /* bf_getcharbuffer */ + (getbufferproc)broken_buffer_getbuffer, /* bf_getbuffer */ + 0, /* bf_releasebuffer */ +}; + +static PyTypeObject _MemoryViewTester_Type = { + PyObject_HEAD_INIT(NULL) + 0, /* Number of items for varobject */ + "memoryviewtester", /* Name of this type */ + sizeof(PyObject), /* Basic object size */ + 0, /* Item size for varobject */ + (destructor)PyObject_Del, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + PyObject_GenericGetAttr, /* tp_getattro */ + 0, /* tp_setattro */ + &memoryviewtester_as_buffer, /* tp_as_buffer */ + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_NEWBUFFER, /* tp_flags */ + 0, /* tp_doc */ + 0, /* tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + 0, /* tp_methods */ + 0, /* tp_members */ + 0, /* tp_getset */ + 0, /* tp_base */ + 0, /* tp_dict */ + 0, /* tp_descr_get */ + 0, /* tp_descr_set */ + 0, /* tp_dictoffset */ + 0, /* tp_init */ + 0, /* tp_alloc */ + PyType_GenericNew, /* tp_new */ +}; + +static PyObject* +test_broken_memoryview(PyObject* self) +{ + PyObject *obj = PyObject_New(PyObject, &_MemoryViewTester_Type); + PyObject *res; + + if (obj == NULL) { + PyErr_Clear(); + PyErr_SetString( + TestError, + "test_broken_memoryview: failed to create object"); + return NULL; + } + + res = PyMemoryView_FromObject(obj); + if (res || !PyErr_Occurred()){ + PyErr_SetString( + TestError, + "test_broken_memoryview: memoryview() didn't raise an Exception"); + Py_XDECREF(res); + Py_DECREF(obj); + return NULL; + } + + PyErr_Clear(); + Py_DECREF(obj); + Py_RETURN_NONE; +} + + +/* Tests of PyLong_{As, From}{Unsigned,}Long(), and (#ifdef HAVE_LONG_LONG) + PyLong_{As, From}{Unsigned,}LongLong(). + + Note that the meat of the test is contained in testcapi_long.h. + This is revolting, but delicate code duplication is worse: "almost + exactly the same" code is needed to test PY_LONG_LONG, but the ubiquitous + dependence on type names makes it impossible to use a parameterized + function. A giant macro would be even worse than this. A C++ template + would be perfect. + + The "report an error" functions are deliberately not part of the #include + file: if the test fails, you can set a breakpoint in the appropriate + error function directly, and crawl back from there in the debugger. +*/ + +#define UNBIND(X) Py_DECREF(X); (X) = NULL + +static PyObject * +raise_test_long_error(const char* msg) +{ + return raiseTestError("test_long_api", msg); +} + +#define TESTNAME test_long_api_inner +#define TYPENAME long +#define F_S_TO_PY PyLong_FromLong +#define F_PY_TO_S PyLong_AsLong +#define F_U_TO_PY PyLong_FromUnsignedLong +#define F_PY_TO_U PyLong_AsUnsignedLong + +#include "testcapi_long.h" + +static PyObject * +test_long_api(PyObject* self) +{ + return TESTNAME(raise_test_long_error); +} + +#undef TESTNAME +#undef TYPENAME +#undef F_S_TO_PY +#undef F_PY_TO_S +#undef F_U_TO_PY +#undef F_PY_TO_U + +#ifdef HAVE_LONG_LONG + +static PyObject * +raise_test_longlong_error(const char* msg) +{ + return raiseTestError("test_longlong_api", msg); +} + +#define TESTNAME test_longlong_api_inner +#define TYPENAME PY_LONG_LONG +#define F_S_TO_PY PyLong_FromLongLong +#define F_PY_TO_S PyLong_AsLongLong +#define F_U_TO_PY PyLong_FromUnsignedLongLong +#define F_PY_TO_U PyLong_AsUnsignedLongLong + +#include "testcapi_long.h" + +static PyObject * +test_longlong_api(PyObject* self, PyObject *args) +{ + return TESTNAME(raise_test_longlong_error); +} + +#undef TESTNAME +#undef TYPENAME +#undef F_S_TO_PY +#undef F_PY_TO_S +#undef F_U_TO_PY +#undef F_PY_TO_U + +/* Test the PyLong_AsLongAndOverflow API. General conversion to PY_LONG + is tested by test_long_api_inner. This test will concentrate on proper + handling of overflow. +*/ + +static PyObject * +test_long_and_overflow(PyObject *self) +{ + PyObject *num, *one, *temp; + long value; + int overflow; + + /* Test that overflow is set properly for a large value. */ + /* num is a number larger than LONG_MAX even on 64-bit platforms */ + num = PyLong_FromString("FFFFFFFFFFFFFFFFFFFFFFFF", NULL, 16); + if (num == NULL) + return NULL; + overflow = 1234; + value = PyLong_AsLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != -1) + return raiseTestError("test_long_and_overflow", + "return value was not set to -1"); + if (overflow != 1) + return raiseTestError("test_long_and_overflow", + "overflow was not set to 1"); + + /* Same again, with num = LONG_MAX + 1 */ + num = PyLong_FromLong(LONG_MAX); + if (num == NULL) + return NULL; + one = PyLong_FromLong(1L); + if (one == NULL) { + Py_DECREF(num); + return NULL; + } + temp = PyNumber_Add(num, one); + Py_DECREF(one); + Py_DECREF(num); + num = temp; + if (num == NULL) + return NULL; + overflow = 0; + value = PyLong_AsLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != -1) + return raiseTestError("test_long_and_overflow", + "return value was not set to -1"); + if (overflow != 1) + return raiseTestError("test_long_and_overflow", + "overflow was not set to 1"); + + /* Test that overflow is set properly for a large negative value. */ + /* num is a number smaller than LONG_MIN even on 64-bit platforms */ + num = PyLong_FromString("-FFFFFFFFFFFFFFFFFFFFFFFF", NULL, 16); + if (num == NULL) + return NULL; + overflow = 1234; + value = PyLong_AsLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != -1) + return raiseTestError("test_long_and_overflow", + "return value was not set to -1"); + if (overflow != -1) + return raiseTestError("test_long_and_overflow", + "overflow was not set to -1"); + + /* Same again, with num = LONG_MIN - 1 */ + num = PyLong_FromLong(LONG_MIN); + if (num == NULL) + return NULL; + one = PyLong_FromLong(1L); + if (one == NULL) { + Py_DECREF(num); + return NULL; + } + temp = PyNumber_Subtract(num, one); + Py_DECREF(one); + Py_DECREF(num); + num = temp; + if (num == NULL) + return NULL; + overflow = 0; + value = PyLong_AsLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != -1) + return raiseTestError("test_long_and_overflow", + "return value was not set to -1"); + if (overflow != -1) + return raiseTestError("test_long_and_overflow", + "overflow was not set to -1"); + + /* Test that overflow is cleared properly for small values. */ + num = PyLong_FromString("FF", NULL, 16); + if (num == NULL) + return NULL; + overflow = 1234; + value = PyLong_AsLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != 0xFF) + return raiseTestError("test_long_and_overflow", + "expected return value 0xFF"); + if (overflow != 0) + return raiseTestError("test_long_and_overflow", + "overflow was not cleared"); + + num = PyLong_FromString("-FF", NULL, 16); + if (num == NULL) + return NULL; + overflow = 0; + value = PyLong_AsLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != -0xFF) + return raiseTestError("test_long_and_overflow", + "expected return value 0xFF"); + if (overflow != 0) + return raiseTestError("test_long_and_overflow", + "overflow was set incorrectly"); + + num = PyLong_FromLong(LONG_MAX); + if (num == NULL) + return NULL; + overflow = 1234; + value = PyLong_AsLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != LONG_MAX) + return raiseTestError("test_long_and_overflow", + "expected return value LONG_MAX"); + if (overflow != 0) + return raiseTestError("test_long_and_overflow", + "overflow was not cleared"); + + num = PyLong_FromLong(LONG_MIN); + if (num == NULL) + return NULL; + overflow = 0; + value = PyLong_AsLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != LONG_MIN) + return raiseTestError("test_long_and_overflow", + "expected return value LONG_MIN"); + if (overflow != 0) + return raiseTestError("test_long_and_overflow", + "overflow was not cleared"); + + Py_INCREF(Py_None); + return Py_None; +} + +/* Test the PyLong_AsLongLongAndOverflow API. General conversion to + PY_LONG_LONG is tested by test_long_api_inner. This test will + concentrate on proper handling of overflow. +*/ + +static PyObject * +test_long_long_and_overflow(PyObject *self) +{ + PyObject *num, *one, *temp; + PY_LONG_LONG value; + int overflow; + + /* Test that overflow is set properly for a large value. */ + /* num is a number larger than PY_LLONG_MAX on a typical machine. */ + num = PyLong_FromString("FFFFFFFFFFFFFFFFFFFFFFFF", NULL, 16); + if (num == NULL) + return NULL; + overflow = 1234; + value = PyLong_AsLongLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != -1) + return raiseTestError("test_long_long_and_overflow", + "return value was not set to -1"); + if (overflow != 1) + return raiseTestError("test_long_long_and_overflow", + "overflow was not set to 1"); + + /* Same again, with num = PY_LLONG_MAX + 1 */ + num = PyLong_FromLongLong(PY_LLONG_MAX); + if (num == NULL) + return NULL; + one = PyLong_FromLong(1L); + if (one == NULL) { + Py_DECREF(num); + return NULL; + } + temp = PyNumber_Add(num, one); + Py_DECREF(one); + Py_DECREF(num); + num = temp; + if (num == NULL) + return NULL; + overflow = 0; + value = PyLong_AsLongLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != -1) + return raiseTestError("test_long_long_and_overflow", + "return value was not set to -1"); + if (overflow != 1) + return raiseTestError("test_long_long_and_overflow", + "overflow was not set to 1"); + + /* Test that overflow is set properly for a large negative value. */ + /* num is a number smaller than PY_LLONG_MIN on a typical platform */ + num = PyLong_FromString("-FFFFFFFFFFFFFFFFFFFFFFFF", NULL, 16); + if (num == NULL) + return NULL; + overflow = 1234; + value = PyLong_AsLongLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != -1) + return raiseTestError("test_long_long_and_overflow", + "return value was not set to -1"); + if (overflow != -1) + return raiseTestError("test_long_long_and_overflow", + "overflow was not set to -1"); + + /* Same again, with num = PY_LLONG_MIN - 1 */ + num = PyLong_FromLongLong(PY_LLONG_MIN); + if (num == NULL) + return NULL; + one = PyLong_FromLong(1L); + if (one == NULL) { + Py_DECREF(num); + return NULL; + } + temp = PyNumber_Subtract(num, one); + Py_DECREF(one); + Py_DECREF(num); + num = temp; + if (num == NULL) + return NULL; + overflow = 0; + value = PyLong_AsLongLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != -1) + return raiseTestError("test_long_long_and_overflow", + "return value was not set to -1"); + if (overflow != -1) + return raiseTestError("test_long_long_and_overflow", + "overflow was not set to -1"); + + /* Test that overflow is cleared properly for small values. */ + num = PyLong_FromString("FF", NULL, 16); + if (num == NULL) + return NULL; + overflow = 1234; + value = PyLong_AsLongLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != 0xFF) + return raiseTestError("test_long_long_and_overflow", + "expected return value 0xFF"); + if (overflow != 0) + return raiseTestError("test_long_long_and_overflow", + "overflow was not cleared"); + + num = PyLong_FromString("-FF", NULL, 16); + if (num == NULL) + return NULL; + overflow = 0; + value = PyLong_AsLongLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != -0xFF) + return raiseTestError("test_long_long_and_overflow", + "expected return value 0xFF"); + if (overflow != 0) + return raiseTestError("test_long_long_and_overflow", + "overflow was set incorrectly"); + + num = PyLong_FromLongLong(PY_LLONG_MAX); + if (num == NULL) + return NULL; + overflow = 1234; + value = PyLong_AsLongLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != PY_LLONG_MAX) + return raiseTestError("test_long_long_and_overflow", + "expected return value PY_LLONG_MAX"); + if (overflow != 0) + return raiseTestError("test_long_long_and_overflow", + "overflow was not cleared"); + + num = PyLong_FromLongLong(PY_LLONG_MIN); + if (num == NULL) + return NULL; + overflow = 0; + value = PyLong_AsLongLongAndOverflow(num, &overflow); + Py_DECREF(num); + if (value == -1 && PyErr_Occurred()) + return NULL; + if (value != PY_LLONG_MIN) + return raiseTestError("test_long_long_and_overflow", + "expected return value PY_LLONG_MIN"); + if (overflow != 0) + return raiseTestError("test_long_long_and_overflow", + "overflow was not cleared"); + + Py_INCREF(Py_None); + return Py_None; +} + +/* Test the L code for PyArg_ParseTuple. This should deliver a PY_LONG_LONG + for both long and int arguments. The test may leak a little memory if + it fails. +*/ +static PyObject * +test_L_code(PyObject *self) +{ + PyObject *tuple, *num; + PY_LONG_LONG value; + + tuple = PyTuple_New(1); + if (tuple == NULL) + return NULL; + + num = PyLong_FromLong(42); + if (num == NULL) + return NULL; + + PyTuple_SET_ITEM(tuple, 0, num); + + value = -1; + if (PyArg_ParseTuple(tuple, "L:test_L_code", &value) < 0) + return NULL; + if (value != 42) + return raiseTestError("test_L_code", + "L code returned wrong value for long 42"); + + Py_DECREF(num); + num = PyInt_FromLong(42); + if (num == NULL) + return NULL; + + PyTuple_SET_ITEM(tuple, 0, num); + + value = -1; + if (PyArg_ParseTuple(tuple, "L:test_L_code", &value) < 0) + return NULL; + if (value != 42) + return raiseTestError("test_L_code", + "L code returned wrong value for int 42"); + + Py_DECREF(tuple); + Py_INCREF(Py_None); + return Py_None; +} + +#endif /* ifdef HAVE_LONG_LONG */ + +/* Test tuple argument processing */ +static PyObject * +getargs_tuple(PyObject *self, PyObject *args) +{ + int a, b, c; + if (!PyArg_ParseTuple(args, "i(ii)", &a, &b, &c)) + return NULL; + return Py_BuildValue("iii", a, b, c); +} + +/* test PyArg_ParseTupleAndKeywords */ +static PyObject *getargs_keywords(PyObject *self, PyObject *args, PyObject *kwargs) +{ + static char *keywords[] = {"arg1","arg2","arg3","arg4","arg5", NULL}; + static char *fmt="(ii)i|(i(ii))(iii)i"; + int int_args[10]={-1, -1, -1, -1, -1, -1, -1, -1, -1, -1}; + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, fmt, keywords, + &int_args[0], &int_args[1], &int_args[2], &int_args[3], &int_args[4], + &int_args[5], &int_args[6], &int_args[7], &int_args[8], &int_args[9])) + return NULL; + return Py_BuildValue("iiiiiiiiii", + int_args[0], int_args[1], int_args[2], int_args[3], int_args[4], + int_args[5], int_args[6], int_args[7], int_args[8], int_args[9]); +} + +/* Functions to call PyArg_ParseTuple with integer format codes, + and return the result. +*/ +static PyObject * +getargs_b(PyObject *self, PyObject *args) +{ + unsigned char value; + if (!PyArg_ParseTuple(args, "b", &value)) + return NULL; + return PyLong_FromUnsignedLong((unsigned long)value); +} + +static PyObject * +getargs_B(PyObject *self, PyObject *args) +{ + unsigned char value; + if (!PyArg_ParseTuple(args, "B", &value)) + return NULL; + return PyLong_FromUnsignedLong((unsigned long)value); +} + +static PyObject * +getargs_h(PyObject *self, PyObject *args) +{ + short value; + if (!PyArg_ParseTuple(args, "h", &value)) + return NULL; + return PyLong_FromLong((long)value); +} + +static PyObject * +getargs_H(PyObject *self, PyObject *args) +{ + unsigned short value; + if (!PyArg_ParseTuple(args, "H", &value)) + return NULL; + return PyLong_FromUnsignedLong((unsigned long)value); +} + +static PyObject * +getargs_I(PyObject *self, PyObject *args) +{ + unsigned int value; + if (!PyArg_ParseTuple(args, "I", &value)) + return NULL; + return PyLong_FromUnsignedLong((unsigned long)value); +} + +static PyObject * +getargs_k(PyObject *self, PyObject *args) +{ + unsigned long value; + if (!PyArg_ParseTuple(args, "k", &value)) + return NULL; + return PyLong_FromUnsignedLong(value); +} + +static PyObject * +getargs_i(PyObject *self, PyObject *args) +{ + int value; + if (!PyArg_ParseTuple(args, "i", &value)) + return NULL; + return PyLong_FromLong((long)value); +} + +static PyObject * +getargs_l(PyObject *self, PyObject *args) +{ + long value; + if (!PyArg_ParseTuple(args, "l", &value)) + return NULL; + return PyLong_FromLong(value); +} + +static PyObject * +getargs_n(PyObject *self, PyObject *args) +{ + Py_ssize_t value; + if (!PyArg_ParseTuple(args, "n", &value)) + return NULL; + return PyInt_FromSsize_t(value); +} + +#ifdef HAVE_LONG_LONG +static PyObject * +getargs_L(PyObject *self, PyObject *args) +{ + PY_LONG_LONG value; + if (!PyArg_ParseTuple(args, "L", &value)) + return NULL; + return PyLong_FromLongLong(value); +} + +static PyObject * +getargs_K(PyObject *self, PyObject *args) +{ + unsigned PY_LONG_LONG value; + if (!PyArg_ParseTuple(args, "K", &value)) + return NULL; + return PyLong_FromUnsignedLongLong(value); +} +#endif + +/* This function not only tests the 'k' getargs code, but also the + PyInt_AsUnsignedLongMask() and PyInt_AsUnsignedLongMask() functions. */ +static PyObject * +test_k_code(PyObject *self) +{ + PyObject *tuple, *num; + unsigned long value; + + tuple = PyTuple_New(1); + if (tuple == NULL) + return NULL; + + /* a number larger than ULONG_MAX even on 64-bit platforms */ + num = PyLong_FromString("FFFFFFFFFFFFFFFFFFFFFFFF", NULL, 16); + if (num == NULL) + return NULL; + + value = PyInt_AsUnsignedLongMask(num); + if (value != ULONG_MAX) + return raiseTestError("test_k_code", + "PyInt_AsUnsignedLongMask() returned wrong value for long 0xFFF...FFF"); + + PyTuple_SET_ITEM(tuple, 0, num); + + value = 0; + if (PyArg_ParseTuple(tuple, "k:test_k_code", &value) < 0) + return NULL; + if (value != ULONG_MAX) + return raiseTestError("test_k_code", + "k code returned wrong value for long 0xFFF...FFF"); + + Py_DECREF(num); + num = PyLong_FromString("-FFFFFFFF000000000000000042", NULL, 16); + if (num == NULL) + return NULL; + + value = PyInt_AsUnsignedLongMask(num); + if (value != (unsigned long)-0x42) + return raiseTestError("test_k_code", + "PyInt_AsUnsignedLongMask() returned wrong value for long 0xFFF...FFF"); + + PyTuple_SET_ITEM(tuple, 0, num); + + value = 0; + if (PyArg_ParseTuple(tuple, "k:test_k_code", &value) < 0) + return NULL; + if (value != (unsigned long)-0x42) + return raiseTestError("test_k_code", + "k code returned wrong value for long -0xFFF..000042"); + + Py_DECREF(tuple); + Py_INCREF(Py_None); + return Py_None; +} + +#ifdef Py_USING_UNICODE + +static volatile int x; + +/* Test the u and u# codes for PyArg_ParseTuple. May leak memory in case + of an error. +*/ +static PyObject * +test_u_code(PyObject *self) +{ + PyObject *tuple, *obj; + Py_UNICODE *value; + int len; + + /* issue4122: Undefined reference to _Py_ascii_whitespace on Windows */ + /* Just use the macro and check that it compiles */ + x = Py_UNICODE_ISSPACE(25); + + tuple = PyTuple_New(1); + if (tuple == NULL) + return NULL; + + obj = PyUnicode_Decode("test", strlen("test"), + "ascii", NULL); + if (obj == NULL) + return NULL; + + PyTuple_SET_ITEM(tuple, 0, obj); + + value = 0; + if (PyArg_ParseTuple(tuple, "u:test_u_code", &value) < 0) + return NULL; + if (value != PyUnicode_AS_UNICODE(obj)) + return raiseTestError("test_u_code", + "u code returned wrong value for u'test'"); + value = 0; + if (PyArg_ParseTuple(tuple, "u#:test_u_code", &value, &len) < 0) + return NULL; + if (value != PyUnicode_AS_UNICODE(obj) || + len != PyUnicode_GET_SIZE(obj)) + return raiseTestError("test_u_code", + "u# code returned wrong values for u'test'"); + + Py_DECREF(tuple); + Py_INCREF(Py_None); + return Py_None; +} + +static PyObject * +test_widechar(PyObject *self) +{ +#if defined(SIZEOF_WCHAR_T) && (SIZEOF_WCHAR_T == 4) + const wchar_t wtext[2] = {(wchar_t)0x10ABCDu}; + size_t wtextlen = 1; +#else + const wchar_t wtext[3] = {(wchar_t)0xDBEAu, (wchar_t)0xDFCDu}; + size_t wtextlen = 2; +#endif + PyObject *wide, *utf8; + + wide = PyUnicode_FromWideChar(wtext, wtextlen); + if (wide == NULL) + return NULL; + + utf8 = PyUnicode_FromString("\xf4\x8a\xaf\x8d"); + if (utf8 == NULL) { + Py_DECREF(wide); + return NULL; + } + + if (PyUnicode_GET_SIZE(wide) != PyUnicode_GET_SIZE(utf8)) { + Py_DECREF(wide); + Py_DECREF(utf8); + return raiseTestError("test_widechar", + "wide string and utf8 string have different length"); + } + if (PyUnicode_Compare(wide, utf8)) { + Py_DECREF(wide); + Py_DECREF(utf8); + if (PyErr_Occurred()) + return NULL; + return raiseTestError("test_widechar", + "wide string and utf8 string are differents"); + } + + Py_DECREF(wide); + Py_DECREF(utf8); + Py_RETURN_NONE; +} + +static PyObject * +test_empty_argparse(PyObject *self) +{ + /* Test that formats can begin with '|'. See issue #4720. */ + PyObject *tuple, *dict = NULL; + static char *kwlist[] = {NULL}; + int result; + tuple = PyTuple_New(0); + if (!tuple) + return NULL; + if ((result = PyArg_ParseTuple(tuple, "|:test_empty_argparse")) < 0) + goto done; + dict = PyDict_New(); + if (!dict) + goto done; + result = PyArg_ParseTupleAndKeywords(tuple, dict, "|:test_empty_argparse", kwlist); + done: + Py_DECREF(tuple); + Py_XDECREF(dict); + if (result < 0) + return NULL; + else { + Py_RETURN_NONE; + } +} + +static PyObject * +codec_incrementalencoder(PyObject *self, PyObject *args) +{ + const char *encoding, *errors = NULL; + if (!PyArg_ParseTuple(args, "s|s:test_incrementalencoder", + &encoding, &errors)) + return NULL; + return PyCodec_IncrementalEncoder(encoding, errors); +} + +static PyObject * +codec_incrementaldecoder(PyObject *self, PyObject *args) +{ + const char *encoding, *errors = NULL; + if (!PyArg_ParseTuple(args, "s|s:test_incrementaldecoder", + &encoding, &errors)) + return NULL; + return PyCodec_IncrementalDecoder(encoding, errors); +} + +#endif + +/* Simple test of _PyLong_NumBits and _PyLong_Sign. */ +static PyObject * +test_long_numbits(PyObject *self) +{ + struct triple { + long input; + size_t nbits; + int sign; + } testcases[] = {{0, 0, 0}, + {1L, 1, 1}, + {-1L, 1, -1}, + {2L, 2, 1}, + {-2L, 2, -1}, + {3L, 2, 1}, + {-3L, 2, -1}, + {4L, 3, 1}, + {-4L, 3, -1}, + {0x7fffL, 15, 1}, /* one Python long digit */ + {-0x7fffL, 15, -1}, + {0xffffL, 16, 1}, + {-0xffffL, 16, -1}, + {0xfffffffL, 28, 1}, + {-0xfffffffL, 28, -1}}; + int i; + + for (i = 0; i < sizeof(testcases) / sizeof(struct triple); ++i) { + PyObject *plong = PyLong_FromLong(testcases[i].input); + size_t nbits = _PyLong_NumBits(plong); + int sign = _PyLong_Sign(plong); + + Py_DECREF(plong); + if (nbits != testcases[i].nbits) + return raiseTestError("test_long_numbits", + "wrong result for _PyLong_NumBits"); + if (sign != testcases[i].sign) + return raiseTestError("test_long_numbits", + "wrong result for _PyLong_Sign"); + } + Py_INCREF(Py_None); + return Py_None; +} + +/* Example passing NULLs to PyObject_Str(NULL) and PyObject_Unicode(NULL). */ + +static PyObject * +test_null_strings(PyObject *self) +{ + PyObject *o1 = PyObject_Str(NULL), *o2 = PyObject_Unicode(NULL); + PyObject *tuple = PyTuple_Pack(2, o1, o2); + Py_XDECREF(o1); + Py_XDECREF(o2); + return tuple; +} + +static PyObject * +raise_exception(PyObject *self, PyObject *args) +{ + PyObject *exc; + PyObject *exc_args, *v; + int num_args, i; + + if (!PyArg_ParseTuple(args, "Oi:raise_exception", + &exc, &num_args)) + return NULL; + if (!PyExceptionClass_Check(exc)) { + PyErr_Format(PyExc_TypeError, "an exception class is required"); + return NULL; + } + + exc_args = PyTuple_New(num_args); + if (exc_args == NULL) + return NULL; + for (i = 0; i < num_args; ++i) { + v = PyInt_FromLong(i); + if (v == NULL) { + Py_DECREF(exc_args); + return NULL; + } + PyTuple_SET_ITEM(exc_args, i, v); + } + PyErr_SetObject(exc, exc_args); + Py_DECREF(exc_args); + return NULL; +} + + +static int test_run_counter = 0; + +static PyObject * +test_datetime_capi(PyObject *self, PyObject *args) { + if (PyDateTimeAPI) { + if (test_run_counter) { + /* Probably regrtest.py -R */ + Py_RETURN_NONE; + } + else { + PyErr_SetString(PyExc_AssertionError, + "PyDateTime_CAPI somehow initialized"); + return NULL; + } + } + test_run_counter++; + PyDateTime_IMPORT; + if (PyDateTimeAPI) + Py_RETURN_NONE; + else + return NULL; +} + + +#ifdef WITH_THREAD + +/* test_thread_state spawns a thread of its own, and that thread releases + * `thread_done` when it's finished. The driver code has to know when the + * thread finishes, because the thread uses a PyObject (the callable) that + * may go away when the driver finishes. The former lack of this explicit + * synchronization caused rare segfaults, so rare that they were seen only + * on a Mac buildbot (although they were possible on any box). + */ +static PyThread_type_lock thread_done = NULL; + +static int +_make_call(void *callable) +{ + PyObject *rc; + int success; + PyGILState_STATE s = PyGILState_Ensure(); + rc = PyObject_CallFunction((PyObject *)callable, ""); + success = (rc != NULL); + Py_XDECREF(rc); + PyGILState_Release(s); + return success; +} + +/* Same thing, but releases `thread_done` when it returns. This variant + * should be called only from threads spawned by test_thread_state(). + */ +static void +_make_call_from_thread(void *callable) +{ + _make_call(callable); + PyThread_release_lock(thread_done); +} + +static PyObject * +test_thread_state(PyObject *self, PyObject *args) +{ + PyObject *fn; + int success = 1; + + if (!PyArg_ParseTuple(args, "O:test_thread_state", &fn)) + return NULL; + + if (!PyCallable_Check(fn)) { + PyErr_Format(PyExc_TypeError, "'%s' object is not callable", + fn->ob_type->tp_name); + return NULL; + } + + /* Ensure Python is set up for threading */ + PyEval_InitThreads(); + thread_done = PyThread_allocate_lock(); + if (thread_done == NULL) + return PyErr_NoMemory(); + PyThread_acquire_lock(thread_done, 1); + + /* Start a new thread with our callback. */ + PyThread_start_new_thread(_make_call_from_thread, fn); + /* Make the callback with the thread lock held by this thread */ + success &= _make_call(fn); + /* Do it all again, but this time with the thread-lock released */ + Py_BEGIN_ALLOW_THREADS + success &= _make_call(fn); + PyThread_acquire_lock(thread_done, 1); /* wait for thread to finish */ + Py_END_ALLOW_THREADS + + /* And once more with and without a thread + XXX - should use a lock and work out exactly what we are trying + to test + */ + Py_BEGIN_ALLOW_THREADS + PyThread_start_new_thread(_make_call_from_thread, fn); + success &= _make_call(fn); + PyThread_acquire_lock(thread_done, 1); /* wait for thread to finish */ + Py_END_ALLOW_THREADS + + /* Release lock we acquired above. This is required on HP-UX. */ + PyThread_release_lock(thread_done); + + PyThread_free_lock(thread_done); + if (!success) + return NULL; + Py_RETURN_NONE; +} + +/* test Py_AddPendingCalls using threads */ +static int _pending_callback(void *arg) +{ + /* we assume the argument is callable object to which we own a reference */ + PyObject *callable = (PyObject *)arg; + PyObject *r = PyObject_CallObject(callable, NULL); + Py_DECREF(callable); + Py_XDECREF(r); + return r != NULL ? 0 : -1; +} + +/* The following requests n callbacks to _pending_callback. It can be + * run from any python thread. + */ +PyObject *pending_threadfunc(PyObject *self, PyObject *arg) +{ + PyObject *callable; + int r; + if (PyArg_ParseTuple(arg, "O", &callable) == 0) + return NULL; + + /* create the reference for the callbackwhile we hold the lock */ + Py_INCREF(callable); + + Py_BEGIN_ALLOW_THREADS + r = Py_AddPendingCall(&_pending_callback, callable); + Py_END_ALLOW_THREADS + + if (r<0) { + Py_DECREF(callable); /* unsuccessful add, destroy the extra reference */ + Py_INCREF(Py_False); + return Py_False; + } + Py_INCREF(Py_True); + return Py_True; +} +#endif + +/* Some tests of PyString_FromFormat(). This needs more tests. */ +static PyObject * +test_string_from_format(PyObject *self, PyObject *args) +{ + PyObject *result; + char *msg; + +#define CHECK_1_FORMAT(FORMAT, TYPE) \ + result = PyString_FromFormat(FORMAT, (TYPE)1); \ + if (result == NULL) \ + return NULL; \ + if (strcmp(PyString_AsString(result), "1")) { \ + msg = FORMAT " failed at 1"; \ + goto Fail; \ + } \ + Py_DECREF(result) + + CHECK_1_FORMAT("%d", int); + CHECK_1_FORMAT("%ld", long); + /* The z width modifier was added in Python 2.5. */ + CHECK_1_FORMAT("%zd", Py_ssize_t); + + /* The u type code was added in Python 2.5. */ + CHECK_1_FORMAT("%u", unsigned int); + CHECK_1_FORMAT("%lu", unsigned long); + CHECK_1_FORMAT("%zu", size_t); + + /* "%lld" and "%llu" support added in Python 2.7. */ +#ifdef HAVE_LONG_LONG + CHECK_1_FORMAT("%llu", unsigned PY_LONG_LONG); + CHECK_1_FORMAT("%lld", PY_LONG_LONG); +#endif + + Py_RETURN_NONE; + + Fail: + Py_XDECREF(result); + return raiseTestError("test_string_from_format", msg); + +#undef CHECK_1_FORMAT +} + +/* Coverage testing of capsule objects. */ + +static const char *capsule_name = "capsule name"; +static char *capsule_pointer = "capsule pointer"; +static char *capsule_context = "capsule context"; +static const char *capsule_error = NULL; +static int +capsule_destructor_call_count = 0; + +static void +capsule_destructor(PyObject *o) { + capsule_destructor_call_count++; + if (PyCapsule_GetContext(o) != capsule_context) { + capsule_error = "context did not match in destructor!"; + } else if (PyCapsule_GetDestructor(o) != capsule_destructor) { + capsule_error = "destructor did not match in destructor! (woah!)"; + } else if (PyCapsule_GetName(o) != capsule_name) { + capsule_error = "name did not match in destructor!"; + } else if (PyCapsule_GetPointer(o, capsule_name) != capsule_pointer) { + capsule_error = "pointer did not match in destructor!"; + } +} + +typedef struct { + char *name; + char *module; + char *attribute; +} known_capsule; + +static PyObject * +test_capsule(PyObject *self, PyObject *args) +{ + PyObject *object; + const char *error = NULL; + void *pointer; + void *pointer2; + known_capsule known_capsules[] = { + #define KNOWN_CAPSULE(module, name) { module "." name, module, name } + KNOWN_CAPSULE("_socket", "CAPI"), + KNOWN_CAPSULE("_curses", "_C_API"), + KNOWN_CAPSULE("datetime", "datetime_CAPI"), + { NULL, NULL }, + }; + known_capsule *known = &known_capsules[0]; + +#define FAIL(x) { error = (x); goto exit; } + +#define CHECK_DESTRUCTOR \ + if (capsule_error) { \ + FAIL(capsule_error); \ + } \ + else if (!capsule_destructor_call_count) { \ + FAIL("destructor not called!"); \ + } \ + capsule_destructor_call_count = 0; \ + + object = PyCapsule_New(capsule_pointer, capsule_name, capsule_destructor); + PyCapsule_SetContext(object, capsule_context); + capsule_destructor(object); + CHECK_DESTRUCTOR; + Py_DECREF(object); + CHECK_DESTRUCTOR; + + object = PyCapsule_New(known, "ignored", NULL); + PyCapsule_SetPointer(object, capsule_pointer); + PyCapsule_SetName(object, capsule_name); + PyCapsule_SetDestructor(object, capsule_destructor); + PyCapsule_SetContext(object, capsule_context); + capsule_destructor(object); + CHECK_DESTRUCTOR; + /* intentionally access using the wrong name */ + pointer2 = PyCapsule_GetPointer(object, "the wrong name"); + if (!PyErr_Occurred()) { + FAIL("PyCapsule_GetPointer should have failed but did not!"); + } + PyErr_Clear(); + if (pointer2) { + if (pointer2 == capsule_pointer) { + FAIL("PyCapsule_GetPointer should not have" + " returned the internal pointer!"); + } else { + FAIL("PyCapsule_GetPointer should have " + "returned NULL pointer but did not!"); + } + } + PyCapsule_SetDestructor(object, NULL); + Py_DECREF(object); + if (capsule_destructor_call_count) { + FAIL("destructor called when it should not have been!"); + } + + for (known = &known_capsules[0]; known->module != NULL; known++) { + /* yeah, ordinarily I wouldn't do this either, + but it's fine for this test harness. + */ + static char buffer[256]; +#undef FAIL +#define FAIL(x) \ + { \ + sprintf(buffer, "%s module: \"%s\" attribute: \"%s\"", \ + x, known->module, known->attribute); \ + error = buffer; \ + goto exit; \ + } \ + + PyObject *module = PyImport_ImportModule(known->module); + if (module) { + pointer = PyCapsule_Import(known->name, 0); + if (!pointer) { + Py_DECREF(module); + FAIL("PyCapsule_GetPointer returned NULL unexpectedly!"); + } + object = PyObject_GetAttrString(module, known->attribute); + if (!object) { + Py_DECREF(module); + return NULL; + } + pointer2 = PyCapsule_GetPointer(object, + "weebles wobble but they don't fall down"); + if (!PyErr_Occurred()) { + Py_DECREF(object); + Py_DECREF(module); + FAIL("PyCapsule_GetPointer should have failed but did not!"); + } + PyErr_Clear(); + if (pointer2) { + Py_DECREF(module); + Py_DECREF(object); + if (pointer2 == pointer) { + FAIL("PyCapsule_GetPointer should not have" + " returned its internal pointer!"); + } else { + FAIL("PyCapsule_GetPointer should have" + " returned NULL pointer but did not!"); + } + } + Py_DECREF(object); + Py_DECREF(module); + } + else + PyErr_Clear(); + } + + exit: + if (error) { + return raiseTestError("test_capsule", error); + } + Py_RETURN_NONE; +#undef FAIL +} + +/* This is here to provide a docstring for test_descr. */ +static PyObject * +test_with_docstring(PyObject *self) +{ + Py_RETURN_NONE; +} + +/* To test the format of tracebacks as printed out. */ +static PyObject * +traceback_print(PyObject *self, PyObject *args) +{ + PyObject *file; + PyObject *traceback; + int result; + + if (!PyArg_ParseTuple(args, "OO:traceback_print", + &traceback, &file)) + return NULL; + + result = PyTraceBack_Print(traceback, file); + if (result < 0) + return NULL; + Py_RETURN_NONE; +} + +/* To test that the result of PyCode_NewEmpty has the right members. */ +static PyObject * +code_newempty(PyObject *self, PyObject *args) +{ + const char *filename; + const char *funcname; + int firstlineno; + + if (!PyArg_ParseTuple(args, "ssi:code_newempty", + &filename, &funcname, &firstlineno)) + return NULL; + + return (PyObject *)PyCode_NewEmpty(filename, funcname, firstlineno); +} + +/* Test PyErr_NewExceptionWithDoc (also exercise PyErr_NewException). + Run via Lib/test/test_exceptions.py */ +static PyObject * +make_exception_with_doc(PyObject *self, PyObject *args, PyObject *kwargs) +{ + char *name; + char *doc = NULL; + PyObject *base = NULL; + PyObject *dict = NULL; + + static char *kwlist[] = {"name", "doc", "base", "dict", NULL}; + + if (!PyArg_ParseTupleAndKeywords(args, kwargs, + "s|sOO:make_exception_with_doc", kwlist, + &name, &doc, &base, &dict)) + return NULL; + + return PyErr_NewExceptionWithDoc(name, doc, base, dict); +} + +static PyMethodDef TestMethods[] = { + {"raise_exception", raise_exception, METH_VARARGS}, + {"test_config", (PyCFunction)test_config, METH_NOARGS}, + {"test_datetime_capi", test_datetime_capi, METH_NOARGS}, + {"test_list_api", (PyCFunction)test_list_api, METH_NOARGS}, + {"test_dict_iteration", (PyCFunction)test_dict_iteration,METH_NOARGS}, + {"test_lazy_hash_inheritance", (PyCFunction)test_lazy_hash_inheritance,METH_NOARGS}, + {"test_broken_memoryview", (PyCFunction)test_broken_memoryview,METH_NOARGS}, + {"test_long_api", (PyCFunction)test_long_api, METH_NOARGS}, + {"test_long_and_overflow", (PyCFunction)test_long_and_overflow, + METH_NOARGS}, + {"test_long_numbits", (PyCFunction)test_long_numbits, METH_NOARGS}, + {"test_k_code", (PyCFunction)test_k_code, METH_NOARGS}, + {"test_empty_argparse", (PyCFunction)test_empty_argparse,METH_NOARGS}, + {"test_null_strings", (PyCFunction)test_null_strings, METH_NOARGS}, + {"test_string_from_format", (PyCFunction)test_string_from_format, METH_NOARGS}, + {"test_with_docstring", (PyCFunction)test_with_docstring, METH_NOARGS, + PyDoc_STR("This is a pretty normal docstring.")}, + + {"getargs_tuple", getargs_tuple, METH_VARARGS}, + {"getargs_keywords", (PyCFunction)getargs_keywords, + METH_VARARGS|METH_KEYWORDS}, + {"getargs_b", getargs_b, METH_VARARGS}, + {"getargs_B", getargs_B, METH_VARARGS}, + {"getargs_h", getargs_h, METH_VARARGS}, + {"getargs_H", getargs_H, METH_VARARGS}, + {"getargs_I", getargs_I, METH_VARARGS}, + {"getargs_k", getargs_k, METH_VARARGS}, + {"getargs_i", getargs_i, METH_VARARGS}, + {"getargs_l", getargs_l, METH_VARARGS}, + {"getargs_n", getargs_n, METH_VARARGS}, +#ifdef HAVE_LONG_LONG + {"getargs_L", getargs_L, METH_VARARGS}, + {"getargs_K", getargs_K, METH_VARARGS}, + {"test_longlong_api", test_longlong_api, METH_NOARGS}, + {"test_long_long_and_overflow", + (PyCFunction)test_long_long_and_overflow, METH_NOARGS}, + {"test_L_code", (PyCFunction)test_L_code, METH_NOARGS}, + {"codec_incrementalencoder", + (PyCFunction)codec_incrementalencoder, METH_VARARGS}, + {"codec_incrementaldecoder", + (PyCFunction)codec_incrementaldecoder, METH_VARARGS}, +#endif +#ifdef Py_USING_UNICODE + {"test_u_code", (PyCFunction)test_u_code, METH_NOARGS}, + {"test_widechar", (PyCFunction)test_widechar, METH_NOARGS}, +#endif +#ifdef WITH_THREAD + {"_test_thread_state", test_thread_state, METH_VARARGS}, + {"_pending_threadfunc", pending_threadfunc, METH_VARARGS}, +#endif + {"test_capsule", (PyCFunction)test_capsule, METH_NOARGS}, + {"traceback_print", traceback_print, METH_VARARGS}, + {"code_newempty", code_newempty, METH_VARARGS}, + {"make_exception_with_doc", (PyCFunction)make_exception_with_doc, + METH_VARARGS | METH_KEYWORDS}, + {NULL, NULL} /* sentinel */ +}; + +#define AddSym(d, n, f, v) {PyObject *o = f(v); PyDict_SetItemString(d, n, o); Py_DECREF(o);} + +typedef struct { + char bool_member; + char byte_member; + unsigned char ubyte_member; + short short_member; + unsigned short ushort_member; + int int_member; + unsigned int uint_member; + long long_member; + unsigned long ulong_member; + float float_member; + double double_member; + char inplace_member[6]; +#ifdef HAVE_LONG_LONG + PY_LONG_LONG longlong_member; + unsigned PY_LONG_LONG ulonglong_member; +#endif +} all_structmembers; + +typedef struct { + PyObject_HEAD + all_structmembers structmembers; +} test_structmembers; + +static struct PyMemberDef test_members[] = { + {"T_BOOL", T_BOOL, offsetof(test_structmembers, structmembers.bool_member), 0, NULL}, + {"T_BYTE", T_BYTE, offsetof(test_structmembers, structmembers.byte_member), 0, NULL}, + {"T_UBYTE", T_UBYTE, offsetof(test_structmembers, structmembers.ubyte_member), 0, NULL}, + {"T_SHORT", T_SHORT, offsetof(test_structmembers, structmembers.short_member), 0, NULL}, + {"T_USHORT", T_USHORT, offsetof(test_structmembers, structmembers.ushort_member), 0, NULL}, + {"T_INT", T_INT, offsetof(test_structmembers, structmembers.int_member), 0, NULL}, + {"T_UINT", T_UINT, offsetof(test_structmembers, structmembers.uint_member), 0, NULL}, + {"T_LONG", T_LONG, offsetof(test_structmembers, structmembers.long_member), 0, NULL}, + {"T_ULONG", T_ULONG, offsetof(test_structmembers, structmembers.ulong_member), 0, NULL}, + {"T_FLOAT", T_FLOAT, offsetof(test_structmembers, structmembers.float_member), 0, NULL}, + {"T_DOUBLE", T_DOUBLE, offsetof(test_structmembers, structmembers.double_member), 0, NULL}, + {"T_STRING_INPLACE", T_STRING_INPLACE, offsetof(test_structmembers, structmembers.inplace_member), 0, NULL}, +#ifdef HAVE_LONG_LONG + {"T_LONGLONG", T_LONGLONG, offsetof(test_structmembers, structmembers.longlong_member), 0, NULL}, + {"T_ULONGLONG", T_ULONGLONG, offsetof(test_structmembers, structmembers.ulonglong_member), 0, NULL}, +#endif + {NULL} +}; + + +static PyObject * +test_structmembers_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) +{ + static char *keywords[] = { + "T_BOOL", "T_BYTE", "T_UBYTE", "T_SHORT", "T_USHORT", + "T_INT", "T_UINT", "T_LONG", "T_ULONG", + "T_FLOAT", "T_DOUBLE", "T_STRING_INPLACE", +#ifdef HAVE_LONG_LONG + "T_LONGLONG", "T_ULONGLONG", +#endif + NULL}; + static char *fmt = "|bbBhHiIlkfds#" +#ifdef HAVE_LONG_LONG + "LK" +#endif + ; + test_structmembers *ob; + const char *s = NULL; + Py_ssize_t string_len = 0; + ob = PyObject_New(test_structmembers, type); + if (ob == NULL) + return NULL; + memset(&ob->structmembers, 0, sizeof(all_structmembers)); + if (!PyArg_ParseTupleAndKeywords(args, kwargs, fmt, keywords, + &ob->structmembers.bool_member, + &ob->structmembers.byte_member, + &ob->structmembers.ubyte_member, + &ob->structmembers.short_member, + &ob->structmembers.ushort_member, + &ob->structmembers.int_member, + &ob->structmembers.uint_member, + &ob->structmembers.long_member, + &ob->structmembers.ulong_member, + &ob->structmembers.float_member, + &ob->structmembers.double_member, + &s, &string_len +#ifdef HAVE_LONG_LONG + , &ob->structmembers.longlong_member, + &ob->structmembers.ulonglong_member +#endif + )) { + Py_DECREF(ob); + return NULL; + } + if (s != NULL) { + if (string_len > 5) { + Py_DECREF(ob); + PyErr_SetString(PyExc_ValueError, "string too long"); + return NULL; + } + strcpy(ob->structmembers.inplace_member, s); + } + else { + strcpy(ob->structmembers.inplace_member, ""); + } + return (PyObject *)ob; +} + +static void +test_structmembers_free(PyObject *ob) +{ + PyObject_FREE(ob); +} + +static PyTypeObject test_structmembersType = { + PyVarObject_HEAD_INIT(NULL, 0) + "test_structmembersType", + sizeof(test_structmembers), /* tp_basicsize */ + 0, /* tp_itemsize */ + test_structmembers_free, /* destructor tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + PyObject_GenericGetAttr, /* tp_getattro */ + PyObject_GenericSetAttr, /* tp_setattro */ + 0, /* tp_as_buffer */ + 0, /* tp_flags */ + "Type containing all structmember types", + 0, /* traverseproc tp_traverse */ + 0, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + 0, /* tp_methods */ + test_members, /* tp_members */ + 0, + 0, + 0, + 0, + 0, + 0, + 0, + 0, + test_structmembers_new, /* tp_new */ +}; + + +PyMODINIT_FUNC +init_testcapi(void) +{ + PyObject *m; + + m = Py_InitModule("_testcapi", TestMethods); + if (m == NULL) + return; + + Py_TYPE(&_HashInheritanceTester_Type)=&PyType_Type; + + Py_TYPE(&test_structmembersType)=&PyType_Type; + Py_INCREF(&test_structmembersType); + /* don't use a name starting with "test", since we don't want + test_capi to automatically call this */ + PyModule_AddObject(m, "_test_structmembersType", (PyObject *)&test_structmembersType); + + PyModule_AddObject(m, "CHAR_MAX", PyInt_FromLong(CHAR_MAX)); + PyModule_AddObject(m, "CHAR_MIN", PyInt_FromLong(CHAR_MIN)); + PyModule_AddObject(m, "UCHAR_MAX", PyInt_FromLong(UCHAR_MAX)); + PyModule_AddObject(m, "SHRT_MAX", PyInt_FromLong(SHRT_MAX)); + PyModule_AddObject(m, "SHRT_MIN", PyInt_FromLong(SHRT_MIN)); + PyModule_AddObject(m, "USHRT_MAX", PyInt_FromLong(USHRT_MAX)); + PyModule_AddObject(m, "INT_MAX", PyLong_FromLong(INT_MAX)); + PyModule_AddObject(m, "INT_MIN", PyLong_FromLong(INT_MIN)); + PyModule_AddObject(m, "UINT_MAX", PyLong_FromUnsignedLong(UINT_MAX)); + PyModule_AddObject(m, "LONG_MAX", PyInt_FromLong(LONG_MAX)); + PyModule_AddObject(m, "LONG_MIN", PyInt_FromLong(LONG_MIN)); + PyModule_AddObject(m, "ULONG_MAX", PyLong_FromUnsignedLong(ULONG_MAX)); + PyModule_AddObject(m, "FLT_MAX", PyFloat_FromDouble(FLT_MAX)); + PyModule_AddObject(m, "FLT_MIN", PyFloat_FromDouble(FLT_MIN)); + PyModule_AddObject(m, "DBL_MAX", PyFloat_FromDouble(DBL_MAX)); + PyModule_AddObject(m, "DBL_MIN", PyFloat_FromDouble(DBL_MIN)); + PyModule_AddObject(m, "LLONG_MAX", PyLong_FromLongLong(PY_LLONG_MAX)); + PyModule_AddObject(m, "LLONG_MIN", PyLong_FromLongLong(PY_LLONG_MIN)); + PyModule_AddObject(m, "ULLONG_MAX", PyLong_FromUnsignedLongLong(PY_ULLONG_MAX)); + PyModule_AddObject(m, "PY_SSIZE_T_MAX", PyInt_FromSsize_t(PY_SSIZE_T_MAX)); + PyModule_AddObject(m, "PY_SSIZE_T_MIN", PyInt_FromSsize_t(PY_SSIZE_T_MIN)); + PyModule_AddObject(m, "SIZEOF_PYGC_HEAD", PyInt_FromSsize_t(sizeof(PyGC_Head))); + + TestError = PyErr_NewException("_testcapi.error", NULL, NULL); + Py_INCREF(TestError); + PyModule_AddObject(m, "error", TestError); +} Added: pypy/branch/fast-forward/lib_pypy/testcapi_long.h ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/lib_pypy/testcapi_long.h Fri Nov 5 19:35:52 2010 @@ -0,0 +1,182 @@ +/* Poor-man's template. Macros used: + TESTNAME name of the test (like test_long_api_inner) + TYPENAME the signed type (like long) + F_S_TO_PY convert signed to pylong; TYPENAME -> PyObject* + F_PY_TO_S convert pylong to signed; PyObject* -> TYPENAME + F_U_TO_PY convert unsigned to pylong; unsigned TYPENAME -> PyObject* + F_PY_TO_U convert pylong to unsigned; PyObject* -> unsigned TYPENAME +*/ + +static PyObject * +TESTNAME(PyObject *error(const char*)) +{ + const int NBITS = sizeof(TYPENAME) * 8; + unsigned TYPENAME base; + PyObject *pyresult; + int i; + + /* Note: This test lets PyObjects leak if an error is raised. Since + an error should never be raised, leaks are impossible . */ + + /* Test native -> PyLong -> native roundtrip identity. + * Generate all powers of 2, and test them and their negations, + * plus the numbers +-1 off from them. + */ + base = 1; + for (i = 0; + i < NBITS + 1; /* on last, base overflows to 0 */ + ++i, base <<= 1) + { + int j; + for (j = 0; j < 6; ++j) { + TYPENAME in, out; + unsigned TYPENAME uin, uout; + + /* For 0, 1, 2 use base; for 3, 4, 5 use -base */ + uin = j < 3 ? base + : (unsigned TYPENAME)(-(TYPENAME)base); + + /* For 0 & 3, subtract 1. + * For 1 & 4, leave alone. + * For 2 & 5, add 1. + */ + uin += (unsigned TYPENAME)(TYPENAME)(j % 3 - 1); + + pyresult = F_U_TO_PY(uin); + if (pyresult == NULL) + return error( + "unsigned unexpected null result"); + + uout = F_PY_TO_U(pyresult); + if (uout == (unsigned TYPENAME)-1 && PyErr_Occurred()) + return error( + "unsigned unexpected -1 result"); + if (uout != uin) + return error( + "unsigned output != input"); + UNBIND(pyresult); + + in = (TYPENAME)uin; + pyresult = F_S_TO_PY(in); + if (pyresult == NULL) + return error( + "signed unexpected null result"); + + out = F_PY_TO_S(pyresult); + if (out == (TYPENAME)-1 && PyErr_Occurred()) + return error( + "signed unexpected -1 result"); + if (out != in) + return error( + "signed output != input"); + UNBIND(pyresult); + } + } + + /* Overflow tests. The loop above ensured that all limit cases that + * should not overflow don't overflow, so all we need to do here is + * provoke one-over-the-limit cases (not exhaustive, but sharp). + */ + { + PyObject *one, *x, *y; + TYPENAME out; + unsigned TYPENAME uout; + + one = PyLong_FromLong(1); + if (one == NULL) + return error( + "unexpected NULL from PyLong_FromLong"); + + /* Unsigned complains about -1? */ + x = PyNumber_Negative(one); + if (x == NULL) + return error( + "unexpected NULL from PyNumber_Negative"); + + uout = F_PY_TO_U(x); + if (uout != (unsigned TYPENAME)-1 || !PyErr_Occurred()) + return error( + "PyLong_AsUnsignedXXX(-1) didn't complain"); + if (!PyErr_ExceptionMatches(PyExc_OverflowError)) + return error( + "PyLong_AsUnsignedXXX(-1) raised " + "something other than OverflowError"); + PyErr_Clear(); + UNBIND(x); + + /* Unsigned complains about 2**NBITS? */ + y = PyLong_FromLong((long)NBITS); + if (y == NULL) + return error( + "unexpected NULL from PyLong_FromLong"); + + x = PyNumber_Lshift(one, y); /* 1L << NBITS, == 2**NBITS */ + UNBIND(y); + if (x == NULL) + return error( + "unexpected NULL from PyNumber_Lshift"); + + uout = F_PY_TO_U(x); + if (uout != (unsigned TYPENAME)-1 || !PyErr_Occurred()) + return error( + "PyLong_AsUnsignedXXX(2**NBITS) didn't " + "complain"); + if (!PyErr_ExceptionMatches(PyExc_OverflowError)) + return error( + "PyLong_AsUnsignedXXX(2**NBITS) raised " + "something other than OverflowError"); + PyErr_Clear(); + + /* Signed complains about 2**(NBITS-1)? + x still has 2**NBITS. */ + y = PyNumber_Rshift(x, one); /* 2**(NBITS-1) */ + UNBIND(x); + if (y == NULL) + return error( + "unexpected NULL from PyNumber_Rshift"); + + out = F_PY_TO_S(y); + if (out != (TYPENAME)-1 || !PyErr_Occurred()) + return error( + "PyLong_AsXXX(2**(NBITS-1)) didn't " + "complain"); + if (!PyErr_ExceptionMatches(PyExc_OverflowError)) + return error( + "PyLong_AsXXX(2**(NBITS-1)) raised " + "something other than OverflowError"); + PyErr_Clear(); + + /* Signed complains about -2**(NBITS-1)-1?; + y still has 2**(NBITS-1). */ + x = PyNumber_Negative(y); /* -(2**(NBITS-1)) */ + UNBIND(y); + if (x == NULL) + return error( + "unexpected NULL from PyNumber_Negative"); + + y = PyNumber_Subtract(x, one); /* -(2**(NBITS-1))-1 */ + UNBIND(x); + if (y == NULL) + return error( + "unexpected NULL from PyNumber_Subtract"); + + out = F_PY_TO_S(y); + if (out != (TYPENAME)-1 || !PyErr_Occurred()) + return error( + "PyLong_AsXXX(-2**(NBITS-1)-1) didn't " + "complain"); + if (!PyErr_ExceptionMatches(PyExc_OverflowError)) + return error( + "PyLong_AsXXX(-2**(NBITS-1)-1) raised " + "something other than OverflowError"); + PyErr_Clear(); + UNBIND(y); + + Py_XDECREF(x); + Py_XDECREF(y); + Py_DECREF(one); + } + + Py_INCREF(Py_None); + return Py_None; +} From afa at codespeak.net Fri Nov 5 19:36:24 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 19:36:24 +0100 (CET) Subject: [pypy-svn] r78764 - pypy/branch/fast-forward/pypy/module/cpyext Message-ID: <20101105183624.737F0282C05@codespeak.net> Author: afa Date: Fri Nov 5 19:36:22 2010 New Revision: 78764 Modified: pypy/branch/fast-forward/pypy/module/cpyext/api.py Log: A useful symbol for _testcapi Modified: pypy/branch/fast-forward/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/api.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/api.py Fri Nov 5 19:36:22 2010 @@ -744,6 +744,7 @@ ("SIZEOF_LONG_LONG", rffi.LONGLONG), ("SIZEOF_VOID_P", rffi.VOIDP), ("SIZEOF_SIZE_T", rffi.SIZE_T), + ("SIZEOF_TIME_T", rffi.TIME_T), ("SIZEOF_LONG", rffi.LONG), ("SIZEOF_SHORT", rffi.SHORT), ("SIZEOF_INT", rffi.INT) From david at codespeak.net Fri Nov 5 19:58:30 2010 From: david at codespeak.net (david at codespeak.net) Date: Fri, 5 Nov 2010 19:58:30 +0100 (CET) Subject: [pypy-svn] r78765 - pypy/extradoc/planning/hg-migration Message-ID: <20101105185830.3CC1A282BFD@codespeak.net> Author: david Date: Fri Nov 5 19:58:27 2010 New Revision: 78765 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: Add my email Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Fri Nov 5 19:58:27 2010 @@ -99,7 +99,7 @@ aft=Andrew Thompson jgilbert=Joshua Gilbert asigfrid=Anders Sigfridsson -david=David Schneider +david=David Schneider mcherm=Michael Chermside tav=tav blais=Martin Blais From antocuni at codespeak.net Fri Nov 5 20:20:52 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 5 Nov 2010 20:20:52 +0100 (CET) Subject: [pypy-svn] r78766 - pypy/extradoc/planning/hg-migration Message-ID: <20101105192052.22775282BFD@codespeak.net> Author: antocuni Date: Fri Nov 5 20:20:51 2010 New Revision: 78766 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: remove trailing commas Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Fri Nov 5 20:20:51 2010 @@ -43,7 +43,7 @@ afayolle=Alexandre Fayolle agaynor=Alex Gaynor mgedmin=Marius Gedminas -getxsick=Bartosz Skowron,,, +getxsick=Bartosz Skowron gvanrossum=Guido van Rossum dialtone=Valentino Volonghi akuhn=Adrian Kuhn From dan at codespeak.net Fri Nov 5 21:12:59 2010 From: dan at codespeak.net (dan at codespeak.net) Date: Fri, 5 Nov 2010 21:12:59 +0100 (CET) Subject: [pypy-svn] r78767 - pypy/extradoc/planning/hg-migration Message-ID: <20101105201259.BADCF282B9D@codespeak.net> Author: dan Date: Fri Nov 5 21:12:26 2010 New Revision: 78767 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: Added my email Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Fri Nov 5 21:12:26 2010 @@ -79,7 +79,7 @@ bea=Beatrice During elmom=Elmo Mntynen ancestor=Andreas Friedge -dan=Daniel Roberts +dan=Daniel Roberts quest=Anders Qvist amcintyre=Alan McIntyre bert=Bert Freudenberg From afa at codespeak.net Fri Nov 5 21:57:57 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 21:57:57 +0100 (CET) Subject: [pypy-svn] r78768 - in pypy/branch/fast-forward: lib_pypy/_ctypes pypy/module/_rawffi pypy/module/_rawffi/test Message-ID: <20101105205757.E272D282BD4@codespeak.net> Author: afa Date: Fri Nov 5 21:57:52 2010 New Revision: 78768 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py Log: Implement _rawffi.wcharp2unicode(), and use it to replace two XXX in ctypes. Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/array.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/array.py Fri Nov 5 21:57:52 2010 @@ -5,14 +5,6 @@ from _ctypes.basics import keepalive_key, store_reference, ensure_objects from _ctypes.basics import CArgObject -def _create_unicode(buffer, maxlength): - res = [] - for i in range(maxlength): - if buffer[i] == '\x00': - break - res.append(buffer[i]) - return u''.join(res) - class ArrayMeta(_CDataMeta): def __new__(self, name, cls, typedict): res = type.__new__(self, name, cls, typedict) @@ -44,8 +36,8 @@ res.raw = property(getraw, setraw) elif subletter == 'u': def getvalue(self): - # rawffi support anyone? - return _create_unicode(self._buffer, self._length_) + return _rawffi.wcharp2unicode(self._buffer.buffer, + self._length_) def setvalue(self, val): # we don't want to have buffers here @@ -79,7 +71,7 @@ if self._type_ is c_char: return _rawffi.charp2string(resarray.buffer, self._length_) if self._type_ is c_wchar: - xxx + return _rawffi.wcharp2unicode(resarray.buffer, self._length_) res = self.__new__(self) ffiarray = self._ffiarray.fromaddress(resarray.buffer, self._length_) res._buffer = ffiarray Modified: pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py Fri Nov 5 21:57:52 2010 @@ -22,6 +22,7 @@ 'sizeof' : 'interp_rawffi.sizeof', 'alignment' : 'interp_rawffi.alignment', 'charp2string' : 'interp_rawffi.charp2string', + 'wcharp2unicode' : 'interp_rawffi.wcharp2unicode', 'charp2rawstring' : 'interp_rawffi.charp2rawstring', 'CallbackPtr' : 'callback.W_CallbackPtr', '_num_of_allocated_objects' : 'tracker.num_of_allocated_objects', Modified: pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py (original) +++ pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py Fri Nov 5 21:57:52 2010 @@ -487,6 +487,13 @@ return space.wrap(s) charp2string.unwrap_spec = [ObjSpace, r_uint, int] +def wcharp2unicode(space, address, maxlength=sys.maxint): + if address == 0: + return space.w_None + s = rffi.wcharp2unicoden(rffi.cast(rffi.CWCHARP, address), maxlength) + return space.wrap(s) +wcharp2unicode.unwrap_spec = [ObjSpace, r_uint, int] + def charp2rawstring(space, address, maxlength=-1): if maxlength == -1: return charp2string(space, address) Modified: pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py (original) +++ pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py Fri Nov 5 21:57:52 2010 @@ -298,6 +298,14 @@ arg1.free() arg2.free() + def test_returning_unicode(self): + import _rawffi + A = _rawffi.Array('u') + a = A(6, u'xx\x00\x00xx') + res = _rawffi.wcharp2unicode(a.buffer) + assert isinstance(res, unicode) + assert res == u'xx' + def test_raw_callable(self): import _rawffi lib = _rawffi.CDLL(self.lib_name) From afa at codespeak.net Fri Nov 5 22:06:10 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 22:06:10 +0100 (CET) Subject: [pypy-svn] r78769 - in pypy/branch/fast-forward: lib_pypy/_ctypes pypy/module/_rawffi Message-ID: <20101105210610.8916A282BFD@codespeak.net> Author: afa Date: Fri Nov 5 22:06:08 2010 New Revision: 78769 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/builtin.py pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py Log: implement _rawffi.wcharp2rawunicode() and use it in _ctypes Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/builtin.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/builtin.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/builtin.py Fri Nov 5 22:06:08 2010 @@ -23,13 +23,4 @@ def _wstring_at_addr(addr, lgt): import ctypes arg = ctypes.c_void_p._CData_value(addr) - # XXX purely applevel - if lgt == -1: - lgt = sys.maxint - a = _rawffi.Array('u').fromaddress(arg, lgt) - res = [] - for i in xrange(lgt): - if lgt == sys.maxint and a[i] == '\x00': - break - res.append(a[i]) - return u''.join(res) + return _rawffi.wcharp2rawunicode(arg, lgt) Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py Fri Nov 5 22:06:08 2010 @@ -142,13 +142,12 @@ result.value = property(_getvalue, _setvalue) elif tp == 'Z': # c_wchar_p - from _ctypes import _wstring_at_addr def _getvalue(self): addr = self._buffer[0] if addr == 0: return None else: - return _wstring_at_addr(addr, -1) + return _rawffi.wcharp2unicode(addr) def _setvalue(self, value): if isinstance(value, basestring): @@ -217,14 +216,13 @@ SysAllocStringLen = windll.oleaut32.SysAllocStringLen SysStringLen = windll.oleaut32.SysStringLen SysFreeString = windll.oleaut32.SysFreeString - from _ctypes import _wstring_at_addr def _getvalue(self): addr = self._buffer[0] if addr == 0: return None else: size = SysStringLen(addr) - return _wstring_at_addr(addr, size) + return _rawffi.wcharp2rawunicode(addr, size) def _setvalue(self, value): if isinstance(value, basestring): Modified: pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/_rawffi/__init__.py Fri Nov 5 22:06:08 2010 @@ -24,6 +24,7 @@ 'charp2string' : 'interp_rawffi.charp2string', 'wcharp2unicode' : 'interp_rawffi.wcharp2unicode', 'charp2rawstring' : 'interp_rawffi.charp2rawstring', + 'wcharp2rawunicode' : 'interp_rawffi.wcharp2rawunicode', 'CallbackPtr' : 'callback.W_CallbackPtr', '_num_of_allocated_objects' : 'tracker.num_of_allocated_objects', 'get_libc' : 'interp_rawffi.get_libc', Modified: pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py (original) +++ pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py Fri Nov 5 22:06:08 2010 @@ -501,6 +501,13 @@ return space.wrap(s) charp2rawstring.unwrap_spec = [ObjSpace, r_uint, int] +def wcharp2rawunicode(space, address, maxlength=-1): + if maxlength == -1: + return wcharp2unicode(space, address) + s = rffi.wcharpsize2unicode(rffi.cast(rffi.CCHARP, address), maxlength) + return space.wrap(s) +charp2rawstring.unwrap_spec = [ObjSpace, r_uint, int] + if _MS_WINDOWS: def FormatError(space, code): return space.wrap(rwin32.FormatError(code)) From benjamin at codespeak.net Fri Nov 5 22:38:10 2010 From: benjamin at codespeak.net (benjamin at codespeak.net) Date: Fri, 5 Nov 2010 22:38:10 +0100 (CET) Subject: [pypy-svn] r78771 - pypy/extradoc/planning/hg-migration Message-ID: <20101105213810.D94D2282BEC@codespeak.net> Author: benjamin Date: Fri Nov 5 22:38:09 2010 New Revision: 78771 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: add email address Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Fri Nov 5 22:38:09 2010 @@ -16,7 +16,7 @@ rxe=Richard Emslie ac=Anders Chrigstrom auc=Aurelien Campeas -benjamin=Benjamin Peterson +benjamin=Benjamin Peterson ale=Anders Lehmann nik=Niklaus Haldimann sanxiyn=Seo Sanghyeon From trundle at codespeak.net Fri Nov 5 23:02:46 2010 From: trundle at codespeak.net (trundle at codespeak.net) Date: Fri, 5 Nov 2010 23:02:46 +0100 (CET) Subject: [pypy-svn] r78772 - pypy/extradoc/planning/hg-migration Message-ID: <20101105220246.31EAC282BDC@codespeak.net> Author: trundle Date: Fri Nov 5 23:02:43 2010 New Revision: 78772 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: Added myself. Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Fri Nov 5 23:02:43 2010 @@ -104,3 +104,4 @@ tav=tav blais=Martin Blais haypo=Victor Stinner +trundle=Andreas St?hrk From afa at codespeak.net Fri Nov 5 23:08:03 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 23:08:03 +0100 (CET) Subject: [pypy-svn] r78773 - pypy/branch/fast-forward/lib_pypy Message-ID: <20101105220803.6BCB2282BF0@codespeak.net> Author: afa Date: Fri Nov 5 23:08:01 2010 New Revision: 78773 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes_test.py Log: Fix compilation on 64bit linux Modified: pypy/branch/fast-forward/lib_pypy/_ctypes_test.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes_test.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes_test.py Fri Nov 5 23:08:01 2010 @@ -20,8 +20,13 @@ # Compile .c file include_dir = os.path.join(thisdir, '..', 'include') + if sys.platform == 'win32': + ccflags = [] + else: + ccflags = ['-fPIC'] res = compiler.compile([os.path.join(thisdir, '_ctypes_test.c')], - include_dirs=[include_dir]) + include_dirs=[include_dir], + extra_preargs=ccflags) object_filename = res[0] # set link options From afa at codespeak.net Fri Nov 5 23:14:52 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 23:14:52 +0100 (CET) Subject: [pypy-svn] r78774 - in pypy/branch/fast-forward/pypy/module/thread: . test Message-ID: <20101105221452.B572D282C03@codespeak.net> Author: afa Date: Fri Nov 5 23:14:51 2010 New Revision: 78774 Modified: pypy/branch/fast-forward/pypy/module/thread/os_local.py pypy/branch/fast-forward/pypy/module/thread/test/test_local.py Log: thread._local.__init__ accepts no argument. Modified: pypy/branch/fast-forward/pypy/module/thread/os_local.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/thread/os_local.py (original) +++ pypy/branch/fast-forward/pypy/module/thread/os_local.py Fri Nov 5 23:14:51 2010 @@ -47,15 +47,20 @@ self.dicts[ident] = w_dict def descr_local__new__(space, w_subtype, __args__): - # XXX check __args__ local = space.allocate_instance(Local, w_subtype) Local.__init__(local, space, __args__) return space.wrap(local) + descr_local__new__.unwrap_spec=[ObjSpace, W_Root, Arguments] + + def descr_local__init__(self, space): + # No arguments allowed + pass + descr_local__init__.unwrap_spec=['self', ObjSpace] Local.typedef = TypeDef("thread._local", __doc__ = "Thread-local data", - __new__ = interp2app(Local.descr_local__new__.im_func, - unwrap_spec=[ObjSpace, W_Root, Arguments]), + __new__ = interp2app(Local.descr_local__new__.im_func), + __init__ = interp2app(Local.descr_local__init__), __dict__ = GetSetProperty(descr_get_dict, descr_set_dict, cls=Local), ) Modified: pypy/branch/fast-forward/pypy/module/thread/test/test_local.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/thread/test/test_local.py (original) +++ pypy/branch/fast-forward/pypy/module/thread/test/test_local.py Fri Nov 5 23:14:51 2010 @@ -51,6 +51,9 @@ tags = [1, 2, 3, 4, 5, 54321] seen = [] + raises(TypeError, thread._local, a=1) + raises(TypeError, thread._local, 1) + class X(thread._local): def __init__(self, n): assert n == 42 From afa at codespeak.net Fri Nov 5 23:23:07 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 23:23:07 +0100 (CET) Subject: [pypy-svn] r78775 - pypy/branch/fast-forward/lib-python/modified-2.7.0/test Message-ID: <20101105222307.B4F7C282C08@codespeak.net> Author: afa Date: Fri Nov 5 23:23:06 2010 New Revision: 78775 Added: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_threading_local.py - copied, changed from r78767, pypy/branch/fast-forward/lib-python/2.7.0/test/test_threading_local.py Log: in Pypy, thread._local objects support setting __dict__. skip the corresponding test Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_threading_local.py (from r78767, pypy/branch/fast-forward/lib-python/2.7.0/test/test_threading_local.py) ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_threading_local.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_threading_local.py Fri Nov 5 23:23:06 2010 @@ -173,8 +173,9 @@ obj = cls() obj.x = 5 self.assertEqual(obj.__dict__, {'x': 5}) - with self.assertRaises(AttributeError): - obj.__dict__ = {} + if test_support.check_impl_detail(): + with self.assertRaises(AttributeError): + obj.__dict__ = {} with self.assertRaises(AttributeError): del obj.__dict__ From afa at codespeak.net Fri Nov 5 23:51:09 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 23:51:09 +0100 (CET) Subject: [pypy-svn] r78776 - in pypy/branch/fast-forward/lib-python/2.7.0: . ctypes distutils distutils/tests email email/test json json/tests lib-tk lib-tk/test/test_ttk lib2to3 lib2to3/fixes lib2to3/tests lib2to3/tests/data logging multiprocessing test test/tracedmodules unittest unittest/test xml/dom xml/sax Message-ID: <20101105225109.41ECB282C06@codespeak.net> Author: afa Date: Fri Nov 5 23:50:59 2010 New Revision: 78776 Modified: pypy/branch/fast-forward/lib-python/2.7.0/CGIHTTPServer.py pypy/branch/fast-forward/lib-python/2.7.0/SimpleXMLRPCServer.py pypy/branch/fast-forward/lib-python/2.7.0/_pyio.py pypy/branch/fast-forward/lib-python/2.7.0/argparse.py pypy/branch/fast-forward/lib-python/2.7.0/asyncore.py pypy/branch/fast-forward/lib-python/2.7.0/base64.py pypy/branch/fast-forward/lib-python/2.7.0/calendar.py pypy/branch/fast-forward/lib-python/2.7.0/collections.py pypy/branch/fast-forward/lib-python/2.7.0/ctypes/util.py pypy/branch/fast-forward/lib-python/2.7.0/distutils/dir_util.py pypy/branch/fast-forward/lib-python/2.7.0/distutils/sysconfig.py pypy/branch/fast-forward/lib-python/2.7.0/distutils/tests/test_build_ext.py pypy/branch/fast-forward/lib-python/2.7.0/distutils/tests/test_dir_util.py pypy/branch/fast-forward/lib-python/2.7.0/doctest.py pypy/branch/fast-forward/lib-python/2.7.0/email/_parseaddr.py pypy/branch/fast-forward/lib-python/2.7.0/email/quoprimime.py pypy/branch/fast-forward/lib-python/2.7.0/email/test/test_email.py pypy/branch/fast-forward/lib-python/2.7.0/gettext.py pypy/branch/fast-forward/lib-python/2.7.0/gzip.py pypy/branch/fast-forward/lib-python/2.7.0/httplib.py pypy/branch/fast-forward/lib-python/2.7.0/json/__init__.py pypy/branch/fast-forward/lib-python/2.7.0/json/decoder.py pypy/branch/fast-forward/lib-python/2.7.0/json/tests/test_encode_basestring_ascii.py pypy/branch/fast-forward/lib-python/2.7.0/json/tests/test_unicode.py pypy/branch/fast-forward/lib-python/2.7.0/lib-tk/test/test_ttk/test_widgets.py pypy/branch/fast-forward/lib-python/2.7.0/lib-tk/turtle.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/Grammar.txt pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixer_base.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixer_util.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_apply.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_basestring.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_buffer.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_callable.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_dict.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_except.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_exec.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_execfile.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_exitfunc.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_filter.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_funcattrs.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_future.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_getcwdu.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_has_key.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_idioms.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_import.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_imports.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_input.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_intern.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_isinstance.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_itertools.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_itertools_imports.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_long.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_map.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_metaclass.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_methodattrs.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_next.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_nonzero.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_operator.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_paren.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_print.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_raise.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_raw_input.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_reduce.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_renames.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_repr.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_set_literal.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_standarderror.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_sys_exc.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_throw.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_tuple_params.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_types.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_unicode.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_urllib.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_xrange.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_xreadlines.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_zip.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/patcomp.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/pygram.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/pytree.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/refactor.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/tests/data/bom.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/tests/test_fixers.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/tests/test_pytree.py pypy/branch/fast-forward/lib-python/2.7.0/logging/__init__.py pypy/branch/fast-forward/lib-python/2.7.0/logging/config.py pypy/branch/fast-forward/lib-python/2.7.0/mimetypes.py pypy/branch/fast-forward/lib-python/2.7.0/multiprocessing/__init__.py pypy/branch/fast-forward/lib-python/2.7.0/ntpath.py pypy/branch/fast-forward/lib-python/2.7.0/posixpath.py pypy/branch/fast-forward/lib-python/2.7.0/runpy.py pypy/branch/fast-forward/lib-python/2.7.0/site.py pypy/branch/fast-forward/lib-python/2.7.0/smtpd.py pypy/branch/fast-forward/lib-python/2.7.0/ssl.py pypy/branch/fast-forward/lib-python/2.7.0/sysconfig.py pypy/branch/fast-forward/lib-python/2.7.0/tarfile.py pypy/branch/fast-forward/lib-python/2.7.0/test/regrtest.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_abc.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_argparse.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_bz2.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_class.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_collections.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_compiler.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_decimal.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_doctest.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_file2k.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_fileio.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_float.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_ftplib.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_gettext.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_gzip.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_httplib.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_httpservers.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_io.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_long.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_mailbox.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_module.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_modulefinder.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_multibytecodec.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_multibytecodec_support.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_multiprocessing.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_ntpath.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_old_mailbox.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_os.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_pipes.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_pkgimport.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_posixpath.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_quopri.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_sax.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_scope.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_shutil.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_signal.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_smtpnet.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_socket.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_ssl.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_subprocess.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_support.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_sys_setprofile.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_threading.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_timeout.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_tokenize.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_trace.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_unicode.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_urllib2net.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_uu.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_winreg.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_xml_etree.py pypy/branch/fast-forward/lib-python/2.7.0/test/test_xpickle.py pypy/branch/fast-forward/lib-python/2.7.0/test/tracedmodules/testmod.py pypy/branch/fast-forward/lib-python/2.7.0/test/win_console_handler.py pypy/branch/fast-forward/lib-python/2.7.0/trace.py pypy/branch/fast-forward/lib-python/2.7.0/unittest/result.py pypy/branch/fast-forward/lib-python/2.7.0/unittest/suite.py pypy/branch/fast-forward/lib-python/2.7.0/unittest/test/test_suite.py pypy/branch/fast-forward/lib-python/2.7.0/urllib.py pypy/branch/fast-forward/lib-python/2.7.0/urllib2.py pypy/branch/fast-forward/lib-python/2.7.0/uu.py pypy/branch/fast-forward/lib-python/2.7.0/uuid.py pypy/branch/fast-forward/lib-python/2.7.0/xml/dom/expatbuilder.py pypy/branch/fast-forward/lib-python/2.7.0/xml/sax/saxutils.py Log: New copy of the 2.7 std library, with probably many fixes to cleanly close resources without relying on the refcounting garbage collector. Modified: pypy/branch/fast-forward/lib-python/2.7.0/CGIHTTPServer.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/CGIHTTPServer.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/CGIHTTPServer.py Fri Nov 5 23:50:59 2010 @@ -29,6 +29,7 @@ import BaseHTTPServer import SimpleHTTPServer import select +import copy class CGIHTTPRequestHandler(SimpleHTTPServer.SimpleHTTPRequestHandler): @@ -154,7 +155,7 @@ # Reference: http://hoohoo.ncsa.uiuc.edu/cgi/env.html # XXX Much of the following could be prepared ahead of time! - env = {} + env = copy.deepcopy(os.environ) env['SERVER_SOFTWARE'] = self.version_string() env['SERVER_NAME'] = self.server.server_name env['GATEWAY_INTERFACE'] = 'CGI/1.1' @@ -216,7 +217,6 @@ for k in ('QUERY_STRING', 'REMOTE_HOST', 'CONTENT_LENGTH', 'HTTP_USER_AGENT', 'HTTP_COOKIE', 'HTTP_REFERER'): env.setdefault(k, "") - os.environ.update(env) self.send_response(200, "Script output follows") @@ -248,7 +248,7 @@ pass os.dup2(self.rfile.fileno(), 0) os.dup2(self.wfile.fileno(), 1) - os.execve(scriptfile, args, os.environ) + os.execve(scriptfile, args, env) except: self.server.handle_error(self.request, self.client_address) os._exit(127) @@ -274,7 +274,8 @@ p = subprocess.Popen(cmdline, stdin = subprocess.PIPE, stdout = subprocess.PIPE, - stderr = subprocess.PIPE + stderr = subprocess.PIPE, + env = env ) if self.command.lower() == "post" and nbytes > 0: data = self.rfile.read(nbytes) @@ -288,6 +289,8 @@ self.wfile.write(stdout) if stderr: self.log_error('%s', stderr) + p.stderr.close() + p.stdout.close() status = p.returncode if status: self.log_error("CGI script exit status %#x", status) Modified: pypy/branch/fast-forward/lib-python/2.7.0/SimpleXMLRPCServer.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/SimpleXMLRPCServer.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/SimpleXMLRPCServer.py Fri Nov 5 23:50:59 2010 @@ -163,7 +163,7 @@ This class is used to register XML-RPC method handlers and then to dispatch them. This class doesn't need to be instanced directly when used by SimpleXMLRPCServer but it - can be instanced when used by the MultiPathXMLRPCServer + can be instanced when used by the MultiPathXMLRPCServer. """ def __init__(self, allow_none=False, encoding=None): Modified: pypy/branch/fast-forward/lib-python/2.7.0/_pyio.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/_pyio.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/_pyio.py Fri Nov 5 23:50:59 2010 @@ -546,6 +546,8 @@ return self.readall() b = bytearray(n.__index__()) n = self.readinto(b) + if n is None: + return None del b[n:] return bytes(b) @@ -563,7 +565,7 @@ """Read up to len(b) bytes into b. Returns number of bytes read (0 for EOF), or None if the object - is set not to block as has no data to read. + is set not to block and has no data to read. """ self._unsupported("readinto") Modified: pypy/branch/fast-forward/lib-python/2.7.0/argparse.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/argparse.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/argparse.py Fri Nov 5 23:50:59 2010 @@ -65,13 +65,20 @@ __all__ = [ 'ArgumentParser', 'ArgumentError', - 'Namespace', - 'Action', + 'ArgumentTypeError', 'FileType', 'HelpFormatter', + 'ArgumentDefaultsHelpFormatter', 'RawDescriptionHelpFormatter', 'RawTextHelpFormatter', - 'ArgumentDefaultsHelpFormatter', + 'Namespace', + 'Action', + 'ONE_OR_MORE', + 'OPTIONAL', + 'PARSER', + 'REMAINDER', + 'SUPPRESS', + 'ZERO_OR_MORE', ] @@ -95,6 +102,7 @@ ONE_OR_MORE = '+' PARSER = 'A...' REMAINDER = '...' +_UNRECOGNIZED_ARGS_ATTR = '_unrecognized_args' # ============================= # Utility functions and classes @@ -385,10 +393,16 @@ for action in group._group_actions: group_actions.add(action) if not group.required: - inserts[start] = '[' + if start in inserts: + inserts[start] += ' [' + else: + inserts[start] = '[' inserts[end] = ']' else: - inserts[start] = '(' + if start in inserts: + inserts[start] += ' (' + else: + inserts[start] = '(' inserts[end] = ')' for i in range(start + 1, end): inserts[i] = '|' @@ -1070,7 +1084,12 @@ raise ArgumentError(self, msg) # parse all the remaining options into the namespace - parser.parse_args(arg_strings, namespace) + # store any unrecognized options on the object, so that the top + # level parser can decide what to do with them + namespace, arg_strings = parser.parse_known_args(arg_strings, namespace) + if arg_strings: + vars(namespace).setdefault(_UNRECOGNIZED_ARGS_ATTR, []) + getattr(namespace, _UNRECOGNIZED_ARGS_ATTR).extend(arg_strings) # ============== @@ -1688,7 +1707,11 @@ # parse the arguments and exit if there are any errors try: - return self._parse_known_args(args, namespace) + namespace, args = self._parse_known_args(args, namespace) + if hasattr(namespace, _UNRECOGNIZED_ARGS_ATTR): + args.extend(getattr(namespace, _UNRECOGNIZED_ARGS_ATTR)) + delattr(namespace, _UNRECOGNIZED_ARGS_ATTR) + return namespace, args except ArgumentError: err = _sys.exc_info()[1] self.error(str(err)) @@ -1789,13 +1812,13 @@ chars = self.prefix_chars if arg_count == 0 and option_string[1] not in chars: action_tuples.append((action, [], option_string)) - for char in self.prefix_chars: - option_string = char + explicit_arg[0] - explicit_arg = explicit_arg[1:] or None - optionals_map = self._option_string_actions - if option_string in optionals_map: - action = optionals_map[option_string] - break + char = option_string[0] + option_string = char + explicit_arg[0] + new_explicit_arg = explicit_arg[1:] or None + optionals_map = self._option_string_actions + if option_string in optionals_map: + action = optionals_map[option_string] + explicit_arg = new_explicit_arg else: msg = _('ignored explicit argument %r') raise ArgumentError(action, msg % explicit_arg) Modified: pypy/branch/fast-forward/lib-python/2.7.0/asyncore.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/asyncore.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/asyncore.py Fri Nov 5 23:50:59 2010 @@ -350,12 +350,15 @@ # XXX can return either an address pair or None try: conn, addr = self.socket.accept() - return conn, addr - except socket.error, why: - if why.args[0] == EWOULDBLOCK: - pass + except TypeError: + return None + except socket.error as why: + if why.args[0] in (EWOULDBLOCK, ECONNABORTED): + return None else: raise + else: + return conn, addr def send(self, data): try: Modified: pypy/branch/fast-forward/lib-python/2.7.0/base64.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/base64.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/base64.py Fri Nov 5 23:50:59 2010 @@ -343,7 +343,8 @@ if o == '-u': func = decode if o == '-t': test1(); return if args and args[0] != '-': - func(open(args[0], 'rb'), sys.stdout) + with open(args[0], 'rb') as f: + func(f, sys.stdout) else: func(sys.stdin, sys.stdout) Modified: pypy/branch/fast-forward/lib-python/2.7.0/calendar.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/calendar.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/calendar.py Fri Nov 5 23:50:59 2010 @@ -95,7 +95,7 @@ def isleap(year): - """Return 1 for leap years, 0 for non-leap years.""" + """Return True for leap years, False for non-leap years.""" return year % 4 == 0 and (year % 100 != 0 or year % 400 == 0) Modified: pypy/branch/fast-forward/lib-python/2.7.0/collections.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/collections.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/collections.py Fri Nov 5 23:50:59 2010 @@ -12,6 +12,32 @@ import heapq as _heapq from itertools import repeat as _repeat, chain as _chain, starmap as _starmap, \ ifilter as _ifilter, imap as _imap +try: + from thread import get_ident +except AttributeError: + from dummy_thread import get_ident + +def _recursive_repr(user_function): + 'Decorator to make a repr function return "..." for a recursive call' + repr_running = set() + + def wrapper(self): + key = id(self), get_ident() + if key in repr_running: + return '...' + repr_running.add(key) + try: + result = user_function(self) + finally: + repr_running.discard(key) + return result + + # Can't use functools.wraps() here because of bootstrap issues + wrapper.__module__ = getattr(user_function, '__module__') + wrapper.__doc__ = getattr(user_function, '__doc__') + wrapper.__name__ = getattr(user_function, '__name__') + return wrapper + ################################################################################ ### OrderedDict @@ -142,6 +168,7 @@ value = self.pop(key) return key, value + @_recursive_repr def __repr__(self): 'od.__repr__() <==> repr(od)' if not self: Modified: pypy/branch/fast-forward/lib-python/2.7.0/ctypes/util.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/ctypes/util.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/ctypes/util.py Fri Nov 5 23:50:59 2010 @@ -185,7 +185,7 @@ def _findLib_ldconfig(name): # XXX assuming GLIBC's ldconfig (with option -p) expr = r'/[^\(\)\s]*lib%s\.[^\(\)\s]*' % re.escape(name) - f = os.popen('LANG=C /sbin/ldconfig -p 2>/dev/null') + f = os.popen('LC_ALL=C LANG=C /sbin/ldconfig -p 2>/dev/null') try: data = f.read() finally: Modified: pypy/branch/fast-forward/lib-python/2.7.0/distutils/dir_util.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/distutils/dir_util.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/distutils/dir_util.py Fri Nov 5 23:50:59 2010 @@ -2,7 +2,7 @@ Utility functions for manipulating directories and directory trees.""" -__revision__ = "$Id: dir_util.py 76956 2009-12-21 01:22:46Z tarek.ziade $" +__revision__ = "$Id: dir_util.py 84862 2010-09-17 16:40:01Z senthil.kumaran $" import os from distutils.errors import DistutilsFileError, DistutilsInternalError @@ -68,7 +68,7 @@ if not dry_run: try: - os.mkdir(head) + os.mkdir(head, mode) created_dirs.append(head) except OSError, exc: raise DistutilsFileError, \ Modified: pypy/branch/fast-forward/lib-python/2.7.0/distutils/sysconfig.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/distutils/sysconfig.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/distutils/sysconfig.py Fri Nov 5 23:50:59 2010 @@ -9,7 +9,7 @@ Email: """ -__revision__ = "$Id: sysconfig.py 80804 2010-05-05 19:09:31Z ronald.oussoren $" +__revision__ = "$Id: sysconfig.py 85358 2010-10-10 09:54:59Z antoine.pitrou $" import os import re @@ -318,6 +318,11 @@ fp.close() + # strip spurious spaces + for k, v in done.items(): + if isinstance(v, str): + done[k] = v.strip() + # save the results in the global dictionary g.update(done) return g Modified: pypy/branch/fast-forward/lib-python/2.7.0/distutils/tests/test_build_ext.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/distutils/tests/test_build_ext.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/distutils/tests/test_build_ext.py Fri Nov 5 23:50:59 2010 @@ -49,6 +49,25 @@ sys.platform == 'cygwin') super(BuildExtTestCase, self).tearDown() + def _fixup_command(self, cmd): + # When Python was build with --enable-shared, -L. is not good enough + # to find the libpython.so. This is because regrtest runs it + # under a tempdir, not in the top level where the .so lives. By the + # time we've gotten here, Python's already been chdir'd to the + # tempdir. + # + # To further add to the fun, we can't just add library_dirs to the + # Extension() instance because that doesn't get plumbed through to the + # final compiler command. + if (sysconfig.get_config_var('Py_ENABLE_SHARED') and + not sys.platform.startswith('win')): + runshared = sysconfig.get_config_var('RUNSHARED') + if runshared is None: + cmd.library_dirs = ['.'] + else: + name, equals, value = runshared.partition('=') + cmd.library_dirs = value.split(os.pathsep) + @unittest.skipIf(not os.path.exists(_XX_MODULE_PATH), 'xxmodule.c not found') def test_build_ext(self): @@ -58,6 +77,7 @@ dist = Distribution({'name': 'xx', 'ext_modules': [xx_ext]}) dist.package_dir = self.tmp_dir cmd = build_ext(dist) + self._fixup_command(cmd) if os.name == "nt": # On Windows, we must build a debug version iff running # a debug build of Python @@ -250,6 +270,7 @@ dist = Distribution({'name': 'xx', 'ext_modules': [ext]}) cmd = build_ext(dist) + self._fixup_command(cmd) cmd.ensure_finalized() self.assertEquals(len(cmd.get_outputs()), 1) Modified: pypy/branch/fast-forward/lib-python/2.7.0/distutils/tests/test_dir_util.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/distutils/tests/test_dir_util.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/distutils/tests/test_dir_util.py Fri Nov 5 23:50:59 2010 @@ -1,7 +1,9 @@ """Tests for distutils.dir_util.""" import unittest import os +import stat import shutil +import sys from distutils.dir_util import (mkpath, remove_tree, create_tree, copy_tree, ensure_relative) @@ -48,6 +50,19 @@ wanted = ["removing '%s' (and everything under it)" % self.root_target] self.assertEquals(self._logs, wanted) + @unittest.skipIf(sys.platform.startswith('win'), + "This test is only appropriate for POSIX-like systems.") + def test_mkpath_with_custom_mode(self): + # Get and set the current umask value for testing mode bits. + umask = os.umask(0o002) + os.umask(umask) + mkpath(self.target, 0o700) + self.assertEqual( + stat.S_IMODE(os.stat(self.target).st_mode), 0o700 & ~umask) + mkpath(self.target2, 0o555) + self.assertEqual( + stat.S_IMODE(os.stat(self.target2).st_mode), 0o555 & ~umask) + def test_create_tree_verbosity(self): create_tree(self.root_target, ['one', 'two', 'three'], verbose=0) Modified: pypy/branch/fast-forward/lib-python/2.7.0/doctest.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/doctest.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/doctest.py Fri Nov 5 23:50:59 2010 @@ -216,7 +216,8 @@ # get_data() opens files as 'rb', so one must do the equivalent # conversion as universal newlines would do. return file_contents.replace(os.linesep, '\n'), filename - return open(filename).read(), filename + with open(filename) as f: + return f.read(), filename # Use sys.stdout encoding for ouput. _encoding = getattr(sys.__stdout__, 'encoding', None) or 'utf-8' @@ -1327,13 +1328,15 @@ self.tries += t __LINECACHE_FILENAME_RE = re.compile(r'[\w\.]+)' + r'(?P.+)' r'\[(?P\d+)\]>$') def __patched_linecache_getlines(self, filename, module_globals=None): m = self.__LINECACHE_FILENAME_RE.match(filename) if m and m.group('name') == self.test.name: example = self.test.examples[int(m.group('examplenum'))] - source = example.source.encode('ascii', 'backslashreplace') + source = example.source + if isinstance(source, unicode): + source = source.encode('ascii', 'backslashreplace') return source.splitlines(True) else: return self.save_linecache_getlines(filename, module_globals) Modified: pypy/branch/fast-forward/lib-python/2.7.0/email/_parseaddr.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/email/_parseaddr.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/email/_parseaddr.py Fri Nov 5 23:50:59 2010 @@ -160,7 +160,12 @@ def quote(str): - """Add quotes around a string.""" + """Prepare string to be used in a quoted string. + + Turns backslash and double quote characters into quoted pairs. These + are the only characters that need to be quoted inside a quoted string. + Does not add the surrounding double quotes. + """ return str.replace('\\', '\\\\').replace('"', '\\"') @@ -318,7 +323,7 @@ aslist.append('.') self.pos += 1 elif self.field[self.pos] == '"': - aslist.append('"%s"' % self.getquote()) + aslist.append('"%s"' % quote(self.getquote())) elif self.field[self.pos] in self.atomends: break else: Modified: pypy/branch/fast-forward/lib-python/2.7.0/email/quoprimime.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/email/quoprimime.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/email/quoprimime.py Fri Nov 5 23:50:59 2010 @@ -333,4 +333,4 @@ the high level email.header class for that functionality. """ s = s.replace('_', ' ') - return re.sub(r'=\w{2}', _unquote_match, s) + return re.sub(r'=[a-fA-F0-9]{2}', _unquote_match, s) Modified: pypy/branch/fast-forward/lib-python/2.7.0/email/test/test_email.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/email/test/test_email.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/email/test/test_email.py Fri Nov 5 23:50:59 2010 @@ -1621,6 +1621,12 @@ dh = decode_header(s % q) self.assertEqual(dh, [(a, 'iso-8859-1')]) + def test_rfc2047_Q_invalid_digits(self): + # issue 10004. + s = '=?iso-8659-1?Q?andr=e9=zz?=' + self.assertEqual(decode_header(s), + [(b'andr\xe9=zz', 'iso-8659-1')]) + # Test the MIMEMessage class class TestMIMEMessage(TestEmailBase): @@ -2281,6 +2287,24 @@ # formataddr() quotes the name if there's a dot in it self.assertEqual(Utils.formataddr((a, b)), y) + def test_parseaddr_preserves_quoted_pairs_in_addresses(self): + # issue 10005. Note that in the third test the second pair of + # backslashes is not actually a quoted pair because it is not inside a + # comment or quoted string: the address being parsed has a quoted + # string containing a quoted backslash, followed by 'example' and two + # backslashes, followed by another quoted string containing a space and + # the word 'example'. parseaddr copies those two backslashes + # literally. Per rfc5322 this is not technically correct since a \ may + # not appear in an address outside of a quoted string. It is probably + # a sensible Postel interpretation, though. + eq = self.assertEqual + eq(Utils.parseaddr('""example" example"@example.com'), + ('', '""example" example"@example.com')) + eq(Utils.parseaddr('"\\"example\\" example"@example.com'), + ('', '"\\"example\\" example"@example.com')) + eq(Utils.parseaddr('"\\\\"example\\\\" example"@example.com'), + ('', '"\\\\"example\\\\" example"@example.com')) + def test_multiline_from_comment(self): x = """\ Foo Modified: pypy/branch/fast-forward/lib-python/2.7.0/gettext.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/gettext.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/gettext.py Fri Nov 5 23:50:59 2010 @@ -471,7 +471,7 @@ # once. result = None for mofile in mofiles: - key = os.path.abspath(mofile) + key = (class_, os.path.abspath(mofile)) t = _translations.get(key) if t is None: with open(mofile, 'rb') as fp: Modified: pypy/branch/fast-forward/lib-python/2.7.0/gzip.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/gzip.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/gzip.py Fri Nov 5 23:50:59 2010 @@ -138,6 +138,13 @@ s = repr(self.fileobj) return '' + def _check_closed(self): + """Raises a ValueError if the underlying file object has been closed. + + """ + if self.closed: + raise ValueError('I/O operation on closed file.') + def _init_write(self, filename): self.name = filename self.crc = zlib.crc32("") & 0xffffffffL @@ -202,6 +209,7 @@ self.fileobj.read(2) # Read & discard the 16-bit header CRC def write(self,data): + self._check_closed() if self.mode != WRITE: import errno raise IOError(errno.EBADF, "write() on read-only GzipFile object") @@ -222,6 +230,7 @@ return len(data) def read(self, size=-1): + self._check_closed() if self.mode != READ: import errno raise IOError(errno.EBADF, "read() on write-only GzipFile object") @@ -359,6 +368,7 @@ self.myfileobj = None def flush(self,zlib_mode=zlib.Z_SYNC_FLUSH): + self._check_closed() if self.mode == WRITE: # Ensure the compressor's buffer is flushed self.fileobj.write(self.compress.flush(zlib_mode)) Modified: pypy/branch/fast-forward/lib-python/2.7.0/httplib.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/httplib.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/httplib.py Fri Nov 5 23:50:59 2010 @@ -639,6 +639,9 @@ amt -= len(chunk) return ''.join(s) + def fileno(self): + return self.fp.fileno() + def getheader(self, name, default=None): if self.msg is None: raise ResponseNotReady() @@ -749,8 +752,8 @@ self.__response = None self.__state = _CS_IDLE - def send(self, str): - """Send `str' to the server.""" + def send(self, data): + """Send `data' to the server.""" if self.sock is None: if self.auto_open: self.connect() @@ -758,16 +761,16 @@ raise NotConnected() if self.debuglevel > 0: - print "send:", repr(str) + print "send:", repr(data) blocksize = 8192 - if hasattr(str,'read') and not isinstance(str, array): + if hasattr(data,'read') and not isinstance(data, array): if self.debuglevel > 0: print "sendIng a read()able" - data = str.read(blocksize) - while data: - self.sock.sendall(data) - data = str.read(blocksize) + datablock = data.read(blocksize) + while datablock: + self.sock.sendall(datablock) + datablock = data.read(blocksize) else: - self.sock.sendall(str) + self.sock.sendall(data) def _output(self, s): """Add a line of output to the current request buffer. @@ -839,9 +842,9 @@ self._method = method if not url: url = '/' - str = '%s %s %s' % (method, url, self._http_vsn_str) + hdr = '%s %s %s' % (method, url, self._http_vsn_str) - self._output(str) + self._output(hdr) if self._http_vsn == 11: # Issue some standard headers for better HTTP/1.1 compliance @@ -912,8 +915,8 @@ if self.__state != _CS_REQ_STARTED: raise CannotSendHeader() - str = '%s: %s' % (header, '\r\n\t'.join(values)) - self._output(str) + hdr = '%s: %s' % (header, '\r\n\t'.join([str(v) for v in values])) + self._output(hdr) def endheaders(self, message_body=None): """Indicate that the last header line has been sent to the server. Modified: pypy/branch/fast-forward/lib-python/2.7.0/json/__init__.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/json/__init__.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/json/__init__.py Fri Nov 5 23:50:59 2010 @@ -160,7 +160,7 @@ To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the ``.default()`` method to serialize additional types), specify it with - the ``cls`` kwarg. + the ``cls`` kwarg; otherwise ``JSONEncoder`` is used. """ # cached encoder @@ -220,7 +220,7 @@ To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the ``.default()`` method to serialize additional types), specify it with - the ``cls`` kwarg. + the ``cls`` kwarg; otherwise ``JSONEncoder`` is used. """ # cached encoder @@ -259,8 +259,16 @@ ``object_hook`` will be used instead of the ``dict``. This feature can be used to implement custom decoders (e.g. JSON-RPC class hinting). + ``object_pairs_hook`` is an optional function that will be called with the + result of any object literal decoded with an ordered list of pairs. The + return value of ``object_pairs_hook`` will be used instead of the ``dict``. + This feature can be used to implement custom decoders that rely on the + order that the key and value pairs are decoded (for example, + collections.OrderedDict will remember the order of insertion). If + ``object_hook`` is also defined, the ``object_pairs_hook`` takes priority. + To use a custom ``JSONDecoder`` subclass, specify it with the ``cls`` - kwarg. + kwarg; otherwise ``JSONDecoder`` is used. """ return loads(fp.read(), @@ -285,6 +293,14 @@ ``object_hook`` will be used instead of the ``dict``. This feature can be used to implement custom decoders (e.g. JSON-RPC class hinting). + ``object_pairs_hook`` is an optional function that will be called with the + result of any object literal decoded with an ordered list of pairs. The + return value of ``object_pairs_hook`` will be used instead of the ``dict``. + This feature can be used to implement custom decoders that rely on the + order that the key and value pairs are decoded (for example, + collections.OrderedDict will remember the order of insertion). If + ``object_hook`` is also defined, the ``object_pairs_hook`` takes priority. + ``parse_float``, if specified, will be called with the string of every JSON float to be decoded. By default this is equivalent to float(num_str). This can be used to use another datatype or parser @@ -301,7 +317,7 @@ are encountered. To use a custom ``JSONDecoder`` subclass, specify it with the ``cls`` - kwarg. + kwarg; otherwise ``JSONDecoder`` is used. """ if (cls is None and encoding is None and object_hook is None and Modified: pypy/branch/fast-forward/lib-python/2.7.0/json/decoder.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/json/decoder.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/json/decoder.py Fri Nov 5 23:50:59 2010 @@ -310,6 +310,15 @@ place of the given ``dict``. This can be used to provide custom deserializations (e.g. to support JSON-RPC class hinting). + ``object_pairs_hook``, if specified will be called with the result of + every JSON object decoded with an ordered list of pairs. The return + value of ``object_pairs_hook`` will be used instead of the ``dict``. + This feature can be used to implement custom decoders that rely on the + order that the key and value pairs are decoded (for example, + collections.OrderedDict will remember the order of insertion). If + ``object_hook`` is also defined, the ``object_pairs_hook`` takes + priority. + ``parse_float``, if specified, will be called with the string of every JSON float to be decoded. By default this is equivalent to float(num_str). This can be used to use another datatype or parser @@ -325,6 +334,11 @@ This can be used to raise an exception if invalid JSON numbers are encountered. + If ``strict`` is false (true is the default), then control + characters will be allowed inside strings. Control characters in + this context are those with character codes in the 0-31 range, + including ``'\\t'`` (tab), ``'\\n'``, ``'\\r'`` and ``'\\0'``. + """ self.encoding = encoding self.object_hook = object_hook Modified: pypy/branch/fast-forward/lib-python/2.7.0/json/tests/test_encode_basestring_ascii.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/json/tests/test_encode_basestring_ascii.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/json/tests/test_encode_basestring_ascii.py Fri Nov 5 23:50:59 2010 @@ -1,6 +1,8 @@ from unittest import TestCase import json.encoder +from json import dumps +from collections import OrderedDict CASES = [ (u'/\\"\ucafe\ubabe\uab98\ufcde\ubcda\uef4a\x08\x0c\n\r\t`1~!@#$%^&*()_+-=[]{}|;:\',./<>?', '"/\\\\\\"\\ucafe\\ubabe\\uab98\\ufcde\\ubcda\\uef4a\\b\\f\\n\\r\\t`1~!@#$%^&*()_+-=[]{}|;:\',./<>?"'), @@ -37,3 +39,9 @@ self.assertEquals(result, expect, '{0!r} != {1!r} for {2}({3!r})'.format( result, expect, fname, input_string)) + + def test_ordered_dict(self): + # See issue 6105 + items = [('one', 1), ('two', 2), ('three', 3), ('four', 4), ('five', 5)] + s = json.dumps(OrderedDict(items)) + self.assertEqual(s, '{"one": 1, "two": 2, "three": 3, "four": 4, "five": 5}') Modified: pypy/branch/fast-forward/lib-python/2.7.0/json/tests/test_unicode.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/json/tests/test_unicode.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/json/tests/test_unicode.py Fri Nov 5 23:50:59 2010 @@ -78,3 +78,5 @@ self.assertEquals(type(json.loads(u'""')), unicode) self.assertEquals(type(json.loads(u'"a"')), unicode) self.assertEquals(type(json.loads(u'["a"]')[0]), unicode) + # Issue 10038. + self.assertEquals(type(json.loads('"foo"')), unicode) Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib-tk/test/test_ttk/test_widgets.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib-tk/test/test_ttk/test_widgets.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib-tk/test/test_ttk/test_widgets.py Fri Nov 5 23:50:59 2010 @@ -13,7 +13,7 @@ def setUp(self): support.root_deiconify() - self.widget = ttk.Button() + self.widget = ttk.Button(width=0, text="Text") self.widget.pack() self.widget.wait_visibility() @@ -24,7 +24,10 @@ def test_identify(self): self.widget.update_idletasks() - self.assertEqual(self.widget.identify(5, 5), "label") + self.assertEqual(self.widget.identify( + int(self.widget.winfo_width() / 2), + int(self.widget.winfo_height() / 2) + ), "label") self.assertEqual(self.widget.identify(-1, -1), "") self.assertRaises(Tkinter.TclError, self.widget.identify, None, 5) @@ -530,7 +533,7 @@ def setUp(self): support.root_deiconify() - self.nb = ttk.Notebook() + self.nb = ttk.Notebook(padding=0) self.child1 = ttk.Label() self.child2 = ttk.Label() self.nb.add(self.child1, text='a') @@ -717,6 +720,7 @@ self.nb.tab(self.child1, text='a', underline=0) self.nb.enable_traversal() self.nb.focus_force() + support.simulate_mouse_click(self.nb, 5, 5) self.nb.event_generate('') self.assertEqual(self.nb.select(), str(self.child1)) @@ -725,7 +729,7 @@ def setUp(self): support.root_deiconify() - self.tv = ttk.Treeview() + self.tv = ttk.Treeview(padding=0) def tearDown(self): self.tv.destroy() Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib-tk/turtle.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib-tk/turtle.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib-tk/turtle.py Fri Nov 5 23:50:59 2010 @@ -1495,7 +1495,10 @@ >>> turtle.left(90) >>> turtle.heading() 90 - >>> turtle.degrees(400.0) # angle measurement in gon + + Change angle measurement unit to grad (also known as gon, + grade, or gradian and equals 1/100-th of the right angle.) + >>> turtle.degrees(400.0) >>> turtle.heading() 100 Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/Grammar.txt ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/Grammar.txt (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/Grammar.txt Fri Nov 5 23:50:59 2010 @@ -128,7 +128,7 @@ '`' testlist1 '`' | NAME | NUMBER | STRING+ | '.' '.' '.') listmaker: (test|star_expr) ( comp_for | (',' (test|star_expr))* [','] ) -testlist_gexp: test ( comp_for | (',' (test|star_expr))* [','] ) +testlist_gexp: (test|star_expr) ( comp_for | (',' (test|star_expr))* [','] ) lambdef: 'lambda' [varargslist] ':' test trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME subscriptlist: subscript (',' subscript)* [','] Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixer_base.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixer_base.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixer_base.py Fri Nov 5 23:50:59 2010 @@ -24,6 +24,7 @@ PATTERN = None # Most subclasses should override with a string literal pattern = None # Compiled pattern, set by compile_pattern() + pattern_tree = None # Tree representation of the pattern options = None # Options object passed to initializer filename = None # The filename (set by set_filename) logger = None # A logger (set by set_filename) @@ -36,6 +37,12 @@ _accept_type = None # [Advanced and not public] This tells RefactoringTool # which node type to accept when there's not a pattern. + keep_line_order = False # For the bottom matcher: match with the + # original line order + BM_compatible = False # Compatibility with the bottom matching + # module; every fixer should set this + # manually + # Shortcut for access to Python grammar symbols syms = pygram.python_symbols @@ -58,7 +65,9 @@ self.{pattern,PATTERN} in .match(). """ if self.PATTERN is not None: - self.pattern = PatternCompiler().compile_pattern(self.PATTERN) + PC = PatternCompiler() + self.pattern, self.pattern_tree = PC.compile_pattern(self.PATTERN, + with_tree=True) def set_filename(self, filename): """Set the filename, and a logger derived from it. Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixer_util.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixer_util.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixer_util.py Fri Nov 5 23:50:59 2010 @@ -295,8 +295,8 @@ """ Works like `does_tree_import` but adds an import statement if it was not imported. """ def is_import_stmt(node): - return node.type == syms.simple_stmt and node.children and \ - is_import(node.children[0]) + return (node.type == syms.simple_stmt and node.children and + is_import(node.children[0])) root = find_root(node) @@ -319,8 +319,8 @@ # if that also fails, we stick to the beginning of the file if insert_pos == 0: for idx, node in enumerate(root.children): - if node.type == syms.simple_stmt and node.children and \ - node.children[0].type == token.STRING: + if (node.type == syms.simple_stmt and node.children and + node.children[0].type == token.STRING): insert_pos = idx + 1 break Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_apply.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_apply.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_apply.py Fri Nov 5 23:50:59 2010 @@ -12,6 +12,7 @@ from ..fixer_util import Call, Comma, parenthesize class FixApply(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ power< 'apply' Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_basestring.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_basestring.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_basestring.py Fri Nov 5 23:50:59 2010 @@ -6,6 +6,7 @@ from ..fixer_util import Name class FixBasestring(fixer_base.BaseFix): + BM_compatible = True PATTERN = "'basestring'" Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_buffer.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_buffer.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_buffer.py Fri Nov 5 23:50:59 2010 @@ -9,6 +9,7 @@ class FixBuffer(fixer_base.BaseFix): + BM_compatible = True explicit = True # The user must ask for this fixer Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_callable.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_callable.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_callable.py Fri Nov 5 23:50:59 2010 @@ -11,6 +11,9 @@ from lib2to3.fixer_util import Call, Name, String, Attr, touch_import class FixCallable(fixer_base.BaseFix): + BM_compatible = True + + order = "pre" # Ignore callable(*args) or use of keywords. # Either could be a hint that the builtin callable() is not being used. Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_dict.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_dict.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_dict.py Fri Nov 5 23:50:59 2010 @@ -40,6 +40,8 @@ class FixDict(fixer_base.BaseFix): + BM_compatible = True + PATTERN = """ power< head=any+ trailer< '.' method=('keys'|'items'|'values'| Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_except.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_except.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_except.py Fri Nov 5 23:50:59 2010 @@ -34,6 +34,7 @@ yield (n, nodes[i+2]) class FixExcept(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ try_stmt< 'try' ':' (simple_stmt | suite) Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_exec.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_exec.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_exec.py Fri Nov 5 23:50:59 2010 @@ -16,6 +16,7 @@ class FixExec(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ exec_stmt< 'exec' a=any 'in' b=any [',' c=any] > Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_execfile.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_execfile.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_execfile.py Fri Nov 5 23:50:59 2010 @@ -13,6 +13,7 @@ class FixExecfile(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ power< 'execfile' trailer< '(' arglist< filename=any [',' globals=any [',' locals=any ] ] > ')' > > Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_exitfunc.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_exitfunc.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_exitfunc.py Fri Nov 5 23:50:59 2010 @@ -9,6 +9,8 @@ class FixExitfunc(fixer_base.BaseFix): + keep_line_order = True + BM_compatible = True PATTERN = """ ( Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_filter.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_filter.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_filter.py Fri Nov 5 23:50:59 2010 @@ -19,6 +19,7 @@ from ..fixer_util import Name, Call, ListComp, in_special_context class FixFilter(fixer_base.ConditionalFix): + BM_compatible = True PATTERN = """ filter_lambda=power< Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_funcattrs.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_funcattrs.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_funcattrs.py Fri Nov 5 23:50:59 2010 @@ -7,6 +7,8 @@ class FixFuncattrs(fixer_base.BaseFix): + BM_compatible = True + PATTERN = """ power< any+ trailer< '.' attr=('func_closure' | 'func_doc' | 'func_globals' | 'func_name' | 'func_defaults' | 'func_code' Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_future.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_future.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_future.py Fri Nov 5 23:50:59 2010 @@ -9,6 +9,8 @@ from ..fixer_util import BlankLine class FixFuture(fixer_base.BaseFix): + BM_compatible = True + PATTERN = """import_from< 'from' module_name="__future__" 'import' any >""" # This should be run last -- some things check for the import Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_getcwdu.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_getcwdu.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_getcwdu.py Fri Nov 5 23:50:59 2010 @@ -8,6 +8,7 @@ from ..fixer_util import Name class FixGetcwdu(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ power< 'os' trailer< dot='.' name='getcwdu' > any* > Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_has_key.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_has_key.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_has_key.py Fri Nov 5 23:50:59 2010 @@ -37,6 +37,7 @@ class FixHasKey(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ anchor=power< Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_idioms.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_idioms.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_idioms.py Fri Nov 5 23:50:59 2010 @@ -35,7 +35,6 @@ TYPE = "power< 'type' trailer< '(' x=any ')' > >" class FixIdioms(fixer_base.BaseFix): - explicit = True # The user must ask for this fixer PATTERN = r""" Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_import.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_import.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_import.py Fri Nov 5 23:50:59 2010 @@ -36,6 +36,7 @@ class FixImport(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ import_from< 'from' imp=any 'import' ['('] any [')'] > Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_imports.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_imports.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_imports.py Fri Nov 5 23:50:59 2010 @@ -84,6 +84,8 @@ class FixImports(fixer_base.BaseFix): + BM_compatible = True + keep_line_order = True # This is overridden in fix_imports2. mapping = MAPPING Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_input.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_input.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_input.py Fri Nov 5 23:50:59 2010 @@ -11,7 +11,7 @@ class FixInput(fixer_base.BaseFix): - + BM_compatible = True PATTERN = """ power< 'input' args=trailer< '(' [any] ')' > > """ Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_intern.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_intern.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_intern.py Fri Nov 5 23:50:59 2010 @@ -12,6 +12,8 @@ class FixIntern(fixer_base.BaseFix): + BM_compatible = True + order = "pre" PATTERN = """ power< 'intern' Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_isinstance.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_isinstance.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_isinstance.py Fri Nov 5 23:50:59 2010 @@ -14,7 +14,7 @@ class FixIsinstance(fixer_base.BaseFix): - + BM_compatible = True PATTERN = """ power< 'isinstance' Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_itertools.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_itertools.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_itertools.py Fri Nov 5 23:50:59 2010 @@ -12,6 +12,7 @@ from ..fixer_util import Name class FixItertools(fixer_base.BaseFix): + BM_compatible = True it_funcs = "('imap'|'ifilter'|'izip'|'ifilterfalse')" PATTERN = """ power< it='itertools' Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_itertools_imports.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_itertools_imports.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_itertools_imports.py Fri Nov 5 23:50:59 2010 @@ -6,6 +6,7 @@ class FixItertoolsImports(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ import_from< 'from' 'itertools' 'import' imports=any > """ %(locals()) Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_long.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_long.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_long.py Fri Nov 5 23:50:59 2010 @@ -10,7 +10,7 @@ class FixLong(fixer_base.BaseFix): - + BM_compatible = True PATTERN = "'long'" def transform(self, node, results): Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_map.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_map.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_map.py Fri Nov 5 23:50:59 2010 @@ -26,6 +26,7 @@ from ..pygram import python_symbols as syms class FixMap(fixer_base.ConditionalFix): + BM_compatible = True PATTERN = """ map_none=power< Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_metaclass.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_metaclass.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_metaclass.py Fri Nov 5 23:50:59 2010 @@ -143,6 +143,7 @@ class FixMetaclass(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ classdef Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_methodattrs.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_methodattrs.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_methodattrs.py Fri Nov 5 23:50:59 2010 @@ -13,6 +13,7 @@ } class FixMethodattrs(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ power< any+ trailer< '.' attr=('im_func' | 'im_self' | 'im_class') > any* > """ Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_next.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_next.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_next.py Fri Nov 5 23:50:59 2010 @@ -15,6 +15,7 @@ class FixNext(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ power< base=any+ trailer< '.' attr='next' > trailer< '(' ')' > > | Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_nonzero.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_nonzero.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_nonzero.py Fri Nov 5 23:50:59 2010 @@ -6,6 +6,7 @@ from ..fixer_util import Name, syms class FixNonzero(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ classdef< 'class' any+ ':' suite< any* Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_operator.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_operator.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_operator.py Fri Nov 5 23:50:59 2010 @@ -14,7 +14,16 @@ from lib2to3.fixer_util import Call, Name, String, touch_import +def invocation(s): + def dec(f): + f.invocation = s + return f + return dec + + class FixOperator(fixer_base.BaseFix): + BM_compatible = True + order = "pre" methods = """ method=('isCallable'|'sequenceIncludes' @@ -34,34 +43,34 @@ if method is not None: return method(node, results) + @invocation("operator.contains(%s)") def _sequenceIncludes(self, node, results): - """operator.contains(%s)""" return self._handle_rename(node, results, u"contains") + @invocation("hasattr(%s, '__call__')") def _isCallable(self, node, results): - """hasattr(%s, '__call__')""" obj = results["obj"] args = [obj.clone(), String(u", "), String(u"'__call__'")] return Call(Name(u"hasattr"), args, prefix=node.prefix) + @invocation("operator.mul(%s)") def _repeat(self, node, results): - """operator.mul(%s)""" return self._handle_rename(node, results, u"mul") + @invocation("operator.imul(%s)") def _irepeat(self, node, results): - """operator.imul(%s)""" return self._handle_rename(node, results, u"imul") + @invocation("isinstance(%s, collections.Sequence)") def _isSequenceType(self, node, results): - """isinstance(%s, collections.Sequence)""" return self._handle_type2abc(node, results, u"collections", u"Sequence") + @invocation("isinstance(%s, collections.Mapping)") def _isMappingType(self, node, results): - """isinstance(%s, collections.Mapping)""" return self._handle_type2abc(node, results, u"collections", u"Mapping") + @invocation("isinstance(%s, numbers.Number)") def _isNumberType(self, node, results): - """isinstance(%s, numbers.Number)""" return self._handle_type2abc(node, results, u"numbers", u"Number") def _handle_rename(self, node, results, name): @@ -82,6 +91,6 @@ return method else: sub = (unicode(results["obj"]),) - invocation_str = unicode(method.__doc__) % sub + invocation_str = unicode(method.invocation) % sub self.warning(node, u"You should use '%s' here." % invocation_str) return None Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_paren.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_paren.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_paren.py Fri Nov 5 23:50:59 2010 @@ -10,6 +10,8 @@ # XXX This doesn't support nested for loops like [x for x in 1, 2 for x in 1, 2] class FixParen(fixer_base.BaseFix): + BM_compatible = True + PATTERN = """ atom< ('[' | '(') (listmaker< any Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_print.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_print.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_print.py Fri Nov 5 23:50:59 2010 @@ -28,6 +28,8 @@ class FixPrint(fixer_base.BaseFix): + BM_compatible = True + PATTERN = """ simple_stmt< any* bare='print' any* > | print_stmt """ Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_raise.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_raise.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_raise.py Fri Nov 5 23:50:59 2010 @@ -4,6 +4,7 @@ raise E -> raise E raise E, V -> raise E(V) raise E, V, T -> raise E(V).with_traceback(T) +raise E, None, T -> raise E.with_traceback(T) raise (((E, E'), E''), E'''), V -> raise E(V) raise "foo", V, T -> warns about string exceptions @@ -29,6 +30,7 @@ class FixRaise(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ raise_stmt< 'raise' exc=any [',' val=any [',' tb=any]] > """ @@ -37,8 +39,9 @@ syms = self.syms exc = results["exc"].clone() - if exc.type is token.STRING: - self.cannot_convert(node, "Python 3 does not support string exceptions") + if exc.type == token.STRING: + msg = "Python 3 does not support string exceptions" + self.cannot_convert(node, msg) return # Python 2 supports @@ -52,7 +55,7 @@ # exc.children[1:-1] is the unparenthesized tuple # exc.children[1].children[0] is the first element of the tuple exc = exc.children[1].children[0].clone() - exc.prefix = " " + exc.prefix = u" " if "val" not in results: # One-argument raise @@ -71,7 +74,12 @@ tb = results["tb"].clone() tb.prefix = u"" - e = Call(exc, args) + e = exc + # If there's a traceback and None is passed as the value, then don't + # add a call, since the user probably just wants to add a + # traceback. See issue #9661. + if val.type != token.NAME or val.value != u"None": + e = Call(exc, args) with_tb = Attr(e, Name(u'with_traceback')) + [ArgList([tb])] new = pytree.Node(syms.simple_stmt, [Name(u"raise")] + with_tb) new.prefix = node.prefix Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_raw_input.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_raw_input.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_raw_input.py Fri Nov 5 23:50:59 2010 @@ -7,6 +7,7 @@ class FixRawInput(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ power< name='raw_input' trailer< '(' [any] ')' > any* > """ Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_reduce.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_reduce.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_reduce.py Fri Nov 5 23:50:59 2010 @@ -14,6 +14,9 @@ class FixReduce(fixer_base.BaseFix): + BM_compatible = True + order = "pre" + PATTERN = """ power< 'reduce' trailer< '(' Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_renames.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_renames.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_renames.py Fri Nov 5 23:50:59 2010 @@ -40,6 +40,7 @@ class FixRenames(fixer_base.BaseFix): + BM_compatible = True PATTERN = "|".join(build_pattern()) order = "pre" # Pre-order tree traversal Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_repr.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_repr.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_repr.py Fri Nov 5 23:50:59 2010 @@ -10,6 +10,7 @@ class FixRepr(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ atom < '`' expr=any '`' > """ Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_set_literal.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_set_literal.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_set_literal.py Fri Nov 5 23:50:59 2010 @@ -11,6 +11,7 @@ class FixSetLiteral(fixer_base.BaseFix): + BM_compatible = True explicit = True PATTERN = """power< 'set' trailer< '(' Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_standarderror.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_standarderror.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_standarderror.py Fri Nov 5 23:50:59 2010 @@ -9,7 +9,7 @@ class FixStandarderror(fixer_base.BaseFix): - + BM_compatible = True PATTERN = """ 'StandardError' """ Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_sys_exc.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_sys_exc.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_sys_exc.py Fri Nov 5 23:50:59 2010 @@ -14,6 +14,7 @@ class FixSysExc(fixer_base.BaseFix): # This order matches the ordering of sys.exc_info(). exc_info = [u"exc_type", u"exc_value", u"exc_traceback"] + BM_compatible = True PATTERN = """ power< 'sys' trailer< dot='.' attribute=(%s) > > """ % '|'.join("'%s'" % e for e in exc_info) Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_throw.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_throw.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_throw.py Fri Nov 5 23:50:59 2010 @@ -14,7 +14,7 @@ from ..fixer_util import Name, Call, ArgList, Attr, is_tuple class FixThrow(fixer_base.BaseFix): - + BM_compatible = True PATTERN = """ power< any trailer< '.' 'throw' > trailer< '(' args=arglist< exc=any ',' val=any [',' tb=any] > ')' > Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_tuple_params.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_tuple_params.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_tuple_params.py Fri Nov 5 23:50:59 2010 @@ -29,6 +29,10 @@ stmt.children[0].type == token.STRING class FixTupleParams(fixer_base.BaseFix): + run_order = 4 #use a lower order since lambda is part of other + #patterns + BM_compatible = True + PATTERN = """ funcdef< 'def' any parameters< '(' args=any ')' > ['->' any] ':' suite=any+ > Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_types.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_types.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_types.py Fri Nov 5 23:50:59 2010 @@ -52,7 +52,7 @@ _pats = ["power< 'types' trailer< '.' name='%s' > >" % t for t in _TYPE_MAPPING] class FixTypes(fixer_base.BaseFix): - + BM_compatible = True PATTERN = '|'.join(_pats) def transform(self, node, results): Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_unicode.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_unicode.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_unicode.py Fri Nov 5 23:50:59 2010 @@ -10,7 +10,7 @@ _literal_re = re.compile(ur"[uU][rR]?[\'\"]") class FixUnicode(fixer_base.BaseFix): - + BM_compatible = True PATTERN = "STRING | 'unicode' | 'unichr'" def transform(self, node, results): Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_urllib.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_urllib.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_urllib.py Fri Nov 5 23:50:59 2010 @@ -8,7 +8,7 @@ from lib2to3.fixes.fix_imports import alternates, FixImports from lib2to3 import fixer_base from lib2to3.fixer_util import (Name, Comma, FromImport, Newline, - find_indentation) + find_indentation, Node, syms) MAPPING = {"urllib": [ ("urllib.request", @@ -121,26 +121,37 @@ mod_dict = {} members = results["members"] for member in members: - member = member.value # we only care about the actual members - if member != u",": + if member.type == syms.import_as_name: + as_name = member.children[2].value + member_name = member.children[0].value + else: + member_name = member.value + as_name = None + if member_name != u",": for change in MAPPING[mod_member.value]: - if member in change[1]: - if change[0] in mod_dict: - mod_dict[change[0]].append(member) - else: - mod_dict[change[0]] = [member] + if member_name in change[1]: + if change[0] not in mod_dict: modules.append(change[0]) + mod_dict.setdefault(change[0], []).append(member) new_nodes = [] indentation = find_indentation(node) first = True + def handle_name(name, prefix): + if name.type == syms.import_as_name: + kids = [Name(name.children[0].value, prefix=prefix), + name.children[1].clone(), + name.children[2].clone()] + return [Node(syms.import_as_name, kids)] + return [Name(name.value, prefix=prefix)] for module in modules: elts = mod_dict[module] names = [] for elt in elts[:-1]: - names.extend([Name(elt, prefix=pref), Comma()]) - names.append(Name(elts[-1], prefix=pref)) + names.extend(handle_name(elt, pref)) + names.append(Comma()) + names.extend(handle_name(elts[-1], pref)) new = FromImport(module, names) if not first or node.parent.prefix.endswith(indentation): new.prefix = indentation Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_xrange.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_xrange.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_xrange.py Fri Nov 5 23:50:59 2010 @@ -10,7 +10,7 @@ class FixXrange(fixer_base.BaseFix): - + BM_compatible = True PATTERN = """ power< (name='range'|name='xrange') trailer< '(' args=any ')' > Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_xreadlines.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_xreadlines.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_xreadlines.py Fri Nov 5 23:50:59 2010 @@ -9,6 +9,7 @@ class FixXreadlines(fixer_base.BaseFix): + BM_compatible = True PATTERN = """ power< call=any+ trailer< '.' 'xreadlines' > trailer< '(' ')' > > | Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_zip.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_zip.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/fixes/fix_zip.py Fri Nov 5 23:50:59 2010 @@ -13,6 +13,7 @@ class FixZip(fixer_base.ConditionalFix): + BM_compatible = True PATTERN = """ power< 'zip' args=trailer< '(' [any] ')' > > Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/patcomp.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/patcomp.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/patcomp.py Fri Nov 5 23:50:59 2010 @@ -52,14 +52,17 @@ self.pysyms = pygram.python_symbols self.driver = driver.Driver(self.grammar, convert=pattern_convert) - def compile_pattern(self, input, debug=False): + def compile_pattern(self, input, debug=False, with_tree=False): """Compiles a pattern string to a nested pytree.*Pattern object.""" tokens = tokenize_wrapper(input) try: root = self.driver.parse_tokens(tokens, debug=debug) - except parse.ParseError, e: + except parse.ParseError as e: raise PatternSyntaxError(str(e)) - return self.compile_node(root) + if with_tree: + return self.compile_node(root), root + else: + return self.compile_node(root) def compile_node(self, node): """Compiles a node, recursively. Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/pygram.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/pygram.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/pygram.py Fri Nov 5 23:50:59 2010 @@ -13,6 +13,8 @@ # The grammar file _GRAMMAR_FILE = os.path.join(os.path.dirname(__file__), "Grammar.txt") +_PATTERN_GRAMMAR_FILE = os.path.join(os.path.dirname(__file__), + "PatternGrammar.txt") class Symbols(object): @@ -33,3 +35,6 @@ python_grammar_no_print_statement = python_grammar.copy() del python_grammar_no_print_statement.keywords["print"] + +pattern_grammar = driver.load_grammar(_PATTERN_GRAMMAR_FILE) +pattern_symbols = Symbols(pattern_grammar) Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/pytree.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/pytree.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/pytree.py Fri Nov 5 23:50:59 2010 @@ -16,7 +16,6 @@ import warnings from StringIO import StringIO - HUGE = 0x7FFFFFFF # maximum repeat count, default max _type_reprs = {} @@ -30,7 +29,6 @@ if type(val) == int: _type_reprs[val] = name return _type_reprs.setdefault(type_num, type_num) - class Base(object): """ @@ -47,6 +45,7 @@ parent = None # Parent node pointer, or None children = () # Tuple of subnodes was_changed = False + was_checked = False def __new__(cls, *args, **kwds): """Constructor that prevents Base from being instantiated.""" @@ -213,6 +212,16 @@ return None return self.parent.children[i-1] + def leaves(self): + for child in self.children: + for x in child.leaves(): + yield x + + def depth(self): + if self.parent is None: + return 0 + return 1 + self.parent.depth() + def get_suffix(self): """ Return the string immediately following the invocant node. This is @@ -227,12 +236,14 @@ def __str__(self): return unicode(self).encode("ascii") - class Node(Base): """Concrete implementation for interior nodes.""" - def __init__(self, type, children, context=None, prefix=None): + def __init__(self,type, children, + context=None, + prefix=None, + fixers_applied=None): """ Initializer. @@ -249,6 +260,10 @@ ch.parent = self if prefix is not None: self.prefix = prefix + if fixers_applied: + self.fixers_applied = fixers_applied[:] + else: + self.fixers_applied = None def __repr__(self): """Return a canonical string representation.""" @@ -273,7 +288,8 @@ def clone(self): """Return a cloned (deep) copy of self.""" - return Node(self.type, [ch.clone() for ch in self.children]) + return Node(self.type, [ch.clone() for ch in self.children], + fixers_applied=self.fixers_applied) def post_order(self): """Return a post-order iterator for the tree.""" @@ -341,7 +357,10 @@ lineno = 0 # Line where this token starts in the input column = 0 # Column where this token tarts in the input - def __init__(self, type, value, context=None, prefix=None): + def __init__(self, type, value, + context=None, + prefix=None, + fixers_applied=[]): """ Initializer. @@ -355,6 +374,7 @@ self.value = value if prefix is not None: self._prefix = prefix + self.fixers_applied = fixers_applied[:] def __repr__(self): """Return a canonical string representation.""" @@ -380,7 +400,11 @@ def clone(self): """Return a cloned (deep) copy of self.""" return Leaf(self.type, self.value, - (self.prefix, (self.lineno, self.column))) + (self.prefix, (self.lineno, self.column)), + fixers_applied=self.fixers_applied) + + def leaves(self): + yield self def post_order(self): """Return a post-order iterator for the tree.""" Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/refactor.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/refactor.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/refactor.py Fri Nov 5 23:50:59 2010 @@ -24,7 +24,10 @@ # Local imports from .pgen2 import driver, tokenize, token +from .fixer_util import find_root from . import pytree, pygram +from . import btm_utils as bu +from . import btm_matcher as bm def get_all_fix_names(fixer_pkg, remove_prefix=True): @@ -201,11 +204,28 @@ logger=self.logger) self.pre_order, self.post_order = self.get_fixers() - self.pre_order_heads = _get_headnode_dict(self.pre_order) - self.post_order_heads = _get_headnode_dict(self.post_order) self.files = [] # List of files that were or should be modified + self.BM = bm.BottomMatcher() + self.bmi_pre_order = [] # Bottom Matcher incompatible fixers + self.bmi_post_order = [] + + for fixer in chain(self.post_order, self.pre_order): + if fixer.BM_compatible: + self.BM.add_fixer(fixer) + # remove fixers that will be handled by the bottom-up + # matcher + elif fixer in self.pre_order: + self.bmi_pre_order.append(fixer) + elif fixer in self.post_order: + self.bmi_post_order.append(fixer) + + self.bmi_pre_order_heads = _get_headnode_dict(self.bmi_pre_order) + self.bmi_post_order_heads = _get_headnode_dict(self.bmi_post_order) + + + def get_fixers(self): """Inspects the options to load the requested patterns and handlers. @@ -268,6 +288,7 @@ def refactor(self, items, write=False, doctests_only=False): """Refactor a list of files and directories.""" + for dir_or_file in items: if os.path.isdir(dir_or_file): self.refactor_dir(dir_or_file, write, doctests_only) @@ -299,7 +320,7 @@ """ try: f = open(filename, "rb") - except IOError, err: + except IOError as err: self.log_error("Can't open %s: %s", filename, err) return None, None try: @@ -348,7 +369,7 @@ self.driver.grammar = pygram.python_grammar_no_print_statement try: tree = self.driver.parse_string(data) - except Exception, err: + except Exception as err: self.log_error("Can't parse %s: %s: %s", name, err.__class__.__name__, err) return @@ -378,6 +399,10 @@ def refactor_tree(self, tree, name): """Refactors a parse tree (modifying the tree in place). + For compatible patterns the bottom matcher module is + used. Otherwise the tree is traversed node-to-node for + matches. + Args: tree: a pytree.Node instance representing the root of the tree to be refactored. @@ -386,11 +411,65 @@ Returns: True if the tree was modified, False otherwise. """ + for fixer in chain(self.pre_order, self.post_order): fixer.start_tree(tree, name) - self.traverse_by(self.pre_order_heads, tree.pre_order()) - self.traverse_by(self.post_order_heads, tree.post_order()) + #use traditional matching for the incompatible fixers + self.traverse_by(self.bmi_pre_order_heads, tree.pre_order()) + self.traverse_by(self.bmi_post_order_heads, tree.post_order()) + + # obtain a set of candidate nodes + match_set = self.BM.run(tree.leaves()) + + while any(match_set.values()): + for fixer in self.BM.fixers: + if fixer in match_set and match_set[fixer]: + #sort by depth; apply fixers from bottom(of the AST) to top + match_set[fixer].sort(key=pytree.Base.depth, reverse=True) + + if fixer.keep_line_order: + #some fixers(eg fix_imports) must be applied + #with the original file's line order + match_set[fixer].sort(key=pytree.Base.get_lineno) + + for node in list(match_set[fixer]): + if node in match_set[fixer]: + match_set[fixer].remove(node) + + try: + find_root(node) + except AssertionError: + # this node has been cut off from a + # previous transformation ; skip + continue + + if node.fixers_applied and fixer in node.fixers_applied: + # do not apply the same fixer again + continue + + results = fixer.match(node) + + if results: + new = fixer.transform(node, results) + if new is not None: + node.replace(new) + #new.fixers_applied.append(fixer) + for node in new.post_order(): + # do not apply the fixer again to + # this or any subnode + if not node.fixers_applied: + node.fixers_applied = [] + node.fixers_applied.append(fixer) + + # update the original match set for + # the added code + new_matches = self.BM.run(new.leaves()) + for fxr in new_matches: + if not fxr in match_set: + match_set[fxr]=[] + + match_set[fxr].extend(new_matches[fxr]) for fixer in chain(self.pre_order, self.post_order): fixer.finish_tree(tree, name) @@ -448,12 +527,12 @@ """ try: f = _open_with_encoding(filename, "w", encoding=encoding) - except os.error, err: + except os.error as err: self.log_error("Can't create %s: %s", filename, err) return try: f.write(_to_system_newlines(new_text)) - except os.error, err: + except os.error as err: self.log_error("Can't write %s: %s", filename, err) finally: f.close() @@ -516,7 +595,7 @@ """ try: tree = self.parse_block(block, lineno, indent) - except Exception, err: + except Exception as err: if self.logger.isEnabledFor(logging.DEBUG): for line in block: self.log_debug("Source: %s", line.rstrip(u"\n")) Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/tests/data/bom.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/tests/data/bom.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/tests/data/bom.py Fri Nov 5 23:50:59 2010 @@ -1,3 +1,2 @@ ???# coding: utf-8 print "BOM BOOM!" - Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/tests/test_fixers.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/tests/test_fixers.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/tests/test_fixers.py Fri Nov 5 23:50:59 2010 @@ -868,6 +868,11 @@ raise Exception(5).with_traceback(6) # foo""" self.check(b, a) + def test_None_value(self): + b = """raise Exception(5), None, tb""" + a = """raise Exception(5).with_traceback(tb)""" + self.check(b, a) + def test_tuple_value(self): b = """raise Exception, (5, 6, 7)""" a = """raise Exception(5, 6, 7)""" @@ -1812,6 +1817,9 @@ b = "from %s import %s as foo_bar" % (old, member) a = "from %s import %s as foo_bar" % (new, member) self.check(b, a) + b = "from %s import %s as blah, %s" % (old, member, member) + a = "from %s import %s as blah, %s" % (new, member, member) + self.check(b, a) def test_star(self): for old in self.modules: Modified: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/tests/test_pytree.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/tests/test_pytree.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/tests/test_pytree.py Fri Nov 5 23:50:59 2010 @@ -178,6 +178,27 @@ self.assertEqual(str(n1), "foo**bar") self.assertTrue(isinstance(n1.children, list)) + def test_leaves(self): + l1 = pytree.Leaf(100, "foo") + l2 = pytree.Leaf(100, "bar") + l3 = pytree.Leaf(100, "fooey") + n2 = pytree.Node(1000, [l1, l2]) + n3 = pytree.Node(1000, [l3]) + n1 = pytree.Node(1000, [n2, n3]) + + self.assertEqual(list(n1.leaves()), [l1, l2, l3]) + + def test_depth(self): + l1 = pytree.Leaf(100, "foo") + l2 = pytree.Leaf(100, "bar") + n2 = pytree.Node(1000, [l1, l2]) + n3 = pytree.Node(1000, []) + n1 = pytree.Node(1000, [n2, n3]) + + self.assertEqual(l1.depth(), 2) + self.assertEqual(n3.depth(), 1) + self.assertEqual(n1.depth(), 0) + def test_post_order(self): l1 = pytree.Leaf(100, "foo") l2 = pytree.Leaf(100, "bar") Modified: pypy/branch/fast-forward/lib-python/2.7.0/logging/__init__.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/logging/__init__.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/logging/__init__.py Fri Nov 5 23:50:59 2010 @@ -1211,7 +1211,7 @@ if filename == _srcfile: f = f.f_back continue - rv = (filename, f.f_lineno, co.co_name) + rv = (co.co_filename, f.f_lineno, co.co_name) break return rv @@ -1263,20 +1263,23 @@ """ Add the specified handler to this logger. """ - if not (hdlr in self.handlers): - self.handlers.append(hdlr) + _acquireLock() + try: + if not (hdlr in self.handlers): + self.handlers.append(hdlr) + finally: + _releaseLock() def removeHandler(self, hdlr): """ Remove the specified handler from this logger. """ - if hdlr in self.handlers: - #hdlr.close() - hdlr.acquire() - try: + _acquireLock() + try: + if hdlr in self.handlers: self.handlers.remove(hdlr) - finally: - hdlr.release() + finally: + _releaseLock() def callHandlers(self, record): """ @@ -1653,9 +1656,15 @@ a NullHandler and add it to the top-level logger of the library module or package. """ + def handle(self, record): + pass + def emit(self, record): pass + def createLock(self): + self.lock = None + # Warnings integration _warnings_showwarning = None Modified: pypy/branch/fast-forward/lib-python/2.7.0/logging/config.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/logging/config.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/logging/config.py Fri Nov 5 23:50:59 2010 @@ -866,6 +866,7 @@ logging._acquireLock() abort = self.abort logging._releaseLock() + self.socket.close() class Server(threading.Thread): @@ -895,8 +896,10 @@ Stop the listening server which was created with a call to listen(). """ global _listener - if _listener: - logging._acquireLock() - _listener.abort = 1 - _listener = None + logging._acquireLock() + try: + if _listener: + _listener.abort = 1 + _listener = None + finally: logging._releaseLock() Modified: pypy/branch/fast-forward/lib-python/2.7.0/mimetypes.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/mimetypes.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/mimetypes.py Fri Nov 5 23:50:59 2010 @@ -199,9 +199,8 @@ list of standard types, else to the list of non-standard types. """ - fp = open(filename) - self.readfp(fp, strict) - fp.close() + with open(filename) as fp: + self.readfp(fp, strict) def readfp(self, fp, strict=True): """ @@ -258,18 +257,19 @@ with _winreg.OpenKey(_winreg.HKEY_CLASSES_ROOT, r'MIME\Database\Content Type') as mimedb: for ctype in enum_types(mimedb): - with _winreg.OpenKey(mimedb, ctype) as key: - try: - suffix, datatype = _winreg.QueryValueEx(key, 'Extension') - except EnvironmentError: - continue - if datatype != _winreg.REG_SZ: - continue - try: - suffix = suffix.encode(default_encoding) # omit in 3.x! - except UnicodeEncodeError: - continue - self.add_type(ctype, suffix, strict) + try: + with _winreg.OpenKey(mimedb, ctype) as key: + suffix, datatype = _winreg.QueryValueEx(key, + 'Extension') + except EnvironmentError: + continue + if datatype != _winreg.REG_SZ: + continue + try: + suffix = suffix.encode(default_encoding) # omit in 3.x! + except UnicodeEncodeError: + continue + self.add_type(ctype, suffix, strict) def guess_type(url, strict=True): @@ -356,7 +356,7 @@ files = knownfiles for file in files: if os.path.isfile(file): - db.readfp(open(file)) + db.read(file) encodings_map = db.encodings_map suffix_map = db.suffix_map types_map = db.types_map[True] Modified: pypy/branch/fast-forward/lib-python/2.7.0/multiprocessing/__init__.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/multiprocessing/__init__.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/multiprocessing/__init__.py Fri Nov 5 23:50:59 2010 @@ -116,7 +116,8 @@ num = 0 elif 'bsd' in sys.platform or sys.platform == 'darwin': try: - num = int(os.popen('sysctl -n hw.ncpu').read()) + with os.popen('sysctl -n hw.ncpu') as p: + num = int(p.read()) except ValueError: num = 0 else: Modified: pypy/branch/fast-forward/lib-python/2.7.0/ntpath.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/ntpath.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/ntpath.py Fri Nov 5 23:50:59 2010 @@ -483,27 +483,38 @@ supports_unicode_filenames = (hasattr(sys, "getwindowsversion") and sys.getwindowsversion()[3] >= 2) +def _abspath_split(path): + abs = abspath(normpath(path)) + prefix, rest = splitunc(abs) + is_unc = bool(prefix) + if not is_unc: + prefix, rest = splitdrive(abs) + return is_unc, prefix, [x for x in rest.split(sep) if x] + def relpath(path, start=curdir): """Return a relative version of a path""" if not path: raise ValueError("no path specified") - start_list = abspath(start).split(sep) - path_list = abspath(path).split(sep) - if start_list[0].lower() != path_list[0].lower(): - unc_path, rest = splitunc(path) - unc_start, rest = splitunc(start) - if bool(unc_path) ^ bool(unc_start): - raise ValueError("Cannot mix UNC and non-UNC paths (%s and %s)" - % (path, start)) + + start_is_unc, start_prefix, start_list = _abspath_split(start) + path_is_unc, path_prefix, path_list = _abspath_split(path) + + if path_is_unc ^ start_is_unc: + raise ValueError("Cannot mix UNC and non-UNC paths (%s and %s)" + % (path, start)) + if path_prefix.lower() != start_prefix.lower(): + if path_is_unc: + raise ValueError("path is on UNC root %s, start on UNC root %s" + % (path_prefix, start_prefix)) else: raise ValueError("path is on drive %s, start on drive %s" - % (path_list[0], start_list[0])) + % (path_prefix, start_prefix)) # Work out how much of the filepath is shared by start and path. - for i in range(min(len(start_list), len(path_list))): - if start_list[i].lower() != path_list[i].lower(): + i = 0 + for e1, e2 in zip(start_list, path_list): + if e1.lower() != e2.lower(): break - else: i += 1 rel_list = [pardir] * (len(start_list)-i) + path_list[i:] Modified: pypy/branch/fast-forward/lib-python/2.7.0/posixpath.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/posixpath.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/posixpath.py Fri Nov 5 23:50:59 2010 @@ -11,6 +11,7 @@ """ import os +import sys import stat import genericpath import warnings @@ -394,7 +395,7 @@ path = normpath(resolved) return path -supports_unicode_filenames = False +supports_unicode_filenames = (sys.platform == 'darwin') def relpath(path, start=curdir): """Return a relative version of a path""" @@ -402,8 +403,8 @@ if not path: raise ValueError("no path specified") - start_list = abspath(start).split(sep) - path_list = abspath(path).split(sep) + start_list = [x for x in abspath(start).split(sep) if x] + path_list = [x for x in abspath(path).split(sep) if x] # Work out how much of the filepath is shared by start and path. i = len(commonprefix([start_list, path_list])) Modified: pypy/branch/fast-forward/lib-python/2.7.0/runpy.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/runpy.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/runpy.py Fri Nov 5 23:50:59 2010 @@ -261,7 +261,7 @@ _ModifiedArgv0(path_name): mod_globals = temp_module.module.__dict__ return _run_code(code, mod_globals, init_globals, - run_name, fname, loader, pkg_name) + run_name, fname, loader, pkg_name).copy() finally: try: sys.path.remove(path_name) Modified: pypy/branch/fast-forward/lib-python/2.7.0/site.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/site.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/site.py Fri Nov 5 23:50:59 2010 @@ -76,7 +76,11 @@ def makepath(*paths): - dir = os.path.abspath(os.path.join(*paths)) + dir = os.path.join(*paths) + try: + dir = os.path.abspath(dir) + except OSError: + pass return dir, os.path.normcase(dir) @@ -87,8 +91,8 @@ continue # don't mess with a PEP 302-supplied __file__ try: m.__file__ = os.path.abspath(m.__file__) - except AttributeError: - continue + except (AttributeError, OSError): + pass def removeduppaths(): Modified: pypy/branch/fast-forward/lib-python/2.7.0/smtpd.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/smtpd.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/smtpd.py Fri Nov 5 23:50:59 2010 @@ -35,7 +35,6 @@ and if remoteport is not given, then 25 is used. """ - # Overview: # # This file implements the minimal SMTP protocol as defined in RFC 821. It @@ -96,7 +95,6 @@ COMMASPACE = ', ' - def usage(code, msg=''): print >> sys.stderr, __doc__ % globals() if msg: @@ -104,7 +102,6 @@ sys.exit(code) - class SMTPChannel(asynchat.async_chat): COMMAND = 0 DATA = 1 @@ -276,7 +273,6 @@ self.push('354 End data with .') - class SMTPServer(asyncore.dispatcher): def __init__(self, localaddr, remoteaddr): self._localaddr = localaddr @@ -299,22 +295,11 @@ localaddr, remoteaddr) def handle_accept(self): - try: - conn, addr = self.accept() - except TypeError: - # sometimes accept() might return None - return - except socket.error, err: - # ECONNABORTED might be thrown - if err[0] != errno.ECONNABORTED: - raise - return - else: - # sometimes addr == None instead of (ip, port) - if addr == None: - return - print >> DEBUGSTREAM, 'Incoming connection from %s' % repr(addr) - channel = SMTPChannel(self, conn, addr) + pair = self.accept() + if pair is not None: + conn, addr = pair + print >> DEBUGSTREAM, 'Incoming connection from %s' % repr(addr) + channel = SMTPChannel(self, conn, addr) # API for "doing something useful with the message" def process_message(self, peer, mailfrom, rcpttos, data): @@ -342,7 +327,6 @@ raise NotImplementedError - class DebuggingServer(SMTPServer): # Do something with the gathered message def process_message(self, peer, mailfrom, rcpttos, data): @@ -358,7 +342,6 @@ print '------------ END MESSAGE ------------' - class PureProxy(SMTPServer): def process_message(self, peer, mailfrom, rcpttos, data): lines = data.split('\n') @@ -399,7 +382,6 @@ return refused - class MailmanProxy(PureProxy): def process_message(self, peer, mailfrom, rcpttos, data): from cStringIO import StringIO @@ -478,13 +460,11 @@ msg.Enqueue(mlist, torequest=1) - class Options: setuid = 1 classname = 'PureProxy' - def parseargs(): global DEBUGSTREAM try: @@ -541,7 +521,6 @@ return options - if __name__ == '__main__': options = parseargs() # Become nobody Modified: pypy/branch/fast-forward/lib-python/2.7.0/ssl.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/ssl.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/ssl.py Fri Nov 5 23:50:59 2010 @@ -184,14 +184,16 @@ else: return v else: - return socket.send(self, data, flags) + return self._sock.send(data, flags) - def sendto(self, data, addr, flags=0): + def sendto(self, data, flags_or_addr, addr=None): if self._sslobj: raise ValueError("sendto not allowed on instances of %s" % self.__class__) + elif addr is None: + return self._sock.sendto(data, flags_or_addr) else: - return socket.sendto(self, data, addr, flags) + return self._sock.sendto(data, flags_or_addr, addr) def sendall(self, data, flags=0): if self._sslobj: @@ -216,7 +218,7 @@ self.__class__) return self.read(buflen) else: - return socket.recv(self, buflen, flags) + return self._sock.recv(buflen, flags) def recv_into(self, buffer, nbytes=None, flags=0): if buffer and (nbytes is None): @@ -233,21 +235,21 @@ buffer[:v] = tmp_buffer return v else: - return socket.recv_into(self, buffer, nbytes, flags) + return self._sock.recv_into(buffer, nbytes, flags) - def recvfrom(self, addr, buflen=1024, flags=0): + def recvfrom(self, buflen=1024, flags=0): if self._sslobj: raise ValueError("recvfrom not allowed on instances of %s" % self.__class__) else: - return socket.recvfrom(self, addr, buflen, flags) + return self._sock.recvfrom(buflen, flags) def recvfrom_into(self, buffer, nbytes=None, flags=0): if self._sslobj: raise ValueError("recvfrom_into not allowed on instances of %s" % self.__class__) else: - return socket.recvfrom_into(self, buffer, nbytes, flags) + return self._sock.recvfrom_into(buffer, nbytes, flags) def pending(self): if self._sslobj: Modified: pypy/branch/fast-forward/lib-python/2.7.0/sysconfig.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/sysconfig.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/sysconfig.py Fri Nov 5 23:50:59 2010 @@ -93,21 +93,28 @@ _EXEC_PREFIX = os.path.normpath(sys.exec_prefix) _CONFIG_VARS = None _USER_BASE = None + +def _safe_realpath(path): + try: + return realpath(path) + except OSError: + return path + if sys.executable: - _PROJECT_BASE = os.path.dirname(realpath(sys.executable)) + _PROJECT_BASE = os.path.dirname(_safe_realpath(sys.executable)) else: # sys.executable can be empty if argv[0] has been changed and Python is # unable to retrieve the real program name - _PROJECT_BASE = realpath(os.getcwd()) + _PROJECT_BASE = _safe_realpath(os.getcwd()) if os.name == "nt" and "pcbuild" in _PROJECT_BASE[-8:].lower(): - _PROJECT_BASE = realpath(os.path.join(_PROJECT_BASE, pardir)) + _PROJECT_BASE = _safe_realpath(os.path.join(_PROJECT_BASE, pardir)) # PC/VS7.1 if os.name == "nt" and "\\pc\\v" in _PROJECT_BASE[-10:].lower(): - _PROJECT_BASE = realpath(os.path.join(_PROJECT_BASE, pardir, pardir)) + _PROJECT_BASE = _safe_realpath(os.path.join(_PROJECT_BASE, pardir, pardir)) # PC/AMD64 if os.name == "nt" and "\\pcbuild\\amd64" in _PROJECT_BASE[-14:].lower(): - _PROJECT_BASE = realpath(os.path.join(_PROJECT_BASE, pardir, pardir)) + _PROJECT_BASE = _safe_realpath(os.path.join(_PROJECT_BASE, pardir, pardir)) def is_python_build(): for fn in ("Setup.dist", "Setup.local"): @@ -251,6 +258,11 @@ else: # bogus variable reference; just drop it since we can't deal del notdone[name] + # strip spurious spaces + for k, v in done.items(): + if isinstance(v, str): + done[k] = v.strip() + # save the results in the global dictionary vars.update(done) return vars @@ -277,7 +289,8 @@ # load the installed pyconfig.h: config_h = get_config_h_filename() try: - parse_config_h(open(config_h), vars) + with open(config_h) as f: + parse_config_h(f, vars) except IOError, e: msg = "invalid Python installation: unable to open %s" % config_h if hasattr(e, "strerror"): @@ -314,7 +327,7 @@ vars['SO'] = '.pyd' vars['EXE'] = '.exe' vars['VERSION'] = _PY_VERSION_SHORT_NO_DOT - vars['BINDIR'] = os.path.dirname(realpath(sys.executable)) + vars['BINDIR'] = os.path.dirname(_safe_realpath(sys.executable)) # # public APIs @@ -434,8 +447,12 @@ # from a different directory. if _PYTHON_BUILD and os.name == "posix": base = _PROJECT_BASE + try: + cwd = os.getcwd() + except OSError: + cwd = None if (not os.path.isabs(_CONFIG_VARS['srcdir']) and - base != os.getcwd()): + base != cwd): # srcdir is relative and we are not in the same directory # as the executable. Assume executable is in the build # directory and make srcdir absolute. Modified: pypy/branch/fast-forward/lib-python/2.7.0/tarfile.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/tarfile.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/tarfile.py Fri Nov 5 23:50:59 2010 @@ -30,13 +30,13 @@ """Read from and write to tar format archives. """ -__version__ = "$Revision: 81667 $" +__version__ = "$Revision: 85213 $" # $Source$ version = "0.9.0" __author__ = "Lars Gust?bel (lars at gustaebel.de)" -__date__ = "$Date: 2010-06-03 14:34:14 +0200 (Thu, 03 Jun 2010) $" -__cvsid__ = "$Id: tarfile.py 81667 2010-06-03 12:34:14Z lars.gustaebel $" +__date__ = "$Date: 2010-10-04 17:37:53 +0200 (Mon, 04 Oct 2010) $" +__cvsid__ = "$Id: tarfile.py 85213 2010-10-04 15:37:53Z lars.gustaebel $" __credits__ = "Gustavo Niemeyer, Niels Gust?bel, Richard Townsend." #--------- @@ -928,8 +928,8 @@ self.chksum = 0 # header checksum self.type = REGTYPE # member type self.linkname = "" # link name - self.uname = "root" # user name - self.gname = "root" # group name + self.uname = "" # user name + self.gname = "" # group name self.devmajor = 0 # device major number self.devminor = 0 # device minor number @@ -1112,8 +1112,8 @@ info.get("type", REGTYPE), stn(info.get("linkname", ""), 100), stn(info.get("magic", POSIX_MAGIC), 8), - stn(info.get("uname", "root"), 32), - stn(info.get("gname", "root"), 32), + stn(info.get("uname", ""), 32), + stn(info.get("gname", ""), 32), itn(info.get("devmajor", 0), 8, format), itn(info.get("devminor", 0), 8, format), stn(info.get("prefix", ""), 155) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/regrtest.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/regrtest.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/regrtest.py Fri Nov 5 23:50:59 2010 @@ -133,11 +133,7 @@ decimal - Test the decimal module against a large suite that verifies compliance with standards. - compiler - Test the compiler package by compiling all the source - in the standard library and test suite. This takes - a long time. Enabling this resource also allows - test_tokenize to verify round-trip lexing on every - file in the test library. + cpu - Used for certain CPU-heavy tests. subprocess Run all tests for the subprocess module. @@ -215,7 +211,7 @@ from test import test_support RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'bsddb', - 'decimal', 'compiler', 'subprocess', 'urlfetch', 'gui', + 'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui', 'xpickle') TEMPDIR = os.path.abspath(tempfile.gettempdir()) @@ -365,9 +361,6 @@ usage(2, "-T and -j don't go together!") if use_mp and findleaks: usage(2, "-l and -j don't go together!") - if use_mp and max(sys.flags): - # TODO: inherit the environment and the flags - print "Warning: flags and environment variables are ignored with -j option" good = [] bad = [] @@ -496,6 +489,8 @@ ) yield (test, args_tuple) pending = tests_and_args() + opt_args = test_support.args_from_interpreter_flags() + base_cmd = [sys.executable] + opt_args + ['-m', 'test.regrtest'] def work(): # A worker thread. try: @@ -506,8 +501,7 @@ output.put((None, None, None, None)) return # -E is needed by some tests, e.g. test_import - popen = Popen([sys.executable, '-E', '-m', 'test.regrtest', - '--slaveargs', json.dumps(args_tuple)], + popen = Popen(base_cmd + ['--slaveargs', json.dumps(args_tuple)], stdout=PIPE, stderr=PIPE, universal_newlines=True, close_fds=(os.name != 'nt')) @@ -805,11 +799,12 @@ def get_asyncore_socket_map(self): asyncore = sys.modules.get('asyncore') - return asyncore and asyncore.socket_map or {} + # XXX Making a copy keeps objects alive until __exit__ gets called. + return asyncore and asyncore.socket_map.copy() or {} def restore_asyncore_socket_map(self, saved_map): asyncore = sys.modules.get('asyncore') if asyncore is not None: - asyncore.socket_map.clear() + asyncore.close_all(ignore_all=True) asyncore.socket_map.update(saved_map) def resource_info(self): @@ -825,9 +820,11 @@ return self def __exit__(self, exc_type, exc_val, exc_tb): + saved_values = self.saved_values + del self.saved_values for name, get, restore in self.resource_info(): current = get() - original = self.saved_values[name] + original = saved_values.pop(name) # Check for changes to the resource's value if current != original: self.changed = True @@ -929,6 +926,10 @@ def cleanup_test_droppings(testname, verbose): import shutil import stat + import gc + + # First kill any dangling references to open files etc. + gc.collect() # Try to clean up junk commonly left behind. While tests shouldn't leave # any files or directories behind, when a test fails that can be tedious @@ -1258,6 +1259,7 @@ test_bsddb3 test_curses test_epoll + test_gdb test_gdbm test_largefile test_locale Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_abc.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_abc.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_abc.py Fri Nov 5 23:50:59 2010 @@ -70,6 +70,13 @@ self.assertFalse(issubclass(OldstyleClass, A)) self.assertFalse(issubclass(A, OldstyleClass)) + def test_type_has_no_abstractmethods(self): + # type pretends not to have __abstractmethods__. + self.assertRaises(AttributeError, getattr, type, "__abstractmethods__") + class meta(type): + pass + self.assertRaises(AttributeError, getattr, meta, "__abstractmethods__") + def test_isinstance_class(self): class A: __metaclass__ = abc.ABCMeta Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_argparse.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_argparse.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_argparse.py Fri Nov 5 23:50:59 2010 @@ -1,6 +1,7 @@ # Author: Steven J. Bethard . import codecs +import inspect import os import shutil import sys @@ -27,6 +28,13 @@ print(obj2) super(TestCase, self).assertEqual(obj1, obj2) + def setUp(self): + # The tests assume that line wrapping occurs at 80 columns, but this + # behaviour can be overridden by setting the COLUMNS environment + # variable. To ensure that this assumption is true, unset COLUMNS. + env = test_support.EnvironmentVarGuard() + env.unset("COLUMNS") + self.addCleanup(env.__exit__) class TempDirMixin(object): @@ -460,6 +468,30 @@ ('/ba +f', NS(f=True, bar=None, baz=42)) ] + +class TestOptionalsAlternatePrefixCharsMultipleShortArgs(ParserTestCase): + """Verify that Optionals must be called with their defined prefixes""" + + parser_signature = Sig(prefix_chars='+-', add_help=False) + argument_signatures = [ + Sig('-x', action='store_true'), + Sig('+y', action='store_true'), + Sig('+z', action='store_true'), + ] + failures = ['-w', + '-xyz', + '+x', + '-y', + '+xyz', + ] + successes = [ + ('', NS(x=False, y=False, z=False)), + ('-x', NS(x=True, y=False, z=False)), + ('+y -x', NS(x=True, y=True, z=False)), + ('+yz -x', NS(x=True, y=True, z=True)), + ] + + class TestOptionalsShortLong(ParserTestCase): """Test a combination of single- and double-dash option strings""" @@ -1726,6 +1758,7 @@ return parser def setUp(self): + super(TestAddSubparsers, self).setUp() self.parser = self._get_parser() self.command_help_parser = self._get_parser(subparser_help=True) @@ -1751,6 +1784,28 @@ NS(foo=True, bar=0.125, w=None, x='c'), ) + def test_parse_known_args(self): + self.assertEqual( + self.parser.parse_known_args('0.5 1 b -w 7'.split()), + (NS(foo=False, bar=0.5, w=7, x='b'), []), + ) + self.assertEqual( + self.parser.parse_known_args('0.5 -p 1 b -w 7'.split()), + (NS(foo=False, bar=0.5, w=7, x='b'), ['-p']), + ) + self.assertEqual( + self.parser.parse_known_args('0.5 1 b -w 7 -p'.split()), + (NS(foo=False, bar=0.5, w=7, x='b'), ['-p']), + ) + self.assertEqual( + self.parser.parse_known_args('0.5 1 b -q -rs -w 7'.split()), + (NS(foo=False, bar=0.5, w=7, x='b'), ['-q', '-rs']), + ) + self.assertEqual( + self.parser.parse_known_args('0.5 -W 1 b -X Y -w 7 Z'.split()), + (NS(foo=False, bar=0.5, w=7, x='b'), ['-W', '-X', 'Y', 'Z']), + ) + def test_dest(self): parser = ErrorRaisingArgumentParser() parser.add_argument('--foo', action='store_true') @@ -1953,6 +2008,7 @@ self.assertRaises(ArgumentParserError, *args, **kwargs) def setUp(self): + super(TestParentParsers, self).setUp() self.wxyz_parent = ErrorRaisingArgumentParser(add_help=False) self.wxyz_parent.add_argument('--w') x_group = self.wxyz_parent.add_argument_group('x') @@ -2140,6 +2196,25 @@ raises(ValueError, add_argument, 'bar', nargs=1) raises(ValueError, add_argument, 'bar', nargs=argparse.PARSER) + def test_help(self): + parser = ErrorRaisingArgumentParser(prog='PROG') + group1 = parser.add_mutually_exclusive_group() + group1.add_argument('--foo', action='store_true') + group1.add_argument('--bar', action='store_false') + group2 = parser.add_mutually_exclusive_group() + group2.add_argument('--soup', action='store_true') + group2.add_argument('--nuts', action='store_false') + expected = '''\ + usage: PROG [-h] [--foo | --bar] [--soup | --nuts] + + optional arguments: + -h, --help show this help message and exit + --foo + --bar + --soup + --nuts + ''' + self.assertEqual(parser.format_help(), textwrap.dedent(expected)) class MEMixin(object): @@ -4171,7 +4246,8 @@ def _test_module_encoding(self, path): path, _ = os.path.splitext(path) path += ".py" - codecs.open(path, 'r', 'utf8').read() + with codecs.open(path, 'r', 'utf8') as f: + f.read() def test_argparse_module_encoding(self): self._test_module_encoding(argparse.__file__) @@ -4246,6 +4322,15 @@ for name in argparse.__all__: self.assertTrue(hasattr(argparse, name)) + def test_all_exports_everything_but_modules(self): + items = [ + name + for name, value in vars(argparse).items() + if not name.startswith("_") + if not inspect.ismodule(value) + ] + self.assertEqual(sorted(items), sorted(argparse.__all__)) + def test_main(): # silence warnings about version argument - these are expected with test_support.check_warnings( Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_bz2.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_bz2.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_bz2.py Fri Nov 5 23:50:59 2010 @@ -54,68 +54,68 @@ os.unlink(self.filename) def createTempFile(self, crlf=0): - f = open(self.filename, "wb") - if crlf: - data = self.DATA_CRLF - else: - data = self.DATA - f.write(data) - f.close() + with open(self.filename, "wb") as f: + if crlf: + data = self.DATA_CRLF + else: + data = self.DATA + f.write(data) def testRead(self): # "Test BZ2File.read()" self.createTempFile() - bz2f = BZ2File(self.filename) - self.assertRaises(TypeError, bz2f.read, None) - self.assertEqual(bz2f.read(), self.TEXT) - bz2f.close() + with BZ2File(self.filename) as bz2f: + self.assertRaises(TypeError, bz2f.read, None) + self.assertEqual(bz2f.read(), self.TEXT) + + def testRead0(self): + # Test BBZ2File.read(0)" + self.createTempFile() + with BZ2File(self.filename) as bz2f: + self.assertRaises(TypeError, bz2f.read, None) + self.assertEqual(bz2f.read(0), "") def testReadChunk10(self): # "Test BZ2File.read() in chunks of 10 bytes" self.createTempFile() - bz2f = BZ2File(self.filename) - text = '' - while 1: - str = bz2f.read(10) - if not str: - break - text += str - self.assertEqual(text, text) - bz2f.close() + with BZ2File(self.filename) as bz2f: + text = '' + while 1: + str = bz2f.read(10) + if not str: + break + text += str + self.assertEqual(text, text) def testRead100(self): # "Test BZ2File.read(100)" self.createTempFile() - bz2f = BZ2File(self.filename) - self.assertEqual(bz2f.read(100), self.TEXT[:100]) - bz2f.close() + with BZ2File(self.filename) as bz2f: + self.assertEqual(bz2f.read(100), self.TEXT[:100]) def testReadLine(self): # "Test BZ2File.readline()" self.createTempFile() - bz2f = BZ2File(self.filename) - self.assertRaises(TypeError, bz2f.readline, None) - sio = StringIO(self.TEXT) - for line in sio.readlines(): - self.assertEqual(bz2f.readline(), line) - bz2f.close() + with BZ2File(self.filename) as bz2f: + self.assertRaises(TypeError, bz2f.readline, None) + sio = StringIO(self.TEXT) + for line in sio.readlines(): + self.assertEqual(bz2f.readline(), line) def testReadLines(self): # "Test BZ2File.readlines()" self.createTempFile() - bz2f = BZ2File(self.filename) - self.assertRaises(TypeError, bz2f.readlines, None) - sio = StringIO(self.TEXT) - self.assertEqual(bz2f.readlines(), sio.readlines()) - bz2f.close() + with BZ2File(self.filename) as bz2f: + self.assertRaises(TypeError, bz2f.readlines, None) + sio = StringIO(self.TEXT) + self.assertEqual(bz2f.readlines(), sio.readlines()) def testIterator(self): # "Test iter(BZ2File)" self.createTempFile() - bz2f = BZ2File(self.filename) - sio = StringIO(self.TEXT) - self.assertEqual(list(iter(bz2f)), sio.readlines()) - bz2f.close() + with BZ2File(self.filename) as bz2f: + sio = StringIO(self.TEXT) + self.assertEqual(list(iter(bz2f)), sio.readlines()) def testClosedIteratorDeadlock(self): # "Test that iteration on a closed bz2file releases the lock." @@ -154,104 +154,91 @@ def testWrite(self): # "Test BZ2File.write()" - bz2f = BZ2File(self.filename, "w") - self.assertRaises(TypeError, bz2f.write) - bz2f.write(self.TEXT) - bz2f.close() - f = open(self.filename, 'rb') - self.assertEqual(self.decompress(f.read()), self.TEXT) - f.close() + with BZ2File(self.filename, "w") as bz2f: + self.assertRaises(TypeError, bz2f.write) + bz2f.write(self.TEXT) + with open(self.filename, 'rb') as f: + self.assertEqual(self.decompress(f.read()), self.TEXT) def testWriteChunks10(self): # "Test BZ2File.write() with chunks of 10 bytes" - bz2f = BZ2File(self.filename, "w") - n = 0 - while 1: - str = self.TEXT[n*10:(n+1)*10] - if not str: - break - bz2f.write(str) - n += 1 - bz2f.close() - f = open(self.filename, 'rb') - self.assertEqual(self.decompress(f.read()), self.TEXT) - f.close() + with BZ2File(self.filename, "w") as bz2f: + n = 0 + while 1: + str = self.TEXT[n*10:(n+1)*10] + if not str: + break + bz2f.write(str) + n += 1 + with open(self.filename, 'rb') as f: + self.assertEqual(self.decompress(f.read()), self.TEXT) def testWriteLines(self): # "Test BZ2File.writelines()" - bz2f = BZ2File(self.filename, "w") - self.assertRaises(TypeError, bz2f.writelines) - sio = StringIO(self.TEXT) - bz2f.writelines(sio.readlines()) - bz2f.close() + with BZ2File(self.filename, "w") as bz2f: + self.assertRaises(TypeError, bz2f.writelines) + sio = StringIO(self.TEXT) + bz2f.writelines(sio.readlines()) # patch #1535500 self.assertRaises(ValueError, bz2f.writelines, ["a"]) - f = open(self.filename, 'rb') - self.assertEqual(self.decompress(f.read()), self.TEXT) - f.close() + with open(self.filename, 'rb') as f: + self.assertEqual(self.decompress(f.read()), self.TEXT) def testWriteMethodsOnReadOnlyFile(self): - bz2f = BZ2File(self.filename, "w") - bz2f.write("abc") - bz2f.close() + with BZ2File(self.filename, "w") as bz2f: + bz2f.write("abc") - bz2f = BZ2File(self.filename, "r") - self.assertRaises(IOError, bz2f.write, "a") - self.assertRaises(IOError, bz2f.writelines, ["a"]) + with BZ2File(self.filename, "r") as bz2f: + self.assertRaises(IOError, bz2f.write, "a") + self.assertRaises(IOError, bz2f.writelines, ["a"]) def testSeekForward(self): # "Test BZ2File.seek(150, 0)" self.createTempFile() - bz2f = BZ2File(self.filename) - self.assertRaises(TypeError, bz2f.seek) - bz2f.seek(150) - self.assertEqual(bz2f.read(), self.TEXT[150:]) - bz2f.close() + with BZ2File(self.filename) as bz2f: + self.assertRaises(TypeError, bz2f.seek) + bz2f.seek(150) + self.assertEqual(bz2f.read(), self.TEXT[150:]) def testSeekBackwards(self): # "Test BZ2File.seek(-150, 1)" self.createTempFile() - bz2f = BZ2File(self.filename) - bz2f.read(500) - bz2f.seek(-150, 1) - self.assertEqual(bz2f.read(), self.TEXT[500-150:]) - bz2f.close() + with BZ2File(self.filename) as bz2f: + bz2f.read(500) + bz2f.seek(-150, 1) + self.assertEqual(bz2f.read(), self.TEXT[500-150:]) def testSeekBackwardsFromEnd(self): # "Test BZ2File.seek(-150, 2)" self.createTempFile() - bz2f = BZ2File(self.filename) - bz2f.seek(-150, 2) - self.assertEqual(bz2f.read(), self.TEXT[len(self.TEXT)-150:]) - bz2f.close() + with BZ2File(self.filename) as bz2f: + bz2f.seek(-150, 2) + self.assertEqual(bz2f.read(), self.TEXT[len(self.TEXT)-150:]) def testSeekPostEnd(self): # "Test BZ2File.seek(150000)" self.createTempFile() - bz2f = BZ2File(self.filename) - bz2f.seek(150000) - self.assertEqual(bz2f.tell(), len(self.TEXT)) - self.assertEqual(bz2f.read(), "") - bz2f.close() + with BZ2File(self.filename) as bz2f: + bz2f.seek(150000) + self.assertEqual(bz2f.tell(), len(self.TEXT)) + self.assertEqual(bz2f.read(), "") def testSeekPostEndTwice(self): # "Test BZ2File.seek(150000) twice" self.createTempFile() - bz2f = BZ2File(self.filename) - bz2f.seek(150000) - bz2f.seek(150000) - self.assertEqual(bz2f.tell(), len(self.TEXT)) - self.assertEqual(bz2f.read(), "") - bz2f.close() + with BZ2File(self.filename) as bz2f: + bz2f.seek(150000) + bz2f.seek(150000) + self.assertEqual(bz2f.tell(), len(self.TEXT)) + self.assertEqual(bz2f.read(), "") def testSeekPreStart(self): # "Test BZ2File.seek(-150, 0)" self.createTempFile() - bz2f = BZ2File(self.filename) - bz2f.seek(-150) - self.assertEqual(bz2f.tell(), 0) - self.assertEqual(bz2f.read(), self.TEXT) - bz2f.close() + with BZ2File(self.filename) as bz2f: + bz2f.seek(-150) + self.assertEqual(bz2f.tell(), 0) + self.assertEqual(bz2f.read(), self.TEXT) def testOpenDel(self): # "Test opening and deleting a file many times" @@ -277,16 +264,13 @@ def testBug1191043(self): # readlines() for files containing no newline data = 'BZh91AY&SY\xd9b\x89]\x00\x00\x00\x03\x80\x04\x00\x02\x00\x0c\x00 \x00!\x9ah3M\x13<]\xc9\x14\xe1BCe\x8a%t' - f = open(self.filename, "wb") - f.write(data) - f.close() - bz2f = BZ2File(self.filename) - lines = bz2f.readlines() - bz2f.close() + with open(self.filename, "wb") as f: + f.write(data) + with BZ2File(self.filename) as bz2f: + lines = bz2f.readlines() self.assertEqual(lines, ['Test']) - bz2f = BZ2File(self.filename) - xlines = list(bz2f.xreadlines()) - bz2f.close() + with BZ2File(self.filename) as bz2f: + xlines = list(bz2f.readlines()) self.assertEqual(xlines, ['Test']) def testContextProtocol(self): @@ -316,8 +300,7 @@ # Using a BZ2File from several threads doesn't deadlock (issue #7205). data = "1" * 2**20 nthreads = 10 - f = bz2.BZ2File(self.filename, 'wb') - try: + with bz2.BZ2File(self.filename, 'wb') as f: def comp(): for i in range(5): f.write(data) @@ -326,27 +309,19 @@ t.start() for t in threads: t.join() - finally: - f.close() def testMixedIterationReads(self): # Issue #8397: mixed iteration and reads should be forbidden. - f = bz2.BZ2File(self.filename, 'wb') - try: + with bz2.BZ2File(self.filename, 'wb') as f: # The internal buffer size is hard-wired to 8192 bytes, we must # write out more than that for the test to stop half through # the buffer. f.write(self.TEXT * 100) - finally: - f.close() - f = bz2.BZ2File(self.filename, 'rb') - try: + with bz2.BZ2File(self.filename, 'rb') as f: next(f) self.assertRaises(ValueError, f.read) self.assertRaises(ValueError, f.readline) self.assertRaises(ValueError, f.readlines) - finally: - f.close() class BZ2CompressorTest(BaseTest): def testCompress(self): Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_class.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_class.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_class.py Fri Nov 5 23:50:59 2010 @@ -513,7 +513,7 @@ callLst[:] = [] as_long = long(mixIntAndLong) - self.assertEquals(type(as_long), int) + self.assertEquals(type(as_long), long) self.assertEquals(as_long, 64) self.assertCallStack([('__long__', (mixIntAndLong,))]) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_collections.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_collections.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_collections.py Fri Nov 5 23:50:59 2010 @@ -926,6 +926,13 @@ self.assertEqual(eval(repr(od)), od) self.assertEqual(repr(OrderedDict()), "OrderedDict()") + def test_repr_recursive(self): + # See issue #9826 + od = OrderedDict.fromkeys('abc') + od['x'] = od + self.assertEqual(repr(od), + "OrderedDict([('a', None), ('b', None), ('c', None), ('x', ...)])") + def test_setdefault(self): pairs = [('c', 1), ('b', 2), ('a', 3), ('d', 4), ('e', 5), ('f', 6)] shuffle(pairs) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_compiler.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_compiler.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_compiler.py Fri Nov 5 23:50:59 2010 @@ -310,7 +310,7 @@ def test_main(): global TEST_ALL - TEST_ALL = test.test_support.is_resource_enabled("compiler") + TEST_ALL = test.test_support.is_resource_enabled("cpu") test.test_support.run_unittest(CompilerTest) if __name__ == "__main__": Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_decimal.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_decimal.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_decimal.py Fri Nov 5 23:50:59 2010 @@ -224,14 +224,15 @@ if skip_expected: raise unittest.SkipTest return - for line in open(file): - line = line.replace('\r\n', '').replace('\n', '') - #print line - try: - t = self.eval_line(line) - except DecimalException, exception: - #Exception raised where there shoudn't have been one. - self.fail('Exception "'+exception.__class__.__name__ + '" raised on line '+line) + with open(file) as f: + for line in f: + line = line.replace('\r\n', '').replace('\n', '') + #print line + try: + t = self.eval_line(line) + except DecimalException as exception: + #Exception raised where there shoudn't have been one. + self.fail('Exception "'+exception.__class__.__name__ + '" raised on line '+line) return Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_doctest.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_doctest.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_doctest.py Fri Nov 5 23:50:59 2010 @@ -1721,10 +1721,13 @@ >>> doc = ''' ... >>> x = 42 + ... >>> raise Exception('cl??') + ... Traceback (most recent call last): + ... Exception: cl?? ... >>> import pdb; pdb.set_trace() ... ''' >>> parser = doctest.DocTestParser() - >>> test = parser.get_doctest(doc, {}, "foo", "foo.py", 0) + >>> test = parser.get_doctest(doc, {}, "foo-b??r at baz", "foo-b??r at baz.py", 0) >>> runner = doctest.DocTestRunner(verbose=False) To demonstrate this, we'll create a fake standard input that @@ -1740,12 +1743,12 @@ >>> try: runner.run(test) ... finally: sys.stdin = real_stdin --Return-- - > (1)()->None + > (1)()->None -> import pdb; pdb.set_trace() (Pdb) print x 42 (Pdb) continue - TestResults(failed=0, attempted=2) + TestResults(failed=0, attempted=3) You can also put pdb.set_trace in a function called from a test: @@ -1757,7 +1760,7 @@ ... >>> x=1 ... >>> calls_set_trace() ... ''' - >>> test = parser.get_doctest(doc, globals(), "foo", "foo.py", 0) + >>> test = parser.get_doctest(doc, globals(), "foo-b??r at baz", "foo-b??r at baz.py", 0) >>> real_stdin = sys.stdin >>> sys.stdin = _FakeInput([ ... 'print y', # print data defined in the function @@ -1776,7 +1779,7 @@ (Pdb) print y 2 (Pdb) up - > (1)() + > (1)() -> calls_set_trace() (Pdb) print x 1 @@ -1794,7 +1797,7 @@ ... ... import pdb; pdb.set_trace() ... >>> f(3) ... ''' - >>> test = parser.get_doctest(doc, globals(), "foo", "foo.py", 0) + >>> test = parser.get_doctest(doc, globals(), "foo-b??r at baz", "foo-b??r at baz.py", 0) >>> real_stdin = sys.stdin >>> sys.stdin = _FakeInput([ ... 'list', # list source from example 2 @@ -1808,7 +1811,7 @@ ... finally: sys.stdin = real_stdin ... # doctest: +NORMALIZE_WHITESPACE --Return-- - > (3)g()->None + > (3)g()->None -> import pdb; pdb.set_trace() (Pdb) list 1 def g(x): @@ -1817,7 +1820,7 @@ [EOF] (Pdb) next --Return-- - > (2)f()->None + > (2)f()->None -> g(x*2) (Pdb) list 1 def f(x): @@ -1825,14 +1828,14 @@ [EOF] (Pdb) next --Return-- - > (1)()->None + > (1)()->None -> f(3) (Pdb) list 1 -> f(3) [EOF] (Pdb) continue ********************************************************************** - File "foo.py", line 7, in foo + File "foo-b??r at baz.py", line 7, in foo-b??r at baz Failed example: f(3) Expected nothing @@ -1866,7 +1869,7 @@ ... ''' >>> parser = doctest.DocTestParser() >>> runner = doctest.DocTestRunner(verbose=False) - >>> test = parser.get_doctest(doc, globals(), "foo", "foo.py", 0) + >>> test = parser.get_doctest(doc, globals(), "foo-b??r at baz", "foo-b??r at baz.py", 0) >>> real_stdin = sys.stdin >>> sys.stdin = _FakeInput([ ... 'print y', # print data defined in the function @@ -1918,7 +1921,7 @@ (Pdb) print y 1 (Pdb) up - > (1)() + > (1)() -> calls_set_trace() (Pdb) print foo *** NameError: name 'foo' is not defined Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_file2k.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_file2k.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_file2k.py Fri Nov 5 23:50:59 2010 @@ -135,6 +135,14 @@ def testReadWhenWriting(self): self.assertRaises(IOError, self.f.read) + def testNastyWritelinesGenerator(self): + def nasty(): + for i in range(5): + if i == 3: + self.f.close() + yield str(i) + self.assertRaises(ValueError, self.f.writelines, nasty()) + def testIssue5677(self): # Remark: Do not perform more than one test per open file, # since that does NOT catch the readline error on Windows. @@ -172,7 +180,7 @@ class OtherFileTests(unittest.TestCase): def testOpenDir(self): - this_dir = os.path.dirname(__file__) + this_dir = os.path.dirname(__file__) or os.curdir for mode in (None, "w"): try: if mode: Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_fileio.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_fileio.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_fileio.py Fri Nov 5 23:50:59 2010 @@ -263,7 +263,6 @@ # OS'es that don't support /dev/tty. pass else: - f = _FileIO("/dev/tty", "a") self.assertEquals(f.readable(), False) self.assertEquals(f.writable(), True) if sys.platform != "darwin" and \ @@ -342,6 +341,7 @@ f.truncate(15) self.assertEqual(f.tell(), 5) self.assertEqual(f.seek(0, os.SEEK_END), 15) + f.close() def testTruncateOnWindows(self): def bug801631(): Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_float.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_float.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_float.py Fri Nov 5 23:50:59 2010 @@ -540,19 +540,20 @@ @requires_IEEE_754 def test_format_testfile(self): - for line in open(format_testfile): - if line.startswith('--'): - continue - line = line.strip() - if not line: - continue - - lhs, rhs = map(str.strip, line.split('->')) - fmt, arg = lhs.split() - arg = float(arg) - self.assertEqual(fmt % arg, rhs) - if not math.isnan(arg) and copysign(1.0, arg) > 0.0: - self.assertEqual(fmt % -arg, '-' + rhs) + with open(format_testfile) as testfile: + for line in open(format_testfile): + if line.startswith('--'): + continue + line = line.strip() + if not line: + continue + + lhs, rhs = map(str.strip, line.split('->')) + fmt, arg = lhs.split() + arg = float(arg) + self.assertEqual(fmt % arg, rhs) + if not math.isnan(arg) and copysign(1.0, arg) > 0.0: + self.assertEqual(fmt % -arg, '-' + rhs) def test_issue5864(self): self.assertEquals(format(123.456, '.4'), '123.5') Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_ftplib.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_ftplib.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_ftplib.py Fri Nov 5 23:50:59 2010 @@ -99,7 +99,7 @@ addr = map(int, arg.split(',')) ip = '%d.%d.%d.%d' %tuple(addr[:4]) port = (addr[4] * 256) + addr[5] - s = socket.create_connection((ip, port), timeout=2) + s = socket.create_connection((ip, port), timeout=10) self.dtp = self.dtp_handler(s, baseclass=self) self.push('200 active data connection established') @@ -107,7 +107,7 @@ sock = socket.socket() sock.bind((self.socket.getsockname()[0], 0)) sock.listen(5) - sock.settimeout(2) + sock.settimeout(10) ip, port = sock.getsockname()[:2] ip = ip.replace('.', ',') p1, p2 = divmod(port, 256) @@ -118,7 +118,7 @@ def cmd_eprt(self, arg): af, ip, port = arg.split(arg[0])[1:-1] port = int(port) - s = socket.create_connection((ip, port), timeout=2) + s = socket.create_connection((ip, port), timeout=10) self.dtp = self.dtp_handler(s, baseclass=self) self.push('200 active data connection established') @@ -126,7 +126,7 @@ sock = socket.socket(socket.AF_INET6) sock.bind((self.socket.getsockname()[0], 0)) sock.listen(5) - sock.settimeout(2) + sock.settimeout(10) port = sock.getsockname()[1] self.push('229 entering extended passive mode (|||%d|)' %port) conn, addr = sock.accept() @@ -403,7 +403,7 @@ def setUp(self): self.server = DummyFTPServer((HOST, 0)) self.server.start() - self.client = ftplib.FTP(timeout=2) + self.client = ftplib.FTP(timeout=10) self.client.connect(self.server.host, self.server.port) def tearDown(self): @@ -545,7 +545,7 @@ def test_makepasv(self): host, port = self.client.makepasv() - conn = socket.create_connection((host, port), 2) + conn = socket.create_connection((host, port), 10) conn.close() # IPv4 is in use, just make sure send_epsv has not been used self.assertEqual(self.server.handler.last_received_cmd, 'pasv') @@ -572,7 +572,7 @@ def test_makepasv(self): host, port = self.client.makepasv() - conn = socket.create_connection((host, port), 2) + conn = socket.create_connection((host, port), 10) conn.close() self.assertEqual(self.server.handler.last_received_cmd, 'epsv') @@ -595,7 +595,7 @@ def setUp(self): self.server = DummyTLS_FTPServer((HOST, 0)) self.server.start() - self.client = ftplib.FTP_TLS(timeout=2) + self.client = ftplib.FTP_TLS(timeout=10) self.client.connect(self.server.host, self.server.port) # enable TLS self.client.auth() @@ -608,7 +608,7 @@ def setUp(self): self.server = DummyTLS_FTPServer((HOST, 0)) self.server.start() - self.client = ftplib.FTP_TLS(timeout=2) + self.client = ftplib.FTP_TLS(timeout=10) self.client.connect(self.server.host, self.server.port) def tearDown(self): Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_gettext.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_gettext.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_gettext.py Fri Nov 5 23:50:59 2010 @@ -334,6 +334,37 @@ 'John Doe \nJane Foobar ') +class DummyGNUTranslations(gettext.GNUTranslations): + def foo(self): + return 'foo' + + +class GettextCacheTestCase(GettextBaseTest): + def test_cache(self): + self.localedir = os.curdir + self.mofile = MOFILE + + self.assertEqual(len(gettext._translations), 0) + + t = gettext.translation('gettext', self.localedir) + + self.assertEqual(len(gettext._translations), 1) + + t = gettext.translation('gettext', self.localedir, + class_=DummyGNUTranslations) + + self.assertEqual(len(gettext._translations), 2) + self.assertEqual(t.__class__, DummyGNUTranslations) + + # Calling it again doesn't add to the cache + + t = gettext.translation('gettext', self.localedir, + class_=DummyGNUTranslations) + + self.assertEqual(len(gettext._translations), 2) + self.assertEqual(t.__class__, DummyGNUTranslations) + + def test_main(): test_support.run_unittest(__name__) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_gzip.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_gzip.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_gzip.py Fri Nov 5 23:50:59 2010 @@ -33,14 +33,15 @@ def test_write(self): - f = gzip.GzipFile(self.filename, 'wb') ; f.write(data1 * 50) + with gzip.GzipFile(self.filename, 'wb') as f: + f.write(data1 * 50) - # Try flush and fileno. - f.flush() - f.fileno() - if hasattr(os, 'fsync'): - os.fsync(f.fileno()) - f.close() + # Try flush and fileno. + f.flush() + f.fileno() + if hasattr(os, 'fsync'): + os.fsync(f.fileno()) + f.close() # Test multiple close() calls. f.close() @@ -48,37 +49,59 @@ def test_read(self): self.test_write() # Try reading. - f = gzip.GzipFile(self.filename, 'r') ; d = f.read() ; f.close() + with gzip.GzipFile(self.filename, 'r') as f: + d = f.read() self.assertEqual(d, data1*50) + def test_io_on_closed_object(self): + # Test that I/O operations on closed GzipFile objects raise a + # ValueError, just like the corresponding functions on file objects. + + # Write to a file, open it for reading, then close it. + self.test_write() + f = gzip.GzipFile(self.filename, 'r') + f.close() + with self.assertRaises(ValueError): + f.read(1) + with self.assertRaises(ValueError): + f.seek(0) + with self.assertRaises(ValueError): + f.tell() + # Open the file for writing, then close it. + f = gzip.GzipFile(self.filename, 'w') + f.close() + with self.assertRaises(ValueError): + f.write('') + with self.assertRaises(ValueError): + f.flush() + def test_append(self): self.test_write() # Append to the previous file - f = gzip.GzipFile(self.filename, 'ab') ; f.write(data2 * 15) ; f.close() + with gzip.GzipFile(self.filename, 'ab') as f: + f.write(data2 * 15) - f = gzip.GzipFile(self.filename, 'rb') ; d = f.read() ; f.close() + with gzip.GzipFile(self.filename, 'rb') as f: + d = f.read() self.assertEqual(d, (data1*50) + (data2*15)) def test_many_append(self): # Bug #1074261 was triggered when reading a file that contained # many, many members. Create such a file and verify that reading it # works. - f = gzip.open(self.filename, 'wb', 9) - f.write('a') - f.close() - for i in range(0,200): - f = gzip.open(self.filename, "ab", 9) # append + with gzip.open(self.filename, 'wb', 9) as f: f.write('a') - f.close() + for i in range(0, 200): + with gzip.open(self.filename, "ab", 9) as f: # append + f.write('a') # Try reading the file - zgfile = gzip.open(self.filename, "rb") - contents = "" - while 1: - ztxt = zgfile.read(8192) - contents += ztxt - if not ztxt: break - zgfile.close() + with gzip.open(self.filename, "rb") as zgfile: + contents = "" + while 1: + ztxt = zgfile.read(8192) + contents += ztxt + if not ztxt: break self.assertEquals(contents, 'a'*201) def test_buffered_reader(self): @@ -86,9 +109,9 @@ # performance. self.test_write() - f = gzip.GzipFile(self.filename, 'rb') - with io.BufferedReader(f) as r: - lines = [line for line in r] + with gzip.GzipFile(self.filename, 'rb') as f: + with io.BufferedReader(f) as r: + lines = [line for line in r] self.assertEqual(lines, 50 * data1.splitlines(True)) @@ -96,140 +119,127 @@ self.test_write() # Try .readline() with varying line lengths - f = gzip.GzipFile(self.filename, 'rb') - line_length = 0 - while 1: - L = f.readline(line_length) - if L == "" and line_length != 0: break - self.assertTrue(len(L) <= line_length) - line_length = (line_length + 1) % 50 - f.close() + with gzip.GzipFile(self.filename, 'rb') as f: + line_length = 0 + while 1: + L = f.readline(line_length) + if not L and line_length != 0: break + self.assertTrue(len(L) <= line_length) + line_length = (line_length + 1) % 50 def test_readlines(self): self.test_write() # Try .readlines() - f = gzip.GzipFile(self.filename, 'rb') - L = f.readlines() - f.close() + with gzip.GzipFile(self.filename, 'rb') as f: + L = f.readlines() - f = gzip.GzipFile(self.filename, 'rb') - while 1: - L = f.readlines(150) - if L == []: break - f.close() + with gzip.GzipFile(self.filename, 'rb') as f: + while 1: + L = f.readlines(150) + if L == []: break def test_seek_read(self): self.test_write() # Try seek, read test - f = gzip.GzipFile(self.filename) - while 1: - oldpos = f.tell() - line1 = f.readline() - if not line1: break - newpos = f.tell() - f.seek(oldpos) # negative seek - if len(line1)>10: - amount = 10 - else: - amount = len(line1) - line2 = f.read(amount) - self.assertEqual(line1[:amount], line2) - f.seek(newpos) # positive seek - f.close() + with gzip.GzipFile(self.filename) as f: + while 1: + oldpos = f.tell() + line1 = f.readline() + if not line1: break + newpos = f.tell() + f.seek(oldpos) # negative seek + if len(line1)>10: + amount = 10 + else: + amount = len(line1) + line2 = f.read(amount) + self.assertEqual(line1[:amount], line2) + f.seek(newpos) # positive seek def test_seek_whence(self): self.test_write() # Try seek(whence=1), read test - f = gzip.GzipFile(self.filename) - f.read(10) - f.seek(10, whence=1) - y = f.read(10) - f.close() + with gzip.GzipFile(self.filename) as f: + f.read(10) + f.seek(10, whence=1) + y = f.read(10) self.assertEquals(y, data1[20:30]) def test_seek_write(self): # Try seek, write test - f = gzip.GzipFile(self.filename, 'w') - for pos in range(0, 256, 16): - f.seek(pos) - f.write('GZ\n') - f.close() + with gzip.GzipFile(self.filename, 'w') as f: + for pos in range(0, 256, 16): + f.seek(pos) + f.write('GZ\n') def test_mode(self): self.test_write() - f = gzip.GzipFile(self.filename, 'r') - self.assertEqual(f.myfileobj.mode, 'rb') - f.close() + with gzip.GzipFile(self.filename, 'r') as f: + self.assertEqual(f.myfileobj.mode, 'rb') def test_1647484(self): for mode in ('wb', 'rb'): - f = gzip.GzipFile(self.filename, mode) - self.assertTrue(hasattr(f, "name")) - self.assertEqual(f.name, self.filename) - f.close() + with gzip.GzipFile(self.filename, mode) as f: + self.assertTrue(hasattr(f, "name")) + self.assertEqual(f.name, self.filename) def test_mtime(self): mtime = 123456789 - fWrite = gzip.GzipFile(self.filename, 'w', mtime = mtime) - fWrite.write(data1) - fWrite.close() - fRead = gzip.GzipFile(self.filename) - dataRead = fRead.read() - self.assertEqual(dataRead, data1) - self.assertTrue(hasattr(fRead, 'mtime')) - self.assertEqual(fRead.mtime, mtime) - fRead.close() + with gzip.GzipFile(self.filename, 'w', mtime = mtime) as fWrite: + fWrite.write(data1) + with gzip.GzipFile(self.filename) as fRead: + dataRead = fRead.read() + self.assertEqual(dataRead, data1) + self.assertTrue(hasattr(fRead, 'mtime')) + self.assertEqual(fRead.mtime, mtime) def test_metadata(self): mtime = 123456789 - fWrite = gzip.GzipFile(self.filename, 'w', mtime = mtime) - fWrite.write(data1) - fWrite.close() - - fRead = open(self.filename, 'rb') - - # see RFC 1952: http://www.faqs.org/rfcs/rfc1952.html + with gzip.GzipFile(self.filename, 'w', mtime = mtime) as fWrite: + fWrite.write(data1) - idBytes = fRead.read(2) - self.assertEqual(idBytes, '\x1f\x8b') # gzip ID + with open(self.filename, 'rb') as fRead: + # see RFC 1952: http://www.faqs.org/rfcs/rfc1952.html - cmByte = fRead.read(1) - self.assertEqual(cmByte, '\x08') # deflate + idBytes = fRead.read(2) + self.assertEqual(idBytes, '\x1f\x8b') # gzip ID - flagsByte = fRead.read(1) - self.assertEqual(flagsByte, '\x08') # only the FNAME flag is set + cmByte = fRead.read(1) + self.assertEqual(cmByte, '\x08') # deflate - mtimeBytes = fRead.read(4) - self.assertEqual(mtimeBytes, struct.pack('Data\r\n') + + def log_message(self, format, *args): + pass + + class TestServerThread(threading.Thread): def __init__(self, test_object, request_handler): @@ -67,6 +87,63 @@ self.connection.request(method, uri, body, headers) return self.connection.getresponse() +class BaseHTTPRequestHandlerTestCase(unittest.TestCase): + """Test the functionaility of the BaseHTTPServer focussing on + BaseHTTPRequestHandler. + """ + + HTTPResponseMatch = re.compile('HTTP/1.[0-9]+ 200 OK') + + def setUp (self): + self.handler = SocketlessRequestHandler() + + def send_typical_request(self, message): + input = StringIO(message) + output = StringIO() + self.handler.rfile = input + self.handler.wfile = output + self.handler.handle_one_request() + output.seek(0) + return output.readlines() + + def verify_get_called(self): + self.assertTrue(self.handler.get_called) + + def verify_expected_headers(self, headers): + for fieldName in 'Server: ', 'Date: ', 'Content-Type: ': + self.assertEqual(sum(h.startswith(fieldName) for h in headers), 1) + + def verify_http_server_response(self, response): + match = self.HTTPResponseMatch.search(response) + self.assertTrue(match is not None) + + def test_http_1_1(self): + result = self.send_typical_request('GET / HTTP/1.1\r\n\r\n') + self.verify_http_server_response(result[0]) + self.verify_expected_headers(result[1:-1]) + self.verify_get_called() + self.assertEqual(result[-1], 'Data\r\n') + + def test_http_1_0(self): + result = self.send_typical_request('GET / HTTP/1.0\r\n\r\n') + self.verify_http_server_response(result[0]) + self.verify_expected_headers(result[1:-1]) + self.verify_get_called() + self.assertEqual(result[-1], 'Data\r\n') + + def test_http_0_9(self): + result = self.send_typical_request('GET / HTTP/0.9\r\n\r\n') + self.assertEqual(len(result), 1) + self.assertEqual(result[0], 'Data\r\n') + self.verify_get_called() + + def test_with_continue_1_0(self): + result = self.send_typical_request('GET / HTTP/1.0\r\nExpect: 100-continue\r\n\r\n') + self.verify_http_server_response(result[0]) + self.verify_expected_headers(result[1:-1]) + self.verify_get_called() + self.assertEqual(result[-1], 'Data\r\n') + class BaseHTTPServerTestCase(BaseTestCase): class request_handler(NoLogRequestHandler, BaseHTTPRequestHandler): @@ -179,6 +256,7 @@ res = self.con.getresponse() self.assertEqual(res.getheader('Connection'), 'keep-alive') self.con.request('TEST', '/') + self.addCleanup(self.con.close) def test_internal_key_error(self): self.con.request('KEYERROR', '/') @@ -398,14 +476,22 @@ self.assertEqual(('Hello World\n', 'text/html', 200), (res.read(), res.getheader('Content-type'), res.status)) + def test_os_environ_is_not_altered(self): + signature = "Test CGI Server" + os.environ['SERVER_SOFTWARE'] = signature + res = self.request('/cgi-bin/file1.py') + self.assertEqual((b'Hello World\n', 'text/html', 200), + (res.read(), res.getheader('Content-type'), res.status)) + self.assertEqual(os.environ['SERVER_SOFTWARE'], signature) def test_main(verbose=None): try: cwd = os.getcwd() - test_support.run_unittest(BaseHTTPServerTestCase, - SimpleHTTPServerTestCase, - CGIHTTPServerTestCase - ) + test_support.run_unittest(BaseHTTPRequestHandlerTestCase, + BaseHTTPServerTestCase, + SimpleHTTPServerTestCase, + CGIHTTPServerTestCase + ) finally: os.chdir(cwd) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_io.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_io.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_io.py Fri Nov 5 23:50:59 2010 @@ -53,7 +53,9 @@ return f._CHUNK_SIZE -class MockRawIO: +class MockRawIOWithoutRead: + """A RawIO implementation without read(), so as to exercise the default + RawIO.read() which calls readinto().""" def __init__(self, read_stack=()): self._read_stack = list(read_stack) @@ -61,14 +63,6 @@ self._reads = 0 self._extraneous_reads = 0 - def read(self, n=None): - self._reads += 1 - try: - return self._read_stack.pop(0) - except: - self._extraneous_reads += 1 - return b"" - def write(self, b): self._write_stack.append(bytes(b)) return len(b) @@ -115,6 +109,23 @@ def truncate(self, pos=None): return pos +class CMockRawIOWithoutRead(MockRawIOWithoutRead, io.RawIOBase): + pass + +class PyMockRawIOWithoutRead(MockRawIOWithoutRead, pyio.RawIOBase): + pass + + +class MockRawIO(MockRawIOWithoutRead): + + def read(self, n=None): + self._reads += 1 + try: + return self._read_stack.pop(0) + except: + self._extraneous_reads += 1 + return b"" + class CMockRawIO(MockRawIO, io.RawIOBase): pass @@ -560,6 +571,19 @@ f.close() self.assertRaises(ValueError, f.flush) + def test_RawIOBase_read(self): + # Exercise the default RawIOBase.read() implementation (which calls + # readinto() internally). + rawio = self.MockRawIOWithoutRead((b"abc", b"d", None, b"efg", None)) + self.assertEqual(rawio.read(2), b"ab") + self.assertEqual(rawio.read(2), b"c") + self.assertEqual(rawio.read(2), b"d") + self.assertEqual(rawio.read(2), None) + self.assertEqual(rawio.read(2), b"ef") + self.assertEqual(rawio.read(2), b"g") + self.assertEqual(rawio.read(2), None) + self.assertEqual(rawio.read(2), b"") + class CIOTest(IOTest): pass @@ -788,6 +812,7 @@ self.assertEquals(b"abcdefg", bufio.read()) @unittest.skipUnless(threading, 'Threading required for this test.') + @support.requires_resource('cpu') def test_threads(self): try: # Write out many bytes with exactly the same number of 0's, @@ -1057,6 +1082,7 @@ self.assertEqual(f.read(), b"abc") @unittest.skipUnless(threading, 'Threading required for this test.') + @support.requires_resource('cpu') def test_threads(self): try: # Write out many bytes from many threads and test they were @@ -2558,7 +2584,7 @@ # Put the namespaces of the IO module we are testing and some useful mock # classes in the __dict__ of each test. mocks = (MockRawIO, MisbehavedRawIO, MockFileIO, CloseFailureIO, - MockNonBlockWriterIO) + MockNonBlockWriterIO, MockRawIOWithoutRead) all_members = io.__all__ + ["IncrementalNewlineDecoder"] c_io_ns = dict((name, getattr(io, name)) for name in all_members) py_io_ns = dict((name, getattr(pyio, name)) for name in all_members) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_long.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_long.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_long.py Fri Nov 5 23:50:59 2010 @@ -601,6 +601,22 @@ slicemin, slicemax = X()[-2L**100:2L**100] self.assertEqual(X()[slicemin:slicemax], (slicemin, slicemax)) + def test_issue9869(self): + # Issue 9869: Interpreter crash when initializing an instance + # of a long subclass from an object whose __long__ method returns + # a plain int. + class BadLong(object): + def __long__(self): + return 1000000 + + class MyLong(long): + pass + + x = MyLong(BadLong()) + self.assertIsInstance(x, long) + self.assertEqual(x, 1000000) + + # ----------------------------------- tests of auto int->long conversion def test_auto_overflow(self): Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_mailbox.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_mailbox.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_mailbox.py Fri Nov 5 23:50:59 2010 @@ -598,12 +598,10 @@ # Remove old files from 'tmp' foo_path = os.path.join(self._path, 'tmp', 'foo') bar_path = os.path.join(self._path, 'tmp', 'bar') - f = open(foo_path, 'w') - f.write("@") - f.close() - f = open(bar_path, 'w') - f.write("@") - f.close() + with open(foo_path, 'w') as f: + f.write("@") + with open(bar_path, 'w') as f: + f.write("@") self._box.clean() self.assertTrue(os.path.exists(foo_path)) self.assertTrue(os.path.exists(bar_path)) @@ -1088,13 +1086,12 @@ def test_initialize_with_file(self): # Initialize based on contents of file - f = open(self._path, 'w+') - f.write(_sample_message) - f.seek(0) - msg = self._factory(f) - self._post_initialize_hook(msg) - self._check_sample(msg) - f.close() + with open(self._path, 'w+') as f: + f.write(_sample_message) + f.seek(0) + msg = self._factory(f) + self._post_initialize_hook(msg) + self._check_sample(msg) def test_initialize_with_nothing(self): # Initialize without arguments @@ -1812,18 +1809,16 @@ filename = os.extsep.join((str(t), str(pid), "myhostname", "mydomain")) tmpname = os.path.join(self._dir, "tmp", filename) newname = os.path.join(self._dir, dir, filename) - fp = open(tmpname, "w") - self._msgfiles.append(tmpname) - if mbox: - fp.write(FROM_) - fp.write(DUMMY_MESSAGE) - fp.close() + with open(tmpname, "w") as fp: + self._msgfiles.append(tmpname) + if mbox: + fp.write(FROM_) + fp.write(DUMMY_MESSAGE) if hasattr(os, "link"): os.link(tmpname, newname) else: - fp = open(newname, "w") - fp.write(DUMMY_MESSAGE) - fp.close() + with open(newname, "w") as fp: + fp.write(DUMMY_MESSAGE) self._msgfiles.append(newname) return tmpname Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_module.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_module.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_module.py Fri Nov 5 23:50:59 2010 @@ -1,6 +1,6 @@ # Test the module type import unittest -from test.test_support import run_unittest +from test.test_support import run_unittest, gc_collect import sys ModuleType = type(sys) @@ -55,14 +55,29 @@ {"__name__": "foo", "__doc__": "foodoc", "bar": 42}) self.assertTrue(foo.__dict__ is d) + @unittest.expectedFailure def test_dont_clear_dict(self): # See issue 7140. def f(): foo = ModuleType("foo") foo.bar = 4 return foo + gc_collect() self.assertEqual(f().__dict__["bar"], 4) + def test_clear_dict_in_ref_cycle(self): + destroyed = [] + m = ModuleType("foo") + m.destroyed = destroyed + s = """class A: + def __del__(self): + destroyed.append(1) +a = A()""" + exec(s, m.__dict__) + del m + gc_collect() + self.assertEqual(destroyed, [1]) + def test_main(): run_unittest(ModuleTests) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_modulefinder.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_modulefinder.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_modulefinder.py Fri Nov 5 23:50:59 2010 @@ -211,11 +211,17 @@ def create_package(source): ofi = None - for line in source.splitlines(): - if line.startswith(" ") or line.startswith("\t"): - ofi.write(line.strip() + "\n") - else: - ofi = open_file(os.path.join(TEST_DIR, line.strip())) + try: + for line in source.splitlines(): + if line.startswith(" ") or line.startswith("\t"): + ofi.write(line.strip() + "\n") + else: + if ofi: + ofi.close() + ofi = open_file(os.path.join(TEST_DIR, line.strip())) + finally: + if ofi: + ofi.close() class ModuleFinderTest(unittest.TestCase): def _do_test(self, info, report=False): Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_multibytecodec.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_multibytecodec.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_multibytecodec.py Fri Nov 5 23:50:59 2010 @@ -46,12 +46,9 @@ 'apple\x92ham\x93spam', 'test.cjktest') def test_codingspec(self): - try: - for enc in ALL_CJKENCODINGS: - print >> open(TESTFN, 'w'), '# coding:', enc - exec open(TESTFN) - finally: - os.unlink(TESTFN) + for enc in ALL_CJKENCODINGS: + code = '# coding: {}\n'.format(enc) + exec code def test_init_segfault(self): # bug #3305: this used to segfault Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_multibytecodec_support.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_multibytecodec_support.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_multibytecodec_support.py Fri Nov 5 23:50:59 2010 @@ -252,7 +252,7 @@ def __init__(self, *args, **kw): unittest.TestCase.__init__(self, *args, **kw) try: - self.open_mapping_file() # test it to report the error early + self.open_mapping_file().close() # test it to report the error early except (IOError, HTTPException): self.skipTest("Could not retrieve "+self.mapfileurl) @@ -270,36 +270,38 @@ unichrs = lambda s: u''.join(_unichr(c) for c in s.split('+')) urt_wa = {} - for line in self.open_mapping_file(): - if not line: - break - data = line.split('#')[0].strip().split() - if len(data) != 2: - continue - - csetval = eval(data[0]) - if csetval <= 0x7F: - csetch = chr(csetval & 0xff) - elif csetval >= 0x1000000: - csetch = chr(csetval >> 24) + chr((csetval >> 16) & 0xff) + \ - chr((csetval >> 8) & 0xff) + chr(csetval & 0xff) - elif csetval >= 0x10000: - csetch = chr(csetval >> 16) + \ - chr((csetval >> 8) & 0xff) + chr(csetval & 0xff) - elif csetval >= 0x100: - csetch = chr(csetval >> 8) + chr(csetval & 0xff) - else: - continue - - unich = unichrs(data[1]) - if unich == u'\ufffd' or unich in urt_wa: - continue - urt_wa[unich] = csetch + with self.open_mapping_file() as f: + for line in f: + if not line: + break + data = line.split('#')[0].strip().split() + if len(data) != 2: + continue + + csetval = eval(data[0]) + if csetval <= 0x7F: + csetch = chr(csetval & 0xff) + elif csetval >= 0x1000000: + csetch = chr(csetval >> 24) + chr((csetval >> 16) & 0xff) + \ + chr((csetval >> 8) & 0xff) + chr(csetval & 0xff) + elif csetval >= 0x10000: + csetch = chr(csetval >> 16) + \ + chr((csetval >> 8) & 0xff) + chr(csetval & 0xff) + elif csetval >= 0x100: + csetch = chr(csetval >> 8) + chr(csetval & 0xff) + else: + continue + + unich = unichrs(data[1]) + if unich == u'\ufffd' or unich in urt_wa: + continue + urt_wa[unich] = csetch - self._testpoint(csetch, unich) + self._testpoint(csetch, unich) def _test_mapping_file_ucm(self): - ucmdata = self.open_mapping_file().read() + with self.open_mapping_file() as f: + ucmdata = f.read() uc = re.findall('', ucmdata) for uni, coded in uc: unich = unichr(int(uni, 16)) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_multiprocessing.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_multiprocessing.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_multiprocessing.py Fri Nov 5 23:50:59 2010 @@ -34,6 +34,12 @@ from multiprocessing import util +try: + from multiprocessing.sharedctypes import Value, copy + HAS_SHAREDCTYPES = True +except ImportError: + HAS_SHAREDCTYPES = False + # # # @@ -72,16 +78,6 @@ Structure = object c_int = c_double = None -try: - from ctypes import Value -except ImportError: - Value = None - -try: - from ctypes import copy as ctypes_copy -except ImportError: - ctypes_copy = None - # # Creates a wrapper for a function which records the time it takes to finish # @@ -119,6 +115,13 @@ else: return self.assertEqual(value, res) + # For the sanity of Windows users, rather than crashing or freezing in + # multiple ways. + def __reduce__(self, *args): + raise NotImplementedError("shouldn't try to pickle a test case") + + __reduce_ex__ = __reduce__ + # # Return the value of a semaphore # @@ -157,12 +160,13 @@ self.assertEqual(current.ident, os.getpid()) self.assertEqual(current.exitcode, None) - def _test(self, q, *args, **kwds): - current = self.current_process() + @classmethod + def _test(cls, q, *args, **kwds): + current = cls.current_process() q.put(args) q.put(kwds) q.put(current.name) - if self.TYPE != 'threads': + if cls.TYPE != 'threads': q.put(bytes(current.authkey)) q.put(current.pid) @@ -205,7 +209,8 @@ self.assertEquals(p.is_alive(), False) self.assertNotIn(p, self.active_children()) - def _test_terminate(self): + @classmethod + def _test_terminate(cls): time.sleep(1000) def test_terminate(self): @@ -254,13 +259,14 @@ p.join() self.assertNotIn(p, self.active_children()) - def _test_recursion(self, wconn, id): + @classmethod + def _test_recursion(cls, wconn, id): from multiprocessing import forking wconn.send(id) if len(id) < 2: for i in range(2): - p = self.Process( - target=self._test_recursion, args=(wconn, id+[i]) + p = cls.Process( + target=cls._test_recursion, args=(wconn, id+[i]) ) p.start() p.join() @@ -343,7 +349,8 @@ class _TestQueue(BaseTestCase): - def _test_put(self, queue, child_can_start, parent_can_continue): + @classmethod + def _test_put(cls, queue, child_can_start, parent_can_continue): child_can_start.wait() for i in range(6): queue.get() @@ -407,7 +414,8 @@ proc.join() - def _test_get(self, queue, child_can_start, parent_can_continue): + @classmethod + def _test_get(cls, queue, child_can_start, parent_can_continue): child_can_start.wait() #queue.put(1) queue.put(2) @@ -468,7 +476,8 @@ proc.join() - def _test_fork(self, queue): + @classmethod + def _test_fork(cls, queue): for i in range(10, 20): queue.put(i) # note that at this point the items may only be buffered, so the @@ -516,7 +525,8 @@ q.get() self.assertEqual(q.qsize(), 0) - def _test_task_done(self, q): + @classmethod + def _test_task_done(cls, q): for obj in iter(q.get, None): time.sleep(DELTA) q.task_done() @@ -628,7 +638,8 @@ class _TestCondition(BaseTestCase): - def f(self, cond, sleeping, woken, timeout=None): + @classmethod + def f(cls, cond, sleeping, woken, timeout=None): cond.acquire() sleeping.release() cond.wait(timeout) @@ -760,7 +771,8 @@ class _TestEvent(BaseTestCase): - def _test_event(self, event): + @classmethod + def _test_event(cls, event): time.sleep(TIMEOUT2) event.set() @@ -800,6 +812,8 @@ # # + at unittest.skipUnless(HAS_SHAREDCTYPES, + "requires multiprocessing.sharedctypes") class _TestValue(BaseTestCase): ALLOWED_TYPES = ('processes',) @@ -811,12 +825,12 @@ ('c', latin('x'), latin('y')) ] - def _test(self, values): - for sv, cv in zip(values, self.codes_values): + @classmethod + def _test(cls, values): + for sv, cv in zip(values, cls.codes_values): sv.value = cv[2] - @unittest.skipIf(c_int is None, "requires _ctypes") def test_value(self, raw=False): if raw: values = [self.RawValue(code, value) @@ -835,11 +849,9 @@ for sv, cv in zip(values, self.codes_values): self.assertEqual(sv.value, cv[2]) - @unittest.skipIf(c_int is None, "requires _ctypes") def test_rawvalue(self): self.test_value(raw=True) - @unittest.skipIf(c_int is None, "requires _ctypes") def test_getobj_getlock(self): val1 = self.Value('i', 5) lock1 = val1.get_lock() @@ -870,7 +882,8 @@ ALLOWED_TYPES = ('processes',) - def f(self, seq): + @classmethod + def f(cls, seq): for i in range(1, len(seq)): seq[i] += seq[i-1] @@ -1215,7 +1228,8 @@ ALLOWED_TYPES = ('manager',) - def _putter(self, address, authkey): + @classmethod + def _putter(cls, address, authkey): manager = QueueManager2( address=address, authkey=authkey, serializer=SERIALIZER ) @@ -1253,7 +1267,8 @@ class _TestManagerRestart(BaseTestCase): - def _putter(self, address, authkey): + @classmethod + def _putter(cls, address, authkey): manager = QueueManager( address=address, authkey=authkey, serializer=SERIALIZER) manager.connect() @@ -1264,7 +1279,11 @@ authkey = os.urandom(32) manager = QueueManager( address=('localhost', 0), authkey=authkey, serializer=SERIALIZER) - addr = manager.get_server().address + srvr = manager.get_server() + addr = srvr.address + # Close the connection.Listener socket which gets opened as a part + # of manager.get_server(). It's not needed for the test. + srvr.listener.close() manager.start() p = self.Process(target=self._putter, args=(manager.address, authkey)) @@ -1288,7 +1307,8 @@ ALLOWED_TYPES = ('processes', 'threads') - def _echo(self, conn): + @classmethod + def _echo(cls, conn): for msg in iter(conn.recv_bytes, SENTINEL): conn.send_bytes(msg) conn.close() @@ -1437,8 +1457,9 @@ ALLOWED_TYPES = ('processes', 'threads') - def _test(self, address): - conn = self.connection.Client(address) + @classmethod + def _test(cls, address): + conn = cls.connection.Client(address) conn.send('hello') conn.close() @@ -1593,11 +1614,14 @@ ('y', c_double) ] + at unittest.skipUnless(HAS_SHAREDCTYPES, + "requires multiprocessing.sharedctypes") class _TestSharedCTypes(BaseTestCase): ALLOWED_TYPES = ('processes',) - def _double(self, x, y, foo, arr, string): + @classmethod + def _double(cls, x, y, foo, arr, string): x.value *= 2 y.value *= 2 foo.x *= 2 @@ -1606,14 +1630,13 @@ for i in range(len(arr)): arr[i] *= 2 - @unittest.skipIf(Value is None, "requires ctypes.Value") def test_sharedctypes(self, lock=False): x = Value('i', 7, lock=lock) y = Value(c_double, 1.0/3.0, lock=lock) foo = Value(_Foo, 3, 2, lock=lock) arr = self.Array('d', range(10), lock=lock) string = self.Array('c', 20, lock=lock) - string.value = 'hello' + string.value = latin('hello') p = self.Process(target=self._double, args=(x, y, foo, arr, string)) p.start() @@ -1627,14 +1650,12 @@ self.assertAlmostEqual(arr[i], i*2) self.assertEqual(string.value, latin('hellohello')) - @unittest.skipIf(Value is None, "requires ctypes.Value") def test_synchronize(self): self.test_sharedctypes(lock=True) - @unittest.skipIf(ctypes_copy is None, "requires ctypes.copy") def test_copy(self): foo = _Foo(2, 5.0) - bar = ctypes_copy(foo) + bar = copy(foo) foo.x = 0 foo.y = 0 self.assertEqual(bar.x, 2) @@ -1648,7 +1669,8 @@ ALLOWED_TYPES = ('processes',) - def _test_finalize(self, conn): + @classmethod + def _test_finalize(cls, conn): class Foo(object): pass @@ -1742,7 +1764,8 @@ logger.info('nor will this') logger.setLevel(LOG_LEVEL) - def _test_level(self, conn): + @classmethod + def _test_level(cls, conn): logger = multiprocessing.get_logger() conn.send(logger.getEffectiveLevel()) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_ntpath.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_ntpath.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_ntpath.py Fri Nov 5 23:50:59 2010 @@ -178,6 +178,16 @@ tester('ntpath.relpath("a", "b/c")', '..\\..\\a') tester('ntpath.relpath("//conky/mountpoint/a", "//conky/mountpoint/b/c")', '..\\..\\a') tester('ntpath.relpath("a", "a")', '.') + tester('ntpath.relpath("/foo/bar/bat", "/x/y/z")', '..\\..\\..\\foo\\bar\\bat') + tester('ntpath.relpath("/foo/bar/bat", "/foo/bar")', 'bat') + tester('ntpath.relpath("/foo/bar/bat", "/")', 'foo\\bar\\bat') + tester('ntpath.relpath("/", "/foo/bar/bat")', '..\\..\\..') + tester('ntpath.relpath("/foo/bar/bat", "/x")', '..\\foo\\bar\\bat') + tester('ntpath.relpath("/x", "/foo/bar/bat")', '..\\..\\..\\x') + tester('ntpath.relpath("/", "/")', '.') + tester('ntpath.relpath("/a", "/a")', '.') + tester('ntpath.relpath("/a/b", "/a/b")', '.') + tester('ntpath.relpath("c:/foo", "C:/FOO")', '.') class NtCommonTest(test_genericpath.CommonTest): Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_old_mailbox.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_old_mailbox.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_old_mailbox.py Fri Nov 5 23:50:59 2010 @@ -48,18 +48,16 @@ filename = os.extsep.join((str(t), str(pid), "myhostname", "mydomain")) tmpname = os.path.join(self._dir, "tmp", filename) newname = os.path.join(self._dir, dir, filename) - fp = open(tmpname, "w") - self._msgfiles.append(tmpname) - if mbox: - fp.write(FROM_) - fp.write(DUMMY_MESSAGE) - fp.close() + with open(tmpname, "w") as fp: + self._msgfiles.append(tmpname) + if mbox: + fp.write(FROM_) + fp.write(DUMMY_MESSAGE) if hasattr(os, "link"): os.link(tmpname, newname) else: - fp = open(newname, "w") - fp.write(DUMMY_MESSAGE) - fp.close() + with open(newname, "w") as fp: + fp.write(DUMMY_MESSAGE) self._msgfiles.append(newname) return tmpname @@ -102,11 +100,12 @@ import email.parser fname = self.createMessage("cur", True) n = 0 - for msg in mailbox.PortableUnixMailbox(open(fname), + with open(fname) as f: + for msg in mailbox.PortableUnixMailbox(f, email.parser.Parser().parse): - n += 1 - self.assertEqual(msg["subject"], "Simple Test") - self.assertEqual(len(str(msg)), len(FROM_)+len(DUMMY_MESSAGE)) + n += 1 + self.assertEqual(msg["subject"], "Simple Test") + self.assertEqual(len(str(msg)), len(FROM_)+len(DUMMY_MESSAGE)) self.assertEqual(n, 1) class MboxTestCase(unittest.TestCase): @@ -119,8 +118,8 @@ def test_from_regex (self): # Testing new regex from bug #1633678 - f = open(self._path, 'w') - f.write("""From fred at example.com Mon May 31 13:24:50 2004 +0200 + with open(self._path, 'w') as f: + f.write("""From fred at example.com Mon May 31 13:24:50 2004 +0200 Subject: message 1 body1 @@ -137,9 +136,9 @@ body4 """) - f.close() - box = mailbox.UnixMailbox(open(self._path, 'r')) - self.assertTrue(len(list(iter(box))) == 4) + with open(self._path, 'r') as f: + box = mailbox.UnixMailbox(f) + self.assertTrue(len(list(iter(box))) == 4) # XXX We still need more tests! Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_os.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_os.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_os.py Fri Nov 5 23:50:59 2010 @@ -11,6 +11,8 @@ import subprocess import time from test import test_support +import mmap +import uuid warnings.filterwarnings("ignore", "tempnam", RuntimeWarning, __name__) warnings.filterwarnings("ignore", "tmpnam", RuntimeWarning, __name__) @@ -342,8 +344,9 @@ def test_update2(self): if os.path.exists("/bin/sh"): os.environ.update(HELLO="World") - value = os.popen("/bin/sh -c 'echo $HELLO'").read().strip() - self.assertEquals(value, "World") + with os.popen("/bin/sh -c 'echo $HELLO'") as popen: + value = popen.read().strip() + self.assertEquals(value, "World") class WalkTests(unittest.TestCase): """Tests for os.walk().""" @@ -706,6 +709,9 @@ stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE) + self.addCleanup(proc.stdout.close) + self.addCleanup(proc.stderr.close) + self.addCleanup(proc.stdin.close) count, max = 0, 100 while count < max and proc.poll() is None: @@ -736,13 +742,23 @@ self._kill(100) def _kill_with_event(self, event, name): + tagname = "test_os_%s" % uuid.uuid1() + m = mmap.mmap(-1, 1, tagname) + m[0] = '0' # Run a script which has console control handling enabled. proc = subprocess.Popen([sys.executable, os.path.join(os.path.dirname(__file__), - "win_console_handler.py")], + "win_console_handler.py"), tagname], creationflags=subprocess.CREATE_NEW_PROCESS_GROUP) # Let the interpreter startup before we send signals. See #3137. - time.sleep(0.5) + count, max = 0, 20 + while count < max and proc.poll() is None: + if m[0] == '1': + break + time.sleep(0.5) + count += 1 + else: + self.fail("Subprocess didn't finish initialization") os.kill(proc.pid, event) # proc.send_signal(event) could also be done here. # Allow time for the signal to be passed and the process to exit. Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_pipes.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_pipes.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_pipes.py Fri Nov 5 23:50:59 2010 @@ -23,17 +23,21 @@ f = t.open(TESTFN, 'w') f.write('hello world #1') f.close() - self.assertEqual(open(TESTFN).read(), 'HELLO WORLD #1') + with open(TESTFN) as f: + self.assertEqual(f.read(), 'HELLO WORLD #1') def testSimplePipe2(self): - file(TESTFN, 'w').write('hello world #2') + with open(TESTFN, 'w') as f: + f.write('hello world #2') t = pipes.Template() t.append(s_command + ' < $IN > $OUT', pipes.FILEIN_FILEOUT) t.copy(TESTFN, TESTFN2) - self.assertEqual(open(TESTFN2).read(), 'HELLO WORLD #2') + with open(TESTFN2) as f: + self.assertEqual(f.read(), 'HELLO WORLD #2') def testSimplePipe3(self): - file(TESTFN, 'w').write('hello world #2') + with open(TESTFN, 'w') as f: + f.write('hello world #2') t = pipes.Template() t.append(s_command + ' < $IN', pipes.FILEIN_STDOUT) with t.open(TESTFN, 'r') as f: @@ -42,16 +46,20 @@ def testEmptyPipeline1(self): # copy through empty pipe d = 'empty pipeline test COPY' - file(TESTFN, 'w').write(d) - file(TESTFN2, 'w').write('') + with open(TESTFN, 'w') as f: + f.write(d) + with open(TESTFN2, 'w') as f: + f.write('') t=pipes.Template() t.copy(TESTFN, TESTFN2) - self.assertEqual(open(TESTFN2).read(), d) + with open(TESTFN2) as f: + self.assertEqual(f.read(), d) def testEmptyPipeline2(self): # read through empty pipe d = 'empty pipeline test READ' - file(TESTFN, 'w').write(d) + with open(TESTFN, 'w') as f: + f.write(d) t=pipes.Template() with t.open(TESTFN, 'r') as f: self.assertEqual(f.read(), d) @@ -60,8 +68,10 @@ # write through empty pipe d = 'empty pipeline test WRITE' t = pipes.Template() - t.open(TESTFN, 'w').write(d) - self.assertEqual(open(TESTFN).read(), d) + with t.open(TESTFN, 'w') as f: + f.write(d) + with open(TESTFN) as f: + self.assertEqual(f.read(), d) def testQuoting(self): safeunquoted = string.ascii_letters + string.digits + '@%_-+=:,./' Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_pkgimport.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_pkgimport.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_pkgimport.py Fri Nov 5 23:50:59 2010 @@ -22,7 +22,8 @@ self.package_dir = os.path.join(self.test_dir, self.package_name) os.mkdir(self.package_dir) - open(os.path.join(self.package_dir, '__init__'+os.extsep+'py'), 'w') + open(os.path.join( + self.package_dir, '__init__'+os.extsep+'py'), 'w').close() self.module_path = os.path.join(self.package_dir, 'foo'+os.extsep+'py') def tearDown(self): Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_posixpath.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_posixpath.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_posixpath.py Fri Nov 5 23:50:59 2010 @@ -322,6 +322,15 @@ self.assertEqual(posixpath.relpath("a/b", "../c"), "../"+curdir+"/a/b") self.assertEqual(posixpath.relpath("a", "b/c"), "../../a") self.assertEqual(posixpath.relpath("a", "a"), ".") + self.assertEqual(posixpath.relpath("/foo/bar/bat", "/x/y/z"), '../../../foo/bar/bat') + self.assertEqual(posixpath.relpath("/foo/bar/bat", "/foo/bar"), 'bat') + self.assertEqual(posixpath.relpath("/foo/bar/bat", "/"), 'foo/bar/bat') + self.assertEqual(posixpath.relpath("/", "/foo/bar/bat"), '../../..') + self.assertEqual(posixpath.relpath("/foo/bar/bat", "/x"), '../foo/bar/bat') + self.assertEqual(posixpath.relpath("/x", "/foo/bar/bat"), '../../../x') + self.assertEqual(posixpath.relpath("/", "/"), '.') + self.assertEqual(posixpath.relpath("/a", "/a"), '.') + self.assertEqual(posixpath.relpath("/a/b", "/a/b"), '.') finally: os.getcwd = real_getcwd Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_quopri.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_quopri.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_quopri.py Fri Nov 5 23:50:59 2010 @@ -178,6 +178,7 @@ (p, e) = self.STRINGS[-1] process = subprocess.Popen([sys.executable, "-mquopri"], stdin=subprocess.PIPE, stdout=subprocess.PIPE) + self.addCleanup(process.stdout.close) cout, cerr = process.communicate(p) # On Windows, Python will output the result to stdout using # CRLF, as the mode of stdout is text mode. To compare this @@ -188,6 +189,7 @@ (p, e) = self.STRINGS[-1] process = subprocess.Popen([sys.executable, "-mquopri", "-d"], stdin=subprocess.PIPE, stdout=subprocess.PIPE) + self.addCleanup(process.stdout.close) cout, cerr = process.communicate(e) self.assertEqual(cout.splitlines(), p.splitlines()) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_sax.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_sax.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_sax.py Fri Nov 5 23:50:59 2010 @@ -1,5 +1,5 @@ -# regression test for SAX 2.0 -*- coding: iso-8859-1 -*- -# $Id: test_sax.py 78919 2010-03-13 12:41:48Z florent.xicluna $ +# regression test for SAX 2.0 -*- coding: utf-8 -*- +# $Id: test_sax.py 85863 2010-10-27 18:58:04Z antoine.pitrou $ from xml.sax import make_parser, ContentHandler, \ SAXException, SAXReaderNotAvailable, SAXParseException @@ -11,6 +11,7 @@ from xml.sax.saxutils import XMLGenerator, escape, unescape, quoteattr, \ XMLFilterBase from xml.sax.expatreader import create_parser +from xml.sax.handler import feature_namespaces from xml.sax.xmlreader import InputSource, AttributesImpl, AttributesNSImpl from cStringIO import StringIO from test.test_support import findfile, run_unittest @@ -108,7 +109,7 @@ "<Donald Duck & Co>") def test_escape_extra(self): - self.assertEquals(escape("Hei p? deg", {"?" : "å"}), + self.assertEquals(escape("Hei p?? deg", {"??" : "å"}), "Hei på deg") # ===== unescape @@ -120,7 +121,7 @@ "") def test_unescape_extra(self): - self.assertEquals(unescape("Hei p? deg", {"?" : "å"}), + self.assertEquals(unescape("Hei p?? deg", {"??" : "å"}), "Hei på deg") def test_unescape_amp_extra(self): @@ -290,6 +291,60 @@ self.assertEquals(result.getvalue(), start+'') + def test_5027_1(self): + # The xml prefix (as in xml:lang below) is reserved and bound by + # definition to http://www.w3.org/XML/1998/namespace. XMLGenerator had + # a bug whereby a KeyError is thrown because this namespace is missing + # from a dictionary. + # + # This test demonstrates the bug by parsing a document. + test_xml = StringIO( + '' + '' + 'Hello' + '') + + parser = make_parser() + parser.setFeature(feature_namespaces, True) + result = StringIO() + gen = XMLGenerator(result) + parser.setContentHandler(gen) + parser.parse(test_xml) + + self.assertEquals(result.getvalue(), + start + ( + '' + 'Hello' + '')) + + def test_5027_2(self): + # The xml prefix (as in xml:lang below) is reserved and bound by + # definition to http://www.w3.org/XML/1998/namespace. XMLGenerator had + # a bug whereby a KeyError is thrown because this namespace is missing + # from a dictionary. + # + # This test demonstrates the bug by direct manipulation of the + # XMLGenerator. + result = StringIO() + gen = XMLGenerator(result) + + gen.startDocument() + gen.startPrefixMapping('a', 'http://example.com/ns') + gen.startElementNS(('http://example.com/ns', 'g1'), 'g1', {}) + lang_attr = {('http://www.w3.org/XML/1998/namespace', 'lang'): 'en'} + gen.startElementNS(('http://example.com/ns', 'g2'), 'g2', lang_attr) + gen.characters('Hello') + gen.endElementNS(('http://example.com/ns', 'g2'), 'g2') + gen.endElementNS(('http://example.com/ns', 'g1'), 'g1') + gen.endPrefixMapping('a') + gen.endDocument() + + self.assertEquals(result.getvalue(), + start + ( + '' + 'Hello' + '')) + class XMLFilterBaseTest(unittest.TestCase): def test_filter_basic(self): Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_scope.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_scope.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_scope.py Fri Nov 5 23:50:59 2010 @@ -647,6 +647,13 @@ self.assertEqual(2, global_ns["result2"]) self.assertEqual(9, global_ns["result9"]) + def testTopIsNotSignificant(self): + # See #9997. + def top(a): + pass + def b(): + global a + def test_main(): with check_warnings(("import \* only allowed at module level", Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_shutil.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_shutil.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_shutil.py Fri Nov 5 23:50:59 2010 @@ -277,7 +277,8 @@ os.link(src, dst) self.assertRaises(shutil.Error, shutil.copyfile, src, dst) - self.assertEqual(open(src,'r').read(), 'cheddar') + with open(src, 'r') as f: + self.assertEqual(f.read(), 'cheddar') os.remove(dst) # Using `src` here would mean we end up with a symlink pointing @@ -285,7 +286,8 @@ # TESTFN/cheese. os.symlink('cheese', dst) self.assertRaises(shutil.Error, shutil.copyfile, src, dst) - self.assertEqual(open(src,'r').read(), 'cheddar') + with open(src, 'r') as f: + self.assertEqual(f.read(), 'cheddar') os.remove(dst) finally: try: @@ -588,9 +590,11 @@ pass def _check_move_file(self, src, dst, real_dst): - contents = open(src, "rb").read() + with open(src, "rb") as f: + contents = f.read() shutil.move(src, dst) - self.assertEqual(contents, open(real_dst, "rb").read()) + with open(real_dst, "rb") as f: + self.assertEqual(contents, f.read()) self.assertFalse(os.path.exists(src)) def _check_move_dir(self, src, dst, real_dst): Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_signal.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_signal.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_signal.py Fri Nov 5 23:50:59 2010 @@ -212,13 +212,13 @@ @unittest.skipUnless(sys.platform == "win32", "Windows specific") class WindowsSignalTests(unittest.TestCase): def test_issue9324(self): + # Updated for issue #10003, adding SIGBREAK handler = lambda x, y: None - signal.signal(signal.SIGABRT, handler) - signal.signal(signal.SIGFPE, handler) - signal.signal(signal.SIGILL, handler) - signal.signal(signal.SIGINT, handler) - signal.signal(signal.SIGSEGV, handler) - signal.signal(signal.SIGTERM, handler) + for sig in (signal.SIGABRT, signal.SIGBREAK, signal.SIGFPE, + signal.SIGILL, signal.SIGINT, signal.SIGSEGV, + signal.SIGTERM): + # Set and then reset a handler for signals that work on windows + signal.signal(sig, signal.signal(sig, handler)) with self.assertRaises(ValueError): signal.signal(-1, handler) @@ -438,8 +438,8 @@ self.assertEqual(self.hndl_called, True) # Issue 3864. Unknown if this affects earlier versions of freebsd also. - @unittest.skipIf(sys.platform=='freebsd6', - 'itimer not reliable (does not mix well with threading) on freebsd6') + @unittest.skipIf(sys.platform in ('freebsd6', 'netbsd5'), + 'itimer not reliable (does not mix well with threading) on some BSDs.') def test_itimer_virtual(self): self.itimer = signal.ITIMER_VIRTUAL signal.signal(signal.SIGVTALRM, self.sig_vtalrm) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_smtpnet.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_smtpnet.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_smtpnet.py Fri Nov 5 23:50:59 2010 @@ -12,7 +12,8 @@ def test_connect(self): test_support.get_attribute(smtplib, 'SMTP_SSL') - server = smtplib.SMTP_SSL(self.testServer, self.remotePort) + with test_support.transient_internet(self.testServer): + server = smtplib.SMTP_SSL(self.testServer, self.remotePort) server.ehlo() server.quit() Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_socket.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_socket.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_socket.py Fri Nov 5 23:50:59 2010 @@ -15,6 +15,7 @@ import contextlib from weakref import proxy import signal +import math def try_address(host, port=0, family=socket.AF_INET): """Try to bind a socket on the given host:port and return True @@ -185,6 +186,11 @@ def clientSetUp(self): self.cli = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + def clientTearDown(self): + self.cli.close() + self.cli = None + ThreadableTest.clientTearDown(self) + class SocketConnectedTest(ThreadedTCPSocketTest): def __init__(self, methodName='runTest'): @@ -350,8 +356,10 @@ # Find one service that exists, then check all the related interfaces. # I've ordered this by protocols that have both a tcp and udp # protocol, at least for modern Linuxes. - if sys.platform in ('linux2', 'freebsd4', 'freebsd5', 'freebsd6', - 'freebsd7', 'freebsd8', 'darwin'): + if (sys.platform.startswith('linux') or + sys.platform.startswith('freebsd') or + sys.platform.startswith('netbsd') or + sys.platform == 'darwin'): # avoid the 'echo' service on this platform, as there is an # assumption breaking non-standard port/protocol entry services = ('daytime', 'qotd', 'domain') @@ -509,6 +517,7 @@ # Testing getsockname() port = self._get_unused_port() sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.addCleanup(sock.close) sock.bind(("0.0.0.0", port)) name = sock.getsockname() # XXX(nnorwitz): http://tinyurl.com/os5jz seems to indicate @@ -522,12 +531,14 @@ # Testing getsockopt() # We know a socket should start without reuse==0 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.addCleanup(sock.close) reuse = sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR) self.assertFalse(reuse != 0, "initial mode is reuse") def testSetSockOpt(self): # Testing setsockopt() sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.addCleanup(sock.close) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) reuse = sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR) self.assertFalse(reuse == 0, "failed to set reuse mode") @@ -560,15 +571,15 @@ finally: sock.close() + @unittest.skipUnless(os.name == "nt", "Windows specific") def test_sock_ioctl(self): - if os.name != "nt": - return self.assertTrue(hasattr(socket.socket, 'ioctl')) self.assertTrue(hasattr(socket, 'SIO_RCVALL')) self.assertTrue(hasattr(socket, 'RCVALL_ON')) self.assertTrue(hasattr(socket, 'RCVALL_OFF')) self.assertTrue(hasattr(socket, 'SIO_KEEPALIVE_VALS')) s = socket.socket() + self.addCleanup(s.close) self.assertRaises(ValueError, s.ioctl, -1, None) s.ioctl(socket.SIO_KEEPALIVE_VALS, (1, 100, 100)) @@ -611,6 +622,42 @@ socket.AI_PASSIVE) + def check_sendall_interrupted(self, with_timeout): + # socketpair() is not stricly required, but it makes things easier. + if not hasattr(signal, 'alarm') or not hasattr(socket, 'socketpair'): + self.skipTest("signal.alarm and socket.socketpair required for this test") + # Our signal handlers clobber the C errno by calling a math function + # with an invalid domain value. + def ok_handler(*args): + self.assertRaises(ValueError, math.acosh, 0) + def raising_handler(*args): + self.assertRaises(ValueError, math.acosh, 0) + 1 // 0 + c, s = socket.socketpair() + old_alarm = signal.signal(signal.SIGALRM, raising_handler) + try: + if with_timeout: + # Just above the one second minimum for signal.alarm + c.settimeout(1.5) + with self.assertRaises(ZeroDivisionError): + signal.alarm(1) + c.sendall(b"x" * (1024**2)) + if with_timeout: + signal.signal(signal.SIGALRM, ok_handler) + signal.alarm(1) + self.assertRaises(socket.timeout, c.sendall, b"x" * (1024**2)) + finally: + signal.signal(signal.SIGALRM, old_alarm) + c.close() + s.close() + + def test_sendall_interrupted(self): + self.check_sendall_interrupted(False) + + def test_sendall_interrupted_with_timeout(self): + self.check_sendall_interrupted(True) + + @unittest.skipUnless(thread, 'Threading required for this test.') class BasicTCPTest(SocketConnectedTest): @@ -673,12 +720,23 @@ return # On Windows, this doesn't exist fd = self.cli_conn.fileno() sock = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM) + self.addCleanup(sock.close) msg = sock.recv(1024) self.assertEqual(msg, MSG) def _testFromFd(self): self.serv_conn.send(MSG) + def testDup(self): + # Testing dup() + sock = self.cli_conn.dup() + self.addCleanup(sock.close) + msg = sock.recv(1024) + self.assertEqual(msg, MSG) + + def _testDup(self): + self.serv_conn.send(MSG) + def testShutdown(self): # Testing shutdown() msg = self.cli_conn.recv(1024) @@ -789,6 +847,7 @@ read, write, err = select.select([self.serv], [], []) if self.serv in read: conn, addr = self.serv.accept() + conn.close() else: self.fail("Error trying to do accept after select.") @@ -799,6 +858,7 @@ def testConnect(self): # Testing non-blocking connect conn, addr = self.serv.accept() + conn.close() def _testConnect(self): self.cli.settimeout(10) @@ -817,6 +877,7 @@ read, write, err = select.select([conn], [], []) if conn in read: msg = conn.recv(len(MSG)) + conn.close() self.assertEqual(msg, MSG) else: self.fail("Error during select call to non-blocking socket.") @@ -1066,6 +1127,7 @@ def test_connect(self): port = test_support.find_unused_port() cli = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.addCleanup(cli.close) with self.assertRaises(socket.error) as cm: cli.connect((HOST, port)) self.assertEqual(cm.exception.errno, errno.ECONNREFUSED) @@ -1103,16 +1165,19 @@ def _justAccept(self): conn, addr = self.serv.accept() + conn.close() testFamily = _justAccept def _testFamily(self): self.cli = socket.create_connection((HOST, self.port), timeout=30) + self.addCleanup(self.cli.close) self.assertEqual(self.cli.family, 2) testSourceAddress = _justAccept def _testSourceAddress(self): self.cli = socket.create_connection((HOST, self.port), timeout=30, source_address=('', self.source_port)) + self.addCleanup(self.cli.close) self.assertEqual(self.cli.getsockname()[1], self.source_port) # The port number being used is sufficient to show that the bind() # call happened. @@ -1124,6 +1189,7 @@ socket.setdefaulttimeout(42) try: self.cli = socket.create_connection((HOST, self.port)) + self.addCleanup(self.cli.close) finally: socket.setdefaulttimeout(None) self.assertEquals(self.cli.gettimeout(), 42) @@ -1135,6 +1201,7 @@ socket.setdefaulttimeout(30) try: self.cli = socket.create_connection((HOST, self.port), timeout=None) + self.addCleanup(self.cli.close) finally: socket.setdefaulttimeout(None) self.assertEqual(self.cli.gettimeout(), None) @@ -1147,6 +1214,7 @@ testTimeoutValueNonamed = _justAccept def _testTimeoutValueNonamed(self): self.cli = socket.create_connection((HOST, self.port), 30) + self.addCleanup(self.cli.close) self.assertEqual(self.cli.gettimeout(), 30) @unittest.skipUnless(thread, 'Threading required for this test.') @@ -1166,6 +1234,7 @@ def testInsideTimeout(self): conn, addr = self.serv.accept() + self.addCleanup(conn.close) time.sleep(3) conn.send("done!") testOutsideTimeout = testInsideTimeout Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_ssl.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_ssl.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_ssl.py Fri Nov 5 23:50:59 2010 @@ -156,18 +156,19 @@ if not test_support.is_resource_enabled('network'): return remote = ("svn.python.org", 443) - s = ssl.wrap_socket(socket.socket(socket.AF_INET), - cert_reqs=ssl.CERT_NONE, ciphers="ALL") - s.connect(remote) - s = ssl.wrap_socket(socket.socket(socket.AF_INET), - cert_reqs=ssl.CERT_NONE, ciphers="DEFAULT") - s.connect(remote) - # Error checking occurs when connecting, because the SSL context - # isn't created before. - s = ssl.wrap_socket(socket.socket(socket.AF_INET), - cert_reqs=ssl.CERT_NONE, ciphers="^$:,;?*'dorothyx") - with self.assertRaisesRegexp(ssl.SSLError, "No cipher can be selected"): + with test_support.transient_internet(remote[0]): + s = ssl.wrap_socket(socket.socket(socket.AF_INET), + cert_reqs=ssl.CERT_NONE, ciphers="ALL") + s.connect(remote) + s = ssl.wrap_socket(socket.socket(socket.AF_INET), + cert_reqs=ssl.CERT_NONE, ciphers="DEFAULT") s.connect(remote) + # Error checking occurs when connecting, because the SSL context + # isn't created before. + s = ssl.wrap_socket(socket.socket(socket.AF_INET), + cert_reqs=ssl.CERT_NONE, ciphers="^$:,;?*'dorothyx") + with self.assertRaisesRegexp(ssl.SSLError, "No cipher can be selected"): + s.connect(remote) @test_support.cpython_only def test_refcycle(self): @@ -179,6 +180,19 @@ del ss self.assertEqual(wr(), None) + def test_wrapped_unconnected(self): + # The _delegate_methods in socket.py are correctly delegated to by an + # unconnected SSLSocket, so they will raise a socket.error rather than + # something unexpected like TypeError. + s = socket.socket(socket.AF_INET) + ss = ssl.wrap_socket(s) + self.assertRaises(socket.error, ss.recv, 1) + self.assertRaises(socket.error, ss.recv_into, bytearray(b'x')) + self.assertRaises(socket.error, ss.recvfrom, 1) + self.assertRaises(socket.error, ss.recvfrom_into, bytearray(b'x'), 1) + self.assertRaises(socket.error, ss.send, b'x') + self.assertRaises(socket.error, ss.sendto, b'x', ('0.0.0.0', 0)) + class NetworkedTests(unittest.TestCase): @@ -1285,7 +1299,7 @@ not os.path.exists(SVN_PYTHON_ORG_ROOT_CERT)): raise test_support.TestFailed("Can't read certificate files!") - tests = [BasicTests] + tests = [BasicTests, BasicSocketTests] if test_support.is_resource_enabled('network'): tests.append(NetworkedTests) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_subprocess.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_subprocess.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_subprocess.py Fri Nov 5 23:50:59 2010 @@ -117,6 +117,8 @@ # .stdin is None when not redirected p = subprocess.Popen([sys.executable, "-c", 'print "banana"'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) + self.addCleanup(p.stdout.close) + self.addCleanup(p.stderr.close) p.wait() self.assertEqual(p.stdin, None) @@ -127,6 +129,8 @@ 'test of stdout in a different ' 'process ..."'], stdin=subprocess.PIPE, stderr=subprocess.PIPE) + self.addCleanup(p.stdin.close) + self.addCleanup(p.stderr.close) p.wait() self.assertEqual(p.stdout, None) @@ -134,6 +138,8 @@ # .stderr is None when not redirected p = subprocess.Popen([sys.executable, "-c", 'print "banana"'], stdin=subprocess.PIPE, stdout=subprocess.PIPE) + self.addCleanup(p.stdout.close) + self.addCleanup(p.stdin.close) p.wait() self.assertEqual(p.stderr, None) @@ -194,6 +200,7 @@ p = subprocess.Popen([sys.executable, "-c", 'import sys; sys.stdout.write("orange")'], stdout=subprocess.PIPE) + self.addCleanup(p.stdout.close) self.assertEqual(p.stdout.read(), "orange") def test_stdout_filedes(self): @@ -222,6 +229,7 @@ p = subprocess.Popen([sys.executable, "-c", 'import sys; sys.stderr.write("strawberry")'], stderr=subprocess.PIPE) + self.addCleanup(p.stderr.close) self.assertStderrEqual(p.stderr.read(), "strawberry") def test_stderr_filedes(self): @@ -254,6 +262,7 @@ 'sys.stderr.write("orange")'], stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + self.addCleanup(p.stdout.close) self.assertStderrEqual(p.stdout.read(), "appleorange") def test_stdout_stderr_file(self): @@ -289,6 +298,7 @@ 'sys.stdout.write(os.getcwd())'], stdout=subprocess.PIPE, cwd=tmpdir) + self.addCleanup(p.stdout.close) normcase = os.path.normcase self.assertEqual(normcase(p.stdout.read()), normcase(tmpdir)) @@ -300,6 +310,7 @@ 'sys.stdout.write(os.getenv("FRUIT"))'], stdout=subprocess.PIPE, env=newenv) + self.addCleanup(p.stdout.close) self.assertEqual(p.stdout.read(), "orange") def test_communicate_stdin(self): @@ -334,6 +345,9 @@ stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + self.addCleanup(p.stdout.close) + self.addCleanup(p.stderr.close) + self.addCleanup(p.stdin.close) (stdout, stderr) = p.communicate("banana") self.assertEqual(stdout, "banana") self.assertStderrEqual(stderr, "pineapple") @@ -382,6 +396,9 @@ stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + self.addCleanup(p.stdout.close) + self.addCleanup(p.stderr.close) + self.addCleanup(p.stdin.close) string_to_write = "abc"*pipe_buf (stdout, stderr) = p.communicate(string_to_write) self.assertEqual(stdout, string_to_write) @@ -394,6 +411,9 @@ stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + self.addCleanup(p.stdout.close) + self.addCleanup(p.stderr.close) + self.addCleanup(p.stdin.close) p.stdin.write("banana") (stdout, stderr) = p.communicate("split") self.assertEqual(stdout, "bananasplit") @@ -415,6 +435,7 @@ 'sys.stdout.write("\\nline6");'], stdout=subprocess.PIPE, universal_newlines=1) + self.addCleanup(p.stdout.close) stdout = p.stdout.read() if hasattr(file, 'newlines'): # Interpreter with universal newline support @@ -442,6 +463,8 @@ 'sys.stdout.write("\\nline6");'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=1) + self.addCleanup(p.stdout.close) + self.addCleanup(p.stderr.close) (stdout, stderr) = p.communicate() if hasattr(file, 'newlines'): # Interpreter with universal newline support @@ -454,20 +477,40 @@ def test_no_leaking(self): # Make sure we leak no resources - if not hasattr(test_support, "is_resource_enabled") \ - or test_support.is_resource_enabled("subprocess") and not mswindows: + if not mswindows: max_handles = 1026 # too much for most UNIX systems else: - max_handles = 65 - for i in range(max_handles): - p = subprocess.Popen([sys.executable, "-c", - "import sys;sys.stdout.write(sys.stdin.read())"], - stdin=subprocess.PIPE, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - data = p.communicate("lime")[0] - self.assertEqual(data, "lime") - + max_handles = 2050 # too much for (at least some) Windows setups + handles = [] + try: + for i in range(max_handles): + try: + handles.append(os.open(test_support.TESTFN, + os.O_WRONLY | os.O_CREAT)) + except OSError as e: + if e.errno != errno.EMFILE: + raise + break + else: + self.skipTest("failed to reach the file descriptor limit " + "(tried %d)" % max_handles) + # Close a couple of them (should be enough for a subprocess) + for i in range(10): + os.close(handles.pop()) + # Loop creating some subprocesses. If one of them leaks some fds, + # the next loop iteration will fail by reaching the max fd limit. + for i in range(15): + p = subprocess.Popen([sys.executable, "-c", + "import sys;" + "sys.stdout.write(sys.stdin.read())"], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + data = p.communicate(b"lime")[0] + self.assertEqual(data, b"lime") + finally: + for h in handles: + os.close(h) def test_list2cmdline(self): self.assertEqual(subprocess.list2cmdline(['a b c', 'd', 'e']), @@ -530,7 +573,7 @@ subprocess.Popen(['nonexisting_i_hope'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) - if c.exception.errno != 2: # ignore "no such file" + if c.exception.errno != errno.ENOENT: # ignore "no such file" raise c.exception def test_handles_closed_on_exception(self): @@ -619,6 +662,7 @@ "sys.stdout.write(os.getenv('FRUIT'))"], stdout=subprocess.PIPE, preexec_fn=lambda: os.putenv("FRUIT", "apple")) + self.addCleanup(p.stdout.close) self.assertEqual(p.stdout.read(), "apple") def test_args_string(self): @@ -652,6 +696,7 @@ p = subprocess.Popen(["echo $FRUIT"], shell=1, stdout=subprocess.PIPE, env=newenv) + self.addCleanup(p.stdout.close) self.assertEqual(p.stdout.read().strip(), "apple") def test_shell_string(self): @@ -661,6 +706,7 @@ p = subprocess.Popen("echo $FRUIT", shell=1, stdout=subprocess.PIPE, env=newenv) + self.addCleanup(p.stdout.close) self.assertEqual(p.stdout.read().strip(), "apple") def test_call_string(self): @@ -692,29 +738,26 @@ for sh in shells: p = subprocess.Popen("echo $0", executable=sh, shell=True, stdout=subprocess.PIPE) + self.addCleanup(p.stdout.close) self.assertEqual(p.stdout.read().strip(), sh) def _kill_process(self, method, *args): # Do not inherit file handles from the parent. # It should fix failures on some platforms. - p = subprocess.Popen([sys.executable, "-c", "input()"], close_fds=True, - stdin=subprocess.PIPE, stderr=subprocess.PIPE) - - # Let the process initialize (Issue #3137) - time.sleep(0.1) - # The process should not terminate prematurely - self.assertIsNone(p.poll()) - # Retry if the process do not receive the signal. - count, maxcount = 0, 3 - while count < maxcount and p.poll() is None: - getattr(p, method)(*args) - time.sleep(0.1) - count += 1 - - self.assertIsNotNone(p.poll(), "the subprocess did not terminate") - if count > 1: - print >>sys.stderr, ("p.{}{} succeeded after " - "{} attempts".format(method, args, count)) + p = subprocess.Popen([sys.executable, "-c", """if 1: + import sys, time + sys.stdout.write('x\\n') + sys.stdout.flush() + time.sleep(30) + """], + close_fds=True, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + # Wait for the interpreter to be completely initialized before + # sending any signal. + p.stdout.read(1) + getattr(p, method)(*args) return p def test_send_signal(self): @@ -788,6 +831,7 @@ p = subprocess.Popen(["set"], shell=1, stdout=subprocess.PIPE, env=newenv) + self.addCleanup(p.stdout.close) self.assertIn("physalis", p.stdout.read()) def test_shell_string(self): @@ -797,6 +841,7 @@ p = subprocess.Popen("set", shell=1, stdout=subprocess.PIPE, env=newenv) + self.addCleanup(p.stdout.close) self.assertIn("physalis", p.stdout.read()) def test_call_string(self): @@ -807,28 +852,25 @@ def _kill_process(self, method, *args): # Some win32 buildbot raises EOFError if stdin is inherited - p = subprocess.Popen([sys.executable, "-c", "input()"], - stdin=subprocess.PIPE, stderr=subprocess.PIPE) - - # Let the process initialize (Issue #3137) - time.sleep(0.1) - # The process should not terminate prematurely - self.assertIsNone(p.poll()) - # Retry if the process do not receive the signal. - count, maxcount = 0, 3 - while count < maxcount and p.poll() is None: - getattr(p, method)(*args) - time.sleep(0.1) - count += 1 - - returncode = p.poll() - self.assertIsNotNone(returncode, "the subprocess did not terminate") - if count > 1: - print >>sys.stderr, ("p.{}{} succeeded after " - "{} attempts".format(method, args, count)) + p = subprocess.Popen([sys.executable, "-c", """if 1: + import sys, time + sys.stdout.write('x\\n') + sys.stdout.flush() + time.sleep(30) + """], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + self.addCleanup(p.stdout.close) + self.addCleanup(p.stderr.close) + self.addCleanup(p.stdin.close) + # Wait for the interpreter to be completely initialized before + # sending any signal. + p.stdout.read(1) + getattr(p, method)(*args) _, stderr = p.communicate() self.assertStderrEqual(stderr, '') - self.assertEqual(p.wait(), returncode) + returncode = p.wait() self.assertNotEqual(returncode, 0) def test_send_signal(self): @@ -891,6 +933,7 @@ def with_spaces(self, *args, **kwargs): kwargs['stdout'] = subprocess.PIPE p = subprocess.Popen(*args, **kwargs) + self.addCleanup(p.stdout.close) self.assertEqual( p.stdout.read ().decode("mbcs"), "2 [%r, 'ab cd']" % self.fname Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_support.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_support.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_support.py Fri Nov 5 23:50:59 2010 @@ -1004,7 +1004,7 @@ return obj def requires_resource(resource): - if resource_is_enabled(resource): + if is_resource_enabled(resource): return _id else: return unittest.skip("resource {0!r} is not enabled".format(resource)) @@ -1199,3 +1199,23 @@ return b"".join(chr(x) for x in b) except TypeError: return bytes(b) + +def args_from_interpreter_flags(): + """Return a list of command-line arguments reproducing the current + settings in sys.flags.""" + flag_opt_map = { + 'bytes_warning': 'b', + 'dont_write_bytecode': 'B', + 'ignore_environment': 'E', + 'no_user_site': 's', + 'no_site': 'S', + 'optimize': 'O', + 'py3k_warning': '3', + 'verbose': 'v', + } + args = [] + for flag, opt in flag_opt_map.items(): + v = getattr(sys.flags, flag) + if v > 0: + args.append('-' + opt * v) + return args Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_sys_setprofile.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_sys_setprofile.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_sys_setprofile.py Fri Nov 5 23:50:59 2010 @@ -1,3 +1,4 @@ +import gc import pprint import sys import unittest @@ -12,14 +13,14 @@ sys.setprofile(None) def test_empty(self): - assert sys.getprofile() is None + self.assertIsNone(sys.getprofile()) def test_setget(self): def fn(*args): pass sys.setprofile(fn) - assert sys.getprofile() == fn + self.assertIs(sys.getprofile(), fn) class HookWatcher: def __init__(self): @@ -352,19 +353,19 @@ def capture_events(callable, p=None): - try: - sys.setprofile() - except TypeError: - pass - else: - raise test_support.TestFailed( - 'sys.setprofile() did not raise TypeError') - if p is None: p = HookWatcher() - sys.setprofile(p.callback) - protect(callable, p) - sys.setprofile(None) + # Disable the garbage collector. This prevents __del__s from showing up in + # traces. + old_gc = gc.isenabled() + gc.disable() + try: + sys.setprofile(p.callback) + protect(callable, p) + sys.setprofile(None) + finally: + if old_gc: + gc.enable() return p.get_events()[1:-1] Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_threading.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_threading.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_threading.py Fri Nov 5 23:50:59 2010 @@ -307,7 +307,7 @@ # Issue1733757 # Avoid a deadlock when sys.settrace steps into threading._shutdown import subprocess - rc = subprocess.call([sys.executable, "-c", """if 1: + p = subprocess.Popen([sys.executable, "-c", """if 1: import sys, threading # A deadlock-killer, to prevent the @@ -327,9 +327,16 @@ return func sys.settrace(func) - """]) + """], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + self.addCleanup(p.stdout.close) + self.addCleanup(p.stderr.close) + stdout, stderr = p.communicate() + rc = p.returncode self.assertFalse(rc == 2, "interpreted was blocked") - self.assertTrue(rc == 0, "Unexpected error") + self.assertTrue(rc == 0, + "Unexpected error: " + repr(stderr)) def test_join_nondaemon_on_shutdown(self): # Issue 1722344 @@ -350,6 +357,8 @@ """], stdout=subprocess.PIPE, stderr=subprocess.PIPE) + self.addCleanup(p.stdout.close) + self.addCleanup(p.stderr.close) stdout, stderr = p.communicate() self.assertEqual(stdout.strip(), "Woke up, sleep function is: ") @@ -423,6 +432,7 @@ p = subprocess.Popen([sys.executable, "-c", script], stdout=subprocess.PIPE) rc = p.wait() data = p.stdout.read().replace('\r', '') + p.stdout.close() self.assertEqual(data, "end of main\nend of thread\n") self.assertFalse(rc == 2, "interpreter was blocked") self.assertTrue(rc == 0, "Unexpected error") @@ -466,7 +476,8 @@ return # Skip platforms with known problems forking from a worker thread. # See http://bugs.python.org/issue3863. - if sys.platform in ('freebsd4', 'freebsd5', 'freebsd6', 'os2emx'): + if sys.platform in ('freebsd4', 'freebsd5', 'freebsd6', 'netbsd5', + 'os2emx'): print >>sys.stderr, ('Skipping test_3_join_in_forked_from_thread' ' due to known OS bugs on'), sys.platform return Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_timeout.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_timeout.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_timeout.py Fri Nov 5 23:50:59 2010 @@ -130,17 +130,19 @@ def testRecvTimeout(self): # Test recv() timeout _timeout = 0.02 - self.sock.connect(self.addr_remote) - self.sock.settimeout(_timeout) - - _t1 = time.time() - self.assertRaises(socket.error, self.sock.recv, 1024) - _t2 = time.time() - _delta = abs(_t1 - _t2) - self.assertTrue(_delta < _timeout + self.fuzz, - "timeout (%g) is %g seconds more than expected (%g)" - %(_delta, self.fuzz, _timeout)) + with test_support.transient_internet(self.addr_remote[0]): + self.sock.connect(self.addr_remote) + self.sock.settimeout(_timeout) + + _t1 = time.time() + self.assertRaises(socket.timeout, self.sock.recv, 1024) + _t2 = time.time() + + _delta = abs(_t1 - _t2) + self.assertTrue(_delta < _timeout + self.fuzz, + "timeout (%g) is %g seconds more than expected (%g)" + %(_delta, self.fuzz, _timeout)) def testAcceptTimeout(self): # Test accept() timeout Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_tokenize.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_tokenize.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_tokenize.py Fri Nov 5 23:50:59 2010 @@ -493,13 +493,13 @@ True Test roundtrip on random python modules. -pass the '-ucompiler' option to process the full directory. +pass the '-ucpu' option to process the full directory. >>> >>> tempdir = os.path.dirname(f) or os.curdir >>> testfiles = glob.glob(os.path.join(tempdir, "test*.py")) - >>> if not test_support.is_resource_enabled("compiler"): + >>> if not test_support.is_resource_enabled("cpu"): ... testfiles = random.sample(testfiles, 10) ... >>> for testfile in testfiles: Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_trace.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_trace.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_trace.py Fri Nov 5 23:50:59 2010 @@ -186,7 +186,6 @@ } self.assertEqual(tracer.results().counts, expected) - class TestRunExecCounts(unittest.TestCase): """A simple sanity test of line-counting, via runctx (exec)""" def setUp(self): @@ -283,8 +282,9 @@ rmtree(TESTFN) unlink(TESTFN) - def _coverage(self, tracer): - tracer.run('from test import test_pprint; test_pprint.test_main()') + def _coverage(self, tracer, + cmd='from test import test_pprint; test_pprint.test_main()'): + tracer.run(cmd) r = tracer.results() r.write_results(show_missing=True, summary=True, coverdir=TESTFN) @@ -311,6 +311,27 @@ files = os.listdir(TESTFN) self.assertEquals(files, []) + def test_issue9936(self): + tracer = trace.Trace(trace=0, count=1) + modname = 'test.tracedmodules.testmod' + # Ensure that the module is executed in import + if modname in sys.modules: + del sys.modules[modname] + cmd = ("import test.tracedmodules.testmod as t;" + "t.func(0); t.func2();") + with captured_stdout() as stdout: + self._coverage(tracer, cmd) + stdout.seek(0) + stdout.readline() + coverage = {} + for line in stdout: + lines, cov, module = line.split()[:3] + coverage[module] = (int(lines), int(cov[:-1])) + # XXX This is needed to run regrtest.py as a script + modname = trace.fullmodname(sys.modules[modname].__file__) + self.assertIn(modname, coverage) + self.assertEqual(coverage[modname], (5, 100)) + def test_main(): run_unittest(__name__) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_unicode.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_unicode.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_unicode.py Fri Nov 5 23:50:59 2010 @@ -1,4 +1,3 @@ -# -*- coding: iso-8859-1 -*- """ Test script for the Unicode implementation. Written by Marc-Andre Lemburg (mal at lemburg.com). Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_urllib2net.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_urllib2net.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_urllib2net.py Fri Nov 5 23:50:59 2010 @@ -156,10 +156,35 @@ def test_urlwithfrag(self): urlwith_frag = "http://docs.python.org/glossary.html#glossary" - req = urllib2.Request(urlwith_frag) - res = urllib2.urlopen(req) - self.assertEqual(res.geturl(), - "http://docs.python.org/glossary.html") + with test_support.transient_internet(urlwith_frag): + req = urllib2.Request(urlwith_frag) + res = urllib2.urlopen(req) + self.assertEqual(res.geturl(), + "http://docs.python.org/glossary.html") + + def test_fileno(self): + req = urllib2.Request("http://www.python.org") + opener = urllib2.build_opener() + res = opener.open(req) + try: + res.fileno() + except AttributeError: + self.fail("HTTPResponse object should return a valid fileno") + finally: + res.close() + + def test_custom_headers(self): + url = "http://www.example.com" + with test_support.transient_internet(url): + opener = urllib2.build_opener() + request = urllib2.Request(url) + self.assertFalse(request.header_items()) + opener.open(request) + self.assertTrue(request.header_items()) + self.assertTrue(request.has_header('User-agent')) + request.add_header('User-Agent','Test-Agent') + opener.open(request) + self.assertEqual(request.get_header('User-agent'),'Test-Agent') def _test_urls(self, urls, handlers, retry=True): import time @@ -175,29 +200,30 @@ url, req, expected_err = url else: req = expected_err = None - debug(url) - try: - f = urlopen(url, req, TIMEOUT) - except EnvironmentError, err: - debug(err) - if expected_err: - msg = ("Didn't get expected error(s) %s for %s %s, got %s: %s" % - (expected_err, url, req, type(err), err)) - self.assertIsInstance(err, expected_err, msg) - except urllib2.URLError as err: - if isinstance(err[0], socket.timeout): - print >>sys.stderr, "" % url - continue - else: - raise - else: + with test_support.transient_internet(url): + debug(url) try: - with test_support.transient_internet(url): - buf = f.read() - debug("read %d bytes" % len(buf)) - except socket.timeout: - print >>sys.stderr, "" % url - f.close() + f = urlopen(url, req, TIMEOUT) + except EnvironmentError as err: + debug(err) + if expected_err: + msg = ("Didn't get expected error(s) %s for %s %s, got %s: %s" % + (expected_err, url, req, type(err), err)) + self.assertIsInstance(err, expected_err, msg) + except urllib2.URLError as err: + if isinstance(err[0], socket.timeout): + print >>sys.stderr, "" % url + continue + else: + raise + else: + try: + with test_support.transient_internet(url): + buf = f.read() + debug("read %d bytes" % len(buf)) + except socket.timeout: + print >>sys.stderr, "" % url + f.close() debug("******** next url coming up...") time.sleep(0.1) @@ -214,59 +240,71 @@ class TimeoutTest(unittest.TestCase): def test_http_basic(self): self.assertTrue(socket.getdefaulttimeout() is None) - u = _urlopen_with_retry("http://www.python.org") - self.assertTrue(u.fp._sock.fp._sock.gettimeout() is None) + url = "http://www.python.org" + with test_support.transient_internet(url, timeout=None): + u = _urlopen_with_retry(url) + self.assertTrue(u.fp._sock.fp._sock.gettimeout() is None) def test_http_default_timeout(self): self.assertTrue(socket.getdefaulttimeout() is None) - socket.setdefaulttimeout(60) - try: - u = _urlopen_with_retry("http://www.python.org") - finally: - socket.setdefaulttimeout(None) - self.assertEqual(u.fp._sock.fp._sock.gettimeout(), 60) + url = "http://www.python.org" + with test_support.transient_internet(url): + socket.setdefaulttimeout(60) + try: + u = _urlopen_with_retry(url) + finally: + socket.setdefaulttimeout(None) + self.assertEqual(u.fp._sock.fp._sock.gettimeout(), 60) def test_http_no_timeout(self): self.assertTrue(socket.getdefaulttimeout() is None) - socket.setdefaulttimeout(60) - try: - u = _urlopen_with_retry("http://www.python.org", timeout=None) - finally: - socket.setdefaulttimeout(None) - self.assertTrue(u.fp._sock.fp._sock.gettimeout() is None) + url = "http://www.python.org" + with test_support.transient_internet(url): + socket.setdefaulttimeout(60) + try: + u = _urlopen_with_retry(url, timeout=None) + finally: + socket.setdefaulttimeout(None) + self.assertTrue(u.fp._sock.fp._sock.gettimeout() is None) def test_http_timeout(self): - u = _urlopen_with_retry("http://www.python.org", timeout=120) - self.assertEqual(u.fp._sock.fp._sock.gettimeout(), 120) + url = "http://www.python.org" + with test_support.transient_internet(url): + u = _urlopen_with_retry(url, timeout=120) + self.assertEqual(u.fp._sock.fp._sock.gettimeout(), 120) FTP_HOST = "ftp://ftp.mirror.nl/pub/gnu/" def test_ftp_basic(self): self.assertTrue(socket.getdefaulttimeout() is None) - u = _urlopen_with_retry(self.FTP_HOST) - self.assertTrue(u.fp.fp._sock.gettimeout() is None) + with test_support.transient_internet(self.FTP_HOST, timeout=None): + u = _urlopen_with_retry(self.FTP_HOST) + self.assertTrue(u.fp.fp._sock.gettimeout() is None) def test_ftp_default_timeout(self): self.assertTrue(socket.getdefaulttimeout() is None) - socket.setdefaulttimeout(60) - try: - u = _urlopen_with_retry(self.FTP_HOST) - finally: - socket.setdefaulttimeout(None) - self.assertEqual(u.fp.fp._sock.gettimeout(), 60) + with test_support.transient_internet(self.FTP_HOST): + socket.setdefaulttimeout(60) + try: + u = _urlopen_with_retry(self.FTP_HOST) + finally: + socket.setdefaulttimeout(None) + self.assertEqual(u.fp.fp._sock.gettimeout(), 60) def test_ftp_no_timeout(self): self.assertTrue(socket.getdefaulttimeout() is None) - socket.setdefaulttimeout(60) - try: - u = _urlopen_with_retry(self.FTP_HOST, timeout=None) - finally: - socket.setdefaulttimeout(None) - self.assertTrue(u.fp.fp._sock.gettimeout() is None) + with test_support.transient_internet(self.FTP_HOST): + socket.setdefaulttimeout(60) + try: + u = _urlopen_with_retry(self.FTP_HOST, timeout=None) + finally: + socket.setdefaulttimeout(None) + self.assertTrue(u.fp.fp._sock.gettimeout() is None) def test_ftp_timeout(self): - u = _urlopen_with_retry(self.FTP_HOST, timeout=60) - self.assertEqual(u.fp.fp._sock.gettimeout(), 60) + with test_support.transient_internet(self.FTP_HOST): + u = _urlopen_with_retry(self.FTP_HOST, timeout=60) + self.assertEqual(u.fp.fp._sock.gettimeout(), 60) def test_main(): Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_uu.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_uu.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_uu.py Fri Nov 5 23:50:59 2010 @@ -161,6 +161,23 @@ finally: self._kill(f) + def test_decode_filename(self): + f = None + try: + test_support.unlink(self.tmpin) + f = open(self.tmpin, 'w') + f.write(encodedtextwrapped % (0644, self.tmpout)) + f.close() + + uu.decode(self.tmpin) + + f = open(self.tmpout, 'r') + s = f.read() + f.close() + self.assertEqual(s, plaintext) + finally: + self._kill(f) + def test_decodetwice(self): # Verify that decode() will refuse to overwrite an existing file f = None Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_winreg.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_winreg.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_winreg.py Fri Nov 5 23:50:59 2010 @@ -261,7 +261,8 @@ finally: done = True thread.join() - DeleteKey(HKEY_CURRENT_USER, test_key_name+'\\changing_value') + with OpenKey(HKEY_CURRENT_USER, test_key_name, 0, KEY_ALL_ACCESS) as key: + DeleteKey(key, 'changing_value') DeleteKey(HKEY_CURRENT_USER, test_key_name) def test_long_key(self): @@ -275,7 +276,8 @@ num_subkeys, num_values, t = QueryInfoKey(key) EnumKey(key, 0) finally: - DeleteKey(HKEY_CURRENT_USER, '\\'.join((test_key_name, name))) + with OpenKey(HKEY_CURRENT_USER, test_key_name, 0, KEY_ALL_ACCESS) as key: + DeleteKey(key, name) DeleteKey(HKEY_CURRENT_USER, test_key_name) def test_dynamic_key(self): Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_xml_etree.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_xml_etree.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_xml_etree.py Fri Nov 5 23:50:59 2010 @@ -586,10 +586,13 @@ + >>> with open(SIMPLE_XMLFILE) as f: + ... data = f.read() + >>> parser = ET.XMLParser() >>> parser.version # doctest: +ELLIPSIS 'Expat ...' - >>> parser.feed(open(SIMPLE_XMLFILE).read()) + >>> parser.feed(data) >>> print serialize(parser.close()) text @@ -598,7 +601,7 @@ >>> parser = ET.XMLTreeBuilder() # 1.2 compatibility - >>> parser.feed(open(SIMPLE_XMLFILE).read()) + >>> parser.feed(data) >>> print serialize(parser.close()) text @@ -608,7 +611,7 @@ >>> target = ET.TreeBuilder() >>> parser = ET.XMLParser(target=target) - >>> parser.feed(open(SIMPLE_XMLFILE).read()) + >>> parser.feed(data) >>> print serialize(parser.close()) text @@ -711,7 +714,8 @@ end-ns None >>> events = ("start", "end", "bogus") - >>> context = iterparse(SIMPLE_XMLFILE, events) + >>> with open(SIMPLE_XMLFILE, "rb") as f: + ... iterparse(f, events) Traceback (most recent call last): ValueError: unknown event 'bogus' @@ -763,6 +767,8 @@ """ Test parser w. custom builder. + >>> with open(SIMPLE_XMLFILE) as f: + ... data = f.read() >>> class Builder: ... def start(self, tag, attrib): ... print "start", tag @@ -772,7 +778,7 @@ ... pass >>> builder = Builder() >>> parser = ET.XMLParser(target=builder) - >>> parser.feed(open(SIMPLE_XMLFILE, "r").read()) + >>> parser.feed(data) start root start element end element @@ -782,6 +788,8 @@ end empty-element end root + >>> with open(SIMPLE_NS_XMLFILE) as f: + ... data = f.read() >>> class Builder: ... def start(self, tag, attrib): ... print "start", tag @@ -795,7 +803,7 @@ ... print "comment", repr(data) >>> builder = Builder() >>> parser = ET.XMLParser(target=builder) - >>> parser.feed(open(SIMPLE_NS_XMLFILE, "r").read()) + >>> parser.feed(data) pi pi 'data' comment ' comment ' start {namespace}root @@ -813,7 +821,8 @@ """ Test Element.getchildren() - >>> tree = ET.parse(open(SIMPLE_XMLFILE, "r")) + >>> with open(SIMPLE_XMLFILE, "r") as f: + ... tree = ET.parse(f) >>> for elem in tree.getroot().iter(): ... summarize_list(elem.getchildren()) ['element', 'element', 'empty-element'] Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/test_xpickle.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_xpickle.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/test_xpickle.py Fri Nov 5 23:50:59 2010 @@ -57,7 +57,8 @@ return cPickle.loads(buf) def have_python_version(name): - """Check whether the given name is a valid Python binary. + """Check whether the given name is a valid Python binary and has + test.test_support. This respects your PATH. @@ -67,7 +68,7 @@ Returns: True if the name is valid, False otherwise. """ - return os.system(name + " -c 'import sys; sys.exit()'") == 0 + return os.system(name + " -c 'import test.test_support'") == 0 class AbstractCompatTests(AbstractPickleTests): Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/tracedmodules/testmod.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/tracedmodules/testmod.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/tracedmodules/testmod.py Fri Nov 5 23:50:59 2010 @@ -1,3 +1,9 @@ def func(x): b = x + 1 return b + 2 + +def func2(): + """Test function for issue 9936 """ + return (1, + 2, + 3) Modified: pypy/branch/fast-forward/lib-python/2.7.0/test/win_console_handler.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/win_console_handler.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/test/win_console_handler.py Fri Nov 5 23:50:59 2010 @@ -8,12 +8,14 @@ similar example in C. """ -from ctypes import wintypes +from ctypes import wintypes, WINFUNCTYPE import signal import ctypes +import mmap +import sys # Function prototype for the handler function. Returns BOOL, takes a DWORD. -HandlerRoutine = wintypes.WINFUNCTYPE(wintypes.BOOL, wintypes.DWORD) +HandlerRoutine = WINFUNCTYPE(wintypes.BOOL, wintypes.DWORD) def _ctrl_handler(sig): """Handle a sig event and return 0 to terminate the process""" @@ -38,6 +40,10 @@ print("Unable to add SetConsoleCtrlHandler") exit(-1) + # Awaken mail process + m = mmap.mmap(-1, 1, sys.argv[1]) + m[0] = '1' + # Do nothing but wait for the signal while True: pass Modified: pypy/branch/fast-forward/lib-python/2.7.0/trace.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/trace.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/trace.py Fri Nov 5 23:50:59 2010 @@ -58,7 +58,7 @@ import tokenize import inspect import gc - +import dis try: import cPickle pickle = cPickle @@ -379,13 +379,7 @@ """Return dict where keys are lines in the line number table.""" linenos = {} - line_increments = [ord(c) for c in code.co_lnotab[1::2]] - table_length = len(line_increments) - docstring = False - - lineno = code.co_firstlineno - for li in line_increments: - lineno += li + for _, lineno in dis.findlinestarts(code): if lineno not in strs: linenos[lineno] = 1 Modified: pypy/branch/fast-forward/lib-python/2.7.0/unittest/result.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/unittest/result.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/unittest/result.py Fri Nov 5 23:50:59 2010 @@ -35,6 +35,7 @@ formatted traceback of the error that occurred. """ _previousTestClass = None + _testRunEntered = False _moduleSetUpFailed = False def __init__(self, stream=None, descriptions=None, verbosity=None): self.failfast = False Modified: pypy/branch/fast-forward/lib-python/2.7.0/unittest/suite.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/unittest/suite.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/unittest/suite.py Fri Nov 5 23:50:59 2010 @@ -80,23 +80,11 @@ subclassing, do not forget to call the base class constructor. """ + def run(self, result, debug=False): + topLevel = False + if getattr(result, '_testRunEntered', False) is False: + result._testRunEntered = topLevel = True - def run(self, result): - self._wrapped_run(result) - self._tearDownPreviousClass(None, result) - self._handleModuleTearDown(result) - return result - - def debug(self): - """Run the tests without collecting errors in a TestResult""" - debug = _DebugResult() - self._wrapped_run(debug, True) - self._tearDownPreviousClass(None, debug) - self._handleModuleTearDown(debug) - - ################################ - # private methods - def _wrapped_run(self, result, debug=False): for test in self: if result.shouldStop: break @@ -111,13 +99,23 @@ getattr(result, '_moduleSetUpFailed', False)): continue - if hasattr(test, '_wrapped_run'): - test._wrapped_run(result, debug) - elif not debug: + if not debug: test(result) else: test.debug() + if topLevel: + self._tearDownPreviousClass(None, result) + self._handleModuleTearDown(result) + return result + + def debug(self): + """Run the tests without collecting errors in a TestResult""" + debug = _DebugResult() + self.run(debug, True) + + ################################ + def _handleClassSetUp(self, test, result): previousClass = getattr(result, '_previousTestClass', None) currentClass = test.__class__ Modified: pypy/branch/fast-forward/lib-python/2.7.0/unittest/test/test_suite.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/unittest/test/test_suite.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/unittest/test/test_suite.py Fri Nov 5 23:50:59 2010 @@ -345,5 +345,19 @@ self.assertEqual(result.testsRun, 2) + def test_overriding_call(self): + class MySuite(unittest.TestSuite): + called = False + def __call__(self, *args, **kw): + self.called = True + unittest.TestSuite.__call__(self, *args, **kw) + + suite = MySuite() + wrapper = unittest.TestSuite() + wrapper.addTest(suite) + wrapper(unittest.TestResult()) + self.assertTrue(suite.called) + + if __name__ == '__main__': unittest.main() Modified: pypy/branch/fast-forward/lib-python/2.7.0/urllib.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/urllib.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/urllib.py Fri Nov 5 23:50:59 2010 @@ -1189,8 +1189,7 @@ 'abcdefghijklmnopqrstuvwxyz' '0123456789' '_.-') _safe_map = {} -for i in xrange(256): - c = chr(i) +for i, c in zip(xrange(256), str(bytearray(xrange(256)))): _safe_map[c] = c if (i < 128 and c in always_safe) else '%{:02X}'.format(i) _safe_quoters = {} Modified: pypy/branch/fast-forward/lib-python/2.7.0/urllib2.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/urllib2.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/urllib2.py Fri Nov 5 23:50:59 2010 @@ -1127,8 +1127,10 @@ h = http_class(host, timeout=req.timeout) # will parse host:port h.set_debuglevel(self._debuglevel) - headers = dict(req.headers) - headers.update(req.unredirected_hdrs) + headers = dict(req.unredirected_hdrs) + headers.update(dict((k, v) for k, v in req.headers.items() + if k not in headers)) + # We want to make an HTTP/1.1 request, but the addinfourl # class isn't prepared to deal with a persistent connection. # It will try to read all remaining data from the socket, Modified: pypy/branch/fast-forward/lib-python/2.7.0/uu.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/uu.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/uu.py Fri Nov 5 23:50:59 2010 @@ -44,40 +44,47 @@ # # If in_file is a pathname open it and change defaults # - if in_file == '-': - in_file = sys.stdin - elif isinstance(in_file, basestring): + opened_files = [] + try: + if in_file == '-': + in_file = sys.stdin + elif isinstance(in_file, basestring): + if name is None: + name = os.path.basename(in_file) + if mode is None: + try: + mode = os.stat(in_file).st_mode + except AttributeError: + pass + in_file = open(in_file, 'rb') + opened_files.append(in_file) + # + # Open out_file if it is a pathname + # + if out_file == '-': + out_file = sys.stdout + elif isinstance(out_file, basestring): + out_file = open(out_file, 'wb') + opened_files.append(out_file) + # + # Set defaults for name and mode + # if name is None: - name = os.path.basename(in_file) + name = '-' if mode is None: - try: - mode = os.stat(in_file).st_mode - except AttributeError: - pass - in_file = open(in_file, 'rb') - # - # Open out_file if it is a pathname - # - if out_file == '-': - out_file = sys.stdout - elif isinstance(out_file, basestring): - out_file = open(out_file, 'w') - # - # Set defaults for name and mode - # - if name is None: - name = '-' - if mode is None: - mode = 0666 - # - # Write the data - # - out_file.write('begin %o %s\n' % ((mode&0777),name)) - data = in_file.read(45) - while len(data) > 0: - out_file.write(binascii.b2a_uu(data)) + mode = 0666 + # + # Write the data + # + out_file.write('begin %o %s\n' % ((mode&0777),name)) data = in_file.read(45) - out_file.write(' \nend\n') + while len(data) > 0: + out_file.write(binascii.b2a_uu(data)) + data = in_file.read(45) + out_file.write(' \nend\n') + finally: + for f in opened_files: + f.close() def decode(in_file, out_file=None, mode=None, quiet=0): @@ -85,65 +92,68 @@ # # Open the input file, if needed. # + opened_files = [] if in_file == '-': in_file = sys.stdin elif isinstance(in_file, basestring): in_file = open(in_file) - # - # Read until a begin is encountered or we've exhausted the file - # - while True: - hdr = in_file.readline() - if not hdr: - raise Error('No valid begin line found in input file') - if not hdr.startswith('begin'): - continue - hdrfields = hdr.split(' ', 2) - if len(hdrfields) == 3 and hdrfields[0] == 'begin': + opened_files.append(in_file) + try: + # + # Read until a begin is encountered or we've exhausted the file + # + while True: + hdr = in_file.readline() + if not hdr: + raise Error('No valid begin line found in input file') + if not hdr.startswith('begin'): + continue + hdrfields = hdr.split(' ', 2) + if len(hdrfields) == 3 and hdrfields[0] == 'begin': + try: + int(hdrfields[1], 8) + break + except ValueError: + pass + if out_file is None: + out_file = hdrfields[2].rstrip() + if os.path.exists(out_file): + raise Error('Cannot overwrite existing file: %s' % out_file) + if mode is None: + mode = int(hdrfields[1], 8) + # + # Open the output file + # + if out_file == '-': + out_file = sys.stdout + elif isinstance(out_file, basestring): + fp = open(out_file, 'wb') try: - int(hdrfields[1], 8) - break - except ValueError: + os.path.chmod(out_file, mode) + except AttributeError: pass - if out_file is None: - out_file = hdrfields[2].rstrip() - if os.path.exists(out_file): - raise Error('Cannot overwrite existing file: %s' % out_file) - if mode is None: - mode = int(hdrfields[1], 8) - # - # Open the output file - # - opened = False - if out_file == '-': - out_file = sys.stdout - elif isinstance(out_file, basestring): - fp = open(out_file, 'wb') - try: - os.path.chmod(out_file, mode) - except AttributeError: - pass - out_file = fp - opened = True - # - # Main decoding loop - # - s = in_file.readline() - while s and s.strip() != 'end': - try: - data = binascii.a2b_uu(s) - except binascii.Error, v: - # Workaround for broken uuencoders by /Fredrik Lundh - nbytes = (((ord(s[0])-32) & 63) * 4 + 5) // 3 - data = binascii.a2b_uu(s[:nbytes]) - if not quiet: - sys.stderr.write("Warning: %s\n" % v) - out_file.write(data) + out_file = fp + opened_files.append(out_file) + # + # Main decoding loop + # s = in_file.readline() - if not s: - raise Error('Truncated input file') - if opened: - out_file.close() + while s and s.strip() != 'end': + try: + data = binascii.a2b_uu(s) + except binascii.Error, v: + # Workaround for broken uuencoders by /Fredrik Lundh + nbytes = (((ord(s[0])-32) & 63) * 4 + 5) // 3 + data = binascii.a2b_uu(s[:nbytes]) + if not quiet: + sys.stderr.write("Warning: %s\n" % v) + out_file.write(data) + s = in_file.readline() + if not s: + raise Error('Truncated input file') + finally: + for f in opened_files: + f.close() def test(): """uuencode/uudecode main program""" Modified: pypy/branch/fast-forward/lib-python/2.7.0/uuid.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/uuid.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/uuid.py Fri Nov 5 23:50:59 2010 @@ -302,15 +302,15 @@ # LC_ALL to get English output, 2>/dev/null to # prevent output on stderr cmd = 'LC_ALL=C %s %s 2>/dev/null' % (executable, args) - pipe = os.popen(cmd) + with os.popen(cmd) as pipe: + for line in pipe: + words = line.lower().split() + for i in range(len(words)): + if words[i] in hw_identifiers: + return int( + words[get_index(i)].replace(':', ''), 16) except IOError: continue - - for line in pipe: - words = line.lower().split() - for i in range(len(words)): - if words[i] in hw_identifiers: - return int(words[get_index(i)].replace(':', ''), 16) return None def _ifconfig_getnode(): @@ -353,10 +353,13 @@ pipe = os.popen(os.path.join(dir, 'ipconfig') + ' /all') except IOError: continue - for line in pipe: - value = line.split(':')[-1].strip().lower() - if re.match('([0-9a-f][0-9a-f]-){5}[0-9a-f][0-9a-f]', value): - return int(value.replace('-', ''), 16) + else: + for line in pipe: + value = line.split(':')[-1].strip().lower() + if re.match('([0-9a-f][0-9a-f]-){5}[0-9a-f][0-9a-f]', value): + return int(value.replace('-', ''), 16) + finally: + pipe.close() def _netbios_getnode(): """Get the hardware address on Windows using NetBIOS calls. Modified: pypy/branch/fast-forward/lib-python/2.7.0/xml/dom/expatbuilder.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/xml/dom/expatbuilder.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/xml/dom/expatbuilder.py Fri Nov 5 23:50:59 2010 @@ -242,7 +242,7 @@ doctype = self.document.implementation.createDocumentType( doctypeName, publicId, systemId) doctype.ownerDocument = self.document - self.document.childNodes.append(doctype) + _append_child(self.document, doctype) self.document.doctype = doctype if self._filter and self._filter.acceptNode(doctype) == FILTER_REJECT: self.document.doctype = None Modified: pypy/branch/fast-forward/lib-python/2.7.0/xml/sax/saxutils.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/xml/sax/saxutils.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/xml/sax/saxutils.py Fri Nov 5 23:50:59 2010 @@ -103,6 +103,12 @@ def _qname(self, name): """Builds a qualified name from a (ns_url, localname) pair""" if name[0]: + # Per http://www.w3.org/XML/1998/namespace, The 'xml' prefix is + # bound by definition to http://www.w3.org/XML/1998/namespace. It + # does not need to be declared and will not usually be found in + # self._current_context. + if 'http://www.w3.org/XML/1998/namespace' == name[0]: + return 'xml:' + name[1] # The name is in a non-empty namespace prefix = self._current_context[name[0]] if prefix: From afa at codespeak.net Fri Nov 5 23:55:25 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 5 Nov 2010 23:55:25 +0100 (CET) Subject: [pypy-svn] r78777 - in pypy/branch/fast-forward/lib-python/modified-2.7.0: . multiprocessing test Message-ID: <20101105225525.2EC9E282C06@codespeak.net> Author: afa Date: Fri Nov 5 23:55:23 2010 New Revision: 78777 Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/multiprocessing/__init__.py pypy/branch/fast-forward/lib-python/modified-2.7.0/sysconfig.py pypy/branch/fast-forward/lib-python/modified-2.7.0/test/regrtest.py pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_file2k.py pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_fileio.py pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_scope.py pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_socket.py pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_threading.py Log: Merge the previous change into the modified version of the files Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/multiprocessing/__init__.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.7.0/multiprocessing/__init__.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/multiprocessing/__init__.py Fri Nov 5 23:55:23 2010 @@ -116,7 +116,8 @@ num = 0 elif 'bsd' in sys.platform or sys.platform == 'darwin': try: - num = int(os.popen('sysctl -n hw.ncpu').read()) + with os.popen('sysctl -n hw.ncpu') as p: + num = int(p.read()) except ValueError: num = 0 else: Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/sysconfig.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.7.0/sysconfig.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/sysconfig.py Fri Nov 5 23:55:23 2010 @@ -93,21 +93,28 @@ _EXEC_PREFIX = os.path.normpath(sys.exec_prefix) _CONFIG_VARS = None _USER_BASE = None + +def _safe_realpath(path): + try: + return realpath(path) + except OSError: + return path + if sys.executable: - _PROJECT_BASE = os.path.dirname(realpath(sys.executable)) + _PROJECT_BASE = os.path.dirname(_safe_realpath(sys.executable)) else: # sys.executable can be empty if argv[0] has been changed and Python is # unable to retrieve the real program name - _PROJECT_BASE = realpath(os.getcwd()) + _PROJECT_BASE = _safe_realpath(os.getcwd()) if os.name == "nt" and "pcbuild" in _PROJECT_BASE[-8:].lower(): - _PROJECT_BASE = realpath(os.path.join(_PROJECT_BASE, pardir)) + _PROJECT_BASE = _safe_realpath(os.path.join(_PROJECT_BASE, pardir)) # PC/VS7.1 if os.name == "nt" and "\\pc\\v" in _PROJECT_BASE[-10:].lower(): - _PROJECT_BASE = realpath(os.path.join(_PROJECT_BASE, pardir, pardir)) + _PROJECT_BASE = _safe_realpath(os.path.join(_PROJECT_BASE, pardir, pardir)) # PC/AMD64 if os.name == "nt" and "\\pcbuild\\amd64" in _PROJECT_BASE[-14:].lower(): - _PROJECT_BASE = realpath(os.path.join(_PROJECT_BASE, pardir, pardir)) + _PROJECT_BASE = _safe_realpath(os.path.join(_PROJECT_BASE, pardir, pardir)) def is_python_build(): for fn in ("Setup.dist", "Setup.local"): @@ -188,7 +195,7 @@ vars['SO'] = '.pyd' vars['EXE'] = '.exe' vars['VERSION'] = _PY_VERSION_SHORT_NO_DOT - vars['BINDIR'] = os.path.dirname(realpath(sys.executable)) + vars['BINDIR'] = os.path.dirname(_safe_realpath(sys.executable)) # # public APIs @@ -308,8 +315,12 @@ # from a different directory. if _PYTHON_BUILD and os.name == "posix": base = _PROJECT_BASE + try: + cwd = os.getcwd() + except OSError: + cwd = None if (not os.path.isabs(_CONFIG_VARS['srcdir']) and - base != os.getcwd()): + base != cwd): # srcdir is relative and we are not in the same directory # as the executable. Assume executable is in the build # directory and make srcdir absolute. Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/regrtest.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/regrtest.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/regrtest.py Fri Nov 5 23:55:23 2010 @@ -133,11 +133,7 @@ decimal - Test the decimal module against a large suite that verifies compliance with standards. - compiler - Test the compiler package by compiling all the source - in the standard library and test suite. This takes - a long time. Enabling this resource also allows - test_tokenize to verify round-trip lexing on every - file in the test library. + cpu - Used for certain CPU-heavy tests. subprocess Run all tests for the subprocess module. @@ -215,7 +211,7 @@ from test import test_support RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'bsddb', - 'decimal', 'compiler', 'subprocess', 'urlfetch', 'gui', + 'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui', 'xpickle') TEMPDIR = os.path.abspath(tempfile.gettempdir()) @@ -365,9 +361,6 @@ usage(2, "-T and -j don't go together!") if use_mp and findleaks: usage(2, "-l and -j don't go together!") - if use_mp and max(sys.flags): - # TODO: inherit the environment and the flags - print "Warning: flags and environment variables are ignored with -j option" good = [] bad = [] @@ -496,6 +489,8 @@ ) yield (test, args_tuple) pending = tests_and_args() + opt_args = test_support.args_from_interpreter_flags() + base_cmd = [sys.executable] + opt_args + ['-m', 'test.regrtest'] def work(): # A worker thread. try: @@ -506,8 +501,7 @@ output.put((None, None, None, None)) return # -E is needed by some tests, e.g. test_import - popen = Popen([sys.executable, '-E', '-m', 'test.regrtest', - '--slaveargs', json.dumps(args_tuple)], + popen = Popen(base_cmd + ['--slaveargs', json.dumps(args_tuple)], stdout=PIPE, stderr=PIPE, universal_newlines=True, close_fds=(os.name != 'nt')) @@ -810,11 +804,12 @@ def get_asyncore_socket_map(self): asyncore = sys.modules.get('asyncore') - return asyncore and asyncore.socket_map or {} + # XXX Making a copy keeps objects alive until __exit__ gets called. + return asyncore and asyncore.socket_map.copy() or {} def restore_asyncore_socket_map(self, saved_map): asyncore = sys.modules.get('asyncore') if asyncore is not None: - asyncore.socket_map.clear() + asyncore.close_all(ignore_all=True) asyncore.socket_map.update(saved_map) def resource_info(self): @@ -830,9 +825,11 @@ return self def __exit__(self, exc_type, exc_val, exc_tb): + saved_values = self.saved_values + del self.saved_values for name, get, restore in self.resource_info(): current = get() - original = self.saved_values[name] + original = saved_values.pop(name) # Check for changes to the resource's value if current != original: self.changed = True @@ -933,6 +930,10 @@ def cleanup_test_droppings(testname, verbose): import shutil import stat + import gc + + # First kill any dangling references to open files etc. + gc.collect() # Try to clean up junk commonly left behind. While tests shouldn't leave # any files or directories behind, when a test fails that can be tedious @@ -1273,6 +1274,7 @@ test_bsddb3 test_curses test_epoll + test_gdb test_gdbm test_largefile test_locale Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_file2k.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_file2k.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_file2k.py Fri Nov 5 23:55:23 2010 @@ -136,6 +136,14 @@ def testReadWhenWriting(self): self.assertRaises(IOError, self.f.read) + def testNastyWritelinesGenerator(self): + def nasty(): + for i in range(5): + if i == 3: + self.f.close() + yield str(i) + self.assertRaises(ValueError, self.f.writelines, nasty()) + def testIssue5677(self): # Remark: Do not perform more than one test per open file, # since that does NOT catch the readline error on Windows. @@ -173,7 +181,7 @@ class OtherFileTests(unittest.TestCase): def testOpenDir(self): - this_dir = os.path.dirname(__file__) + this_dir = os.path.dirname(__file__) or os.curdir for mode in (None, "w"): try: if mode: Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_fileio.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_fileio.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_fileio.py Fri Nov 5 23:55:23 2010 @@ -269,7 +269,6 @@ # OS'es that don't support /dev/tty. pass else: - f = _FileIO("/dev/tty", "a") self.assertEquals(f.readable(), False) self.assertEquals(f.writable(), True) if sys.platform != "darwin" and \ @@ -348,6 +347,7 @@ f.truncate(15) self.assertEqual(f.tell(), 5) self.assertEqual(f.seek(0, os.SEEK_END), 15) + f.close() def testTruncateOnWindows(self): def bug801631(): Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py Fri Nov 5 23:55:23 2010 @@ -34,6 +34,12 @@ from multiprocessing import util +try: + from multiprocessing.sharedctypes import Value, copy + HAS_SHAREDCTYPES = True +except ImportError: + HAS_SHAREDCTYPES = False + # # # @@ -72,16 +78,6 @@ Structure = object c_int = c_double = None -try: - from ctypes import Value -except ImportError: - Value = None - -try: - from ctypes import copy as ctypes_copy -except ImportError: - ctypes_copy = None - # # Creates a wrapper for a function which records the time it takes to finish # @@ -119,6 +115,13 @@ else: return self.assertEqual(value, res) + # For the sanity of Windows users, rather than crashing or freezing in + # multiple ways. + def __reduce__(self, *args): + raise NotImplementedError("shouldn't try to pickle a test case") + + __reduce_ex__ = __reduce__ + # # Return the value of a semaphore # @@ -157,12 +160,13 @@ self.assertEqual(current.ident, os.getpid()) self.assertEqual(current.exitcode, None) - def _test(self, q, *args, **kwds): - current = self.current_process() + @classmethod + def _test(cls, q, *args, **kwds): + current = cls.current_process() q.put(args) q.put(kwds) q.put(current.name) - if self.TYPE != 'threads': + if cls.TYPE != 'threads': q.put(bytes(current.authkey)) q.put(current.pid) @@ -205,7 +209,8 @@ self.assertEquals(p.is_alive(), False) self.assertNotIn(p, self.active_children()) - def _test_terminate(self): + @classmethod + def _test_terminate(cls): time.sleep(1000) def test_terminate(self): @@ -254,13 +259,14 @@ p.join() self.assertNotIn(p, self.active_children()) - def _test_recursion(self, wconn, id): + @classmethod + def _test_recursion(cls, wconn, id): from multiprocessing import forking wconn.send(id) if len(id) < 2: for i in range(2): - p = self.Process( - target=self._test_recursion, args=(wconn, id+[i]) + p = cls.Process( + target=cls._test_recursion, args=(wconn, id+[i]) ) p.start() p.join() @@ -343,7 +349,8 @@ class _TestQueue(BaseTestCase): - def _test_put(self, queue, child_can_start, parent_can_continue): + @classmethod + def _test_put(cls, queue, child_can_start, parent_can_continue): child_can_start.wait() for i in range(6): queue.get() @@ -407,7 +414,8 @@ proc.join() - def _test_get(self, queue, child_can_start, parent_can_continue): + @classmethod + def _test_get(cls, queue, child_can_start, parent_can_continue): child_can_start.wait() #queue.put(1) queue.put(2) @@ -468,7 +476,8 @@ proc.join() - def _test_fork(self, queue): + @classmethod + def _test_fork(cls, queue): for i in range(10, 20): queue.put(i) # note that at this point the items may only be buffered, so the @@ -517,7 +526,8 @@ q.get() self.assertEqual(q.qsize(), 0) - def _test_task_done(self, q): + @classmethod + def _test_task_done(cls, q): for obj in iter(q.get, None): time.sleep(DELTA) q.task_done() @@ -630,7 +640,8 @@ class _TestCondition(BaseTestCase): - def f(self, cond, sleeping, woken, timeout=None): + @classmethod + def f(cls, cond, sleeping, woken, timeout=None): cond.acquire() sleeping.release() cond.wait(timeout) @@ -762,7 +773,8 @@ class _TestEvent(BaseTestCase): - def _test_event(self, event): + @classmethod + def _test_event(cls, event): time.sleep(TIMEOUT2) event.set() @@ -802,6 +814,8 @@ # # + at unittest.skipUnless(HAS_SHAREDCTYPES, + "requires multiprocessing.sharedctypes") class _TestValue(BaseTestCase): ALLOWED_TYPES = ('processes',) @@ -813,12 +827,12 @@ ('c', latin('x'), latin('y')) ] - def _test(self, values): - for sv, cv in zip(values, self.codes_values): + @classmethod + def _test(cls, values): + for sv, cv in zip(values, cls.codes_values): sv.value = cv[2] - @unittest.skipIf(c_int is None, "requires _ctypes") def test_value(self, raw=False): if raw: values = [self.RawValue(code, value) @@ -837,11 +851,9 @@ for sv, cv in zip(values, self.codes_values): self.assertEqual(sv.value, cv[2]) - @unittest.skipIf(c_int is None, "requires _ctypes") def test_rawvalue(self): self.test_value(raw=True) - @unittest.skipIf(c_int is None, "requires _ctypes") def test_getobj_getlock(self): val1 = self.Value('i', 5) lock1 = val1.get_lock() @@ -872,7 +884,8 @@ ALLOWED_TYPES = ('processes',) - def f(self, seq): + @classmethod + def f(cls, seq): for i in range(1, len(seq)): seq[i] += seq[i-1] @@ -1218,7 +1231,8 @@ ALLOWED_TYPES = ('manager',) - def _putter(self, address, authkey): + @classmethod + def _putter(cls, address, authkey): manager = QueueManager2( address=address, authkey=authkey, serializer=SERIALIZER ) @@ -1256,7 +1270,8 @@ class _TestManagerRestart(BaseTestCase): - def _putter(self, address, authkey): + @classmethod + def _putter(cls, address, authkey): manager = QueueManager( address=address, authkey=authkey, serializer=SERIALIZER) manager.connect() @@ -1268,7 +1283,11 @@ authkey = os.urandom(32) manager = QueueManager( address=('localhost', 0), authkey=authkey, serializer=SERIALIZER) - addr = manager.get_server().address + srvr = manager.get_server() + addr = srvr.address + # Close the connection.Listener socket which gets opened as a part + # of manager.get_server(). It's not needed for the test. + srvr.listener.close() manager.start() p = self.Process(target=self._putter, args=(manager.address, authkey)) @@ -1292,7 +1311,8 @@ ALLOWED_TYPES = ('processes', 'threads') - def _echo(self, conn): + @classmethod + def _echo(cls, conn): for msg in iter(conn.recv_bytes, SENTINEL): conn.send_bytes(msg) conn.close() @@ -1441,8 +1461,9 @@ ALLOWED_TYPES = ('processes', 'threads') - def _test(self, address): - conn = self.connection.Client(address) + @classmethod + def _test(cls, address): + conn = cls.connection.Client(address) conn.send('hello') conn.close() @@ -1598,11 +1619,14 @@ ('y', c_double) ] + at unittest.skipUnless(HAS_SHAREDCTYPES, + "requires multiprocessing.sharedctypes") class _TestSharedCTypes(BaseTestCase): ALLOWED_TYPES = ('processes',) - def _double(self, x, y, foo, arr, string): + @classmethod + def _double(cls, x, y, foo, arr, string): x.value *= 2 y.value *= 2 foo.x *= 2 @@ -1611,14 +1635,13 @@ for i in range(len(arr)): arr[i] *= 2 - @unittest.skipIf(Value is None, "requires ctypes.Value") def test_sharedctypes(self, lock=False): x = Value('i', 7, lock=lock) y = Value(c_double, 1.0/3.0, lock=lock) foo = Value(_Foo, 3, 2, lock=lock) arr = self.Array('d', range(10), lock=lock) string = self.Array('c', 20, lock=lock) - string.value = 'hello' + string.value = latin('hello') p = self.Process(target=self._double, args=(x, y, foo, arr, string)) p.start() @@ -1632,14 +1655,12 @@ self.assertAlmostEqual(arr[i], i*2) self.assertEqual(string.value, latin('hellohello')) - @unittest.skipIf(Value is None, "requires ctypes.Value") def test_synchronize(self): self.test_sharedctypes(lock=True) - @unittest.skipIf(ctypes_copy is None, "requires ctypes.copy") def test_copy(self): foo = _Foo(2, 5.0) - bar = ctypes_copy(foo) + bar = copy(foo) foo.x = 0 foo.y = 0 self.assertEqual(bar.x, 2) @@ -1653,7 +1674,8 @@ ALLOWED_TYPES = ('processes',) - def _test_finalize(self, conn): + @classmethod + def _test_finalize(cls, conn): class Foo(object): pass @@ -1747,7 +1769,8 @@ logger.info('nor will this') logger.setLevel(LOG_LEVEL) - def _test_level(self, conn): + @classmethod + def _test_level(cls, conn): logger = multiprocessing.get_logger() conn.send(logger.getEffectiveLevel()) Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_scope.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_scope.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_scope.py Fri Nov 5 23:55:23 2010 @@ -648,6 +648,13 @@ self.assertEqual(2, global_ns["result2"]) self.assertEqual(9, global_ns["result9"]) + def testTopIsNotSignificant(self): + # See #9997. + def top(a): + pass + def b(): + global a + def test_main(): with check_warnings(("import \* only allowed at module level", Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_socket.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_socket.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_socket.py Fri Nov 5 23:55:23 2010 @@ -15,6 +15,7 @@ import contextlib from weakref import proxy import signal +import math def try_address(host, port=0, family=socket.AF_INET): """Try to bind a socket on the given host:port and return True @@ -185,6 +186,11 @@ def clientSetUp(self): self.cli = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + def clientTearDown(self): + self.cli.close() + self.cli = None + ThreadableTest.clientTearDown(self) + class SocketConnectedTest(ThreadedTCPSocketTest): def __init__(self, methodName='runTest'): @@ -351,8 +357,10 @@ # Find one service that exists, then check all the related interfaces. # I've ordered this by protocols that have both a tcp and udp # protocol, at least for modern Linuxes. - if sys.platform in ('linux2', 'freebsd4', 'freebsd5', 'freebsd6', - 'freebsd7', 'freebsd8', 'darwin'): + if (sys.platform.startswith('linux') or + sys.platform.startswith('freebsd') or + sys.platform.startswith('netbsd') or + sys.platform == 'darwin'): # avoid the 'echo' service on this platform, as there is an # assumption breaking non-standard port/protocol entry services = ('daytime', 'qotd', 'domain') @@ -510,6 +518,7 @@ # Testing getsockname() port = self._get_unused_port() sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.addCleanup(sock.close) sock.bind(("0.0.0.0", port)) name = sock.getsockname() # XXX(nnorwitz): http://tinyurl.com/os5jz seems to indicate @@ -523,12 +532,14 @@ # Testing getsockopt() # We know a socket should start without reuse==0 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.addCleanup(sock.close) reuse = sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR) self.assertFalse(reuse != 0, "initial mode is reuse") def testSetSockOpt(self): # Testing setsockopt() sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.addCleanup(sock.close) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) reuse = sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR) self.assertFalse(reuse == 0, "failed to set reuse mode") @@ -561,15 +572,15 @@ finally: sock.close() + @unittest.skipUnless(os.name == "nt", "Windows specific") def test_sock_ioctl(self): - if os.name != "nt": - return self.assertTrue(hasattr(socket.socket, 'ioctl')) self.assertTrue(hasattr(socket, 'SIO_RCVALL')) self.assertTrue(hasattr(socket, 'RCVALL_ON')) self.assertTrue(hasattr(socket, 'RCVALL_OFF')) self.assertTrue(hasattr(socket, 'SIO_KEEPALIVE_VALS')) s = socket.socket() + self.addCleanup(s.close) self.assertRaises(ValueError, s.ioctl, -1, None) s.ioctl(socket.SIO_KEEPALIVE_VALS, (1, 100, 100)) @@ -612,6 +623,42 @@ socket.AI_PASSIVE) + def check_sendall_interrupted(self, with_timeout): + # socketpair() is not stricly required, but it makes things easier. + if not hasattr(signal, 'alarm') or not hasattr(socket, 'socketpair'): + self.skipTest("signal.alarm and socket.socketpair required for this test") + # Our signal handlers clobber the C errno by calling a math function + # with an invalid domain value. + def ok_handler(*args): + self.assertRaises(ValueError, math.acosh, 0) + def raising_handler(*args): + self.assertRaises(ValueError, math.acosh, 0) + 1 // 0 + c, s = socket.socketpair() + old_alarm = signal.signal(signal.SIGALRM, raising_handler) + try: + if with_timeout: + # Just above the one second minimum for signal.alarm + c.settimeout(1.5) + with self.assertRaises(ZeroDivisionError): + signal.alarm(1) + c.sendall(b"x" * (1024**2)) + if with_timeout: + signal.signal(signal.SIGALRM, ok_handler) + signal.alarm(1) + self.assertRaises(socket.timeout, c.sendall, b"x" * (1024**2)) + finally: + signal.signal(signal.SIGALRM, old_alarm) + c.close() + s.close() + + def test_sendall_interrupted(self): + self.check_sendall_interrupted(False) + + def test_sendall_interrupted_with_timeout(self): + self.check_sendall_interrupted(True) + + @unittest.skipUnless(thread, 'Threading required for this test.') class BasicTCPTest(SocketConnectedTest): @@ -674,12 +721,23 @@ return # On Windows, this doesn't exist fd = self.cli_conn.fileno() sock = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM) + self.addCleanup(sock.close) msg = sock.recv(1024) self.assertEqual(msg, MSG) def _testFromFd(self): self.serv_conn.send(MSG) + def testDup(self): + # Testing dup() + sock = self.cli_conn.dup() + self.addCleanup(sock.close) + msg = sock.recv(1024) + self.assertEqual(msg, MSG) + + def _testDup(self): + self.serv_conn.send(MSG) + def testShutdown(self): # Testing shutdown() msg = self.cli_conn.recv(1024) @@ -790,6 +848,7 @@ read, write, err = select.select([self.serv], [], []) if self.serv in read: conn, addr = self.serv.accept() + conn.close() else: self.fail("Error trying to do accept after select.") @@ -800,6 +859,7 @@ def testConnect(self): # Testing non-blocking connect conn, addr = self.serv.accept() + conn.close() def _testConnect(self): self.cli.settimeout(10) @@ -818,6 +878,7 @@ read, write, err = select.select([conn], [], []) if conn in read: msg = conn.recv(len(MSG)) + conn.close() self.assertEqual(msg, MSG) else: self.fail("Error during select call to non-blocking socket.") @@ -1067,6 +1128,7 @@ def test_connect(self): port = test_support.find_unused_port() cli = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + self.addCleanup(cli.close) with self.assertRaises(socket.error) as cm: cli.connect((HOST, port)) self.assertEqual(cm.exception.errno, errno.ECONNREFUSED) @@ -1104,16 +1166,19 @@ def _justAccept(self): conn, addr = self.serv.accept() + conn.close() testFamily = _justAccept def _testFamily(self): self.cli = socket.create_connection((HOST, self.port), timeout=30) + self.addCleanup(self.cli.close) self.assertEqual(self.cli.family, 2) testSourceAddress = _justAccept def _testSourceAddress(self): self.cli = socket.create_connection((HOST, self.port), timeout=30, source_address=('', self.source_port)) + self.addCleanup(self.cli.close) self.assertEqual(self.cli.getsockname()[1], self.source_port) # The port number being used is sufficient to show that the bind() # call happened. @@ -1125,6 +1190,7 @@ socket.setdefaulttimeout(42) try: self.cli = socket.create_connection((HOST, self.port)) + self.addCleanup(self.cli.close) finally: socket.setdefaulttimeout(None) self.assertEquals(self.cli.gettimeout(), 42) @@ -1136,6 +1202,7 @@ socket.setdefaulttimeout(30) try: self.cli = socket.create_connection((HOST, self.port), timeout=None) + self.addCleanup(self.cli.close) finally: socket.setdefaulttimeout(None) self.assertEqual(self.cli.gettimeout(), None) @@ -1148,6 +1215,7 @@ testTimeoutValueNonamed = _justAccept def _testTimeoutValueNonamed(self): self.cli = socket.create_connection((HOST, self.port), 30) + self.addCleanup(self.cli.close) self.assertEqual(self.cli.gettimeout(), 30) @unittest.skipUnless(thread, 'Threading required for this test.') @@ -1167,6 +1235,7 @@ def testInsideTimeout(self): conn, addr = self.serv.accept() + self.addCleanup(conn.close) time.sleep(3) conn.send("done!") testOutsideTimeout = testInsideTimeout Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_support.py Fri Nov 5 23:55:23 2010 @@ -1008,7 +1008,7 @@ return obj def requires_resource(resource): - if resource_is_enabled(resource): + if is_resource_enabled(resource): return _id else: return unittest.skip("resource {0!r} is not enabled".format(resource)) @@ -1203,3 +1203,23 @@ return b"".join(chr(x) for x in b) except TypeError: return bytes(b) + +def args_from_interpreter_flags(): + """Return a list of command-line arguments reproducing the current + settings in sys.flags.""" + flag_opt_map = { + 'bytes_warning': 'b', + 'dont_write_bytecode': 'B', + 'ignore_environment': 'E', + 'no_user_site': 's', + 'no_site': 'S', + 'optimize': 'O', + 'py3k_warning': '3', + 'verbose': 'v', + } + args = [] + for flag, opt in flag_opt_map.items(): + v = getattr(sys.flags, flag) + if v > 0: + args.append('-' + opt * v) + return args Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_threading.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_threading.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_threading.py Fri Nov 5 23:55:23 2010 @@ -307,7 +307,7 @@ # Issue1733757 # Avoid a deadlock when sys.settrace steps into threading._shutdown import subprocess - rc = subprocess.call([sys.executable, "-c", """if 1: + p = subprocess.Popen([sys.executable, "-c", """if 1: import sys, threading # A deadlock-killer, to prevent the @@ -327,9 +327,16 @@ return func sys.settrace(func) - """]) + """], + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + self.addCleanup(p.stdout.close) + self.addCleanup(p.stderr.close) + stdout, stderr = p.communicate() + rc = p.returncode self.assertFalse(rc == 2, "interpreted was blocked") - self.assertTrue(rc == 0, "Unexpected error") + self.assertTrue(rc == 0, + "Unexpected error: " + repr(stderr)) def test_join_nondaemon_on_shutdown(self): # Issue 1722344 @@ -350,6 +357,8 @@ """], stdout=subprocess.PIPE, stderr=subprocess.PIPE) + self.addCleanup(p.stdout.close) + self.addCleanup(p.stderr.close) stdout, stderr = p.communicate() self.assertEqual(stdout.strip(), "Woke up, sleep function is: ") @@ -423,6 +432,7 @@ p = subprocess.Popen([sys.executable, "-c", script], stdout=subprocess.PIPE) rc = p.wait() data = p.stdout.read().replace('\r', '') + p.stdout.close() self.assertEqual(data, "end of main\nend of thread\n") self.assertFalse(rc == 2, "interpreter was blocked") self.assertTrue(rc == 0, "Unexpected error") @@ -467,7 +477,8 @@ return # Skip platforms with known problems forking from a worker thread. # See http://bugs.python.org/issue3863. - if sys.platform in ('freebsd4', 'freebsd5', 'freebsd6', 'os2emx'): + if sys.platform in ('freebsd4', 'freebsd5', 'freebsd6', 'netbsd5', + 'os2emx'): print >>sys.stderr, ('Skipping test_3_join_in_forked_from_thread' ' due to known OS bugs on'), sys.platform return From afa at codespeak.net Sat Nov 6 00:12:28 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sat, 6 Nov 2010 00:12:28 +0100 (CET) Subject: [pypy-svn] r78778 - pypy/branch/fast-forward/pypy/module/_rawffi Message-ID: <20101105231228.3BB03282C09@codespeak.net> Author: afa Date: Sat Nov 6 00:12:26 2010 New Revision: 78778 Modified: pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py Log: Fix translation Modified: pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py (original) +++ pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py Sat Nov 6 00:12:26 2010 @@ -506,7 +506,7 @@ return wcharp2unicode(space, address) s = rffi.wcharpsize2unicode(rffi.cast(rffi.CCHARP, address), maxlength) return space.wrap(s) -charp2rawstring.unwrap_spec = [ObjSpace, r_uint, int] +wcharp2rawunicode.unwrap_spec = [ObjSpace, r_uint, int] if _MS_WINDOWS: def FormatError(space, code): From afa at codespeak.net Sat Nov 6 00:28:17 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sat, 6 Nov 2010 00:28:17 +0100 (CET) Subject: [pypy-svn] r78779 - in pypy/branch/fast-forward/pypy/module/_hashlib: . test Message-ID: <20101105232817.E5AB5282B9D@codespeak.net> Author: afa Date: Sat Nov 6 00:28:16 2010 New Revision: 78779 Modified: pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py Log: All hashlib functions should accept buffers Modified: pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py (original) +++ pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py Sat Nov 6 00:28:16 2010 @@ -123,7 +123,7 @@ # shortcut functions def make_new_hash(name, funcname): @func_renamer(funcname) - @unwrap_spec(ObjSpace, str) + @unwrap_spec(ObjSpace, 'bufferstr') def new_hash(space, string=''): return new(space, name, string) return new_hash Modified: pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py (original) +++ pypy/branch/fast-forward/pypy/module/_hashlib/test/test_hashlib.py Sat Nov 6 00:28:16 2010 @@ -77,4 +77,5 @@ h = _hashlib.new('md5', b) h.update(b) assert h.digest() == _hashlib.openssl_md5('x' * 20).digest() + _hashlib.openssl_sha1(b).digest() From afa at codespeak.net Sat Nov 6 00:31:04 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sat, 6 Nov 2010 00:31:04 +0100 (CET) Subject: [pypy-svn] r78780 - pypy/branch/fast-forward/lib-python/modified-2.7.0/test Message-ID: <20101105233104.6B674282BF5@codespeak.net> Author: afa Date: Sat Nov 6 00:31:02 2010 New Revision: 78780 Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py Log: Pass one more test in test_multiprocessing Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py Sat Nov 6 00:31:02 2010 @@ -1682,6 +1682,7 @@ a = Foo() util.Finalize(a, conn.send, args=('a',)) del a # triggers callback for a + test_support.gc_collect() b = Foo() close_b = util.Finalize(b, conn.send, args=('b',)) From afa at codespeak.net Sat Nov 6 00:38:22 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sat, 6 Nov 2010 00:38:22 +0100 (CET) Subject: [pypy-svn] r78781 - pypy/branch/fast-forward/pypy/module/_rawffi Message-ID: <20101105233822.1BEA0282BF5@codespeak.net> Author: afa Date: Sat Nov 6 00:38:20 2010 New Revision: 78781 Modified: pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py Log: Fix translation again Modified: pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py (original) +++ pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py Sat Nov 6 00:38:20 2010 @@ -504,7 +504,7 @@ def wcharp2rawunicode(space, address, maxlength=-1): if maxlength == -1: return wcharp2unicode(space, address) - s = rffi.wcharpsize2unicode(rffi.cast(rffi.CCHARP, address), maxlength) + s = rffi.wcharpsize2unicode(rffi.cast(rffi.CWCHARP, address), maxlength) return space.wrap(s) wcharp2rawunicode.unwrap_spec = [ObjSpace, r_uint, int] From afa at codespeak.net Sat Nov 6 00:40:11 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sat, 6 Nov 2010 00:40:11 +0100 (CET) Subject: [pypy-svn] r78782 - pypy/branch/fast-forward/lib_pypy Message-ID: <20101105234011.DB8F2282BF5@codespeak.net> Author: afa Date: Sat Nov 6 00:40:10 2010 New Revision: 78782 Removed: pypy/branch/fast-forward/lib_pypy/pstats.py Log: Remove an outdated custom version of the library, and tests pass! From afa at codespeak.net Sat Nov 6 00:41:48 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sat, 6 Nov 2010 00:41:48 +0100 (CET) Subject: [pypy-svn] r78783 - pypy/branch/fast-forward/lib_pypy Message-ID: <20101105234148.7311E282BF5@codespeak.net> Author: afa Date: Sat Nov 6 00:41:46 2010 New Revision: 78783 Removed: pypy/branch/fast-forward/lib_pypy/runpy.py Log: /bin/bash: q: command not found From verte at codespeak.net Sat Nov 6 03:42:16 2010 From: verte at codespeak.net (verte at codespeak.net) Date: Sat, 6 Nov 2010 03:42:16 +0100 (CET) Subject: [pypy-svn] r78785 - pypy/extradoc/planning/hg-migration Message-ID: <20101106024216.D0FD0282BDC@codespeak.net> Author: verte Date: Sat Nov 6 03:42:13 2010 New Revision: 78785 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: add myself to the usermap Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Sat Nov 6 03:42:13 2010 @@ -105,3 +105,4 @@ blais=Martin Blais haypo=Victor Stinner trundle=Andreas St?hrk +verte=William ML Leslie From mwh at codespeak.net Sat Nov 6 03:46:49 2010 From: mwh at codespeak.net (mwh at codespeak.net) Date: Sat, 6 Nov 2010 03:46:49 +0100 (CET) Subject: [pypy-svn] r78786 - pypy/extradoc/planning/hg-migration Message-ID: <20101106024649.4701B282B9E@codespeak.net> Author: mwh Date: Sat Nov 6 03:46:47 2010 New Revision: 78786 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: my email address Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Sat Nov 6 03:46:47 2010 @@ -8,7 +8,7 @@ cfbolz=Carl Friedrich Bolz pedronis=Samuele Pedroni antocuni=Antonio Cuni -mwh=Michael Hudson +mwh=Michael Hudson tismer=Christian Tismer hpk=Holger Krekel afa=Amaury Forgeot d'Arc From rxe at codespeak.net Sat Nov 6 03:50:14 2010 From: rxe at codespeak.net (rxe at codespeak.net) Date: Sat, 6 Nov 2010 03:50:14 +0100 (CET) Subject: [pypy-svn] r78787 - pypy/extradoc/planning/hg-migration Message-ID: <20101106025014.A07AD282BEC@codespeak.net> Author: rxe Date: Sat Nov 6 03:50:13 2010 New Revision: 78787 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: add my email address Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Sat Nov 6 03:50:13 2010 @@ -13,7 +13,7 @@ hpk=Holger Krekel afa=Amaury Forgeot d'Arc ericvrp=Eric van Riet Paap -rxe=Richard Emslie +rxe=Richard Emslie ac=Anders Chrigstrom auc=Aurelien Campeas benjamin=Benjamin Peterson From hakanardo at codespeak.net Sat Nov 6 10:53:48 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Sat, 6 Nov 2010 10:53:48 +0100 (CET) Subject: [pypy-svn] r78788 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt Message-ID: <20101106095348.8ED5B282B9E@codespeak.net> Author: hakanardo Date: Sat Nov 6 10:53:46 2010 New Revision: 78788 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Log: Cleaned up the handling of the arguments of the final jump. They should now be inlined and forced in the exact same way as the arguments of all other resops. Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Sat Nov 6 10:53:46 2010 @@ -54,10 +54,15 @@ if not isinstance(a, Const): inputargs.append(a) - # this loop is equivalent to the main optimization loop in + # This loop is equivalent to the main optimization loop in # Optimizer.propagate_all_forward for newop in loop_operations: - newop.initarglist([self.inline_arg(a) for a in newop.getarglist()]) + if newop.getopnum() == rop.JUMP: + args = inputargs + else: + args = newop.getarglist() + newop.initarglist([self.inline_arg(a) for a in args]) + if newop.result: old_result = newop.result newop.result = newop.result.clonebox() @@ -67,14 +72,6 @@ if isinstance(descr, ResumeGuardDescr): descr.rd_snapshot = self.inline_snapshot(descr.rd_snapshot) - if newop.getopnum() == rop.JUMP: - args = [] - for arg in inputargs: - arg = argmap[arg] - args.append(self.getvalue(arg).force_box()) - newop.initarglist(args + inputargs[len(args):]) - - #print 'P: ', str(newop) self.emit_operation(newop) # Remove jump to make sure forced code are placed before it @@ -98,10 +95,10 @@ if not isinstance(a, Const) and not a in boxes_created_this_iteration: if a not in inputargs: inputargs.append(a) - b = self.getvalue(a).force_box() - if not isinstance(b, Const): - b = self.getvalue(argmap[b]).force_box() - jumpargs.append(b) + box = self.inline_arg(a) + if box in self.optimizer.values: + box = self.optimizer.values[box].force_box() + jumpargs.append(box) jmp.initarglist(jumpargs) self.optimizer.newoperations.append(jmp) From antocuni at codespeak.net Sat Nov 6 11:02:54 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Sat, 6 Nov 2010 11:02:54 +0100 (CET) Subject: [pypy-svn] r78789 - pypy/extradoc/planning/hg-migration Message-ID: <20101106100254.4D8E5282BD4@codespeak.net> Author: antocuni Date: Sat Nov 6 11:02:52 2010 New Revision: 78789 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: regenerate the map by looking at the history of everything under pypy/, not just pypy/trunk. Handle utf-8 characters correctly Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Sat Nov 6 11:02:52 2010 @@ -1,3 +1,5 @@ +# -*- encoding: utf-8 -*- +# # map codespeak usernames to mercurial usernames # # If you want your email address to be in the "official" PyPy repo, please add @@ -8,101 +10,130 @@ cfbolz=Carl Friedrich Bolz pedronis=Samuele Pedroni antocuni=Antonio Cuni -mwh=Michael Hudson -tismer=Christian Tismer hpk=Holger Krekel +mwh=Michael Hudson afa=Amaury Forgeot d'Arc +tismer=Christian Tismer +benjamin=Benjamin Peterson ericvrp=Eric van Riet Paap -rxe=Richard Emslie ac=Anders Chrigstrom -auc=Aurelien Campeas -benjamin=Benjamin Peterson +xoraxax=Alexander Schremmer +rxe=Richard Emslie ale=Anders Lehmann +auc=Aurelien Campeas +getxsick=Bartosz Skowron nik=Niklaus Haldimann +cami=Camillo Bruni +lac=Laura Creighton +david=David Schneider sanxiyn=Seo Sanghyeon santagada=Leonardo Santagada +tverwaes=Toon Verwaest +adim=Adrien Di Mascio rhymes=Lawrence Oluyede -xoraxax=Alexander Schremmer -jlg=Jakub Gustak +jacob=Jacob Hallen guido=Guido Wesdorp -niko=Niko Matsakis ludal=Ludovic Aubry +jlg=Jakub Gustak +bea=Beatrice During +hakanardo=Hakan Ardo +niko=Niko Matsakis alex=Alex Martelli -tverwaes=Toon Verwaest +jcreigh=Jason Creighton +iko=Anders Hammarquist +agaynor=Alex Gaynor stephan=Stephan Diehl -adim=Adrien Di Mascio +jandem=Jan de Mooij +pmaupin=Patrick Maupin sschwarzer=Stefan Schwarzer tomek=Tomek Meka -pmaupin=Patrick Maupin -jacob=Jacob Hallen -lac=Laura Creighton -bob=Bob Ippolito -cami=Camillo Bruni -simonb=Simon Burton +exarkun=Jean-Paul Calderone bgola=Bruno Gola +dan=Daniel Roberts +bob=Bob Ippolito afayolle=Alexandre Fayolle -agaynor=Alex Gaynor +simonb=Simon Burton +alastair=alastair mgedmin=Marius Gedminas -getxsick=Bartosz Skowron -gvanrossum=Guido van Rossum +witulski=John Witulski +nico=Nicolas Chauvat dialtone=Valentino Volonghi +magcius=Jean-Philippe St. Pierre +trundle=Andreas St?hrk +gvanrossum=Guido van Rossum +vinogradov=Pavel Vinogradov +jum=Jens-Uwe Mager +wlav=Wim Lavrijsen akuhn=Adrian Kuhn pdg=Paul deGrandis +gbrandl=Georg Brandl gromit=Gerald Klix wanja=Wanja Saatkamp -iko=Anders Hammarquist +boria=boria +davide=Davide Ancona oscar=Oscar Nierstrasz -jandem=Jan de Mooij goden=Eugene Oden -exarkun=Jean-Paul Calderone -lukas=Lukas Renggli -hakanardo=Hakan Ardo +leuschel=Michael Leuschel +docgok=Henry Mason guenter=Guenter Jantzen -dinu=Dinu Gherman -gbrandl=Georg Brandl +bert=Bert Freudenberg +lukas=Lukas Renggli +lene=Lene Wagner +regmee=Amit Regmi +adurdin=Andrew Durdin benyoung=Ben Young -nico=Nicolas Chauvat -rocco=Rocco Moretti +bigdog=Michael Schneider +briandorsey=Brian Dorsey +njriley=Nicholas Riley +igorto=Igor Trindade Oliveira micktwomey=Michael Twomey -boria=boria -jared.grubb Jared Grubb +rocco=Rocco Moretti +wildchild=Gabriel Lavoie +lucian=Lucian Branescu Mihaila +dinu=Dinu Gherman +jared.grubb=Jared Grubb +karlb=Karl Bartel odie=Olivier Dormond +haypo=Victor Stinner +antoine=Antoine Pitrou +atobe=Toby Watson +micke=Mikael Sch?nenberg +nshepperd=Neil Shepperd stuart=Stuart Williams -jum=Jens-Uwe Mager -jcreigh=Jason Creighton +hruske=Gasper Zejn justas=Justas Sadzevicius -wildchild=Gabriel Lavoie -micke=Mikael Schnenberg -briandorsey=Brian Dorsey -lucian=Lucian Branescu Mihaila +syt=Sylvain Thenault +alecu=Alejandro J. Cura +electronicru=Alexander Sedov +elmom=Elmo M?ntynen jriehl=Jonathan David Riehl -bea=Beatrice During -elmom=Elmo Mntynen -ancestor=Andreas Friedge -dan=Daniel Roberts quest=Anders Qvist amcintyre=Alan McIntyre -bert=Bert Freudenberg +tobami=Miquel Torres +alix=Alix Einfeldt pzieschang=Pieter Zieschang +aft=Andrew Thompson +blais=Martin Blais +busemann=Stephan Busemann esmljaos=Jacob Oscarson -lutz_p Lutz Paelike -hruske=Gasper Zejn -bigdog=Michael Schneider +henrikv=Henrik Vendelbo +iammisc=Travis Francis Athougies laszlo=Artur Lisiecki -lene=Lene Wagner -radix=Christopher Armstrong -jacek=Jacek Generowicz -busemann=Stephan Busemann -yusei=Yusei Tahara +lucio=Lucio Torre +lutz_p=Lutz Paelike +tav=tav +asigfrid=Anders Sigfridsson gotcha=Godefroid Chappelle -atobe=Toby Watson -aft=Andrew Thompson +jacek=Jacek Generowicz +jbaker=Jim Baker jgilbert=Joshua Gilbert -asigfrid=Anders Sigfridsson -david=David Schneider +misto=Fabrizio Milo +niemeyer=Gustavo Niemeyer +radix=Christopher Armstrong +verte=William Leslie +yusei=Yusei Tahara +anthon=Anthon van der Neut +jan=Jan Balster +lamby=Chris Lamb mcherm=Michael Chermside -tav=tav -blais=Martin Blais -haypo=Victor Stinner -trundle=Andreas St?hrk -verte=William ML Leslie +zooko=Zooko Wilcox-O Hearn From xoraxax at codespeak.net Sat Nov 6 11:08:38 2010 From: xoraxax at codespeak.net (xoraxax at codespeak.net) Date: Sat, 6 Nov 2010 11:08:38 +0100 (CET) Subject: [pypy-svn] r78790 - pypy/extradoc/planning/hg-migration Message-ID: <20101106100838.3F04A282BD4@codespeak.net> Author: xoraxax Date: Sat Nov 6 11:08:36 2010 New Revision: 78790 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: Added my mail address. Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Sat Nov 6 11:08:36 2010 @@ -17,7 +17,7 @@ benjamin=Benjamin Peterson ericvrp=Eric van Riet Paap ac=Anders Chrigstrom -xoraxax=Alexander Schremmer +xoraxax=Alexander Schremmer rxe=Richard Emslie ale=Anders Lehmann auc=Aurelien Campeas From afa at codespeak.net Sat Nov 6 13:25:37 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sat, 6 Nov 2010 13:25:37 +0100 (CET) Subject: [pypy-svn] r78791 - pypy/branch/fast-forward/lib-python/modified-2.7.0/test Message-ID: <20101106122537.ED5D8282B9D@codespeak.net> Author: afa Date: Sat Nov 6 13:25:35 2010 New Revision: 78791 Added: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_syntax.py - copied, changed from r78767, pypy/branch/fast-forward/lib-python/2.7.0/test/test_syntax.py Log: Fixes in test_syntax: - a limitation of CPython (20 nested blocks) - a CPython inconstency in SyntaxError messages Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_syntax.py (from r78767, pypy/branch/fast-forward/lib-python/2.7.0/test/test_syntax.py) ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_syntax.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_syntax.py Sat Nov 6 13:25:35 2010 @@ -5,7 +5,8 @@ >>> def f(x): ... global x Traceback (most recent call last): -SyntaxError: name 'x' is local and global (, line 1) + File "", line 1 +SyntaxError: name 'x' is local and global The tests are all raise SyntaxErrors. They were created by checking each C call that raises SyntaxError. There are several modules that @@ -375,7 +376,7 @@ In 2.5 there was a missing exception and an assert was triggered in a debug build. The number of blocks must be greater than CO_MAXBLOCKS. SF #1565514 - >>> while 1: + >>> while 1: # doctest:+SKIP ... while 2: ... while 3: ... while 4: From afa at codespeak.net Sat Nov 6 13:33:41 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sat, 6 Nov 2010 13:33:41 +0100 (CET) Subject: [pypy-svn] r78792 - pypy/branch/fast-forward/pypy/module/zlib Message-ID: <20101106123341.5AEB7282B9E@codespeak.net> Author: afa Date: Sat Nov 6 13:33:40 2010 New Revision: 78792 Modified: pypy/branch/fast-forward/pypy/module/zlib/interp_zlib.py Log: Fix failures in module/zlib/test Modified: pypy/branch/fast-forward/pypy/module/zlib/interp_zlib.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/zlib/interp_zlib.py (original) +++ pypy/branch/fast-forward/pypy/module/zlib/interp_zlib.py Sat Nov 6 13:33:40 2010 @@ -315,7 +315,7 @@ decompress.unwrap_spec = ['self', 'bufferstr', int] - def flush(self, length=0): + def flush(self, length=sys.maxint): """ flush( [length] ) -- This is kept for backward compatibility, because each call to decompress() immediately returns as much From afa at codespeak.net Sat Nov 6 13:40:49 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sat, 6 Nov 2010 13:40:49 +0100 (CET) Subject: [pypy-svn] r78793 - pypy/branch/fast-forward/lib-python Message-ID: <20101106124049.D3A36282BD4@codespeak.net> Author: afa Date: Sat Nov 6 13:40:48 2010 New Revision: 78793 Modified: pypy/branch/fast-forward/lib-python/TODO Log: Update TODO list for 2.7 Modified: pypy/branch/fast-forward/lib-python/TODO ============================================================================== --- pypy/branch/fast-forward/lib-python/TODO (original) +++ pypy/branch/fast-forward/lib-python/TODO Sat Nov 6 13:40:48 2010 @@ -31,6 +31,10 @@ - in test_os.py, fix posix.setregid(-1, -1), posix.setreuid(-1, -1). This proably requires to use the git_t typedef instead of rffi.INT. +- missing posix.(confstr|pathconf|fpathconf)(_names)? + +- some set() methods now accept a variable number of arguments + Medium tasks ------------ From david at codespeak.net Sat Nov 6 15:27:57 2010 From: david at codespeak.net (david at codespeak.net) Date: Sat, 6 Nov 2010 15:27:57 +0100 (CET) Subject: [pypy-svn] r78794 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . test Message-ID: <20101106142757.D9CC3282B90@codespeak.net> Author: david Date: Sat Nov 6 15:27:56 2010 New Revision: 78794 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Log: Implement extra load store instructions for half word and byte access and refactor tests a bit Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Sat Nov 6 15:27:56 2010 @@ -30,6 +30,77 @@ | (w & 0x1) << 21 | reg_operation(rt, rn, rm, imm, s, shifttype)) return f + +def define_extra_load_store_func(name, table): + def check_registers(r1, r2): + assert r1 % 2 == 0 + assert r1 + 1 == r2 + assert r1 != 14 + + n = ((table['op1'] & 0x1F) << 20 + | 0x1 << 7 + | (table['op2'] & 0x3) << 5 + | 0x1 << 4) + p = 1 + w = 0 + rncond = ('rn' in table and table['rn'] == '!0xF') + dual = (name[-4] == 'D') + + if dual: + if name[-2:] == 'rr': + def f(self, rt, rt2, rn, rm, cond=cond.AL): + check_registers(rt, rt2) + assert not (rncond and rn == 0xF) + self.write32(n + | cond << 28 + | (p & 0x1) << 24 + | (1 & 0x1) << 23 + | (w & 0x1) << 21 + | (rn & 0xF) << 16 + | (rt & 0xF) << 12 + | (rm & 0xF)) + else: + def f(self, rt, rt2, rn, imm=0, cond=cond.AL): + check_registers(rt, rt2) + assert not (rncond and rn == 0xF) + u, imm = self._encode_imm(imm) + self.write32(n + | cond << 28 + | (p & 0x1) << 24 + | (u & 0x1) << 23 + | (w & 0x1) << 21 + | (rn & 0xF) << 16 + | (rt & 0xF) << 12 + | ((imm >> 0x4) & 0xF) << 8 + | (imm & 0xF)) + + else: + if name[-2:] == 'rr': + def f(self, rt, rn, rm, cond=cond.AL): + assert not (rncond and rn == 0xF) + self.write32(n + | cond << 28 + | (p & 0x1) << 24 + | (1 & 0x1) << 23 + | (w & 0x1) << 21 + | (rn & 0xF) << 16 + | (rt & 0xF) << 12 + | (rm & 0xF)) + else: + def f(self, rt, rn, imm=0, cond=cond.AL): + assert not (rncond and rn == 0xF) + u, imm = self._encode_imm(imm) + self.write32(n + | cond << 28 + | (p & 0x1) << 24 + | (u & 0x1) << 23 + | (w & 0x1) << 21 + | (rn & 0xF) << 16 + | (rt & 0xF) << 12 + | ((imm >> 0x4) & 0xF) << 8 + | (imm & 0xF)) + return f + def define_data_proc_imm(name, table): n = (0x1 << 25 | (table['op'] & 0x1F) << 20) @@ -194,11 +265,11 @@ def reg_operation(rt, rn, rm, imm, s, shifttype): # XXX encode shiftype correctly return ((s & 0x1) << 20 - | (rn & 0xFF) << 16 - | (rt & 0xFF) << 12 + | (rn & 0xF) << 16 + | (rt & 0xF) << 12 | (imm & 0x1F) << 7 | (shifttype & 0x3) << 5 - | (rm & 0xFF)) + | (rm & 0xF)) def define_instruction(builder, key, val, target): f = builder(key, val) @@ -206,6 +277,7 @@ def define_instructions(target): i_g_map = [(instructions.load_store, define_load_store_func), + (instructions.extra_load_store, define_extra_load_store_func), (instructions.data_proc, define_data_proc), (instructions.data_proc_imm, define_data_proc_imm), (instructions.supervisor_and_coproc, define_supervisor_and_coproc), Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py Sat Nov 6 15:27:56 2010 @@ -10,6 +10,21 @@ 'LDRB_ri': {'A':0, 'op1': 0x5, 'B': 0, 'imm': True}, 'LDRB_rr': {'A':1, 'op1': 0x5, 'B': 0, 'imm': False}, } +extra_load_store = { #Section 5.2.8 + 'STRH_rr': {'op2': 0x1, 'op1': 0x0}, + 'LDRH_rr': {'op2': 0x1, 'op1': 0x1}, + 'STRH_ri': {'op2': 0x1, 'op1': 0x4}, + 'LDRH_ri': {'op2': 0x1, 'op1': 0x5, 'rn':'!0xF'}, + 'LDRD_rr': {'op2': 0x2, 'op1': 0x0}, + 'LDRSB_rr': {'op2': 0x2, 'op1': 0x1}, + 'LDRD_ri': {'op2': 0x2, 'op1': 0x4}, + 'LDRSB_ri': {'op2': 0x2, 'op1': 0x5, 'rn':'!0xF'}, + 'STRD_rr': {'op2': 0x3, 'op1': 0x0}, + 'LDRSH_rr': {'op2': 0x3, 'op1': 0x1}, + 'STRD_ri': {'op2': 0x3, 'op1': 0x4}, + 'LDRSH_ri': {'op2': 0x3, 'op1': 0x5, 'rn':'!0xF'}, +} + data_proc = { 'AND_rr': {'op1':0x0, 'op2':0, 'op3':0, 'result':True, 'base':True}, Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Sat Nov 6 15:27:56 2010 @@ -52,9 +52,8 @@ self.assert_equal('MOV r9, #255') def test_mov_ri_max(self): - py.test.skip('Check the actual largest thing') - self.cb.MOV_ri(r.r9.value, 0xFFF) - self.assert_equal('MOV r9, #4095') + self.cb.MOV_ri(r.r9.value, 0xFF) + self.assert_equal('MOV r9, #255') def test_str_ri(self): self.cb.STR_ri(r.r9.value, r.r14.value) @@ -109,9 +108,8 @@ self.assert_equal('SUB r2, r4, #123') def test_sub_ri2(self): - py.test.skip('XXX check the actual largest value') - self.cb.SUB_ri(r.r3.value, r.r7.value, 0xFFF) - self.assert_equal('SUB r3, r7, #4095') + self.cb.SUB_ri(r.r3.value, r.r7.value, 0xFF) + self.assert_equal('SUB r3, r7, #255') def test_cmp_ri(self): self.cb.CMP_ri(r.r3.value, 123) @@ -157,6 +155,19 @@ test = builder(key, value) setattr(TestInstrCodeBuilderForGeneratedInstr, test_name % key, test) +def gen_test_function(name, asm, args, kwargs=None): + if kwargs is None: + kwargs = {} + def f(self): + func = getattr(self.cb, name) + func(*args, **kwargs) + try: + f_name = name[:name.index('_')] + except ValueError, e: + f_name = name + self.assert_equal(asm % f_name) + return f + def gen_test_data_proc_imm_func(name, table): if table['result'] and table['base']: def f(self): @@ -164,32 +175,28 @@ func(r.r3.value, r.r7.value, 23) self.assert_equal('%s r3, r7, #23' % name[:name.index('_')]) py.test.raises(ValueError, 'func(r.r3.value, r.r7.value, -12)') - elif not table['base']: - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, 23) - self.assert_equal('%s r3, #23' % name[:name.index('_')]) else: - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, 23) - self.assert_equal('%s r3, #23' % name[:name.index('_')]) + f = gen_test_function(name, '%s r3, #23', [r.r3.value, 23]) return f def gen_test_imm_func(name, table): - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, r.r7.value, 23) - self.assert_equal('%s r3, [r7, #23]' % name[:name.index('_')]) - return f + return gen_test_function(name, '%s r3, [r7, #23]', [r.r3.value, r.r7.value, 23]) def gen_test_reg_func(name, table): - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, r.r7.value, r.r12.value) - self.assert_equal('%s r3, [r7, r12]' % name[:name.index('_')]) - return f + return gen_test_function(name, '%s r3, [r7, r12]', [r.r3.value, r.r7.value, r.r12.value]) +def gen_test_extra_load_store_func(name, table): + if name[-4] == 'D': + if name[-2:] == 'rr': + f = gen_test_function(name, '%s r4, [r8, r12]', [r.r4.value, r.r5.value, r.r8.value, r.r12.value]) + else: + f = gen_test_function(name, '%s r4, [r8, #223]', [r.r4.value, r.r5.value, r.r8.value, 223]) + else: + if name[-2:] == 'rr': + f = gen_test_function(name, '%s r4, [r5, r12]', [r.r4.value, r.r5.value, r.r12.value]) + else: + f = gen_test_function(name, '%s r4, [r5, #223]', [r.r4.value, r.r5.value, 223]) + return f def gen_test_mul_func(name, table): if 'acc' in table and table['acc']: if 'update_flags' in table and table['update_flags']: @@ -212,35 +219,20 @@ func(r.r3.value, r.r13.value, r.r7.value, r.r12.value) self.assert_equal("""%(name)s r3, r13, r7, r12 """ % {'name':name}) else: - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, r.r7.value, r.r12.value) - self.assert_equal("""%(name)s r3, r7, r12 """ % {'name':name}) + f = gen_test_function(name, '%s r3, r7, r12 ', [r.r3.value, r.r7.value, r.r12.value]) return f def gen_test_data_reg_shift_reg_func(name, table): if name[-2:] == 'rr': - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, r.r7.value, r.r12.value) - self.assert_equal('%s r3, r7, r12' % name[:name.index('_')]) - + f = gen_test_function(name, '%s r3, r7, r12', [r.r3.value, r.r7.value, r.r12.value]) else: - if 'result' in table and not table['result']: - result = False - else: - result = True + result = 'result' not in table or table['result'] if result: - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, r.r7.value, r.r8.value, r.r11.value, shifttype=0x2) - self.assert_equal('%s r3, r7, r8, ASR r11' % name[:name.index('_')]) + f = gen_test_function(name, '%s r3, r7, r8, ASR r11', + [r.r3.value, r.r7.value, r.r8.value, r.r11.value], {'shifttype':0x2}) else: - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, r.r7.value, r.r11.value, shifttype=0x2) - self.assert_equal('%s r3, r7, ASR r11' % name[:name.index('_')]) - + f = gen_test_function(name, '%s r3, r7, ASR r11', + [r.r3.value, r.r7.value, r.r11.value], {'shifttype':0x2}) return f def gen_test_data_reg_func(name, table): @@ -263,10 +255,7 @@ %(name)sS r3, r7, r12 """ % {'name':op_name}) else: - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, r.r7.value) - self.assert_equal("""%(name)s r3, r7""" % {'name':op_name}) + f = gen_test_function(name, '%s r3, r7', [r.r3.value, r.r7.value]) return f @@ -279,6 +268,9 @@ f = gen_test_reg_func build_test(f, key, value, test_name) + for key, value in instructions.extra_load_store.iteritems(): + build_test(gen_test_extra_load_store_func, key, value, 'test_extra_load_store_%s' % test_name) + for key, value, in instructions.data_proc.iteritems(): build_test(gen_test_data_reg_func, key, value, test_name) From wildchild at codespeak.net Sat Nov 6 20:38:03 2010 From: wildchild at codespeak.net (wildchild at codespeak.net) Date: Sat, 6 Nov 2010 20:38:03 +0100 (CET) Subject: [pypy-svn] r78797 - pypy/extradoc/planning/hg-migration Message-ID: <20101106193803.AE8E6282BDB@codespeak.net> Author: wildchild Date: Sat Nov 6 20:38:02 2010 New Revision: 78797 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: Adding my e-mail address. Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Sat Nov 6 20:38:02 2010 @@ -1,4 +1,4 @@ -# -*- encoding: utf-8 -*- +?# -*- encoding: utf-8 -*- # # map codespeak usernames to mercurial usernames # @@ -88,7 +88,7 @@ igorto=Igor Trindade Oliveira micktwomey=Michael Twomey rocco=Rocco Moretti -wildchild=Gabriel Lavoie +wildchild=Gabriel Lavoie lucian=Lucian Branescu Mihaila dinu=Dinu Gherman jared.grubb=Jared Grubb From afa at codespeak.net Sat Nov 6 23:37:18 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sat, 6 Nov 2010 23:37:18 +0100 (CET) Subject: [pypy-svn] r78798 - pypy/branch/fast-forward/pypy/module/cpyext Message-ID: <20101106223718.098C3282B9D@codespeak.net> Author: afa Date: Sat Nov 6 23:37:15 2010 New Revision: 78798 Modified: pypy/branch/fast-forward/pypy/module/cpyext/structmember.py Log: Reduce duplicated code Modified: pypy/branch/fast-forward/pypy/module/cpyext/structmember.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/structmember.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/structmember.py Sat Nov 6 23:37:15 2010 @@ -9,35 +9,33 @@ from pypy.module.cpyext.stringobject import (PyString_FromString, PyString_FromStringAndSize) from pypy.module.cpyext.typeobjectdefs import PyMemberDef +from pypy.rlib.unroll import unrolling_iterable + +integer_converters = unrolling_iterable([ + (structmemberdefs.T_SHORT, rffi.SHORT, PyInt_AsLong), + (structmemberdefs.T_INT, rffi.INT, PyInt_AsLong), + (structmemberdefs.T_LONG, rffi.LONG, PyInt_AsLong), + (structmemberdefs.T_USHORT, rffi.USHORT, PyInt_AsUnsignedLong), + (structmemberdefs.T_UINT, rffi.UINT, PyInt_AsUnsignedLong), + (structmemberdefs.T_ULONG, rffi.ULONG, PyInt_AsUnsignedLong), + (structmemberdefs.T_BYTE, rffi.UCHAR, PyInt_AsLong), + ]) @cpython_api([PyObject, lltype.Ptr(PyMemberDef)], PyObject) def PyMember_GetOne(space, obj, w_member): addr = rffi.cast(ADDR, obj) addr += w_member.c_offset + member_type = rffi.cast(lltype.Signed, w_member.c_type) - if member_type == structmemberdefs.T_SHORT: - result = rffi.cast(rffi.SHORTP, addr) - w_result = space.wrap(result[0]) - elif member_type == structmemberdefs.T_INT: - result = rffi.cast(rffi.INTP, addr) - w_result = space.wrap(result[0]) - elif member_type == structmemberdefs.T_LONG: - result = rffi.cast(rffi.LONGP, addr) - w_result = space.wrap(result[0]) - elif member_type == structmemberdefs.T_USHORT: - result = rffi.cast(rffi.USHORTP, addr) - w_result = space.wrap(result[0]) - elif member_type == structmemberdefs.T_UINT: - result = rffi.cast(rffi.UINTP, addr) - w_result = space.wrap(result[0]) - elif member_type == structmemberdefs.T_ULONG: - result = rffi.cast(rffi.ULONGP, addr) - w_result = space.wrap(result[0]) - elif member_type == structmemberdefs.T_BYTE: - result = rffi.cast(rffi.CCHARP, addr) - w_result = space.wrap(result[0]) - elif member_type == structmemberdefs.T_STRING: + for converter in integer_converters: + typ, lltype, _ = converter + if typ == member_type + result = rffi.cast(rffi.CArrayPtr(lltype), addr) + w_result = space.wrap(result[0]) + return w_result + + if member_type == structmemberdefs.T_STRING: result = rffi.cast(rffi.CCHARPP, addr) if result[0]: w_result = PyString_FromString(space, result[0]) @@ -49,16 +47,19 @@ elif member_type == structmemberdefs.T_CHAR: result = rffi.cast(rffi.CCHARP, addr) w_result = space.wrap(result[0]) - elif member_type in [structmemberdefs.T_OBJECT, - structmemberdefs.T_OBJECT_EX]: + elif member_type == structmemberdefs.T_OBJECT: obj_ptr = rffi.cast(PyObjectP, addr) if obj_ptr[0]: w_result = from_ref(space, obj_ptr[0]) else: - if member_type == structmemberdefs.T_OBJECT_EX: - w_name = space.wrap(rffi.charp2str(w_member.c_name)) - raise OperationError(space.w_AttributeError, w_name) w_result = space.w_None + elif member_type == T_OBJECT_EX: + obj_ptr = rffi.cast(PyObjectP, addr) + if obj_ptr[0]: + w_result = from_ref(space, obj_ptr[0]) + else: + w_name = space.wrap(rffi.charp2str(w_member.c_name)) + raise OperationError(space.w_AttributeError, w_name) else: raise OperationError(space.w_SystemError, space.wrap("bad memberdescr type")) @@ -86,35 +87,15 @@ raise OperationError(space.w_TypeError, space.wrap("can't delete numeric/char attribute")) - if member_type == structmemberdefs.T_SHORT: - w_long_value = PyInt_AsLong(space, w_value) - array = rffi.cast(rffi.SHORTP, addr) - array[0] = rffi.cast(rffi.SHORT, w_long_value) - elif member_type == structmemberdefs.T_INT: - w_long_value = PyInt_AsLong(space, w_value) - array = rffi.cast(rffi.INTP, addr) - array[0] = rffi.cast(rffi.INT, w_long_value) - elif member_type == structmemberdefs.T_LONG: - w_long_value = PyInt_AsLong(space, w_value) - array = rffi.cast(rffi.LONGP, addr) - array[0] = rffi.cast(rffi.LONG, w_long_value) - elif member_type == structmemberdefs.T_USHORT: - w_long_value = PyInt_AsUnsignedLong(space, w_value) - array = rffi.cast(rffi.USHORTP, addr) - array[0] = rffi.cast(rffi.USHORT, w_long_value) - elif member_type == structmemberdefs.T_UINT: - w_long_value = PyInt_AsUnsignedLong(space, w_value) - array = rffi.cast(rffi.UINTP, addr) - array[0] = rffi.cast(rffi.UINT, w_long_value) - elif member_type == structmemberdefs.T_ULONG: - w_long_value = PyInt_AsUnsignedLong(space, w_value) - array = rffi.cast(rffi.ULONGP, addr) - array[0] = rffi.cast(rffi.ULONG, w_long_value) - elif member_type == structmemberdefs.T_BYTE: - w_long_value = PyInt_AsLong(space, w_value) - array = rffi.cast(rffi.CCHARP, addr) - array[0] = rffi.cast(rffi.CHAR, w_long_value) - elif member_type == structmemberdefs.T_CHAR: + for converter in integer_converters: + typ, lltype, getter = converter + if typ == member_type: + value = getter(space, w_value) + array = rffi.cast(rffi.CarrayPtr(lltype), addr) + array[0] = rffi.cast(lltype, value) + return 0 + + if member_type == structmemberdefs.T_CHAR: str_value = space.str_w(w_value) if len(str_value) != 1: raise OperationError(space.w_TypeError, From afa at codespeak.net Sat Nov 6 23:55:00 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sat, 6 Nov 2010 23:55:00 +0100 (CET) Subject: [pypy-svn] r78799 - pypy/branch/fast-forward/pypy/module/cpyext Message-ID: <20101106225500.A82FF282BE0@codespeak.net> Author: afa Date: Sat Nov 6 23:54:57 2010 New Revision: 78799 Modified: pypy/branch/fast-forward/pypy/module/cpyext/structmember.py Log: Run tests, and fix mistakes Modified: pypy/branch/fast-forward/pypy/module/cpyext/structmember.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/structmember.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/structmember.py Sat Nov 6 23:54:57 2010 @@ -1,7 +1,7 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.typedef import TypeDef, GetSetProperty from pypy.rpython.lltypesystem import rffi, lltype -from pypy.module.cpyext import structmemberdefs +from pypy.module.cpyext.structmemberdefs import * from pypy.module.cpyext.api import ADDR, PyObjectP, cpython_api from pypy.module.cpyext.intobject import PyInt_AsLong, PyInt_AsUnsignedLong from pypy.module.cpyext.pyerrors import PyErr_Occurred @@ -12,13 +12,13 @@ from pypy.rlib.unroll import unrolling_iterable integer_converters = unrolling_iterable([ - (structmemberdefs.T_SHORT, rffi.SHORT, PyInt_AsLong), - (structmemberdefs.T_INT, rffi.INT, PyInt_AsLong), - (structmemberdefs.T_LONG, rffi.LONG, PyInt_AsLong), - (structmemberdefs.T_USHORT, rffi.USHORT, PyInt_AsUnsignedLong), - (structmemberdefs.T_UINT, rffi.UINT, PyInt_AsUnsignedLong), - (structmemberdefs.T_ULONG, rffi.ULONG, PyInt_AsUnsignedLong), - (structmemberdefs.T_BYTE, rffi.UCHAR, PyInt_AsLong), + (T_SHORT, rffi.SHORT, PyInt_AsLong), + (T_INT, rffi.INT, PyInt_AsLong), + (T_LONG, rffi.LONG, PyInt_AsLong), + (T_USHORT, rffi.USHORT, PyInt_AsUnsignedLong), + (T_UINT, rffi.UINT, PyInt_AsUnsignedLong), + (T_ULONG, rffi.ULONG, PyInt_AsUnsignedLong), + (T_BYTE, rffi.UCHAR, PyInt_AsLong), ]) @@ -29,25 +29,25 @@ member_type = rffi.cast(lltype.Signed, w_member.c_type) for converter in integer_converters: - typ, lltype, _ = converter - if typ == member_type - result = rffi.cast(rffi.CArrayPtr(lltype), addr) + typ, lltyp, _ = converter + if typ == member_type: + result = rffi.cast(rffi.CArrayPtr(lltyp), addr) w_result = space.wrap(result[0]) return w_result - if member_type == structmemberdefs.T_STRING: + if member_type == T_STRING: result = rffi.cast(rffi.CCHARPP, addr) if result[0]: w_result = PyString_FromString(space, result[0]) else: w_result = space.w_None - elif member_type == structmemberdefs.T_STRING_INPLACE: + elif member_type == T_STRING_INPLACE: result = rffi.cast(rffi.CCHARP, addr) w_result = PyString_FromString(space, result) - elif member_type == structmemberdefs.T_CHAR: + elif member_type == T_CHAR: result = rffi.cast(rffi.CCHARP, addr) w_result = space.wrap(result[0]) - elif member_type == structmemberdefs.T_OBJECT: + elif member_type == T_OBJECT: obj_ptr = rffi.cast(PyObjectP, addr) if obj_ptr[0]: w_result = from_ref(space, obj_ptr[0]) @@ -73,37 +73,35 @@ member_type = rffi.cast(lltype.Signed, w_member.c_type) flags = rffi.cast(lltype.Signed, w_member.c_flags) - if (flags & structmemberdefs.READONLY or - member_type in [structmemberdefs.T_STRING, - structmemberdefs.T_STRING_INPLACE]): + if (flags & READONLY or + member_type in [T_STRING, T_STRING_INPLACE]): raise OperationError(space.w_TypeError, space.wrap("readonly attribute")) elif w_value is None: - if member_type == structmemberdefs.T_OBJECT_EX: + if member_type == T_OBJECT_EX: if not rffi.cast(PyObjectP, addr)[0]: w_name = space.wrap(rffi.charp2str(w_member.c_name)) raise OperationError(space.w_AttributeError, w_name) - elif member_type != structmemberdefs.T_OBJECT: + elif member_type != T_OBJECT: raise OperationError(space.w_TypeError, space.wrap("can't delete numeric/char attribute")) for converter in integer_converters: - typ, lltype, getter = converter + typ, lltyp, getter = converter if typ == member_type: value = getter(space, w_value) - array = rffi.cast(rffi.CarrayPtr(lltype), addr) - array[0] = rffi.cast(lltype, value) + array = rffi.cast(rffi.CArrayPtr(lltyp), addr) + array[0] = rffi.cast(lltyp, value) return 0 - if member_type == structmemberdefs.T_CHAR: + if member_type == T_CHAR: str_value = space.str_w(w_value) if len(str_value) != 1: raise OperationError(space.w_TypeError, space.wrap("string of length 1 expected")) array = rffi.cast(rffi.CCHARP, addr) array[0] = str_value[0] - elif member_type in [structmemberdefs.T_OBJECT, - structmemberdefs.T_OBJECT_EX]: + elif member_type in [T_OBJECT, T_OBJECT_EX]: array = rffi.cast(PyObjectP, addr) if array[0]: Py_DecRef(space, array[0]) From afa at codespeak.net Sun Nov 7 00:04:15 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 00:04:15 +0100 (CET) Subject: [pypy-svn] r78800 - in pypy/branch/fast-forward/pypy/module/cpyext: . include Message-ID: <20101106230415.929AB282BE0@codespeak.net> Author: afa Date: Sun Nov 7 00:04:14 2010 New Revision: 78800 Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/structmember.h pypy/branch/fast-forward/pypy/module/cpyext/structmember.py pypy/branch/fast-forward/pypy/module/cpyext/structmemberdefs.py Log: More types in structmembers Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/structmember.h ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/include/structmember.h (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/structmember.h Sun Nov 7 00:04:14 2010 @@ -24,17 +24,23 @@ #define T_SHORT 0 #define T_INT 1 #define T_LONG 2 +#define T_FLOAT 3 +#define T_DOUBLE 4 #define T_STRING 5 #define T_OBJECT 6 #define T_CHAR 7 /* 1-character string */ #define T_BYTE 8 /* 8-bit signed int */ +#define T_UBYTE 9 #define T_USHORT 10 #define T_UINT 11 #define T_ULONG 12 -#define T_STRING_INPLACE 13 /* Strings contained in the structure */ +#define T_STRING_INPLACE 13 /* Strings contained in the structure */ +#define T_BOOL 14 #define T_OBJECT_EX 16 /* Like T_OBJECT, but raises AttributeError when the value is NULL, instead of converting to None. */ +#define T_LONGLONG 17 +#define T_ULONGLONG 18 /* Flags */ #define READONLY 1 Modified: pypy/branch/fast-forward/pypy/module/cpyext/structmember.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/structmember.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/structmember.py Sun Nov 7 00:04:14 2010 @@ -6,8 +6,11 @@ from pypy.module.cpyext.intobject import PyInt_AsLong, PyInt_AsUnsignedLong from pypy.module.cpyext.pyerrors import PyErr_Occurred from pypy.module.cpyext.pyobject import PyObject, Py_DecRef, from_ref, make_ref -from pypy.module.cpyext.stringobject import (PyString_FromString, - PyString_FromStringAndSize) +from pypy.module.cpyext.stringobject import ( + PyString_FromString, PyString_FromStringAndSize) +from pypy.module.cpyext.floatobject import PyFloat_AsDouble +from pypy.module.cpyext.longobject import ( + PyLong_AsLongLong, PyLong_AsUnsignedLongLong) from pypy.module.cpyext.typeobjectdefs import PyMemberDef from pypy.rlib.unroll import unrolling_iterable @@ -19,6 +22,12 @@ (T_UINT, rffi.UINT, PyInt_AsUnsignedLong), (T_ULONG, rffi.ULONG, PyInt_AsUnsignedLong), (T_BYTE, rffi.UCHAR, PyInt_AsLong), + (T_UBYTE, rffi.UCHAR, PyInt_AsUnsignedLong), + (T_BOOL, rffi.UCHAR, PyInt_AsLong), + (T_FLOAT, rffi.FLOAT, PyFloat_AsDouble), + (T_DOUBLE, rffi.DOUBLE, PyFloat_AsDouble), + (T_LONGLONG, rffi.LONGLONG, PyLong_AsLongLong), + (T_ULONGLONG, rffi.ULONGLONG, PyLong_AsUnsignedLongLong), ]) Modified: pypy/branch/fast-forward/pypy/module/cpyext/structmemberdefs.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/structmemberdefs.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/structmemberdefs.py Sun Nov 7 00:04:14 2010 @@ -1,14 +1,20 @@ T_SHORT = 0 T_INT = 1 T_LONG = 2 +T_FLOAT = 3 +T_DOUBLE = 4 T_STRING = 5 T_OBJECT = 6 T_CHAR = 7 T_BYTE = 8 +T_UBYTE = 9 T_USHORT = 10 T_UINT = 11 T_ULONG = 12 T_STRING_INPLACE = 13 +T_BOOL = 14 T_OBJECT_EX = 16 +T_LONGLONG = 17 +T_ULONGLONG = 18 READONLY = RO = 1 From afa at codespeak.net Sun Nov 7 00:32:56 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 00:32:56 +0100 (CET) Subject: [pypy-svn] r78801 - in pypy/branch/fast-forward/pypy/module/cpyext: . test Message-ID: <20101106233256.140A3282BDB@codespeak.net> Author: afa Date: Sun Nov 7 00:32:54 2010 New Revision: 78801 Modified: pypy/branch/fast-forward/pypy/module/cpyext/longobject.py pypy/branch/fast-forward/pypy/module/cpyext/test/test_longobject.py Log: cpyext: implement PyLong_AsLongAndOverflow Modified: pypy/branch/fast-forward/pypy/module/cpyext/longobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/longobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/longobject.py Sun Nov 7 00:32:54 2010 @@ -62,6 +62,48 @@ raised.""" return rffi.cast(rffi.ULONGLONG, space.r_ulonglong_w(w_long)) + at cpython_api([PyObject, rffi.CArrayPtr(rffi.INT_real)], lltype.Signed, + error=-1) +def PyLong_AsLongAndOverflow(space, w_long, overflow_ptr): + """ + Return a C long representation of the contents of pylong. If pylong is + greater than LONG_MAX or less than LONG_MIN, set *overflow to 1 or -1, + respectively, and return -1; otherwise, set *overflow to 0. If any other + exception occurs (for example a TypeError or MemoryError), then -1 will be + returned and *overflow will be 0.""" + overflow_ptr[0] = 0 + try: + return space.int_w(w_long) + except OperationError, e: + if not e.match(space, space.w_OverflowError): + raise + if space.is_true(space.gt(w_long, space.wrap(0))): + overflow_ptr[0] = 1 + else: + overflow_ptr[0] = -1 + return -1 + + at cpython_api([PyObject, rffi.CArrayPtr(rffi.INT_real)], rffi.LONGLONG, + error=-1) +def PyLong_AsLongLongAndOverflow(space, w_long, overflow_ptr): + """ + Return a C long long representation of the contents of pylong. If pylong is + greater than PY_LLONG_MAX or less than PY_LLONG_MIN, set *overflow to 1 or + -1, respectively, and return -1; otherwise, set *overflow to 0. If any + other exception occurs (for example a TypeError or MemoryError), then -1 + will be returned and *overflow will be 0.""" + overflow_ptr[0] = 0 + try: + return rffi.cast(rffi.LONGLONG, space.r_longlong_w(w_long)) + except OperationError, e: + if not e.match(space, space.w_OverflowError): + raise + if space.is_true(space.gt(w_long, space.wrap(0))): + overflow_ptr[0] = 1 + else: + overflow_ptr[0] = -1 + return -1 + @cpython_api([lltype.Float], PyObject) def PyLong_FromDouble(space, val): """Return a new PyLongObject object from v, or NULL on failure.""" Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_longobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/test/test_longobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_longobject.py Sun Nov 7 00:32:54 2010 @@ -64,6 +64,30 @@ assert api.PyErr_Occurred() api.PyErr_Clear() + def test_as_long_and_overflow(self, space, api): + overflow = lltype.malloc(rffi.INTP.TO, 1, flavor='raw') + assert api.PyLong_AsLongAndOverflow( + space.wrap(sys.maxint), overflow) == sys.maxint + assert api.PyLong_AsLongAndOverflow( + space.wrap(-sys.maxint - 2), overflow) == -1 + assert not api.PyErr_Occurred() + assert overflow[0] == -1 + lltype.free(overflow, flavor='raw') + + def test_as_longlong_and_overflow(self, space, api): + overflow = lltype.malloc(rffi.INTP.TO, 1, flavor='raw') + assert api.PyLong_AsLongLongAndOverflow( + space.wrap(1<<62), overflow) == 1<<62 + assert api.PyLong_AsLongLongAndOverflow( + space.wrap(1<<63), overflow) == -1 + assert not api.PyErr_Occurred() + assert overflow[0] == 1 + assert api.PyLong_AsLongLongAndOverflow( + space.wrap(-1<<64), overflow) == -1 + assert not api.PyErr_Occurred() + assert overflow[0] == -1 + lltype.free(overflow, flavor='raw') + def test_as_voidptr(self, space, api): w_l = api.PyLong_FromVoidPtr(lltype.nullptr(rffi.VOIDP.TO)) assert space.unwrap(w_l) == 0L From afa at codespeak.net Sun Nov 7 00:42:24 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 00:42:24 +0100 (CET) Subject: [pypy-svn] r78802 - pypy/branch/fast-forward/pypy/module/cpyext Message-ID: <20101106234224.C9625282BDB@codespeak.net> Author: afa Date: Sun Nov 7 00:42:23 2010 New Revision: 78802 Modified: pypy/branch/fast-forward/pypy/module/cpyext/longobject.py Log: These INT/Signed casts will drive me nuts. Modified: pypy/branch/fast-forward/pypy/module/cpyext/longobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/longobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/longobject.py Sun Nov 7 00:42:23 2010 @@ -71,16 +71,16 @@ respectively, and return -1; otherwise, set *overflow to 0. If any other exception occurs (for example a TypeError or MemoryError), then -1 will be returned and *overflow will be 0.""" - overflow_ptr[0] = 0 + overflow_ptr[0] = rffi.cast(rffi.INT_real, 0) try: return space.int_w(w_long) except OperationError, e: if not e.match(space, space.w_OverflowError): raise if space.is_true(space.gt(w_long, space.wrap(0))): - overflow_ptr[0] = 1 + overflow_ptr[0] = rffi.cast(rffi.INT_real, 1) else: - overflow_ptr[0] = -1 + overflow_ptr[0] = rffi.cast(rffi.INT_real, -1) return -1 @cpython_api([PyObject, rffi.CArrayPtr(rffi.INT_real)], rffi.LONGLONG, @@ -92,16 +92,16 @@ -1, respectively, and return -1; otherwise, set *overflow to 0. If any other exception occurs (for example a TypeError or MemoryError), then -1 will be returned and *overflow will be 0.""" - overflow_ptr[0] = 0 + overflow_ptr[0] = rffi.cast(rffi.INT_real, 0) try: return rffi.cast(rffi.LONGLONG, space.r_longlong_w(w_long)) except OperationError, e: if not e.match(space, space.w_OverflowError): raise if space.is_true(space.gt(w_long, space.wrap(0))): - overflow_ptr[0] = 1 + overflow_ptr[0] = rffi.cast(rffi.INT_real, 1) else: - overflow_ptr[0] = -1 + overflow_ptr[0] = rffi.cast(rffi.INT_real, -1) return -1 @cpython_api([lltype.Float], PyObject) From afa at codespeak.net Sun Nov 7 11:03:52 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 11:03:52 +0100 (CET) Subject: [pypy-svn] r78804 - pypy/branch/fast-forward/pypy/module/cpyext Message-ID: <20101107100352.AD944282B9D@codespeak.net> Author: afa Date: Sun Nov 7 11:03:51 2010 New Revision: 78804 Modified: pypy/branch/fast-forward/pypy/module/cpyext/structmember.py Log: Fix translation Modified: pypy/branch/fast-forward/pypy/module/cpyext/structmember.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/structmember.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/structmember.py Sun Nov 7 11:03:51 2010 @@ -41,7 +41,10 @@ typ, lltyp, _ = converter if typ == member_type: result = rffi.cast(rffi.CArrayPtr(lltyp), addr) - w_result = space.wrap(result[0]) + if lltyp is rffi.FLOAT: + w_result = space.wrap(rffi.cast(rffi.DOUBLE, result[0])) + else: + w_result = space.wrap(result[0]) return w_result if member_type == T_STRING: From jlg at codespeak.net Sun Nov 7 11:23:50 2010 From: jlg at codespeak.net (jlg at codespeak.net) Date: Sun, 7 Nov 2010 11:23:50 +0100 (CET) Subject: [pypy-svn] r78805 - pypy/extradoc/planning/hg-migration Message-ID: <20101107102350.09257282BDA@codespeak.net> Author: jlg Date: Sun Nov 7 11:23:49 2010 New Revision: 78805 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: added my email Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Sun Nov 7 11:23:49 2010 @@ -34,7 +34,7 @@ jacob=Jacob Hallen guido=Guido Wesdorp ludal=Ludovic Aubry -jlg=Jakub Gustak +jlg=Jakub Gustak bea=Beatrice During hakanardo=Hakan Ardo niko=Niko Matsakis From dan at codespeak.net Sun Nov 7 11:34:31 2010 From: dan at codespeak.net (dan at codespeak.net) Date: Sun, 7 Nov 2010 11:34:31 +0100 (CET) Subject: [pypy-svn] r78806 - pypy/branch/ootype-virtualrefs/pypy/rlib Message-ID: <20101107103431.9809C282BDB@codespeak.net> Author: dan Date: Sun Nov 7 11:34:29 2010 New Revision: 78806 Modified: pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py Log: Passing another test by implementing OOVRefRepr.convert_const(), hopefully correctly. Modified: pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py ============================================================================== --- pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py (original) +++ pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py Sun Nov 7 11:34:29 2010 @@ -8,6 +8,8 @@ from pypy.rpython.lltypesystem import lltype from pypy.rpython.error import TyperError +from pypy.rpython.ootypesystem import ootype + class SomeVRef(annmodel.SomeObject): @@ -61,6 +63,16 @@ class OOVRefRepr(VRefRepr): lowleveltype = OBJECT + def rtype_simple_call(self, hop): + [v] = hop.inputargs(self) + #v = hop.genop('jit_force_virtual', [v], resulttype = OBJECT) + return hop.genop('ooupcast', [v], resulttype = hop.r_result) + + def convert_const(self, value): + if value() is not None: + raise TypeError("only supports virtual_ref_None as a" + " prebuilt virtual_ref") + return ootype.ROOT._null vrefrepr = VRefRepr() oovrefrepr = OOVRefRepr() From dan at codespeak.net Sun Nov 7 13:06:25 2010 From: dan at codespeak.net (dan at codespeak.net) Date: Sun, 7 Nov 2010 13:06:25 +0100 (CET) Subject: [pypy-svn] r78807 - in pypy/branch/ootype-virtualrefs/pypy/rlib: . test Message-ID: <20101107120625.40E16282BDC@codespeak.net> Author: dan Date: Sun Nov 7 13:06:23 2010 New Revision: 78807 Modified: pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py Log: Passing all tests, changed ooupcast to oodowncast and fixed a test helper function. Modified: pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py ============================================================================== --- pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py (original) +++ pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py Sun Nov 7 13:06:23 2010 @@ -65,8 +65,9 @@ lowleveltype = OBJECT def rtype_simple_call(self, hop): [v] = hop.inputargs(self) - #v = hop.genop('jit_force_virtual', [v], resulttype = OBJECT) - return hop.genop('ooupcast', [v], resulttype = hop.r_result) + v = hop.genop('jit_force_virtual', [v], resulttype = OBJECT) + #RESULTTYPE, ptr + return hop.genop('oodowncast', [v], resulttype = hop.r_result) def convert_const(self, value): if value() is not None: Modified: pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py ============================================================================== --- pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py (original) +++ pypy/branch/ootype-virtualrefs/pypy/rlib/test/test__jit_vref.py Sun Nov 7 13:06:23 2010 @@ -11,6 +11,8 @@ from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin +from pypy.rpython.ootypesystem import ootype + class X(object): pass @@ -125,4 +127,4 @@ class TestOOtype(BaseTestVRef, OORtypeMixin): OBJECTTYPE = OBJECT def castable(self, TO, var): - return ootype.isSubclass(TO, lltype.typeOf(var)) + return ootype.isSubclass(lltype.typeOf(var), TO) From arigo at codespeak.net Sun Nov 7 13:54:07 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 13:54:07 +0100 (CET) Subject: [pypy-svn] r78808 - pypy/branch/fast-forward/pypy/module/unicodedata Message-ID: <20101107125407.072AB282BDC@codespeak.net> Author: arigo Date: Sun Nov 7 13:54:05 2010 New Revision: 78808 Modified: pypy/branch/fast-forward/pypy/module/unicodedata/generate_unicodedb.py Log: A "sorted()" was missing, producing hard-to-understand diffs whenever we regenerate the unicodedb_x_y_z file. Modified: pypy/branch/fast-forward/pypy/module/unicodedata/generate_unicodedb.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/unicodedata/generate_unicodedb.py (original) +++ pypy/branch/fast-forward/pypy/module/unicodedata/generate_unicodedb.py Sun Nov 7 13:54:05 2010 @@ -207,7 +207,7 @@ print >> outfile, '}' print >> outfile print >> outfile, '%s_corrected = {' % name - for key in base_dict: + for key in sorted(base_dict): if key not in dictionary: print >> outfile, '%r: None,' % key print >> outfile, '}' From arigo at codespeak.net Sun Nov 7 13:54:27 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 13:54:27 +0100 (CET) Subject: [pypy-svn] r78809 - pypy/branch/fast-forward/pypy/module/unicodedata Message-ID: <20101107125427.6331D282BDD@codespeak.net> Author: arigo Date: Sun Nov 7 13:54:25 2010 New Revision: 78809 Modified: pypy/branch/fast-forward/pypy/module/unicodedata/unicodedb_3_2_0.py Log: Regenerate this file with the missing sorted(). Modified: pypy/branch/fast-forward/pypy/module/unicodedata/unicodedb_3_2_0.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/unicodedata/unicodedb_3_2_0.py (original) +++ pypy/branch/fast-forward/pypy/module/unicodedata/unicodedb_3_2_0.py Sun Nov 7 13:54:25 2010 @@ -16721,6 +16721,17 @@ } _decimal_corrected = { +1984: None, +1985: None, +1986: None, +1987: None, +1988: None, +1989: None, +1990: None, +1991: None, +1992: None, +1993: None, +3046: None, 4240: None, 4241: None, 4242: None, @@ -16731,26 +16742,6 @@ 4247: None, 4248: None, 4249: None, -43216: None, -43217: None, -43218: None, -43219: None, -43220: None, -43221: None, -43222: None, -43223: None, -43224: None, -43225: None, -43264: None, -43265: None, -43266: None, -43267: None, -43268: None, -43269: None, -43270: None, -43271: None, -43272: None, -43273: None, 6470: None, 6471: None, 6472: None, @@ -16761,27 +16752,17 @@ 6477: None, 6478: None, 6479: None, -43472: None, -43473: None, +6608: None, +6609: None, 6610: None, -43475: None, -43476: None, -43477: None, -43478: None, -43479: None, -43480: None, -43481: None, +6611: None, +6612: None, +6613: None, +6614: None, +6615: None, +6616: None, +6617: None, 6618: None, -43600: None, -43601: None, -43602: None, -43603: None, -43604: None, -43605: None, -43606: None, -43607: None, -43608: None, -43609: None, 6784: None, 6785: None, 6786: None, @@ -16802,14 +16783,6 @@ 6807: None, 6808: None, 6809: None, -6608: None, -6611: None, -6612: None, -6613: None, -6614: None, -6615: None, -6616: None, -6617: None, 6992: None, 6993: None, 6994: None, @@ -16830,18 +16803,6 @@ 7095: None, 7096: None, 7097: None, -43474: None, -3046: None, -44016: None, -44017: None, -44018: None, -44019: None, -44020: None, -44021: None, -44022: None, -44023: None, -44024: None, -44025: None, 7232: None, 7233: None, 7234: None, @@ -16862,16 +16823,6 @@ 7255: None, 7256: None, 7257: None, -66720: None, -66721: None, -66722: None, -66723: None, -66724: None, -66725: None, -66726: None, -66727: None, -66728: None, -66729: None, 42528: None, 42529: None, 42530: None, @@ -16882,54 +16833,6 @@ 42535: None, 42536: None, 42537: None, -6609: None, -1984: None, -1985: None, -1986: None, -1987: None, -1988: None, -1989: None, -1990: None, -1991: None, -1992: None, -1993: None, -} -_digit = { -} - -_digit_corrected = { -69216: None, -43475: None, -43476: None, -69221: None, -7248: None, -7249: None, -7250: None, -7251: None, -7252: None, -7253: None, -7254: None, -7255: None, -7256: None, -7257: None, -4240: None, -4241: None, -4242: None, -4244: None, -4246: None, -4247: None, -4248: None, -4249: None, -66720: None, -66721: None, -66722: None, -66723: None, -66724: None, -66725: None, -66726: None, -66727: None, -66728: None, -66729: None, 43216: None, 43217: None, 43218: None, @@ -16940,8 +16843,6 @@ 43223: None, 43224: None, 43225: None, -127238: None, -9471: None, 43264: None, 43265: None, 43266: None, @@ -16950,9 +16851,74 @@ 43269: None, 43270: None, 43271: None, +43272: None, 43273: None, -127242: None, -69220: None, +43472: None, +43473: None, +43474: None, +43475: None, +43476: None, +43477: None, +43478: None, +43479: None, +43480: None, +43481: None, +43600: None, +43601: None, +43602: None, +43603: None, +43604: None, +43605: None, +43606: None, +43607: None, +43608: None, +43609: None, +44016: None, +44017: None, +44018: None, +44019: None, +44020: None, +44021: None, +44022: None, +44023: None, +44024: None, +44025: None, +66720: None, +66721: None, +66722: None, +66723: None, +66724: None, +66725: None, +66726: None, +66727: None, +66728: None, +66729: None, +} +_digit = { +} + +_digit_corrected = { +1984: None, +1985: None, +1986: None, +1987: None, +1988: None, +1989: None, +1990: None, +1991: None, +1992: None, +1993: None, +3046: None, +4240: None, +4241: None, +4242: None, +4243: None, +4244: None, +4245: None, +4246: None, +4247: None, +4248: None, +4249: None, 6470: None, 6471: None, 6472: None, @@ -16963,17 +16929,6 @@ 6477: None, 6478: None, 6479: None, -7232: None, -7233: None, -7234: None, -127232: None, -7235: None, -7236: None, -7237: None, -7238: None, -7239: None, -7240: None, -7241: None, 6608: None, 6609: None, 6610: None, @@ -16981,46 +16936,14 @@ 6612: None, 6613: None, 6614: None, +6615: None, 6616: None, 6617: None, 6618: None, -43600: None, -43603: None, -43604: None, -43605: None, -43606: None, -127233: None, -43607: None, -127234: None, -43608: None, -127235: None, -43609: None, -127236: None, -69224: None, -127237: None, -42528: None, -42529: None, -42530: None, -42531: None, -42532: None, -42533: None, -42534: None, -42535: None, -42536: None, -42537: None, -127239: None, -43272: None, -127241: None, -68160: None, -68161: None, -68162: None, -68163: None, -69217: None, -69218: None, -127240: None, 6784: None, 6785: None, 6786: None, +6787: None, 6788: None, 6789: None, 6790: None, @@ -17037,17 +16960,6 @@ 6807: None, 6808: None, 6809: None, -43602: None, -43479: None, -43472: None, -69223: None, -43477: None, -43478: None, -6615: None, -43480: None, -6787: None, -43481: None, -69219: None, 6992: None, 6993: None, 6994: None, @@ -17058,10 +16970,6 @@ 6999: None, 7000: None, 7001: None, -4243: None, -4245: None, -44016: None, -69222: None, 7088: None, 7089: None, 7090: None, @@ -17072,20 +16980,78 @@ 7095: None, 7096: None, 7097: None, -43601: None, -1984: None, -1985: None, -1986: None, -1987: None, -1988: None, -1989: None, -1990: None, -1991: None, -1992: None, -1993: None, +7232: None, +7233: None, +7234: None, +7235: None, +7236: None, +7237: None, +7238: None, +7239: None, +7240: None, +7241: None, +7248: None, +7249: None, +7250: None, +7251: None, +7252: None, +7253: None, +7254: None, +7255: None, +7256: None, +7257: None, +9471: None, +42528: None, +42529: None, +42530: None, +42531: None, +42532: None, +42533: None, +42534: None, +42535: None, +42536: None, +42537: None, +43216: None, +43217: None, +43218: None, +43219: None, +43220: None, +43221: None, +43222: None, +43223: None, +43224: None, +43225: None, +43264: None, +43265: None, +43266: None, +43267: None, +43268: None, +43269: None, +43270: None, +43271: None, +43272: None, +43273: None, +43472: None, 43473: None, -3046: None, 43474: None, +43475: None, +43476: None, +43477: None, +43478: None, +43479: None, +43480: None, +43481: None, +43600: None, +43601: None, +43602: None, +43603: None, +43604: None, +43605: None, +43606: None, +43607: None, +43608: None, +43609: None, +44016: None, 44017: None, 44018: None, 44019: None, @@ -17095,6 +17061,40 @@ 44023: None, 44024: None, 44025: None, +66720: None, +66721: None, +66722: None, +66723: None, +66724: None, +66725: None, +66726: None, +66727: None, +66728: None, +66729: None, +68160: None, +68161: None, +68162: None, +68163: None, +69216: None, +69217: None, +69218: None, +69219: None, +69220: None, +69221: None, +69222: None, +69223: None, +69224: None, +127232: None, +127233: None, +127234: None, +127235: None, +127236: None, +127237: None, +127238: None, +127239: None, +127240: None, +127241: None, +127242: None, } _numeric = { 188: 1.0 / 4.0, @@ -17131,31 +17131,31 @@ } _numeric_corrected = { -131073: None, -65878: None, -65879: None, -65882: None, -74757: None, -14378: None, -65885: None, -65887: None, -65890: None, -65891: None, -67672: None, -65892: None, -67674: None, -67675: None, -67676: None, -67677: None, -67678: None, -65893: None, -65880: None, -131172: None, -74769: None, -69222: None, -65881: None, -119656: None, -74775: None, +1984: None, +1985: None, +1986: None, +1987: None, +1988: None, +1989: None, +1990: None, +1991: None, +1992: None, +1993: None, +2552: None, +3046: None, +3192: None, +3193: None, +3194: None, +3195: None, +3196: None, +3197: None, +3198: None, +3440: None, +3441: None, +3442: None, +3443: None, +3444: None, +3445: None, 4240: None, 4241: None, 4242: None, @@ -17166,93 +17166,16 @@ 4247: None, 4248: None, 4249: None, -74811: None, -65904: None, -69219: None, -65905: None, -74816: None, -74780: None, -194704: None, -65906: None, -69240: None, -65883: None, -65909: None, -65910: None, -74817: None, -74785: None, -65911: None, -43216: None, -43217: None, -43218: None, -43219: None, -43220: None, -43221: None, -43222: None, -43223: None, -43224: None, -43225: None, -67862: None, -68476: None, -127240: None, -43264: None, -43265: None, -43266: None, -43267: None, -13443: None, -43269: None, -43270: None, -43271: None, -43272: None, -43273: None, -65802: None, -65803: None, -65804: None, -65805: None, -65806: None, -65807: None, -65808: None, -65809: None, -65810: None, -65811: None, -65812: None, -65813: None, -65815: None, -65816: None, -65817: None, -65818: None, -65819: None, -65820: None, -65821: None, -65822: None, -65823: None, -65824: None, -43056: None, -65826: None, -65827: None, -65828: None, -65829: None, -65830: None, -43057: None, -65832: None, -65833: None, -65834: None, -65835: None, -65836: None, -43058: None, -65838: None, -65839: None, -65840: None, -65841: None, -65842: None, -43059: None, -43060: None, -43061: None, -65856: None, -65857: None, -65858: None, -65859: None, -65860: None, -65861: None, +6128: None, +6129: None, +6130: None, +6131: None, +6132: None, +6133: None, +6134: None, +6135: None, +6136: None, +6137: None, 6470: None, 6471: None, 6472: None, @@ -17263,29 +17186,6 @@ 6477: None, 6478: None, 6479: None, -8528: None, -8529: None, -8530: None, -133507: None, -8581: None, -8582: None, -8583: None, -8584: None, -8585: None, -65930: None, -74758: None, -74818: None, -141720: None, -74820: None, -42735: None, -133532: None, -74821: None, -74822: None, -74823: None, -74824: None, -63922: None, -74825: None, -65873: None, 6608: None, 6609: None, 6610: None, @@ -17297,29 +17197,6 @@ 6616: None, 6617: None, 6618: None, -74832: None, -74835: None, -2552: None, -63997: None, -74837: None, -65894: None, -74840: None, -67673: None, -65888: None, -74761: None, -74841: None, -67679: None, -68160: None, -65896: None, -68162: None, -68163: None, -68164: None, -68165: None, -68166: None, -68167: None, -74850: None, -43600: None, -68222: None, 6784: None, 6785: None, 6786: None, @@ -17330,7 +17207,6 @@ 6791: None, 6792: None, 6793: None, -119655: None, 6800: None, 6801: None, 6802: None, @@ -17341,31 +17217,6 @@ 6807: None, 6808: None, 6809: None, -65831: None, -74849: None, -131361: None, -66514: None, -74834: None, -43472: None, -65895: None, -43475: None, -43476: None, -133885: None, -43477: None, -74836: None, -43478: None, -43479: None, -43480: None, -43481: None, -65868: None, -146203: None, -65872: None, -69246: None, -21316: None, -66378: None, -63859: None, -21324: None, -15181: None, 6992: None, 6993: None, 6994: None, @@ -17376,23 +17227,6 @@ 6999: None, 7000: None, 7001: None, -68442: None, -68444: None, -68445: None, -68446: None, -119648: None, -119649: None, -119650: None, -119651: None, -119652: None, -119653: None, -68477: None, -68478: None, -66369: None, -65907: None, -140176: None, -65908: None, -65869: None, 7088: None, 7089: None, 7090: None, @@ -17403,83 +17237,6 @@ 7095: None, 7096: None, 7097: None, -21444: None, -43268: None, -66513: None, -43473: None, -66515: None, -66516: None, -66517: None, -43474: None, -63864: None, -3046: None, -44019: None, -44020: None, -44022: None, -44023: None, -44024: None, -44025: None, -74752: None, -74753: None, -74754: None, -74755: None, -74756: None, -13317: None, -63851: None, -74759: None, -74760: None, -65843: None, -74762: None, -74763: None, -74764: None, -74765: None, -74766: None, -74767: None, -74768: None, -68440: None, -74770: None, -74771: None, -74772: None, -74773: None, -74774: None, -68441: None, -74776: None, -74777: None, -74778: None, -74779: None, -65897: None, -74781: None, -74782: None, -74783: None, -74784: None, -65912: None, -74786: None, -68443: None, -74788: None, -74789: None, -74790: None, -74791: None, -74792: None, -74793: None, -74794: None, -74795: None, -74797: None, -74798: None, -74799: None, -74800: None, -74801: None, -74804: None, -74805: None, -74806: None, -74807: None, -74808: None, -74809: None, -74810: None, -68447: None, -74812: None, -74813: None, -74814: None, -74815: None, 7232: None, 7233: None, 7234: None, @@ -17490,12 +17247,6 @@ 7239: None, 7240: None, 7241: None, -74826: None, -74827: None, -74828: None, -74829: None, -74830: None, -74831: None, 7248: None, 7249: None, 7250: None, @@ -17506,82 +17257,33 @@ 7255: None, 7256: None, 7257: None, -74842: None, -74843: None, -74844: None, -74845: None, -74846: None, -74847: None, -3192: None, -3193: None, -3194: None, -3195: None, -3196: None, -3197: None, -3198: None, -66720: None, -66721: None, -66722: None, -66723: None, -66724: None, -66725: None, -66726: None, -66727: None, -66728: None, -66729: None, -65898: None, +8528: None, +8529: None, +8530: None, +8581: None, +8582: None, +8583: None, +8584: None, +8585: None, +9471: None, +11517: None, +13317: None, +13443: None, +14378: None, +15181: None, +20118: None, +20160: None, +20200: None, +21316: None, +21324: None, +21444: None, +24186: None, +24318: None, +28422: None, 36019: None, +38433: None, +42528: None, 42529: None, -68472: None, -68473: None, -68474: None, -68475: None, -9471: None, -119658: None, -42734: None, -119659: None, -133866: None, -74787: None, -65899: None, -119660: None, -133913: None, -131298: None, -74819: None, -3440: None, -3441: None, -3442: None, -3443: None, -3444: None, -3445: None, -65876: None, -119662: None, -68161: None, -74833: None, -119663: None, -119657: None, -65886: None, -127233: None, -119664: None, -127234: None, -127239: None, -65900: None, -119665: None, -43601: None, -43602: None, -11517: None, -43603: None, -43604: None, -43605: None, -127232: None, -43606: None, -127236: None, -43607: None, -43608: None, -127235: None, -43609: None, -127237: None, -42528: None, -38433: None, 42530: None, 42531: None, 42532: None, @@ -17590,205 +17292,446 @@ 42535: None, 42536: None, 42537: None, -65799: None, -65800: None, -65801: None, -127242: None, -69216: None, -127238: None, -65862: None, -69217: None, -42728: None, -69218: None, -74796: None, -69221: None, -69226: None, -69223: None, -69228: None, -69229: None, -69230: None, -69231: None, -69232: None, -69224: None, -69234: None, -69235: None, -65901: None, -69237: None, -69238: None, -69225: None, -69241: None, -24186: None, -69243: None, -69244: None, -69245: None, -65864: None, -69227: None, -65814: None, -67863: None, -63955: None, -156269: None, -67864: None, -68479: None, -20118: None, -67865: None, -127241: None, -67866: None, -69220: None, -67867: None, -69233: None, -69236: None, -133516: None, -20160: None, -65825: None, -69239: None, -119661: None, -65867: None, -69242: None, 42726: None, 42727: None, -20200: None, +42728: None, 42729: None, 42730: None, 42731: None, 42732: None, 42733: None, -133418: None, -24318: None, -28422: None, -65902: None, -65837: None, -68221: None, -119654: None, -65884: None, -65889: None, -65903: None, +42734: None, +42735: None, +43056: None, +43057: None, +43058: None, +43059: None, +43060: None, +43061: None, +43216: None, +43217: None, +43218: None, +43219: None, +43220: None, +43221: None, +43222: None, +43223: None, +43224: None, +43225: None, +43264: None, +43265: None, +43266: None, +43267: None, +43268: None, +43269: None, +43270: None, +43271: None, +43272: None, +43273: None, +43472: None, +43473: None, +43474: None, +43475: None, +43476: None, +43477: None, +43478: None, +43479: None, +43480: None, +43481: None, +43600: None, +43601: None, +43602: None, +43603: None, +43604: None, +43605: None, +43606: None, +43607: None, +43608: None, +43609: None, 44016: None, 44017: None, -65863: None, 44018: None, -74848: None, +44019: None, +44020: None, +44021: None, +44022: None, +44023: None, +44024: None, +44025: None, +63851: None, +63859: None, +63864: None, +63922: None, +63953: None, +63955: None, +63997: None, +65799: None, +65800: None, +65801: None, +65802: None, +65803: None, +65804: None, +65805: None, +65806: None, +65807: None, +65808: None, +65809: None, +65810: None, +65811: None, +65812: None, +65813: None, +65814: None, +65815: None, +65816: None, +65817: None, +65818: None, +65819: None, +65820: None, +65821: None, +65822: None, +65823: None, +65824: None, +65825: None, +65826: None, +65827: None, +65828: None, +65829: None, +65830: None, +65831: None, +65832: None, +65833: None, +65834: None, +65835: None, +65836: None, +65837: None, +65838: None, +65839: None, +65840: None, +65841: None, +65842: None, +65843: None, +65856: None, +65857: None, +65858: None, +65859: None, +65860: None, +65861: None, +65862: None, +65863: None, +65864: None, 65865: None, 65866: None, -44021: None, -1984: None, -1985: None, -1986: None, -1987: None, -1988: None, -1989: None, -1990: None, -1991: None, -1992: None, -1993: None, -65875: None, +65867: None, +65868: None, +65869: None, 65870: None, -63953: None, 65871: None, +65872: None, +65873: None, 65874: None, -6128: None, -6129: None, -6130: None, -6131: None, -6132: None, -6133: None, -6134: None, -6135: None, -6136: None, -6137: None, +65875: None, +65876: None, 65877: None, +65878: None, +65879: None, +65880: None, +65881: None, +65882: None, +65883: None, +65884: None, +65885: None, +65886: None, +65887: None, +65888: None, +65889: None, +65890: None, +65891: None, +65892: None, +65893: None, +65894: None, +65895: None, +65896: None, +65897: None, +65898: None, +65899: None, +65900: None, +65901: None, +65902: None, +65903: None, +65904: None, +65905: None, +65906: None, +65907: None, +65908: None, +65909: None, +65910: None, +65911: None, +65912: None, +65930: None, +66369: None, +66378: None, +66513: None, +66514: None, +66515: None, +66516: None, +66517: None, +66720: None, +66721: None, +66722: None, +66723: None, +66724: None, +66725: None, +66726: None, +66727: None, +66728: None, +66729: None, +67672: None, +67673: None, +67674: None, +67675: None, +67676: None, +67677: None, +67678: None, +67679: None, +67862: None, +67863: None, +67864: None, +67865: None, +67866: None, +67867: None, +68160: None, +68161: None, +68162: None, +68163: None, +68164: None, +68165: None, +68166: None, +68167: None, +68221: None, +68222: None, +68440: None, +68441: None, +68442: None, +68443: None, +68444: None, +68445: None, +68446: None, +68447: None, +68472: None, +68473: None, +68474: None, +68475: None, +68476: None, +68477: None, +68478: None, +68479: None, +69216: None, +69217: None, +69218: None, +69219: None, +69220: None, +69221: None, +69222: None, +69223: None, +69224: None, +69225: None, +69226: None, +69227: None, +69228: None, +69229: None, +69230: None, +69231: None, +69232: None, +69233: None, +69234: None, +69235: None, +69236: None, +69237: None, +69238: None, +69239: None, +69240: None, +69241: None, +69242: None, +69243: None, +69244: None, +69245: None, +69246: None, +74752: None, +74753: None, +74754: None, +74755: None, +74756: None, +74757: None, +74758: None, +74759: None, +74760: None, +74761: None, +74762: None, +74763: None, +74764: None, +74765: None, +74766: None, +74767: None, +74768: None, +74769: None, +74770: None, +74771: None, +74772: None, +74773: None, +74774: None, +74775: None, +74776: None, +74777: None, +74778: None, +74779: None, +74780: None, +74781: None, +74782: None, +74783: None, +74784: None, +74785: None, +74786: None, +74787: None, +74788: None, +74789: None, +74790: None, +74791: None, +74792: None, +74793: None, +74794: None, +74795: None, +74796: None, +74797: None, +74798: None, +74799: None, +74800: None, +74801: None, +74804: None, +74805: None, +74806: None, +74807: None, +74808: None, +74809: None, +74810: None, +74811: None, +74812: None, +74813: None, +74814: None, +74815: None, +74816: None, +74817: None, +74818: None, +74819: None, +74820: None, +74821: None, +74822: None, +74823: None, +74824: None, +74825: None, +74826: None, +74827: None, +74828: None, +74829: None, +74830: None, +74831: None, +74832: None, +74833: None, +74834: None, +74835: None, +74836: None, +74837: None, +74840: None, +74841: None, +74842: None, +74843: None, +74844: None, +74845: None, +74846: None, +74847: None, +74848: None, +74849: None, +74850: None, +119648: None, +119649: None, +119650: None, +119651: None, +119652: None, +119653: None, +119654: None, +119655: None, +119656: None, +119657: None, +119658: None, +119659: None, +119660: None, +119661: None, +119662: None, +119663: None, +119664: None, +119665: None, +127232: None, +127233: None, +127234: None, +127235: None, +127236: None, +127237: None, +127238: None, +127239: None, +127240: None, +127241: None, +127242: None, +131073: None, +131172: None, +131298: None, +131361: None, +133418: None, +133507: None, +133516: None, +133532: None, +133866: None, +133885: None, +133913: None, +140176: None, +141720: None, +146203: None, +156269: None, +194704: None, } -def decimal(code): - try: - return _decimal[code] - except KeyError: - if base_mod is not None and code not in _decimal_corrected: - return base_mod._decimal[code] - else: - raise - -def digit(code): - try: - return _digit[code] - except KeyError: - if base_mod is not None and code not in _digit_corrected: - return base_mod._digit[code] - else: - raise - -def numeric(code): - try: - return _numeric[code] - except KeyError: - if base_mod is not None and code not in _numeric_corrected: - return base_mod._numeric[code] - else: - raise - -_toupper = { -1010: 931, -} - -_toupper_corrected = { -11525: None, -11443: None, -11457: None, -11533: None, -11321: None, -11322: None, -11527: None, -42595: None, -11323: None, -11539: None, -42625: None, -42827: None, -11529: None, -42831: None, -11325: None, -42627: None, -11326: None, -11315: None, -11531: None, -11463: None, -11327: None, -7545: None, -42603: None, -11425: None, -11535: None, -11316: None, -11317: None, -11318: None, -66638: None, -8526: None, -11332: None, -66639: None, -11537: None, -42605: None, +def decimal(code): + try: + return _decimal[code] + except KeyError: + if base_mod is not None and code not in _decimal_corrected: + return base_mod._decimal[code] + else: + raise + +def digit(code): + try: + return _digit[code] + except KeyError: + if base_mod is not None and code not in _digit_corrected: + return base_mod._digit[code] + else: + raise + +def numeric(code): + try: + return _numeric[code] + except KeyError: + if base_mod is not None and code not in _numeric_corrected: + return base_mod._numeric[code] + else: + raise + +_toupper = { +1010: 931, +} + +_toupper_corrected = { 384: None, -11328: None, -8580: None, -11329: None, -11330: None, -11331: None, 410: None, -11334: None, -11333: None, -11335: None, -11471: None, -11336: None, -11337: None, -11339: None, -11341: None, -11342: None, -11343: None, -11541: None, -11473: None, -11338: None, -11543: None, -11475: None, -11358: None, -11324: None, -42641: None, 572: None, 575: None, 576: None, @@ -17801,247 +17744,314 @@ 592: None, 593: None, 594: None, -11447: None, -11365: None, -11477: None, 619: None, 625: None, -42643: None, 637: None, 649: None, 652: None, -11547: None, -11479: None, -42849: None, -11352: None, -11379: None, -11344: None, -42787: None, -11549: None, -11313: None, -42851: None, -42789: None, -11557: None, -11345: None, -42791: None, -42579: None, -42795: None, -11489: None, -11346: None, -11393: None, -11551: None, -11483: None, -42853: None, -11421: None, -11347: None, -11312: None, -42803: None, -11353: None, -42879: None, -42805: None, -11348: None, -11403: None, -42807: None, -11553: None, -11405: None, -42601: None, -11485: None, -42855: None, -11407: None, -11349: None, -11409: None, -42583: None, 881: None, 883: None, 887: None, 891: None, 892: None, 893: None, -11413: None, -42817: None, -11555: None, -11415: None, -11487: None, -42819: None, -11417: None, -11351: None, -11340: None, -42821: None, -11419: None, -42585: None, -11427: None, 983: None, -42833: None, -42587: None, -42835: None, -11354: None, -11433: None, 1016: None, 1019: None, -42837: None, -11435: None, -11491: None, -42839: None, -11423: None, -11437: None, +1231: None, +1271: None, +1275: None, +1277: None, +1279: None, +1297: None, +1299: None, +1301: None, +1303: None, +1305: None, +1307: None, +1309: None, +1311: None, +1313: None, +1315: None, +1317: None, +7545: None, +7549: None, +7931: None, +7933: None, +7935: None, +8526: None, +8580: None, +11312: None, +11313: None, +11314: None, +11315: None, +11316: None, +11317: None, +11318: None, +11319: None, +11320: None, +11321: None, +11322: None, +11323: None, +11324: None, +11325: None, +11326: None, +11327: None, +11328: None, +11329: None, +11330: None, +11331: None, +11332: None, +11333: None, +11334: None, +11335: None, +11336: None, +11337: None, +11338: None, +11339: None, +11340: None, +11341: None, +11342: None, +11343: None, +11344: None, +11345: None, +11346: None, +11347: None, +11348: None, +11349: None, +11350: None, +11351: None, +11352: None, +11353: None, +11354: None, 11355: None, -42841: None, -11439: None, -42589: None, -42843: None, -11441: None, +11356: None, +11357: None, +11358: None, +11361: None, +11365: None, 11366: None, 11368: None, 11370: None, 11372: None, +11379: None, 11382: None, +11393: None, 11395: None, 11397: None, 11399: None, 11401: None, +11403: None, +11405: None, +11407: None, +11409: None, +11411: None, +11413: None, +11415: None, +11417: None, +11419: None, +11421: None, +11423: None, +11425: None, +11427: None, 11429: None, +11431: None, +11433: None, +11435: None, +11437: None, +11439: None, +11441: None, +11443: None, +11445: None, +11447: None, +11449: None, +11451: None, +11453: None, +11455: None, +11457: None, 11459: None, 11461: None, -11361: None, +11463: None, 11465: None, 11467: None, 11469: None, -1231: None, +11471: None, +11473: None, +11475: None, +11477: None, +11479: None, +11481: None, +11483: None, +11485: None, +11487: None, +11489: None, +11491: None, 11500: None, 11502: None, -11431: None, -1271: None, -1275: None, -11356: None, -1277: None, -1279: None, 11520: None, +11521: None, 11522: None, +11523: None, 11524: None, +11525: None, 11526: None, +11527: None, 11528: None, +11529: None, 11530: None, +11531: None, 11532: None, +11533: None, 11534: None, +11535: None, 11536: None, -1297: None, +11537: None, 11538: None, -1299: None, +11539: None, 11540: None, -1301: None, +11541: None, 11542: None, -1303: None, +11543: None, 11544: None, -1305: None, +11545: None, 11546: None, -1307: None, +11547: None, 11548: None, -1309: None, +11549: None, 11550: None, -1311: None, +11551: None, 11552: None, -1313: None, +11553: None, 11554: None, -1315: None, +11555: None, 11556: None, -1317: None, -42887: None, -11411: None, -42892: None, -42599: None, +11557: None, 42561: None, -42874: None, -11357: None, 42563: None, -7549: None, 42565: None, 42567: None, 42569: None, 42571: None, -11314: None, -42876: None, 42573: None, 42575: None, -11453: None, 42577: None, -42591: None, -11445: None, +42579: None, 42581: None, -11481: None, -42881: None, +42583: None, +42585: None, +42587: None, +42589: None, +42591: None, +42595: None, +42597: None, +42599: None, +42601: None, +42603: None, +42605: None, +42625: None, +42627: None, +42629: None, +42631: None, 42633: None, -11545: None, -42883: None, 42635: None, -42885: None, -7931: None, -7933: None, -7935: None, -11449: None, -42629: None, +42637: None, +42639: None, +42641: None, +42643: None, +42645: None, +42647: None, +42787: None, +42789: None, +42791: None, 42793: None, -42631: None, +42795: None, 42797: None, 42799: None, +42803: None, +42805: None, +42807: None, 42809: None, 42811: None, 42813: None, 42815: None, +42817: None, +42819: None, +42821: None, 42823: None, 42825: None, -42597: None, +42827: None, 42829: None, -42637: None, -11350: None, -42639: None, +42831: None, +42833: None, +42835: None, +42837: None, +42839: None, +42841: None, +42843: None, 42845: None, 42847: None, +42849: None, +42851: None, +42853: None, +42855: None, 42857: None, 42859: None, 42861: None, 42863: None, -42645: None, -11451: None, -42647: None, -11521: None, -11523: None, -11455: None, -11319: None, -11320: None, +42874: None, +42876: None, +42879: None, +42881: None, +42883: None, +42885: None, +42887: None, +42892: None, +66638: None, +66639: None, } _tolower = { } _tolower_corrected = { -11264: None, -11265: None, -11293: None, -11266: None, -11267: None, -11268: None, -11269: None, -11270: None, -11458: None, -11271: None, -11272: None, -11273: None, -11274: None, -42636: None, -11279: None, -11280: None, -11460: None, -11281: None, -11282: None, -11283: None, -42626: None, -11284: None, -11285: None, -11286: None, -11287: None, -11288: None, -11289: None, -11290: None, -11294: None, +570: None, +571: None, +573: None, +574: None, +577: None, +579: None, +580: None, +581: None, +582: None, +584: None, +586: None, +588: None, +590: None, +880: None, +882: None, +886: None, +975: None, +1015: None, +1017: None, +1018: None, +1021: None, +1022: None, +1023: None, +1216: None, +1270: None, +1274: None, +1276: None, +1278: None, +1296: None, +1298: None, +1300: None, +1302: None, +1304: None, +1306: None, +1308: None, +1310: None, +1312: None, +1314: None, +1316: None, 4256: None, 4257: None, 4258: None, @@ -18074,217 +18084,205 @@ 4285: None, 4286: None, 4287: None, -11301: None, -11302: None, -11303: None, -11304: None, -11305: None, -11306: None, -11307: None, -11308: None, -11309: None, -11310: None, -11486: None, -11468: None, +4288: None, +4289: None, +4290: None, +4291: None, +4292: None, +4293: None, +7838: None, +7930: None, +7932: None, +7934: None, +8498: None, 8579: None, -42628: None, -570: None, -571: None, -573: None, -574: None, -577: None, -579: None, -580: None, -581: None, -582: None, -584: None, -586: None, -588: None, +11264: None, +11265: None, +11266: None, +11267: None, +11268: None, +11269: None, +11270: None, +11271: None, +11272: None, +11273: None, +11274: None, +11275: None, +11276: None, +11277: None, +11278: None, +11279: None, +11280: None, +11281: None, +11282: None, +11283: None, +11284: None, +11285: None, +11286: None, +11287: None, +11288: None, +11289: None, +11290: None, 11291: None, -590: None, -42642: None, +11292: None, +11293: None, +11294: None, +11295: None, +11296: None, +11297: None, +11298: None, +11299: None, +11300: None, +11301: None, +11302: None, +11303: None, +11304: None, +11305: None, +11306: None, +11307: None, +11308: None, +11309: None, +11310: None, +11360: None, +11362: None, +11363: None, 11364: None, -11297: None, +11367: None, 11369: None, +11371: None, +11373: None, 11374: None, +11375: None, 11376: None, 11378: None, -11275: None, 11381: None, -11276: None, -11277: None, -42792: None, 11390: None, -42794: None, +11391: None, 11392: None, -42796: None, -11278: None, -42798: None, -11416: None, +11394: None, +11396: None, +11398: None, +11400: None, 11402: None, 11404: None, -42582: None, -42808: None, 11406: None, -42810: None, 11408: None, -42812: None, 11410: None, -880: None, -882: None, -42814: None, -886: None, 11412: None, 11414: None, -11488: None, -42824: None, +11416: None, +11418: None, 11420: None, 11422: None, -42826: None, 11424: None, -42560: None, -42828: None, -42634: None, 11426: None, -975: None, 11428: None, 11430: None, -11490: None, 11432: None, -1015: None, -1017: None, -1018: None, -1021: None, -1022: None, -1023: None, +11434: None, +11436: None, +11438: None, +11440: None, +11442: None, 11444: None, -11363: None, -11367: None, -11371: None, -11373: None, -11375: None, -42856: None, -42858: None, -11391: None, -4288: None, -11394: None, -11396: None, -11398: None, -4289: None, -11400: None, -42860: None, -4290: None, -4291: None, -42862: None, -4292: None, -66598: None, -4293: None, +11446: None, +11448: None, +11450: None, +11452: None, +11454: None, +11456: None, +11458: None, +11460: None, 11462: None, -11360: None, 11464: None, -1216: None, +11466: None, +11468: None, 11470: None, -42873: None, 11472: None, -11362: None, -8498: None, 11474: None, -1270: None, 11476: None, -1274: None, -1276: None, -1278: None, 11478: None, -1296: None, 11480: None, -1298: None, -1300: None, -1302: None, -1304: None, -1306: None, -1308: None, 11482: None, -1310: None, -1312: None, -1314: None, -1316: None, -11296: None, 11484: None, -11434: None, -11298: None, -11299: None, +11486: None, +11488: None, +11490: None, 11499: None, -11466: None, -42562: None, 11501: None, -11436: None, +42560: None, +42562: None, 42564: None, -11300: None, 42566: None, 42568: None, 42570: None, 42572: None, -11438: None, 42574: None, 42576: None, -42891: None, 42578: None, 42580: None, -11295: None, -11440: None, +42582: None, 42584: None, 42586: None, 42588: None, 42590: None, -11442: None, -11456: None, 42594: None, 42596: None, 42598: None, 42600: None, -11454: None, 42602: None, 42604: None, -42646: None, -7838: None, -11446: None, -11418: None, -7930: None, -11448: None, -7932: None, -7934: None, 42624: None, +42626: None, +42628: None, +42630: None, +42632: None, +42634: None, +42636: None, +42638: None, +42640: None, +42642: None, +42644: None, +42646: None, 42786: None, 42788: None, -42630: None, 42790: None, -42632: None, +42792: None, +42794: None, +42796: None, +42798: None, 42802: None, 42804: None, 42806: None, -11450: None, +42808: None, +42810: None, +42812: None, +42814: None, 42816: None, 42818: None, 42820: None, 42822: None, +42824: None, +42826: None, +42828: None, 42830: None, 42832: None, 42834: None, 42836: None, -42638: None, 42838: None, 42840: None, 42842: None, 42844: None, 42846: None, 42848: None, -42640: None, 42850: None, 42852: None, 42854: None, -11292: None, -11452: None, -42644: None, +42856: None, +42858: None, +42860: None, +42862: None, +42873: None, 42875: None, 42877: None, 42878: None, @@ -18292,6 +18290,8 @@ 42882: None, 42884: None, 42886: None, +42891: None, +66598: None, 66599: None, } _totitle = { @@ -18299,288 +18299,288 @@ } _totitle_corrected = { -11525: None, -11443: None, -11457: None, -11533: None, -11321: None, -11322: None, -11527: None, -42595: None, -11323: None, -11539: None, -42625: None, -42827: None, -11529: None, -42831: None, -11325: None, -42627: None, -11326: None, -11315: None, -11531: None, -11463: None, -11327: None, +384: None, +410: None, +453: None, +456: None, +459: None, +498: None, +572: None, +575: None, +576: None, +578: None, +583: None, +585: None, +587: None, +589: None, +591: None, +592: None, +593: None, +594: None, +619: None, +625: None, +637: None, +649: None, +652: None, +881: None, +883: None, +887: None, +891: None, +892: None, +893: None, +983: None, +1016: None, +1019: None, +1231: None, +1271: None, +1275: None, +1277: None, +1279: None, +1297: None, +1299: None, +1301: None, +1303: None, +1305: None, +1307: None, +1309: None, +1311: None, +1313: None, +1315: None, +1317: None, 7545: None, -42603: None, -11425: None, -11535: None, +7549: None, +7931: None, +7933: None, +7935: None, +8526: None, +8580: None, +11312: None, +11313: None, +11314: None, +11315: None, 11316: None, 11317: None, 11318: None, -66638: None, -8526: None, -11332: None, -66639: None, -11537: None, -42605: None, -384: None, +11319: None, +11320: None, +11321: None, +11322: None, +11323: None, +11324: None, +11325: None, +11326: None, +11327: None, 11328: None, -8580: None, 11329: None, 11330: None, 11331: None, -410: None, -11334: None, +11332: None, 11333: None, +11334: None, 11335: None, -11471: None, 11336: None, 11337: None, +11338: None, 11339: None, -453: None, -456: None, -459: None, +11340: None, 11341: None, 11342: None, 11343: None, -11541: None, -11473: None, -498: None, -11338: None, -11543: None, -11475: None, -11358: None, -11324: None, -42641: None, -572: None, -575: None, -576: None, -578: None, -583: None, -585: None, -587: None, -589: None, -591: None, -592: None, -593: None, -594: None, -11447: None, -11365: None, -11477: None, -619: None, -625: None, -42643: None, -637: None, -649: None, -652: None, -11547: None, -11479: None, -42849: None, -11352: None, -11379: None, 11344: None, -42787: None, -11549: None, -11313: None, -42851: None, -42789: None, -11557: None, 11345: None, -42791: None, -42579: None, -42795: None, -11489: None, 11346: None, -11393: None, -11551: None, -11483: None, -42853: None, -11421: None, 11347: None, -11312: None, -42803: None, -11353: None, -42879: None, -42805: None, 11348: None, -11403: None, -42807: None, -11553: None, -11405: None, -42601: None, -11485: None, -42855: None, -11407: None, 11349: None, -11409: None, -42583: None, -881: None, -883: None, -887: None, -891: None, -892: None, -893: None, -11413: None, -42817: None, -11555: None, -11415: None, -11487: None, -42819: None, -11417: None, +11350: None, 11351: None, -11340: None, -42821: None, -11419: None, -42585: None, -11427: None, -983: None, -42833: None, -42587: None, -42835: None, +11352: None, +11353: None, 11354: None, -11433: None, -1016: None, -1019: None, -42837: None, -11435: None, -11491: None, -42839: None, -11423: None, -11437: None, 11355: None, -42841: None, -11439: None, -42589: None, -42843: None, -11441: None, +11356: None, +11357: None, +11358: None, +11361: None, +11365: None, 11366: None, 11368: None, 11370: None, 11372: None, +11379: None, 11382: None, +11393: None, 11395: None, 11397: None, 11399: None, 11401: None, +11403: None, +11405: None, +11407: None, +11409: None, +11411: None, +11413: None, +11415: None, +11417: None, +11419: None, +11421: None, +11423: None, +11425: None, +11427: None, 11429: None, +11431: None, +11433: None, +11435: None, +11437: None, +11439: None, +11441: None, +11443: None, +11445: None, +11447: None, +11449: None, +11451: None, +11453: None, +11455: None, +11457: None, 11459: None, 11461: None, -11361: None, +11463: None, 11465: None, 11467: None, 11469: None, -1231: None, +11471: None, +11473: None, +11475: None, +11477: None, +11479: None, +11481: None, +11483: None, +11485: None, +11487: None, +11489: None, +11491: None, 11500: None, 11502: None, -11431: None, -1271: None, -1275: None, -11356: None, -1277: None, -1279: None, 11520: None, +11521: None, 11522: None, +11523: None, 11524: None, +11525: None, 11526: None, +11527: None, 11528: None, +11529: None, 11530: None, +11531: None, 11532: None, +11533: None, 11534: None, +11535: None, 11536: None, -1297: None, +11537: None, 11538: None, -1299: None, +11539: None, 11540: None, -1301: None, +11541: None, 11542: None, -1303: None, +11543: None, 11544: None, -1305: None, +11545: None, 11546: None, -1307: None, +11547: None, 11548: None, -1309: None, +11549: None, 11550: None, -1311: None, +11551: None, 11552: None, -1313: None, +11553: None, 11554: None, -1315: None, +11555: None, 11556: None, -1317: None, -42887: None, -11411: None, -42892: None, -42599: None, +11557: None, 42561: None, -42874: None, -11357: None, 42563: None, -7549: None, 42565: None, 42567: None, 42569: None, 42571: None, -11314: None, -42876: None, 42573: None, 42575: None, -11453: None, 42577: None, -42591: None, -11445: None, +42579: None, 42581: None, -11481: None, -42881: None, +42583: None, +42585: None, +42587: None, +42589: None, +42591: None, +42595: None, +42597: None, +42599: None, +42601: None, +42603: None, +42605: None, +42625: None, +42627: None, +42629: None, +42631: None, 42633: None, -11545: None, -42883: None, 42635: None, -42885: None, -7931: None, -7933: None, -7935: None, -11449: None, -42629: None, +42637: None, +42639: None, +42641: None, +42643: None, +42645: None, +42647: None, +42787: None, +42789: None, +42791: None, 42793: None, -42631: None, +42795: None, 42797: None, 42799: None, +42803: None, +42805: None, +42807: None, 42809: None, 42811: None, 42813: None, 42815: None, +42817: None, +42819: None, +42821: None, 42823: None, 42825: None, -42597: None, +42827: None, 42829: None, -42637: None, -11350: None, -42639: None, +42831: None, +42833: None, +42835: None, +42837: None, +42839: None, +42841: None, +42843: None, 42845: None, 42847: None, +42849: None, +42851: None, +42853: None, +42855: None, 42857: None, 42859: None, 42861: None, 42863: None, -42645: None, -11451: None, -42647: None, -11521: None, -11523: None, -11455: None, -11319: None, -11320: None, +42874: None, +42876: None, +42879: None, +42881: None, +42883: None, +42885: None, +42887: None, +42892: None, +66638: None, +66639: None, } def toupper(code): @@ -18620,9 +18620,6 @@ _raw_decomposition_corrected = { 1017: None, -69786: None, -69788: None, -69803: None, 4348: None, 6918: None, 6920: None, @@ -18744,7 +18741,6 @@ 8529: None, 8530: None, 8585: None, -42864: None, 11388: None, 11389: None, 11631: None, @@ -18769,108 +18765,7 @@ 13278: None, 13279: None, 13311: None, -120001: None, -120484: None, -120485: None, -120778: None, -120779: None, -127232: None, -127233: None, -127234: None, -127235: None, -127236: None, -127237: None, -127238: None, -127239: None, -127240: None, -127241: None, -127242: None, -127248: None, -127249: None, -127250: None, -127251: None, -127252: None, -127253: None, -127254: None, -127255: None, -127256: None, -127257: None, -127258: None, -127259: None, -127260: None, -127261: None, -127262: None, -127263: None, -127264: None, -127265: None, -127266: None, -127267: None, -127268: None, -127269: None, -127270: None, -127271: None, -127272: None, -127273: None, -127274: None, -127275: None, -127276: None, -127277: None, -127278: None, -127281: None, -127293: None, -127295: None, -127298: None, -127302: None, -127306: None, -127307: None, -127308: None, -127309: None, -127310: None, -127376: None, -127488: None, -127504: None, -127505: None, -127506: None, -127507: None, -127508: None, -127509: None, -127510: None, -127511: None, -127512: None, -127513: None, -127514: None, -127515: None, -127516: None, -127517: None, -127518: None, -127519: None, -127520: None, -127521: None, -127522: None, -127523: None, -127524: None, -127525: None, -127526: None, -127527: None, -127528: None, -127529: None, -127530: None, -127531: None, -127532: None, -127533: None, -127534: None, -127535: None, -127536: None, -127537: None, -127552: None, -127553: None, -127554: None, -127555: None, -127556: None, -127557: None, -127558: None, -127559: None, -127560: None, +42864: None, 64107: None, 64108: None, 64109: None, @@ -18992,6 +18887,111 @@ 65049: None, 65095: None, 65096: None, +69786: None, +69788: None, +69803: None, +120001: None, +120484: None, +120485: None, +120778: None, +120779: None, +127232: None, +127233: None, +127234: None, +127235: None, +127236: None, +127237: None, +127238: None, +127239: None, +127240: None, +127241: None, +127242: None, +127248: None, +127249: None, +127250: None, +127251: None, +127252: None, +127253: None, +127254: None, +127255: None, +127256: None, +127257: None, +127258: None, +127259: None, +127260: None, +127261: None, +127262: None, +127263: None, +127264: None, +127265: None, +127266: None, +127267: None, +127268: None, +127269: None, +127270: None, +127271: None, +127272: None, +127273: None, +127274: None, +127275: None, +127276: None, +127277: None, +127278: None, +127281: None, +127293: None, +127295: None, +127298: None, +127302: None, +127306: None, +127307: None, +127308: None, +127309: None, +127310: None, +127376: None, +127488: None, +127504: None, +127505: None, +127506: None, +127507: None, +127508: None, +127509: None, +127510: None, +127511: None, +127512: None, +127513: None, +127514: None, +127515: None, +127516: None, +127517: None, +127518: None, +127519: None, +127520: None, +127521: None, +127522: None, +127523: None, +127524: None, +127525: None, +127526: None, +127527: None, +127528: None, +127529: None, +127530: None, +127531: None, +127532: None, +127533: None, +127534: None, +127535: None, +127536: None, +127537: None, +127552: None, +127553: None, +127554: None, +127555: None, +127556: None, +127557: None, +127558: None, +127559: None, +127560: None, } def decomposition(code): @@ -19934,12 +19934,17 @@ } _canon_decomposition_corrected = { -64195: None, -69786: None, -69788: None, -69803: None, -64201: None, -64139: None, +6918: None, +6920: None, +6922: None, +6924: None, +6926: None, +6930: None, +6971: None, +6973: None, +6976: None, +6977: None, +6979: None, 64107: None, 64108: None, 64109: None, @@ -19970,6 +19975,7 @@ 64136: None, 64137: None, 64138: None, +64139: None, 64140: None, 64141: None, 64142: None, @@ -20025,38 +20031,32 @@ 64192: None, 64193: None, 64194: None, +64195: None, 64196: None, 64197: None, 64198: None, 64199: None, 64200: None, +64201: None, 64202: None, 64203: None, 64204: None, 64205: None, 64206: None, +64207: None, 64208: None, 64209: None, 64210: None, 64211: None, 64212: None, +64213: None, 64214: None, 64215: None, 64216: None, 64217: None, -6918: None, -6920: None, -6922: None, -6924: None, -6926: None, -6930: None, -6971: None, -6973: None, -6976: None, -6977: None, -6979: None, -64207: None, -64213: None, +69786: None, +69788: None, +69803: None, } _compat_decomposition = { 194664: [136042], @@ -20068,9 +20068,6 @@ _compat_decomposition_corrected = { 1017: None, -69786: None, -69788: None, -69803: None, 4348: None, 6918: None, 6920: None, @@ -20192,7 +20189,6 @@ 8529: None, 8530: None, 8585: None, -42864: None, 11388: None, 11389: None, 11631: None, @@ -20217,108 +20213,7 @@ 13278: None, 13279: None, 13311: None, -120001: None, -120484: None, -120485: None, -120778: None, -120779: None, -127232: None, -127233: None, -127234: None, -127235: None, -127236: None, -127237: None, -127238: None, -127239: None, -127240: None, -127241: None, -127242: None, -127248: None, -127249: None, -127250: None, -127251: None, -127252: None, -127253: None, -127254: None, -127255: None, -127256: None, -127257: None, -127258: None, -127259: None, -127260: None, -127261: None, -127262: None, -127263: None, -127264: None, -127265: None, -127266: None, -127267: None, -127268: None, -127269: None, -127270: None, -127271: None, -127272: None, -127273: None, -127274: None, -127275: None, -127276: None, -127277: None, -127278: None, -127281: None, -127293: None, -127295: None, -127298: None, -127302: None, -127306: None, -127307: None, -127308: None, -127309: None, -127310: None, -127376: None, -127488: None, -127504: None, -127505: None, -127506: None, -127507: None, -127508: None, -127509: None, -127510: None, -127511: None, -127512: None, -127513: None, -127514: None, -127515: None, -127516: None, -127517: None, -127518: None, -127519: None, -127520: None, -127521: None, -127522: None, -127523: None, -127524: None, -127525: None, -127526: None, -127527: None, -127528: None, -127529: None, -127530: None, -127531: None, -127532: None, -127533: None, -127534: None, -127535: None, -127536: None, -127537: None, -127552: None, -127553: None, -127554: None, -127555: None, -127556: None, -127557: None, -127558: None, -127559: None, -127560: None, +42864: None, 64107: None, 64108: None, 64109: None, @@ -20440,6 +20335,111 @@ 65049: None, 65095: None, 65096: None, +69786: None, +69788: None, +69803: None, +120001: None, +120484: None, +120485: None, +120778: None, +120779: None, +127232: None, +127233: None, +127234: None, +127235: None, +127236: None, +127237: None, +127238: None, +127239: None, +127240: None, +127241: None, +127242: None, +127248: None, +127249: None, +127250: None, +127251: None, +127252: None, +127253: None, +127254: None, +127255: None, +127256: None, +127257: None, +127258: None, +127259: None, +127260: None, +127261: None, +127262: None, +127263: None, +127264: None, +127265: None, +127266: None, +127267: None, +127268: None, +127269: None, +127270: None, +127271: None, +127272: None, +127273: None, +127274: None, +127275: None, +127276: None, +127277: None, +127278: None, +127281: None, +127293: None, +127295: None, +127298: None, +127302: None, +127306: None, +127307: None, +127308: None, +127309: None, +127310: None, +127376: None, +127488: None, +127504: None, +127505: None, +127506: None, +127507: None, +127508: None, +127509: None, +127510: None, +127511: None, +127512: None, +127513: None, +127514: None, +127515: None, +127516: None, +127517: None, +127518: None, +127519: None, +127520: None, +127521: None, +127522: None, +127523: None, +127524: None, +127525: None, +127526: None, +127527: None, +127528: None, +127529: None, +127530: None, +127531: None, +127532: None, +127533: None, +127534: None, +127535: None, +127536: None, +127537: None, +127552: None, +127553: None, +127554: None, +127555: None, +127556: None, +127557: None, +127558: None, +127559: None, +127560: None, } def canon_decomposition(code): From arigo at codespeak.net Sun Nov 7 14:16:00 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 14:16:00 +0100 (CET) Subject: [pypy-svn] r78811 - in pypy/branch/fast-forward/pypy/rpython: . test Message-ID: <20101107131600.B7D1C282BDC@codespeak.net> Author: arigo Date: Sun Nov 7 14:15:58 2010 New Revision: 78811 Modified: pypy/branch/fast-forward/pypy/rpython/rint.py pypy/branch/fast-forward/pypy/rpython/test/test_rint.py Log: Test and improvement of hash() of an r_longlong. Modified: pypy/branch/fast-forward/pypy/rpython/rint.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/rint.py (original) +++ pypy/branch/fast-forward/pypy/rpython/rint.py Sun Nov 7 14:15:58 2010 @@ -291,6 +291,9 @@ return None def get_ll_hash_function(self): + if (sys.maxint == 2147483647 and + self.lowleveltype in (SignedLongLong, UnsignedLongLong)): + return ll_hash_long_long return ll_hash_int get_ll_fasthash_function = get_ll_hash_function @@ -411,6 +414,9 @@ def ll_hash_int(n): return intmask(n) +def ll_hash_long_long(n): + return intmask(intmask(n) + 9 * intmask(n >> 32)) + def ll_check_chr(n): if 0 <= n <= 255: return Modified: pypy/branch/fast-forward/pypy/rpython/test/test_rint.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/test/test_rint.py (original) +++ pypy/branch/fast-forward/pypy/rpython/test/test_rint.py Sun Nov 7 14:15:58 2010 @@ -5,6 +5,7 @@ from pypy.rpython.test import snippet from pypy.rlib.rarithmetic import r_int, r_uint, r_longlong, r_ulonglong from pypy.rlib.rarithmetic import ovfcheck +from pypy.rlib import objectmodel from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin if r_longlong is not r_int: @@ -353,6 +354,18 @@ res = self.interpret(f, [3]) assert res == 3 + def test_hash(self): + def f(x): + return objectmodel.compute_hash(x) + res = self.interpret(f, [123456789]) + assert res == 123456789 + res = self.interpret(f, [int64(123456789012345678)]) + if sys.maxint == 2147483647: + # check the way we compute such a hash so far + assert res == -1506741426 + 9 * 28744523 + else: + assert res == 123456789012345678 + class TestLLtype(BaseTestRint, LLRtypeMixin): pass From arigo at codespeak.net Sun Nov 7 14:17:28 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 14:17:28 +0100 (CET) Subject: [pypy-svn] r78812 - pypy/branch/fast-forward/lib-python Message-ID: <20101107131728.575B8282BDC@codespeak.net> Author: arigo Date: Sun Nov 7 14:17:26 2010 New Revision: 78812 Modified: pypy/branch/fast-forward/lib-python/TODO Log: Done (r78810). Modified: pypy/branch/fast-forward/lib-python/TODO ============================================================================== --- pypy/branch/fast-forward/lib-python/TODO (original) +++ pypy/branch/fast-forward/lib-python/TODO Sun Nov 7 14:17:26 2010 @@ -62,7 +62,7 @@ - Change the way the unicodedata module stores its database: in unicodedb_5_2_0.py, the "_composition" map contains values > 2**32 which - causes the translation to fail. + causes the translation to fail. DONE - "Shorter float representation": copy dtoa.c from CPython and use it to format/parse floats. Enable this with a translation option. From arigo at codespeak.net Sun Nov 7 15:15:59 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 15:15:59 +0100 (CET) Subject: [pypy-svn] r78813 - in pypy/branch/fast-forward: lib-python pypy/module/signal pypy/module/signal/test pypy/translator/c/src Message-ID: <20101107141559.A2E52282BDC@codespeak.net> Author: arigo Date: Sun Nov 7 15:15:57 2010 New Revision: 78813 Modified: pypy/branch/fast-forward/lib-python/TODO pypy/branch/fast-forward/pypy/module/signal/__init__.py pypy/branch/fast-forward/pypy/module/signal/interp_signal.py pypy/branch/fast-forward/pypy/module/signal/test/test_signal.py pypy/branch/fast-forward/pypy/translator/c/src/signals.h Log: signal.set_wakeup_fd(). Modified: pypy/branch/fast-forward/lib-python/TODO ============================================================================== --- pypy/branch/fast-forward/lib-python/TODO (original) +++ pypy/branch/fast-forward/lib-python/TODO Sun Nov 7 15:15:57 2010 @@ -44,7 +44,7 @@ - missing builtin: memoryview -- signal.set_wakeup_fd() +- signal.set_wakeup_fd() DONE - add 'unicode' in ObjSpace.MethodTable + probably a default implementation that falls back to space.str(). Modified: pypy/branch/fast-forward/pypy/module/signal/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/signal/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/signal/__init__.py Sun Nov 7 15:15:57 2010 @@ -6,6 +6,7 @@ interpleveldefs = { 'signal': 'interp_signal.signal', 'getsignal': 'interp_signal.getsignal', + 'set_wakeup_fd': 'interp_signal.set_wakeup_fd', 'NSIG': 'space.wrap(interp_signal.NSIG)', 'SIG_DFL': 'space.wrap(interp_signal.SIG_DFL)', 'SIG_IGN': 'space.wrap(interp_signal.SIG_IGN)', Modified: pypy/branch/fast-forward/pypy/module/signal/interp_signal.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/signal/interp_signal.py (original) +++ pypy/branch/fast-forward/pypy/module/signal/interp_signal.py Sun Nov 7 15:15:57 2010 @@ -7,6 +7,7 @@ import py from pypy.tool import autopath from pypy.rlib import jit +from pypy.rlib.rarithmetic import intmask def setup(): for key, value in cpy_signal.__dict__.items(): @@ -25,6 +26,7 @@ include_dirs = [str(py.path.local(autopath.pypydir).join('translator', 'c'))], export_symbols = ['pypysig_poll', 'pypysig_default', 'pypysig_ignore', 'pypysig_setflag', + 'pypysig_set_wakeup_fd', 'pypysig_getaddr_occurred'], ) @@ -34,6 +36,7 @@ pypysig_ignore = external('pypysig_ignore', [rffi.INT], lltype.Void) pypysig_default = external('pypysig_default', [rffi.INT], lltype.Void) pypysig_setflag = external('pypysig_setflag', [rffi.INT], lltype.Void) +pypysig_set_wakeup_fd = external('pypysig_set_wakeup_fd', [rffi.INT], rffi.INT) pypysig_poll = external('pypysig_poll', [], rffi.INT, threadsafe=False) # don't bother releasing the GIL around a call to pypysig_poll: it's # pointless and a performance issue @@ -213,3 +216,21 @@ action.handlers_w[signum] = w_handler return old_handler signal.unwrap_spec = [ObjSpace, int, W_Root] + +def set_wakeup_fd(space, fd): + """Sets the fd to be written to (with '\0') when a signal + comes in. Returns the old fd. A library can use this to + wakeup select or poll. The previous fd is returned. + + The fd must be non-blocking. + """ + if space.config.objspace.usemodules.thread: + main_ec = space.threadlocals.getmainthreadvalue() + ec = space.getexecutioncontext() + if ec is not main_ec: + raise OperationError( + space.w_ValueError, + space.wrap("set_wakeup_fd only works in main thread")) + old_fd = pypysig_set_wakeup_fd(fd) + return space.wrap(intmask(old_fd)) +set_wakeup_fd.unwrap_spec = [ObjSpace, int] Modified: pypy/branch/fast-forward/pypy/module/signal/test/test_signal.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/signal/test/test_signal.py (original) +++ pypy/branch/fast-forward/pypy/module/signal/test/test_signal.py Sun Nov 7 15:15:57 2010 @@ -136,6 +136,40 @@ finally: signal(SIGALRM, SIG_DFL) + def test_set_wakeup_fd(self): + import signal, posix, fcntl + def myhandler(signum, frame): + pass + signal.signal(signal.SIGUSR1, myhandler) + # + def cannot_read(): + try: + posix.read(fd_read, 1) + except OSError: + pass + else: + raise AssertionError, "os.read(fd_read, 1) succeeded?" + # + fd_read, fd_write = posix.pipe() + flags = fcntl.fcntl(fd_write, fcntl.F_GETFL, 0) + flags = flags | posix.O_NONBLOCK + fcntl.fcntl(fd_write, fcntl.F_SETFL, flags) + flags = fcntl.fcntl(fd_read, fcntl.F_GETFL, 0) + flags = flags | posix.O_NONBLOCK + fcntl.fcntl(fd_read, fcntl.F_SETFL, flags) + # + old_wakeup = signal.set_wakeup_fd(fd_write) + try: + cannot_read() + posix.kill(posix.getpid(), signal.SIGUSR1) + res = posix.read(fd_read, 1) + assert res == '\x00' + cannot_read() + finally: + old_wakeup = signal.set_wakeup_fd(old_wakeup) + # + signal.signal(signal.SIGUSR1, signal.SIG_DFL) + class AppTestSignalSocket: @@ -168,4 +202,3 @@ alarm(0) finally: signal(SIGALRM, SIG_DFL) - Modified: pypy/branch/fast-forward/pypy/translator/c/src/signals.h ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/src/signals.h (original) +++ pypy/branch/fast-forward/pypy/translator/c/src/signals.h Sun Nov 7 15:15:57 2010 @@ -43,6 +43,7 @@ void pypysig_default(int signum); /* signal will do default action (SIG_DFL) */ void pypysig_setflag(int signum); /* signal will set a flag which can be queried with pypysig_poll() */ +int pypysig_set_wakeup_fd(int fd); /* utility to poll for signals that arrived */ int pypysig_poll(void); /* => signum or -1 */ @@ -76,6 +77,7 @@ struct pypysig_long_struct pypysig_occurred; static volatile long *pypysig_occurred_v = (volatile long *)&pypysig_occurred.value; static volatile int pypysig_flags[NSIG]; +static int wakeup_fd = -1; void pypysig_ignore(int signum) { @@ -112,6 +114,9 @@ /* the point of "*pypysig_occurred_v" instead of just "pypysig_occurred" is the volatile declaration */ *pypysig_occurred_v |= PENDING_SIGNAL_BIT; + + if (wakeup_fd != -1) + write(wakeup_fd, "\0", 1); } void pypysig_setflag(int signum) @@ -151,6 +156,13 @@ return -1; /* no pending signal */ } +int pypysig_set_wakeup_fd(int fd) +{ + int old_fd = wakeup_fd; + wakeup_fd = fd; + return old_fd; +} + #endif #endif From arigo at codespeak.net Sun Nov 7 15:32:59 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 15:32:59 +0100 (CET) Subject: [pypy-svn] r78814 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101107143259.28DA1282BDC@codespeak.net> Author: arigo Date: Sun Nov 7 15:32:57 2010 New Revision: 78814 Modified: pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py pypy/branch/fast-forward/pypy/objspace/std/setobject.py pypy/branch/fast-forward/pypy/objspace/std/settype.py pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Log: Set's isdisjoint() method. Modified: pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py Sun Nov 7 15:32:57 2010 @@ -33,6 +33,10 @@ frozenset_reduce = SMM('__reduce__',1, doc='Return state information for' ' pickling.') +# 2.6 methods +frozenset_isdisjoint = SMM('isdisjoint', 2, + doc='Return True if two sets have a' + ' null intersection.') register_all(vars(), globals()) Modified: pypy/branch/fast-forward/pypy/objspace/std/setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/setobject.py Sun Nov 7 15:32:57 2010 @@ -167,6 +167,14 @@ result[w_key] = None return result +def _isdisjoint_dict(ld, rd): + if len(ld) > len(rd): + ld, rd = rd, ld # loop over the smaller dict + for w_key in ld: + if w_key in rd: + return False + return True + def _symmetric_difference_dict(space, ld, rd): result = newset(space) for w_key in ld: @@ -510,6 +518,25 @@ inplace_and__Set_Frozenset = inplace_and__Set_Set +def set_isdisjoint__Set_Set(space, w_left, w_other): + # optimization only (the general case works too) + ld, rd = w_left.setdata, w_other.setdata + disjoint = _isdisjoint_dict(ld, rd) + return space.newbool(disjoint) + +set_isdisjoint__Set_Frozenset = set_isdisjoint__Set_Set +set_isdisjoint__Frozenset_Frozenset = set_isdisjoint__Set_Set +set_isdisjoint__Frozenset_Set = set_isdisjoint__Set_Set + +def set_isdisjoint__Set_ANY(space, w_left, w_other): + ld = w_left.setdata + for w_key in space.listview(w_other): + if w_key in ld: + return space.w_False + return space.w_True + +frozenset_isdisjoint__Frozenset_ANY = set_isdisjoint__Set_ANY + def set_symmetric_difference__Set_Set(space, w_left, w_other): # optimization only (the general case works too) ld, rd = w_left.setdata, w_other.setdata Modified: pypy/branch/fast-forward/pypy/objspace/std/settype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/settype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/settype.py Sun Nov 7 15:32:57 2010 @@ -62,6 +62,10 @@ set_reduce = SMM('__reduce__',1, doc='Return state information for' ' pickling.') +# 2.6 methods +set_isdisjoint = SMM('isdisjoint', 2, + doc='Return True if two sets have a' + ' null intersection.') register_all(vars(), globals()) Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Sun Nov 7 15:32:57 2010 @@ -270,3 +270,13 @@ efs = [f, Frozenset(f)] # All empty frozenset subclass instances should have different ids assert len(set(map(id, efs))) == len(efs) + + def test_isdisjoint(self): + assert set([1,2,3]).isdisjoint(set([4,5,6])) + assert set([1,2,3]).isdisjoint(frozenset([4,5,6])) + assert set([1,2,3]).isdisjoint([4,5,6]) + assert set([1,2,3]).isdisjoint((4,5,6)) + assert not set([1,2,5]).isdisjoint(set([4,5,6])) + assert not set([1,2,5]).isdisjoint(frozenset([4,5,6])) + assert not set([1,2,5]).isdisjoint([4,5,6]) + assert not set([1,2,5]).isdisjoint((4,5,6)) From arigo at codespeak.net Sun Nov 7 15:43:54 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 15:43:54 +0100 (CET) Subject: [pypy-svn] r78815 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101107144354.C079C282BDD@codespeak.net> Author: arigo Date: Sun Nov 7 15:43:53 2010 New Revision: 78815 Modified: pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py pypy/branch/fast-forward/pypy/objspace/std/setobject.py pypy/branch/fast-forward/pypy/objspace/std/settype.py pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Log: set.union(more_than_one_argument). Modified: pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py Sun Nov 7 15:43:53 2010 @@ -26,10 +26,9 @@ ' two sets as a new set.\n\n(i.e.' ' all elements that are in exactly' ' one of the sets.)') -frozenset_union = SMM('union', 2, - doc='Return the union of two sets as a' - ' new set.\n\n(i.e. all elements' - ' that are in either set.)') +frozenset_union = SMM('union', 1, varargs_w=True, + doc='Return a new set with elements' + ' from the set and all others.') frozenset_reduce = SMM('__reduce__',1, doc='Return state information for' ' pickling.') Modified: pypy/branch/fast-forward/pypy/objspace/std/setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/setobject.py Sun Nov 7 15:43:53 2010 @@ -582,30 +582,28 @@ inplace_xor__Set_Frozenset = inplace_xor__Set_Set -def set_union__Set_Set(space, w_left, w_other): - # optimization only (the general case works too) +def or__Set_Set(space, w_left, w_other): ld, rd = w_left.setdata, w_other.setdata result = ld.copy() result.update(rd) return w_left._newobj(space, result) -set_union__Set_Frozenset = set_union__Set_Set -set_union__Frozenset_Set = set_union__Set_Set -set_union__Frozenset_Frozenset = set_union__Set_Set -or__Set_Set = set_union__Set_Set -or__Set_Frozenset = set_union__Set_Set -or__Frozenset_Set = set_union__Set_Set -or__Frozenset_Frozenset = set_union__Set_Set +or__Set_Frozenset = or__Set_Set +or__Frozenset_Set = or__Set_Set +or__Frozenset_Frozenset = or__Set_Set - -def set_union__Set_ANY(space, w_left, w_other): +def set_union__Set(space, w_left, others_w): ld = w_left.setdata result = ld.copy() - for w_key in space.listview(w_other): - result[w_key] = None + for w_other in others_w: + if isinstance(w_other, W_BaseSetObject): + result.update(w_other.setdata) # optimization only + else: + for w_key in space.listview(w_other): + result[w_key] = None return w_left._newobj(space, result) -frozenset_union__Frozenset_ANY = set_union__Set_ANY +frozenset_union__Frozenset = set_union__Set def len__Set(space, w_left): return space.newint(len(w_left.setdata)) Modified: pypy/branch/fast-forward/pypy/objspace/std/settype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/settype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/settype.py Sun Nov 7 15:43:53 2010 @@ -52,10 +52,9 @@ set_symmetric_difference_update = SMM('symmetric_difference_update', 2, doc='Update a set with the symmetric' ' difference of itself and another.') -set_union = SMM('union', 2, - doc='Return the union of two sets as a' - ' new set.\n\n(i.e. all elements' - ' that are in either set.)') +set_union = SMM('union', 1, varargs_w=True, + doc='Return a new set with elements' + ' from the set and all others.') set_update = SMM('update', 2, doc='Update a set with the union of' ' itself and another.') Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Sun Nov 7 15:43:53 2010 @@ -61,6 +61,10 @@ a = set([4, 5]) b = a.union([5, 7]) assert sorted(b) == [4, 5, 7] + c = a.union([5, 7], [1], set([9,7]), frozenset([2]), frozenset()) + assert sorted(c) == [1, 2, 4, 5, 7, 9] + d = a.union() + assert d == a def test_compare(self): raises(TypeError, cmp, set('abc'), set('abd')) From arigo at codespeak.net Sun Nov 7 16:01:11 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 16:01:11 +0100 (CET) Subject: [pypy-svn] r78816 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101107150111.E2E7F282BE0@codespeak.net> Author: arigo Date: Sun Nov 7 16:01:10 2010 New Revision: 78816 Modified: pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py pypy/branch/fast-forward/pypy/objspace/std/setobject.py pypy/branch/fast-forward/pypy/objspace/std/settype.py pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Log: set.intersection(more_than_one_argument). Modified: pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py Sun Nov 7 16:01:10 2010 @@ -11,10 +11,9 @@ ' as a new set.\n\n(i.e. all' ' elements that are in this set but' ' not the other.)') -frozenset_intersection = SMM('intersection', 2, - doc='Return the intersection of two sets' - ' as a new set.\n\n(i.e. all' - ' elements that are in both sets.)') +frozenset_intersection = SMM('intersection', 1, varargs_w=True, + doc='Return a new set with elements common' + ' to the set and all others.') frozenset_issubset = SMM('issubset', 2, doc='Report whether another set contains' ' this set.') Modified: pypy/branch/fast-forward/pypy/objspace/std/setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/setobject.py Sun Nov 7 16:01:10 2010 @@ -471,30 +471,32 @@ del w_left.setdata[w_key] return w_key -def set_intersection__Set_Set(space, w_left, w_other): - # optimization only (the general case works too) +def and__Set_Set(space, w_left, w_other): ld, rd = w_left.setdata, w_other.setdata new_ld = _intersection_dict(space, ld, rd) return w_left._newobj(space, new_ld) -set_intersection__Set_Frozenset = set_intersection__Set_Set -set_intersection__Frozenset_Frozenset = set_intersection__Set_Set -set_intersection__Frozenset_Set = set_intersection__Set_Set - -def set_intersection__Set_ANY(space, w_left, w_other): - result = newset(space) - ld = w_left.setdata - for w_key in space.listview(w_other): - if w_key in ld: - result[w_key] = None +and__Set_Frozenset = and__Set_Set +and__Frozenset_Set = and__Set_Set +and__Frozenset_Frozenset = and__Set_Set + +def set_intersection__Set(space, w_left, others_w): + result = w_left.setdata + if len(others_w) == 0: + result = result.copy() + for w_other in others_w: + if isinstance(w_other, W_BaseSetObject): + # optimization only + result = _intersection_dict(space, result, w_other.setdata) + else: + result2 = newset(space) + for w_key in space.listview(w_other): + if w_key in result: + result2[w_key] = None + result = result2 return w_left._newobj(space, result) -frozenset_intersection__Frozenset_ANY = set_intersection__Set_ANY - -and__Set_Set = set_intersection__Set_Set -and__Set_Frozenset = set_intersection__Set_Set -and__Frozenset_Set = set_intersection__Set_Set -and__Frozenset_Frozenset = set_intersection__Set_Set +frozenset_intersection__Frozenset = set_intersection__Set def set_intersection_update__Set_Set(space, w_left, w_other): # optimization only (the general case works too) Modified: pypy/branch/fast-forward/pypy/objspace/std/settype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/settype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/settype.py Sun Nov 7 16:01:10 2010 @@ -23,10 +23,9 @@ doc='Remove an element from a set if it' ' is a member.\n\nIf the element is' ' not a member, do nothing.') -set_intersection = SMM('intersection', 2, - doc='Return the intersection of two sets' - ' as a new set.\n\n(i.e. all' - ' elements that are in both sets.)') +set_intersection = SMM('intersection', 1, varargs_w=True, + doc='Return a new set with elements common' + ' to the set and all others.') set_intersection_update = SMM('intersection_update', 2, doc='Update a set with the intersection' ' of itself and another.') Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Sun Nov 7 16:01:10 2010 @@ -11,8 +11,8 @@ from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject from pypy.objspace.std.setobject import _initialize_set from pypy.objspace.std.setobject import newset, make_setdata_from_w_iterable -from pypy.objspace.std.setobject import set_intersection__Set_Set -from pypy.objspace.std.setobject import set_intersection__Set_ANY +from pypy.objspace.std.setobject import and__Set_Set +from pypy.objspace.std.setobject import set_intersection__Set from pypy.objspace.std.setobject import eq__Set_Set letters = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ' @@ -35,10 +35,10 @@ _initialize_set(self.space, t0, self.otherword) t1 = W_FrozensetObject(self.space, make_setdata_from_w_iterable(self.space, self.otherword)) - r0 = set_intersection__Set_Set(self.space, s, t0) - r1 = set_intersection__Set_Set(self.space, s, t1) + r0 = and__Set_Set(self.space, s, t0) + r1 = and__Set_Set(self.space, s, t1) assert eq__Set_Set(self.space, r0, r1) == self.true - sr = set_intersection__Set_ANY(self.space, s, self.otherword) + sr = set_intersection__Set(self.space, s, [self.otherword]) assert eq__Set_Set(self.space, r0, sr) == self.true def test_compare(self): @@ -284,3 +284,19 @@ assert not set([1,2,5]).isdisjoint(frozenset([4,5,6])) assert not set([1,2,5]).isdisjoint([4,5,6]) assert not set([1,2,5]).isdisjoint((4,5,6)) + + def test_intersection(self): + assert set([1,2,3]).intersection(set([2,3,4])) == set([2,3]) + assert set([1,2,3]).intersection(frozenset([2,3,4])) == set([2,3]) + assert set([1,2,3]).intersection([2,3,4]) == set([2,3]) + assert set([1,2,3]).intersection((2,3,4)) == set([2,3]) + assert frozenset([1,2,3]).intersection(set([2,3,4])) == frozenset([2,3]) + assert frozenset([1,2,3]).intersection(frozenset([2,3,4]))== frozenset([2,3]) + assert frozenset([1,2,3]).intersection([2,3,4]) == frozenset([2,3]) + assert frozenset([1,2,3]).intersection((2,3,4)) == frozenset([2,3]) + assert set([1,2,3,4]).intersection([2,3,4,5], set((1,2,3))) == set([2,3]) + assert frozenset([1,2,3,4]).intersection((2,3,4,5), [1,2,3]) == \ + frozenset([2,3]) + s = set([1,2,3]) + assert s.intersection() == s + assert s.intersection() is not s From arigo at codespeak.net Sun Nov 7 16:07:09 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 16:07:09 +0100 (CET) Subject: [pypy-svn] r78817 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101107150709.87F35282BE3@codespeak.net> Author: arigo Date: Sun Nov 7 16:07:07 2010 New Revision: 78817 Modified: pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py pypy/branch/fast-forward/pypy/objspace/std/setobject.py pypy/branch/fast-forward/pypy/objspace/std/settype.py pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Log: set.difference(more_than_one_argument). Modified: pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/frozensettype.py Sun Nov 7 16:07:07 2010 @@ -6,11 +6,9 @@ frozenset_copy = SMM('copy', 1, doc='Return a shallow copy of a set.') -frozenset_difference = SMM('difference', 2, - doc='Return the difference of two sets' - ' as a new set.\n\n(i.e. all' - ' elements that are in this set but' - ' not the other.)') +frozenset_difference = SMM('difference', 1, varargs_w=True, + doc='Return a new set with elements in' + ' the set that are not in the others.') frozenset_intersection = SMM('intersection', 1, varargs_w=True, doc='Return a new set with elements common' ' to the set and all others.') Modified: pypy/branch/fast-forward/pypy/objspace/std/setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/setobject.py Sun Nov 7 16:07:07 2010 @@ -233,26 +233,28 @@ def set_clear__Set(space, w_left): w_left.setdata.clear() -def set_difference__Set_Set(space, w_left, w_other): - # optimization only (the general case works too) +def sub__Set_Set(space, w_left, w_other): ld, rd = w_left.setdata, w_other.setdata new_ld = _difference_dict(space, ld, rd) return w_left._newobj(space, new_ld) -set_difference__Set_Frozenset = set_difference__Set_Set -frozenset_difference__Frozenset_Set = set_difference__Set_Set -frozenset_difference__Frozenset_Frozenset = set_difference__Set_Set -sub__Set_Set = set_difference__Set_Set -sub__Set_Frozenset = set_difference__Set_Set -sub__Frozenset_Set = set_difference__Set_Set -sub__Frozenset_Frozenset = set_difference__Set_Set +sub__Set_Frozenset = sub__Set_Set +sub__Frozenset_Set = sub__Set_Set +sub__Frozenset_Frozenset = sub__Set_Set -def set_difference__Set_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) - new_ld = _difference_dict(space, ld, rd) - return w_left._newobj(space, new_ld) +def set_difference__Set(space, w_left, others_w): + result = w_left.setdata + if len(others_w) == 0: + result = result.copy() + for w_other in others_w: + if isinstance(w_other, W_BaseSetObject): + rd = w_other.setdata # optimization only + else: + rd = make_setdata_from_w_iterable(space, w_other) + result = _difference_dict(space, result, rd) + return w_left._newobj(space, result) -frozenset_difference__Frozenset_ANY = set_difference__Set_ANY +frozenset_difference__Frozenset = set_difference__Set def set_difference_update__Set_Set(space, w_left, w_other): Modified: pypy/branch/fast-forward/pypy/objspace/std/settype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/settype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/settype.py Sun Nov 7 16:07:07 2010 @@ -11,11 +11,9 @@ doc='Remove all elements from this set.') set_copy = SMM('copy', 1, doc='Return a shallow copy of a set.') -set_difference = SMM('difference', 2, - doc='Return the difference of two sets' - ' as a new set.\n\n(i.e. all' - ' elements that are in this set but' - ' not the other.)') +set_difference = SMM('difference', 1, varargs_w=True, + doc='Return a new set with elements in' + ' the set that are not in the others.') set_difference_update = SMM('difference_update', 2, doc='Remove all elements of another set' ' from this set.') Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Sun Nov 7 16:07:07 2010 @@ -300,3 +300,18 @@ s = set([1,2,3]) assert s.intersection() == s assert s.intersection() is not s + + def test_difference(self): + assert set([1,2,3]).difference(set([2,3,4])) == set([1]) + assert set([1,2,3]).difference(frozenset([2,3,4])) == set([1]) + assert set([1,2,3]).difference([2,3,4]) == set([1]) + assert set([1,2,3]).difference((2,3,4)) == set([1]) + assert frozenset([1,2,3]).difference(set([2,3,4])) == frozenset([1]) + assert frozenset([1,2,3]).difference(frozenset([2,3,4]))== frozenset([1]) + assert frozenset([1,2,3]).difference([2,3,4]) == frozenset([1]) + assert frozenset([1,2,3]).difference((2,3,4)) == frozenset([1]) + assert set([1,2,3,4]).difference([4,5], set((0,1))) == set([2,3]) + assert frozenset([1,2,3,4]).difference((4,5), [0,1]) == frozenset([2,3]) + s = set([1,2,3]) + assert s.difference() == s + assert s.difference() is not s From arigo at codespeak.net Sun Nov 7 16:13:46 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 16:13:46 +0100 (CET) Subject: [pypy-svn] r78818 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101107151346.B753E282BE8@codespeak.net> Author: arigo Date: Sun Nov 7 16:13:45 2010 New Revision: 78818 Modified: pypy/branch/fast-forward/pypy/objspace/std/setobject.py pypy/branch/fast-forward/pypy/objspace/std/settype.py pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Log: set.update(more_than_one_argument). Modified: pypy/branch/fast-forward/pypy/objspace/std/setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/setobject.py Sun Nov 7 16:13:45 2010 @@ -197,19 +197,19 @@ #end helper functions -def set_update__Set_BaseSet(space, w_left, w_other): - # optimization only (the general case works too) - ld, rd = w_left.setdata, w_other.setdata - ld.update(rd) - -def set_update__Set_ANY(space, w_left, w_other): +def set_update__Set(space, w_left, others_w): """Update a set with the union of itself and another.""" ld = w_left.setdata - for w_item in space.listview(w_other): - ld[w_item] = None + for w_other in others_w: + if isinstance(w_other, W_BaseSetObject): + ld.update(w_other.setdata) # optimization only + else: + for w_key in space.listview(w_other): + ld[w_key] = None def inplace_or__Set_Set(space, w_left, w_other): - set_update__Set_BaseSet(space, w_left, w_other) + ld, rd = w_left.setdata, w_other.setdata + ld.update(rd) return w_left inplace_or__Set_Frozenset = inplace_or__Set_Set @@ -597,8 +597,7 @@ or__Frozenset_Frozenset = or__Set_Set def set_union__Set(space, w_left, others_w): - ld = w_left.setdata - result = ld.copy() + result = w_left.setdata.copy() for w_other in others_w: if isinstance(w_other, W_BaseSetObject): result.update(w_other.setdata) # optimization only Modified: pypy/branch/fast-forward/pypy/objspace/std/settype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/settype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/settype.py Sun Nov 7 16:13:45 2010 @@ -52,9 +52,9 @@ set_union = SMM('union', 1, varargs_w=True, doc='Return a new set with elements' ' from the set and all others.') -set_update = SMM('update', 2, - doc='Update a set with the union of' - ' itself and another.') +set_update = SMM('update', 1, varargs_w=True, + doc='Update the set, adding elements' + ' from all others.') set_reduce = SMM('__reduce__',1, doc='Return state information for' ' pickling.') Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Sun Nov 7 16:13:45 2010 @@ -144,6 +144,15 @@ s1 = set('abc') s1.update(frozenset('fro')) assert s1 == set('abcfro') + s1 = set('abc') + s1.update('def') + assert s1 == set('abcdef') + s1 = set('abc') + s1.update() + assert s1 == set('abc') + s1 = set('abc') + s1.update('d', 'ef', frozenset('g')) + assert s1 == set('abcdefg') def test_recursive_repr(self): class A(object): From arigo at codespeak.net Sun Nov 7 16:19:23 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 16:19:23 +0100 (CET) Subject: [pypy-svn] r78819 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101107151923.17158282BE9@codespeak.net> Author: arigo Date: Sun Nov 7 16:19:22 2010 New Revision: 78819 Modified: pypy/branch/fast-forward/pypy/objspace/std/setobject.py pypy/branch/fast-forward/pypy/objspace/std/settype.py pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Log: set.intersection_update(more_than_one_argument). Modified: pypy/branch/fast-forward/pypy/objspace/std/setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/setobject.py Sun Nov 7 16:19:22 2010 @@ -482,10 +482,8 @@ and__Frozenset_Set = and__Set_Set and__Frozenset_Frozenset = and__Set_Set -def set_intersection__Set(space, w_left, others_w): +def _intersection_multiple(space, w_left, others_w): result = w_left.setdata - if len(others_w) == 0: - result = result.copy() for w_other in others_w: if isinstance(w_other, W_BaseSetObject): # optimization only @@ -496,28 +494,25 @@ if w_key in result: result2[w_key] = None result = result2 + return result + +def set_intersection__Set(space, w_left, others_w): + if len(others_w) == 0: + result = w_left.setdata.copy() + else: + result = _intersection_multiple(space, w_left, others_w) return w_left._newobj(space, result) frozenset_intersection__Frozenset = set_intersection__Set -def set_intersection_update__Set_Set(space, w_left, w_other): - # optimization only (the general case works too) - ld, rd = w_left.setdata, w_other.setdata - new_ld = _intersection_dict(space, ld, rd) - w_left.setdata = new_ld - -set_intersection_update__Set_Frozenset = set_intersection_update__Set_Set - -def set_intersection_update__Set_ANY(space, w_left, w_other): - result = newset(space) - ld = w_left.setdata - for w_key in space.listview(w_other): - if w_key in ld: - result[w_key] = None +def set_intersection_update__Set(space, w_left, others_w): + result = _intersection_multiple(space, w_left, others_w) w_left.setdata = result def inplace_and__Set_Set(space, w_left, w_other): - set_intersection_update__Set_Set(space, w_left, w_other) + ld, rd = w_left.setdata, w_other.setdata + new_ld = _intersection_dict(space, ld, rd) + w_left.setdata = new_ld return w_left inplace_and__Set_Frozenset = inplace_and__Set_Set Modified: pypy/branch/fast-forward/pypy/objspace/std/settype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/settype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/settype.py Sun Nov 7 16:19:22 2010 @@ -24,9 +24,9 @@ set_intersection = SMM('intersection', 1, varargs_w=True, doc='Return a new set with elements common' ' to the set and all others.') -set_intersection_update = SMM('intersection_update', 2, - doc='Update a set with the intersection' - ' of itself and another.') +set_intersection_update = SMM('intersection_update', 1, varargs_w=True, + doc='Update the set, keeping only elements' + ' found in it and all others.') set_issubset = SMM('issubset', 2, doc='Report whether another set contains' ' this set.') Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Sun Nov 7 16:19:22 2010 @@ -324,3 +324,12 @@ s = set([1,2,3]) assert s.difference() == s assert s.difference() is not s + + def test_intersection_update(self): + s = set([1,2,3,4,7]) + s.intersection_update([0,1,2,3,4,5,6]) + assert s == set([1,2,3,4]) + s.intersection_update((2,3,4,5), frozenset([0,1,2,3])) + assert s == set([2,3]) + s.intersection_update() + assert s == set([2,3]) From arigo at codespeak.net Sun Nov 7 16:24:31 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 16:24:31 +0100 (CET) Subject: [pypy-svn] r78820 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101107152431.9EC1836C224@codespeak.net> Author: arigo Date: Sun Nov 7 16:24:30 2010 New Revision: 78820 Modified: pypy/branch/fast-forward/pypy/objspace/std/setobject.py pypy/branch/fast-forward/pypy/objspace/std/settype.py pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Log: set.difference_update(more_than_one_argument). Modified: pypy/branch/fast-forward/pypy/objspace/std/setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/setobject.py Sun Nov 7 16:24:30 2010 @@ -257,23 +257,22 @@ frozenset_difference__Frozenset = set_difference__Set -def set_difference_update__Set_Set(space, w_left, w_other): - # optimization only (the general case works too) - ld, rd = w_left.setdata, w_other.setdata - _difference_dict_update(space, ld, rd) - -set_difference_update__Set_Frozenset = set_difference_update__Set_Set - -def set_difference_update__Set_ANY(space, w_left, w_other): +def set_difference_update__Set(space, w_left, others_w): ld = w_left.setdata - for w_key in space.listview(w_other): - try: - del ld[w_key] - except KeyError: - pass + for w_other in others_w: + if isinstance(w_other, W_BaseSetObject): + # optimization only + _difference_dict_update(space, ld, w_other.setdata) + else: + for w_key in space.listview(w_other): + try: + del ld[w_key] + except KeyError: + pass def inplace_sub__Set_Set(space, w_left, w_other): - set_difference_update__Set_Set(space, w_left, w_other) + ld, rd = w_left.setdata, w_other.setdata + _difference_dict_update(space, ld, rd) return w_left inplace_sub__Set_Frozenset = inplace_sub__Set_Set Modified: pypy/branch/fast-forward/pypy/objspace/std/settype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/settype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/settype.py Sun Nov 7 16:24:30 2010 @@ -14,9 +14,9 @@ set_difference = SMM('difference', 1, varargs_w=True, doc='Return a new set with elements in' ' the set that are not in the others.') -set_difference_update = SMM('difference_update', 2, - doc='Remove all elements of another set' - ' from this set.') +set_difference_update = SMM('difference_update', 1, varargs_w=True, + doc='Update the set, removing elements' + ' found in others.') set_discard = SMM('discard', 2, doc='Remove an element from a set if it' ' is a member.\n\nIf the element is' Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_setobject.py Sun Nov 7 16:24:30 2010 @@ -333,3 +333,14 @@ assert s == set([2,3]) s.intersection_update() assert s == set([2,3]) + + def test_difference_update(self): + s = set([1,2,3,4,7]) + s.difference_update([0,7,8,9]) + assert s == set([1,2,3,4]) + s.difference_update((0,1), frozenset([4,5,6])) + assert s == set([2,3]) + s.difference_update() + assert s == set([2,3]) + s.difference_update(s) + assert s == set([]) From arigo at codespeak.net Sun Nov 7 16:36:17 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 16:36:17 +0100 (CET) Subject: [pypy-svn] r78821 - pypy/branch/fast-forward/lib-python Message-ID: <20101107153617.7F6CD282BEA@codespeak.net> Author: arigo Date: Sun Nov 7 16:36:16 2010 New Revision: 78821 Modified: pypy/branch/fast-forward/lib-python/TODO Log: Did this. Modified: pypy/branch/fast-forward/lib-python/TODO ============================================================================== --- pypy/branch/fast-forward/lib-python/TODO (original) +++ pypy/branch/fast-forward/lib-python/TODO Sun Nov 7 16:36:16 2010 @@ -33,7 +33,7 @@ - missing posix.(confstr|pathconf|fpathconf)(_names)? -- some set() methods now accept a variable number of arguments +- some set() methods now accept a variable number of arguments DONE Medium tasks From arigo at codespeak.net Sun Nov 7 16:48:00 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 16:48:00 +0100 (CET) Subject: [pypy-svn] r78822 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101107154800.F3BD7282BDD@codespeak.net> Author: arigo Date: Sun Nov 7 16:47:59 2010 New Revision: 78822 Modified: pypy/branch/fast-forward/pypy/objspace/std/stringobject.py pypy/branch/fast-forward/pypy/objspace/std/test/test_stringobject.py Log: str.translate(None, 'delete-chars') Modified: pypy/branch/fast-forward/pypy/objspace/std/stringobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/stringobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/stringobject.py Sun Nov 7 16:47:59 2010 @@ -919,7 +919,9 @@ return space.wrap(buf.build()) - + +DEFAULT_NOOP_TABLE = ''.join([chr(i) for i in range(256)]) + def str_translate__String_ANY_ANY(space, w_string, w_table, w_deletechars=''): """charfilter - unicode handling is not implemented @@ -929,11 +931,14 @@ which must be a string of length 256""" # XXX CPython accepts buffers, too, not sure what we should do - table = space.str_w(w_table) - if len(table) != 256: - raise OperationError( - space.w_ValueError, - space.wrap("translation table must be 256 characters long")) + if space.is_w(w_table, space.w_None): + table = DEFAULT_NOOP_TABLE + else: + table = space.str_w(w_table) + if len(table) != 256: + raise OperationError( + space.w_ValueError, + space.wrap("translation table must be 256 characters long")) string = w_string._value chars = [] Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_stringobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_stringobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_stringobject.py Sun Nov 7 16:47:59 2010 @@ -594,6 +594,8 @@ raises(ValueError, 'xyz'.translate, 'too short') raises(ValueError, 'xyz'.translate, 'too long'*33) + assert 'yz' == 'xyz'.translate(None, 'x') # 2.6 + def test_iter(self): l=[] for i in iter("42"): From arigo at codespeak.net Sun Nov 7 16:52:35 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 16:52:35 +0100 (CET) Subject: [pypy-svn] r78823 - pypy/trunk/pypy/objspace/std Message-ID: <20101107155235.E9DF1282BEC@codespeak.net> Author: arigo Date: Sun Nov 7 16:52:29 2010 New Revision: 78823 Modified: pypy/trunk/pypy/objspace/std/stringobject.py Log: Optimize the code for str.translate(..., deletechars). Modified: pypy/trunk/pypy/objspace/std/stringobject.py ============================================================================== --- pypy/trunk/pypy/objspace/std/stringobject.py (original) +++ pypy/trunk/pypy/objspace/std/stringobject.py Sun Nov 7 16:52:29 2010 @@ -937,10 +937,17 @@ string = w_string._value chars = [] - for char in string: - w_char = W_StringObject.PREBUILT[ord(char)] - if not space.is_true(space.contains(w_deletechars, w_char)): - chars.append(table[ord(char)]) + deletechars = space.str_w(w_deletechars) + if len(deletechars) == 0: + for char in string: + chars.append(table[ord(char)]) + else: + deletion_table = [False] * 256 + for c in deletechars: + deletion_table[ord(c)] = True + for char in string: + if not deletion_table[ord(char)]: + chars.append(table[ord(char)]) return W_StringObject(''.join(chars)) def str_decode__String_ANY_ANY(space, w_string, w_encoding=None, w_errors=None): From arigo at codespeak.net Sun Nov 7 17:34:05 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 17:34:05 +0100 (CET) Subject: [pypy-svn] r78824 - in pypy/branch/fast-forward/pypy: interpreter module/__builtin__ module/__builtin__/test Message-ID: <20101107163405.7C8C9282BEB@codespeak.net> Author: arigo Date: Sun Nov 7 17:34:04 2010 New Revision: 78824 Modified: pypy/branch/fast-forward/pypy/interpreter/buffer.py pypy/branch/fast-forward/pypy/module/__builtin__/__init__.py pypy/branch/fast-forward/pypy/module/__builtin__/operation.py pypy/branch/fast-forward/pypy/module/__builtin__/test/test_buffer.py Log: Attempt to kill the W_Memoryview class, and just use buffers. Doing so is a lot more efficient, because it will select the right subclass of Buffer when constructing it. Also rename memoryview.to_bytes() to memoryview.tobytes(), according to the CPython API documentation. This actually adds tobytes() as a method on all buffer objects. I suppose it's fine. Modified: pypy/branch/fast-forward/pypy/interpreter/buffer.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/buffer.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/buffer.py Sun Nov 7 17:34:04 2010 @@ -207,6 +207,7 @@ __mul__ = interp2app(Buffer.descr_mul), __rmul__ = interp2app(Buffer.descr_mul), __repr__ = interp2app(Buffer.descr_repr), + tobytes = interp2app(Buffer.descr_str), ) Buffer.typedef.acceptable_as_base_class = False Modified: pypy/branch/fast-forward/pypy/module/__builtin__/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/__init__.py Sun Nov 7 17:34:04 2010 @@ -53,7 +53,7 @@ 'bytes' : '(space.w_str)', 'unicode' : '(space.w_unicode)', 'buffer' : 'operation.W_Buffer', - 'memoryview' : 'operation.W_Memoryview', + 'memoryview' : 'operation.W_Buffer', 'file' : 'state.get(space).w_file', 'open' : 'state.get(space).w_file', Modified: pypy/branch/fast-forward/pypy/module/__builtin__/operation.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/operation.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/operation.py Sun Nov 7 17:34:04 2010 @@ -16,23 +16,6 @@ W_Buffer = buffer.Buffer -class W_Memoryview(buffer.StringLikeBuffer): - @unwrap_spec(ObjSpace, W_Root, W_Root) - def descr_new(space, w_subtype, w_object): - self = space.allocate_instance(W_Memoryview, w_subtype) - buffer.StringLikeBuffer.__init__(self, space, w_object) - return space.wrap(self) - - @unwrap_spec('self', ObjSpace) - def to_bytes_w(self, space): - return space.wrap(self.as_str()) - -W_Memoryview.typedef = TypeDef( - "memoryview", W_Buffer.typedef, - __new__=interp2app(W_Memoryview.descr_new.im_func), - to_bytes=interp2app(W_Memoryview.to_bytes_w), - ) - def abs(space, w_val): "abs(number) -> number\n\nReturn the absolute value of the argument." return space.abs(w_val) Modified: pypy/branch/fast-forward/pypy/module/__builtin__/test/test_buffer.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/test/test_buffer.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/test/test_buffer.py Sun Nov 7 17:34:04 2010 @@ -163,6 +163,6 @@ class AppTestMemoryview: def test_basic(self): v = memoryview("abc") - assert v.to_bytes() == "abc" + assert v.tobytes() == "abc" assert len(v) == 3 assert list(v) == ['a', 'b', 'c'] From david at codespeak.net Sun Nov 7 17:50:29 2010 From: david at codespeak.net (david at codespeak.net) Date: Sun, 7 Nov 2010 17:50:29 +0100 (CET) Subject: [pypy-svn] r78825 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101107165029.9C78E282BF0@codespeak.net> Author: david Date: Sun Nov 7 17:50:27 2010 New Revision: 78825 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Log: Implement setfield_gc and getfield_gc operations Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Sun Nov 7 17:50:27 2010 @@ -5,7 +5,8 @@ from pypy.jit.backend.arm.codebuilder import ARMv7Builder, ARMv7InMemoryBuilder from pypy.jit.backend.arm.regalloc import ARMRegisterManager, ARMFrameManager from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity -from pypy.jit.metainterp.history import ConstInt, BoxInt, Box, BasicFailDescr +from pypy.jit.metainterp.history import (ConstInt, BoxInt, Box, BasicFailDescr, + INT, REF, FLOAT) from pypy.jit.metainterp.resoperation import rop from pypy.rlib import rgc from pypy.rpython.annlowlevel import llhelper @@ -13,18 +14,20 @@ from pypy.jit.backend.arm.opassembler import (GuardOpAssembler, IntOpAsslember, OpAssembler, - UnaryIntOpAssembler) + UnaryIntOpAssembler, + FieldOpAssembler) # XXX Move to llsupport from pypy.jit.backend.x86.support import values_array class AssemblerARM(GuardOpAssembler, IntOpAsslember, - OpAssembler, UnaryIntOpAssembler): + OpAssembler, UnaryIntOpAssembler, FieldOpAssembler): def __init__(self, cpu, failargs_limit=1000): self.mc = ARMv7Builder() self.cpu = cpu self.fail_boxes_int = values_array(lltype.Signed, failargs_limit) + self.fail_boxes_ptr = values_array(llmemory.GCREF, failargs_limit) self._debug_asm = True self._exit_code_addr = self.mc.curraddr() @@ -58,11 +61,16 @@ i += 1 fail_index += 1 res = enc[i] - if res == '\xFE': - continue if res == '\xFF': break + if res == '\xFE': + continue + + group = res + i += 1 + res = enc[i] if res == '\xFD': + assert group == '\xEF' # imm value value = self.decode32(enc, i+1) i += 4 @@ -75,7 +83,13 @@ reg = ord(enc[i]) value = self.decode32(stack, reg*WORD) - self.fail_boxes_int.setitem(fail_index, value) + if group == '\xEF': # INT + self.fail_boxes_int.setitem(fail_index, value) + elif group == '\xEE': # REF + self.fail_boxes_ptr.setitem(fail_index, rffi.cast(llmemory.GCREF, value)) + else: + assert 0, 'unknown type' + assert enc[i] == '\xFF' descr = self.decode32(enc, i+1) @@ -114,6 +128,10 @@ def _gen_path_to_exit_path(self, op, args, regalloc, fcond=c.AL): """ + types: + \xEE = REF + \xEF = INT + location: \xFC = stack location \xFD = imm location \xFE = Empty arg @@ -122,21 +140,31 @@ box = Box() reg = regalloc.force_allocate_reg(box) # XXX free this memory + # XXX allocate correct amount of memory mem = lltype.malloc(rffi.CArray(lltype.Char), (len(args)+5)*4, flavor='raw') i = 0 j = 0 while(i < len(args)): if args[i]: loc = regalloc.loc(args[i]) + if args[i].type == INT: + mem[j] = '\xEF' + j += 1 + elif args[i].type == REF: + mem[j] = '\xEE' + j += 1 + else: + assert 0, 'unknown type' + if loc.is_reg(): mem[j] = chr(loc.value) j += 1 elif loc.is_imm(): + assert args[i].type == INT mem[j] = '\xFD' self.encode32(mem, j+1, loc.getint()) j += 5 else: - #print 'Encoding a stack location' mem[j] = '\xFC' self.encode32(mem, j+1, loc.position) j += 5 @@ -178,8 +206,14 @@ def gen_bootstrap_code(self, inputargs, regalloc, looptoken): regs = [] for i in range(len(inputargs)): - reg = regalloc.force_allocate_reg(inputargs[i]) - addr = self.fail_boxes_int.get_addr_for_num(i) + loc = inputargs[i] + reg = regalloc.force_allocate_reg(loc) + if loc.type == REF: + addr = self.fail_boxes_ptr.get_addr_for_num(i) + elif loc.type == INT: + addr = self.fail_boxes_int.get_addr_for_num(i) + else: + raise ValueError self.mc.gen_load_int(reg.value, addr) self.mc.LDR_ri(reg.value, reg.value) regs.append(reg) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Sun Nov 7 17:50:27 2010 @@ -10,6 +10,7 @@ gen_emit_op_ri, gen_emit_cmp_op) from pypy.jit.backend.arm.codebuilder import ARMv7Builder, ARMv7InMemoryBuilder from pypy.jit.backend.arm.regalloc import ARMRegisterManager +from pypy.jit.backend.llsupport.descr import BaseFieldDescr, BaseArrayDescr from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity, TempBox from pypy.jit.metainterp.history import ConstInt, BoxInt, Box, BasicFailDescr from pypy.jit.metainterp.resoperation import rop @@ -220,3 +221,46 @@ regalloc.force_allocate_reg(op.result, selected_reg=r.r0) regalloc.after_call(op.result) regalloc.possibly_free_vars(locs) + +class FieldOpAssembler(object): + + def emit_op_setfield_gc(self, op, regalloc, fcond): + ofs, size, ptr = self._unpack_fielddescr(op.getdescr()) + #ofs_loc = regalloc.make_sure_var_in_reg(ConstInt(ofs)) + #size_loc = regalloc.make_sure_var_in_reg(ofs) + base_loc = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) + value_loc = regalloc.make_sure_var_in_reg(op.getarg(1), imm_fine=False) + if size == 4: + f = self.mc.STR_ri + elif size == 2: + f = self.mc.STRH_ri + elif size == 1: + f = self.mc.STRB_ri + else: + assert 0 + f(value_loc.value, base_loc.value, ofs) + return fcond + + def emit_op_getfield_gc(self, op, regalloc, fcond): + ofs, size, ptr = self._unpack_fielddescr(op.getdescr()) + # ofs_loc = regalloc.make_sure_var_in_reg(ConstInt(ofs)) + base_loc = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) + res = regalloc.force_allocate_reg(op.result) + if size == 4: + f = self.mc.LDR_ri + elif size == 2: + f = self.mc.LDRH_ri + elif size == 1: + f = self.mc.LDRB_ri + else: + assert 0 + f(res.value, base_loc.value, ofs) + return fcond + + #XXX from ../x86/regalloc.py:791 + def _unpack_fielddescr(self, fielddescr): + assert isinstance(fielddescr, BaseFieldDescr) + ofs = fielddescr.offset + size = fielddescr.get_field_size(self.cpu.translate_support_code) + ptr = fielddescr.is_pointer_field() + return ofs, size, ptr Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Sun Nov 7 17:50:27 2010 @@ -2,6 +2,8 @@ RegisterManager, compute_vars_longevity from pypy.jit.backend.arm import registers as r from pypy.jit.backend.arm import locations +from pypy.jit.metainterp.history import ConstInt +from pypy.rpython.lltypesystem import rffi, lltype class ARMRegisterManager(RegisterManager): all_regs = r.all_regs @@ -15,14 +17,19 @@ def update_bindings(self, enc, inputargs): j = 0 for i in range(len(inputargs)): - # XXX decode imm and and stack locs + # XXX decode imm and and stack locs and REFs while enc[j] == '\xFE': j += 1 + assert enc[j] == '\xEF' + j += 1 self.force_allocate_reg(inputargs[i], selected_reg=r.all_regs[ord(enc[j])]) j += 1 def convert_to_imm(self, c): - return locations.ImmLocation(c.value) + if isinstance(c, ConstInt): + return locations.ImmLocation(c.value) + else: + return locations.ImmLocation(rffi.cast(lltype.Signed, c.value)) def call_result_location(self, v): return r.r0 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/registers.py Sun Nov 7 17:50:27 2010 @@ -13,6 +13,6 @@ all_regs = [r0, r1, r2, r3, r4, r5, r6, r7, r8, r9, r10] caller_resp = [r0, r1, r2, r3] -callee_resp = [r4, r5, r6, r7, r8, r9, r10, r11] +callee_resp = [r4, r5, r6, r7, r8, r9, r10, fp] callee_saved_registers = callee_resp+[lr] callee_restored_registers = callee_resp+[pc] Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Sun Nov 7 17:50:27 2010 @@ -24,9 +24,15 @@ def set_future_value_int(self, index, intvalue): self.assembler.fail_boxes_int.setitem(index, intvalue) + def set_future_value_ref(self, index, ptrvalue): + self.assembler.fail_boxes_ptr.setitem(index, ptrvalue) + def get_latest_value_int(self, index): return self.assembler.fail_boxes_int.getitem(index) + def get_latest_value_ref(self, index): + return self.assembler.fail_boxes_ptr.getitem(index) + def get_latest_value_count(self): return self.assembler.fail_boxes_count From fijal at codespeak.net Sun Nov 7 17:50:51 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 7 Nov 2010 17:50:51 +0100 (CET) Subject: [pypy-svn] r78826 - pypy/trunk/pypy/module/sys Message-ID: <20101107165051.58521282BF1@codespeak.net> Author: fijal Date: Sun Nov 7 17:50:50 2010 New Revision: 78826 Modified: pypy/trunk/pypy/module/sys/version.py Log: Bump a number (should be done a long time ago) Modified: pypy/trunk/pypy/module/sys/version.py ============================================================================== --- pypy/trunk/pypy/module/sys/version.py (original) +++ pypy/trunk/pypy/module/sys/version.py Sun Nov 7 17:50:50 2010 @@ -8,7 +8,7 @@ CPYTHON_VERSION = (2, 5, 2, "beta", 42) #XXX # sync patchlevel.h CPYTHON_API_VERSION = 1012 #XXX # sync with include/modsupport.h -PYPY_VERSION = (1, 3, 0, "beta", '?') #XXX # sync patchlevel.h +PYPY_VERSION = (1, 4, 0, "beta", '?') #XXX # sync patchlevel.h # the last item is replaced by the svn revision ^^^ TRIM_URL_UP_TO = 'svn/pypy/' From fijal at codespeak.net Sun Nov 7 17:51:22 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 7 Nov 2010 17:51:22 +0100 (CET) Subject: [pypy-svn] r78827 - pypy/release/1.4.x Message-ID: <20101107165122.E3670282BF3@codespeak.net> Author: fijal Date: Sun Nov 7 17:51:21 2010 New Revision: 78827 Added: pypy/release/1.4.x/ (props changed) - copied from r78826, pypy/trunk/ Log: Branch for a release From fijal at codespeak.net Sun Nov 7 18:35:43 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 7 Nov 2010 18:35:43 +0100 (CET) Subject: [pypy-svn] r78828 - in pypy/trunk/pypy/jit: backend/llgraph metainterp metainterp/test tool Message-ID: <20101107173543.681EE282B90@codespeak.net> Author: fijal Date: Sun Nov 7 18:35:41 2010 New Revision: 78828 Modified: pypy/trunk/pypy/jit/backend/llgraph/llimpl.py pypy/trunk/pypy/jit/metainterp/logger.py pypy/trunk/pypy/jit/metainterp/pyjitpl.py pypy/trunk/pypy/jit/metainterp/resoperation.py pypy/trunk/pypy/jit/metainterp/test/test_logger.py pypy/trunk/pypy/jit/metainterp/test/test_oparser.py pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py pypy/trunk/pypy/jit/tool/oparser.py Log: Make debug_merge_point handle as well recursion level (as a second argument). Likely breaks a lot of tools Modified: pypy/trunk/pypy/jit/backend/llgraph/llimpl.py ============================================================================== --- pypy/trunk/pypy/jit/backend/llgraph/llimpl.py (original) +++ pypy/trunk/pypy/jit/backend/llgraph/llimpl.py Sun Nov 7 18:35:41 2010 @@ -152,7 +152,7 @@ 'unicodegetitem' : (('ref', 'int'), 'int'), 'unicodesetitem' : (('ref', 'int', 'int'), 'int'), 'cast_ptr_to_int' : (('ref',), 'int'), - 'debug_merge_point': (('ref',), None), + 'debug_merge_point': (('ref', 'int'), None), 'force_token' : ((), 'int'), 'call_may_force' : (('int', 'varargs'), 'intorptr'), 'guard_not_forced': ((), None), @@ -568,7 +568,7 @@ # return _op_default_implementation - def op_debug_merge_point(self, _, value): + def op_debug_merge_point(self, _, value, recdepth): from pypy.jit.metainterp.warmspot import get_stats loc = ConstPtr(value)._get_str() get_stats().add_merge_point_location(loc) Modified: pypy/trunk/pypy/jit/metainterp/logger.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/logger.py (original) +++ pypy/trunk/pypy/jit/metainterp/logger.py Sun Nov 7 18:35:41 2010 @@ -81,7 +81,8 @@ op = operations[i] if op.getopnum() == rop.DEBUG_MERGE_POINT: loc = op.getarg(0)._get_str() - debug_print("debug_merge_point('%s')" % (loc,)) + reclev = op.getarg(1).getint() + debug_print("debug_merge_point('%s', %s)" % (loc, reclev)) continue args = ", ".join([self.repr_of_arg(memo, op.getarg(i)) for i in range(op.numargs())]) if op.result is not None: Modified: pypy/trunk/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/trunk/pypy/jit/metainterp/pyjitpl.py Sun Nov 7 18:35:41 2010 @@ -820,7 +820,8 @@ jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex] self.verify_green_args(jitdriver_sd, greenboxes) # xxx we may disable the following line in some context later - self.debug_merge_point(jitdriver_sd, greenboxes) + self.debug_merge_point(jitdriver_sd, self.metainterp.in_recursion, + greenboxes) if self.metainterp.seen_loop_header_for_jdindex < 0: if not jitdriver_sd.no_loop_header or not any_operation: return @@ -860,13 +861,13 @@ assembler_call=True) raise ChangeFrame - def debug_merge_point(self, jitdriver_sd, greenkey): + def debug_merge_point(self, jitdriver_sd, in_recursion, greenkey): # debugging: produce a DEBUG_MERGE_POINT operation loc = jitdriver_sd.warmstate.get_location_str(greenkey) debug_print(loc) constloc = self.metainterp.cpu.ts.conststr(loc) self.metainterp.history.record(rop.DEBUG_MERGE_POINT, - [constloc], None) + [constloc, ConstInt(in_recursion)], None) @arguments("box", "label") def opimpl_goto_if_exception_mismatch(self, vtablebox, next_exc_target): Modified: pypy/trunk/pypy/jit/metainterp/resoperation.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/resoperation.py (original) +++ pypy/trunk/pypy/jit/metainterp/resoperation.py Sun Nov 7 18:35:41 2010 @@ -458,7 +458,7 @@ 'NEWUNICODE/1', #'RUNTIMENEW/1', # ootype operation 'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier) - 'DEBUG_MERGE_POINT/1', # debugging only + 'DEBUG_MERGE_POINT/2', # debugging only 'JIT_DEBUG/*', # debugging only 'VIRTUAL_REF_FINISH/2', # removed before it's passed to the backend 'COPYSTRCONTENT/5', # src, dst, srcstart, dststart, length Modified: pypy/trunk/pypy/jit/metainterp/test/test_logger.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_logger.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_logger.py Sun Nov 7 18:35:41 2010 @@ -97,7 +97,7 @@ def test_debug_merge_point(self): inp = ''' [] - debug_merge_point("info") + debug_merge_point("info", 0) ''' loop, oloop = self.reparse(inp, check_equal=False) assert loop.operations[0].getarg(0)._get_str() == 'info' Modified: pypy/trunk/pypy/jit/metainterp/test/test_oparser.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_oparser.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_oparser.py Sun Nov 7 18:35:41 2010 @@ -141,10 +141,10 @@ def test_debug_merge_point(): x = ''' [] - debug_merge_point("info") - debug_merge_point('info') - debug_merge_point(' info') - debug_merge_point('(stuff) #1') + debug_merge_point("info", 0) + debug_merge_point('info', 1) + debug_merge_point(' info', 1) + debug_merge_point('(stuff) #1', 1) ''' loop = parse(x) assert loop.operations[0].getarg(0)._get_str() == 'info' @@ -168,7 +168,7 @@ i6 = int_sub(i1, 1) i8 = int_gt(i6, 3) guard_true(i8, descr=) [i4, i6] -debug_merge_point('(no jitdriver.get_printable_location!)') +debug_merge_point('(no jitdriver.get_printable_location!)', 0) jump(i6, i4, descr=) ''' Modified: pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py Sun Nov 7 18:35:41 2010 @@ -1390,7 +1390,7 @@ ops = """ [p1] i1 = getfield_gc(p1, descr=valuedescr) - debug_merge_point(15) + debug_merge_point(15, 0) i2 = getfield_gc(p1, descr=valuedescr) escape(i1) escape(i2) @@ -1399,7 +1399,7 @@ expected = """ [p1] i1 = getfield_gc(p1, descr=valuedescr) - debug_merge_point(15) + debug_merge_point(15, 0) escape(i1) escape(i1) jump(p1) Modified: pypy/trunk/pypy/jit/tool/oparser.py ============================================================================== --- pypy/trunk/pypy/jit/tool/oparser.py (original) +++ pypy/trunk/pypy/jit/tool/oparser.py Sun Nov 7 18:35:41 2010 @@ -189,7 +189,7 @@ descr = None if argspec.strip(): if opname == 'debug_merge_point': - allargs = [argspec] + allargs = argspec.rsplit(', ', 1) else: allargs = [arg for arg in argspec.split(",") if arg != ''] From fijal at codespeak.net Sun Nov 7 18:36:56 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 7 Nov 2010 18:36:56 +0100 (CET) Subject: [pypy-svn] r78829 - pypy/trunk/pypy/jit/tool/test Message-ID: <20101107173656.CB5BA282B90@codespeak.net> Author: fijal Date: Sun Nov 7 18:36:55 2010 New Revision: 78829 Modified: pypy/trunk/pypy/jit/tool/test/test_traceviewer.py Log: update the test. It passes though Modified: pypy/trunk/pypy/jit/tool/test/test_traceviewer.py ============================================================================== --- pypy/trunk/pypy/jit/tool/test/test_traceviewer.py (original) +++ pypy/trunk/pypy/jit/tool/test/test_traceviewer.py Sun Nov 7 18:36:55 2010 @@ -19,7 +19,7 @@ def test_no_of_loops(self): data = [preparse(""" # Loop 0 : loop with 39 ops - debug_merge_point('') + debug_merge_point('', 0) guard_class(p4, 141310752, descr=) [p0, p1] p60 = getfield_gc(p4, descr=) guard_nonnull(p60, descr=) [p0, p1] @@ -51,7 +51,7 @@ assert loop.right.content == 'extra' def test_postparse(self): - real_loops = [FinalBlock("debug_merge_point(' #40 POP_TOP')", None)] + real_loops = [FinalBlock("debug_merge_point(' #40 POP_TOP', 0)", None)] postprocess(real_loops, real_loops[:], {}) assert real_loops[0].header.startswith("_runCallbacks, file '/tmp/x/twisted-trunk/twisted/internet/defer.py', line 357") From arigo at codespeak.net Sun Nov 7 18:59:08 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 18:59:08 +0100 (CET) Subject: [pypy-svn] r78830 - in pypy/branch/fast-forward/pypy: interpreter module/__builtin__ module/__builtin__/test Message-ID: <20101107175908.57200282B9E@codespeak.net> Author: arigo Date: Sun Nov 7 18:59:06 2010 New Revision: 78830 Added: pypy/branch/fast-forward/pypy/module/__builtin__/memoryview.py Modified: pypy/branch/fast-forward/pypy/interpreter/buffer.py pypy/branch/fast-forward/pypy/module/__builtin__/__init__.py pypy/branch/fast-forward/pypy/module/__builtin__/operation.py pypy/branch/fast-forward/pypy/module/__builtin__/test/test_buffer.py Log: Implement the built-in 'memoryview' type as a thin wrapper around an interp-level buffer. Modified: pypy/branch/fast-forward/pypy/interpreter/buffer.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/buffer.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/buffer.py Sun Nov 7 18:59:06 2010 @@ -207,7 +207,6 @@ __mul__ = interp2app(Buffer.descr_mul), __rmul__ = interp2app(Buffer.descr_mul), __repr__ = interp2app(Buffer.descr_repr), - tobytes = interp2app(Buffer.descr_str), ) Buffer.typedef.acceptable_as_base_class = False Modified: pypy/branch/fast-forward/pypy/module/__builtin__/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/__init__.py Sun Nov 7 18:59:06 2010 @@ -52,8 +52,8 @@ 'object' : '(space.w_object)', 'bytes' : '(space.w_str)', 'unicode' : '(space.w_unicode)', - 'buffer' : 'operation.W_Buffer', - 'memoryview' : 'operation.W_Buffer', + 'buffer' : 'memoryview.W_Buffer', + 'memoryview' : 'memoryview.W_MemoryView', 'file' : 'state.get(space).w_file', 'open' : 'state.get(space).w_file', Added: pypy/branch/fast-forward/pypy/module/__builtin__/memoryview.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/pypy/module/__builtin__/memoryview.py Sun Nov 7 18:59:06 2010 @@ -0,0 +1,138 @@ +""" +Implementation of the 'buffer' and 'memoryview' types. +""" +from pypy.interpreter.baseobjspace import Wrappable +from pypy.interpreter import gateway, buffer +from pypy.interpreter.gateway import interp2app, unwrap_spec, ObjSpace, W_Root +from pypy.interpreter.typedef import TypeDef, GetSetProperty +from pypy.interpreter.error import OperationError + +W_Buffer = buffer.Buffer # actually implemented in pypy.interpreter.buffer + + +class W_MemoryView(Wrappable): + """Implement the built-in 'memoryview' type as a thin wrapper around + an interp-level buffer. + """ + + def __init__(self, buf): + assert isinstance(buf, buffer.Buffer) + self.buf = buf + + def _make_descr__cmp(name): + def descr__cmp(self, space, w_other): + other = space.interpclass_w(w_other) + if not isinstance(other, W_MemoryView): + return space.w_NotImplemented + # xxx not the most efficient implementation + str1 = self.as_str() + str2 = other.as_str() + return space.wrap(getattr(operator, name)(str1, str2)) + descr__cmp.unwrap_spec = ['self', ObjSpace, W_Root] + descr__cmp.func_name = name + return descr__cmp + + descr_eq = _make_descr__cmp('eq') + descr_ne = _make_descr__cmp('ne') + descr_lt = _make_descr__cmp('lt') + descr_le = _make_descr__cmp('le') + descr_gt = _make_descr__cmp('gt') + descr_ge = _make_descr__cmp('ge') + + def as_str(self): + return self.buf.as_str() + + def getlength(self): + return self.buf.getlength() + + def getslice(self, start, stop): + if start < 0: + start = 0 + size = stop - start + if size < 0: + size = 0 + buf = self.buf + if isinstance(buf, buffer.RWBuffer): + buf = buffer.RWSubBuffer(buf, start, size) + else: + buf = buffer.SubBuffer(buf, start, size) + return W_MemoryView(buf) + + @unwrap_spec('self', ObjSpace) + def descr_buffer(self, space): + return space.wrap(self.buf) + + @unwrap_spec('self', ObjSpace) + def descr_tobytes(self, space): + return space.wrap(self.as_str()) + + @unwrap_spec('self', ObjSpace) + def descr_tolist(self, space): + buf = self.buf + result = [] + for i in range(buf.getlength()): + result.append(space.wrap(ord(buf.getitem(i)))) + return space.newlist(result) + + @unwrap_spec('self', ObjSpace, W_Root) + def descr_getitem(self, space, w_index): + start, stop, step = space.decode_index(w_index, self.getlength()) + if step == 0: # index only + return space.wrap(self.buf.getitem(start)) + elif step == 1: + res = self.getslice(start, stop) + return space.wrap(res) + else: + raise OperationError(space.w_ValueError, + space.wrap("memoryview object does not support" + " slicing with a step")) + + @unwrap_spec('self', ObjSpace, W_Root, 'bufferstr') + def descr_setitem(self, space, w_index, newstring): + buf = self.buf + if isinstance(buf, buffer.RWBuffer): + buf.descr_setitem(space, w_index, newstring) + else: + raise OperationError(space.w_TypeError, + space.wrap("cannot modify read-only memory")) + + @unwrap_spec('self', ObjSpace) + def descr_len(self, space): + return self.buf.descr_len(space) + + def is_readonly(space, self): + return space.wrap(not isinstance(self.buf, buffer.RWBuffer)) + + + at unwrap_spec(ObjSpace, W_Root, W_Root) +def descr_new(space, w_subtype, w_object): + memoryview = W_MemoryView(space.buffer(w_object)) + return space.wrap(memoryview) + +W_MemoryView.typedef = TypeDef( + "memoryview", + __doc__ = """\ +Create a new memoryview object which references the given object. +""", + __new__ = interp2app(descr_new), + __buffer__ = interp2app(W_MemoryView.descr_buffer), + __eq__ = interp2app(W_MemoryView.descr_eq), + __ge__ = interp2app(W_MemoryView.descr_ge), + __getitem__ = interp2app(W_MemoryView.descr_getitem), + __gt__ = interp2app(W_MemoryView.descr_gt), + __le__ = interp2app(W_MemoryView.descr_le), + __len__ = interp2app(W_MemoryView.descr_len), + __lt__ = interp2app(W_MemoryView.descr_lt), + __ne__ = interp2app(W_MemoryView.descr_ne), + __setitem__ = interp2app(W_MemoryView.descr_setitem), + tobytes = interp2app(W_MemoryView.descr_tobytes), + tolist = interp2app(W_MemoryView.descr_tolist), + #format + #itemsize + #ndim + readonly = GetSetProperty(W_MemoryView.is_readonly) + #shape + #strides + #suboffsets + ) +W_MemoryView.typedef.acceptable_as_base_class = False Modified: pypy/branch/fast-forward/pypy/module/__builtin__/operation.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/operation.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/operation.py Sun Nov 7 18:59:06 2010 @@ -2,10 +2,10 @@ Interp-level implementation of the basic space operations. """ -from pypy.interpreter import gateway, buffer +from pypy.interpreter import gateway from pypy.interpreter.baseobjspace import ObjSpace, W_Root from pypy.interpreter.error import OperationError -from pypy.interpreter.gateway import interp2app, unwrap_spec +from pypy.interpreter.gateway import interp2app from pypy.interpreter.typedef import TypeDef from pypy.rlib.runicode import UNICHR from pypy.rlib.rarithmetic import isnan, isinf @@ -14,8 +14,6 @@ import __builtin__ NoneNotWrapped = gateway.NoneNotWrapped -W_Buffer = buffer.Buffer - def abs(space, w_val): "abs(number) -> number\n\nReturn the absolute value of the argument." return space.abs(w_val) Modified: pypy/branch/fast-forward/pypy/module/__builtin__/test/test_buffer.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/test/test_buffer.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/test/test_buffer.py Sun Nov 7 18:59:06 2010 @@ -160,9 +160,27 @@ raises(ValueError, buffer, a, -1) raises(ValueError, buffer, a, 0, -2) -class AppTestMemoryview: +class AppTestMemoryView: def test_basic(self): v = memoryview("abc") assert v.tobytes() == "abc" assert len(v) == 3 assert list(v) == ['a', 'b', 'c'] + assert v.tolist() == [97, 98, 99] + assert v[1] == "b" + assert v[-1] == "c" + raises(TypeError, "v[1] = 'x'") + assert v.readonly is True + w = v[1:234] + assert isinstance(w, memoryview) + assert len(w) == 2 + + def test_rw(self): + data = bytearray('abcefg') + v = memoryview(data) + assert v.readonly is False + v[0] = 'z' + assert data == bytearray(b'zbcefg') + v[1:4] = '123' + assert data == bytearray(b'z123fg') + raises((ValueError, TypeError), "v[2] = 'spam'") From arigo at codespeak.net Sun Nov 7 19:01:25 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 19:01:25 +0100 (CET) Subject: [pypy-svn] r78831 - pypy/branch/fast-forward/lib-python Message-ID: <20101107180125.0616B282BDA@codespeak.net> Author: arigo Date: Sun Nov 7 19:01:24 2010 New Revision: 78831 Modified: pypy/branch/fast-forward/lib-python/TODO Log: Mark as mostly DONE. Modified: pypy/branch/fast-forward/lib-python/TODO ============================================================================== --- pypy/branch/fast-forward/lib-python/TODO (original) +++ pypy/branch/fast-forward/lib-python/TODO Sun Nov 7 19:01:24 2010 @@ -42,7 +42,7 @@ - Ast objects should be picklable, see in pypy/module/_ast/test/test_ast.py: test_pickle() -- missing builtin: memoryview +- missing builtin: memoryview DONE (but not very much tested) - signal.set_wakeup_fd() DONE From arigo at codespeak.net Sun Nov 7 19:10:20 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 7 Nov 2010 19:10:20 +0100 (CET) Subject: [pypy-svn] r78832 - in pypy/branch/fast-forward/pypy/module/__builtin__: . test Message-ID: <20101107181020.C6CDF282BDD@codespeak.net> Author: arigo Date: Sun Nov 7 19:10:19 2010 New Revision: 78832 Added: pypy/branch/fast-forward/pypy/module/__builtin__/interp_memoryview.py - copied, changed from r78830, pypy/branch/fast-forward/pypy/module/__builtin__/memoryview.py Removed: pypy/branch/fast-forward/pypy/module/__builtin__/memoryview.py Modified: pypy/branch/fast-forward/pypy/module/__builtin__/__init__.py pypy/branch/fast-forward/pypy/module/__builtin__/test/test_buffer.py Log: * rename memoryview.py into interp_memoryview.py to avoid conflicts with the built-in name from 2.7. * implement the (dummy) attributes. Modified: pypy/branch/fast-forward/pypy/module/__builtin__/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/__init__.py Sun Nov 7 19:10:19 2010 @@ -52,8 +52,8 @@ 'object' : '(space.w_object)', 'bytes' : '(space.w_str)', 'unicode' : '(space.w_unicode)', - 'buffer' : 'memoryview.W_Buffer', - 'memoryview' : 'memoryview.W_MemoryView', + 'buffer' : 'interp_memoryview.W_Buffer', + 'memoryview' : 'interp_memoryview.W_MemoryView', 'file' : 'state.get(space).w_file', 'open' : 'state.get(space).w_file', Copied: pypy/branch/fast-forward/pypy/module/__builtin__/interp_memoryview.py (from r78830, pypy/branch/fast-forward/pypy/module/__builtin__/memoryview.py) ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/memoryview.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/interp_memoryview.py Sun Nov 7 19:10:19 2010 @@ -100,8 +100,18 @@ def descr_len(self, space): return self.buf.descr_len(space) - def is_readonly(space, self): + def w_get_format(space, self): + return space.wrap("B") + def w_get_itemsize(space, self): + return space.wrap(1) + def w_get_ndim(space, self): + return space.wrap(1) + def w_is_readonly(space, self): return space.wrap(not isinstance(self.buf, buffer.RWBuffer)) + def w_get_shape(space, self): + return space.newtuple([space.wrap(self.getlength())]) + def w_get_strides(space, self): + return space.newtuple([space.wrap(1)]) @unwrap_spec(ObjSpace, W_Root, W_Root) @@ -127,12 +137,12 @@ __setitem__ = interp2app(W_MemoryView.descr_setitem), tobytes = interp2app(W_MemoryView.descr_tobytes), tolist = interp2app(W_MemoryView.descr_tolist), - #format - #itemsize - #ndim - readonly = GetSetProperty(W_MemoryView.is_readonly) - #shape - #strides - #suboffsets + format = GetSetProperty(W_MemoryView.w_get_format), + itemsize = GetSetProperty(W_MemoryView.w_get_itemsize), + ndim = GetSetProperty(W_MemoryView.w_get_ndim), + readonly = GetSetProperty(W_MemoryView.w_is_readonly), + shape = GetSetProperty(W_MemoryView.w_get_shape), + strides = GetSetProperty(W_MemoryView.w_get_strides), + #suboffsets? undocumented ) W_MemoryView.typedef.acceptable_as_base_class = False Modified: pypy/branch/fast-forward/pypy/module/__builtin__/test/test_buffer.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/test/test_buffer.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/test/test_buffer.py Sun Nov 7 19:10:19 2010 @@ -184,3 +184,11 @@ v[1:4] = '123' assert data == bytearray(b'z123fg') raises((ValueError, TypeError), "v[2] = 'spam'") + + def test_memoryview_attrs(self): + v = memoryview("a"*100) + assert v.format == "B" + assert v.itemsize == 1 + assert v.shape == (100,) + assert v.ndim == 1 + assert v.strides == (1,) From afa at codespeak.net Sun Nov 7 20:38:43 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 20:38:43 +0100 (CET) Subject: [pypy-svn] r78833 - in pypy/branch/fast-forward/pypy/module/cpyext: . test Message-ID: <20101107193843.7B43F282B90@codespeak.net> Author: afa Date: Sun Nov 7 20:38:40 2010 New Revision: 78833 Added: pypy/branch/fast-forward/pypy/module/cpyext/memoryobject.py (contents, props changed) pypy/branch/fast-forward/pypy/module/cpyext/test/test_memoryobject.py (contents, props changed) Modified: pypy/branch/fast-forward/pypy/module/cpyext/__init__.py Log: Add PyMemoryView_FromObject Modified: pypy/branch/fast-forward/pypy/module/cpyext/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/__init__.py Sun Nov 7 20:38:40 2010 @@ -69,6 +69,7 @@ import pypy.module.cpyext.weakrefobject import pypy.module.cpyext.funcobject import pypy.module.cpyext.classobject +import pypy.module.cpyext.memoryobject # now that all rffi_platform.Struct types are registered, configure them api.configure_types() Added: pypy/branch/fast-forward/pypy/module/cpyext/memoryobject.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/pypy/module/cpyext/memoryobject.py Sun Nov 7 20:38:40 2010 @@ -0,0 +1,6 @@ +from pypy.module.cpyext.api import cpython_api +from pypy.module.cpyext.pyobject import PyObject + + at cpython_api([PyObject], PyObject) +def PyMemoryView_FromObject(space, w_obj): + return space.call_method(space.builtin, "memoryview", w_obj) Added: pypy/branch/fast-forward/pypy/module/cpyext/test/test_memoryobject.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_memoryobject.py Sun Nov 7 20:38:40 2010 @@ -0,0 +1,8 @@ +from pypy.module.cpyext.test.test_api import BaseApiTest + +class TestMemoryViewObject(BaseApiTest): + def test_fromobject(self, space, api): + w_hello = space.wrap("hello") + w_view = api.PyMemoryView_FromObject(w_hello) + w_bytes = space.call_method(w_view, "tobytes") + assert space.unwrap(w_bytes) == "hello" From afa at codespeak.net Sun Nov 7 20:38:57 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 20:38:57 +0100 (CET) Subject: [pypy-svn] r78834 - pypy/branch/fast-forward/pypy/module/cpyext/include Message-ID: <20101107193857.4ECBF282B9E@codespeak.net> Author: afa Date: Sun Nov 7 20:38:55 2010 New Revision: 78834 Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/complexobject.h (contents, props changed) pypy/branch/fast-forward/pypy/module/cpyext/include/datetime.h (contents, props changed) Log: fixeol Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/complexobject.h ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/include/complexobject.h (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/complexobject.h Sun Nov 7 20:38:55 2010 @@ -1,27 +1,27 @@ -/* Complex object interface */ - -#ifndef Py_COMPLEXOBJECT_H -#define Py_COMPLEXOBJECT_H -#ifdef __cplusplus -extern "C" { -#endif - -typedef struct Py_complex_t { - double real; - double imag; -} Py_complex; - -/* generated function */ -PyAPI_FUNC(void) _PyComplex_AsCComplex(PyObject *, Py_complex *); - -Py_LOCAL_INLINE(Py_complex) PyComplex_AsCComplex(PyObject *obj) -{ - Py_complex result; - _PyComplex_AsCComplex(obj, &result); - return result; -} - -#ifdef __cplusplus -} -#endif -#endif /* !Py_COMPLEXOBJECT_H */ +/* Complex object interface */ + +#ifndef Py_COMPLEXOBJECT_H +#define Py_COMPLEXOBJECT_H +#ifdef __cplusplus +extern "C" { +#endif + +typedef struct Py_complex_t { + double real; + double imag; +} Py_complex; + +/* generated function */ +PyAPI_FUNC(void) _PyComplex_AsCComplex(PyObject *, Py_complex *); + +Py_LOCAL_INLINE(Py_complex) PyComplex_AsCComplex(PyObject *obj) +{ + Py_complex result; + _PyComplex_AsCComplex(obj, &result); + return result; +} + +#ifdef __cplusplus +} +#endif +#endif /* !Py_COMPLEXOBJECT_H */ Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/datetime.h ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/include/datetime.h (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/datetime.h Sun Nov 7 20:38:55 2010 @@ -1,30 +1,30 @@ -#ifndef DATETIME_H -#define DATETIME_H -#ifdef __cplusplus -extern "C" { -#endif - -/* Define structure for C API. */ -typedef struct { - /* type objects */ - PyTypeObject *DateType; - PyTypeObject *DateTimeType; - PyTypeObject *TimeType; - PyTypeObject *DeltaType; -} PyDateTime_CAPI; - -PyAPI_DATA(PyDateTime_CAPI*) PyDateTimeAPI; -#define PyDateTime_IMPORT \ - do { \ - if(PyDateTimeAPI==NULL) \ - PyDateTimeAPI = _PyDateTime_Import(); \ - } while (0) - -typedef struct { - PyObject_HEAD -} PyDateTime_Delta; - -#ifdef __cplusplus -} -#endif -#endif +#ifndef DATETIME_H +#define DATETIME_H +#ifdef __cplusplus +extern "C" { +#endif + +/* Define structure for C API. */ +typedef struct { + /* type objects */ + PyTypeObject *DateType; + PyTypeObject *DateTimeType; + PyTypeObject *TimeType; + PyTypeObject *DeltaType; +} PyDateTime_CAPI; + +PyAPI_DATA(PyDateTime_CAPI*) PyDateTimeAPI; +#define PyDateTime_IMPORT \ + do { \ + if(PyDateTimeAPI==NULL) \ + PyDateTimeAPI = _PyDateTime_Import(); \ + } while (0) + +typedef struct { + PyObject_HEAD +} PyDateTime_Delta; + +#ifdef __cplusplus +} +#endif +#endif From dan at codespeak.net Sun Nov 7 20:39:50 2010 From: dan at codespeak.net (dan at codespeak.net) Date: Sun, 7 Nov 2010 20:39:50 +0100 (CET) Subject: [pypy-svn] r78835 - pypy/branch/ootype-virtualrefs/pypy/rlib Message-ID: <20101107193950.EDD10282BDA@codespeak.net> Author: dan Date: Sun Nov 7 20:39:49 2010 New Revision: 78835 Modified: pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py pypy/branch/ootype-virtualrefs/pypy/rlib/jit.py Log: We now make it through rtyping, and rlib/test/test_jit.py passes. Modified: pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py ============================================================================== --- pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py (original) +++ pypy/branch/ootype-virtualrefs/pypy/rlib/_jit_vref.py Sun Nov 7 20:39:49 2010 @@ -66,7 +66,6 @@ def rtype_simple_call(self, hop): [v] = hop.inputargs(self) v = hop.genop('jit_force_virtual', [v], resulttype = OBJECT) - #RESULTTYPE, ptr return hop.genop('oodowncast', [v], resulttype = hop.r_result) def convert_const(self, value): Modified: pypy/branch/ootype-virtualrefs/pypy/rlib/jit.py ============================================================================== --- pypy/branch/ootype-virtualrefs/pypy/rlib/jit.py (original) +++ pypy/branch/ootype-virtualrefs/pypy/rlib/jit.py Sun Nov 7 20:39:49 2010 @@ -457,8 +457,8 @@ r_green = hop.args_r[i] v_green = hop.inputarg(r_green, arg=i) else: - if hop.rtyper.type_system.name == 'ootypesystem': - py.test.skip("lltype only") + #if hop.rtyper.type_system.name == 'ootypesystem': + #py.test.skip("lltype only") objname, fieldname = name.split('.') # see test_green_field assert objname in driver.reds i = kwds_i['i_' + objname] @@ -474,7 +474,10 @@ "field %r not found in %r" % (name, r_red.lowleveltype.TO)) r_red = r_red.rbase - GTYPE = r_red.lowleveltype.TO + if hop.rtyper.type_system.name == 'ootypesystem': + GTYPE = r_red.lowleveltype + else: + GTYPE = r_red.lowleveltype.TO assert GTYPE._immutable_field(mangled_name), ( "field %r must be declared as immutable" % name) if not hasattr(driver, 'll_greenfields'): From afa at codespeak.net Sun Nov 7 20:48:07 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 20:48:07 +0100 (CET) Subject: [pypy-svn] r78836 - pypy/branch/fast-forward/pypy/module/cpyext/include Message-ID: <20101107194807.CAA02282BDA@codespeak.net> Author: afa Date: Sun Nov 7 20:48:06 2010 New Revision: 78836 Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/pyconfig.h pypy/branch/fast-forward/pypy/module/cpyext/include/pyport.h Log: Add constants: PY_LLONG_MAX &co Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/pyconfig.h ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/include/pyconfig.h (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/pyconfig.h Sun Nov 7 20:48:06 2010 @@ -10,7 +10,6 @@ #define HAVE_LONG_LONG 1 #define HAVE_STDARG_PROTOTYPES 1 #define PY_FORMAT_LONG_LONG "ll" -#define PY_LONG_LONG long long #define PY_FORMAT_SIZE_T "z" #define WITH_DOC_STRINGS #define HAVE_UNICODE Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/pyport.h ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/include/pyport.h (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/pyport.h Sun Nov 7 20:48:06 2010 @@ -5,6 +5,29 @@ #include #endif +/* typedefs for some C9X-defined synonyms for integral types. */ +#ifdef HAVE_LONG_LONG +#ifndef PY_LONG_LONG +#define PY_LONG_LONG long long +#if defined(LLONG_MAX) +/* If LLONG_MAX is defined in limits.h, use that. */ +#define PY_LLONG_MIN LLONG_MIN +#define PY_LLONG_MAX LLONG_MAX +#define PY_ULLONG_MAX ULLONG_MAX +#elif defined(__LONG_LONG_MAX__) +/* Otherwise, if GCC has a builtin define, use that. */ +#define PY_LLONG_MAX __LONG_LONG_MAX__ +#define PY_LLONG_MIN (-PY_LLONG_MAX-1) +#define PY_ULLONG_MAX (__LONG_LONG_MAX__*2ULL + 1ULL) +#else +/* Otherwise, rely on two's complement. */ +#define PY_ULLONG_MAX (~0ULL) +#define PY_LLONG_MAX ((long long)(PY_ULLONG_MAX>>1)) +#define PY_LLONG_MIN (-PY_LLONG_MAX-1) +#endif /* LLONG_MAX */ +#endif +#endif /* HAVE_LONG_LONG */ + /* Largest possible value of size_t. SIZE_MAX is part of C99, so it might be defined on some platforms. If it is not defined, (size_t)-1 is a portable From fijal at codespeak.net Sun Nov 7 20:54:12 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 7 Nov 2010 20:54:12 +0100 (CET) Subject: [pypy-svn] r78837 - pypy/extradoc/planning Message-ID: <20101107195412.8A09B282BDC@codespeak.net> Author: fijal Date: Sun Nov 7 20:54:10 2010 New Revision: 78837 Modified: pypy/extradoc/planning/jit.txt Log: Add an item before I forget Modified: pypy/extradoc/planning/jit.txt ============================================================================== --- pypy/extradoc/planning/jit.txt (original) +++ pypy/extradoc/planning/jit.txt Sun Nov 7 20:54:10 2010 @@ -86,6 +86,20 @@ - xxx (find more examples :-) +BACKEND TASKS +------------- + +Look into avoiding double load of memory into register on 64bit. +In case we want to first read a value, increment it and store (for example), +we end up with double load of memory into register. Like: + +movabs 0xsomemem,r11 +mov (r11), r10 +add 0x1, r10 +movabs 0xsomemem,r11 +mov r10, (r11) + +(second movabs could have been avoided) LATER (maybe) TASKS ------------------- From fijal at codespeak.net Sun Nov 7 20:56:42 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 7 Nov 2010 20:56:42 +0100 (CET) Subject: [pypy-svn] r78838 - in pypy/branch/fast-forward: lib-python pypy/module/itertools pypy/module/itertools/test Message-ID: <20101107195642.8C7BD282BDC@codespeak.net> Author: fijal Date: Sun Nov 7 20:56:41 2010 New Revision: 78838 Modified: pypy/branch/fast-forward/lib-python/TODO pypy/branch/fast-forward/pypy/module/itertools/__init__.py pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py pypy/branch/fast-forward/pypy/module/itertools/test/test_itertools.py Log: (dcolish) product and compress for itertools Modified: pypy/branch/fast-forward/lib-python/TODO ============================================================================== --- pypy/branch/fast-forward/lib-python/TODO (original) +++ pypy/branch/fast-forward/lib-python/TODO Sun Nov 7 20:56:41 2010 @@ -27,6 +27,7 @@ - Fix fcntl.fcntl(fd, fcntl.F_NOTIFY, fcntl.DN_MULTISHOT) on 32bit platform. - missing functions in itertools: combinations, product, compress... + compress, product DONE - in test_os.py, fix posix.setregid(-1, -1), posix.setreuid(-1, -1). This proably requires to use the git_t typedef instead of rffi.INT. Modified: pypy/branch/fast-forward/pypy/module/itertools/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/itertools/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/itertools/__init__.py Sun Nov 7 20:56:41 2010 @@ -26,6 +26,7 @@ interpleveldefs = { 'chain' : 'interp_itertools.W_Chain', + 'compress' : 'interp_itertools.W_Compress', 'count' : 'interp_itertools.W_Count', 'cycle' : 'interp_itertools.W_Cycle', 'dropwhile' : 'interp_itertools.W_DropWhile', @@ -36,6 +37,7 @@ 'islice' : 'interp_itertools.W_ISlice', 'izip' : 'interp_itertools.W_IZip', 'izip_longest' : 'interp_itertools.W_IZipLongest', + 'product' : 'interp_itertools.W_Product', 'repeat' : 'interp_itertools.W_Repeat', 'starmap' : 'interp_itertools.W_StarMap', 'takewhile' : 'interp_itertools.W_TakeWhile', Modified: pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py (original) +++ pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py Sun Nov 7 20:56:41 2010 @@ -942,3 +942,137 @@ __iter__ = interp2app(W_GroupByIterator.iter_w, unwrap_spec=['self']), next = interp2app(W_GroupByIterator.next_w, unwrap_spec=['self'])) W_GroupByIterator.typedef.acceptable_as_base_class = False + + +class W_Compress(Wrappable): + def __init__(self, space, w_data, w_selectors): + self.space = space + self.w_data = space.iter(w_data) + self.w_selectors = space.iter(w_selectors) + + def iter_w(self): + return self.space.wrap(self) + + def next_w(self): + if not self.w_data or not self.w_selectors: + raise OperationError(self.space.w_StopIteration, self.space.w_None) + + while True: + w_next_item = self.space.next(self.w_data) + w_next_selector = self.space.next(self.w_selectors) + if self.space.is_true(w_next_selector): + return w_next_item + + +def W_Compress__new__(space, w_subtype, w_data, w_selectors): + return space.wrap(W_Compress(space, w_data, w_selectors)) + +W_Compress.typedef = TypeDef( + 'compress', + __new__ = interp2app(W_Compress__new__, + unwrap_spec=[ObjSpace, W_Root, W_Root, W_Root]), + __iter__ = interp2app(W_Compress.iter_w, unwrap_spec=['self']), + next = interp2app(W_Compress.next_w, unwrap_spec=['self']), + __doc__ = """Make an iterator that filters elements from *data* returning + only those that have a corresponding element in *selectors* that evaluates to + ``True``. Stops when either the *data* or *selectors* iterables has been + exhausted. + Equivalent to:: + + def compress(data, selectors): + # compress('ABCDEF', [1,0,1,0,1,1]) --> A C E F + return (d for d, s in izip(data, selectors) if s) +""") + + +class W_Product(Wrappable): + + def __init__(self, space, args_w, repeat_w): + self.space = space + self.gears_w = [x for x in args_w] * repeat_w.intval + self.num_gears = len(self.gears_w) + # initialization of indicies to loop over + self.indicies = [(0, len(self.gears_w[x].wrappeditems)) + for x in range(0, self.num_gears)] + self.cont = True + + def roll_gears(self): + # Starting from the end of the gear indicies work to the front + # incrementing the gear until the limit is reached. When the limit + # is reached carry operation to the next gear + should_carry = True + + for n in range(0, self.num_gears): + nth_gear = self.num_gears - n - 1 + if should_carry: + count, lim = self.indicies[nth_gear] + count += 1 + if count == lim and nth_gear == 0: + self.cont = False + if count == lim: + should_carry = True + count = 0 + else: + should_carry = False + self.indicies[nth_gear] = (count, lim) + else: + break + + def iter_w(self): + return self.space.wrap(self) + + def next_w(self): + if not self.cont: + raise OperationError(self.space.w_StopIteration, + self.space.w_None) + l = [] + for x in range(0, self.num_gears): + index, limit = self.indicies[x] + l.append(self.gears_w[x].wrappeditems[index]) + self.roll_gears() + return self.space.newtuple(l) + + +def W_Product__new__(space, args_w): + star_args_w, kw_args_w = args_w.unpack() + if len(kw_args_w) > 1: + raise OperationError(space.w_TypeError, + space.wrap("product() takes at most 1 argument (%d given)" % + len(kw_args_w))) + repeat = kw_args_w.get('repeat', space.wrap(1)) + return space.wrap(W_Product(space, star_args_w[1:], repeat)) + +W_Product.typedef = TypeDef( + 'product', + __new__ = interp2app(W_Product__new__, + unwrap_spec=[ObjSpace, Arguments]), + __iter__ = interp2app(W_Product.iter_w, unwrap_spec=['self']), + next = interp2app(W_Product.next_w, unwrap_spec=['self']), + __doc__ = """ + Cartesian product of input iterables. + + Equivalent to nested for-loops in a generator expression. For example, + ``product(A, B)`` returns the same as ``((x,y) for x in A for y in B)``. + + The nested loops cycle like an odometer with the rightmost element advancing + on every iteration. This pattern creates a lexicographic ordering so that if + the input's iterables are sorted, the product tuples are emitted in sorted + order. + + To compute the product of an iterable with itself, specify the number of + repetitions with the optional *repeat* keyword argument. For example, + ``product(A, repeat=4)`` means the same as ``product(A, A, A, A)``. + + This function is equivalent to the following code, except that the + actual implementation does not build up intermediate results in memory:: + + def product(*args, **kwds): + # product('ABCD', 'xy') --> Ax Ay Bx By Cx Cy Dx Dy + # product(range(2), repeat=3) --> 000 001 010 011 100 101 110 111 + pools = map(tuple, args) * kwds.get('repeat', 1) + result = [[]] + for pool in pools: + result = [x+[y] for x in result for y in pool] + for prod in result: + yield tuple(prod) +""") Modified: pypy/branch/fast-forward/pypy/module/itertools/test/test_itertools.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/itertools/test/test_itertools.py (original) +++ pypy/branch/fast-forward/pypy/module/itertools/test/test_itertools.py Sun Nov 7 20:56:41 2010 @@ -4,6 +4,19 @@ def setup_class(cls): cls.space = gettestobjspace(usemodules=['itertools']) + def test_compress(self): + import itertools + + it = itertools.compress(['a', 'b', 'c'], [0, 1, 0]) + + assert list(it) == ['b'] + + def test_compress_diff_len(self): + import itertools + + it = itertools.compress(['a'], []) + raises(StopIteration, it.next) + def test_count(self): import itertools @@ -698,3 +711,44 @@ assert it.next() == (1, 2) assert it.next()== (1, 2) raises(RuntimeError, it.next) + + def test_product(self): + from itertools import product + l = [1, 2] + m = ['a', 'b'] + + prodlist = product(l, m) + assert list(prodlist) == [(1, 'a'), (1, 'b'), (2, 'a'), (2, 'b')] + + def test_product_repeat(self): + from itertools import product + l = [1, 2] + m = ['a', 'b'] + + prodlist = product(l, m, repeat=2) + ans = [(1, 'a', 1, 'a'), (1, 'a', 1, 'b'), (1, 'a', 2, 'a'), + (1, 'a', 2, 'b'), (1, 'b', 1, 'a'), (1, 'b', 1, 'b'), + (1, 'b', 2, 'a'), (1, 'b', 2, 'b'), (2, 'a', 1, 'a'), + (2, 'a', 1, 'b'), (2, 'a', 2, 'a'), (2, 'a', 2, 'b'), + (2, 'b', 1, 'a'), (2, 'b', 1, 'b'), (2, 'b', 2, 'a'), + (2, 'b', 2, 'b')] + assert list(prodlist) == ans + + def test_product_diff_sizes(self): + from itertools import product + l = [1, 2] + m = ['a'] + + prodlist = product(l, m) + assert list(prodlist) == [(1, 'a'), (2, 'a')] + + l = [1] + m = ['a', 'b'] + prodlist = product(l, m) + assert list(prodlist) == [(1, 'a'), (1, 'b')] + + def test_product_toomany_args(self): + from itertools import product + l = [1, 2] + m = ['a'] + raises(TypeError, product, l, m, repeat=1, foo=2) From fijal at codespeak.net Sun Nov 7 21:02:43 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 7 Nov 2010 21:02:43 +0100 (CET) Subject: [pypy-svn] r78839 - pypy/trunk/pypy/jit/backend/x86 Message-ID: <20101107200243.B9AAE282BE0@codespeak.net> Author: fijal Date: Sun Nov 7 21:01:58 2010 New Revision: 78839 Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py Log: I think it's fine to count bridges only *once* Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/assembler.py (original) +++ pypy/trunk/pypy/jit/backend/x86/assembler.py Sun Nov 7 21:01:58 2010 @@ -358,7 +358,6 @@ assert ([loc.assembler() for loc in arglocs] == [loc.assembler() for loc in faildescr._x86_debug_faillocs]) regalloc = RegAlloc(self, self.cpu.translate_support_code) - operations = self._inject_debugging_code(operations) fail_depths = faildescr._x86_current_depths regalloc.prepare_bridge(fail_depths, inputargs, arglocs, operations) From afa at codespeak.net Sun Nov 7 21:12:28 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 21:12:28 +0100 (CET) Subject: [pypy-svn] r78840 - pypy/branch/fast-forward/pypy/module/cpyext/include Message-ID: <20101107201228.29CC3282B9E@codespeak.net> Author: afa Date: Sun Nov 7 21:12:14 2010 New Revision: 78840 Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/Python.h Log: Fix #include order: should fix definition of PY_LONG_LONG Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/Python.h ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/include/Python.h (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/Python.h Sun Nov 7 21:12:14 2010 @@ -78,6 +78,7 @@ #include #include "patchlevel.h" +#include "pyconfig.h" #include "object.h" #include "pyport.h" @@ -90,8 +91,6 @@ #include #include -#include "pyconfig.h" - #include "boolobject.h" #include "floatobject.h" #include "complexobject.h" From afa at codespeak.net Sun Nov 7 21:36:58 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 21:36:58 +0100 (CET) Subject: [pypy-svn] r78841 - in pypy/branch/fast-forward/pypy/module/cpyext: . test Message-ID: <20101107203658.BEFFC282B9D@codespeak.net> Author: afa Date: Sun Nov 7 21:36:55 2010 New Revision: 78841 Modified: pypy/branch/fast-forward/pypy/module/cpyext/intobject.py pypy/branch/fast-forward/pypy/module/cpyext/longobject.py pypy/branch/fast-forward/pypy/module/cpyext/stubs.py pypy/branch/fast-forward/pypy/module/cpyext/test/test_intobject.py Log: Implement PyInt_AsUnsignedLongMask Modified: pypy/branch/fast-forward/pypy/module/cpyext/intobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/intobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/intobject.py Sun Nov 7 21:36:55 2010 @@ -2,7 +2,7 @@ from pypy.rpython.lltypesystem import rffi, lltype from pypy.module.cpyext.api import (cpython_api, PyObject, CANNOT_FAIL, build_type_checkers, Py_ssize_t) - +from pypy.rlib.rarithmetic import r_uint PyInt_Check, PyInt_CheckExact = build_type_checkers("Int") @@ -28,6 +28,20 @@ raised.""" return space.uint_w(space.int(w_obj)) + at cpython_api([PyObject], rffi.ULONG, error=-1) +def PyInt_AsUnsignedLongMask(space, w_obj): + """Will first attempt to cast the object to a PyIntObject or + PyLongObject, if it is not already one, and then return its value as + unsigned long. This function does not check for overflow. + """ + w_int = space.int(w_obj) + if space.is_true(space.isinstance(w_int, space.w_int)): + num = space.int_w(w_int) + return r_uint(num) + else: + num = space.bigint_w(w_int) + return num.uintmask() + @cpython_api([PyObject], lltype.Signed, error=CANNOT_FAIL) def PyInt_AS_LONG(space, w_int): """Return the value of the object w_int. No error checking is performed.""" Modified: pypy/branch/fast-forward/pypy/module/cpyext/longobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/longobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/longobject.py Sun Nov 7 21:36:55 2010 @@ -3,6 +3,7 @@ CONST_STRING, ADDR) from pypy.objspace.std.longobject import W_LongObject from pypy.interpreter.error import OperationError +from pypy.module.cpyext.intobject import PyInt_AsUnsignedLongMask PyLong_Check, PyLong_CheckExact = build_type_checkers("Long") @@ -38,6 +39,13 @@ raised.""" return rffi.cast(rffi.ULONG, space.uint_w(w_long)) + at cpython_api([PyObject], rffi.ULONG, error=-1) +def PyLong_AsUnsignedLongMask(space, w_long): + """Return a C unsigned long from a Python long integer, without checking + for overflow. + """ + return PyInt_AsUnsignedLongMask(space, w_long) + @cpython_api([PyObject], lltype.Signed, error=-1) def PyLong_AsLong(space, w_long): """ Modified: pypy/branch/fast-forward/pypy/module/cpyext/stubs.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/stubs.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/stubs.py Sun Nov 7 21:36:55 2010 @@ -1873,14 +1873,6 @@ """ raise NotImplementedError - at cpython_api([PyObject], rffi.ULONG, error=-1) -def PyInt_AsUnsignedLongMask(space, io): - """Will first attempt to cast the object to a PyIntObject or - PyLongObject, if it is not already one, and then return its value as - unsigned long. This function does not check for overflow. - """ - raise NotImplementedError - @cpython_api([PyObject], rffi.LONGLONG, error=-1) def PyInt_AsUnsignedLongLongMask(space, io): """Will first attempt to cast the object to a PyIntObject or @@ -2013,13 +2005,6 @@ """ raise NotImplementedError - at cpython_api([PyObject], rffi.ULONG, error=-1) -def PyLong_AsUnsignedLongMask(space, io): - """Return a C unsigned long from a Python long integer, without checking - for overflow. - """ - raise NotImplementedError - @cpython_api([PyObject], rffi.ULONGLONG, error=-1) def PyLong_AsUnsignedLongLongMask(space, io): """Return a C unsigned long long from a Python long integer, without Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_intobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/test/test_intobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_intobject.py Sun Nov 7 21:36:55 2010 @@ -24,6 +24,11 @@ assert api.PyErr_Occurred() is space.w_ValueError api.PyErr_Clear() + assert (api.PyInt_AsUnsignedLongMask(space.wrap(sys.maxint)) + == sys.maxint) + assert (api.PyInt_AsUnsignedLongMask(space.wrap(10**30)) + == 10**30 % ((sys.maxint + 1) * 2)) + def test_coerce(self, space, api): class Coerce(object): def __int__(self): From afa at codespeak.net Sun Nov 7 21:53:49 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 21:53:49 +0100 (CET) Subject: [pypy-svn] r78842 - in pypy/branch/fast-forward/pypy/module/cpyext: . test Message-ID: <20101107205349.884E0282B9E@codespeak.net> Author: afa Date: Sun Nov 7 21:53:42 2010 New Revision: 78842 Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py pypy/branch/fast-forward/pypy/module/cpyext/unicodeobject.py Log: Add PyUnicode_FromString, PyUnicode_FromStringAndSize Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py Sun Nov 7 21:53:42 2010 @@ -65,6 +65,15 @@ assert rffi.wcharp2unicode(buf) == 'a' rffi.free_wcharp(buf) + def test_fromstring(self, space, api): + s = rffi.str2charp(u'sp?m'.encode("utf-8")) + w_res = api.PyUnicode_FromString(s) + assert space.unwrap(w_res) == u'sp?m' + + w_res = api.PyUnicode_FromStringAndSize(s, 4) + assert space.unwrap(w_res) == u'sp?' + rffi.free_charp(s) + def test_AsUTF8String(self, space, api): w_u = space.wrap(u'sp?m') w_res = api.PyUnicode_AsUTF8String(w_u) Modified: pypy/branch/fast-forward/pypy/module/cpyext/unicodeobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/unicodeobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/unicodeobject.py Sun Nov 7 21:53:42 2010 @@ -287,6 +287,25 @@ space.wrap("decoding Unicode is not supported")) return space.call_function(w_meth, w_encoding, w_errors) + at cpython_api([CONST_STRING], PyObject) +def PyUnicode_FromString(space, s): + """Create a Unicode object from an UTF-8 encoded null-terminated char buffer""" + w_str = space.wrap(rffi.charp2str(s)) + return space.call_method(w_str, 'decode', space.wrap("utf-8")) + + at cpython_api([CONST_STRING, Py_ssize_t], PyObject) +def PyUnicode_FromStringAndSize(space, s, size): + """Create a Unicode Object from the char buffer u. The bytes will be + interpreted as being UTF-8 encoded. u may also be NULL which causes the + contents to be undefined. It is the user's responsibility to fill in the + needed data. The buffer is copied into the new object. If the buffer is not + NULL, the return value might be a shared object. Therefore, modification of + the resulting Unicode object is only allowed when u is NULL.""" + if not s: + raise NotImplementedError + w_str = space.wrap(rffi.charpsize2str(s, size)) + return space.call_method(w_str, 'decode', space.wrap("utf-8")) + @cpython_api([PyObject], PyObject) def PyUnicode_AsUTF8String(space, w_unicode): """Encode a Unicode object using UTF-8 and return the result as Python string From afa at codespeak.net Sun Nov 7 22:14:08 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 22:14:08 +0100 (CET) Subject: [pypy-svn] r78843 - in pypy/branch/fast-forward/pypy/module/cpyext: . include src test Message-ID: <20101107211408.7180B282B9D@codespeak.net> Author: afa Date: Sun Nov 7 22:14:03 2010 New Revision: 78843 Added: pypy/branch/fast-forward/pypy/module/cpyext/include/pycapsule.h (contents, props changed) pypy/branch/fast-forward/pypy/module/cpyext/src/capsule.c (contents, props changed) pypy/branch/fast-forward/pypy/module/cpyext/test/test_capsule.py (contents, props changed) Modified: pypy/branch/fast-forward/pypy/module/cpyext/api.py pypy/branch/fast-forward/pypy/module/cpyext/import_.py pypy/branch/fast-forward/pypy/module/cpyext/include/Python.h pypy/branch/fast-forward/pypy/module/cpyext/include/object.h pypy/branch/fast-forward/pypy/module/cpyext/stubs.py Log: Add PyCapsule object and all its API. Implement it directly in C, because it's only useful there. Modified: pypy/branch/fast-forward/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/api.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/api.py Sun Nov 7 22:14:03 2010 @@ -320,6 +320,11 @@ 'PyCObject_GetDesc', 'PyCObject_Import', 'PyCObject_SetVoidPtr', 'PyCObject_Type', 'init_pycobject', + 'PyCapsule_New', 'PyCapsule_IsValid', 'PyCapsule_GetPointer', + 'PyCapsule_GetName', 'PyCapsule_GetDestructor', 'PyCapsule_GetContext', + 'PyCapsule_SetPointer', 'PyCapsule_SetName', 'PyCapsule_SetDestructor', + 'PyCapsule_SetContext', 'PyCapsule_Import', 'PyCapsule_Type', 'init_capsule', + 'PyObject_AsReadBuffer', 'PyObject_AsWriteBuffer', 'PyObject_CheckReadBuffer', ] TYPES = {} @@ -562,9 +567,11 @@ def setup_init_functions(eci): init_buffer = rffi.llexternal('init_bufferobject', [], lltype.Void, compilation_info=eci) init_pycobject = rffi.llexternal('init_pycobject', [], lltype.Void, compilation_info=eci) + init_capsule = rffi.llexternal('init_capsule', [], lltype.Void, compilation_info=eci) INIT_FUNCTIONS.extend([ lambda space: init_buffer(), lambda space: init_pycobject(), + lambda space: init_capsule(), ]) def init_function(func): @@ -846,6 +853,7 @@ source_dir / "bufferobject.c", source_dir / "object.c", source_dir / "cobject.c", + source_dir / "capsule.c", ], separate_module_sources = [code, struct_source], export_symbols=export_symbols_eci, Modified: pypy/branch/fast-forward/pypy/module/cpyext/import_.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/import_.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/import_.py Sun Nov 7 22:14:03 2010 @@ -39,3 +39,9 @@ @cpython_api([CONST_STRING], PyObject) def PyImport_ImportModule(space, name): return PyImport_Import(space, space.wrap(rffi.charp2str(name))) + + at cpython_api([CONST_STRING], PyObject) +def PyImport_ImportModuleNoBlock(space, name): + space.warn('PyImport_ImportModuleNoBlock() is not non-blocking', + space.w_RuntimeWarning) + return PyImport_Import(space, space.wrap(rffi.charp2str(name))) Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/Python.h ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/include/Python.h (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/Python.h Sun Nov 7 22:14:03 2010 @@ -110,6 +110,7 @@ #include "eval.h" #include "pymem.h" #include "pycobject.h" +#include "pycapsule.h" #include "bufferobject.h" #include "sliceobject.h" #include "datetime.h" Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/object.h ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/include/object.h (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/object.h Sun Nov 7 22:14:03 2010 @@ -55,11 +55,9 @@ } \ } while (0) -#if 0 /* This will be added with python 2.6 */ #define Py_REFCNT(ob) (((PyObject*)(ob))->ob_refcnt) #define Py_TYPE(ob) (((PyObject*)(ob))->ob_type) #define Py_SIZE(ob) (((PyVarObject*)(ob))->ob_size) -#endif /* This will be added with python 2.6 */ #define Py_None (&_Py_NoneStruct) Added: pypy/branch/fast-forward/pypy/module/cpyext/include/pycapsule.h ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/pycapsule.h Sun Nov 7 22:14:03 2010 @@ -0,0 +1,56 @@ + +/* Capsule objects let you wrap a C "void *" pointer in a Python + object. They're a way of passing data through the Python interpreter + without creating your own custom type. + + Capsules are used for communication between extension modules. + They provide a way for an extension module to export a C interface + to other extension modules, so that extension modules can use the + Python import mechanism to link to one another. + + For more information, please see "c-api/capsule.html" in the + documentation. +*/ + +#ifndef Py_CAPSULE_H +#define Py_CAPSULE_H +#ifdef __cplusplus +extern "C" { +#endif + +PyAPI_DATA(PyTypeObject) PyCapsule_Type; + +typedef void (*PyCapsule_Destructor)(PyObject *); + +#define PyCapsule_CheckExact(op) (Py_TYPE(op) == &PyCapsule_Type) + + +PyAPI_FUNC(PyObject *) PyCapsule_New( + void *pointer, + const char *name, + PyCapsule_Destructor destructor); + +PyAPI_FUNC(void *) PyCapsule_GetPointer(PyObject *capsule, const char *name); + +PyAPI_FUNC(PyCapsule_Destructor) PyCapsule_GetDestructor(PyObject *capsule); + +PyAPI_FUNC(const char *) PyCapsule_GetName(PyObject *capsule); + +PyAPI_FUNC(void *) PyCapsule_GetContext(PyObject *capsule); + +PyAPI_FUNC(int) PyCapsule_IsValid(PyObject *capsule, const char *name); + +PyAPI_FUNC(int) PyCapsule_SetPointer(PyObject *capsule, void *pointer); + +PyAPI_FUNC(int) PyCapsule_SetDestructor(PyObject *capsule, PyCapsule_Destructor destructor); + +PyAPI_FUNC(int) PyCapsule_SetName(PyObject *capsule, const char *name); + +PyAPI_FUNC(int) PyCapsule_SetContext(PyObject *capsule, void *context); + +PyAPI_FUNC(void *) PyCapsule_Import(const char *name, int no_block); + +#ifdef __cplusplus +} +#endif +#endif /* !Py_CAPSULE_H */ Added: pypy/branch/fast-forward/pypy/module/cpyext/src/capsule.c ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/pypy/module/cpyext/src/capsule.c Sun Nov 7 22:14:03 2010 @@ -0,0 +1,328 @@ +/* Wrap void * pointers to be passed between C modules */ + +#include "Python.h" + +/* Internal structure of PyCapsule */ +typedef struct { + PyObject_HEAD + void *pointer; + const char *name; + void *context; + PyCapsule_Destructor destructor; +} PyCapsule; + + + +static int +_is_legal_capsule(PyCapsule *capsule, const char *invalid_capsule) +{ + if (!capsule || !PyCapsule_CheckExact(capsule) || capsule->pointer == NULL) { + PyErr_SetString(PyExc_ValueError, invalid_capsule); + return 0; + } + return 1; +} + +#define is_legal_capsule(capsule, name) \ + (_is_legal_capsule(capsule, \ + name " called with invalid PyCapsule object")) + + +static int +name_matches(const char *name1, const char *name2) { + /* if either is NULL, */ + if (!name1 || !name2) { + /* they're only the same if they're both NULL. */ + return name1 == name2; + } + return !strcmp(name1, name2); +} + + + +PyObject * +PyCapsule_New(void *pointer, const char *name, PyCapsule_Destructor destructor) +{ + PyCapsule *capsule; + + if (!pointer) { + PyErr_SetString(PyExc_ValueError, "PyCapsule_New called with null pointer"); + return NULL; + } + + capsule = PyObject_NEW(PyCapsule, &PyCapsule_Type); + if (capsule == NULL) { + return NULL; + } + + capsule->pointer = pointer; + capsule->name = name; + capsule->context = NULL; + capsule->destructor = destructor; + + return (PyObject *)capsule; +} + + +int +PyCapsule_IsValid(PyObject *o, const char *name) +{ + PyCapsule *capsule = (PyCapsule *)o; + + return (capsule != NULL && + PyCapsule_CheckExact(capsule) && + capsule->pointer != NULL && + name_matches(capsule->name, name)); +} + + +void * +PyCapsule_GetPointer(PyObject *o, const char *name) +{ + PyCapsule *capsule = (PyCapsule *)o; + + if (!is_legal_capsule(capsule, "PyCapsule_GetPointer")) { + return NULL; + } + + if (!name_matches(name, capsule->name)) { + PyErr_SetString(PyExc_ValueError, "PyCapsule_GetPointer called with incorrect name"); + return NULL; + } + + return capsule->pointer; +} + + +const char * +PyCapsule_GetName(PyObject *o) +{ + PyCapsule *capsule = (PyCapsule *)o; + + if (!is_legal_capsule(capsule, "PyCapsule_GetName")) { + return NULL; + } + return capsule->name; +} + + +PyCapsule_Destructor +PyCapsule_GetDestructor(PyObject *o) +{ + PyCapsule *capsule = (PyCapsule *)o; + + if (!is_legal_capsule(capsule, "PyCapsule_GetDestructor")) { + return NULL; + } + return capsule->destructor; +} + + +void * +PyCapsule_GetContext(PyObject *o) +{ + PyCapsule *capsule = (PyCapsule *)o; + + if (!is_legal_capsule(capsule, "PyCapsule_GetContext")) { + return NULL; + } + return capsule->context; +} + + +int +PyCapsule_SetPointer(PyObject *o, void *pointer) +{ + PyCapsule *capsule = (PyCapsule *)o; + + if (!pointer) { + PyErr_SetString(PyExc_ValueError, "PyCapsule_SetPointer called with null pointer"); + return -1; + } + + if (!is_legal_capsule(capsule, "PyCapsule_SetPointer")) { + return -1; + } + + capsule->pointer = pointer; + return 0; +} + + +int +PyCapsule_SetName(PyObject *o, const char *name) +{ + PyCapsule *capsule = (PyCapsule *)o; + + if (!is_legal_capsule(capsule, "PyCapsule_SetName")) { + return -1; + } + + capsule->name = name; + return 0; +} + + +int +PyCapsule_SetDestructor(PyObject *o, PyCapsule_Destructor destructor) +{ + PyCapsule *capsule = (PyCapsule *)o; + + if (!is_legal_capsule(capsule, "PyCapsule_SetDestructor")) { + return -1; + } + + capsule->destructor = destructor; + return 0; +} + + +int +PyCapsule_SetContext(PyObject *o, void *context) +{ + PyCapsule *capsule = (PyCapsule *)o; + + if (!is_legal_capsule(capsule, "PyCapsule_SetContext")) { + return -1; + } + + capsule->context = context; + return 0; +} + + +void * +PyCapsule_Import(const char *name, int no_block) +{ + PyObject *object = NULL; + void *return_value = NULL; + char *trace; + size_t name_length = (strlen(name) + 1) * sizeof(char); + char *name_dup = (char *)PyMem_MALLOC(name_length); + + if (!name_dup) { + return NULL; + } + + memcpy(name_dup, name, name_length); + + trace = name_dup; + while (trace) { + char *dot = strchr(trace, '.'); + if (dot) { + *dot++ = '\0'; + } + + if (object == NULL) { + if (no_block) { + object = PyImport_ImportModuleNoBlock(trace); + } else { + object = PyImport_ImportModule(trace); + if (!object) { + PyErr_Format(PyExc_ImportError, "PyCapsule_Import could not import module \"%s\"", trace); + } + } + } else { + PyObject *object2 = PyObject_GetAttrString(object, trace); + Py_DECREF(object); + object = object2; + } + if (!object) { + goto EXIT; + } + + trace = dot; + } + + /* compare attribute name to module.name by hand */ + if (PyCapsule_IsValid(object, name)) { + PyCapsule *capsule = (PyCapsule *)object; + return_value = capsule->pointer; + } else { + PyErr_Format(PyExc_AttributeError, + "PyCapsule_Import \"%s\" is not valid", + name); + } + +EXIT: + Py_XDECREF(object); + if (name_dup) { + PyMem_FREE(name_dup); + } + return return_value; +} + + +static void +capsule_dealloc(PyObject *o) +{ + PyCapsule *capsule = (PyCapsule *)o; + if (capsule->destructor) { + capsule->destructor(o); + } + PyObject_DEL(o); +} + + +static PyObject * +capsule_repr(PyObject *o) +{ + PyCapsule *capsule = (PyCapsule *)o; + const char *name; + const char *quote; + + if (capsule->name) { + quote = "\""; + name = capsule->name; + } else { + quote = ""; + name = "NULL"; + } + + return PyString_FromFormat("", + quote, name, quote, capsule); +} + + + +PyDoc_STRVAR(PyCapsule_Type__doc__, +"Capsule objects let you wrap a C \"void *\" pointer in a Python\n\ +object. They're a way of passing data through the Python interpreter\n\ +without creating your own custom type.\n\ +\n\ +Capsules are used for communication between extension modules.\n\ +They provide a way for an extension module to export a C interface\n\ +to other extension modules, so that extension modules can use the\n\ +Python import mechanism to link to one another.\n\ +"); + +PyTypeObject PyCapsule_Type = { + PyVarObject_HEAD_INIT(&PyType_Type, 0) + "PyCapsule", /*tp_name*/ + sizeof(PyCapsule), /*tp_basicsize*/ + 0, /*tp_itemsize*/ + /* methods */ + capsule_dealloc, /*tp_dealloc*/ + 0, /*tp_print*/ + 0, /*tp_getattr*/ + 0, /*tp_setattr*/ + 0, /*tp_reserved*/ + capsule_repr, /*tp_repr*/ + 0, /*tp_as_number*/ + 0, /*tp_as_sequence*/ + 0, /*tp_as_mapping*/ + 0, /*tp_hash*/ + 0, /*tp_call*/ + 0, /*tp_str*/ + 0, /*tp_getattro*/ + 0, /*tp_setattro*/ + 0, /*tp_as_buffer*/ + 0, /*tp_flags*/ + PyCapsule_Type__doc__ /*tp_doc*/ +}; + +void init_capsule() +{ + PyType_Ready(&PyCapsule_Type); +} + Modified: pypy/branch/fast-forward/pypy/module/cpyext/stubs.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/stubs.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/stubs.py Sun Nov 7 22:14:03 2010 @@ -372,18 +372,6 @@ used as the positional and keyword parameters to the object's constructor.""" raise NotImplementedError - at cpython_api([PyObject], rffi.VOIDP_real, error=lltype.nullptr(rffi.VOIDP.TO)) -def PyCObject_GetDesc(space, self): - """Return the description void * that the PyCObject self was - created with.""" - raise NotImplementedError - - at cpython_api([PyObject, rffi.VOIDP_real], rffi.INT_real, error=0) -def PyCObject_SetVoidPtr(space, self, cobj): - """Set the void pointer inside self to cobj. The PyCObject must not - have an associated destructor. Return true on success, false on failure.""" - raise NotImplementedError - @cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL) def PyCode_Check(space, co): """Return true if co is a code object""" Added: pypy/branch/fast-forward/pypy/module/cpyext/test/test_capsule.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_capsule.py Sun Nov 7 22:14:03 2010 @@ -0,0 +1,29 @@ +from pypy.module.cpyext.test.test_cpyext import AppTestCpythonExtensionBase + +class AppTestCapsule(AppTestCpythonExtensionBase): + def test_capsule_import(self): + module = self.import_extension('foo', [ + ("set_ptr", "METH_O", + """ + PyObject *capsule, *module; + void *ptr = PyLong_AsVoidPtr(args); + if (PyErr_Occurred()) return NULL; + capsule = PyCapsule_New(ptr, "foo._ptr", NULL); + if (PyErr_Occurred()) return NULL; + module = PyImport_ImportModule("foo"); + PyModule_AddObject(module, "_ptr", capsule); + Py_DECREF(module); + if (PyErr_Occurred()) return NULL; + Py_RETURN_NONE; + """), + ("get_ptr", "METH_NOARGS", + """ + void *ptr = PyCapsule_Import("foo._ptr", 0); + if (PyErr_Occurred()) return NULL; + return PyLong_FromVoidPtr(ptr); + """)]) + module.set_ptr(1234) + assert 'capsule object "foo._ptr" at ' in str(module._ptr) + import gc; gc.collect() + assert module.get_ptr() == 1234 + del module._ptr From afa at codespeak.net Sun Nov 7 22:37:13 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 22:37:13 +0100 (CET) Subject: [pypy-svn] r78844 - in pypy/branch/fast-forward/pypy/module/cpyext: . test Message-ID: <20101107213713.36444282BE3@codespeak.net> Author: afa Date: Sun Nov 7 22:37:11 2010 New Revision: 78844 Added: pypy/branch/fast-forward/pypy/module/cpyext/codecs.py (contents, props changed) pypy/branch/fast-forward/pypy/module/cpyext/test/test_codecs.py (contents, props changed) Modified: pypy/branch/fast-forward/pypy/module/cpyext/__init__.py Log: Add PyCodec_IncrementalEncoder, PyCodec_IncrementalDecoder Modified: pypy/branch/fast-forward/pypy/module/cpyext/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/__init__.py Sun Nov 7 22:37:11 2010 @@ -70,6 +70,7 @@ import pypy.module.cpyext.funcobject import pypy.module.cpyext.classobject import pypy.module.cpyext.memoryobject +import pypy.module.cpyext.codecs # now that all rffi_platform.Struct types are registered, configure them api.configure_types() Added: pypy/branch/fast-forward/pypy/module/cpyext/codecs.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/pypy/module/cpyext/codecs.py Sun Nov 7 22:37:11 2010 @@ -0,0 +1,22 @@ +from pypy.rpython.lltypesystem import rffi +from pypy.module.cpyext.api import cpython_api, PyObject, CONST_STRING +from pypy.module._codecs import interp_codecs + + at cpython_api([CONST_STRING, CONST_STRING], PyObject) +def PyCodec_IncrementalEncoder(space, encoding, errors): + w_codec = interp_codecs.lookup_codec(space, rffi.charp2str(encoding)) + if errors: + w_errors = space.wrap(rffi.charp2str(errors)) + return space.call_method(w_codec, "incrementalencoder", w_errors) + else: + return space.call_method(w_codec, "incrementalencoder") + + at cpython_api([CONST_STRING, CONST_STRING], PyObject) +def PyCodec_IncrementalDecoder(space, encoding, errors): + w_codec = interp_codecs.lookup_codec(space, rffi.charp2str(encoding)) + if errors: + w_errors = space.wrap(rffi.charp2str(errors)) + return space.call_method(w_codec, "incrementaldecoder", w_errors) + else: + return space.call_method(w_codec, "incrementaldecoder") + Added: pypy/branch/fast-forward/pypy/module/cpyext/test/test_codecs.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_codecs.py Sun Nov 7 22:37:11 2010 @@ -0,0 +1,14 @@ +# encoding: iso-8859-15 +from pypy.module.cpyext.test.test_api import BaseApiTest +from pypy.rpython.lltypesystem import rffi, lltype + +class TestCodecs(BaseApiTest): + def test_incremental(self, space, api): + utf8 = rffi.str2charp('utf-8') + w_encoder = api.PyCodec_IncrementalEncoder(utf8, None) + w_encoded = space.call_method(w_encoder, 'encode', space.wrap(u'sp?m')) + w_decoder = api.PyCodec_IncrementalDecoder(utf8, None) + w_decoded = space.call_method(w_decoder, 'decode', w_encoded) + assert space.unwrap(w_decoded) == u'sp?m' + rffi.free_charp(utf8) + From afa at codespeak.net Sun Nov 7 23:01:21 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 23:01:21 +0100 (CET) Subject: [pypy-svn] r78845 - in pypy/branch/fast-forward/pypy/module/cpyext: . test Message-ID: <20101107220121.2A22C282B90@codespeak.net> Author: afa Date: Sun Nov 7 23:01:18 2010 New Revision: 78845 Modified: pypy/branch/fast-forward/pypy/module/cpyext/longobject.py pypy/branch/fast-forward/pypy/module/cpyext/test/test_longobject.py Log: Add _PyLong_NumBits, _PyLong_Sign Modified: pypy/branch/fast-forward/pypy/module/cpyext/longobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/longobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/longobject.py Sun Nov 7 23:01:18 2010 @@ -1,6 +1,6 @@ from pypy.rpython.lltypesystem import lltype, rffi from pypy.module.cpyext.api import (cpython_api, PyObject, build_type_checkers, - CONST_STRING, ADDR) + CONST_STRING, ADDR, CANNOT_FAIL) from pypy.objspace.std.longobject import W_LongObject from pypy.interpreter.error import OperationError from pypy.module.cpyext.intobject import PyInt_AsUnsignedLongMask @@ -159,3 +159,13 @@ For values outside 0..LONG_MAX, both signed and unsigned integers are accepted.""" return rffi.cast(rffi.VOIDP_real, space.uint_w(w_long)) + at cpython_api([PyObject], rffi.SIZE_T, error=-1) +def _PyLong_NumBits(space, w_long): + return space.uint_w(space.call_method(w_long, "bit_length")) + + at cpython_api([PyObject], rffi.INT_real, error=CANNOT_FAIL) +def _PyLong_Sign(space, w_long): + assert isinstance(w_long, W_LongObject) + return w_long.num.sign + + Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_longobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/test/test_longobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_longobject.py Sun Nov 7 23:01:18 2010 @@ -93,6 +93,19 @@ assert space.unwrap(w_l) == 0L assert api.PyLong_AsVoidPtr(w_l) == lltype.nullptr(rffi.VOIDP_real.TO) + def test_sign_and_bits(self, space, api): + assert api._PyLong_Sign(space.wrap(0L)) == 0 + assert api._PyLong_Sign(space.wrap(2L)) == 1 + assert api._PyLong_Sign(space.wrap(-2L)) == -1 + + assert api._PyLong_NumBits(space.wrap(0)) == 0 + assert api._PyLong_NumBits(space.wrap(1)) == 1 + assert api._PyLong_NumBits(space.wrap(-1)) == 1 + assert api._PyLong_NumBits(space.wrap(2)) == 2 + assert api._PyLong_NumBits(space.wrap(-2)) == 2 + assert api._PyLong_NumBits(space.wrap(3)) == 2 + assert api._PyLong_NumBits(space.wrap(-3)) == 2 + class AppTestLongObject(AppTestCpythonExtensionBase): def test_fromunsignedlong(self): module = self.import_extension('foo', [ From afa at codespeak.net Sun Nov 7 23:12:49 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 23:12:49 +0100 (CET) Subject: [pypy-svn] r78846 - in pypy/branch/fast-forward/pypy/module/cpyext: . test Message-ID: <20101107221249.BBED0282B9D@codespeak.net> Author: afa Date: Sun Nov 7 23:12:47 2010 New Revision: 78846 Modified: pypy/branch/fast-forward/pypy/module/cpyext/stubs.py pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py pypy/branch/fast-forward/pypy/module/cpyext/unicodeobject.py Log: PyUnicode_FromObject, PyUnicode_Compare Modified: pypy/branch/fast-forward/pypy/module/cpyext/stubs.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/stubs.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/stubs.py Sun Nov 7 23:12:47 2010 @@ -2741,12 +2741,6 @@ possible. This macro does not raise exceptions.""" raise NotImplementedError - at cpython_api([PyObject], PyObject) -def PyUnicode_FromObject(space, obj): - """Shortcut for PyUnicode_FromEncodedObject(obj, NULL, "strict") which is used - throughout the interpreter whenever coercion to Unicode is needed.""" - raise NotImplementedError - @cpython_api([rffi.CWCHARP, Py_ssize_t, rffi.CCHARP, rffi.CCHARP], PyObject) def PyUnicode_Encode(space, s, size, encoding, errors): """Encode the Py_UNICODE buffer of the given size and return a Python @@ -3133,12 +3127,6 @@ require changes in your code for properly supporting 64-bit systems.""" raise NotImplementedError - at cpython_api([PyObject, PyObject], rffi.INT_real, error=-2) -def PyUnicode_Compare(space, left, right): - """Compare two strings and return -1, 0, 1 for less than, equal, and greater than, - respectively.""" - raise NotImplementedError - @cpython_api([PyObject, PyObject, rffi.INT_real], PyObject) def PyUnicode_RichCompare(space, left, right, op): """Rich compare two unicode strings and return one of the following: Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_unicodeobject.py Sun Nov 7 23:12:47 2010 @@ -122,6 +122,12 @@ assert api.Py_UNICODE_TOUPPER(u'?') == u'?' assert api.Py_UNICODE_TOUPPER(u'?') == u'?' + def test_fromobject(self, space, api): + w_u = space.wrap(u'a') + assert api.PyUnicode_FromObject(w_u) is w_u + assert space.unwrap( + api.PyUnicode_FromObject(space.wrap('test'))) == 'test' + def test_decode(self, space, api): b_text = rffi.str2charp('caf\x82xx') b_encoding = rffi.str2charp('cp437') @@ -216,3 +222,6 @@ test("\xFE\xFF\x00\x61\x00\x62\x00\x63\x00\x64", 0, 1) test("\xFF\xFE\x61\x00\x62\x00\x63\x00\x64\x00", 0, -1) + + def test_compare(self, space, api): + assert api.PyUnicode_Compare(space.wrap('a'), space.wrap('b')) == -1 Modified: pypy/branch/fast-forward/pypy/module/cpyext/unicodeobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/unicodeobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/unicodeobject.py Sun Nov 7 23:12:47 2010 @@ -253,6 +253,15 @@ w_errors = space.w_None return space.call_method(w_str, 'decode', w_encoding, w_errors) + at cpython_api([PyObject], PyObject) +def PyUnicode_FromObject(space, w_obj): + """Shortcut for PyUnicode_FromEncodedObject(obj, NULL, "strict") which is used + throughout the interpreter whenever coercion to Unicode is needed.""" + if space.is_w(space.type(w_obj), space.w_unicode): + return w_obj + else: + return space.call_function(space.w_unicode, w_obj) + @cpython_api([PyObject, CONST_STRING, CONST_STRING], PyObject) def PyUnicode_FromEncodedObject(space, w_obj, encoding, errors): """Coerce an encoded object obj to an Unicode object and return a reference with @@ -424,3 +433,10 @@ else: w_errors = space.w_None return space.call_method(w_str, 'decode', w_encoding, w_errors) + + at cpython_api([PyObject, PyObject], rffi.INT_real, error=-2) +def PyUnicode_Compare(space, w_left, w_right): + """Compare two strings and return -1, 0, 1 for less than, equal, and greater + than, respectively.""" + return space.int_w(space.cmp(w_left, w_right)) + From afa at codespeak.net Sun Nov 7 23:21:08 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 23:21:08 +0100 (CET) Subject: [pypy-svn] r78847 - pypy/branch/fast-forward/pypy/module/cpyext/include Message-ID: <20101107222108.4B6CA36C221@codespeak.net> Author: afa Date: Sun Nov 7 23:21:06 2010 New Revision: 78847 Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/pyerrors.h Log: Add PyExceptionClass_Check Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/pyerrors.h ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/include/pyerrors.h (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/pyerrors.h Sun Nov 7 23:21:06 2010 @@ -7,6 +7,10 @@ extern "C" { #endif +#define PyExceptionClass_Check(x) \ + (PyClass_Check((x)) || (PyType_Check((x)) && \ + PyObject_IsSubclass((x), PyExc_BaseException))) + PyObject *PyErr_NewException(char *name, PyObject *base, PyObject *dict); PyObject *PyErr_Format(PyObject *exception, const char *format, ...); From afa at codespeak.net Sun Nov 7 23:26:40 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 23:26:40 +0100 (CET) Subject: [pypy-svn] r78848 - in pypy/branch/fast-forward/pypy/module/cpyext: . include src Message-ID: <20101107222640.3EE5A282B9E@codespeak.net> Author: afa Date: Sun Nov 7 23:26:38 2010 New Revision: 78848 Modified: pypy/branch/fast-forward/pypy/module/cpyext/api.py pypy/branch/fast-forward/pypy/module/cpyext/include/pyerrors.h pypy/branch/fast-forward/pypy/module/cpyext/src/pyerrors.c Log: PyErr_NewExceptionWithDoc, copied from CPython. Modified: pypy/branch/fast-forward/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/api.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/api.py Sun Nov 7 23:26:38 2010 @@ -308,7 +308,7 @@ 'PyModule_AddObject', 'PyModule_AddIntConstant', 'PyModule_AddStringConstant', 'Py_BuildValue', 'Py_VaBuildValue', 'PyTuple_Pack', - 'PyErr_Format', 'PyErr_NewException', + 'PyErr_Format', 'PyErr_NewException', 'PyErr_NewExceptionWithDoc', 'PyEval_CallFunction', 'PyEval_CallMethod', 'PyObject_CallFunction', 'PyObject_CallMethod', 'PyObject_CallFunctionObjArgs', 'PyObject_CallMethodObjArgs', Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/pyerrors.h ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/include/pyerrors.h (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/pyerrors.h Sun Nov 7 23:26:38 2010 @@ -12,6 +12,7 @@ PyObject_IsSubclass((x), PyExc_BaseException))) PyObject *PyErr_NewException(char *name, PyObject *base, PyObject *dict); +PyObject *PyErr_NewExceptionWithDoc(char *name, char *doc, PyObject *base, PyObject *dict); PyObject *PyErr_Format(PyObject *exception, const char *format, ...); #ifdef __cplusplus Modified: pypy/branch/fast-forward/pypy/module/cpyext/src/pyerrors.c ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/src/pyerrors.c (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/src/pyerrors.c Sun Nov 7 23:26:38 2010 @@ -69,3 +69,37 @@ Py_XDECREF(modulename); return result; } + +/* Create an exception with docstring */ +PyObject * +PyErr_NewExceptionWithDoc(char *name, char *doc, PyObject *base, PyObject *dict) +{ + int result; + PyObject *ret = NULL; + PyObject *mydict = NULL; /* points to the dict only if we create it */ + PyObject *docobj; + + if (dict == NULL) { + dict = mydict = PyDict_New(); + if (dict == NULL) { + return NULL; + } + } + + if (doc != NULL) { + docobj = PyString_FromString(doc); + if (docobj == NULL) + goto failure; + result = PyDict_SetItemString(dict, "__doc__", docobj); + Py_DECREF(docobj); + if (result < 0) + goto failure; + } + + ret = PyErr_NewException(name, base, dict); + failure: + Py_XDECREF(mydict); + return ret; +} + + From afa at codespeak.net Sun Nov 7 23:42:56 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 23:42:56 +0100 (CET) Subject: [pypy-svn] r78849 - pypy/branch/fast-forward/pypy/module/cpyext Message-ID: <20101107224256.2C009282B9E@codespeak.net> Author: afa Date: Sun Nov 7 23:42:54 2010 New Revision: 78849 Modified: pypy/branch/fast-forward/pypy/module/cpyext/pyerrors.py Log: PyTraceBack_Print, don't know how to test it :-( Modified: pypy/branch/fast-forward/pypy/module/cpyext/pyerrors.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/pyerrors.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/pyerrors.py Sun Nov 7 23:42:54 2010 @@ -241,3 +241,12 @@ def PyErr_Print(space): """Alias for PyErr_PrintEx(1).""" PyErr_PrintEx(space, 1) + + at cpython_api([PyObject, PyObject], rffi.INT_real, error=-1): +def PyTraceBack_Print(space, w_tb, w_file): + space.call_method(w_file, "write", space.wrap( + 'Traceback (most recent call last):\n')) + w_traceback = space.call_method(space.builtin, '__import__', + space.wrap("traceback")) + space.call_method(w_traceback, "print_tb", w_tb, space.w_None, w_file) + return 0 From afa at codespeak.net Sun Nov 7 23:46:45 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 7 Nov 2010 23:46:45 +0100 (CET) Subject: [pypy-svn] r78850 - pypy/branch/fast-forward/pypy/module/cpyext Message-ID: <20101107224645.C14F7282B9E@codespeak.net> Author: afa Date: Sun Nov 7 23:46:41 2010 New Revision: 78850 Modified: pypy/branch/fast-forward/pypy/module/cpyext/pyerrors.py Log: Typo Modified: pypy/branch/fast-forward/pypy/module/cpyext/pyerrors.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/pyerrors.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/pyerrors.py Sun Nov 7 23:46:41 2010 @@ -242,7 +242,7 @@ """Alias for PyErr_PrintEx(1).""" PyErr_PrintEx(space, 1) - at cpython_api([PyObject, PyObject], rffi.INT_real, error=-1): + at cpython_api([PyObject, PyObject], rffi.INT_real, error=-1) def PyTraceBack_Print(space, w_tb, w_file): space.call_method(w_file, "write", space.wrap( 'Traceback (most recent call last):\n')) From afa at codespeak.net Mon Nov 8 00:07:41 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 00:07:41 +0100 (CET) Subject: [pypy-svn] r78851 - pypy/branch/fast-forward/pypy/module/__builtin__ Message-ID: <20101107230741.5D68B282B90@codespeak.net> Author: afa Date: Mon Nov 8 00:07:39 2010 New Revision: 78851 Modified: pypy/branch/fast-forward/pypy/module/__builtin__/interp_memoryview.py Log: Try to fix translation. I don't know if operator.lt is RPython... Modified: pypy/branch/fast-forward/pypy/module/__builtin__/interp_memoryview.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/interp_memoryview.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/interp_memoryview.py Mon Nov 8 00:07:39 2010 @@ -6,6 +6,7 @@ from pypy.interpreter.gateway import interp2app, unwrap_spec, ObjSpace, W_Root from pypy.interpreter.typedef import TypeDef, GetSetProperty from pypy.interpreter.error import OperationError +import operator W_Buffer = buffer.Buffer # actually implemented in pypy.interpreter.buffer From afa at codespeak.net Mon Nov 8 00:08:05 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 00:08:05 +0100 (CET) Subject: [pypy-svn] r78852 - in pypy/branch/fast-forward/pypy/module/cpyext: . test Message-ID: <20101107230805.A1F89282BDA@codespeak.net> Author: afa Date: Mon Nov 8 00:08:04 2010 New Revision: 78852 Modified: pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py pypy/branch/fast-forward/pypy/module/cpyext/test/test_funcobject.py Log: Add PyCode_NewEmpty Modified: pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py Mon Nov 8 00:08:04 2010 @@ -1,10 +1,11 @@ from pypy.rpython.lltypesystem import rffi, lltype from pypy.module.cpyext.api import ( - PyObjectFields, generic_cpy_call, + PyObjectFields, generic_cpy_call, CONST_STRING, cpython_api, bootstrap_function, cpython_struct, build_type_checkers) from pypy.module.cpyext.pyobject import ( PyObject, make_ref, from_ref, Py_DecRef, make_typedescr, borrow_from) from pypy.interpreter.function import Function, Method +from pypy.interpreter.pycode import PyCode PyFunctionObjectStruct = lltype.ForwardReference() PyFunctionObject = lltype.Ptr(PyFunctionObjectStruct) @@ -63,3 +64,22 @@ assert isinstance(w_method, Method) return borrow_from(w_method, w_method.w_class) + at cpython_api([CONST_STRING, CONST_STRING, rffi.INT_real], PyObject) +def PyCode_NewEmpty(space, filename, funcname, firstlineno): + """Creates a new empty code object with the specified source location.""" + return space.wrap(PyCode(space, + argcount=0, + nlocals=0, + stacksize=0, + flags=0, + code="", + consts=[], + names=[], + varnames=[], + filename=rffi.charp2str(filename), + name=rffi.charp2str(funcname), + firstlineno=firstlineno, + lnotab="", + freevars=[], + cellvars=[])) + Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_funcobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/test/test_funcobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_funcobject.py Mon Nov 8 00:08:04 2010 @@ -35,3 +35,12 @@ w_method2 = api.PyMethod_New(w_function, w_self, w_class) assert space.eq_w(w_method, w_method2) + + def test_newcode(self, space, api): + filename = rffi.str2charp('filename') + funcname = rffi.str2charp('funcname') + w_code = api.PyCode_NewEmpty(filename, funcname, 3) + assert w_code.co_filename == 'filename' + assert w_code.co_firstlineno == 3 + rffi.free_charp(filename) + rffi.free_charp(funcname) From afa at codespeak.net Mon Nov 8 00:12:55 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 00:12:55 +0100 (CET) Subject: [pypy-svn] r78853 - pypy/branch/fast-forward/pypy/module/cpyext/include Message-ID: <20101107231255.00E4E282BDC@codespeak.net> Author: afa Date: Mon Nov 8 00:12:46 2010 New Revision: 78853 Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/object.h Log: Add a dummy PyGC_Head structure Modified: pypy/branch/fast-forward/pypy/module/cpyext/include/object.h ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/include/object.h (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/include/object.h Mon Nov 8 00:12:46 2010 @@ -459,6 +459,11 @@ #define PyObject_GC_New(type, typeobj) \ ( (type *) _PyObject_GC_New(typeobj) ) +/* A dummy PyGC_Head, just to please some tests. Don't use it! */ +typedef union _gc_head { + char dummy; +} PyGC_Head; + /* Utility macro to help write tp_traverse functions. * To use this macro, the tp_traverse function must name its arguments * "visit" and "arg". This is intended to keep tp_traverse functions From afa at codespeak.net Mon Nov 8 01:39:24 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 01:39:24 +0100 (CET) Subject: [pypy-svn] r78854 - pypy/branch/fast-forward/pypy/module/cpyext Message-ID: <20101108003924.0D268282B9E@codespeak.net> Author: afa Date: Mon Nov 8 01:39:22 2010 New Revision: 78854 Modified: pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py Log: Skip this function for now, don't know how to make it translate. Modified: pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py Mon Nov 8 01:39:22 2010 @@ -4,6 +4,7 @@ cpython_api, bootstrap_function, cpython_struct, build_type_checkers) from pypy.module.cpyext.pyobject import ( PyObject, make_ref, from_ref, Py_DecRef, make_typedescr, borrow_from) +from pypy.interpreter.error import OperationError from pypy.interpreter.function import Function, Method from pypy.interpreter.pycode import PyCode @@ -67,13 +68,17 @@ @cpython_api([CONST_STRING, CONST_STRING, rffi.INT_real], PyObject) def PyCode_NewEmpty(space, filename, funcname, firstlineno): """Creates a new empty code object with the specified source location.""" + raise OperationError(space.w_NotImplementedError, space.wrap( + "PyCode_NewEmpty")) + # XXX I keep getting a "TooLateForChange" around the annotation of + # the consts variable return space.wrap(PyCode(space, argcount=0, nlocals=0, stacksize=0, flags=0, code="", - consts=[], + consts=list([]), names=[], varnames=[], filename=rffi.charp2str(filename), From afa at codespeak.net Mon Nov 8 01:53:39 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 01:53:39 +0100 (CET) Subject: [pypy-svn] r78855 - pypy/branch/fast-forward/pypy/module/itertools Message-ID: <20101108005339.7DCB5282B9E@codespeak.net> Author: afa Date: Mon Nov 8 01:53:38 2010 New Revision: 78855 Modified: pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py Log: Try to fix translation Modified: pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py (original) +++ pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py Mon Nov 8 01:53:38 2010 @@ -992,8 +992,8 @@ self.gears_w = [x for x in args_w] * repeat_w.intval self.num_gears = len(self.gears_w) # initialization of indicies to loop over - self.indicies = [(0, len(self.gears_w[x].wrappeditems)) - for x in range(0, self.num_gears)] + self.indicies = [(0, space.int_w(space.len(w_gear))) + for w_gear in self.gears_w] self.cont = True def roll_gears(self): @@ -1028,7 +1028,8 @@ l = [] for x in range(0, self.num_gears): index, limit = self.indicies[x] - l.append(self.gears_w[x].wrappeditems[index]) + l.append(self.space.getitem(self.gears_w[x], + self.space.wrap(index))) self.roll_gears() return self.space.newtuple(l) From afa at codespeak.net Mon Nov 8 02:10:29 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 02:10:29 +0100 (CET) Subject: [pypy-svn] r78856 - pypy/branch/fast-forward/pypy/module/itertools Message-ID: <20101108011029.50B4E282BDD@codespeak.net> Author: afa Date: Mon Nov 8 02:10:27 2010 New Revision: 78856 Modified: pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py Log: More translation fixes Modified: pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py (original) +++ pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py Mon Nov 8 02:10:27 2010 @@ -987,9 +987,9 @@ class W_Product(Wrappable): - def __init__(self, space, args_w, repeat_w): + def __init__(self, space, args_w, w_repeat): self.space = space - self.gears_w = [x for x in args_w] * repeat_w.intval + self.gears_w = [x for x in args_w] * space.int_w(w_repeat) self.num_gears = len(self.gears_w) # initialization of indicies to loop over self.indicies = [(0, space.int_w(space.len(w_gear))) @@ -1040,8 +1040,8 @@ raise OperationError(space.w_TypeError, space.wrap("product() takes at most 1 argument (%d given)" % len(kw_args_w))) - repeat = kw_args_w.get('repeat', space.wrap(1)) - return space.wrap(W_Product(space, star_args_w[1:], repeat)) + w_repeat = kw_args_w.get('repeat', space.wrap(1)) + return space.wrap(W_Product(space, star_args_w[1:], w_repeat)) W_Product.typedef = TypeDef( 'product', From afa at codespeak.net Mon Nov 8 02:29:58 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 02:29:58 +0100 (CET) Subject: [pypy-svn] r78857 - pypy/branch/fast-forward/pypy/module/itertools Message-ID: <20101108012958.7ECC1282BE7@codespeak.net> Author: afa Date: Mon Nov 8 02:29:57 2010 New Revision: 78857 Modified: pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py Log: This list must not be resized Modified: pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py (original) +++ pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py Mon Nov 8 02:29:57 2010 @@ -1025,11 +1025,11 @@ if not self.cont: raise OperationError(self.space.w_StopIteration, self.space.w_None) - l = [] + l = [None] * self.num_gears for x in range(0, self.num_gears): index, limit = self.indicies[x] - l.append(self.space.getitem(self.gears_w[x], - self.space.wrap(index))) + l[x] = self.space.getitem(self.gears_w[x], + self.space.wrap(index))) self.roll_gears() return self.space.newtuple(l) From benjamin at codespeak.net Mon Nov 8 03:51:19 2010 From: benjamin at codespeak.net (benjamin at codespeak.net) Date: Mon, 8 Nov 2010 03:51:19 +0100 (CET) Subject: [pypy-svn] r78858 - pypy/branch/fast-forward/pypy/module/itertools Message-ID: <20101108025119.351A5282BDD@codespeak.net> Author: benjamin Date: Mon Nov 8 03:51:17 2010 New Revision: 78858 Modified: pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py Log: fix extra paren Modified: pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py (original) +++ pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py Mon Nov 8 03:51:17 2010 @@ -1029,7 +1029,7 @@ for x in range(0, self.num_gears): index, limit = self.indicies[x] l[x] = self.space.getitem(self.gears_w[x], - self.space.wrap(index))) + self.space.wrap(index)) self.roll_gears() return self.space.newtuple(l) From afa at codespeak.net Mon Nov 8 10:08:33 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 10:08:33 +0100 (CET) Subject: [pypy-svn] r78859 - pypy/branch/fast-forward/pypy/module/cpyext Message-ID: <20101108090833.71EE9282B90@codespeak.net> Author: afa Date: Mon Nov 8 10:08:27 2010 New Revision: 78859 Modified: pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py Log: Work around an annotation bug Modified: pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py Mon Nov 8 10:08:27 2010 @@ -68,17 +68,18 @@ @cpython_api([CONST_STRING, CONST_STRING, rffi.INT_real], PyObject) def PyCode_NewEmpty(space, filename, funcname, firstlineno): """Creates a new empty code object with the specified source location.""" - raise OperationError(space.w_NotImplementedError, space.wrap( - "PyCode_NewEmpty")) - # XXX I keep getting a "TooLateForChange" around the annotation of - # the consts variable + # XXX Something that annotates as SomeInstance(knownType=W_Root) + # XXX looks like a bug in make_sure_not_modified() used by PyCode.___init__ + # XXX which also depends on the annotation order + w_const = space.and_(space.w_True, space.w_None) + return space.wrap(PyCode(space, argcount=0, nlocals=0, stacksize=0, flags=0, code="", - consts=list([]), + consts=[w_const], names=[], varnames=[], filename=rffi.charp2str(filename), From ale at codespeak.net Mon Nov 8 10:16:15 2010 From: ale at codespeak.net (ale at codespeak.net) Date: Mon, 8 Nov 2010 10:16:15 +0100 (CET) Subject: [pypy-svn] r78860 - pypy/extradoc/planning/hg-migration Message-ID: <20101108091615.7326F282B90@codespeak.net> Author: ale Date: Mon Nov 8 10:16:13 2010 New Revision: 78860 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: My email address Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Mon Nov 8 10:16:13 2010 @@ -19,7 +19,7 @@ ac=Anders Chrigstrom xoraxax=Alexander Schremmer rxe=Richard Emslie -ale=Anders Lehmann +ale=Anders Lehmann auc=Aurelien Campeas getxsick=Bartosz Skowron nik=Niklaus Haldimann From david at codespeak.net Mon Nov 8 11:06:20 2010 From: david at codespeak.net (david at codespeak.net) Date: Mon, 8 Nov 2010 11:06:20 +0100 (CET) Subject: [pypy-svn] r78861 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . helper test Message-ID: <20101108100620.062EC282B9D@codespeak.net> Author: david Date: Mon Nov 8 11:06:18 2010 New Revision: 78861 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py Log: Fixed decoding of spilled values on stack Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Mon Nov 8 11:06:18 2010 @@ -55,7 +55,8 @@ the failboxes. Registers are saved on the stack XXX Rest to follow""" - i = -1 + frame_depth = self.decode32(enc, 0) + i = 3 fail_index = -1 while(True): i += 1 @@ -77,7 +78,8 @@ elif res == '\xFC': # stack location stack_loc = self.decode32(enc, i+1) #XXX ffuu use propper calculation here - value = self.decode32(stack, len(r.all_regs)*WORD+40-stack_loc*WORD) + value = self.decode32(stack, + (len(r.all_regs)+frame_depth-stack_loc)*WORD) i += 4 else: # an int for now reg = ord(enc[i]) @@ -142,8 +144,9 @@ # XXX free this memory # XXX allocate correct amount of memory mem = lltype.malloc(rffi.CArray(lltype.Char), (len(args)+5)*4, flavor='raw') + self.encode32(mem, 0, regalloc.frame_manager.frame_depth-1) i = 0 - j = 0 + j = 4 while(i < len(args)): if args[i]: loc = regalloc.loc(args[i]) @@ -217,6 +220,7 @@ self.mc.gen_load_int(reg.value, addr) self.mc.LDR_ri(reg.value, reg.value) regs.append(reg) + regalloc.possibly_free_var(reg) looptoken._arm_arglocs = regs # cpu interface @@ -258,7 +262,7 @@ cb = ARMv7InMemoryBuilder(addr, ARMv7InMemoryBuilder.size_of_gen_load_int) if regalloc.frame_manager.frame_depth == 1: return - n = regalloc.frame_manager.frame_depth*WORD + n = (regalloc.frame_manager.frame_depth - 1)*WORD self._adjust_sp(n, cb) def _adjust_sp(self, n, cb=None, fcond=c.AL): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Mon Nov 8 11:06:18 2010 @@ -4,8 +4,9 @@ def gen_emit_op_unary_cmp(true_cond, false_cond): def f(self, op, regalloc, fcond): - reg = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) - res = regalloc.force_allocate_reg(op.result) + a0 = op.getarg(0) + reg = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + res = regalloc.force_allocate_reg(op.result, [a0]) self.mc.CMP_ri(reg.value, 0) self.mc.MOV_ri(res.value, 1, true_cond) self.mc.MOV_ri(res.value, 0, false_cond) @@ -24,18 +25,18 @@ imm_a1 = self._check_imm_arg(arg1, imm_size, allow_zero=allow_zero) if commutative and imm_a0: l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=imm_a0) - l1 = regalloc.make_sure_var_in_reg(arg1) - res = regalloc.force_allocate_reg(op.result) + l1 = regalloc.make_sure_var_in_reg(arg1, [arg0]) + res = regalloc.force_allocate_reg(op.result, [arg0, arg1]) ri_op(res.value, l1.value, imm=l0.getint(), cond=fcond) elif imm_a1: l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) - l1 = regalloc.make_sure_var_in_reg(arg1, imm_fine=True) - res = regalloc.force_allocate_reg(op.result) + l1 = regalloc.make_sure_var_in_reg(arg1, [arg0], imm_fine=True) + res = regalloc.force_allocate_reg(op.result, [arg0, arg1]) ri_op(res.value, l0.value, imm=l1.getint(), cond=fcond) else: l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) - l1 = regalloc.make_sure_var_in_reg(arg1, imm_fine=False) - res = regalloc.force_allocate_reg(op.result) + l1 = regalloc.make_sure_var_in_reg(arg1, [arg0], imm_fine=False) + res = regalloc.force_allocate_reg(op.result, [arg0, arg1]) rr_op(res.value, l0.value, l1.value) regalloc.possibly_free_vars_for_op(op) return fcond @@ -43,8 +44,10 @@ def gen_emit_op_by_helper_call(opname): def f(self, op, regalloc, fcond): - arg1 = regalloc.make_sure_var_in_reg(op.getarg(0), selected_reg=r.r0, imm_fine=False) - arg2 = regalloc.make_sure_var_in_reg(op.getarg(1), selected_reg=r.r1, imm_fine=False) + a0 = op.getarg(0) + a1 = op.getarg(1) + arg1 = regalloc.make_sure_var_in_reg(a0, selected_reg=r.r0, imm_fine=False) + arg2 = regalloc.make_sure_var_in_reg(a1, [a0], selected_reg=r.r1, imm_fine=False) assert arg1 == r.r0 assert arg2 == r.r1 res = regalloc.force_allocate_reg(op.result, selected_reg=r.r0) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Mon Nov 8 11:06:18 2010 @@ -32,16 +32,16 @@ a0, a1 = a1, a0 if imm_a1: l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) - l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=True) - res = regalloc.force_allocate_reg(op.result) + l1 = regalloc.make_sure_var_in_reg(a1, [a0], imm_fine=True) + res = regalloc.force_allocate_reg(op.result, [a0, a1]) if l1.getint() < 0: self.mc.SUB_ri(res.value, l0.value, -1 * l1.getint(), s=1) else: self.mc.ADD_ri(res.value, l0.value, l1.getint(), s=1) else: l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) - l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=False) - res = regalloc.force_allocate_reg(op.result) + l1 = regalloc.make_sure_var_in_reg(a1, forbidden_vars=[a0], imm_fine=False) + res = regalloc.force_allocate_reg(op.result, forbidden_vars=[a0, a1]) self.mc.ADD_rr(res.value, l0.value, l1.value, s=1) regalloc.possibly_free_vars_for_op(op) @@ -54,8 +54,8 @@ imm_a0 = isinstance(a0, ConstInt) and (a0.getint() <= 0xFF or -1 * a0.getint() <= 0xFF) imm_a1 = isinstance(a1, ConstInt) and (a1.getint() <= 0xFF or -1 * a1.getint() <= 0xFF) l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=imm_a0) - l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=imm_a1) - res = regalloc.force_allocate_reg(op.result) + l1 = regalloc.make_sure_var_in_reg(a1, [a0], imm_fine=imm_a1) + res = regalloc.force_allocate_reg(op.result, [a0, a1]) if imm_a0: value = l0.getint() if value < 0: @@ -78,18 +78,22 @@ return fcond def emit_op_int_mul(self, op, regalloc, fcond): - reg1 = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) - reg2 = regalloc.make_sure_var_in_reg(op.getarg(1), imm_fine=False) - res = regalloc.force_allocate_reg(op.result) + a0 = op.getarg(0) + a1 = op.getarg(1) + reg1 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + reg2 = regalloc.make_sure_var_in_reg(a1, [a0], imm_fine=False) + res = regalloc.force_allocate_reg(op.result, [a0, a1]) self.mc.MUL(res.value, reg1.value, reg2.value) regalloc.possibly_free_vars_for_op(op) return fcond #ref: http://blogs.arm.com/software-enablement/detecting-overflow-from-mul/ def emit_op_int_mul_ovf(self, op, regalloc, fcond): - reg1 = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) - reg2 = regalloc.make_sure_var_in_reg(op.getarg(1), imm_fine=False) - res = regalloc.force_allocate_reg(op.result) + a0 = op.getarg(0) + a1 = op.getarg(1) + reg1 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + reg2 = regalloc.make_sure_var_in_reg(a1, [a0], imm_fine=False) + res = regalloc.force_allocate_reg(op.result, [a0, a1]) self.mc.SMULL(res.value, r.ip.value, reg1.value, reg2.value, cond=fcond) self.mc.CMP_rr(r.ip.value, res.value, shifttype=shift.ASR, s=31, cond=fcond) regalloc.possibly_free_vars_for_op(op) @@ -129,8 +133,9 @@ emit_op_int_is_zero = gen_emit_op_unary_cmp(c.EQ, c.NE) def emit_op_int_invert(self, op, regalloc, fcond): - reg = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) - res = regalloc.force_allocate_reg(op.result) + a0 = op.getarg(0) + reg = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + res = regalloc.force_allocate_reg(op.result, [a0]) self.mc.MVN_rr(res.value, reg.value) regalloc.possibly_free_vars_for_op(op) @@ -139,9 +144,9 @@ #XXX check for a better way of doing this def emit_op_int_neg(self, op, regalloc, fcond): arg = op.getarg(0) - l0 = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) - l1 = regalloc.make_sure_var_in_reg(ConstInt(-1), imm_fine=False) - res = regalloc.force_allocate_reg(op.result) + l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(ConstInt(-1), [arg], imm_fine=False) + res = regalloc.force_allocate_reg(op.result, [a0]) self.mc.MUL(res.value, l0.value, l1.value) regalloc.possibly_free_vars([l0, l1, res]) return fcond @@ -225,11 +230,13 @@ class FieldOpAssembler(object): def emit_op_setfield_gc(self, op, regalloc, fcond): + a0 = op.getarg(0) + a1 = op.getarg(1) ofs, size, ptr = self._unpack_fielddescr(op.getdescr()) #ofs_loc = regalloc.make_sure_var_in_reg(ConstInt(ofs)) #size_loc = regalloc.make_sure_var_in_reg(ofs) - base_loc = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) - value_loc = regalloc.make_sure_var_in_reg(op.getarg(1), imm_fine=False) + base_loc = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + value_loc = regalloc.make_sure_var_in_reg(a1, [a0], imm_fine=False) if size == 4: f = self.mc.STR_ri elif size == 2: @@ -242,10 +249,11 @@ return fcond def emit_op_getfield_gc(self, op, regalloc, fcond): + a0 = op.getarg(0) ofs, size, ptr = self._unpack_fielddescr(op.getdescr()) # ofs_loc = regalloc.make_sure_var_in_reg(ConstInt(ofs)) - base_loc = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) - res = regalloc.force_allocate_reg(op.result) + base_loc = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + res = regalloc.force_allocate_reg(op.result, [a0]) if size == 4: f = self.mc.LDR_ri elif size == 2: Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Mon Nov 8 11:06:18 2010 @@ -15,7 +15,8 @@ RegisterManager.__init__(self, longevity, frame_manager, assembler) def update_bindings(self, enc, inputargs): - j = 0 + # first word contains frame depth + j = 4 for i in range(len(inputargs)): # XXX decode imm and and stack locs and REFs while enc[j] == '\xFE': Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py Mon Nov 8 11:06:18 2010 @@ -1,6 +1,15 @@ from pypy.jit.backend.arm.runner import ArmCPU from pypy.jit.backend.test.runner_test import LLtypeBackendTest from pypy.jit.backend.arm.test.support import skip_unless_arm +from pypy.jit.metainterp.history import (AbstractFailDescr, + AbstractDescr, + BasicFailDescr, + BoxInt, Box, BoxPtr, + LoopToken, + ConstInt, ConstPtr, + BoxObj, Const, + ConstObj, BoxFloat, ConstFloat) +from pypy.jit.metainterp.resoperation import ResOperation, rop skip_unless_arm() @@ -14,3 +23,27 @@ def setup_method(self, meth): self.cpu = ArmCPU(rtyper=None, stats=FakeStats()) + + def test_result_is_spilled(self): + cpu = self.cpu + inp = [BoxInt(i) for i in range(1, 15)] + out = list(inp) + out.reverse() + looptoken = LoopToken() + operations = [ + ResOperation(rop.INT_ADD, [inp[0], inp[1]], inp[2]), + ResOperation(rop.INT_SUB, [inp[1], ConstInt(1)], inp[4]), + ResOperation(rop.INT_ADD, [inp[4], ConstInt(0)], inp[3]), + ResOperation(rop.INT_ADD, [inp[5], inp[6]], inp[7]), + ResOperation(rop.INT_SUB, [inp[8], ConstInt(1)], inp[9]), + ResOperation(rop.INT_ADD, [inp[10], ConstInt(1)], inp[3]), + ResOperation(rop.INT_ADD, [inp[11], inp[12]], inp[13]), + ResOperation(rop.FINISH, out, None, descr=BasicFailDescr(1)), + ] + cpu.compile_loop(inp, operations, looptoken) + for i in range(1, 15): + self.cpu.set_future_value_int(i-1, i) + res = self.cpu.execute_token(looptoken) + output = [self.cpu.get_latest_value_int(i-1) for i in range(1, 15)] + expected = [25,13,12,11,8,9,13,7,6,1,12,3,2,1] + assert output == expected From david at codespeak.net Mon Nov 8 11:11:46 2010 From: david at codespeak.net (david at codespeak.net) Date: Mon, 8 Nov 2010 11:11:46 +0100 (CET) Subject: [pypy-svn] r78862 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101108101146.6BA335080B@codespeak.net> Author: david Date: Mon Nov 8 11:11:45 2010 New Revision: 78862 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Add get_ and set_field_raw operations Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Mon Nov 8 11:11:45 2010 @@ -71,6 +71,7 @@ self.mc.ADD_ri(res.value, l0.value, -1 * value, s=1) else: self.mc.SUB_ri(res.value, l0.value, value, s=1) + self.mc.BKPT() else: self.mc.SUB_rr(res.value, l0.value, l1.value, s=1) @@ -248,6 +249,8 @@ f(value_loc.value, base_loc.value, ofs) return fcond + emit_op_setfield_raw = emit_op_setfield_gc + def emit_op_getfield_gc(self, op, regalloc, fcond): a0 = op.getarg(0) ofs, size, ptr = self._unpack_fielddescr(op.getdescr()) @@ -265,6 +268,12 @@ f(res.value, base_loc.value, ofs) return fcond + emit_op_getfield_raw = emit_op_getfield_gc + emit_op_getfield_raw_pure = emit_op_getfield_gc + emit_op_getfield_gc_pure = emit_op_getfield_gc + + + #XXX from ../x86/regalloc.py:791 def _unpack_fielddescr(self, fielddescr): assert isinstance(fielddescr, BaseFieldDescr) From afa at codespeak.net Mon Nov 8 11:30:51 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 11:30:51 +0100 (CET) Subject: [pypy-svn] r78863 - pypy/branch/fast-forward/pypy/module/cpyext Message-ID: <20101108103051.22FD3282B9E@codespeak.net> Author: afa Date: Mon Nov 8 11:30:44 2010 New Revision: 78863 Modified: pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py Log: More complex workaround Modified: pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py Mon Nov 8 11:30:44 2010 @@ -65,21 +65,33 @@ assert isinstance(w_method, Method) return borrow_from(w_method, w_method.w_class) +# XXX Something that annotates as a list of (W_Root or None) +# XXX looks like a bug in make_sure_not_modified() used by PyCode.___init__ +# XXX which also depends on the annotation order +def consts_w(): + return [] + +from pypy.rpython import extregistry +class For_some_const(extregistry.ExtRegistryEntry): + _about_ = consts_w + + def compute_result_annotation(self): + from pypy.annotation import model as annmodel + from pypy.interpreter.baseobjspace import W_Root + clsdef = self.bookkeeper.getuniqueclassdef(W_Root) + s_value = annmodel.SomeInstance(clsdef, can_be_None=True) + return self.bookkeeper.newlist(s_value) + @cpython_api([CONST_STRING, CONST_STRING, rffi.INT_real], PyObject) def PyCode_NewEmpty(space, filename, funcname, firstlineno): """Creates a new empty code object with the specified source location.""" - # XXX Something that annotates as SomeInstance(knownType=W_Root) - # XXX looks like a bug in make_sure_not_modified() used by PyCode.___init__ - # XXX which also depends on the annotation order - w_const = space.and_(space.w_True, space.w_None) - return space.wrap(PyCode(space, argcount=0, nlocals=0, stacksize=0, flags=0, code="", - consts=[w_const], + consts=consts_w(), names=[], varnames=[], filename=rffi.charp2str(filename), From afa at codespeak.net Mon Nov 8 11:38:45 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 11:38:45 +0100 (CET) Subject: [pypy-svn] r78864 - pypy/branch/fast-forward/pypy/module/cpyext Message-ID: <20101108103845.9D1B15080B@codespeak.net> Author: afa Date: Mon Nov 8 11:38:43 2010 New Revision: 78864 Modified: pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py Log: add cast to fix translation Modified: pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py Mon Nov 8 11:38:43 2010 @@ -96,7 +96,7 @@ varnames=[], filename=rffi.charp2str(filename), name=rffi.charp2str(funcname), - firstlineno=firstlineno, + firstlineno=rffi.cast(lltype.Signed, firstlineno), lnotab="", freevars=[], cellvars=[])) From afa at codespeak.net Mon Nov 8 13:01:14 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 13:01:14 +0100 (CET) Subject: [pypy-svn] r78865 - pypy/branch/fast-forward/pypy/module/cpyext Message-ID: <20101108120114.E6E3B282BDA@codespeak.net> Author: afa Date: Mon Nov 8 13:01:12 2010 New Revision: 78865 Modified: pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py Log: I hope I get it right this time Modified: pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/funcobject.py Mon Nov 8 13:01:12 2010 @@ -82,6 +82,9 @@ s_value = annmodel.SomeInstance(clsdef, can_be_None=True) return self.bookkeeper.newlist(s_value) + def specialize_call(self, hop): + return hop.rtyper.type_system.rlist.newlist(hop.llops, hop.r_result, []) + @cpython_api([CONST_STRING, CONST_STRING, rffi.INT_real], PyObject) def PyCode_NewEmpty(space, filename, funcname, firstlineno): """Creates a new empty code object with the specified source location.""" From david at codespeak.net Mon Nov 8 13:27:29 2010 From: david at codespeak.net (david at codespeak.net) Date: Mon, 8 Nov 2010 13:27:29 +0100 (CET) Subject: [pypy-svn] r78866 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101108122729.C88C8282B9D@codespeak.net> Author: david Date: Mon Nov 8 13:27:28 2010 New Revision: 78866 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/shift.py Log: implement getarrayitem, setarrayitem and arraylen operations Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Mon Nov 8 13:27:28 2010 @@ -15,13 +15,15 @@ IntOpAsslember, OpAssembler, UnaryIntOpAssembler, - FieldOpAssembler) + FieldOpAssembler, + ArrayOpAssember) # XXX Move to llsupport from pypy.jit.backend.x86.support import values_array class AssemblerARM(GuardOpAssembler, IntOpAsslember, - OpAssembler, UnaryIntOpAssembler, FieldOpAssembler): + OpAssembler, UnaryIntOpAssembler, + FieldOpAssembler, ArrayOpAssember): def __init__(self, cpu, failargs_limit=1000): self.mc = ARMv7Builder() Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Mon Nov 8 13:27:28 2010 @@ -71,7 +71,6 @@ self.mc.ADD_ri(res.value, l0.value, -1 * value, s=1) else: self.mc.SUB_ri(res.value, l0.value, value, s=1) - self.mc.BKPT() else: self.mc.SUB_rr(res.value, l0.value, l1.value, s=1) @@ -145,9 +144,9 @@ #XXX check for a better way of doing this def emit_op_int_neg(self, op, regalloc, fcond): arg = op.getarg(0) - l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + l0 = regalloc.make_sure_var_in_reg(arg, imm_fine=False) l1 = regalloc.make_sure_var_in_reg(ConstInt(-1), [arg], imm_fine=False) - res = regalloc.force_allocate_reg(op.result, [a0]) + res = regalloc.force_allocate_reg(op.result, [arg]) self.mc.MUL(res.value, l0.value, l1.value) regalloc.possibly_free_vars([l0, l1, res]) return fcond @@ -281,3 +280,77 @@ size = fielddescr.get_field_size(self.cpu.translate_support_code) ptr = fielddescr.is_pointer_field() return ofs, size, ptr + +class ArrayOpAssember(object): + + def emit_op_arraylen_gc(self, op, regalloc, fcond): + arraydescr = op.getdescr() + assert isinstance(arraydescr, BaseArrayDescr) + ofs = arraydescr.get_ofs_length(self.cpu.translate_support_code) + base_loc = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) + regalloc.possibly_free_vars_for_op(op) + res = regalloc.force_allocate_reg(op.result, forbidden_vars=[base_loc]) + + self.mc.LDR_ri(res.value, base_loc.value, ofs) + regalloc.possibly_free_var(op.getarg(0)) + + def emit_op_setarrayitem_gc(self, op, regalloc, fcond): + a0 = op.getarg(0) + a1 = op.getarg(1) + a2 = op.getarg(2) + scale, ofs, _, ptr = self._unpack_arraydescr(op.getdescr()) + + base_loc = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + ofs_loc = regalloc.make_sure_var_in_reg(a1, imm_fine=False) + #XXX check if imm would be fine here + value_loc = regalloc.make_sure_var_in_reg(a2, imm_fine=False) + + if scale == 2: + self.mc.STR_rr(value_loc.value, base_loc.value, ofs_loc.value, cond=fcond, + imm=scale, shifttype=shift.LSL) + elif scale == 1: + self.mc.LSL_ri(ofs_loc.value, ofs_loc.value, scale) + self.mc.STRH_rr(value_loc.value, base_loc.value, ofs_loc.value, cond=fcond) + elif scale == 0: + self.mc.STRB_rr(value_loc.value, base_loc.value, ofs_loc.value, cond=fcond) + else: + assert 0 + return fcond + + emit_op_setarrayitem_raw = emit_op_setarrayitem_gc + + def emit_op_getarrayitem_gc(self, op, regalloc, fcond): + a0 = op.getarg(0) + a1 = op.getarg(1) + scale, ofs, _, ptr = self._unpack_arraydescr(op.getdescr()) + + base_loc = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + ofs_loc = regalloc.make_sure_var_in_reg(a1, imm_fine=False) + res = regalloc.force_allocate_reg(op.result) + if scale == 2: + f = self.mc.LDR_rr + elif scale == 1: + f = self.mc.LDRH_rr + elif scale == 0: + f = self.mc.LDRB_rr + else: + assert 0 + if scale > 0: + self.mc.LSL_ri(ofs_loc.value, ofs_loc.value, scale) + f(res.value, base_loc.value, ofs_loc.value, cond=fcond) + + emit_op_getarrayitem_raw = emit_op_getarrayitem_gc + emit_op_getarrayitem_gc_pure = emit_op_getarrayitem_gc + + #XXX from ../x86/regalloc.py:779 + def _unpack_arraydescr(self, arraydescr): + assert isinstance(arraydescr, BaseArrayDescr) + ofs_length = arraydescr.get_ofs_length(self.cpu.translate_support_code) + ofs = arraydescr.get_base_size(self.cpu.translate_support_code) + size = arraydescr.get_item_size(self.cpu.translate_support_code) + ptr = arraydescr.is_array_of_pointers() + scale = 0 + while (1 << scale) < size: + scale += 1 + assert (1 << scale) == size + return scale, ofs, ofs_length, ptr Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/shift.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/shift.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/shift.py Mon Nov 8 13:27:28 2010 @@ -1,6 +1,6 @@ # According to A8.4 -LSL = 0xB00 -LSR = 0xB01 -ASR = 0xB10 -ROR = 0xB11 -RRX = 0xB11 # with imm = 0 +LSL = 0x0 +LSR = 0x1 +ASR = 0x2 +ROR = 0x3 +RRX = 0x3 # with imm = 0 From iko at codespeak.net Mon Nov 8 13:27:43 2010 From: iko at codespeak.net (iko at codespeak.net) Date: Mon, 8 Nov 2010 13:27:43 +0100 (CET) Subject: [pypy-svn] r78867 - pypy/extradoc/planning/hg-migration Message-ID: <20101108122743.6F383282BDA@codespeak.net> Author: iko Date: Mon Nov 8 13:27:42 2010 New Revision: 78867 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: add email Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Mon Nov 8 13:27:42 2010 @@ -40,7 +40,7 @@ niko=Niko Matsakis alex=Alex Martelli jcreigh=Jason Creighton -iko=Anders Hammarquist +iko=Anders Hammarquist agaynor=Alex Gaynor stephan=Stephan Diehl jandem=Jan de Mooij From arigo at codespeak.net Mon Nov 8 13:54:40 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 8 Nov 2010 13:54:40 +0100 (CET) Subject: [pypy-svn] r78868 - pypy/trunk/pypy/translator/c Message-ID: <20101108125440.36695282B9D@codespeak.net> Author: arigo Date: Mon Nov 8 13:54:38 2010 New Revision: 78868 Modified: pypy/trunk/pypy/translator/c/genc.py Log: Better support in the Makefile for Ctrl-C during make, not leaving empty files behind. Modified: pypy/trunk/pypy/translator/c/genc.py ============================================================================== --- pypy/trunk/pypy/translator/c/genc.py (original) +++ pypy/trunk/pypy/translator/c/genc.py Mon Nov 8 13:54:38 2010 @@ -623,9 +623,15 @@ mk.definition('OBJECTS', '$(ASMLBLFILES) gcmaptable.s') mk.rule('%.s', '%.c', '$(CC) $(CFLAGS) $(CFLAGSEXTRA) -frandom-seed=$< -o $@ -S $< $(INCLUDEDIRS)') mk.rule('%.lbl.s %.gcmap', '%.s', - python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py -m$(PYPY_MAIN_FUNCTION) -t $< > $*.gcmap') + [python + + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py ' + '-m$(PYPY_MAIN_FUNCTION) -t $< > $*.gctmp', + 'mv $*.gctmp $*.gcmap']) mk.rule('gcmaptable.s', '$(GCMAPFILES)', - python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py $(GCMAPFILES) > $@') + [python + + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py ' + '$(GCMAPFILES) > $@.tmp', + 'mv $@.tmp $@']) mk.rule('.PRECIOUS', '%.s', "# don't remove .s files if Ctrl-C'ed") else: From arigo at codespeak.net Mon Nov 8 13:55:10 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 8 Nov 2010 13:55:10 +0100 (CET) Subject: [pypy-svn] r78869 - pypy/trunk/pypy/jit/tool Message-ID: <20101108125510.32A7850810@codespeak.net> Author: arigo Date: Mon Nov 8 13:55:08 2010 New Revision: 78869 Modified: pypy/trunk/pypy/jit/tool/pypytrace.vim Log: Still playing with the colors. Modified: pypy/trunk/pypy/jit/tool/pypytrace.vim ============================================================================== --- pypy/trunk/pypy/jit/tool/pypytrace.vim (original) +++ pypy/trunk/pypy/jit/tool/pypytrace.vim Mon Nov 8 13:55:08 2010 @@ -21,10 +21,10 @@ hi def link pypyLoopStart Structure "hi def link pypyLoopArgs PreProc -hi def link pypyFailArgs String +hi def link pypyFailArgs Special "hi def link pypyOpName Statement -hi def link pypyDebugMergePoint Comment +hi def link pypyDebugMergePoint String hi def link pypyConstPtr Constant hi def link pypyNumber Number -hi def link pypyDescr String +hi def link pypyDescr PreProc hi def link pypyDescrField Label From arigo at codespeak.net Mon Nov 8 13:55:56 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 8 Nov 2010 13:55:56 +0100 (CET) Subject: [pypy-svn] r78870 - pypy/trunk/pypy/jit/backend/x86/tool Message-ID: <20101108125556.B128F282B9D@codespeak.net> Author: arigo Date: Mon Nov 8 13:55:55 2010 New Revision: 78870 Modified: pypy/trunk/pypy/jit/backend/x86/tool/viewcode.py Log: Don't use the Intel order here, I find it more confusing than helpful. Modified: pypy/trunk/pypy/jit/backend/x86/tool/viewcode.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/tool/viewcode.py (original) +++ pypy/trunk/pypy/jit/backend/x86/tool/viewcode.py Mon Nov 8 13:55:55 2010 @@ -37,7 +37,7 @@ 'x86_64': 'x86-64', 'i386': 'i386', } - objdump = ('objdump -M intel,%(backend)s -b binary -m i386 ' + objdump = ('objdump -M %(backend)s -b binary -m i386 ' '--adjust-vma=%(origin)d -D %(file)s') # f = open(tmpfile, 'wb') From arigo at codespeak.net Mon Nov 8 13:58:42 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 8 Nov 2010 13:58:42 +0100 (CET) Subject: [pypy-svn] r78871 - in pypy/trunk: lib_pypy/ctypes_config_cache lib_pypy/ctypes_config_cache/test pypy/jit/backend pypy/module/__pypy__ pypy/module/__pypy__/test Message-ID: <20101108125842.8833A50811@codespeak.net> Author: arigo Date: Mon Nov 8 13:58:40 2010 New Revision: 78871 Added: pypy/trunk/lib_pypy/ctypes_config_cache/dumpcache.py Modified: pypy/trunk/lib_pypy/ctypes_config_cache/ (props changed) pypy/trunk/lib_pypy/ctypes_config_cache/hashlib.ctc.py pypy/trunk/lib_pypy/ctypes_config_cache/locale.ctc.py pypy/trunk/lib_pypy/ctypes_config_cache/pyexpat.ctc.py pypy/trunk/lib_pypy/ctypes_config_cache/rebuild.py pypy/trunk/lib_pypy/ctypes_config_cache/resource.ctc.py pypy/trunk/lib_pypy/ctypes_config_cache/syslog.ctc.py pypy/trunk/lib_pypy/ctypes_config_cache/test/test_cache.py pypy/trunk/pypy/jit/backend/detect_cpu.py pypy/trunk/pypy/module/__pypy__/__init__.py pypy/trunk/pypy/module/__pypy__/test/test_special.py Log: Tweaks to allow the ctypes_config_cache to contain both 32-bit and 64-bit versions. Previously, it only contained the version of the most recently built pypy-c, and if you tried to use it with a pypy-c built for the other word size, you would get random segfaults. Added: pypy/trunk/lib_pypy/ctypes_config_cache/dumpcache.py ============================================================================== --- (empty file) +++ pypy/trunk/lib_pypy/ctypes_config_cache/dumpcache.py Mon Nov 8 13:58:40 2010 @@ -0,0 +1,23 @@ +import os +from ctypes_configure import dumpcache +from pypy.jit.backend import detect_cpu + +def dumpcache2(basename, config): + model = detect_cpu.autodetect_main_model_and_size() + filename = '_%s_%s_.py' % (basename, model) + dumpcache.dumpcache(__file__, filename, config) + # + filename = os.path.join(os.path.dirname(__file__), + '_%s_cache.py' % (basename,)) + g = open(filename, 'w') + print >> g, '''\ +try: + from __pypy__ import cpumodel +except ImportError: + from pypy.jit.backend import detect_cpu + cpumodel = detect_cpu.autodetect_main_model_and_size() +mod = __import__("ctypes_config_cache._%s_%%s_" %% (cpumodel,), + None, None, ["*"]) +globals().update(mod.__dict__)\ +''' % (basename,) + g.close() Modified: pypy/trunk/lib_pypy/ctypes_config_cache/hashlib.ctc.py ============================================================================== --- pypy/trunk/lib_pypy/ctypes_config_cache/hashlib.ctc.py (original) +++ pypy/trunk/lib_pypy/ctypes_config_cache/hashlib.ctc.py Mon Nov 8 13:58:40 2010 @@ -4,7 +4,8 @@ """ from ctypes import * -from ctypes_configure import configure, dumpcache +from ctypes_configure import configure +import dumpcache class CConfig: @@ -17,4 +18,4 @@ [('digest', c_void_p)]) config = configure.configure(CConfig) -dumpcache.dumpcache(__file__, '_hashlib_cache.py', config) +dumpcache.dumpcache2('hashlib', config) Modified: pypy/trunk/lib_pypy/ctypes_config_cache/locale.ctc.py ============================================================================== --- pypy/trunk/lib_pypy/ctypes_config_cache/locale.ctc.py (original) +++ pypy/trunk/lib_pypy/ctypes_config_cache/locale.ctc.py Mon Nov 8 13:58:40 2010 @@ -5,7 +5,7 @@ from ctypes_configure.configure import (configure, ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger, SimpleType, check_eci) -from ctypes_configure.dumpcache import dumpcache +import dumpcache # ____________________________________________________________ @@ -70,4 +70,4 @@ config['ALL_CONSTANTS'] = tuple(_CONSTANTS) config['HAS_LANGINFO'] = HAS_LANGINFO -dumpcache(__file__, '_locale_cache.py', config) +dumpcache.dumpcache2('locale', config) Modified: pypy/trunk/lib_pypy/ctypes_config_cache/pyexpat.ctc.py ============================================================================== --- pypy/trunk/lib_pypy/ctypes_config_cache/pyexpat.ctc.py (original) +++ pypy/trunk/lib_pypy/ctypes_config_cache/pyexpat.ctc.py Mon Nov 8 13:58:40 2010 @@ -5,7 +5,8 @@ import ctypes from ctypes import c_char_p, c_int, c_void_p, c_char -from ctypes_configure import configure, dumpcache +from ctypes_configure import configure +import dumpcache class CConfigure: @@ -41,4 +42,4 @@ config = configure.configure(CConfigure) -dumpcache.dumpcache(__file__, '_pyexpat_cache.py', config) +dumpcache.dumpcache2('pyexpat', config) Modified: pypy/trunk/lib_pypy/ctypes_config_cache/rebuild.py ============================================================================== --- pypy/trunk/lib_pypy/ctypes_config_cache/rebuild.py (original) +++ pypy/trunk/lib_pypy/ctypes_config_cache/rebuild.py Mon Nov 8 13:58:40 2010 @@ -31,10 +31,25 @@ sys.path[:] = path def try_rebuild(): + from pypy.jit.backend import detect_cpu + model = detect_cpu.autodetect_main_model_and_size() + # remove the files '_*_model_.py' + left = {} + for p in os.listdir(_dirpath): + if p.startswith('_') and (p.endswith('_%s_.py' % model) or + p.endswith('_%s_.pyc' % model)): + os.unlink(os.path.join(_dirpath, p)) + elif p.startswith('_') and (p.endswith('_.py') or + p.endswith('_.pyc')): + for i in range(2, len(p)-4): + left[p[:i]] = True + # remove the files '_*_cache.py' if there is no '_*_*_.py' left around for p in os.listdir(_dirpath): if p.startswith('_') and (p.endswith('_cache.py') or p.endswith('_cache.pyc')): - os.unlink(os.path.join(_dirpath, p)) + if p[:-9] not in left: + os.unlink(p) + # for p in os.listdir(_dirpath): if p.endswith('.ctc.py'): try: Modified: pypy/trunk/lib_pypy/ctypes_config_cache/resource.ctc.py ============================================================================== --- pypy/trunk/lib_pypy/ctypes_config_cache/resource.ctc.py (original) +++ pypy/trunk/lib_pypy/ctypes_config_cache/resource.ctc.py Mon Nov 8 13:58:40 2010 @@ -4,7 +4,7 @@ """ from ctypes import sizeof -from ctypes_configure.dumpcache import dumpcache +import dumpcache from ctypes_configure.configure import (configure, ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger, SimpleType) @@ -58,4 +58,4 @@ del config[key] config['ALL_CONSTANTS'] = _CONSTANTS + tuple(optional_constants) -dumpcache(__file__, '_resource_cache.py', config) +dumpcache.dumpcache2('resource', config) Modified: pypy/trunk/lib_pypy/ctypes_config_cache/syslog.ctc.py ============================================================================== --- pypy/trunk/lib_pypy/ctypes_config_cache/syslog.ctc.py (original) +++ pypy/trunk/lib_pypy/ctypes_config_cache/syslog.ctc.py Mon Nov 8 13:58:40 2010 @@ -5,7 +5,7 @@ from ctypes_configure.configure import (configure, ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger) -from ctypes_configure.dumpcache import dumpcache +import dumpcache _CONSTANTS = ( @@ -72,4 +72,4 @@ all_constants = config.keys() all_constants.sort() config['ALL_CONSTANTS'] = tuple(all_constants) -dumpcache(__file__, '_syslog_cache.py', config) +dumpcache.dumpcache2('syslog', config) Modified: pypy/trunk/lib_pypy/ctypes_config_cache/test/test_cache.py ============================================================================== --- pypy/trunk/lib_pypy/ctypes_config_cache/test/test_cache.py (original) +++ pypy/trunk/lib_pypy/ctypes_config_cache/test/test_cache.py Mon Nov 8 13:58:40 2010 @@ -7,19 +7,27 @@ def run(filename, outputname): filepath = dirpath.join(filename) - tmpdir = udir.ensure('testcache-' + filename, dir=True) - outputpath = tmpdir.join(outputname) - d = {'__file__': str(outputpath)} + tmpdir2 = udir.ensure('testcache-' + filename, dir=True) + tmpdir = tmpdir2.ensure('ctypes_config_cache', dir=True) + tmpdir.join('__init__.py').write('\n') + tmpdir.join('dumpcache.py').write(dirpath.join('dumpcache.py').read()) path = sys.path[:] try: - sys.path.insert(0, str(dirpath)) - execfile(str(filepath), d) + sys.path.insert(0, str(tmpdir)) + execfile(str(filepath), {}) finally: sys.path[:] = path + sys.modules.pop('dumpcache', None) # + outputpath = tmpdir.join(outputname) assert outputpath.check(exists=1) d = {} - execfile(str(outputpath), d) + try: + sys.path.insert(0, str(tmpdir2)) + execfile(str(outputpath), d) + finally: + sys.path[:] = path + sys.modules.pop('ctypes_config_cache', None) return d Modified: pypy/trunk/pypy/jit/backend/detect_cpu.py ============================================================================== --- pypy/trunk/pypy/jit/backend/detect_cpu.py (original) +++ pypy/trunk/pypy/jit/backend/detect_cpu.py Mon Nov 8 13:58:40 2010 @@ -34,7 +34,17 @@ 'x86_64': 'x86', }[mach] except KeyError: - raise ProcessorAutodetectError, "unsupported processor '%s'" % mach + return mach + +def autodetect_main_model_and_size(): + model = autodetect_main_model() + if sys.maxint == 2**31-1: + model += '_32' + elif sys.maxint == 2**63-1: + model += '_64' + else: + raise AssertionError, "bad value for sys.maxint" + return model def autodetect(): model = autodetect_main_model() Modified: pypy/trunk/pypy/module/__pypy__/__init__.py ============================================================================== --- pypy/trunk/pypy/module/__pypy__/__init__.py (original) +++ pypy/trunk/pypy/module/__pypy__/__init__.py Mon Nov 8 13:58:40 2010 @@ -25,3 +25,7 @@ 'interp_magic.reset_method_cache_counter') PYC_MAGIC = get_pyc_magic(self.space) self.extra_interpdef('PYC_MAGIC', 'space.wrap(%d)' % PYC_MAGIC) + # + from pypy.jit.backend import detect_cpu + model = detect_cpu.autodetect_main_model_and_size() + self.extra_interpdef('cpumodel', 'space.wrap(%r)' % model) Modified: pypy/trunk/pypy/module/__pypy__/test/test_special.py ============================================================================== --- pypy/trunk/pypy/module/__pypy__/test/test_special.py (original) +++ pypy/trunk/pypy/module/__pypy__/test/test_special.py Mon Nov 8 13:58:40 2010 @@ -17,3 +17,7 @@ from __pypy__ import isfake import select assert isfake(select) + + def test_cpumodel(self): + import __pypy__ + assert hasattr(__pypy__, 'cpumodel') From fijal at codespeak.net Mon Nov 8 15:31:52 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 8 Nov 2010 15:31:52 +0100 (CET) Subject: [pypy-svn] r78872 - pypy/trunk/pypy/interpreter Message-ID: <20101108143152.CE74E282B9D@codespeak.net> Author: fijal Date: Mon Nov 8 15:31:50 2010 New Revision: 78872 Modified: pypy/trunk/pypy/interpreter/pyopcode.py Log: Make FrameBlock a newstyle class, while we're at it Modified: pypy/trunk/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/trunk/pypy/interpreter/pyopcode.py (original) +++ pypy/trunk/pypy/interpreter/pyopcode.py Mon Nov 8 15:31:50 2010 @@ -1140,7 +1140,7 @@ state_pack_variables = staticmethod(state_pack_variables) -class FrameBlock: +class FrameBlock(object): """Abstract base class for frame blocks from the blockstack, used by the SETUP_XXX and POP_BLOCK opcodes.""" From fijal at codespeak.net Mon Nov 8 15:33:37 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 8 Nov 2010 15:33:37 +0100 (CET) Subject: [pypy-svn] r78873 - pypy/trunk/pypy/jit/backend/llsupport/test Message-ID: <20101108143337.F0B645080E@codespeak.net> Author: fijal Date: Mon Nov 8 15:33:36 2010 New Revision: 78873 Modified: pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py Log: fix test_gc Modified: pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py ============================================================================== --- pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py (original) +++ pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py Mon Nov 8 15:33:36 2010 @@ -270,7 +270,7 @@ def test_get_rid_of_debug_merge_point(self): operations = [ - ResOperation(rop.DEBUG_MERGE_POINT, ['dummy'], None), + ResOperation(rop.DEBUG_MERGE_POINT, ['dummy', 2], None), ] gc_ll_descr = self.gc_ll_descr gc_ll_descr.rewrite_assembler(None, operations) From fijal at codespeak.net Mon Nov 8 15:36:04 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 8 Nov 2010 15:36:04 +0100 (CET) Subject: [pypy-svn] r78874 - pypy/trunk/pypy/jit/backend/x86/test Message-ID: <20101108143604.1D565282BDA@codespeak.net> Author: fijal Date: Mon Nov 8 15:36:02 2010 New Revision: 78874 Modified: pypy/trunk/pypy/jit/backend/x86/test/test_runner.py Log: fix tests Modified: pypy/trunk/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/trunk/pypy/jit/backend/x86/test/test_runner.py Mon Nov 8 15:36:02 2010 @@ -346,7 +346,7 @@ return self.val operations = [ - ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("hello")], None), + ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("hello"), 0], None), ResOperation(rop.INT_ADD, [i0, ConstInt(1)], i1), ResOperation(rop.INT_LE, [i1, ConstInt(9)], i2), ResOperation(rop.GUARD_TRUE, [i2], None, descr=faildescr1), @@ -365,7 +365,7 @@ bridge = [ ResOperation(rop.INT_LE, [i1b, ConstInt(19)], i3), ResOperation(rop.GUARD_TRUE, [i3], None, descr=faildescr2), - ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("bye")], None), + ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("bye"), 0], None), ResOperation(rop.JUMP, [i1b], None, descr=looptoken), ] bridge[1].setfailargs([i1b]) @@ -497,7 +497,7 @@ def test_debugger_on(self): loop = """ [i0] - debug_merge_point('xyz') + debug_merge_point('xyz', 0) i1 = int_add(i0, 1) i2 = int_ge(i1, 10) guard_false(i2) [] From afa at codespeak.net Mon Nov 8 16:01:39 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 16:01:39 +0100 (CET) Subject: [pypy-svn] r78875 - in pypy/branch/fast-forward/pypy: objspace/std/test rlib Message-ID: <20101108150139.3326336C221@codespeak.net> Author: afa Date: Mon Nov 8 16:01:37 2010 New Revision: 78875 Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py pypy/branch/fast-forward/pypy/rlib/rbigint.py Log: Crash in test_int.py! Test and fix Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py Mon Nov 8 16:01:37 2010 @@ -273,3 +273,4 @@ def test_bit_length(self): assert 8L.bit_length() == 4 assert (-1<<40).bit_length() == 41 + assert ((2**31)-1).bit_length() == 31 Modified: pypy/branch/fast-forward/pypy/rlib/rbigint.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rbigint.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rbigint.py Mon Nov 8 16:01:37 2010 @@ -564,7 +564,7 @@ return 0 msd = self.digits[i - 1] msd_bits = 0 - if msd >= 32: + while msd >= 32: msd_bits += 6 msd >>= 6 msd_bits += [ From fijal at codespeak.net Mon Nov 8 16:17:24 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 8 Nov 2010 16:17:24 +0100 (CET) Subject: [pypy-svn] r78876 - pypy/extradoc/planning Message-ID: <20101108151724.84D0E282BDA@codespeak.net> Author: fijal Date: Mon Nov 8 16:17:22 2010 New Revision: 78876 Modified: pypy/extradoc/planning/jit.txt Log: generators, again Modified: pypy/extradoc/planning/jit.txt ============================================================================== --- pypy/extradoc/planning/jit.txt (original) +++ pypy/extradoc/planning/jit.txt Mon Nov 8 16:17:22 2010 @@ -34,6 +34,13 @@ is a compile time constant (and call unrolled version of string formatting loop in this case). +- generators are still fairly inefficient. We get a lot of: + i = ptr_eq(frame, some_other_frame) + guard_value(i, 0) + every second instruction. + + there is also manipulating of valuestackdepth and such. + OPTIMIZATIONS ------------- From david at codespeak.net Mon Nov 8 17:04:06 2010 From: david at codespeak.net (david at codespeak.net) Date: Mon, 8 Nov 2010 17:04:06 +0100 (CET) Subject: [pypy-svn] r78877 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101108160406.06AFB282BDC@codespeak.net> Author: david Date: Mon Nov 8 17:04:04 2010 New Revision: 78877 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Log: Fix a bug when calling functions and placing the sp incorrectly Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Mon Nov 8 17:04:04 2010 @@ -146,7 +146,7 @@ # XXX free this memory # XXX allocate correct amount of memory mem = lltype.malloc(rffi.CArray(lltype.Char), (len(args)+5)*4, flavor='raw') - self.encode32(mem, 0, regalloc.frame_manager.frame_depth-1) + self.encode32(mem, 0, regalloc.frame_manager.frame_depth) i = 0 j = 4 while(i < len(args)): @@ -264,7 +264,7 @@ cb = ARMv7InMemoryBuilder(addr, ARMv7InMemoryBuilder.size_of_gen_load_int) if regalloc.frame_manager.frame_depth == 1: return - n = (regalloc.frame_manager.frame_depth - 1)*WORD + n = (regalloc.frame_manager.frame_depth)*WORD self._adjust_sp(n, cb) def _adjust_sp(self, n, cb=None, fcond=c.AL): From dcolish at codespeak.net Mon Nov 8 17:07:06 2010 From: dcolish at codespeak.net (dcolish at codespeak.net) Date: Mon, 8 Nov 2010 17:07:06 +0100 (CET) Subject: [pypy-svn] r78878 - in pypy/branch/fast-forward: lib_pypy lib_pypy/pypy_test pypy/module/itertools Message-ID: <20101108160706.3C88F5080C@codespeak.net> Author: dcolish Date: Mon Nov 8 17:07:04 2010 New Revision: 78878 Added: pypy/branch/fast-forward/lib_pypy/pypy_test/test_itertools.py Modified: pypy/branch/fast-forward/lib_pypy/itertools.py pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py Log: add non-module equiv of compress and product. simplify W_Compress.next_w Modified: pypy/branch/fast-forward/lib_pypy/itertools.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/itertools.py (original) +++ pypy/branch/fast-forward/lib_pypy/itertools.py Mon Nov 8 17:07:04 2010 @@ -60,6 +60,23 @@ raise TypeError('%s has no next() method' % \ (self._cur_iterable_iter)) + +class compress(object): + def __init__(self, data, selectors): + self.data = iter(data) + self.selectors = iter(selectors) + + def __iter__(self): + return self + + def next(self): + while True: + next_item = self.data.next() + next_selector = self.selectors.next() + if bool(next_selector): + return next_item + + class count(object): """Make an iterator that returns consecutive integers starting with n. If not specified n defaults to zero. Does not currently @@ -409,7 +426,56 @@ except AttributeError: # CPython raises a TypeError when next() is not defined raise TypeError('%s has no next() method' % (i)) - + + +class product(object): + + def __init__(self, *args, **kw): + if len(kw) > 1: + raise TypeError("product() takes at most 1 argument (%d given)" % + len(kw)) + self.repeat = kw.get('repeat', 1) + self.gears = [x for x in args] * self.repeat + self.num_gears = len(self.gears) + # initialization of indicies to loop over + self.indicies = [(0, len(self.gears[x])) + for x in range(0, self.num_gears)] + self.cont = True + + def roll_gears(self): + # Starting from the end of the gear indicies work to the front + # incrementing the gear until the limit is reached. When the limit + # is reached carry operation to the next gear + should_carry = True + for n in range(0, self.num_gears): + nth_gear = self.num_gears - n - 1 + if should_carry: + count, lim = self.indicies[nth_gear] + count += 1 + if count == lim and nth_gear == 0: + self.cont = False + if count == lim: + should_carry = True + count = 0 + else: + should_carry = False + self.indicies[nth_gear] = (count, lim) + else: + break + + def __iter__(self): + return self + + def next(self): + if not self.cont: + raise StopIteration + l = [] + for x in range(0, self.num_gears): + index, limit = self.indicies[x] + l.append(self.gears[x][index]) + self.roll_gears() + return tuple(l) + class repeat(object): """Make an iterator that returns object over and over again. Added: pypy/branch/fast-forward/lib_pypy/pypy_test/test_itertools.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/lib_pypy/pypy_test/test_itertools.py Mon Nov 8 17:07:04 2010 @@ -0,0 +1,50 @@ +from py.test import raises +from lib_pypy import itertools + +class TestItertools(object): + + def test_compress(self): + it = itertools.compress(['a', 'b', 'c'], [0, 1, 0]) + + assert list(it) == ['b'] + + def test_compress_diff_len(self): + it = itertools.compress(['a'], []) + raises(StopIteration, it.next) + + def test_product(self): + l = [1, 2] + m = ['a', 'b'] + + prodlist = itertools.product(l, m) + assert list(prodlist) == [(1, 'a'), (1, 'b'), (2, 'a'), (2, 'b')] + + def test_product_repeat(self): + l = [1, 2] + m = ['a', 'b'] + + prodlist = itertools.product(l, m, repeat=2) + ans = [(1, 'a', 1, 'a'), (1, 'a', 1, 'b'), (1, 'a', 2, 'a'), + (1, 'a', 2, 'b'), (1, 'b', 1, 'a'), (1, 'b', 1, 'b'), + (1, 'b', 2, 'a'), (1, 'b', 2, 'b'), (2, 'a', 1, 'a'), + (2, 'a', 1, 'b'), (2, 'a', 2, 'a'), (2, 'a', 2, 'b'), + (2, 'b', 1, 'a'), (2, 'b', 1, 'b'), (2, 'b', 2, 'a'), + (2, 'b', 2, 'b')] + assert list(prodlist) == ans + + def test_product_diff_sizes(self): + l = [1, 2] + m = ['a'] + + prodlist = itertools.product(l, m) + assert list(prodlist) == [(1, 'a'), (2, 'a')] + + l = [1] + m = ['a', 'b'] + prodlist = itertools.product(l, m) + assert list(prodlist) == [(1, 'a'), (1, 'b')] + + def test_product_toomany_args(self): + l = [1, 2] + m = ['a'] + raises(TypeError, itertools.product, l, m, repeat=1, foo=2) Modified: pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py (original) +++ pypy/branch/fast-forward/pypy/module/itertools/interp_itertools.py Mon Nov 8 17:07:04 2010 @@ -954,9 +954,8 @@ return self.space.wrap(self) def next_w(self): - if not self.w_data or not self.w_selectors: - raise OperationError(self.space.w_StopIteration, self.space.w_None) - + # No need to check for StopIteration since either w_data + # or w_selectors will raise this. The shortest one stops first. while True: w_next_item = self.space.next(self.w_data) w_next_selector = self.space.next(self.w_selectors) From arigo at codespeak.net Mon Nov 8 17:28:13 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 8 Nov 2010 17:28:13 +0100 (CET) Subject: [pypy-svn] r78879 - pypy/trunk/pypy/rpython/memory/gc Message-ID: <20101108162813.7AE36282BDC@codespeak.net> Author: arigo Date: Mon Nov 8 17:28:11 2010 New Revision: 78879 Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py Log: An old change of mine. Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Mon Nov 8 17:28:11 2010 @@ -178,6 +178,7 @@ self.nursery = NULL self.nursery_free = NULL self.nursery_top = NULL + self.debug_always_do_minor_collect = False # # The ArenaCollection() handles the nonmovable objects allocation. if ArenaCollectionClass is None: @@ -245,6 +246,10 @@ # From there on, the GC is fully initialized and the code # below can use it newsize = base.read_from_env('PYPY_GC_NURSERY') + # PYPY_GC_NURSERY=1 forces a minor collect for every malloc. + # Useful to debug external factors, like trackgcroot or the + # handling of the write barrier. + self.debug_always_do_minor_collect = newsize == 1 if newsize <= 0: newsize = generation.estimate_best_nursery_size() if newsize <= 0: @@ -444,6 +449,10 @@ result = self.nursery_free self.nursery_free = result + totalsize ll_assert(self.nursery_free <= self.nursery_top, "nursery overflow") + # + if self.debug_always_do_minor_collect: + self.nursery_free = self.nursery_top + # return result collect_and_reserve._dont_inline_ = True From afa at codespeak.net Mon Nov 8 17:40:18 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 17:40:18 +0100 (CET) Subject: [pypy-svn] r78880 - pypy/branch/fast-forward/lib-python/modified-2.7.0/test Message-ID: <20101108164018.93D5B282BE0@codespeak.net> Author: afa Date: Mon Nov 8 17:40:17 2010 New Revision: 78880 Added: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_traceback.py - copied, changed from r78879, pypy/branch/fast-forward/lib-python/2.7.0/test/test_traceback.py Log: Use a "with" to close the file when done. Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_traceback.py (from r78879, pypy/branch/fast-forward/lib-python/2.7.0/test/test_traceback.py) ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_traceback.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_traceback.py Mon Nov 8 17:40:17 2010 @@ -74,7 +74,8 @@ try: sys.path.insert(0, testdir) testfile = os.path.join(testdir, 'test_bug737473.py') - print >> open(testfile, 'w'), """ + with open(testfile, 'w') as f: + print >> f, """ def test(): raise ValueError""" @@ -96,7 +97,8 @@ # three seconds are needed for this test to pass reliably :-( time.sleep(4) - print >> open(testfile, 'w'), """ + with open(testfile, 'w') as f: + print >> f, """ def test(): raise NotImplementedError""" reload(test_bug737473) From afa at codespeak.net Mon Nov 8 17:52:06 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 17:52:06 +0100 (CET) Subject: [pypy-svn] r78881 - pypy/branch/fast-forward/pypy/interpreter Message-ID: <20101108165206.A39A2282BDA@codespeak.net> Author: afa Date: Mon Nov 8 17:52:04 2010 New Revision: 78881 Modified: pypy/branch/fast-forward/pypy/interpreter/typedef.py Log: Code objects are weakrefable. Modified: pypy/branch/fast-forward/pypy/interpreter/typedef.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/typedef.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/typedef.py Mon Nov 8 17:52:04 2010 @@ -714,6 +714,7 @@ co_name = interp_attrproperty('co_name', cls=PyCode), co_firstlineno = interp_attrproperty('co_firstlineno', cls=PyCode), co_lnotab = interp_attrproperty('co_lnotab', cls=PyCode), + __weakref__ = make_weakref_descr(PyCode), ) PyCode.typedef.acceptable_as_base_class = False From arigo at codespeak.net Mon Nov 8 17:57:56 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 8 Nov 2010 17:57:56 +0100 (CET) Subject: [pypy-svn] r78882 - pypy/trunk/lib_pypy/ctypes_config_cache Message-ID: <20101108165756.1C59C282BE3@codespeak.net> Author: arigo Date: Mon Nov 8 17:57:55 2010 New Revision: 78882 Modified: pypy/trunk/lib_pypy/ctypes_config_cache/rebuild.py Log: Oups. Modified: pypy/trunk/lib_pypy/ctypes_config_cache/rebuild.py ============================================================================== --- pypy/trunk/lib_pypy/ctypes_config_cache/rebuild.py (original) +++ pypy/trunk/lib_pypy/ctypes_config_cache/rebuild.py Mon Nov 8 17:57:55 2010 @@ -48,7 +48,7 @@ if p.startswith('_') and (p.endswith('_cache.py') or p.endswith('_cache.pyc')): if p[:-9] not in left: - os.unlink(p) + os.unlink(os.path.join(_dirpath, p)) # for p in os.listdir(_dirpath): if p.endswith('.ctc.py'): From fijal at codespeak.net Mon Nov 8 18:01:00 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 8 Nov 2010 18:01:00 +0100 (CET) Subject: [pypy-svn] r78883 - pypy/branch/more-asm-debugging Message-ID: <20101108170100.89DA7282BE3@codespeak.net> Author: fijal Date: Mon Nov 8 18:00:59 2010 New Revision: 78883 Added: pypy/branch/more-asm-debugging/ (props changed) - copied from r78882, pypy/trunk/ Log: A branch to try to improve debuggability of asm backend From agaynor at codespeak.net Mon Nov 8 18:03:57 2010 From: agaynor at codespeak.net (agaynor at codespeak.net) Date: Mon, 8 Nov 2010 18:03:57 +0100 (CET) Subject: [pypy-svn] r78884 - in pypy/trunk/pypy/module/pypyjit: . test Message-ID: <20101108170357.1C089282BE8@codespeak.net> Author: agaynor Date: Mon Nov 8 18:03:54 2010 New Revision: 78884 Modified: pypy/trunk/pypy/module/pypyjit/policy.py pypy/trunk/pypy/module/pypyjit/test/test_policy.py Log: Allow the jit to inline property. Modified: pypy/trunk/pypy/module/pypyjit/policy.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/policy.py (original) +++ pypy/trunk/pypy/module/pypyjit/policy.py Mon Nov 8 18:03:54 2010 @@ -6,7 +6,8 @@ if (modname == '__builtin__.operation' or modname == '__builtin__.abstractinst' or modname == '__builtin__.interp_classobj' or - modname == '__builtin__.functional'): + modname == '__builtin__.functional' or + modname == '__builtin__.descriptor'): return True if '.' in modname: modname, _ = modname.split('.', 1) @@ -19,7 +20,7 @@ # this function should never actually return True directly # but instead call the base implementation mod = func.__module__ or '?' - + if mod.startswith('pypy.objspace.'): # gc_id operation if func.__name__ == 'id__ANY': @@ -36,5 +37,5 @@ modname = mod[len('pypy.module.'):] if not self.look_inside_pypy_module(modname): return False - + return True Modified: pypy/trunk/pypy/module/pypyjit/test/test_policy.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/test/test_policy.py (original) +++ pypy/trunk/pypy/module/pypyjit/test/test_policy.py Mon Nov 8 18:03:54 2010 @@ -12,7 +12,7 @@ def test_rlocale(): from pypy.rlib.rlocale import setlocale - assert not pypypolicy.look_inside_function(setlocale) + assert not pypypolicy.look_inside_function(setlocale) def test_geninterp(): d = {'_geninterp_': True} @@ -28,6 +28,10 @@ from pypy.interpreter.pyparser import parser assert not pypypolicy.look_inside_function(parser.Grammar.__init__.im_func) +def test_property(): + from pypy.module.__builtin__.descriptor import W_Property + assert pypypolicy.look_inside_function(W_Property.get.im_func) + def test_pypy_module(): from pypy.module._random.interp_random import W_Random assert not pypypolicy.look_inside_function(W_Random.random) @@ -35,6 +39,7 @@ assert pypypolicy.look_inside_pypy_module('__builtin__.operation') assert pypypolicy.look_inside_pypy_module('__builtin__.abstractinst') assert pypypolicy.look_inside_pypy_module('__builtin__.functional') + assert pypypolicy.look_inside_pypy_module('__builtin__.descriptor') assert pypypolicy.look_inside_pypy_module('exceptions.interp_exceptions') for modname in 'pypyjit', 'signal', 'micronumpy', 'math', 'imp': assert pypypolicy.look_inside_pypy_module(modname) @@ -42,4 +47,3 @@ def test_see_jit_module(): assert pypypolicy.look_inside_pypy_module('pypyjit.interp_jit') - From afa at codespeak.net Mon Nov 8 18:10:21 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 18:10:21 +0100 (CET) Subject: [pypy-svn] r78885 - in pypy/branch/fast-forward/pypy/module/cpyext: . src test Message-ID: <20101108171021.370BC282BE9@codespeak.net> Author: afa Date: Mon Nov 8 18:10:19 2010 New Revision: 78885 Modified: pypy/branch/fast-forward/pypy/module/cpyext/longobject.py pypy/branch/fast-forward/pypy/module/cpyext/src/getargs.c pypy/branch/fast-forward/pypy/module/cpyext/test/test_longobject.py Log: Add PyLong_AsUnsignedLongLongMask, and remove most "#if 0" in getargs.c Modified: pypy/branch/fast-forward/pypy/module/cpyext/longobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/longobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/longobject.py Mon Nov 8 18:10:19 2010 @@ -70,6 +70,15 @@ raised.""" return rffi.cast(rffi.ULONGLONG, space.r_ulonglong_w(w_long)) + at cpython_api([PyObject], rffi.ULONGLONG, error=-1) +def PyLong_AsUnsignedLongLongMask(space, w_long): + """Will first attempt to cast the object to a PyIntObject or + PyLongObject, if it is not already one, and then return its value as + unsigned long long, without checking for overflow. + """ + num = space.bigint_w(w_long) + return num.ulonglongmask() + @cpython_api([PyObject, rffi.CArrayPtr(rffi.INT_real)], lltype.Signed, error=-1) def PyLong_AsLongAndOverflow(space, w_long, overflow_ptr): Modified: pypy/branch/fast-forward/pypy/module/cpyext/src/getargs.c ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/src/getargs.c (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/src/getargs.c Mon Nov 8 18:10:19 2010 @@ -557,8 +557,6 @@ switch (c) { case 'b': { /* unsigned byte -- very short int */ - Py_FatalError("'b' unimplemented for PyArg_*\n"); -#if 0 char *p = va_arg(*p_va, char *); long ival; if (float_argument_error(arg)) @@ -579,13 +577,10 @@ else *p = (unsigned char) ival; break; -#endif } case 'B': {/* byte sized bitfield - both signed and unsigned values allowed */ - Py_FatalError("'B' unimplemented for PyArg_*\n"); -#if 0 char *p = va_arg(*p_va, char *); long ival; if (float_argument_error(arg)) @@ -596,12 +591,9 @@ else *p = (unsigned char) ival; break; -#endif } case 'h': {/* signed short int */ - Py_FatalError("'h' unimplemented for PyArg_*\n"); -#if 0 short *p = va_arg(*p_va, short *); long ival; if (float_argument_error(arg)) @@ -622,13 +614,10 @@ else *p = (short) ival; break; -#endif } case 'H': { /* short int sized bitfield, both signed and unsigned allowed */ - Py_FatalError("'H' unimplemented for PyArg_*\n"); -#if 0 unsigned short *p = va_arg(*p_va, unsigned short *); long ival; if (float_argument_error(arg)) @@ -639,7 +628,6 @@ else *p = (unsigned short) ival; break; -#endif } case 'i': {/* signed int */ int *p = va_arg(*p_va, int *); @@ -665,8 +653,6 @@ } case 'I': { /* int sized bitfield, both signed and unsigned allowed */ - Py_FatalError("'I' unimplemented for PyArg_*\n"); -#if 0 unsigned int *p = va_arg(*p_va, unsigned int *); unsigned int ival; if (float_argument_error(arg)) @@ -677,7 +663,6 @@ else *p = ival; break; -#endif } case 'n': /* Py_ssize_t */ #if SIZEOF_SIZE_T != SIZEOF_LONG @@ -708,8 +693,6 @@ } case 'k': { /* long sized bitfield */ - Py_FatalError("'k' unimplemented for PyArg_*\n"); -#if 0 unsigned long *p = va_arg(*p_va, unsigned long *); unsigned long ival; if (PyInt_Check(arg)) @@ -720,13 +703,10 @@ return converterr("integer", arg, msgbuf, bufsize); *p = ival; break; -#endif } #ifdef HAVE_LONG_LONG case 'L': {/* PY_LONG_LONG */ - Py_FatalError("'L' unimplemented for PyArg_*\n"); -#if 0 PY_LONG_LONG *p = va_arg( *p_va, PY_LONG_LONG * ); PY_LONG_LONG ival = PyLong_AsLongLong( arg ); if (ival == (PY_LONG_LONG)-1 && PyErr_Occurred() ) { @@ -735,12 +715,9 @@ *p = ival; } break; -#endif } case 'K': { /* long long sized bitfield */ - Py_FatalError("'K' unimplemented for PyArg_*\n"); -#if 0 unsigned PY_LONG_LONG *p = va_arg(*p_va, unsigned PY_LONG_LONG *); unsigned PY_LONG_LONG ival; if (PyInt_Check(arg)) @@ -751,7 +728,6 @@ return converterr("integer", arg, msgbuf, bufsize); *p = ival; break; -#endif } #endif // HAVE_LONG_LONG @@ -777,8 +753,6 @@ #ifndef WITHOUT_COMPLEX case 'D': {/* complex double */ - Py_FatalError("'D' unimplemented for PyArg_*\n"); -#if 0 Py_complex *p = va_arg(*p_va, Py_complex *); Py_complex cval; cval = PyComplex_AsCComplex(arg); @@ -787,7 +761,6 @@ else *p = cval; break; -#endif } #endif /* WITHOUT_COMPLEX */ @@ -985,8 +958,6 @@ break; } case 'e': {/* encoded string */ - Py_FatalError("'e' unimplemented for PyArg_*\n"); -#if 0 char **buffer; const char *encoding; PyObject *s; @@ -1150,7 +1121,6 @@ } Py_DECREF(s); break; -#endif } #ifdef Py_USING_UNICODE Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_longobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/test/test_longobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_longobject.py Mon Nov 8 18:10:19 2010 @@ -64,8 +64,11 @@ assert api.PyErr_Occurred() api.PyErr_Clear() + assert api.PyLong_AsUnsignedLongLongMask( + space.wrap(1<<64)) == 0 + def test_as_long_and_overflow(self, space, api): - overflow = lltype.malloc(rffi.INTP.TO, 1, flavor='raw') + overflow = lltype.malloc(rffi.CArrayPtr(rffi.INT_real).TO, 1, flavor='raw') assert api.PyLong_AsLongAndOverflow( space.wrap(sys.maxint), overflow) == sys.maxint assert api.PyLong_AsLongAndOverflow( @@ -75,7 +78,7 @@ lltype.free(overflow, flavor='raw') def test_as_longlong_and_overflow(self, space, api): - overflow = lltype.malloc(rffi.INTP.TO, 1, flavor='raw') + overflow = lltype.malloc(rffi.CArrayPtr(rffi.INT_real).TO, 1, flavor='raw') assert api.PyLong_AsLongLongAndOverflow( space.wrap(1<<62), overflow) == 1<<62 assert api.PyLong_AsLongLongAndOverflow( From agaynor at codespeak.net Mon Nov 8 18:13:55 2010 From: agaynor at codespeak.net (agaynor at codespeak.net) Date: Mon, 8 Nov 2010 18:13:55 +0100 (CET) Subject: [pypy-svn] r78886 - pypy/trunk/pypy/module/__builtin__ Message-ID: <20101108171355.E0ABB282BEA@codespeak.net> Author: agaynor Date: Mon Nov 8 18:13:54 2010 New Revision: 78886 Modified: pypy/trunk/pypy/module/__builtin__/descriptor.py Log: Mark property's fields as bein immutable. Modified: pypy/trunk/pypy/module/__builtin__/descriptor.py ============================================================================== --- pypy/trunk/pypy/module/__builtin__/descriptor.py (original) +++ pypy/trunk/pypy/module/__builtin__/descriptor.py Mon Nov 8 18:13:54 2010 @@ -96,6 +96,8 @@ ) class W_Property(Wrappable): + _immutable_fields_ = ["w_fget", "w_fset", "w_fdel"] + def init(self, space, w_fget=None, w_fset=None, w_fdel=None, w_doc=None): self.w_fget = w_fget self.w_fset = w_fset @@ -183,4 +185,3 @@ fget = interp_attrproperty_w('w_fget', W_Property), fset = interp_attrproperty_w('w_fset', W_Property), ) - From fijal at codespeak.net Mon Nov 8 18:23:54 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 8 Nov 2010 18:23:54 +0100 (CET) Subject: [pypy-svn] r78887 - in pypy/branch/more-asm-debugging/pypy/jit/backend/x86: . test Message-ID: <20101108172354.41F08282BEB@codespeak.net> Author: fijal Date: Mon Nov 8 18:23:52 2010 New Revision: 78887 Modified: pypy/branch/more-asm-debugging/pypy/jit/backend/x86/assembler.py pypy/branch/more-asm-debugging/pypy/jit/backend/x86/test/test_runner.py Log: implement a checksum feature, to be able to gdb on exact loop (not sure how this is useful, but let's see) Modified: pypy/branch/more-asm-debugging/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/more-asm-debugging/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/more-asm-debugging/pypy/jit/backend/x86/assembler.py Mon Nov 8 18:23:52 2010 @@ -303,6 +303,7 @@ _x86_frame_depth _x86_param_depth _x86_arglocs + _x86_debug_checksum """ if not we_are_translated(): # Arguments should be unique @@ -312,7 +313,7 @@ funcname = self._find_debug_merge_point(operations) if log: self._register_counter() - operations = self._inject_debugging_code(operations) + operations = self._inject_debugging_code(looptoken, operations) regalloc = RegAlloc(self, self.cpu.translate_support_code) arglocs = regalloc.prepare_loop(inputargs, operations, looptoken) @@ -350,7 +351,7 @@ funcname = self._find_debug_merge_point(operations) if log: self._register_counter() - operations = self._inject_debugging_code(operations) + operations = self._inject_debugging_code(looptoken, operations) arglocs = self.rebuild_faillocs_from_descr( faildescr._x86_failure_recovery_bytecode) @@ -432,9 +433,13 @@ mc.done() - def _inject_debugging_code(self, operations): + def _inject_debugging_code(self, looptoken, operations): if self._debug: # before doing anything, let's increase a counter + s = 0 + for op in operations: + s += op.getopnum() + looptoken._x86_debug_checksum = s c_adr = ConstInt(rffi.cast(lltype.Signed, self.loop_run_counters[-1][1])) box = BoxInt() Modified: pypy/branch/more-asm-debugging/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/branch/more-asm-debugging/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/branch/more-asm-debugging/pypy/jit/backend/x86/test/test_runner.py Mon Nov 8 18:23:52 2010 @@ -515,3 +515,20 @@ self.cpu.finish_once() lines = py.path.local(self.logfile + ".count").readlines() assert lines[0] == '0:10\n' # '10 xyz\n' + + def test_debugger_checksum(self): + loop = """ + [i0] + debug_merge_point('xyz', 0) + i1 = int_add(i0, 1) + i2 = int_ge(i1, 10) + guard_false(i2) [] + jump(i1) + """ + ops = parse(loop) + self.cpu.assembler.set_debug(True) + self.cpu.compile_loop(ops.inputargs, ops.operations, ops.token) + self.cpu.set_future_value_int(0, 0) + self.cpu.execute_token(ops.token) + assert ops.token._x86_debug_checksum == sum([op.getopnum() + for op in ops.operations]) From fijal at codespeak.net Mon Nov 8 18:28:13 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 8 Nov 2010 18:28:13 +0100 (CET) Subject: [pypy-svn] r78888 - pypy/branch/more-asm-debugging/pypy/jit/backend/x86 Message-ID: <20101108172813.7FE78282BEC@codespeak.net> Author: fijal Date: Mon Nov 8 18:28:12 2010 New Revision: 78888 Modified: pypy/branch/more-asm-debugging/pypy/jit/backend/x86/assembler.py Log: * Use %x for displaying addresses * Fix for bridges Modified: pypy/branch/more-asm-debugging/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/more-asm-debugging/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/more-asm-debugging/pypy/jit/backend/x86/assembler.py Mon Nov 8 18:28:12 2010 @@ -337,8 +337,9 @@ self._assemble_bootstrap_direct_call(arglocs, curadr, frame_depth+param_depth) # - debug_print("Loop #", looptoken.number, "has address", - looptoken._x86_loop_code, "to", self.mc.tell()) + debug_print("Loop #%d has address %x to %x" % (looptoken.number, + looptoken._x86_loop_code, + self.mc.tell())) self.mc.end_function() self.write_pending_failure_recoveries() @@ -351,7 +352,7 @@ funcname = self._find_debug_merge_point(operations) if log: self._register_counter() - operations = self._inject_debugging_code(looptoken, operations) + operations = self._inject_debugging_code(faildescr, operations) arglocs = self.rebuild_faillocs_from_descr( faildescr._x86_failure_recovery_bytecode) @@ -378,9 +379,8 @@ faildescr._x86_bridge_param_depth = param_depth # patch the jump from original guard self.patch_jump_for_descr(faildescr, adr_bridge) - debug_print("Bridge out of guard", - descr_number, - "has address", adr_bridge, "to", self.mc.tell()) + debug_print("Bridge out of guard %d has address %x to %x" % + (descr_number, adr_bridge, self.mc.tell())) self.mc.end_function() self.write_pending_failure_recoveries() @@ -433,6 +433,7 @@ mc.done() + @specialize.argtype(1) def _inject_debugging_code(self, looptoken, operations): if self._debug: # before doing anything, let's increase a counter From fijal at codespeak.net Mon Nov 8 18:32:37 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 8 Nov 2010 18:32:37 +0100 (CET) Subject: [pypy-svn] r78889 - in pypy/trunk/pypy/jit/backend/x86: . test Message-ID: <20101108173237.C1EB5282BF0@codespeak.net> Author: fijal Date: Mon Nov 8 18:32:36 2010 New Revision: 78889 Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py pypy/trunk/pypy/jit/backend/x86/test/test_runner.py Log: Merge more-asm-debugging branch. It adds a checksum on each of loops, so we can grab it and put a gdb on executing a specific loop Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/assembler.py (original) +++ pypy/trunk/pypy/jit/backend/x86/assembler.py Mon Nov 8 18:32:36 2010 @@ -303,6 +303,7 @@ _x86_frame_depth _x86_param_depth _x86_arglocs + _x86_debug_checksum """ if not we_are_translated(): # Arguments should be unique @@ -312,7 +313,7 @@ funcname = self._find_debug_merge_point(operations) if log: self._register_counter() - operations = self._inject_debugging_code(operations) + operations = self._inject_debugging_code(looptoken, operations) regalloc = RegAlloc(self, self.cpu.translate_support_code) arglocs = regalloc.prepare_loop(inputargs, operations, looptoken) @@ -336,8 +337,9 @@ self._assemble_bootstrap_direct_call(arglocs, curadr, frame_depth+param_depth) # - debug_print("Loop #", looptoken.number, "has address", - looptoken._x86_loop_code, "to", self.mc.tell()) + debug_print("Loop #%d has address %x to %x" % (looptoken.number, + looptoken._x86_loop_code, + self.mc.tell())) self.mc.end_function() self.write_pending_failure_recoveries() @@ -350,7 +352,7 @@ funcname = self._find_debug_merge_point(operations) if log: self._register_counter() - operations = self._inject_debugging_code(operations) + operations = self._inject_debugging_code(faildescr, operations) arglocs = self.rebuild_faillocs_from_descr( faildescr._x86_failure_recovery_bytecode) @@ -377,9 +379,8 @@ faildescr._x86_bridge_param_depth = param_depth # patch the jump from original guard self.patch_jump_for_descr(faildescr, adr_bridge) - debug_print("Bridge out of guard", - descr_number, - "has address", adr_bridge, "to", self.mc.tell()) + debug_print("Bridge out of guard %d has address %x to %x" % + (descr_number, adr_bridge, self.mc.tell())) self.mc.end_function() self.write_pending_failure_recoveries() @@ -432,9 +433,14 @@ mc.done() - def _inject_debugging_code(self, operations): + @specialize.argtype(1) + def _inject_debugging_code(self, looptoken, operations): if self._debug: # before doing anything, let's increase a counter + s = 0 + for op in operations: + s += op.getopnum() + looptoken._x86_debug_checksum = s c_adr = ConstInt(rffi.cast(lltype.Signed, self.loop_run_counters[-1][1])) box = BoxInt() Modified: pypy/trunk/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/trunk/pypy/jit/backend/x86/test/test_runner.py Mon Nov 8 18:32:36 2010 @@ -515,3 +515,20 @@ self.cpu.finish_once() lines = py.path.local(self.logfile + ".count").readlines() assert lines[0] == '0:10\n' # '10 xyz\n' + + def test_debugger_checksum(self): + loop = """ + [i0] + debug_merge_point('xyz', 0) + i1 = int_add(i0, 1) + i2 = int_ge(i1, 10) + guard_false(i2) [] + jump(i1) + """ + ops = parse(loop) + self.cpu.assembler.set_debug(True) + self.cpu.compile_loop(ops.inputargs, ops.operations, ops.token) + self.cpu.set_future_value_int(0, 0) + self.cpu.execute_token(ops.token) + assert ops.token._x86_debug_checksum == sum([op.getopnum() + for op in ops.operations]) From fijal at codespeak.net Mon Nov 8 18:32:57 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 8 Nov 2010 18:32:57 +0100 (CET) Subject: [pypy-svn] r78890 - pypy/branch/more-asm-debugging Message-ID: <20101108173257.C6954282BF1@codespeak.net> Author: fijal Date: Mon Nov 8 18:32:56 2010 New Revision: 78890 Removed: pypy/branch/more-asm-debugging/ Log: delete merged branch From arigo at codespeak.net Mon Nov 8 19:14:28 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 8 Nov 2010 19:14:28 +0100 (CET) Subject: [pypy-svn] r78891 - pypy/trunk/pypy/translator/c/gcc Message-ID: <20101108181428.566BC282BDD@codespeak.net> Author: arigo Date: Mon Nov 8 19:14:26 2010 New Revision: 78891 Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py Log: We can now default to verbose=0 and not print the dots of progress. Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py ============================================================================== --- pypy/trunk/pypy/translator/c/gcc/trackgcroot.py (original) +++ pypy/trunk/pypy/translator/c/gcc/trackgcroot.py Mon Nov 8 19:14:26 2010 @@ -1904,7 +1904,7 @@ if __name__ == '__main__': - verbose = 1 + verbose = 0 shuffle = False output_raw_table = False if sys.platform == 'darwin': From afa at codespeak.net Mon Nov 8 21:34:01 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 21:34:01 +0100 (CET) Subject: [pypy-svn] r78892 - in pypy/branch/fast-forward/pypy/interpreter: astcompiler/test pyparser Message-ID: <20101108203401.7DB0D282B9E@codespeak.net> Author: afa Date: Mon Nov 8 21:33:57 2010 New Revision: 78892 Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py Log: "0x", "0b" are not valid tokens and should raise SyntaxError Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/astcompiler/test/test_astbuilder.py Mon Nov 8 21:33:57 2010 @@ -1147,6 +1147,10 @@ for num in ("0b00101", "0B00101", "0b101", "0B101"): assert space.eq_w(get_num(num), space.wrap(5)) + py.test.raises(SyntaxError, self.get_ast, "0x") + py.test.raises(SyntaxError, self.get_ast, "0b") + py.test.raises(SyntaxError, self.get_ast, "0o") + def check_comprehension(self, brackets, ast_type): def brack(s): return brackets % s Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/pyparser/genpytokenize.py Mon Nov 8 21:33:57 2010 @@ -59,17 +59,21 @@ hexNumber = chain(states, newArcPair(states, "0"), groupStr(states, "xX"), - any(states, groupStr(states, "0123456789abcdefABCDEF")), + atleastonce(states, + groupStr(states, "0123456789abcdefABCDEF")), maybe(states, groupStr(states, "lL"))) octNumber = chain(states, newArcPair(states, "0"), - maybe(states, groupStr(states, "oO")), + maybe(states, + chain(states, + groupStr(states, "oO"), + groupStr(states, "01234567"))), any(states, groupStr(states, "01234567")), maybe(states, groupStr(states, "lL"))) binNumber = chain(states, newArcPair(states, "0"), groupStr(states, "bB"), - any(states, groupStr(states, "01")), + atleastonce(states, groupStr(states, "01")), maybe(states, groupStr(states, "lL"))) decNumber = chain(states, groupStr(states, "123456789"), Modified: pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/pyparser/pytokenize.py Mon Nov 8 21:33:57 2010 @@ -25,10 +25,10 @@ accepts = [True, True, True, True, True, True, True, True, True, True, False, True, True, True, True, False, - False, False, True, True, True, True, True, False, - True, False, True, False, True, False, False, - True, False, False, False, False, True, False, - False, False, True] + False, False, True, False, False, True, False, + False, True, False, True, False, True, False, + False, True, False, False, True, True, True, + False, False, True, False, False, False, True] states = [ # 0 {'\t': 0, '\n': 13, '\x0c': 0, @@ -153,16 +153,15 @@ # 18 {automata.DEFAULT: 18, '\n': 27, '\r': 27}, # 19 - {'0': 19, '1': 19, '2': 19, '3': 19, - '4': 19, '5': 19, '6': 19, '7': 19, - '8': 19, '9': 19, 'A': 19, 'B': 19, - 'C': 19, 'D': 19, 'E': 19, 'F': 19, - 'L': 13, 'a': 19, 'b': 19, 'c': 19, - 'd': 19, 'e': 19, 'f': 19, 'l': 13}, + {'0': 34, '1': 34, '2': 34, '3': 34, + '4': 34, '5': 34, '6': 34, '7': 34, + '8': 34, '9': 34, 'A': 34, 'B': 34, + 'C': 34, 'D': 34, 'E': 34, 'F': 34, + 'a': 34, 'b': 34, 'c': 34, 'd': 34, + 'e': 34, 'f': 34}, # 20 - {'0': 20, '1': 20, '2': 20, '3': 20, - '4': 20, '5': 20, '6': 20, '7': 20, - 'L': 13, 'l': 13}, + {'0': 35, '1': 35, '2': 35, '3': 35, + '4': 35, '5': 35, '6': 35, '7': 35}, # 21 {'.': 24, '0': 21, '1': 21, '2': 21, '3': 21, '4': 21, '5': 21, '6': 21, @@ -170,7 +169,7 @@ 'J': 13, 'L': 13, 'e': 25, 'j': 13, 'l': 13}, # 22 - {'0': 22, '1': 22, 'L': 13, 'l': 13}, + {'0': 36, '1': 36}, # 23 {'.': 24, '0': 23, '1': 23, '2': 23, '3': 23, '4': 23, '5': 23, '6': 23, @@ -179,59 +178,72 @@ # 24 {'0': 24, '1': 24, '2': 24, '3': 24, '4': 24, '5': 24, '6': 24, '7': 24, - '8': 24, '9': 24, 'E': 34, 'J': 13, - 'e': 34, 'j': 13}, + '8': 24, '9': 24, 'E': 37, 'J': 13, + 'e': 37, 'j': 13}, # 25 - {'+': 35, '-': 35, '0': 36, '1': 36, - '2': 36, '3': 36, '4': 36, '5': 36, - '6': 36, '7': 36, '8': 36, '9': 36}, + {'+': 38, '-': 38, '0': 39, '1': 39, + '2': 39, '3': 39, '4': 39, '5': 39, + '6': 39, '7': 39, '8': 39, '9': 39}, # 26 {'0': 26, '1': 26, '2': 26, '3': 26, '4': 26, '5': 26, '6': 26, '7': 26, - '8': 26, '9': 26, 'E': 34, 'J': 13, - 'e': 34, 'j': 13}, + '8': 26, '9': 26, 'E': 37, 'J': 13, + 'e': 37, 'j': 13}, # 27 {}, # 28 {"'": 13}, # 29 - {automata.DEFAULT: 37, '\n': 13, '\r': 14}, + {automata.DEFAULT: 40, '\n': 13, '\r': 14}, # 30 {automata.DEFAULT: 30, '\n': 27, '\r': 27, "'": 13, '\\': 29}, # 31 {'"': 13}, # 32 - {automata.DEFAULT: 38, '\n': 13, '\r': 14}, + {automata.DEFAULT: 41, '\n': 13, '\r': 14}, # 33 {automata.DEFAULT: 33, '\n': 27, '\r': 27, '"': 13, '\\': 32}, # 34 - {'+': 39, '-': 39, '0': 40, '1': 40, - '2': 40, '3': 40, '4': 40, '5': 40, - '6': 40, '7': 40, '8': 40, '9': 40}, + {'0': 34, '1': 34, '2': 34, '3': 34, + '4': 34, '5': 34, '6': 34, '7': 34, + '8': 34, '9': 34, 'A': 34, 'B': 34, + 'C': 34, 'D': 34, 'E': 34, 'F': 34, + 'L': 13, 'a': 34, 'b': 34, 'c': 34, + 'd': 34, 'e': 34, 'f': 34, 'l': 13}, # 35 - {'0': 36, '1': 36, '2': 36, '3': 36, - '4': 36, '5': 36, '6': 36, '7': 36, - '8': 36, '9': 36}, + {'0': 35, '1': 35, '2': 35, '3': 35, + '4': 35, '5': 35, '6': 35, '7': 35, + 'L': 13, 'l': 13}, # 36 - {'0': 36, '1': 36, '2': 36, '3': 36, - '4': 36, '5': 36, '6': 36, '7': 36, - '8': 36, '9': 36, 'J': 13, 'j': 13}, + {'0': 36, '1': 36, 'L': 13, 'l': 13}, # 37 - {automata.DEFAULT: 37, '\n': 27, - '\r': 27, "'": 13, '\\': 29}, + {'+': 42, '-': 42, '0': 43, '1': 43, + '2': 43, '3': 43, '4': 43, '5': 43, + '6': 43, '7': 43, '8': 43, '9': 43}, # 38 - {automata.DEFAULT: 38, '\n': 27, - '\r': 27, '"': 13, '\\': 32}, + {'0': 39, '1': 39, '2': 39, '3': 39, + '4': 39, '5': 39, '6': 39, '7': 39, + '8': 39, '9': 39}, # 39 - {'0': 40, '1': 40, '2': 40, '3': 40, - '4': 40, '5': 40, '6': 40, '7': 40, - '8': 40, '9': 40}, + {'0': 39, '1': 39, '2': 39, '3': 39, + '4': 39, '5': 39, '6': 39, '7': 39, + '8': 39, '9': 39, 'J': 13, 'j': 13}, # 40 - {'0': 40, '1': 40, '2': 40, '3': 40, - '4': 40, '5': 40, '6': 40, '7': 40, - '8': 40, '9': 40, 'J': 13, 'j': 13}, + {automata.DEFAULT: 40, '\n': 27, + '\r': 27, "'": 13, '\\': 29}, + # 41 + {automata.DEFAULT: 41, '\n': 27, + '\r': 27, '"': 13, '\\': 32}, + # 42 + {'0': 43, '1': 43, '2': 43, '3': 43, + '4': 43, '5': 43, '6': 43, '7': 43, + '8': 43, '9': 43}, + # 43 + {'0': 43, '1': 43, '2': 43, '3': 43, + '4': 43, '5': 43, '6': 43, '7': 43, + '8': 43, '9': 43, 'J': 13, 'j': 13}, ] pseudoDFA = automata.DFA(states, accepts) @@ -307,9 +319,6 @@ 'b' : None, 'B' : None} -#_______________________________________________________________________ -# End of automatically generated DFA's - for uniPrefix in ("", "u", "U", "b", "B"): for rawPrefix in ("", "r", "R"): prefix = uniPrefix + rawPrefix From afa at codespeak.net Mon Nov 8 21:40:33 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 21:40:33 +0100 (CET) Subject: [pypy-svn] r78893 - pypy/branch/fast-forward/lib_pypy Message-ID: <20101108204033.ED809282B9E@codespeak.net> Author: afa Date: Mon Nov 8 21:40:31 2010 New Revision: 78893 Modified: pypy/branch/fast-forward/lib_pypy/_collections.py Log: deque: Fix "d.extend(d)" Modified: pypy/branch/fast-forward/lib_pypy/_collections.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_collections.py (original) +++ pypy/branch/fast-forward/lib_pypy/_collections.py Mon Nov 8 21:40:31 2010 @@ -76,6 +76,8 @@ self.pop() def extend(self, iterable): + if iterable is self: + iterable = list(iterable) for elem in iterable: self.append(elem) From afa at codespeak.net Mon Nov 8 21:41:44 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 21:41:44 +0100 (CET) Subject: [pypy-svn] r78894 - pypy/branch/fast-forward/lib_pypy Message-ID: <20101108204144.7EA0D282BDC@codespeak.net> Author: afa Date: Mon Nov 8 21:41:43 2010 New Revision: 78894 Modified: pypy/branch/fast-forward/lib_pypy/_collections.py Log: deque: fix "d.extendleft(d)" Modified: pypy/branch/fast-forward/lib_pypy/_collections.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_collections.py (original) +++ pypy/branch/fast-forward/lib_pypy/_collections.py Mon Nov 8 21:41:43 2010 @@ -82,6 +82,8 @@ self.append(elem) def extendleft(self, iterable): + if iterable is self: + iterable = list(iterable) for elem in iterable: self.appendleft(elem) From afa at codespeak.net Mon Nov 8 21:55:09 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 21:55:09 +0100 (CET) Subject: [pypy-svn] r78895 - in pypy/branch/fast-forward: lib-python/modified-2.7.0/test lib_pypy Message-ID: <20101108205509.D54D7282BDD@codespeak.net> Author: afa Date: Mon Nov 8 21:55:08 2010 New Revision: 78895 Added: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_deque.py - copied, changed from r78894, pypy/branch/fast-forward/lib-python/2.7.0/test/test_deque.py Modified: pypy/branch/fast-forward/lib_pypy/_collections.py Log: Fix all tests in test_deque.py Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_deque.py (from r78894, pypy/branch/fast-forward/lib-python/2.7.0/test/test_deque.py) ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_deque.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_deque.py Mon Nov 8 21:55:08 2010 @@ -505,7 +505,7 @@ container = reversed(deque([obj, 1])) obj.x = iter(container) del obj, container - gc.collect() + test_support.gc_collect() self.assertTrue(ref() is None, "Cycle was not collected") class TestVariousIteratorArgs(unittest.TestCase): @@ -621,6 +621,7 @@ p = weakref.proxy(d) self.assertEqual(str(p), str(d)) d = None + test_support.gc_collect() self.assertRaises(ReferenceError, str, p) def test_strange_subclass(self): Modified: pypy/branch/fast-forward/lib_pypy/_collections.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_collections.py (original) +++ pypy/branch/fast-forward/lib_pypy/_collections.py Mon Nov 8 21:55:08 2010 @@ -32,14 +32,19 @@ return self def __init__(self, iterable=(), maxlen=None): + self.clear() if maxlen is not None: if maxlen < 0: raise ValueError("maxlen must be non-negative") - self.maxlen = maxlen + self._maxlen = maxlen add = self.append for elem in iterable: add(elem) + @property + def maxlen(self): + return self._maxlen + def clear(self): self.right = self.left = [None] * BLOCKSIZ self.rightndx = n//2 # points to last written element @@ -350,6 +355,10 @@ else: return NotImplemented + def __iadd__(self, other): + self.extend(other) + return self + class deque_iterator(object): def __init__(self, deq, itergen): From afa at codespeak.net Mon Nov 8 21:58:30 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 21:58:30 +0100 (CET) Subject: [pypy-svn] r78896 - pypy/branch/fast-forward/lib-python Message-ID: <20101108205830.2B95A282BE0@codespeak.net> Author: afa Date: Mon Nov 8 21:58:28 2010 New Revision: 78896 Modified: pypy/branch/fast-forward/lib-python/TODO Log: More TODO items Modified: pypy/branch/fast-forward/lib-python/TODO ============================================================================== --- pypy/branch/fast-forward/lib-python/TODO (original) +++ pypy/branch/fast-forward/lib-python/TODO Mon Nov 8 21:58:28 2010 @@ -36,6 +36,9 @@ - some set() methods now accept a variable number of arguments DONE +- remove code duplication: bit_length() and _count_bits() in rlib/rbigint.py, + objspace/std/longobject.py and objspace/std/longtype.py. + Medium tasks ------------ @@ -47,6 +50,8 @@ - signal.set_wakeup_fd() DONE +- Dictionary views: dict.viewkeys(), dict.viewvalues(), dict.viewitems() + - add 'unicode' in ObjSpace.MethodTable + probably a default implementation that falls back to space.str(). From afa at codespeak.net Mon Nov 8 22:42:40 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 22:42:40 +0100 (CET) Subject: [pypy-svn] r78899 - pypy/branch/fast-forward/lib-python/modified-2.7.0/test Message-ID: <20101108214240.48FF9282B9D@codespeak.net> Author: afa Date: Mon Nov 8 22:42:37 2010 New Revision: 78899 Added: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_bz2.py - copied, changed from r78894, pypy/branch/fast-forward/lib-python/2.7.0/test/test_bz2.py Log: Fix tests in test_bz2.py Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_bz2.py (from r78894, pypy/branch/fast-forward/lib-python/2.7.0/test/test_bz2.py) ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_bz2.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_bz2.py Mon Nov 8 22:42:37 2010 @@ -246,6 +246,8 @@ for i in xrange(10000): o = BZ2File(self.filename) del o + if i % 100 == 0: + test_support.gc_collect() def testOpenNonexistent(self): # "Test opening a nonexistent file" @@ -310,6 +312,7 @@ for t in threads: t.join() + @test_support.impl_detail() def testMixedIterationReads(self): # Issue #8397: mixed iteration and reads should be forbidden. with bz2.BZ2File(self.filename, 'wb') as f: From afa at codespeak.net Mon Nov 8 22:49:26 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 22:49:26 +0100 (CET) Subject: [pypy-svn] r78900 - pypy/branch/fast-forward/lib_pypy Message-ID: <20101108214926.0F7B4282B9D@codespeak.net> Author: afa Date: Mon Nov 8 22:49:25 2010 New Revision: 78900 Modified: pypy/branch/fast-forward/lib_pypy/_testcapi.py Log: Fix _testcapi compilation on 64bit linux Modified: pypy/branch/fast-forward/lib_pypy/_testcapi.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_testcapi.py (original) +++ pypy/branch/fast-forward/lib_pypy/_testcapi.py Mon Nov 8 22:49:25 2010 @@ -13,8 +13,13 @@ # Compile .c file include_dir = os.path.join(thisdir, '..', 'include') + if sys.platform == 'win32': + ccflags = [] + else: + ccflags = ['-fPIC', '-Wimplicit-function-declaration'] res = compiler.compile([os.path.join(thisdir, '_testcapimodule.c')], - include_dirs=[include_dir]) + include_dirs=[include_dir], + extra_preargs=ccflags) object_filename = res[0] # set link options From afa at codespeak.net Mon Nov 8 23:46:58 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 8 Nov 2010 23:46:58 +0100 (CET) Subject: [pypy-svn] r78901 - pypy/branch/fast-forward/lib-python/2.7.0/lib2to3 Message-ID: <20101108224658.D45CC282B9D@codespeak.net> Author: afa Date: Mon Nov 8 23:46:55 2010 New Revision: 78901 Added: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/btm_matcher.py pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/btm_utils.py Log: Add missing files in lib2to3 Added: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/btm_matcher.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/btm_matcher.py Mon Nov 8 23:46:55 2010 @@ -0,0 +1,168 @@ +"""A bottom-up tree matching algorithm implementation meant to speed +up 2to3's matching process. After the tree patterns are reduced to +their rarest linear path, a linear Aho-Corasick automaton is +created. The linear automaton traverses the linear paths from the +leaves to the root of the AST and returns a set of nodes for further +matching. This reduces significantly the number of candidate nodes.""" + +__author__ = "George Boutsioukis " + +import logging +import itertools +from collections import defaultdict + +from . import pytree +from .btm_utils import reduce_tree + +class BMNode(object): + """Class for a node of the Aho-Corasick automaton used in matching""" + count = itertools.count() + def __init__(self): + self.transition_table = {} + self.fixers = [] + self.id = next(BMNode.count) + self.content = '' + +class BottomMatcher(object): + """The main matcher class. After instantiating the patterns should + be added using the add_fixer method""" + + def __init__(self): + self.match = set() + self.root = BMNode() + self.nodes = [self.root] + self.fixers = [] + self.logger = logging.getLogger("RefactoringTool") + + def add_fixer(self, fixer): + """Reduces a fixer's pattern tree to a linear path and adds it + to the matcher(a common Aho-Corasick automaton). The fixer is + appended on the matching states and called when they are + reached""" + self.fixers.append(fixer) + tree = reduce_tree(fixer.pattern_tree) + linear = tree.get_linear_subpattern() + match_nodes = self.add(linear, start=self.root) + for match_node in match_nodes: + match_node.fixers.append(fixer) + + def add(self, pattern, start): + "Recursively adds a linear pattern to the AC automaton" + #print("adding pattern", pattern, "to", start) + if not pattern: + #print("empty pattern") + return [start] + if isinstance(pattern[0], tuple): + #alternatives + #print("alternatives") + match_nodes = [] + for alternative in pattern[0]: + #add all alternatives, and add the rest of the pattern + #to each end node + end_nodes = self.add(alternative, start=start) + for end in end_nodes: + match_nodes.extend(self.add(pattern[1:], end)) + return match_nodes + else: + #single token + #not last + if pattern[0] not in start.transition_table: + #transition did not exist, create new + next_node = BMNode() + start.transition_table[pattern[0]] = next_node + else: + #transition exists already, follow + next_node = start.transition_table[pattern[0]] + + if pattern[1:]: + end_nodes = self.add(pattern[1:], start=next_node) + else: + end_nodes = [next_node] + return end_nodes + + def run(self, leaves): + """The main interface with the bottom matcher. The tree is + traversed from the bottom using the constructed + automaton. Nodes are only checked once as the tree is + retraversed. When the automaton fails, we give it one more + shot(in case the above tree matches as a whole with the + rejected leaf), then we break for the next leaf. There is the + special case of multiple arguments(see code comments) where we + recheck the nodes + + Args: + The leaves of the AST tree to be matched + + Returns: + A dictionary of node matches with fixers as the keys + """ + current_ac_node = self.root + results = defaultdict(list) + for leaf in leaves: + current_ast_node = leaf + while current_ast_node: + current_ast_node.was_checked = True + for child in current_ast_node.children: + # multiple statements, recheck + if isinstance(child, pytree.Leaf) and child.value == u";": + current_ast_node.was_checked = False + break + if current_ast_node.type == 1: + #name + node_token = current_ast_node.value + else: + node_token = current_ast_node.type + + if node_token in current_ac_node.transition_table: + #token matches + current_ac_node = current_ac_node.transition_table[node_token] + for fixer in current_ac_node.fixers: + if not fixer in results: + results[fixer] = [] + results[fixer].append(current_ast_node) + + else: + #matching failed, reset automaton + current_ac_node = self.root + if (current_ast_node.parent is not None + and current_ast_node.parent.was_checked): + #the rest of the tree upwards has been checked, next leaf + break + + #recheck the rejected node once from the root + if node_token in current_ac_node.transition_table: + #token matches + current_ac_node = current_ac_node.transition_table[node_token] + for fixer in current_ac_node.fixers: + if not fixer in results.keys(): + results[fixer] = [] + results[fixer].append(current_ast_node) + + current_ast_node = current_ast_node.parent + return results + + def print_ac(self): + "Prints a graphviz diagram of the BM automaton(for debugging)" + print("digraph g{") + def print_node(node): + for subnode_key in node.transition_table.keys(): + subnode = node.transition_table[subnode_key] + print("%d -> %d [label=%s] //%s" % + (node.id, subnode.id, type_repr(subnode_key), str(subnode.fixers))) + if subnode_key == 1: + print(subnode.content) + print_node(subnode) + print_node(self.root) + print("}") + +# taken from pytree.py for debugging; only used by print_ac +_type_reprs = {} +def type_repr(type_num): + global _type_reprs + if not _type_reprs: + from .pygram import python_symbols + # printing tokens is possible but not as useful + # from .pgen2 import token // token.__dict__.items(): + for name, val in python_symbols.__dict__.items(): + if type(val) == int: _type_reprs[val] = name + return _type_reprs.setdefault(type_num, type_num) Added: pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/btm_utils.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/lib-python/2.7.0/lib2to3/btm_utils.py Mon Nov 8 23:46:55 2010 @@ -0,0 +1,283 @@ +"Utility functions used by the btm_matcher module" + +from . import pytree +from .pgen2 import grammar, token +from .pygram import pattern_symbols, python_symbols + +syms = pattern_symbols +pysyms = python_symbols +tokens = grammar.opmap +token_labels = token + +TYPE_ANY = -1 +TYPE_ALTERNATIVES = -2 +TYPE_GROUP = -3 + +class MinNode(object): + """This class serves as an intermediate representation of the + pattern tree during the conversion to sets of leaf-to-root + subpatterns""" + + def __init__(self, type=None, name=None): + self.type = type + self.name = name + self.children = [] + self.leaf = False + self.parent = None + self.alternatives = [] + self.group = [] + + def __repr__(self): + return str(self.type) + ' ' + str(self.name) + + def leaf_to_root(self): + """Internal method. Returns a characteristic path of the + pattern tree. This method must be run for all leaves until the + linear subpatterns are merged into a single""" + node = self + subp = [] + while node: + if node.type == TYPE_ALTERNATIVES: + node.alternatives.append(subp) + if len(node.alternatives) == len(node.children): + #last alternative + subp = [tuple(node.alternatives)] + node.alternatives = [] + node = node.parent + continue + else: + node = node.parent + subp = None + break + + if node.type == TYPE_GROUP: + node.group.append(subp) + #probably should check the number of leaves + if len(node.group) == len(node.children): + subp = get_characteristic_subpattern(node.group) + node.group = [] + node = node.parent + continue + else: + node = node.parent + subp = None + break + + if node.type == token_labels.NAME and node.name: + #in case of type=name, use the name instead + subp.append(node.name) + else: + subp.append(node.type) + + node = node.parent + return subp + + def get_linear_subpattern(self): + """Drives the leaf_to_root method. The reason that + leaf_to_root must be run multiple times is because we need to + reject 'group' matches; for example the alternative form + (a | b c) creates a group [b c] that needs to be matched. Since + matching multiple linear patterns overcomes the automaton's + capabilities, leaf_to_root merges each group into a single + choice based on 'characteristic'ity, + + i.e. (a|b c) -> (a|b) if b more characteristic than c + + Returns: The most 'characteristic'(as defined by + get_characteristic_subpattern) path for the compiled pattern + tree. + """ + + for l in self.leaves(): + subp = l.leaf_to_root() + if subp: + return subp + + def leaves(self): + "Generator that returns the leaves of the tree" + for child in self.children: + for x in child.leaves(): + yield x + if not self.children: + yield self + +def reduce_tree(node, parent=None): + """ + Internal function. Reduces a compiled pattern tree to an + intermediate representation suitable for feeding the + automaton. This also trims off any optional pattern elements(like + [a], a*). + """ + + new_node = None + #switch on the node type + if node.type == syms.Matcher: + #skip + node = node.children[0] + + if node.type == syms.Alternatives : + #2 cases + if len(node.children) <= 2: + #just a single 'Alternative', skip this node + new_node = reduce_tree(node.children[0], parent) + else: + #real alternatives + new_node = MinNode(type=TYPE_ALTERNATIVES) + #skip odd children('|' tokens) + for child in node.children: + if node.children.index(child)%2: + continue + reduced = reduce_tree(child, new_node) + if reduced is not None: + new_node.children.append(reduced) + elif node.type == syms.Alternative: + if len(node.children) > 1: + + new_node = MinNode(type=TYPE_GROUP) + for child in node.children: + reduced = reduce_tree(child, new_node) + if reduced: + new_node.children.append(reduced) + if not new_node.children: + # delete the group if all of the children were reduced to None + new_node = None + + else: + new_node = reduce_tree(node.children[0], parent) + + elif node.type == syms.Unit: + if (isinstance(node.children[0], pytree.Leaf) and + node.children[0].value == '('): + #skip parentheses + return reduce_tree(node.children[1], parent) + if ((isinstance(node.children[0], pytree.Leaf) and + node.children[0].value == '[') + or + (len(node.children)>1 and + hasattr(node.children[1], "value") and + node.children[1].value == '[')): + #skip whole unit if its optional + return None + + leaf = True + details_node = None + alternatives_node = None + has_repeater = False + repeater_node = None + has_variable_name = False + + for child in node.children: + if child.type == syms.Details: + leaf = False + details_node = child + elif child.type == syms.Repeater: + has_repeater = True + repeater_node = child + elif child.type == syms.Alternatives: + alternatives_node = child + if hasattr(child, 'value') and child.value == '=': # variable name + has_variable_name = True + + #skip variable name + if has_variable_name: + #skip variable name, '=' + name_leaf = node.children[2] + if hasattr(name_leaf, 'value') and name_leaf.value == '(': + # skip parenthesis + name_leaf = node.children[3] + else: + name_leaf = node.children[0] + + #set node type + if name_leaf.type == token_labels.NAME: + #(python) non-name or wildcard + if name_leaf.value == 'any': + new_node = MinNode(type=TYPE_ANY) + else: + if hasattr(token_labels, name_leaf.value): + new_node = MinNode(type=getattr(token_labels, name_leaf.value)) + else: + new_node = MinNode(type=getattr(pysyms, name_leaf.value)) + + elif name_leaf.type == token_labels.STRING: + #(python) name or character; remove the apostrophes from + #the string value + name = name_leaf.value.strip("'") + if name in tokens: + new_node = MinNode(type=tokens[name]) + else: + new_node = MinNode(type=token_labels.NAME, name=name) + elif name_leaf.type == syms.Alternatives: + new_node = reduce_tree(alternatives_node, parent) + + #handle repeaters + if has_repeater: + if repeater_node.children[0].value == '*': + #reduce to None + new_node = None + elif repeater_node.children[0].value == '+': + #reduce to a single occurence i.e. do nothing + pass + else: + #TODO: handle {min, max} repeaters + raise NotImplementedError + pass + + #add children + if details_node and new_node is not None: + for child in details_node.children[1:-1]: + #skip '<', '>' markers + reduced = reduce_tree(child, new_node) + if reduced is not None: + new_node.children.append(reduced) + if new_node: + new_node.parent = parent + return new_node + + +def get_characteristic_subpattern(subpatterns): + """Picks the most characteristic from a list of linear patterns + Current order used is: + names > common_names > common_chars + """ + if not isinstance(subpatterns, list): + return subpatterns + if len(subpatterns)==1: + return subpatterns[0] + + # first pick out the ones containing variable names + subpatterns_with_names = [] + subpatterns_with_common_names = [] + common_names = ['in', 'for', 'if' , 'not', 'None'] + subpatterns_with_common_chars = [] + common_chars = "[]().,:" + for subpattern in subpatterns: + if any(rec_test(subpattern, lambda x: type(x) is str)): + if any(rec_test(subpattern, + lambda x: isinstance(x, str) and x in common_chars)): + subpatterns_with_common_chars.append(subpattern) + elif any(rec_test(subpattern, + lambda x: isinstance(x, str) and x in common_names)): + subpatterns_with_common_names.append(subpattern) + + else: + subpatterns_with_names.append(subpattern) + + if subpatterns_with_names: + subpatterns = subpatterns_with_names + elif subpatterns_with_common_names: + subpatterns = subpatterns_with_common_names + elif subpatterns_with_common_chars: + subpatterns = subpatterns_with_common_chars + # of the remaining subpatterns pick out the longest one + return max(subpatterns, key=len) + +def rec_test(sequence, test_func): + """Tests test_func on all items of sequence and items of included + sub-iterables""" + for x in sequence: + if isinstance(x, (list, tuple)): + for y in rec_test(x, test_func): + yield y + else: + yield test_func(x) From afa at codespeak.net Tue Nov 9 00:03:09 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 00:03:09 +0100 (CET) Subject: [pypy-svn] r78902 - in pypy/branch/fast-forward: lib-python lib-python/modified-2.5.2/test lib_pypy/ctypes_config_cache lib_pypy/ctypes_config_cache/test lib_pypy/pypy_test pypy/annotation pypy/config pypy/config/test pypy/doc pypy/doc/config pypy/interpreter pypy/interpreter/test pypy/jit/backend pypy/jit/backend/llgraph pypy/jit/backend/llsupport/test pypy/jit/backend/x86 pypy/jit/backend/x86/test pypy/jit/backend/x86/tool pypy/jit/codewriter pypy/jit/codewriter/test pypy/jit/metainterp pypy/jit/metainterp/optimizeopt pypy/jit/metainterp/test pypy/jit/tl pypy/jit/tool pypy/jit/tool/test pypy/module/__builtin__ pypy/module/__builtin__/test pypy/module/__pypy__ pypy/module/__pypy__/test pypy/module/_bisect pypy/module/_bisect/test pypy/module/gc pypy/module/gc/test pypy/module/pypyjit pypy/module/pypyjit/test pypy/module/signal pypy/module/signal/test pypy/module/sys pypy/module/thread pypy/module/thread/test pypy/objspace pypy/objspace/std pypy/objspace/std/test pypy/rlib pypy/rlib/test pypy/rpython/lltypesystem pypy/rpython/memory/gc pypy/rpython/test pypy/tool pypy/translator/c pypy/translator/c/gcc pypy/translator/c/gcc/test pypy/translator/c/gcc/test/darwin64 pypy/translator/c/src Message-ID: <20101108230309.6675A5080C@codespeak.net> Author: afa Date: Tue Nov 9 00:03:02 2010 New Revision: 78902 Added: pypy/branch/fast-forward/lib-python/modified-2.5.2/test/test_dict.py - copied unchanged from r78900, pypy/trunk/lib-python/modified-2.5.2/test/test_dict.py pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/dumpcache.py - copied unchanged from r78900, pypy/trunk/lib_pypy/ctypes_config_cache/dumpcache.py pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules._bisect.txt - copied unchanged from r78900, pypy/trunk/pypy/doc/config/objspace.usemodules._bisect.txt pypy/branch/fast-forward/pypy/jit/codewriter/test/test_void_list.py - copied unchanged from r78900, pypy/trunk/pypy/jit/codewriter/test/test_void_list.py pypy/branch/fast-forward/pypy/module/_bisect/ (props changed) - copied from r78900, pypy/trunk/pypy/module/_bisect/ pypy/branch/fast-forward/pypy/module/_bisect/__init__.py - copied unchanged from r78900, pypy/trunk/pypy/module/_bisect/__init__.py pypy/branch/fast-forward/pypy/module/_bisect/app_bisect.py - copied unchanged from r78900, pypy/trunk/pypy/module/_bisect/app_bisect.py pypy/branch/fast-forward/pypy/module/_bisect/interp_bisect.py - copied unchanged from r78900, pypy/trunk/pypy/module/_bisect/interp_bisect.py pypy/branch/fast-forward/pypy/module/_bisect/test/ (props changed) - copied from r78900, pypy/trunk/pypy/module/_bisect/test/ pypy/branch/fast-forward/pypy/module/_bisect/test/__init__.py - copied unchanged from r78900, pypy/trunk/pypy/module/_bisect/test/__init__.py pypy/branch/fast-forward/pypy/module/_bisect/test/test_bisect.py - copied unchanged from r78900, pypy/trunk/pypy/module/_bisect/test/test_bisect.py pypy/branch/fast-forward/pypy/objspace/std/test/test_methodcache.py - copied unchanged from r78900, pypy/trunk/pypy/objspace/std/test/test_methodcache.py pypy/branch/fast-forward/pypy/translator/c/gcc/test/darwin64/ - copied from r78900, pypy/trunk/pypy/translator/c/gcc/test/darwin64/ pypy/branch/fast-forward/pypy/translator/c/gcc/test/darwin64/track0.s - copied unchanged from r78900, pypy/trunk/pypy/translator/c/gcc/test/darwin64/track0.s pypy/branch/fast-forward/pypy/translator/c/gcc/test/darwin64/track1.s - copied unchanged from r78900, pypy/trunk/pypy/translator/c/gcc/test/darwin64/track1.s Removed: pypy/branch/fast-forward/pypy/doc/config/objspace.std.withinlineddict.txt pypy/branch/fast-forward/pypy/doc/config/objspace.std.withshadowtracking.txt pypy/branch/fast-forward/pypy/doc/config/objspace.std.withsharingdict.txt pypy/branch/fast-forward/pypy/objspace/std/inlinedict.py pypy/branch/fast-forward/pypy/objspace/std/sharingdict.py pypy/branch/fast-forward/pypy/objspace/std/test/test_inlinedict.py pypy/branch/fast-forward/pypy/objspace/std/test/test_shadowtracking.py pypy/branch/fast-forward/pypy/objspace/std/test/test_sharingdict.py Modified: pypy/branch/fast-forward/lib-python/conftest.py pypy/branch/fast-forward/lib-python/modified-2.5.2/test/mapping_tests.py pypy/branch/fast-forward/lib-python/modified-2.5.2/test/test_descr.py pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/ (props changed) pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/locale.ctc.py pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/pyexpat.ctc.py pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/rebuild.py pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/resource.ctc.py pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/syslog.ctc.py pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/test/test_cache.py pypy/branch/fast-forward/lib_pypy/pypy_test/test_ctypes_support.py pypy/branch/fast-forward/pypy/annotation/bookkeeper.py pypy/branch/fast-forward/pypy/annotation/classdef.py pypy/branch/fast-forward/pypy/annotation/description.py pypy/branch/fast-forward/pypy/annotation/dictdef.py pypy/branch/fast-forward/pypy/annotation/listdef.py pypy/branch/fast-forward/pypy/annotation/specialize.py pypy/branch/fast-forward/pypy/annotation/unaryop.py pypy/branch/fast-forward/pypy/config/pypyoption.py pypy/branch/fast-forward/pypy/config/test/test_pypyoption.py pypy/branch/fast-forward/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt pypy/branch/fast-forward/pypy/doc/getting-started.txt pypy/branch/fast-forward/pypy/doc/interpreter-optimizations.txt pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py pypy/branch/fast-forward/pypy/interpreter/executioncontext.py pypy/branch/fast-forward/pypy/interpreter/gateway.py pypy/branch/fast-forward/pypy/interpreter/pyopcode.py pypy/branch/fast-forward/pypy/interpreter/test/test_executioncontext.py pypy/branch/fast-forward/pypy/interpreter/typedef.py pypy/branch/fast-forward/pypy/jit/backend/detect_cpu.py pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_gc.py pypy/branch/fast-forward/pypy/jit/backend/model.py pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py pypy/branch/fast-forward/pypy/jit/backend/x86/tool/viewcode.py pypy/branch/fast-forward/pypy/jit/codewriter/call.py pypy/branch/fast-forward/pypy/jit/codewriter/format.py pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py pypy/branch/fast-forward/pypy/jit/codewriter/test/test_list.py pypy/branch/fast-forward/pypy/jit/metainterp/blackhole.py pypy/branch/fast-forward/pypy/jit/metainterp/compile.py pypy/branch/fast-forward/pypy/jit/metainterp/logger.py pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/rewrite.py pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_oparser.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py pypy/branch/fast-forward/pypy/jit/tl/pypyjit.py pypy/branch/fast-forward/pypy/jit/tool/oparser.py pypy/branch/fast-forward/pypy/jit/tool/pypytrace.vim pypy/branch/fast-forward/pypy/jit/tool/showstats.py pypy/branch/fast-forward/pypy/jit/tool/test/test_traceviewer.py pypy/branch/fast-forward/pypy/module/__builtin__/descriptor.py pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py pypy/branch/fast-forward/pypy/module/__pypy__/__init__.py pypy/branch/fast-forward/pypy/module/__pypy__/test/test_special.py pypy/branch/fast-forward/pypy/module/gc/interp_gc.py pypy/branch/fast-forward/pypy/module/gc/test/test_gc.py pypy/branch/fast-forward/pypy/module/pypyjit/interp_jit.py pypy/branch/fast-forward/pypy/module/pypyjit/policy.py pypy/branch/fast-forward/pypy/module/pypyjit/test/test_policy.py pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py pypy/branch/fast-forward/pypy/module/signal/__init__.py pypy/branch/fast-forward/pypy/module/signal/interp_signal.py pypy/branch/fast-forward/pypy/module/signal/test/test_signal.py pypy/branch/fast-forward/pypy/module/sys/__init__.py pypy/branch/fast-forward/pypy/module/sys/version.py pypy/branch/fast-forward/pypy/module/sys/vm.py pypy/branch/fast-forward/pypy/module/thread/gil.py pypy/branch/fast-forward/pypy/module/thread/test/test_gil.py pypy/branch/fast-forward/pypy/objspace/descroperation.py pypy/branch/fast-forward/pypy/objspace/std/callmethod.py pypy/branch/fast-forward/pypy/objspace/std/celldict.py pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py pypy/branch/fast-forward/pypy/objspace/std/inttype.py pypy/branch/fast-forward/pypy/objspace/std/listobject.py pypy/branch/fast-forward/pypy/objspace/std/longtype.py pypy/branch/fast-forward/pypy/objspace/std/mapdict.py pypy/branch/fast-forward/pypy/objspace/std/objspace.py pypy/branch/fast-forward/pypy/objspace/std/proxyobject.py pypy/branch/fast-forward/pypy/objspace/std/smallintobject.py pypy/branch/fast-forward/pypy/objspace/std/stringobject.py pypy/branch/fast-forward/pypy/objspace/std/strutil.py pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py pypy/branch/fast-forward/pypy/objspace/std/typeobject.py pypy/branch/fast-forward/pypy/rlib/jit.py pypy/branch/fast-forward/pypy/rlib/objectmodel.py pypy/branch/fast-forward/pypy/rlib/rbigint.py pypy/branch/fast-forward/pypy/rlib/test/test_rbigint.py pypy/branch/fast-forward/pypy/rpython/lltypesystem/rdict.py pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py pypy/branch/fast-forward/pypy/rpython/test/test_rdict.py pypy/branch/fast-forward/pypy/tool/alarm.py pypy/branch/fast-forward/pypy/tool/readdictinfo.py pypy/branch/fast-forward/pypy/tool/rundictbenchmarks.py pypy/branch/fast-forward/pypy/tool/statistic_irc_log.py pypy/branch/fast-forward/pypy/tool/watchdog.py pypy/branch/fast-forward/pypy/tool/watchdog_nt.py pypy/branch/fast-forward/pypy/translator/c/gcc/test/test_trackgcroot.py pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py pypy/branch/fast-forward/pypy/translator/c/genc.py pypy/branch/fast-forward/pypy/translator/c/src/g_include.h pypy/branch/fast-forward/pypy/translator/c/src/int.h pypy/branch/fast-forward/pypy/translator/c/src/signals.h Log: merge from trunk: svn merge -r78316:78900 ../trunk Modified: pypy/branch/fast-forward/lib-python/conftest.py ============================================================================== --- pypy/branch/fast-forward/lib-python/conftest.py (original) +++ pypy/branch/fast-forward/lib-python/conftest.py Tue Nov 9 00:03:02 2010 @@ -131,7 +131,7 @@ RegrTest('test_binhex.py'), RegrTest('test_binop.py', core=True), - RegrTest('test_bisect.py', core=True), + RegrTest('test_bisect.py', core=True, usemodules='_bisect'), RegrTest('test_bool.py', core=True), RegrTest('test_bsddb.py', skip="unsupported extension module"), RegrTest('test_bsddb185.py', skip="unsupported extension module"), Modified: pypy/branch/fast-forward/lib-python/modified-2.5.2/test/mapping_tests.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.5.2/test/mapping_tests.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.5.2/test/mapping_tests.py Tue Nov 9 00:03:02 2010 @@ -1,6 +1,7 @@ # tests common to dict and UserDict import unittest import UserDict +from test import test_support class BasicTestMappingProtocol(unittest.TestCase): @@ -525,7 +526,8 @@ self.assertEqual(va, int(ka)) kb, vb = tb = b.popitem() self.assertEqual(vb, int(kb)) - self.assert_(not(copymode < 0 and ta != tb)) + if test_support.check_impl_detail(): + self.assert_(not(copymode < 0 and ta != tb)) self.assert_(not a) self.assert_(not b) Modified: pypy/branch/fast-forward/lib-python/modified-2.5.2/test/test_descr.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.5.2/test/test_descr.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.5.2/test/test_descr.py Tue Nov 9 00:03:02 2010 @@ -2028,7 +2028,9 @@ except TypeError, msg: verify(str(msg).find("weak reference") >= 0) else: - verify(0, "weakref.ref(no) should be illegal") + # in PyPy it is (sometimes) possible to take a weakref here + #verify(0, "weakref.ref(no) should be illegal") + pass class Weak(object): __slots__ = ['foo', '__weakref__'] yes = Weak() Modified: pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/locale.ctc.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/locale.ctc.py (original) +++ pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/locale.ctc.py Tue Nov 9 00:03:02 2010 @@ -5,7 +5,7 @@ from ctypes_configure.configure import (configure, ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger, SimpleType, check_eci) -from ctypes_configure.dumpcache import dumpcache +import dumpcache # ____________________________________________________________ @@ -70,4 +70,4 @@ config['ALL_CONSTANTS'] = tuple(_CONSTANTS) config['HAS_LANGINFO'] = HAS_LANGINFO -dumpcache(__file__, '_locale_cache.py', config) +dumpcache.dumpcache2('locale', config) Modified: pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/pyexpat.ctc.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/pyexpat.ctc.py (original) +++ pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/pyexpat.ctc.py Tue Nov 9 00:03:02 2010 @@ -5,7 +5,8 @@ import ctypes from ctypes import c_char_p, c_int, c_void_p, c_char -from ctypes_configure import configure, dumpcache +from ctypes_configure import configure +import dumpcache class CConfigure: @@ -41,4 +42,4 @@ config = configure.configure(CConfigure) -dumpcache.dumpcache(__file__, '_pyexpat_cache.py', config) +dumpcache.dumpcache2('pyexpat', config) Modified: pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/rebuild.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/rebuild.py (original) +++ pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/rebuild.py Tue Nov 9 00:03:02 2010 @@ -31,10 +31,25 @@ sys.path[:] = path def try_rebuild(): + from pypy.jit.backend import detect_cpu + model = detect_cpu.autodetect_main_model_and_size() + # remove the files '_*_model_.py' + left = {} + for p in os.listdir(_dirpath): + if p.startswith('_') and (p.endswith('_%s_.py' % model) or + p.endswith('_%s_.pyc' % model)): + os.unlink(os.path.join(_dirpath, p)) + elif p.startswith('_') and (p.endswith('_.py') or + p.endswith('_.pyc')): + for i in range(2, len(p)-4): + left[p[:i]] = True + # remove the files '_*_cache.py' if there is no '_*_*_.py' left around for p in os.listdir(_dirpath): if p.startswith('_') and (p.endswith('_cache.py') or p.endswith('_cache.pyc')): - os.unlink(os.path.join(_dirpath, p)) + if p[:-9] not in left: + os.unlink(os.path.join(_dirpath, p)) + # for p in os.listdir(_dirpath): if p.endswith('.ctc.py'): try: Modified: pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/resource.ctc.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/resource.ctc.py (original) +++ pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/resource.ctc.py Tue Nov 9 00:03:02 2010 @@ -5,7 +5,7 @@ from ctypes import sizeof -from ctypes_configure.dumpcache import dumpcache +import dumpcache from ctypes_configure.configure import (configure, ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger, SimpleType) @@ -59,4 +59,4 @@ del config[key] config['ALL_CONSTANTS'] = _CONSTANTS + tuple(optional_constants) -dumpcache(__file__, '_resource_cache.py', config) +dumpcache.dumpcache2('resource', config) Modified: pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/syslog.ctc.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/syslog.ctc.py (original) +++ pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/syslog.ctc.py Tue Nov 9 00:03:02 2010 @@ -5,7 +5,7 @@ from ctypes_configure.configure import (configure, ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger) -from ctypes_configure.dumpcache import dumpcache +import dumpcache _CONSTANTS = ( @@ -72,4 +72,4 @@ all_constants = config.keys() all_constants.sort() config['ALL_CONSTANTS'] = tuple(all_constants) -dumpcache(__file__, '_syslog_cache.py', config) +dumpcache.dumpcache2('syslog', config) Modified: pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/test/test_cache.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/test/test_cache.py (original) +++ pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/test/test_cache.py Tue Nov 9 00:03:02 2010 @@ -7,19 +7,27 @@ def run(filename, outputname): filepath = dirpath.join(filename) - tmpdir = udir.ensure('testcache-' + filename, dir=True) - outputpath = tmpdir.join(outputname) - d = {'__file__': str(outputpath)} + tmpdir2 = udir.ensure('testcache-' + filename, dir=True) + tmpdir = tmpdir2.ensure('ctypes_config_cache', dir=True) + tmpdir.join('__init__.py').write('\n') + tmpdir.join('dumpcache.py').write(dirpath.join('dumpcache.py').read()) path = sys.path[:] try: - sys.path.insert(0, str(dirpath)) - execfile(str(filepath), d) + sys.path.insert(0, str(tmpdir)) + execfile(str(filepath), {}) finally: sys.path[:] = path + sys.modules.pop('dumpcache', None) # + outputpath = tmpdir.join(outputname) assert outputpath.check(exists=1) d = {} - execfile(str(outputpath), d) + try: + sys.path.insert(0, str(tmpdir2)) + execfile(str(outputpath), d) + finally: + sys.path[:] = path + sys.modules.pop('ctypes_config_cache', None) return d Modified: pypy/branch/fast-forward/lib_pypy/pypy_test/test_ctypes_support.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/pypy_test/test_ctypes_support.py (original) +++ pypy/branch/fast-forward/lib_pypy/pypy_test/test_ctypes_support.py Tue Nov 9 00:03:02 2010 @@ -22,12 +22,11 @@ assert get_errno() == 0 def test_argument_conversion_and_checks(): - import ctypes - libc = ctypes.cdll.LoadLibrary("libc.so.6") - libc.strlen.argtypes = ctypes.c_char_p, - libc.strlen.restype = ctypes.c_size_t - assert libc.strlen("eggs") == 4 - + strlen = standard_c_lib.strlen + strlen.argtypes = [c_char_p] + strlen.restype = c_size_t + assert strlen("eggs") == 4 + # Should raise ArgumentError, not segfault - py.test.raises(ctypes.ArgumentError, libc.strlen, False) + py.test.raises(ArgumentError, strlen, False) Modified: pypy/branch/fast-forward/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/bookkeeper.py (original) +++ pypy/branch/fast-forward/pypy/annotation/bookkeeper.py Tue Nov 9 00:03:02 2010 @@ -24,7 +24,7 @@ from pypy.rpython import extregistry from pypy.tool.identity_dict import identity_dict -class Stats: +class Stats(object): def __init__(self, bookkeeper): self.bookkeeper = bookkeeper @@ -137,7 +137,7 @@ def consider_dict_delitem(self, dic): return dic -class Bookkeeper: +class Bookkeeper(object): """The log of choices that have been made while analysing the operations. It ensures that the same 'choice objects' will be returned if we ask again during reflowing. Like ExecutionContext, there is an implicit @@ -736,7 +736,7 @@ return True # for parsing call arguments -class RPythonCallsSpace: +class RPythonCallsSpace(object): """Pseudo Object Space providing almost no real operation. For the Arguments class: if it really needs other operations, it means that the call pattern is too complex for R-Python. Modified: pypy/branch/fast-forward/pypy/annotation/classdef.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/classdef.py (original) +++ pypy/branch/fast-forward/pypy/annotation/classdef.py Tue Nov 9 00:03:02 2010 @@ -58,7 +58,7 @@ # same name in all subclasses of A, if any. (Parent class attributes can # be visible in reads from instances of subclasses.) -class Attribute: +class Attribute(object): # readonly-ness # SomeThing-ness # NB. an attribute is readonly if it is a constant class attribute. @@ -402,7 +402,7 @@ # ____________________________________________________________ -class InstanceSource: +class InstanceSource(object): instance_level = True def __init__(self, bookkeeper, obj): Modified: pypy/branch/fast-forward/pypy/annotation/description.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/description.py (original) +++ pypy/branch/fast-forward/pypy/annotation/description.py Tue Nov 9 00:03:02 2010 @@ -6,7 +6,7 @@ from pypy.tool.sourcetools import valid_identifier from pypy.tool.pairtype import extendabletype -class CallFamily: +class CallFamily(object): """A family of Desc objects that could be called from common call sites. The call families are conceptually a partition of all (callable) Desc objects, where the equivalence relation is the transitive closure of @@ -51,7 +51,7 @@ self.total_calltable_size += 1 -class FrozenAttrFamily: +class FrozenAttrFamily(object): """A family of FrozenDesc objects that have any common 'getattr' sites. The attr families are conceptually a partition of FrozenDesc objects, where the equivalence relation is the transitive closure of: @@ -80,7 +80,7 @@ self.attrs[attrname] = s_value -class ClassAttrFamily: +class ClassAttrFamily(object): """A family of ClassDesc objects that have common 'getattr' sites for a given attribute name. The attr families are conceptually a partition of ClassDesc objects, where the equivalence relation is the transitive Modified: pypy/branch/fast-forward/pypy/annotation/dictdef.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/dictdef.py (original) +++ pypy/branch/fast-forward/pypy/annotation/dictdef.py Tue Nov 9 00:03:02 2010 @@ -77,7 +77,7 @@ dictdef.dictvalue = self -class DictDef: +class DictDef(object): """A dict definition remembers how general the keys and values in that particular dict have to be. Every dict creation makes a new DictDef, and the union of two dicts merges the DictKeys and DictValues that each Modified: pypy/branch/fast-forward/pypy/annotation/listdef.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/listdef.py (original) +++ pypy/branch/fast-forward/pypy/annotation/listdef.py Tue Nov 9 00:03:02 2010 @@ -6,7 +6,7 @@ class TooLateForChange(Exception): pass -class ListItem: +class ListItem(object): mutated = False # True for lists mutated after creation resized = False # True for lists resized after creation range_step = None # the step -- only for lists only created by a range() @@ -117,7 +117,7 @@ return updated -class ListDef: +class ListDef(object): """A list definition remembers how general the items in that particular list have to be. Every list creation makes a new ListDef, and the union of two lists merges the ListItems that each ListDef stores.""" Modified: pypy/branch/fast-forward/pypy/annotation/specialize.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/specialize.py (original) +++ pypy/branch/fast-forward/pypy/annotation/specialize.py Tue Nov 9 00:03:02 2010 @@ -100,7 +100,7 @@ # ____________________________________________________________________________ # specializations -class MemoTable: +class MemoTable(object): def __init__(self, funcdesc, args, value): self.funcdesc = funcdesc self.table = {args: value} Modified: pypy/branch/fast-forward/pypy/annotation/unaryop.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/unaryop.py (original) +++ pypy/branch/fast-forward/pypy/annotation/unaryop.py Tue Nov 9 00:03:02 2010 @@ -434,6 +434,9 @@ def method_clear(dct): pass + def method_popitem(dct): + return dct.getanyitem('items') + def _can_only_throw(dic, *ignore): if dic1.dictdef.dictkey.custom_eq_hash: return None # r_dict: can throw anything Modified: pypy/branch/fast-forward/pypy/config/pypyoption.py ============================================================================== --- pypy/branch/fast-forward/pypy/config/pypyoption.py (original) +++ pypy/branch/fast-forward/pypy/config/pypyoption.py Tue Nov 9 00:03:02 2010 @@ -27,11 +27,11 @@ working_modules = default_modules.copy() working_modules.update(dict.fromkeys( ["_socket", "unicodedata", "mmap", "fcntl", - "rctime" , "select", "zipimport", "_lsprof", + "rctime" , "select", "zipimport", "_lsprof", "crypt", "signal", "_rawffi", "termios", "zlib", "bz2", "struct", "_hashlib", "_md5", "_sha", "_minimal_curses", "cStringIO", "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array", - "_multiprocessing", '_warnings'] + "_bisect", "_multiprocessing", '_warnings'] )) working_oo_modules = default_modules.copy() @@ -232,27 +232,15 @@ requires=[("objspace.opcodes.CALL_LIKELY_BUILTIN", False), ("objspace.honor__builtins__", False)]), - BoolOption("withsharingdict", - "use dictionaries that share the keys part", - default=False), - BoolOption("withdictmeasurement", "create huge files with masses of information " "about dictionaries", default=False), - BoolOption("withinlineddict", - "make instances more compact by revoming a level of indirection", - default=False, - requires=[("objspace.std.withshadowtracking", False)]), - BoolOption("withmapdict", "make instances really small but slow without the JIT", default=False, - requires=[("objspace.std.withshadowtracking", False), - ("objspace.std.withinlineddict", False), - ("objspace.std.withsharingdict", False), - ("objspace.std.getattributeshortcut", True), + requires=[("objspace.std.getattributeshortcut", True), ("objspace.std.withtypeversion", True), ]), @@ -269,12 +257,6 @@ # weakrefs needed, because of get_subclasses() requires=[("translation.rweakref", True)]), - BoolOption("withshadowtracking", - "track whether an instance attribute shadows a type" - " attribute", - default=False, - requires=[("objspace.std.withtypeversion", True), - ("translation.rweakref", True)]), BoolOption("withmethodcache", "try to cache method lookups", default=False, @@ -346,9 +328,6 @@ config.objspace.std.suggest(optimized_list_getitem=True) config.objspace.std.suggest(getattributeshortcut=True) config.objspace.std.suggest(newshortcut=True) - if type_system != 'ootype': - config.objspace.std.suggest(withsharingdict=True) - config.objspace.std.suggest(withinlineddict=True) # extra costly optimizations only go in level 3 if level == '3': @@ -377,7 +356,7 @@ # extra optimizations with the JIT if level == 'jit': config.objspace.std.suggest(withcelldict=True) - #config.objspace.std.suggest(withmapdict=True) + config.objspace.std.suggest(withmapdict=True) def enable_allworkingmodules(config): Modified: pypy/branch/fast-forward/pypy/config/test/test_pypyoption.py ============================================================================== --- pypy/branch/fast-forward/pypy/config/test/test_pypyoption.py (original) +++ pypy/branch/fast-forward/pypy/config/test/test_pypyoption.py Tue Nov 9 00:03:02 2010 @@ -47,7 +47,7 @@ def test_set_pypy_opt_level(): conf = get_pypy_config() set_pypy_opt_level(conf, '2') - assert conf.objspace.std.withsharingdict + assert conf.objspace.std.newshortcut conf = get_pypy_config() set_pypy_opt_level(conf, '0') assert not conf.objspace.std.newshortcut @@ -59,7 +59,6 @@ assert not conf.objspace.std.withtypeversion assert not conf.objspace.std.withmethodcache - assert not conf.objspace.std.withshadowtracking def test_check_documentation(): def check_file_exists(fn): Modified: pypy/branch/fast-forward/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt ============================================================================== --- pypy/branch/fast-forward/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt (original) +++ pypy/branch/fast-forward/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt Tue Nov 9 00:03:02 2010 @@ -5,8 +5,6 @@ case. So far, this only works for calls with no keyword, no ``*arg`` and no ``**arg`` but it would be easy to extend. -Gives the best results combined with :config:`objspace.std.withshadowtracking`. - For more information, see the section in `Standard Interpreter Optimizations`_. .. _`Standard Interpreter Optimizations`: ../interpreter-optimizations.html#lookup-method-call-method Modified: pypy/branch/fast-forward/pypy/doc/getting-started.txt ============================================================================== --- pypy/branch/fast-forward/pypy/doc/getting-started.txt (original) +++ pypy/branch/fast-forward/pypy/doc/getting-started.txt Tue Nov 9 00:03:02 2010 @@ -18,6 +18,7 @@ translation process - as opposed to encoding low level details into the language implementation itself. `more...`_ + .. _Python: http://docs.python.org/ref .. _`more...`: architecture.html Modified: pypy/branch/fast-forward/pypy/doc/interpreter-optimizations.txt ============================================================================== --- pypy/branch/fast-forward/pypy/doc/interpreter-optimizations.txt (original) +++ pypy/branch/fast-forward/pypy/doc/interpreter-optimizations.txt Tue Nov 9 00:03:02 2010 @@ -153,8 +153,8 @@ dicts: the representation of the instance dict contains only a list of values. -You can enable this feature with the :config:`objspace.std.withsharingdict` -option. +A more advanced version of sharing dicts, called *map dicts,* is available +with the :config:`objspace.std.withmapdict` option. Builtin-Shadowing +++++++++++++++++ @@ -219,8 +219,7 @@ shadowing the class attribute. If we know that there is no shadowing (since instance dict tells us that) we can save this lookup on the instance dictionary. -You can enable this feature with the :config:`objspace.std.withshadowtracking` -option. +*This was deprecated and is no longer available.* Method Caching Modified: pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py Tue Nov 9 00:03:02 2010 @@ -36,13 +36,10 @@ return space.finditem_str(w_dict, attr) return None - def getdictvalue_attr_is_in_class(self, space, attr): - return self.getdictvalue(space, attr) - - def setdictvalue(self, space, attr, w_value, shadows_type=True): + def setdictvalue(self, space, attr, w_value): w_dict = self.getdict() if w_dict is not None: - space.setitem_str(w_dict, attr, w_value, shadows_type) + space.setitem_str(w_dict, attr, w_value) return True return False @@ -261,10 +258,9 @@ self.interned_strings = {} self.actionflag = ActionFlag() # changed by the signal module + self.check_signal_action = None # changed by the signal module self.user_del_action = UserDelAction(self) self.frame_trace_action = FrameTraceAction(self) - self.actionflag.register_action(self.user_del_action) - self.actionflag.register_action(self.frame_trace_action) from pypy.interpreter.pycode import cpython_magic, default_magic self.our_magic = default_magic @@ -657,7 +653,7 @@ """shortcut for space.int_w(space.hash(w_obj))""" return self.int_w(self.hash(w_obj)) - def setitem_str(self, w_obj, key, w_value, shadows_type=True): + def setitem_str(self, w_obj, key, w_value): return self.setitem(w_obj, self.wrap(key), w_value) def finditem_str(self, w_obj, key): Modified: pypy/branch/fast-forward/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/executioncontext.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/executioncontext.py Tue Nov 9 00:03:02 2010 @@ -5,6 +5,8 @@ from pypy.rlib.unroll import unrolling_iterable from pypy.rlib import jit +TICK_COUNTER_STEP = 100 + def app_profile_call(space, w_callable, frame, event, w_arg): space.call_function(w_callable, space.wrap(frame), @@ -19,6 +21,9 @@ # XXX self.w_tracefunc, self.profilefunc # XXX frame.is_being_profiled + # XXX [fijal] but they're not. is_being_profiled is guarded a bit all + # over the place as well as w_tracefunc + def __init__(self, space): self.space = space self.topframeref = jit.vref_None @@ -163,24 +168,19 @@ if self.w_tracefunc is not None: self._trace(frame, 'return', w_retval) - def bytecode_trace(self, frame): + def bytecode_trace(self, frame, decr_by=TICK_COUNTER_STEP): "Trace function called before each bytecode." # this is split into a fast path and a slower path that is # not invoked every time bytecode_trace() is. actionflag = self.space.actionflag - ticker = actionflag.get() - if actionflag.has_bytecode_counter: # this "if" is constant-folded - ticker += 1 - actionflag.set(ticker) - if ticker & actionflag.interesting_bits: # fast check + if actionflag.decrement_ticker(decr_by) < 0: actionflag.action_dispatcher(self, frame) # slow path bytecode_trace._always_inline_ = True def bytecode_trace_after_exception(self, frame): "Like bytecode_trace(), but without increasing the ticker." actionflag = self.space.actionflag - ticker = actionflag.get() - if ticker & actionflag.interesting_bits: # fast check + if actionflag.get_ticker() < 0: actionflag.action_dispatcher(self, frame) # slow path bytecode_trace_after_exception._always_inline_ = True @@ -314,125 +314,109 @@ frame.last_exception = last_exception self.is_tracing -= 1 + def checksignals(self): + """Similar to PyErr_CheckSignals(). If called in the main thread, + and if signals are pending for the process, deliver them now + (i.e. call the signal handlers).""" + if self.space.check_signal_action is not None: + self.space.check_signal_action.perform(self, None) + def _freeze_(self): raise Exception("ExecutionContext instances should not be seen during" " translation. Now is a good time to inspect the" " traceback and see where this one comes from :-)") -class AbstractActionFlag: - """This holds the global 'action flag'. It is a single bitfield - integer, with bits corresponding to AsyncAction objects that need to - be immediately triggered. The correspondance from bits to - AsyncAction instances is built at translation time. We can quickly - check if there is anything at all to do by checking if any of the - relevant bits is set. If threads are enabled, they consume the 20 - lower bits to hold a counter incremented at each bytecode, to know - when to release the GIL. +class AbstractActionFlag(object): + """This holds in an integer the 'ticker'. If threads are enabled, + it is decremented at each bytecode; when it reaches zero, we release + the GIL. And whether we have threads or not, it is forced to zero + whenever we fire any of the asynchronous actions. """ def __init__(self): self._periodic_actions = [] self._nonperiodic_actions = [] - self.unused_bits = self.FREE_BITS[:] self.has_bytecode_counter = False - self.interesting_bits = 0 + self.fired_actions = None + self.checkinterval_scaled = 100 * TICK_COUNTER_STEP self._rebuild_action_dispatcher() def fire(self, action): - """Request for the action to be run before the next opcode. - The action must have been registered at space initalization time.""" - ticker = self.get() - self.set(ticker | action.bitmask) - - def register_action(self, action): - "NOT_RPYTHON" - assert isinstance(action, AsyncAction) - if action.bitmask == 0: - while True: - action.bitmask = self.unused_bits.pop(0) - if not (action.bitmask & self.interesting_bits): - break - self.interesting_bits |= action.bitmask - if action.bitmask & self.BYTECODE_COUNTER_OVERFLOW_BIT: - assert action.bitmask == self.BYTECODE_COUNTER_OVERFLOW_BIT - self._periodic_actions.append(action) + """Request for the action to be run before the next opcode.""" + if not action._fired: + action._fired = True + if self.fired_actions is None: + self.fired_actions = [] + self.fired_actions.append(action) + # set the ticker to -1 in order to force action_dispatcher() + # to run at the next possible bytecode + self.reset_ticker(-1) + + def register_periodic_action(self, action, use_bytecode_counter): + """NOT_RPYTHON: + Register the PeriodicAsyncAction action to be called whenever the + tick counter becomes smaller than 0. If 'use_bytecode_counter' is + True, make sure that we decrease the tick counter at every bytecode. + This is needed for threads. Note that 'use_bytecode_counter' can be + False for signal handling, because whenever the process receives a + signal, the tick counter is set to -1 by C code in signals.h. + """ + assert isinstance(action, PeriodicAsyncAction) + self._periodic_actions.append(action) + if use_bytecode_counter: self.has_bytecode_counter = True - self.force_tick_counter() - else: - self._nonperiodic_actions.append((action, action.bitmask)) self._rebuild_action_dispatcher() - def setcheckinterval(self, space, interval): - if interval < self.CHECK_INTERVAL_MIN: - interval = self.CHECK_INTERVAL_MIN - elif interval > self.CHECK_INTERVAL_MAX: - interval = self.CHECK_INTERVAL_MAX - space.sys.checkinterval = interval - self.force_tick_counter() - - def force_tick_counter(self): - # force the tick counter to a valid value -- this actually forces - # it to reach BYTECODE_COUNTER_OVERFLOW_BIT at the next opcode. - ticker = self.get() - ticker &= ~ self.BYTECODE_COUNTER_OVERFLOW_BIT - ticker |= self.BYTECODE_COUNTER_MASK - self.set(ticker) + def getcheckinterval(self): + return self.checkinterval_scaled // TICK_COUNTER_STEP + + def setcheckinterval(self, interval): + MAX = sys.maxint // TICK_COUNTER_STEP + if interval < 1: + interval = 1 + elif interval > MAX: + interval = MAX + self.checkinterval_scaled = interval * TICK_COUNTER_STEP def _rebuild_action_dispatcher(self): periodic_actions = unrolling_iterable(self._periodic_actions) - nonperiodic_actions = unrolling_iterable(self._nonperiodic_actions) - has_bytecode_counter = self.has_bytecode_counter @jit.dont_look_inside def action_dispatcher(ec, frame): - # periodic actions - if has_bytecode_counter: - ticker = self.get() - if ticker & self.BYTECODE_COUNTER_OVERFLOW_BIT: - # We must run the periodic actions now, but first - # reset the bytecode counter (the following line - # works by assuming that we just overflowed the - # counter, i.e. BYTECODE_COUNTER_OVERFLOW_BIT is - # set but none of the BYTECODE_COUNTER_MASK bits - # are). - ticker -= ec.space.sys.checkinterval - self.set(ticker) - for action in periodic_actions: - action.perform(ec, frame) + # periodic actions (first reset the bytecode counter) + self.reset_ticker(self.checkinterval_scaled) + for action in periodic_actions: + action.perform(ec, frame) # nonperiodic actions - for action, bitmask in nonperiodic_actions: - ticker = self.get() - if ticker & bitmask: - self.set(ticker & ~ bitmask) + list = self.fired_actions + if list is not None: + self.fired_actions = None + for action in list: + action._fired = False action.perform(ec, frame) action_dispatcher._dont_inline_ = True self.action_dispatcher = action_dispatcher - # Bits reserved for the bytecode counter, if used - BYTECODE_COUNTER_MASK = (1 << 20) - 1 - BYTECODE_COUNTER_OVERFLOW_BIT = (1 << 20) - - # Free bits - FREE_BITS = [1 << _b for _b in range(21, LONG_BIT-1)] - - # The acceptable range of values for sys.checkinterval, so that - # the bytecode_counter fits in 20 bits - CHECK_INTERVAL_MIN = 1 - CHECK_INTERVAL_MAX = BYTECODE_COUNTER_OVERFLOW_BIT - class ActionFlag(AbstractActionFlag): """The normal class for space.actionflag. The signal module provides a different one.""" - _flags = 0 + _ticker = 0 + + def get_ticker(self): + return self._ticker - def get(self): - return self._flags + def reset_ticker(self, value): + self._ticker = value - def set(self, value): - self._flags = value + def decrement_ticker(self, by): + value = self._ticker + if self.has_bytecode_counter: # this 'if' is constant-folded + value -= by + self._ticker = value + return value class AsyncAction(object): @@ -440,7 +424,7 @@ asynchronously with regular bytecode execution, but that still need to occur between two opcodes, not at a completely random time. """ - bitmask = 0 # means 'please choose one bit automatically' + _fired = False def __init__(self, space): self.space = space @@ -453,10 +437,11 @@ def fire_after_thread_switch(self): """Bit of a hack: fire() the action but only the next time the GIL is released and re-acquired (i.e. after a potential thread switch). - Don't call this if threads are not enabled. + Don't call this if threads are not enabled. Currently limited to + one action (i.e. reserved for CheckSignalAction from module/signal). """ from pypy.module.thread.gil import spacestate - spacestate.set_actionflag_bit_after_thread_switch |= self.bitmask + spacestate.action_after_thread_switch = self def perform(self, executioncontext, frame): """To be overridden.""" @@ -466,7 +451,6 @@ """Abstract base class for actions that occur automatically every sys.checkinterval bytecodes. """ - bitmask = ActionFlag.BYTECODE_COUNTER_OVERFLOW_BIT class UserDelAction(AsyncAction): Modified: pypy/branch/fast-forward/pypy/interpreter/gateway.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/gateway.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/gateway.py Tue Nov 9 00:03:02 2010 @@ -28,7 +28,7 @@ # internal non-translatable parts: import py -class SignatureBuilder: +class SignatureBuilder(object): "NOT_RPYTHON" def __init__(self, func=None, argnames=None, varargname=None, kwargname=None, name = None): @@ -51,7 +51,7 @@ #________________________________________________________________ -class UnwrapSpecRecipe: +class UnwrapSpecRecipe(object): "NOT_RPYTHON" bases_order = [Wrappable, W_Root, ObjSpace, Arguments, object] Modified: pypy/branch/fast-forward/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/pyopcode.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/pyopcode.py Tue Nov 9 00:03:02 2010 @@ -1244,7 +1244,7 @@ state_pack_variables = staticmethod(state_pack_variables) -class FrameBlock: +class FrameBlock(object): """Abstract base class for frame blocks from the blockstack, used by the SETUP_XXX and POP_BLOCK opcodes.""" Modified: pypy/branch/fast-forward/pypy/interpreter/test/test_executioncontext.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/test/test_executioncontext.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/test/test_executioncontext.py Tue Nov 9 00:03:02 2010 @@ -19,7 +19,6 @@ space = self.space a1 = DemoAction(space) - space.actionflag.register_action(a1) for i in range(20): # assert does not raise: space.appexec([], """(): @@ -50,7 +49,7 @@ space = self.space a2 = DemoAction(space) - space.actionflag.register_action(a2) + space.actionflag.register_periodic_action(a2, True) try: for i in range(500): space.appexec([], """(): @@ -59,7 +58,8 @@ """) except Finished: pass - assert space.sys.checkinterval / 10 < i < space.sys.checkinterval * 3 + checkinterval = space.actionflag.getcheckinterval() + assert checkinterval / 10 < i < checkinterval * 1.1 def test_llprofile(self): l = [] Modified: pypy/branch/fast-forward/pypy/interpreter/typedef.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/typedef.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/typedef.py Tue Nov 9 00:03:02 2010 @@ -197,6 +197,7 @@ from pypy.objspace.std.mapdict import BaseMapdictObject, ObjectMixin add(BaseMapdictObject) add(ObjectMixin) + body["user_overridden_class"] = True features = () if "user" in features: # generic feature needed by all subcls @@ -256,16 +257,7 @@ return self.slots_w[index] add(Proto) - wantdict = "dict" in features - if wantdict and config.objspace.std.withinlineddict: - from pypy.objspace.std.objectobject import W_ObjectObject - from pypy.objspace.std.inlinedict import make_mixin - if supercls is W_ObjectObject: - Mixin = make_mixin(config) - add(Mixin) - wantdict = False - - if wantdict: + if "dict" in features: base_user_setup = supercls.user_setup.im_func if "user_setup" in body: base_user_setup = body["user_setup"] @@ -284,15 +276,6 @@ def setclass(self, space, w_subtype): # only used by descr_set___class__ self.w__class__ = w_subtype - if space.config.objspace.std.withshadowtracking: - self.w__dict__.set_shadows_anything() - - def getdictvalue_attr_is_in_class(self, space, name): - w_dict = self.w__dict__ - if space.config.objspace.std.withshadowtracking: - if not w_dict.shadows_anything(): - return None - return space.finditem_str(w_dict, name) add(Proto) Modified: pypy/branch/fast-forward/pypy/jit/backend/detect_cpu.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/detect_cpu.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/detect_cpu.py Tue Nov 9 00:03:02 2010 @@ -23,11 +23,6 @@ mach = os.popen('uname -m', 'r').read().strip() if not mach: raise ProcessorAutodetectError, "cannot run 'uname -m'" - if mach == 'x86_64': - if sys.maxint == 2147483647: - mach = 'x86' # it's a 64-bit processor but in 32-bits mode, maybe - else: - assert sys.maxint == 2 ** 63 - 1 try: return {'i386': 'x86', 'i486': 'x86', @@ -36,17 +31,31 @@ 'i86pc': 'x86', # Solaris/Intel 'x86': 'x86', # Apple 'Power Macintosh': 'ppc', - 'x86_64': 'x86_64', + 'x86_64': 'x86', }[mach] except KeyError: - raise ProcessorAutodetectError, "unsupported processor '%s'" % mach + return mach + +def autodetect_main_model_and_size(): + model = autodetect_main_model() + if sys.maxint == 2**31-1: + model += '_32' + elif sys.maxint == 2**63-1: + model += '_64' + else: + raise AssertionError, "bad value for sys.maxint" + return model def autodetect(): model = autodetect_main_model() - if model == 'x86': - from pypy.jit.backend.x86.detect_sse2 import detect_sse2 - if not detect_sse2(): - model = 'x86-without-sse2' + if sys.maxint == 2**63-1: + model += '_64' + else: + assert sys.maxint == 2**31-1 + if model == 'x86': + from pypy.jit.backend.x86.detect_sse2 import detect_sse2 + if not detect_sse2(): + model = 'x86-without-sse2' return model def getcpuclassname(backend_name="auto"): Modified: pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py Tue Nov 9 00:03:02 2010 @@ -152,7 +152,7 @@ 'unicodegetitem' : (('ref', 'int'), 'int'), 'unicodesetitem' : (('ref', 'int', 'int'), 'int'), 'cast_ptr_to_int' : (('ref',), 'int'), - 'debug_merge_point': (('ref',), None), + 'debug_merge_point': (('ref', 'int'), None), 'force_token' : ((), 'int'), 'call_may_force' : (('int', 'varargs'), 'intorptr'), 'guard_not_forced': ((), None), @@ -568,7 +568,7 @@ # return _op_default_implementation - def op_debug_merge_point(self, _, value): + def op_debug_merge_point(self, _, value, recdepth): from pypy.jit.metainterp.warmspot import get_stats loc = ConstPtr(value)._get_str() get_stats().add_merge_point_location(loc) Modified: pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py Tue Nov 9 00:03:02 2010 @@ -118,13 +118,13 @@ self._descrs[key] = descr return descr - def compile_bridge(self, faildescr, inputargs, operations): + def compile_bridge(self, faildescr, inputargs, operations, log=True): c = llimpl.compile_start() self._compile_loop_or_bridge(c, inputargs, operations) old, oldindex = faildescr._compiled_fail llimpl.compile_redirect_fail(old, oldindex, c) - def compile_loop(self, inputargs, operations, loopdescr): + def compile_loop(self, inputargs, operations, loopdescr, log=True): """In a real assembler backend, this should assemble the given list of operations. Here we just generate a similar CompiledLoop instance. The code here is RPython, whereas the code in llimpl Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_gc.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_gc.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_gc.py Tue Nov 9 00:03:02 2010 @@ -270,7 +270,7 @@ def test_get_rid_of_debug_merge_point(self): operations = [ - ResOperation(rop.DEBUG_MERGE_POINT, ['dummy'], None), + ResOperation(rop.DEBUG_MERGE_POINT, ['dummy', 2], None), ] gc_ll_descr = self.gc_ll_descr gc_ll_descr.rewrite_assembler(None, operations) Modified: pypy/branch/fast-forward/pypy/jit/backend/model.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/model.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/model.py Tue Nov 9 00:03:02 2010 @@ -33,14 +33,14 @@ pass - def compile_loop(self, inputargs, operations, looptoken): + def compile_loop(self, inputargs, operations, looptoken, log=True): """Assemble the given loop. Extra attributes should be put in the LoopToken to point to the compiled loop in assembler. """ raise NotImplementedError - def compile_bridge(self, faildescr, inputargs, operations): + def compile_bridge(self, faildescr, inputargs, operations, log=True): """Assemble the bridge. The FailDescr is the descr of the original guard that failed. """ Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py Tue Nov 9 00:03:02 2010 @@ -295,7 +295,7 @@ self.mc.RET() self.mc.done() - def assemble_loop(self, inputargs, operations, looptoken): + def assemble_loop(self, inputargs, operations, looptoken, log): """adds the following attributes to looptoken: _x86_loop_code (an integer giving an address) _x86_bootstrap_code (an integer giving an address) @@ -303,6 +303,7 @@ _x86_frame_depth _x86_param_depth _x86_arglocs + _x86_debug_checksum """ if not we_are_translated(): # Arguments should be unique @@ -310,10 +311,11 @@ self.setup() funcname = self._find_debug_merge_point(operations) - + if log: + self._register_counter() + operations = self._inject_debugging_code(looptoken, operations) regalloc = RegAlloc(self, self.cpu.translate_support_code) - operations = self._inject_debugging_code(operations) arglocs = regalloc.prepare_loop(inputargs, operations, looptoken) looptoken._x86_arglocs = arglocs @@ -335,18 +337,22 @@ self._assemble_bootstrap_direct_call(arglocs, curadr, frame_depth+param_depth) # - debug_print("Loop #", looptoken.number, "has address", - looptoken._x86_loop_code, "to", self.mc.tell()) + debug_print("Loop #%d has address %x to %x" % (looptoken.number, + looptoken._x86_loop_code, + self.mc.tell())) self.mc.end_function() self.write_pending_failure_recoveries() - def assemble_bridge(self, faildescr, inputargs, operations): + def assemble_bridge(self, faildescr, inputargs, operations, log): if not we_are_translated(): # Arguments should be unique assert len(set(inputargs)) == len(inputargs) self.setup() funcname = self._find_debug_merge_point(operations) + if log: + self._register_counter() + operations = self._inject_debugging_code(faildescr, operations) arglocs = self.rebuild_faillocs_from_descr( faildescr._x86_failure_recovery_bytecode) @@ -354,7 +360,6 @@ assert ([loc.assembler() for loc in arglocs] == [loc.assembler() for loc in faildescr._x86_debug_faillocs]) regalloc = RegAlloc(self, self.cpu.translate_support_code) - operations = self._inject_debugging_code(operations) fail_depths = faildescr._x86_current_depths regalloc.prepare_bridge(fail_depths, inputargs, arglocs, operations) @@ -374,9 +379,8 @@ faildescr._x86_bridge_param_depth = param_depth # patch the jump from original guard self.patch_jump_for_descr(faildescr, adr_bridge) - debug_print("Bridge out of guard", - descr_number, - "has address", adr_bridge, "to", self.mc.tell()) + debug_print("Bridge out of guard %d has address %x to %x" % + (descr_number, adr_bridge, self.mc.tell())) self.mc.end_function() self.write_pending_failure_recoveries() @@ -401,12 +405,14 @@ else: funcname = "" % len(self.loop_run_counters) # invent the counter, so we don't get too confused + return funcname + + def _register_counter(self): if self._debug: struct = lltype.malloc(DEBUG_COUNTER, flavor='raw', track_allocation=False) # known to leak struct.i = 0 self.loop_run_counters.append((len(self.loop_run_counters), struct)) - return funcname def patch_jump_for_descr(self, faildescr, adr_new_target): adr_jump_offset = faildescr._x86_adr_jump_offset @@ -427,9 +433,14 @@ mc.done() - def _inject_debugging_code(self, operations): + @specialize.argtype(1) + def _inject_debugging_code(self, looptoken, operations): if self._debug: # before doing anything, let's increase a counter + s = 0 + for op in operations: + s += op.getopnum() + looptoken._x86_debug_checksum = s c_adr = ConstInt(rffi.cast(lltype.Signed, self.loop_run_counters[-1][1])) box = BoxInt() @@ -706,9 +717,11 @@ dispatch_opnum = guard_opnum else: dispatch_opnum = op.getopnum() - res = genop_guard_list[dispatch_opnum](self, op, guard_op, guard_token, - arglocs, resloc) - faildescr._x86_adr_jump_offset = res + genop_guard_list[dispatch_opnum](self, op, guard_op, guard_token, + arglocs, resloc) + if not we_are_translated(): + # must be added by the genop_guard_list[]() + assert hasattr(faildescr, '_x86_adr_jump_offset') def regalloc_perform_guard(self, guard_op, faillocs, arglocs, resloc, current_depths): @@ -765,15 +778,15 @@ if isinstance(op.getarg(0), Const): self.mc.CMP(arglocs[1], arglocs[0]) if guard_opnum == rop.GUARD_FALSE: - return self.implement_guard(guard_token, rev_cond) + self.implement_guard(guard_token, rev_cond) else: - return self.implement_guard(guard_token, false_rev_cond) + self.implement_guard(guard_token, false_rev_cond) else: self.mc.CMP(arglocs[0], arglocs[1]) if guard_opnum == rop.GUARD_FALSE: - return self.implement_guard(guard_token, cond) + self.implement_guard(guard_token, cond) else: - return self.implement_guard(guard_token, false_cond) + self.implement_guard(guard_token, false_cond) return genop_cmp_guard def _cmpop_guard_float(cond, false_cond, need_jp): @@ -787,13 +800,14 @@ if guard_opnum == rop.GUARD_FALSE: if need_jp: self.mc.J_il8(rx86.Conditions['P'], 6) - return self.implement_guard(guard_token, cond) + self.implement_guard(guard_token, cond) else: if need_jp: self.mc.J_il8(rx86.Conditions['P'], 2) self.mc.J_il8(rx86.Conditions[cond], 5) - return self.implement_guard(guard_token) - return self.implement_guard(guard_token, false_cond) + self.implement_guard(guard_token) + else: + self.implement_guard(guard_token, false_cond) return genop_cmp_guard_float def _emit_call(self, x, arglocs, start=0, tmp=eax): @@ -955,11 +969,11 @@ self.mc.ensure_bytes_available(16 + guard_token.recovery_stub_size()) if guard_opnum == rop.GUARD_TRUE: self.mc.J_il8(rx86.Conditions['P'], 6) - return self.implement_guard(guard_token, 'E') + self.implement_guard(guard_token, 'E') else: self.mc.J_il8(rx86.Conditions['P'], 2) self.mc.J_il8(rx86.Conditions['E'], 5) - return self.implement_guard(guard_token) + self.implement_guard(guard_token) def genop_float_neg(self, op, arglocs, resloc): # Following what gcc does: res = x ^ 0x8000000000000000 @@ -979,9 +993,9 @@ guard_opnum = guard_op.getopnum() self.mc.CMP(arglocs[0], imm0) if guard_opnum == rop.GUARD_TRUE: - return self.implement_guard(guard_token, 'Z') + self.implement_guard(guard_token, 'Z') else: - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') def genop_int_is_true(self, op, arglocs, resloc): self.mc.CMP(arglocs[0], imm0) @@ -993,9 +1007,9 @@ guard_opnum = guard_op.getopnum() self.mc.CMP(arglocs[0], imm0) if guard_opnum == rop.GUARD_TRUE: - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') else: - return self.implement_guard(guard_token, 'Z') + self.implement_guard(guard_token, 'Z') def genop_int_is_zero(self, op, arglocs, resloc): self.mc.CMP(arglocs[0], imm0) @@ -1192,13 +1206,13 @@ def genop_guard_guard_true(self, ign_1, guard_op, guard_token, locs, ign_2): loc = locs[0] self.mc.TEST(loc, loc) - return self.implement_guard(guard_token, 'Z') + self.implement_guard(guard_token, 'Z') genop_guard_guard_nonnull = genop_guard_guard_true def genop_guard_guard_no_exception(self, ign_1, guard_op, guard_token, locs, ign_2): self.mc.CMP(heap(self.cpu.pos_exception()), imm0) - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') def genop_guard_guard_exception(self, ign_1, guard_op, guard_token, locs, resloc): @@ -1206,19 +1220,18 @@ loc1 = locs[1] self.mc.MOV(loc1, heap(self.cpu.pos_exception())) self.mc.CMP(loc1, loc) - addr = self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') if resloc is not None: self.mc.MOV(resloc, heap(self.cpu.pos_exc_value())) self.mc.MOV(heap(self.cpu.pos_exception()), imm0) self.mc.MOV(heap(self.cpu.pos_exc_value()), imm0) - return addr def _gen_guard_overflow(self, guard_op, guard_token): guard_opnum = guard_op.getopnum() if guard_opnum == rop.GUARD_NO_OVERFLOW: - return self.implement_guard(guard_token, 'O') + self.implement_guard(guard_token, 'O') elif guard_opnum == rop.GUARD_OVERFLOW: - return self.implement_guard(guard_token, 'NO') + self.implement_guard(guard_token, 'NO') else: not_implemented("int_xxx_ovf followed by %s" % guard_op.getopname()) @@ -1238,7 +1251,7 @@ def genop_guard_guard_false(self, ign_1, guard_op, guard_token, locs, ign_2): loc = locs[0] self.mc.TEST(loc, loc) - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') genop_guard_guard_isnull = genop_guard_guard_false def genop_guard_guard_value(self, ign_1, guard_op, guard_token, locs, ign_2): @@ -1247,7 +1260,7 @@ self.mc.UCOMISD(locs[0], locs[1]) else: self.mc.CMP(locs[0], locs[1]) - return self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') def _cmp_guard_class(self, locs): offset = self.cpu.vtable_offset @@ -1279,7 +1292,7 @@ def genop_guard_guard_class(self, ign_1, guard_op, guard_token, locs, ign_2): self.mc.ensure_bytes_available(256) self._cmp_guard_class(locs) - return self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') def genop_guard_guard_nonnull_class(self, ign_1, guard_op, guard_token, locs, ign_2): @@ -1294,7 +1307,7 @@ assert 0 < offset <= 127 self.mc.overwrite(jb_location-1, [chr(offset)]) # - return self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') def implement_guard_recovery(self, guard_opnum, faildescr, failargs, fail_locs): @@ -1621,13 +1634,15 @@ def implement_guard(self, guard_token, condition=None): self.mc.reserve_bytes(guard_token.recovery_stub_size()) self.pending_guard_tokens.append(guard_token) - # XXX: These jumps are patched later, the self.mc.tell() are just - # dummy values + # These jumps are patched later, the mc.tell() are just + # dummy values. Also, use self.mc._mc to avoid triggering a + # "buffer full" exactly here. + mc = self.mc._mc if condition: - self.mc.J_il(rx86.Conditions[condition], self.mc.tell()) + mc.J_il(rx86.Conditions[condition], mc.tell()) else: - self.mc.JMP_l(self.mc.tell()) - return self.mc.tell() - 4 + mc.JMP_l(mc.tell()) + guard_token.faildescr._x86_adr_jump_offset = mc.tell() - 4 def genop_call(self, op, arglocs, resloc): sizeloc = arglocs[0] @@ -1668,7 +1683,7 @@ self.mc.MOV_bi(FORCE_INDEX_OFS, fail_index) self.genop_call(op, arglocs, result_loc) self.mc.CMP_bi(FORCE_INDEX_OFS, 0) - return self.implement_guard(guard_token, 'L') + self.implement_guard(guard_token, 'L') def genop_guard_call_assembler(self, op, guard_op, guard_token, arglocs, result_loc): @@ -1755,7 +1770,7 @@ assert 0 <= offset <= 127 self.mc.overwrite(jmp_location - 1, [chr(offset)]) self.mc.CMP_bi(FORCE_INDEX_OFS, 0) - return self.implement_guard(guard_token, 'L') + self.implement_guard(guard_token, 'L') def genop_discard_cond_call_gc_wb(self, op, arglocs): # use 'mc._mc' directly instead of 'mc', to avoid Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py Tue Nov 9 00:03:02 2010 @@ -49,11 +49,13 @@ self.assembler.finish_once() self.profile_agent.shutdown() - def compile_loop(self, inputargs, operations, looptoken): - self.assembler.assemble_loop(inputargs, operations, looptoken) - - def compile_bridge(self, faildescr, inputargs, operations): - self.assembler.assemble_bridge(faildescr, inputargs, operations) + def compile_loop(self, inputargs, operations, looptoken, log=True): + self.assembler.assemble_loop(inputargs, operations, looptoken, + log=log) + + def compile_bridge(self, faildescr, inputargs, operations, log=True): + self.assembler.assemble_bridge(faildescr, inputargs, operations, + log=log) def set_future_value_int(self, index, intvalue): self.assembler.fail_boxes_int.setitem(index, intvalue) Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py Tue Nov 9 00:03:02 2010 @@ -346,7 +346,7 @@ return self.val operations = [ - ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("hello")], None), + ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("hello"), 0], None), ResOperation(rop.INT_ADD, [i0, ConstInt(1)], i1), ResOperation(rop.INT_LE, [i1, ConstInt(9)], i2), ResOperation(rop.GUARD_TRUE, [i2], None, descr=faildescr1), @@ -365,7 +365,7 @@ bridge = [ ResOperation(rop.INT_LE, [i1b, ConstInt(19)], i3), ResOperation(rop.GUARD_TRUE, [i3], None, descr=faildescr2), - ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("bye")], None), + ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("bye"), 0], None), ResOperation(rop.JUMP, [i1b], None, descr=looptoken), ] bridge[1].setfailargs([i1b]) @@ -478,6 +478,10 @@ # whether the test segfaults. assert self.cpu.get_latest_value_int(0) == finished.value + def test_overflow_guard_exception(self): + for i in range(50): + self.test_exceptions() + class TestDebuggingAssembler(object): def setup_method(self, meth): @@ -493,7 +497,7 @@ def test_debugger_on(self): loop = """ [i0] - debug_merge_point('xyz') + debug_merge_point('xyz', 0) i1 = int_add(i0, 1) i2 = int_ge(i1, 10) guard_false(i2) [] @@ -511,3 +515,20 @@ self.cpu.finish_once() lines = py.path.local(self.logfile + ".count").readlines() assert lines[0] == '0:10\n' # '10 xyz\n' + + def test_debugger_checksum(self): + loop = """ + [i0] + debug_merge_point('xyz', 0) + i1 = int_add(i0, 1) + i2 = int_ge(i1, 10) + guard_false(i2) [] + jump(i1) + """ + ops = parse(loop) + self.cpu.assembler.set_debug(True) + self.cpu.compile_loop(ops.inputargs, ops.operations, ops.token) + self.cpu.set_future_value_int(0, 0) + self.cpu.execute_token(ops.token) + assert ops.token._x86_debug_checksum == sum([op.getopnum() + for op in ops.operations]) Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/tool/viewcode.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/x86/tool/viewcode.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/x86/tool/viewcode.py Tue Nov 9 00:03:02 2010 @@ -37,7 +37,7 @@ 'x86_64': 'x86-64', 'i386': 'i386', } - objdump = ('objdump -M intel,%(backend)s -b binary -m i386 ' + objdump = ('objdump -M %(backend)s -b binary -m i386 ' '--adjust-vma=%(origin)d -D %(file)s') # f = open(tmpfile, 'wb') Modified: pypy/branch/fast-forward/pypy/jit/codewriter/call.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/codewriter/call.py (original) +++ pypy/branch/fast-forward/pypy/jit/codewriter/call.py Tue Nov 9 00:03:02 2010 @@ -237,6 +237,8 @@ effectinfo) def _canraise(self, op): + if op.opname == 'pseudo_call_cannot_raise': + return False try: return self.raise_analyzer.can_raise(op) except lltype.DelayedPointer: Modified: pypy/branch/fast-forward/pypy/jit/codewriter/format.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/codewriter/format.py (original) +++ pypy/branch/fast-forward/pypy/jit/codewriter/format.py Tue Nov 9 00:03:02 2010 @@ -80,7 +80,8 @@ def assert_format(ssarepr, expected): asm = format_assembler(ssarepr) - expected = str(py.code.Source(expected)).strip() + '\n' + if expected != '': + expected = str(py.code.Source(expected)).strip() + '\n' asmlines = asm.split("\n") explines = expected.split("\n") for asm, exp in zip(asmlines, explines): Modified: pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py (original) +++ pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py Tue Nov 9 00:03:02 2010 @@ -172,6 +172,7 @@ def rewrite_op_same_as(self, op): pass def rewrite_op_cast_pointer(self, op): pass + def rewrite_op_cast_opaque_ptr(self, op): pass # rlib.rerased def rewrite_op_cast_primitive(self, op): pass def rewrite_op_cast_bool_to_int(self, op): pass def rewrite_op_cast_bool_to_uint(self, op): pass @@ -872,6 +873,8 @@ elif oopspec_name == 'jit.assert_green': kind = getkind(args[0].concretetype) return SpaceOperation('%s_assert_green' % kind, args, None) + elif oopspec_name == 'jit.current_trace_length': + return SpaceOperation('current_trace_length', [], op.result) else: raise AssertionError("missing support for %r" % oopspec_name) @@ -893,17 +896,21 @@ prefix = 'do_resizable_' ARRAY = LIST.items.TO if self._array_of_voids(ARRAY): - raise NotSupported("resizable lists of voids") - descrs = (self.cpu.arraydescrof(ARRAY), - self.cpu.fielddescrof(LIST, 'length'), - self.cpu.fielddescrof(LIST, 'items'), - self.cpu.sizeof(LIST)) + prefix += 'void_' + descrs = () + else: + descrs = (self.cpu.arraydescrof(ARRAY), + self.cpu.fielddescrof(LIST, 'length'), + self.cpu.fielddescrof(LIST, 'items'), + self.cpu.sizeof(LIST)) else: prefix = 'do_fixed_' if self._array_of_voids(LIST): - raise NotSupported("fixed lists of voids") - arraydescr = self.cpu.arraydescrof(LIST) - descrs = (arraydescr,) + prefix += 'void_' + descrs = () + else: + arraydescr = self.cpu.arraydescrof(LIST) + descrs = (arraydescr,) # try: meth = getattr(self, prefix + oopspec_name.replace('.', '_')) @@ -942,6 +949,11 @@ descr, args[1]], v_posindex) return v_posindex, [op0, op1] + def _prepare_void_list_getset(self, op): + non_negative, can_raise = self._get_list_nonneg_canraise_flags(op) + if can_raise: + raise NotSupported("list operation can raise") + def _get_initial_newlist_length(self, op, args): # normalize number of arguments to the 'newlist' function if len(args) > 1: @@ -1013,6 +1025,12 @@ def do_fixed_list_ll_arraycopy(self, op, args, arraydescr): return self._handle_oopspec_call(op, args, EffectInfo.OS_ARRAYCOPY) + def do_fixed_void_list_getitem(self, op, args): + self._prepare_void_list_getset(op) + return [] + do_fixed_void_list_getitem_foldable = do_fixed_void_list_getitem + do_fixed_void_list_setitem = do_fixed_void_list_getitem + # ---------- resizable lists ---------- def do_resizable_newlist(self, op, args, arraydescr, lengthdescr, @@ -1048,6 +1066,12 @@ return SpaceOperation('getfield_gc_i', [args[0], lengthdescr], op.result) + def do_resizable_void_list_getitem(self, op, args): + self._prepare_void_list_getset(op) + return [] + do_resizable_void_list_getitem_foldable = do_resizable_void_list_getitem + do_resizable_void_list_setitem = do_resizable_void_list_getitem + # ---------- # Strings and Unicodes. @@ -1076,7 +1100,7 @@ c_func, TP = support.builtin_func_for_spec(self.cpu.rtyper, oopspec_name, argtypes, resulttype) - op = SpaceOperation('pseudo_call', + op = SpaceOperation('pseudo_call_cannot_raise', [c_func] + [varoftype(T) for T in argtypes], varoftype(resulttype)) calldescr = self.callcontrol.getcalldescr(op, oopspecindex) Modified: pypy/branch/fast-forward/pypy/jit/codewriter/test/test_list.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/codewriter/test/test_list.py (original) +++ pypy/branch/fast-forward/pypy/jit/codewriter/test/test_list.py Tue Nov 9 00:03:02 2010 @@ -19,6 +19,7 @@ class FakeCPU: class arraydescrof(AbstractDescr): def __init__(self, ARRAY): + assert ARRAY.OF != lltype.Void self.ARRAY = ARRAY def __repr__(self): return '' Modified: pypy/branch/fast-forward/pypy/jit/metainterp/blackhole.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/blackhole.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/blackhole.py Tue Nov 9 00:03:02 2010 @@ -774,6 +774,10 @@ def bhimpl_float_assert_green(x): pass + @arguments(returns="i") + def bhimpl_current_trace_length(): + return -1 + # ---------- # the main hints and recursive calls Modified: pypy/branch/fast-forward/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/compile.py Tue Nov 9 00:03:02 2010 @@ -123,7 +123,6 @@ if not we_are_translated(): show_loop(metainterp_sd) TreeLoop.check_consistency_of(inputargs, operations) - pass metainterp_sd.profiler.start_backend() debug_start("jit-backend") try: @@ -600,5 +599,5 @@ ResOperation(rop.FINISH, finishargs, None, descr=jd.portal_finishtoken) ] operations[1].setfailargs([]) - cpu.compile_loop(inputargs, operations, loop_token) + cpu.compile_loop(inputargs, operations, loop_token, log=False) return loop_token Modified: pypy/branch/fast-forward/pypy/jit/metainterp/logger.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/logger.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/logger.py Tue Nov 9 00:03:02 2010 @@ -81,7 +81,8 @@ op = operations[i] if op.getopnum() == rop.DEBUG_MERGE_POINT: loc = op.getarg(0)._get_str() - debug_print("debug_merge_point('%s')" % (loc,)) + reclev = op.getarg(1).getint() + debug_print("debug_merge_point('%s', %s)" % (loc, reclev)) continue args = ", ".join([self.repr_of_arg(memo, op.getarg(i)) for i in range(op.numargs())]) if op.result is not None: Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/rewrite.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/rewrite.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/rewrite.py Tue Nov 9 00:03:02 2010 @@ -244,14 +244,11 @@ self.optimizer.exception_might_have_happened = False def optimize_CALL_LOOPINVARIANT(self, op): - funcvalue = self.getvalue(op.getarg(0)) - if not funcvalue.is_constant(): - # XXX this code path is never executed in tests nor in production. - # in fact, it can't even happen since residual_call in codewriter - # expects a compile-time constant - self.emit_operation(op) - return - key = make_hashable_int(op.getarg(0).getint()) + arg = op.getarg(0) + # 'arg' must be a Const, because residual_call in codewriter + # expects a compile-time constant + assert isinstance(arg, Const) + key = make_hashable_int(arg.getint()) resvalue = self.optimizer.loop_invariant_results.get(key, None) if resvalue is not None: self.make_equal_to(op.result, resvalue) Modified: pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py Tue Nov 9 00:03:02 2010 @@ -820,7 +820,8 @@ jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex] self.verify_green_args(jitdriver_sd, greenboxes) # xxx we may disable the following line in some context later - self.debug_merge_point(jitdriver_sd, greenboxes) + self.debug_merge_point(jitdriver_sd, self.metainterp.in_recursion, + greenboxes) if self.metainterp.seen_loop_header_for_jdindex < 0: if not jitdriver_sd.no_loop_header or not any_operation: return @@ -860,13 +861,13 @@ assembler_call=True) raise ChangeFrame - def debug_merge_point(self, jitdriver_sd, greenkey): + def debug_merge_point(self, jitdriver_sd, in_recursion, greenkey): # debugging: produce a DEBUG_MERGE_POINT operation loc = jitdriver_sd.warmstate.get_location_str(greenkey) debug_print(loc) constloc = self.metainterp.cpu.ts.conststr(loc) self.metainterp.history.record(rop.DEBUG_MERGE_POINT, - [constloc], None) + [constloc, ConstInt(in_recursion)], None) @arguments("box", "label") def opimpl_goto_if_exception_mismatch(self, vtablebox, next_exc_target): @@ -948,6 +949,11 @@ opimpl_ref_assert_green = _opimpl_assert_green opimpl_float_assert_green = _opimpl_assert_green + @arguments() + def opimpl_current_trace_length(self): + trace_length = len(self.metainterp.history.operations) + return ConstInt(trace_length) + @arguments("box") def opimpl_virtual_ref(self, box): # Details on the content of metainterp.virtualref_boxes: Modified: pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py Tue Nov 9 00:03:02 2010 @@ -458,7 +458,7 @@ 'NEWUNICODE/1', #'RUNTIMENEW/1', # ootype operation 'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier) - 'DEBUG_MERGE_POINT/1', # debugging only + 'DEBUG_MERGE_POINT/2', # debugging only 'JIT_DEBUG/*', # debugging only 'VIRTUAL_REF_FINISH/2', # removed before it's passed to the backend 'COPYSTRCONTENT/5', # src, dst, srcstart, dststart, length Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py Tue Nov 9 00:03:02 2010 @@ -3,6 +3,7 @@ from pypy.rlib.jit import JitDriver, we_are_jitted, hint, dont_look_inside from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE, loop_invariant from pypy.rlib.jit import jit_debug, assert_green, AssertGreenFailed +from pypy.rlib.jit import unroll_safe, current_trace_length from pypy.jit.metainterp.warmspot import ll_meta_interp, get_stats from pypy.jit.backend.llgraph import runner from pypy.jit.metainterp import pyjitpl, history @@ -1672,6 +1673,31 @@ assert res == 8 py.test.raises(AssertGreenFailed, self.interp_operations, f, [8, 0]) + def test_current_trace_length(self): + myjitdriver = JitDriver(greens = ['g'], reds = ['x']) + @dont_look_inside + def residual(): + print "hi there" + @unroll_safe + def loop(g): + y = 0 + while y < g: + residual() + y += 1 + def f(x, g): + n = 0 + while x > 0: + myjitdriver.can_enter_jit(x=x, g=g) + myjitdriver.jit_merge_point(x=x, g=g) + loop(g) + x -= 1 + n = current_trace_length() + return n + res = self.meta_interp(f, [5, 8]) + assert 14 < res < 42 + res = self.meta_interp(f, [5, 2]) + assert 4 < res < 14 + class TestOOtype(BasicTests, OOJitMixin): Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py Tue Nov 9 00:03:02 2010 @@ -97,7 +97,7 @@ def test_debug_merge_point(self): inp = ''' [] - debug_merge_point("info") + debug_merge_point("info", 0) ''' loop, oloop = self.reparse(inp, check_equal=False) assert loop.operations[0].getarg(0)._get_str() == 'info' Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_oparser.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_oparser.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_oparser.py Tue Nov 9 00:03:02 2010 @@ -141,10 +141,10 @@ def test_debug_merge_point(): x = ''' [] - debug_merge_point("info") - debug_merge_point('info') - debug_merge_point(' info') - debug_merge_point('(stuff) #1') + debug_merge_point("info", 0) + debug_merge_point('info', 1) + debug_merge_point(' info', 1) + debug_merge_point('(stuff) #1', 1) ''' loop = parse(x) assert loop.operations[0].getarg(0)._get_str() == 'info' @@ -168,7 +168,7 @@ i6 = int_sub(i1, 1) i8 = int_gt(i6, 3) guard_true(i8, descr=) [i4, i6] -debug_merge_point('(no jitdriver.get_printable_location!)') +debug_merge_point('(no jitdriver.get_printable_location!)', 0) jump(i6, i4, descr=) ''' Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py Tue Nov 9 00:03:02 2010 @@ -807,10 +807,10 @@ guard_value(i2, 1) [] i3 = call_loopinvariant(1, i1, descr=nonwritedescr) guard_no_exception() [] - guard_value(i2, 1) [] + guard_value(i3, 1) [] i4 = call_loopinvariant(1, i1, descr=nonwritedescr) guard_no_exception() [] - guard_value(i2, 1) [] + guard_value(i4, 1) [] jump(i1) """ expected = """ @@ -1390,7 +1390,7 @@ ops = """ [p1] i1 = getfield_gc(p1, descr=valuedescr) - debug_merge_point(15) + debug_merge_point(15, 0) i2 = getfield_gc(p1, descr=valuedescr) escape(i1) escape(i2) @@ -1399,7 +1399,7 @@ expected = """ [p1] i1 = getfield_gc(p1, descr=valuedescr) - debug_merge_point(15) + debug_merge_point(15, 0) escape(i1) escape(i1) jump(p1) Modified: pypy/branch/fast-forward/pypy/jit/tl/pypyjit.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/tl/pypyjit.py (original) +++ pypy/branch/fast-forward/pypy/jit/tl/pypyjit.py Tue Nov 9 00:03:02 2010 @@ -38,13 +38,12 @@ config.objspace.allworkingmodules = False config.objspace.usemodules.pypyjit = True config.objspace.usemodules.array = True -config.objspace.usemodules._weakref = False +config.objspace.usemodules._weakref = True config.objspace.usemodules._sre = False # config.objspace.usemodules._ffi = True # set_pypy_opt_level(config, level='jit') -config.objspace.std.withinlineddict = True if BACKEND == 'c': config.objspace.std.multimethods = 'mrd' Modified: pypy/branch/fast-forward/pypy/jit/tool/oparser.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/tool/oparser.py (original) +++ pypy/branch/fast-forward/pypy/jit/tool/oparser.py Tue Nov 9 00:03:02 2010 @@ -189,7 +189,7 @@ descr = None if argspec.strip(): if opname == 'debug_merge_point': - allargs = [argspec] + allargs = argspec.rsplit(', ', 1) else: allargs = [arg for arg in argspec.split(",") if arg != ''] Modified: pypy/branch/fast-forward/pypy/jit/tool/pypytrace.vim ============================================================================== --- pypy/branch/fast-forward/pypy/jit/tool/pypytrace.vim (original) +++ pypy/branch/fast-forward/pypy/jit/tool/pypytrace.vim Tue Nov 9 00:03:02 2010 @@ -21,10 +21,10 @@ hi def link pypyLoopStart Structure "hi def link pypyLoopArgs PreProc -hi def link pypyFailArgs String -hi def link pypyOpName Statement -hi def link pypyDebugMergePoint Comment +hi def link pypyFailArgs Special +"hi def link pypyOpName Statement +hi def link pypyDebugMergePoint String hi def link pypyConstPtr Constant hi def link pypyNumber Number -hi def link pypyDescr String +hi def link pypyDescr PreProc hi def link pypyDescrField Label Modified: pypy/branch/fast-forward/pypy/jit/tool/showstats.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/tool/showstats.py (original) +++ pypy/branch/fast-forward/pypy/jit/tool/showstats.py Tue Nov 9 00:03:02 2010 @@ -10,9 +10,10 @@ def main(argv): log = logparser.parse_log_file(argv[0]) + log_count_lines = open(argv[0] + '.count').readlines() parts = logparser.extract_category(log, "jit-log-opt-") for i, oplist in enumerate(parts): - loop = parse(oplist, no_namespace=True) + loop = parse(oplist, no_namespace=True, nonstrict=True) num_ops = 0 num_dmp = 0 num_guards = 0 @@ -23,7 +24,11 @@ num_ops += 1 if op.is_guard(): num_guards += 1 - print "Loop #%d, length: %d, opcodes: %d, guards: %d, %f" % (i, num_ops, num_dmp, num_guards, num_ops/num_dmp) - + if num_dmp == 0: + print "Loop #%d, length: %d, opcodes: %d, guards: %d" % (i, num_ops, num_dmp, num_guards) + else: + print "Loop #%d, length: %d, opcodes: %d, guards: %d, %f" % (i, num_ops, num_dmp, num_guards, num_ops/num_dmp) + print loop.comment, "run", log_count_lines[i].split(":")[1].strip(), "times" + if __name__ == '__main__': main(sys.argv[1:]) Modified: pypy/branch/fast-forward/pypy/jit/tool/test/test_traceviewer.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/tool/test/test_traceviewer.py (original) +++ pypy/branch/fast-forward/pypy/jit/tool/test/test_traceviewer.py Tue Nov 9 00:03:02 2010 @@ -19,7 +19,7 @@ def test_no_of_loops(self): data = [preparse(""" # Loop 0 : loop with 39 ops - debug_merge_point('') + debug_merge_point('', 0) guard_class(p4, 141310752, descr=) [p0, p1] p60 = getfield_gc(p4, descr=) guard_nonnull(p60, descr=) [p0, p1] @@ -51,7 +51,7 @@ assert loop.right.content == 'extra' def test_postparse(self): - real_loops = [FinalBlock("debug_merge_point(' #40 POP_TOP')", None)] + real_loops = [FinalBlock("debug_merge_point(' #40 POP_TOP', 0)", None)] postprocess(real_loops, real_loops[:], {}) assert real_loops[0].header.startswith("_runCallbacks, file '/tmp/x/twisted-trunk/twisted/internet/defer.py', line 357") Modified: pypy/branch/fast-forward/pypy/module/__builtin__/descriptor.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/descriptor.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/descriptor.py Tue Nov 9 00:03:02 2010 @@ -96,6 +96,8 @@ ) class W_Property(Wrappable): + _immutable_fields_ = ["w_fget", "w_fset", "w_fdel"] + def init(self, space, w_fget=None, w_fset=None, w_fdel=None, w_doc=None): self.w_fget = w_fget self.w_fset = w_fset Modified: pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/interp_classobj.py Tue Nov 9 00:03:02 2010 @@ -411,8 +411,7 @@ if w_meth is not None: space.call_function(w_meth, w_name, w_value) else: - # bit obscure: appease normalization - self.setdictvalue(space, name, w_value, True) + self.setdictvalue(space, name, w_value) def descr_delattr(self, space, w_name): name = unwrap_attr(space, w_name) Modified: pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/test/test_classobj.py Tue Nov 9 00:03:02 2010 @@ -981,28 +981,7 @@ raises(TypeError, descr.__delete__, a) -class AppTestOldStyleSharing(AppTestOldstyle): - def setup_class(cls): - cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True}) - if option.runappdirect: - py.test.skip("can only be run on py.py") - def is_sharing(space, w_inst): - from pypy.objspace.std.sharingdict import SharedDictImplementation - w_d = w_inst.getdict() - return space.wrap(isinstance(w_d, SharedDictImplementation) and w_d.r_dict_content is None) - cls.w_is_sharing = cls.space.wrap(gateway.interp2app(is_sharing)) - - - def test_real_sharing(self): - class A: - def __init__(self): - self.x = 42 - A1, A2, A3 = A(), A(), A() - assert self.is_sharing(A3) - assert self.is_sharing(A2) - assert self.is_sharing(A1) - -class AppTestOldStyleModDict(object): +class AppTestOldStyleClassStrDict(object): def setup_class(cls): if option.runappdirect: py.test.skip("can only be run on py.py") Modified: pypy/branch/fast-forward/pypy/module/__pypy__/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__pypy__/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/__pypy__/__init__.py Tue Nov 9 00:03:02 2010 @@ -25,3 +25,7 @@ 'interp_magic.reset_method_cache_counter') PYC_MAGIC = get_pyc_magic(self.space) self.extra_interpdef('PYC_MAGIC', 'space.wrap(%d)' % PYC_MAGIC) + # + from pypy.jit.backend import detect_cpu + model = detect_cpu.autodetect_main_model_and_size() + self.extra_interpdef('cpumodel', 'space.wrap(%r)' % model) Modified: pypy/branch/fast-forward/pypy/module/__pypy__/test/test_special.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__pypy__/test/test_special.py (original) +++ pypy/branch/fast-forward/pypy/module/__pypy__/test/test_special.py Tue Nov 9 00:03:02 2010 @@ -17,3 +17,7 @@ from __pypy__ import isfake import select assert isfake(select) + + def test_cpumodel(self): + import __pypy__ + assert hasattr(__pypy__, 'cpumodel') Modified: pypy/branch/fast-forward/pypy/module/gc/interp_gc.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/gc/interp_gc.py (original) +++ pypy/branch/fast-forward/pypy/module/gc/interp_gc.py Tue Nov 9 00:03:02 2010 @@ -5,6 +5,11 @@ def collect(space): "Run a full collection." + # First clear the method cache. See test_gc for an example of why. + if space.config.objspace.std.withmethodcache: + from pypy.objspace.std.typeobject import MethodCache + cache = space.fromcache(MethodCache) + cache.clear() rgc.collect() return space.wrap(0) Modified: pypy/branch/fast-forward/pypy/module/gc/test/test_gc.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/gc/test/test_gc.py (original) +++ pypy/branch/fast-forward/pypy/module/gc/test/test_gc.py Tue Nov 9 00:03:02 2010 @@ -103,3 +103,22 @@ import gc gc.dump_heap_stats(self.fname) + +class AppTestGcMethodCache(object): + def setup_class(cls): + cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True}) + + def test_clear_method_cache(self): + import gc, weakref + rlist = [] + def f(): + class C(object): + def f(self): + pass + C().f() # Fill the method cache + rlist.append(weakref.ref(C)) + for i in range(5): + f() + gc.collect() # the classes C should all go away here + for r in rlist: + assert r() is None Modified: pypy/branch/fast-forward/pypy/module/pypyjit/interp_jit.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pypyjit/interp_jit.py (original) +++ pypy/branch/fast-forward/pypy/module/pypyjit/interp_jit.py Tue Nov 9 00:03:02 2010 @@ -6,6 +6,7 @@ from pypy.tool.pairtype import extendabletype from pypy.rlib.rarithmetic import r_uint, intmask from pypy.rlib.jit import JitDriver, hint, we_are_jitted, dont_look_inside +from pypy.rlib.jit import current_trace_length import pypy.interpreter.pyopcode # for side-effects from pypy.interpreter.error import OperationError, operationerrfmt from pypy.interpreter.gateway import ObjSpace, Arguments, W_Root @@ -80,9 +81,22 @@ def jump_absolute(self, jumpto, _, ec=None): if we_are_jitted(): + # Normally, the tick counter is decremented by 100 for every + # Python opcode. Here, to better support JIT compilation of + # small loops, we decrement it by a possibly smaller constant. + # We get the maximum 100 when the (unoptimized) trace length + # is at least 3200 (a bit randomly). + trace_length = r_uint(current_trace_length()) + decr_by = trace_length // 32 + if decr_by < 1: + decr_by = 1 + elif decr_by > 100: # also if current_trace_length() returned -1 + decr_by = 100 + # self.last_instr = intmask(jumpto) - ec.bytecode_trace(self) + ec.bytecode_trace(self, intmask(decr_by)) jumpto = r_uint(self.last_instr) + # pypyjitdriver.can_enter_jit(frame=self, ec=ec, next_instr=jumpto, pycode=self.getcode()) return jumpto Modified: pypy/branch/fast-forward/pypy/module/pypyjit/policy.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pypyjit/policy.py (original) +++ pypy/branch/fast-forward/pypy/module/pypyjit/policy.py Tue Nov 9 00:03:02 2010 @@ -6,7 +6,8 @@ if (modname == '__builtin__.operation' or modname == '__builtin__.abstractinst' or modname == '__builtin__.interp_classobj' or - modname == '__builtin__.functional'): + modname == '__builtin__.functional' or + modname == '__builtin__.descriptor'): return True if '.' in modname: modname, _ = modname.split('.', 1) @@ -19,7 +20,7 @@ # this function should never actually return True directly # but instead call the base implementation mod = func.__module__ or '?' - + if mod.startswith('pypy.objspace.'): # gc_id operation if func.__name__ == 'id__ANY': @@ -36,5 +37,5 @@ modname = mod[len('pypy.module.'):] if not self.look_inside_pypy_module(modname): return False - + return True Modified: pypy/branch/fast-forward/pypy/module/pypyjit/test/test_policy.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pypyjit/test/test_policy.py (original) +++ pypy/branch/fast-forward/pypy/module/pypyjit/test/test_policy.py Tue Nov 9 00:03:02 2010 @@ -12,7 +12,7 @@ def test_rlocale(): from pypy.rlib.rlocale import setlocale - assert not pypypolicy.look_inside_function(setlocale) + assert not pypypolicy.look_inside_function(setlocale) def test_geninterp(): d = {'_geninterp_': True} @@ -28,6 +28,10 @@ from pypy.interpreter.pyparser import parser assert not pypypolicy.look_inside_function(parser.Grammar.__init__.im_func) +def test_property(): + from pypy.module.__builtin__.descriptor import W_Property + assert pypypolicy.look_inside_function(W_Property.get.im_func) + def test_pypy_module(): from pypy.module._random.interp_random import W_Random assert not pypypolicy.look_inside_function(W_Random.random) @@ -35,6 +39,7 @@ assert pypypolicy.look_inside_pypy_module('__builtin__.operation') assert pypypolicy.look_inside_pypy_module('__builtin__.abstractinst') assert pypypolicy.look_inside_pypy_module('__builtin__.functional') + assert pypypolicy.look_inside_pypy_module('__builtin__.descriptor') assert pypypolicy.look_inside_pypy_module('exceptions.interp_exceptions') for modname in 'pypyjit', 'signal', 'micronumpy', 'math', 'imp': assert pypypolicy.look_inside_pypy_module(modname) @@ -42,4 +47,3 @@ def test_see_jit_module(): assert pypypolicy.look_inside_pypy_module('pypyjit.interp_jit') - Modified: pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py Tue Nov 9 00:03:02 2010 @@ -93,6 +93,9 @@ # some support code... print >> f, py.code.Source(""" import sys + # we don't want to see the small bridges created + # by the checkinterval reaching the limit + sys.setcheckinterval(10000000) try: # make the file runnable by CPython import pypyjit pypyjit.set_param(threshold=%d) @@ -136,9 +139,10 @@ assert opslogfile.check() log = logparser.parse_log_file(str(opslogfile)) parts = logparser.extract_category(log, 'jit-log-opt-') + self.rawloops = [part for part in parts + if not from_entry_bridge(part, parts)] # skip entry bridges, they can contain random things - self.loops = [parse(part, no_namespace=True) for part in parts - if not from_entry_bridge(part, parts)] + self.loops = [parse(part, no_namespace=True) for part in self.rawloops] self.sliced_loops = [] # contains all bytecodes of all loops self.total_ops = 0 for loop in self.loops: @@ -162,12 +166,11 @@ return [ops for ops in self.sliced_loops if ops.bytecode == name] def print_loops(self): - for loop in self.loops: + for rawloop in self.rawloops: print print '@' * 79 print - for op in loop.operations: - print op + print rawloop.rstrip() print print '@' * 79 @@ -278,7 +281,7 @@ assert len(ops) == 2 assert not ops[0].get_opnames("call") assert not ops[0].get_opnames("new") - assert len(ops[0].get_opnames("guard")) <= 3 # we get 2 withmapdict + assert len(ops[0].get_opnames("guard")) <= 2 assert not ops[1] # second LOOKUP_METHOD folded away ops = self.get_by_bytecode("CALL_METHOD") @@ -294,7 +297,11 @@ ops = self.get_by_bytecode("LOAD_ATTR") assert len(ops) == 2 - assert ops[0].get_opnames() == ["getfield_gc", "getarrayitem_gc", + # With mapdict, we get fast access to (so far) the 5 first + # attributes, which means it is done with only the following + # operations. (For the other attributes there is additionally + # a getarrayitem_gc.) + assert ops[0].get_opnames() == ["getfield_gc", "guard_nonnull_class"] assert not ops[1] # second LOAD_ATTR folded away @@ -323,8 +330,8 @@ assert len(ops) == 2 assert not ops[0].get_opnames("call") assert not ops[0].get_opnames("new") - assert len(ops[0].get_opnames("guard")) <= 3 # we get 2 withmapdict - assert len(ops[0].get_opnames("getfield")) <= 5 # we get <5 withmapdict + assert len(ops[0].get_opnames("guard")) <= 2 + assert len(ops[0].get_opnames("getfield")) <= 4 assert not ops[1] # second LOOKUP_METHOD folded away def test_default_and_kw(self): Modified: pypy/branch/fast-forward/pypy/module/signal/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/signal/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/signal/__init__.py Tue Nov 9 00:03:02 2010 @@ -35,9 +35,7 @@ MixedModule.__init__(self, space, *args) # add the signal-checking callback as an action on the space space.check_signal_action = interp_signal.CheckSignalAction(space) - space.actionflag.register_action(space.check_signal_action) - # use the C-level pypysig_occurred variable as the action flag - # (the result is that the C-level signal handler will directly - # set the flag for the CheckSignalAction) + space.actionflag.register_periodic_action(space.check_signal_action, + use_bytecode_counter=False) space.actionflag.__class__ = interp_signal.SignalActionFlag # xxx yes I know the previous line is a hack Modified: pypy/branch/fast-forward/pypy/module/signal/interp_signal.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/signal/interp_signal.py (original) +++ pypy/branch/fast-forward/pypy/module/signal/interp_signal.py Tue Nov 9 00:03:02 2010 @@ -1,6 +1,7 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.baseobjspace import W_Root, ObjSpace from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag +from pypy.interpreter.executioncontext import PeriodicAsyncAction import signal as cpy_signal from pypy.rpython.lltypesystem import lltype, rffi from pypy.translator.tool.cbuild import ExternalCompilationInfo @@ -55,19 +56,29 @@ class SignalActionFlag(AbstractActionFlag): - def get(self): + # This class uses the C-level pypysig_counter variable as the tick + # counter. The C-level signal handler will reset it to -1 whenever + # a signal is received. + + def get_ticker(self): p = pypysig_getaddr_occurred() return p.c_value - def set(self, value): + + def reset_ticker(self, value): p = pypysig_getaddr_occurred() p.c_value = value + def decrement_ticker(self, by): + p = pypysig_getaddr_occurred() + value = p.c_value + if self.has_bytecode_counter: # this 'if' is constant-folded + value -= by + p.c_value = value + return value -class CheckSignalAction(AsyncAction): - """An action that is automatically invoked when a signal is received.""" - # The C-level signal handler sets the bit 30 of pypysig_occurred: - bitmask = 1 << 30 +class CheckSignalAction(PeriodicAsyncAction): + """An action that is automatically invoked when a signal is received.""" def __init__(self, space): AsyncAction.__init__(self, space) @@ -76,7 +87,6 @@ # need a helper action in case signals arrive in a non-main thread self.pending_signals = {} self.reissue_signal_action = ReissueSignalAction(space) - space.actionflag.register_action(self.reissue_signal_action) else: self.reissue_signal_action = None Modified: pypy/branch/fast-forward/pypy/module/signal/test/test_signal.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/signal/test/test_signal.py (original) +++ pypy/branch/fast-forward/pypy/module/signal/test/test_signal.py Tue Nov 9 00:03:02 2010 @@ -1,6 +1,38 @@ import os, py +import signal as cpy_signal from pypy.conftest import gettestobjspace + +class TestCheckSignals: + + def setup_class(cls): + if not hasattr(os, 'kill') or not hasattr(os, 'getpid'): + py.test.skip("requires os.kill() and os.getpid()") + cls.space = gettestobjspace(usemodules=['signal']) + + def test_checksignals(self): + space = self.space + w_received = space.appexec([], """(): + import signal + received = [] + def myhandler(signum, frame): + received.append(signum) + signal.signal(signal.SIGUSR1, myhandler) + return received""") + # + assert not space.is_true(w_received) + # + # send the signal now + os.kill(os.getpid(), cpy_signal.SIGUSR1) + # + # myhandler() should not be immediately called + assert not space.is_true(w_received) + # + # calling ec.checksignals() should call it + space.getexecutioncontext().checksignals() + assert space.is_true(w_received) + + class AppTestSignal: def setup_class(cls): @@ -24,18 +56,12 @@ signal.signal(signal.SIGUSR1, myhandler) posix.kill(posix.getpid(), signal.SIGUSR1) - for i in range(10000): - # wait a bit for the signal to be delivered to the handler - if received: - break + # the signal should be delivered to the handler immediately assert received == [signal.SIGUSR1] del received[:] posix.kill(posix.getpid(), signal.SIGUSR1) - for i in range(10000): - # wait a bit for the signal to be delivered to the handler - if received: - break + # the signal should be delivered to the handler immediately assert received == [signal.SIGUSR1] del received[:] Modified: pypy/branch/fast-forward/pypy/module/sys/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/sys/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/sys/__init__.py Tue Nov 9 00:03:02 2010 @@ -10,7 +10,6 @@ if space.config.translating: del self.__class__.interpleveldefs['pypy_getudir'] super(Module, self).__init__(space, w_name) - self.checkinterval = 100 self.recursionlimit = 100 self.w_default_encoder = None self.defaultencoding = "ascii" Modified: pypy/branch/fast-forward/pypy/module/sys/version.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/sys/version.py (original) +++ pypy/branch/fast-forward/pypy/module/sys/version.py Tue Nov 9 00:03:02 2010 @@ -8,7 +8,7 @@ CPYTHON_VERSION = (2, 7, 0, "final", 42) #XXX # sync patchlevel.h CPYTHON_API_VERSION = 1012 #XXX # sync with include/modsupport.h -PYPY_VERSION = (1, 3, 0, "beta", '?') #XXX # sync patchlevel.h +PYPY_VERSION = (1, 4, 0, "beta", '?') #XXX # sync patchlevel.h # the last item is replaced by the svn revision ^^^ TRIM_URL_UP_TO = 'svn/pypy/' Modified: pypy/branch/fast-forward/pypy/module/sys/vm.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/sys/vm.py (original) +++ pypy/branch/fast-forward/pypy/module/sys/vm.py Tue Nov 9 00:03:02 2010 @@ -67,7 +67,7 @@ def setcheckinterval(space, interval): """Tell the Python interpreter to check for asynchronous events every n instructions. This also affects how often thread switches occur.""" - space.actionflag.setcheckinterval(space, interval) + space.actionflag.setcheckinterval(interval) setcheckinterval.unwrap_spec = [ObjSpace, int] def getcheckinterval(space): @@ -77,7 +77,7 @@ # return 0. The idea is that according to the CPython docs, <= 0 # means "check every virtual instruction, maximizing responsiveness # as well as overhead". - result = space.sys.checkinterval + result = space.actionflag.getcheckinterval() if result <= 1: result = 0 return space.wrap(result) Modified: pypy/branch/fast-forward/pypy/module/thread/gil.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/thread/gil.py (original) +++ pypy/branch/fast-forward/pypy/module/thread/gil.py Tue Nov 9 00:03:02 2010 @@ -20,7 +20,8 @@ def initialize(self, space): # add the GIL-releasing callback as an action on the space - space.actionflag.register_action(GILReleaseAction(space)) + space.actionflag.register_periodic_action(GILReleaseAction(space), + use_bytecode_counter=True) def setup_threads(self, space): """Enable threads in the object space, if they haven't already been.""" @@ -44,7 +45,6 @@ # test_compile_lock. As a workaround, we repatch these global # fields systematically. spacestate.ll_GIL = self.ll_GIL - spacestate.actionflag = space.actionflag invoke_around_extcall(before_external_call, after_external_call) return result @@ -74,18 +74,17 @@ def _freeze_(self): self.ll_GIL = thread.null_ll_lock - self.actionflag = None - self.set_actionflag_bit_after_thread_switch = 0 + self.action_after_thread_switch = None + # ^^^ set by AsyncAction.fire_after_thread_switch() return False def after_thread_switch(self): # this is support logic for the signal module, to help it deliver # signals to the main thread. - actionflag = self.actionflag - if actionflag is not None: - flag = actionflag.get() - flag |= self.set_actionflag_bit_after_thread_switch - actionflag.set(flag) + action = self.action_after_thread_switch + if action is not None: + self.action_after_thread_switch = None + action.fire() spacestate = SpaceState() spacestate._freeze_() Modified: pypy/branch/fast-forward/pypy/module/thread/test/test_gil.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/thread/test/test_gil.py (original) +++ pypy/branch/fast-forward/pypy/module/thread/test/test_gil.py Tue Nov 9 00:03:02 2010 @@ -8,7 +8,7 @@ pass class FakeActionFlag(object): - def register_action(self, action): + def register_periodic_action(self, action, use_bytecode_counter): pass def get(self): return 0 Modified: pypy/branch/fast-forward/pypy/objspace/descroperation.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/descroperation.py (original) +++ pypy/branch/fast-forward/pypy/objspace/descroperation.py Tue Nov 9 00:03:02 2010 @@ -51,7 +51,7 @@ return space.is_w(w_typ1, w_typ2) -class Object: +class Object(object): def descr__getattribute__(space, w_obj, w_name): name = space.str_w(w_name) w_descr = space.lookup(w_obj, name) @@ -64,9 +64,7 @@ w_type = space.type(w_obj) return space.get_and_call_function(w_get, w_descr, w_obj, w_type) - w_value = w_obj.getdictvalue_attr_is_in_class(space, name) - else: - w_value = w_obj.getdictvalue(space, name) + w_value = w_obj.getdictvalue(space, name) if w_value is not None: return w_value if w_descr is not None: @@ -76,13 +74,11 @@ def descr__setattr__(space, w_obj, w_name, w_value): name = space.str_w(w_name) w_descr = space.lookup(w_obj, name) - shadows_type = False if w_descr is not None: if space.is_data_descr(w_descr): space.set(w_descr, w_obj, w_value) return - shadows_type = True - if w_obj.setdictvalue(space, name, w_value, shadows_type): + if w_obj.setdictvalue(space, name, w_value): return raiseattrerror(space, w_obj, name, w_descr) @@ -100,7 +96,7 @@ def descr__init__(space, w_obj, __args__): pass -class DescrOperation: +class DescrOperation(object): _mixin_ = True def is_data_descr(space, w_obj): @@ -573,7 +569,7 @@ # what is the maximum value slices can get on CPython? # we need to stick to that value, because fake.py etc. -class Temp: +class Temp(object): def __getslice__(self, i, j): return j slice_max = Temp()[:] Modified: pypy/branch/fast-forward/pypy/objspace/std/callmethod.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/callmethod.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/callmethod.py Tue Nov 9 00:03:02 2010 @@ -44,7 +44,7 @@ else: typ = type(w_descr) if typ is function.Function or typ is function.FunctionWithFixedCode: - w_value = w_obj.getdictvalue_attr_is_in_class(space, name) + w_value = w_obj.getdictvalue(space, name) if w_value is None: # fast method path: a function object in the class, # nothing in the instance @@ -103,7 +103,7 @@ w_descr = space.lookup(w_obj, methname) typ = type(w_descr) if typ is function.Function or typ is function.FunctionWithFixedCode: - w_value = w_obj.getdictvalue_attr_is_in_class(space, methname) + w_value = w_obj.getdictvalue(space, methname) if w_value is None: # fast method path: a function object in the class, # nothing in the instance Modified: pypy/branch/fast-forward/pypy/objspace/std/celldict.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/celldict.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/celldict.py Tue Nov 9 00:03:02 2010 @@ -47,7 +47,7 @@ else: self._as_rdict().impl_fallback_setitem(w_key, w_value) - def impl_setitem_str(self, name, w_value, shadows_type=True): + def impl_setitem_str(self, name, w_value): self.getcell(name, True).w_value = w_value def impl_delitem(self, w_key): Modified: pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py Tue Nov 9 00:03:02 2010 @@ -49,14 +49,6 @@ elif space.config.objspace.std.withdictmeasurement: assert w_type is None return MeasuringDictImplementation(space) - elif space.config.objspace.std.withsharingdict and instance: - from pypy.objspace.std.sharingdict import SharedDictImplementation - assert w_type is None - return SharedDictImplementation(space) - elif (space.config.objspace.std.withshadowtracking and instance and - classofinstance is not None): - assert w_type is None - return ShadowDetectingDictImplementation(space, classofinstance) elif instance or strdict or module: assert w_type is None return StrDictImplementation(space) @@ -112,7 +104,7 @@ #return w_value or None raise NotImplementedError("abstract base class") - def impl_setitem_str(self, key, w_value, shadows_type=True): + def impl_setitem_str(self, key, w_value): raise NotImplementedError("abstract base class") def impl_setitem(self, w_key, w_value): @@ -165,12 +157,15 @@ key = OPTIMIZED_BUILTINS[i] return self.impl_getitem_str(key) - # this method will only be seen whan a certain config option is used - def impl_shadows_anything(self): - return True - - def impl_set_shadows_anything(self): - pass + def impl_popitem(self): + # default implementation + space = self.space + iterator = self.impl_iter() + w_key, w_value = iterator.next() + if w_key is None: + raise KeyError + self.impl_delitem(w_key) + return w_key, w_value # _________________________________________________________________ # fallback implementation methods @@ -178,7 +173,7 @@ def impl_fallback_setitem(self, w_key, w_value): self.r_dict_content[w_key] = w_value - def impl_fallback_setitem_str(self, key, w_value, shadows_type=True): + def impl_fallback_setitem_str(self, key, w_value): return self.impl_fallback_setitem(self.space.wrap(key), w_value) def impl_fallback_delitem(self, w_key): @@ -211,18 +206,15 @@ key = OPTIMIZED_BUILTINS[i] return self.impl_fallback_getitem_str(key) - def impl_fallback_shadows_anything(self): - return True - - def impl_fallback_set_shadows_anything(self): - pass + def impl_fallback_popitem(self): + return self.r_dict_content.popitem() implementation_methods = [ ("getitem", 1), ("getitem_str", 1), ("length", 0), - ("setitem_str", 3), + ("setitem_str", 2), ("setitem", 2), ("delitem", 1), ("iter", 0), @@ -231,8 +223,7 @@ ("keys", 0), ("clear", 0), ("get_builtin_indexed", 1), - ("shadows_anything", 0), - ("set_shadows_anything", 0), + ("popitem", 0), ] @@ -312,7 +303,7 @@ else: self._as_rdict().impl_fallback_setitem(w_key, w_value) - def impl_setitem_str(self, key, w_value, shadows_type=True): + def impl_setitem_str(self, key, w_value): self.content[key] = w_value def impl_delitem(self, w_key): @@ -388,47 +379,12 @@ return None, None -class ShadowDetectingDictImplementation(StrDictImplementation): - def __init__(self, space, w_type): - StrDictImplementation.__init__(self, space) - self.w_type = w_type - self.original_version_tag = w_type.version_tag() - if self.original_version_tag is None: - self._shadows_anything = True - else: - self._shadows_anything = False - - def impl_setitem_str(self, key, w_value, shadows_type=True): - if shadows_type: - self._shadows_anything = True - StrDictImplementation.impl_setitem_str( - self, key, w_value, shadows_type) - - def impl_setitem(self, w_key, w_value): - space = self.space - if space.is_w(space.type(w_key), space.w_str): - if not self._shadows_anything: - w_obj = self.w_type.lookup(space.str_w(w_key)) - if w_obj is not None: - self._shadows_anything = True - StrDictImplementation.impl_setitem_str( - self, self.space.str_w(w_key), w_value, False) - else: - self._as_rdict().impl_fallback_setitem(w_key, w_value) - - def impl_shadows_anything(self): - return (self._shadows_anything or - self.w_type.version_tag() is not self.original_version_tag) - - def impl_set_shadows_anything(self): - self._shadows_anything = True - class WaryDictImplementation(StrDictImplementation): def __init__(self, space): StrDictImplementation.__init__(self, space) self.shadowed = [None] * len(BUILTIN_TO_INDEX) - def impl_setitem_str(self, key, w_value, shadows_type=True): + def impl_setitem_str(self, key, w_value): i = BUILTIN_TO_INDEX.get(key, -1) if i != -1: self.shadowed[i] = w_value @@ -558,7 +514,7 @@ self.info.writes += 1 self.content[w_key] = w_value self.info.maxcontents = max(self.info.maxcontents, len(self.content)) - def impl_setitem_str(self, key, w_value, shadows_type=True): + def impl_setitem_str(self, key, w_value): self.info.setitem_strs += 1 self.impl_setitem(self.space.wrap(key), w_value) def impl_delitem(self, w_key): @@ -837,16 +793,11 @@ return w_item def dict_popitem__DictMulti(space, w_dict): - # XXX should somehow use the same trick as CPython: saving the index - # of the last popped item in the hash table, so that the next call to - # popitem() can be more efficient, instead of always starting from the - # beginning of the hash table. - iterator = w_dict.iter() - w_key, w_value = iterator.next() - if w_key is None: + try: + w_key, w_value = w_dict.popitem() + except KeyError: raise OperationError(space.w_KeyError, space.wrap("popitem(): dictionary is empty")) - w_dict.delitem(w_key) return space.newtuple([w_key, w_value]) app = gateway.applevel(''' Modified: pypy/branch/fast-forward/pypy/objspace/std/inttype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/inttype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/inttype.py Tue Nov 9 00:03:02 2010 @@ -2,7 +2,7 @@ from pypy.interpreter.error import OperationError from pypy.objspace.std.register_all import register_all from pypy.objspace.std.stdtypedef import StdTypeDef, SMM -from pypy.objspace.std.strutil import (string_to_int, string_to_w_long, +from pypy.objspace.std.strutil import (string_to_int, string_to_bigint, ParseStringError, ParseStringOverflowError) from pypy.rlib.rarithmetic import r_uint @@ -65,10 +65,12 @@ def retry_to_w_long(space, parser, base=0): parser.rewind() try: - return string_to_w_long(space, None, base=base, parser=parser) + bigint = string_to_bigint(None, base=base, parser=parser) except ParseStringError, e: raise OperationError(space.w_ValueError, space.wrap(e.msg)) + from pypy.objspace.std.longobject import W_LongObject + return W_LongObject(bigint) def descr__new__(space, w_inttype, w_x=0, w_base=gateway.NoneNotWrapped): from pypy.objspace.std.intobject import W_IntObject Modified: pypy/branch/fast-forward/pypy/objspace/std/listobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/listobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/listobject.py Tue Nov 9 00:03:02 2010 @@ -283,7 +283,7 @@ elif start >= 0: del items[start:start+delta] else: - assert delta==0 + assert delta==0 # start<0 is only possible with slicelength==0 elif len2 != slicelength: # No resize for extended slices raise operationerrfmt(space.w_ValueError, "attempt to " "assign sequence of size %d to extended slice of size %d", Modified: pypy/branch/fast-forward/pypy/objspace/std/longtype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/longtype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/longtype.py Tue Nov 9 00:03:02 2010 @@ -2,7 +2,7 @@ from pypy.interpreter import gateway, typedef from pypy.objspace.std.register_all import register_all from pypy.objspace.std.stdtypedef import StdTypeDef, SMM -from pypy.objspace.std.strutil import string_to_w_long, ParseStringError +from pypy.objspace.std.strutil import string_to_bigint, ParseStringError long_conjugate = SMM("conjugate", 1, doc="Returns self, the complex conjugate of any long.") @@ -18,10 +18,10 @@ if w_base is None: # check for easy cases if type(w_value) is W_LongObject: - pass + bigint = w_value.num elif space.is_true(space.isinstance(w_value, space.w_str)): try: - w_value = string_to_w_long(space, space.str_w(w_value)) + bigint = string_to_bigint(space.str_w(w_value)) except ParseStringError, e: raise OperationError(space.w_ValueError, space.wrap(e.msg)) @@ -31,7 +31,7 @@ from pypy.objspace.std.ropeunicodeobject import unicode_to_decimal_w else: from pypy.objspace.std.unicodeobject import unicode_to_decimal_w - w_value = string_to_w_long(space, unicode_to_decimal_w(space, w_value)) + bigint = string_to_bigint(unicode_to_decimal_w(space, w_value)) except ParseStringError, e: raise OperationError(space.w_ValueError, space.wrap(e.msg)) @@ -49,10 +49,11 @@ if space.is_true(space.isinstance(w_obj, space.w_long)): assert isinstance(w_obj, W_LongObject) # XXX this could fail! # XXX find a way to do that even if w_obj is not a W_LongObject - w_value = w_obj + bigint = w_obj.num elif space.is_true(space.isinstance(w_obj, space.w_int)): + from pypy.rlib.rbigint import rbigint intval = space.int_w(w_obj) - w_value = W_LongObject.fromint(space, intval) + bigint = rbigint.fromint(intval) else: raise OperationError(space.w_ValueError, space.wrap("value can't be converted to long")) @@ -70,13 +71,13 @@ space.wrap("long() can't convert non-string " "with explicit base")) try: - w_value = string_to_w_long(space, s, base) + bigint = string_to_bigint(s, base) except ParseStringError, e: raise OperationError(space.w_ValueError, space.wrap(e.msg)) w_obj = space.allocate_instance(W_LongObject, w_longtype) - W_LongObject.__init__(w_obj, w_value.num) + W_LongObject.__init__(w_obj, bigint) return w_obj def descr_get_numerator(space, w_obj): Modified: pypy/branch/fast-forward/pypy/objspace/std/mapdict.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/mapdict.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/mapdict.py Tue Nov 9 00:03:02 2010 @@ -1,4 +1,4 @@ -from pypy.rlib import jit, objectmodel +from pypy.rlib import jit, objectmodel, debug from pypy.interpreter.baseobjspace import W_Root from pypy.objspace.std.dictmultiobject import W_DictMultiObject @@ -10,6 +10,9 @@ # attribute shapes NUM_DIGITS = 4 +NUM_DIGITS_POW2 = 1 << NUM_DIGITS +# note: we use "x * NUM_DIGITS_POW2" instead of "x << NUM_DIGITS" because +# we want to propagate knowledge that the result cannot be negative class AbstractAttribute(object): _immutable_fields_ = ['w_cls'] @@ -69,8 +72,10 @@ attr = self._get_new_attr(selector[0], selector[1]) oldattr = obj._get_mapdict_map() if not jit.we_are_jitted(): - oldattr._size_estimate += attr.size_estimate() - oldattr.size_estimate() - assert oldattr.size_estimate() >= oldattr.length() + size_est = (oldattr._size_estimate + attr.size_estimate() + - oldattr.size_estimate()) + assert size_est >= (oldattr.length() * NUM_DIGITS_POW2) + oldattr._size_estimate = size_est if attr.length() > obj._mapdict_storage_length(): # note that attr.size_estimate() is always at least attr.length() new_storage = [None] * attr.size_estimate() @@ -188,7 +193,7 @@ self.selector = selector self.position = back.length() self.back = back - self._size_estimate = self.length() << NUM_DIGITS + self._size_estimate = self.length() * NUM_DIGITS_POW2 def _copy_attr(self, obj, new_obj): w_value = self.read(obj, self.selector) @@ -291,7 +296,7 @@ def getdictvalue(self, space, attrname): return self._get_mapdict_map().read(self, (attrname, DICT)) - def setdictvalue(self, space, attrname, w_value, shadows_type=True): + def setdictvalue(self, space, attrname, w_value): return self._get_mapdict_map().write(self, (attrname, DICT), w_value) def deldictvalue(self, space, w_name): @@ -356,7 +361,8 @@ def setweakref(self, space, weakreflifeline): from pypy.module._weakref.interp__weakref import WeakrefLifeline - assert isinstance(weakreflifeline, WeakrefLifeline) + assert (isinstance(weakreflifeline, WeakrefLifeline) or + weakreflifeline is None) self._get_mapdict_map().write(self, ("weakref", SPECIAL), weakreflifeline) class ObjectMixin(object): @@ -383,16 +389,18 @@ assert space.config.objspace.std.withmapdict map = w_type.terminator classes = memo_get_subclass_of_correct_size(space, cls) + if SUBCLASSES_MIN_FIELDS == SUBCLASSES_MAX_FIELDS: + return classes[0] size = map.size_estimate() - if not size: - size = 1 - try: - return classes[size - 1] - except IndexError: - return classes[-1] + debug.check_nonneg(size) + if size < len(classes): + return classes[size] + else: + return classes[len(classes)-1] get_subclass_of_correct_size._annspecialcase_ = "specialize:arg(1)" -NUM_SUBCLASSES = 10 # XXX tweak this number +SUBCLASSES_MIN_FIELDS = 5 # XXX tweak these numbers +SUBCLASSES_MAX_FIELDS = 5 def memo_get_subclass_of_correct_size(space, supercls): key = space, supercls @@ -401,8 +409,12 @@ except KeyError: assert not hasattr(supercls, "__del__") result = [] - for i in range(1, NUM_SUBCLASSES+1): + for i in range(SUBCLASSES_MIN_FIELDS, SUBCLASSES_MAX_FIELDS+1): result.append(_make_subclass_size_n(supercls, i)) + for i in range(SUBCLASSES_MIN_FIELDS): + result.insert(0, result[0]) + if SUBCLASSES_MIN_FIELDS == SUBCLASSES_MAX_FIELDS: + assert len(set(result)) == 1 _subclass_cache[key] = result return result memo_get_subclass_of_correct_size._annspecialcase_ = "specialize:memo" @@ -413,7 +425,7 @@ rangen = unroll.unrolling_iterable(range(n)) nmin1 = n - 1 rangenmin1 = unroll.unrolling_iterable(range(nmin1)) - class subcls(ObjectMixin, BaseMapdictObject, supercls): + class subcls(BaseMapdictObject, supercls): def _init_empty(self, map): from pypy.rlib.debug import make_sure_not_resized for i in rangen: @@ -506,8 +518,8 @@ def impl_getitem_str(self, key): return self.w_obj.getdictvalue(self.space, key) - def impl_setitem_str(self, key, w_value, shadows_type=True): - flag = self.w_obj.setdictvalue(self.space, key, w_value, shadows_type) + def impl_setitem_str(self, key, w_value): + flag = self.w_obj.setdictvalue(self.space, key, w_value) assert flag def impl_setitem(self, w_key, w_value): @@ -588,8 +600,6 @@ # ____________________________________________________________ # Magic caching -# XXX we also would like getdictvalue_attr_is_in_class() above - class CacheEntry(object): map = None version_tag = None Modified: pypy/branch/fast-forward/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/objspace.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/objspace.py Tue Nov 9 00:03:02 2010 @@ -437,7 +437,7 @@ if is_data: w_get = self.lookup(w_descr, "__get__") if w_get is None: - w_value = w_obj.getdictvalue_attr_is_in_class(self, name) + w_value = w_obj.getdictvalue(self, name) if w_value is not None: return w_value if not is_data: @@ -489,14 +489,12 @@ return w_obj.getitem(w_key) return ObjSpace.finditem(self, w_obj, w_key) - def setitem_str(self, w_obj, key, w_value, shadows_type=True): + def setitem_str(self, w_obj, key, w_value): """ Same as setitem, but takes string instead of any wrapped object - - XXX what shadows_type means??? """ if (isinstance(w_obj, W_DictMultiObject) and not w_obj.user_overridden_class): - w_obj.setitem_str(key, w_value, shadows_type) + w_obj.setitem_str(key, w_value) else: self.setitem(w_obj, self.wrap(key), w_value) Modified: pypy/branch/fast-forward/pypy/objspace/std/proxyobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/proxyobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/proxyobject.py Tue Nov 9 00:03:02 2010 @@ -43,7 +43,7 @@ raise return None - def setdictvalue(self, space, attr, w_value, shadows_type=True): + def setdictvalue(self, space, attr, w_value): try: space.call_function(self.w_controller, space.wrap('__setattr__'), space.wrap(attr), w_value) Modified: pypy/branch/fast-forward/pypy/objspace/std/smallintobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/smallintobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/smallintobject.py Tue Nov 9 00:03:02 2010 @@ -2,18 +2,13 @@ Implementation of small ints, stored as odd-valued pointers in the translated PyPy. To enable them, see inttype.py. """ -import types -from pypy.interpreter.error import OperationError from pypy.objspace.std import intobject from pypy.objspace.std.model import registerimplementation, W_Object from pypy.objspace.std.register_all import register_all -from pypy.objspace.std.multimethod import FailedToImplementArgs from pypy.objspace.std.noneobject import W_NoneObject -from pypy.rlib.rarithmetic import ovfcheck, ovfcheck_lshift, LONG_BIT, r_uint -from pypy.objspace.std.inttype import wrapint -from pypy.objspace.std.intobject import W_IntObject, _impl_int_int_pow +from pypy.objspace.std.intobject import W_IntObject from pypy.rlib.objectmodel import UnboxedValue -from pypy.rlib.rbigint import rbigint +from pypy.tool.sourcetools import func_with_new_name class W_SmallIntObject(W_Object, UnboxedValue): @@ -46,9 +41,7 @@ new_name = name.replace("Int", "SmallInt") # Copy the function, so the annotator specializes it for # W_SmallIntObject. - ns[new_name] = types.FunctionType(func.func_code, ns, new_name, - func.func_defaults, - func.func_closure) + ns[new_name] = func_with_new_name(func, new_name) ns["get_integer"] = ns["pos__SmallInt"] = ns["int__SmallInt"] ns["get_negint"] = ns["neg__SmallInt"] Modified: pypy/branch/fast-forward/pypy/objspace/std/stringobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/stringobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/stringobject.py Tue Nov 9 00:03:02 2010 @@ -942,10 +942,17 @@ string = w_string._value chars = [] - for char in string: - w_char = W_StringObject.PREBUILT[ord(char)] - if not space.is_true(space.contains(w_deletechars, w_char)): - chars.append(table[ord(char)]) + deletechars = space.str_w(w_deletechars) + if len(deletechars) == 0: + for char in string: + chars.append(table[ord(char)]) + else: + deletion_table = [False] * 256 + for c in deletechars: + deletion_table[ord(c)] = True + for char in string: + if not deletion_table[ord(char)]: + chars.append(table[ord(char)]) return W_StringObject(''.join(chars)) def str_decode__String_ANY_ANY(space, w_string, w_encoding=None, w_errors=None): Modified: pypy/branch/fast-forward/pypy/objspace/std/strutil.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/strutil.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/strutil.py Tue Nov 9 00:03:02 2010 @@ -4,6 +4,7 @@ from pypy.rlib.rarithmetic import ovfcheck, break_up_float, parts_to_float,\ INFINITY, NAN +from pypy.rlib.rbigint import rbigint, parse_digit_string from pypy.interpreter.error import OperationError import math @@ -97,9 +98,10 @@ return -1 def string_to_int(s, base=10): - """Utility to converts a string to an integer (or possibly a long). + """Utility to converts a string to an integer. If base is 0, the proper base is guessed based on the leading characters of 's'. Raises ParseStringError in case of error. + Raises ParseStringOverflowError in case the result does not fit. """ s = literal = strip_spaces(s) p = NumberStringParser(s, literal, base, 'int') @@ -119,11 +121,9 @@ except OverflowError: raise ParseStringOverflowError(p) -def string_to_long(space, s, base=10, parser=None): - return string_to_w_long(space, s, base, parser).longval() - -def string_to_w_long(space, s, base=10, parser=None): - """As string_to_int(), but ignores an optional 'l' or 'L' suffix.""" +def string_to_bigint(s, base=10, parser=None): + """As string_to_int(), but ignores an optional 'l' or 'L' suffix + and returns an rbigint.""" if parser is None: s = literal = strip_spaces(s) if (s.endswith('l') or s.endswith('L')) and base < 22: @@ -132,18 +132,7 @@ p = NumberStringParser(s, literal, base, 'long') else: p = parser - w_base = space.newlong(p.base) - w_result = space.newlong(0) - while True: - digit = p.next_digit() - if digit == -1: - if p.sign == -1: - w_result = space.neg(w_result) - # XXX grumble - from pypy.objspace.std.longobject import W_LongObject - assert isinstance(w_result, W_LongObject) - return w_result - w_result = space.add(space.mul(w_result,w_base), space.newlong(digit)) + return parse_digit_string(p) # Tim's comment: # 57 bits are more than needed in any case. Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py Tue Nov 9 00:03:02 2010 @@ -4,7 +4,6 @@ StrDictImplementation from pypy.objspace.std.celldict import ModuleDictImplementation -from pypy.objspace.std.sharingdict import SharedDictImplementation from pypy.conftest import gettestobjspace @@ -239,7 +238,16 @@ assert len(d) == 0 assert (it!=it1) and (it1==(1,2) or it1==(3,4)) raises(KeyError, d.popitem) - + + def test_popitem_2(self): + class A(object): + pass + d = A().__dict__ + d['x'] = 5 + it1 = d.popitem() + assert it1 == ('x', 5) + raises(KeyError, d.popitem) + def test_setdefault(self): d = {1:2, 3:4} dd = d.copy() @@ -513,32 +521,6 @@ assert getattr(a, s) == 42 -class TestW_DictSharing(TestW_DictObject): - def setup_class(cls): - cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True}) - -class AppTest_DictSharing(AppTest_DictObject): - def setup_class(cls): - cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True}) - - def test_values_does_not_share(self): - class A(object): - pass - a = A() - a.abc = 12 - l = a.__dict__.values() - assert l == [12] - l[0] = 24 - assert a.abc == 12 - - def test_items(self): - class A(object): - pass - a = A() - a.abc = 12 - a.__dict__.items() == [("abc", 12)] - - class AppTestModuleDict(object): def setup_class(cls): cls.space = gettestobjspace(**{"objspace.std.withcelldict": True}) @@ -632,10 +614,8 @@ class objspace: class std: withdictmeasurement = False - withsharingdict = False withsmalldicts = False withcelldict = False - withshadowtracking = False class opcodes: CALL_LIKELY_BUILTIN = False @@ -770,9 +750,6 @@ string = "int" string2 = "isinstance" -class TestSharedDictImplementation(BaseTestRDictImplementation): - ImplementionClass = SharedDictImplementation - class BaseTestDevolvedDictImplementation(BaseTestRDictImplementation): def fill_impl(self): @@ -794,8 +771,6 @@ string = "int" string2 = "isinstance" -class TestDevolvedSharedDictImplementation(BaseTestDevolvedDictImplementation): - ImplementionClass = SharedDictImplementation def test_module_uses_strdict(): fakespace = FakeSpace() Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_longobject.py Tue Nov 9 00:03:02 2010 @@ -47,22 +47,28 @@ assert math.trunc(-1L) == -1L def test_add(self): - assert int(123L + 12443L) == 123 + 12443 - assert -20 + 2 + 3L + True == -14L + x = 123L + assert int(x + 12443L) == 123 + 12443 + x = -20 + assert x + 2 + 3L + True == -14L def test_sub(self): - assert int(58543L - 12332L) == 58543 - 12332 - assert 237123838281233L * 12 == 237123838281233L * 12L + x = 58543L + assert int(x - 12332L) == 58543 - 12332 + x = 237123838281233L + assert x * 12 == x * 12L def test_mul(self): - assert 363L * 2 ** 40 == 363L << 40 + x = 363L + assert x * 2 ** 40 == x << 40 def test_truediv(self): exec "from __future__ import division; a = 31415926L / 10000000L" assert a == 3.1415926 def test_floordiv(self): - a = 31415926L // 10000000L + x = 31415926L + a = x // 10000000L assert a == 3L def test_numerator_denominator(self): @@ -72,37 +78,39 @@ assert (42L).denominator == 1L def test_compare(self): + Z = 0 + ZL = 0L for BIG in (1L, 1L << 62, 1L << 9999): - assert 0 == 0L - assert not (0 != 0L) - assert 0L == 0 - assert not (0L != 0) - assert not (0 == BIG) - assert 0 != BIG - assert not (BIG == 0) - assert BIG != 0 - assert not (0L == BIG) - assert 0L != BIG - assert 0 <= 0L - assert not (0 < 0L) - assert 0 <= BIG - assert 0 < BIG - assert not (BIG <= 0) - assert not (BIG < 0) - assert 0L <= 0L - assert not (0L < 0L) - assert 0L <= BIG - assert 0L < BIG - assert not (BIG <= 0L) - assert not (BIG < 0L) - assert not (0 <= -BIG) - assert not (0 < -BIG) - assert -BIG <= 0 - assert -BIG < 0 - assert not (0L <= -BIG) - assert not (0L < -BIG) - assert -BIG <= 0L - assert -BIG < 0L + assert Z == ZL + assert not (Z != ZL) + assert ZL == Z + assert not (ZL != Z) + assert not (Z == BIG) + assert Z != BIG + assert not (BIG == Z) + assert BIG != Z + assert not (ZL == BIG) + assert ZL != BIG + assert Z <= ZL + assert not (Z < ZL) + assert Z <= BIG + assert Z < BIG + assert not (BIG <= Z) + assert not (BIG < Z) + assert ZL <= ZL + assert not (ZL < ZL) + assert ZL <= BIG + assert ZL < BIG + assert not (BIG <= ZL) + assert not (BIG < ZL) + assert not (Z <= -BIG) + assert not (Z < -BIG) + assert -BIG <= Z + assert -BIG < Z + assert not (ZL <= -BIG) + assert not (ZL < -BIG) + assert -BIG <= ZL + assert -BIG < ZL # assert not (BIG < int(BIG)) assert (BIG <= int(BIG)) @@ -149,7 +157,8 @@ def test_conversion(self): class long2(long): pass - x = long2(1L<<100) + x = 1L + x = long2(x<<100) y = int(x) assert type(y) == long assert type(+long2(5)) is long @@ -164,7 +173,8 @@ assert type(long2(5) // 1) is long def test_pow(self): - assert pow(0L, 0L, 1L) == 0L + x = 0L + assert pow(x, 0L, 1L) == 0L def test_getnewargs(self): assert 0L .__getnewargs__() == (0L,) @@ -205,13 +215,14 @@ assert oct(01234567012345670L) == '01234567012345670L' def test_bits(self): - assert 0xAAAAAAAAL | 0x55555555L == 0xFFFFFFFFL - assert 0xAAAAAAAAL & 0x55555555L == 0x00000000L - assert 0xAAAAAAAAL ^ 0x55555555L == 0xFFFFFFFFL - assert -0xAAAAAAAAL | 0x55555555L == -0xAAAAAAA9L - assert 0xAAAAAAAAL | 0x555555555L == 0x5FFFFFFFFL - assert 0xAAAAAAAAL & 0x555555555L == 0x000000000L - assert 0xAAAAAAAAL ^ 0x555555555L == 0x5FFFFFFFFL + x = 0xAAAAAAAAL + assert x | 0x55555555L == 0xFFFFFFFFL + assert x & 0x55555555L == 0x00000000L + assert x ^ 0x55555555L == 0xFFFFFFFFL + assert -x | 0x55555555L == -0xAAAAAAA9L + assert x | 0x555555555L == 0x5FFFFFFFFL + assert x & 0x555555555L == 0x000000000L + assert x ^ 0x555555555L == 0x5FFFFFFFFL def test_hash(self): # ints have the same hash as equal longs @@ -241,7 +252,8 @@ def test_huge_longs(self): import operator - huge = 1L << 40000L + x = 1L + huge = x << 40000L raises(OverflowError, float, huge) raises(OverflowError, operator.truediv, huge, 3) raises(OverflowError, operator.truediv, huge, 3L) @@ -260,17 +272,23 @@ class myotherlong(long): pass assert long(myotherlong(21)) == 21L - + def test_conjugate(self): assert (7L).conjugate() == 7L assert (-7L).conjugate() == -7L - + class L(long): pass - + assert type(L(7).conjugate()) is long def test_bit_length(self): assert 8L.bit_length() == 4 assert (-1<<40).bit_length() == 41 assert ((2**31)-1).bit_length() == 31 + + + def test_negative_zero(self): + x = eval("-0L") + assert x == 0L + Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py Tue Nov 9 00:03:02 2010 @@ -160,6 +160,13 @@ assert obj.getweakref() is lifeline1 assert obj.getdictvalue(space, "weakref") == 41 + lifeline1 = WeakrefLifeline(space) + obj = c.instantiate() + assert obj.getweakref() is None + obj.setweakref(space, lifeline1) + obj.setweakref(space, None) + + def test_slots(): cls = Class() @@ -777,3 +784,19 @@ assert res == (0, 0, 1) res = self.check(f, 'x') assert res == (0, 0, 1) + +class TestDictSubclassShortcutBug(object): + def setup_class(cls): + cls.space = gettestobjspace( + **{"objspace.std.withmapdict": True, + "objspace.std.withmethodcachecounter": True}) + + def test_bug(self): + w_dict = self.space.appexec([], """(): + class A(dict): + def __getitem__(self, key): + return 1 + assert eval("a", globals(), A()) == 1 + return A() + """) + assert w_dict.user_overridden_class Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_strutil.py Tue Nov 9 00:03:02 2010 @@ -25,7 +25,7 @@ ] for s, expected in cases: assert string_to_int(s) == expected - assert string_to_w_long(space, s).longval() == expected + assert string_to_bigint(s).tolong() == expected def test_string_to_int_base(self): space = self.space @@ -122,17 +122,16 @@ raises(ParseStringError, string_to_int, '+'+s, base) raises(ParseStringError, string_to_int, '-'+s, base) - def test_string_to_w_long(self): - space = self.space - assert string_to_w_long(space, '123L').longval() == 123 - assert string_to_w_long(space, '123L ').longval() == 123 - raises(ParseStringError, string_to_w_long, space, 'L') - raises(ParseStringError, string_to_w_long, space, 'L ') - assert string_to_w_long(space, '123L', 4).longval() == 27 - assert string_to_w_long(space, '123L', 30).longval() == 27000 + 1800 + 90 + 21 - assert string_to_w_long(space, '123L', 22).longval() == 10648 + 968 + 66 + 21 - assert string_to_w_long(space, '123L', 21).longval() == 441 + 42 + 3 - assert string_to_w_long(space, '1891234174197319').longval() == 1891234174197319 + def test_string_to_bigint(self): + assert string_to_bigint('123L').tolong() == 123 + assert string_to_bigint('123L ').tolong() == 123 + raises(ParseStringError, string_to_bigint, 'L') + raises(ParseStringError, string_to_bigint, 'L ') + assert string_to_bigint('123L', 4).tolong() == 27 + assert string_to_bigint('123L', 30).tolong() == 27000 + 1800 + 90 + 21 + assert string_to_bigint('123L', 22).tolong() == 10648 + 968 + 66 + 21 + assert string_to_bigint('123L', 21).tolong() == 441 + 42 + 3 + assert string_to_bigint('1891234174197319').tolong() == 1891234174197319 def test_string_to_float(self): def string_to_float(x): Modified: pypy/branch/fast-forward/pypy/objspace/std/typeobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/typeobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/typeobject.py Tue Nov 9 00:03:02 2010 @@ -61,6 +61,15 @@ self.hits = {} self.misses = {} + def clear(self): + None_None = (None, None) + for i in range(len(self.versions)): + self.versions[i] = None + for i in range(len(self.names)): + self.names[i] = None + for i in range(len(self.lookup_where)): + self.lookup_where[i] = None_None + class W_TypeObject(W_Object): from pypy.objspace.std.typetype import type_typedef as typedef Modified: pypy/branch/fast-forward/pypy/rlib/jit.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/jit.py (original) +++ pypy/branch/fast-forward/pypy/rlib/jit.py Tue Nov 9 00:03:02 2010 @@ -4,6 +4,7 @@ from pypy.rlib.objectmodel import CDefinedIntSymbolic from pypy.rlib.objectmodel import keepalive_until_here from pypy.rlib.unroll import unrolling_iterable +from pypy.rlib.nonconst import NonConstant def purefunction(func): """ Decorate a function as pure. Pure means precisely that: @@ -145,6 +146,14 @@ return hop.inputconst(lltype.Signed, _we_are_jitted) +def current_trace_length(): + """During JIT tracing, returns the current trace length (as a constant). + If not tracing, returns -1.""" + if NonConstant(False): + return 73 + return -1 +current_trace_length.oopspec = 'jit.current_trace_length()' + def jit_debug(string, arg1=-sys.maxint-1, arg2=-sys.maxint-1, arg3=-sys.maxint-1, arg4=-sys.maxint-1): """When JITted, cause an extra operation DEBUG_MERGE_POINT to appear in Modified: pypy/branch/fast-forward/pypy/rlib/objectmodel.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/objectmodel.py (original) +++ pypy/branch/fast-forward/pypy/rlib/objectmodel.py Tue Nov 9 00:03:02 2010 @@ -475,6 +475,10 @@ def setdefault(self, key, default): return self._dict.setdefault(_r_dictkey(self, key), default) + def popitem(self): + dk, value = self._dict.popitem() + return dk.key, value + def copy(self): result = r_dict(self.key_eq, self.key_hash) result.update(self) Modified: pypy/branch/fast-forward/pypy/rlib/rbigint.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rbigint.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rbigint.py Tue Nov 9 00:03:02 2010 @@ -1610,11 +1610,16 @@ # a few internal helpers -DEC_PER_DIGIT = 1 -while int('9' * DEC_PER_DIGIT) < MASK: - DEC_PER_DIGIT += 1 -DEC_PER_DIGIT -= 1 -DEC_MAX = 10 ** DEC_PER_DIGIT +def digits_max_for_base(base): + dec_per_digit = 1 + while base ** dec_per_digit < MASK: + dec_per_digit += 1 + dec_per_digit -= 1 + return base ** dec_per_digit + +BASE_MAX = [0, 0] + [digits_max_for_base(_base) for _base in range(2, 37)] +DEC_MAX = digits_max_for_base(10) +assert DEC_MAX == BASE_MAX[10] def _decimalstr_to_bigint(s): # a string that has been already parsed to be decimal and valid, @@ -1629,7 +1634,6 @@ p += 1 a = rbigint.fromint(0) - cnt = DEC_PER_DIGIT tens = 1 dig = 0 ord0 = ord('0') @@ -1641,8 +1645,26 @@ a = _muladd1(a, tens, dig) tens = 1 dig = 0 - if sign: + if sign and a.sign == 1: a.sign = -1 return a - +def parse_digit_string(parser): + # helper for objspace.std.strutil + a = rbigint.fromint(0) + base = parser.base + digitmax = BASE_MAX[base] + tens, dig = 1, 0 + while True: + digit = parser.next_digit() + if tens == digitmax or digit < 0: + a = _muladd1(a, tens, dig) + if digit < 0: + break + dig = digit + tens = base + else: + dig = dig * base + digit + tens *= base + a.sign *= parser.sign + return a Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rbigint.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/test/test_rbigint.py (original) +++ pypy/branch/fast-forward/pypy/rlib/test/test_rbigint.py Tue Nov 9 00:03:02 2010 @@ -112,6 +112,23 @@ assert rbigint.fromrarith_int(r_uint(2*sys.maxint+1)).eq( rbigint.fromlong(2*sys.maxint+1)) + def test_fromdecimalstr(self): + x = rbigint.fromdecimalstr("12345678901234567890523897987") + assert x.tolong() == 12345678901234567890523897987L + assert x.tobool() is True + x = rbigint.fromdecimalstr("+12345678901234567890523897987") + assert x.tolong() == 12345678901234567890523897987L + assert x.tobool() is True + x = rbigint.fromdecimalstr("-12345678901234567890523897987") + assert x.tolong() == -12345678901234567890523897987L + assert x.tobool() is True + x = rbigint.fromdecimalstr("+0") + assert x.tolong() == 0 + assert x.tobool() is False + x = rbigint.fromdecimalstr("-0") + assert x.tolong() == 0 + assert x.tobool() is False + def test_add(self): x = 123456789123456789000000L y = 123858582373821923936744221L @@ -458,6 +475,35 @@ assert (rbigint.fromlong(-9**50).ulonglongmask() == r_ulonglong(-9**50)) + def test_parse_digit_string(self): + from pypy.rlib.rbigint import parse_digit_string + class Parser: + def __init__(self, base, sign, digits): + self.base = base + self.sign = sign + self.next_digit = iter(digits + [-1]).next + x = parse_digit_string(Parser(10, 1, [6])) + assert x.eq(rbigint.fromint(6)) + x = parse_digit_string(Parser(10, 1, [6, 2, 3])) + assert x.eq(rbigint.fromint(623)) + x = parse_digit_string(Parser(10, -1, [6, 2, 3])) + assert x.eq(rbigint.fromint(-623)) + x = parse_digit_string(Parser(16, 1, [0xA, 0x4, 0xF])) + assert x.eq(rbigint.fromint(0xA4F)) + num = 0 + for i in range(36): + x = parse_digit_string(Parser(36, 1, range(i))) + assert x.eq(rbigint.fromlong(num)) + num = num * 36 + i + x = parse_digit_string(Parser(16, -1, range(15,-1,-1)*99)) + assert x.eq(rbigint.fromlong(long('-0x' + 'FEDCBA9876543210'*99, 16))) + assert x.tobool() is True + x = parse_digit_string(Parser(7, 1, [0, 0, 0])) + assert x.tobool() is False + x = parse_digit_string(Parser(7, -1, [0, 0, 0])) + assert x.tobool() is False + + BASE = 2 ** SHIFT class TestTranslatable(object): Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/rdict.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/lltypesystem/rdict.py (original) +++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/rdict.py Tue Nov 9 00:03:02 2010 @@ -306,6 +306,13 @@ hop.exception_cannot_occur() return hop.gendirectcall(ll_clear, v_dict) + def rtype_method_popitem(self, hop): + v_dict, = hop.inputargs(self) + r_tuple = hop.r_result + cTUPLE = hop.inputconst(lltype.Void, r_tuple.lowleveltype) + hop.exception_is_here() + return hop.gendirectcall(ll_popitem, cTUPLE, v_dict) + class __extend__(pairtype(DictRepr, rmodel.Repr)): def rtype_getitem((r_dict, r_key), hop): @@ -465,6 +472,10 @@ i = ll_dict_lookup(d, key, d.keyhash(key)) if not d.entries.valid(i): raise KeyError + _ll_dict_del(d, i) +ll_dict_delitem.oopspec = 'dict.delitem(d, key)' + +def _ll_dict_del(d, i): d.entries.mark_deleted(i) d.num_items -= 1 # clear the key and the value if they are GC pointers @@ -481,7 +492,6 @@ num_entries = len(d.entries) if num_entries > DICT_INITSIZE and d.num_items < num_entries / 4: ll_dict_resize(d) -ll_dict_delitem.oopspec = 'dict.delitem(d, key)' def ll_dict_resize(d): old_entries = d.entries @@ -810,3 +820,26 @@ i = ll_dict_lookup(d, key, d.keyhash(key)) return d.entries.valid(i) ll_contains.oopspec = 'dict.contains(d, key)' + +POPITEMINDEX = lltype.Struct('PopItemIndex', ('nextindex', lltype.Signed)) +global_popitem_index = lltype.malloc(POPITEMINDEX, zero=True, immortal=True) + +def ll_popitem(ELEM, dic): + entries = dic.entries + dmask = len(entries) - 1 + base = global_popitem_index.nextindex + counter = 0 + while counter <= dmask: + i = (base + counter) & dmask + counter += 1 + if entries.valid(i): + break + else: + raise KeyError + global_popitem_index.nextindex += counter + entry = entries[i] + r = lltype.malloc(ELEM.TO) + r.item0 = recast(ELEM.TO.item0, entry.key) + r.item1 = recast(ELEM.TO.item1, entry.value) + _ll_dict_del(dic, i) + return r Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py Tue Nov 9 00:03:02 2010 @@ -178,6 +178,7 @@ self.nursery = NULL self.nursery_free = NULL self.nursery_top = NULL + self.debug_always_do_minor_collect = False # # The ArenaCollection() handles the nonmovable objects allocation. if ArenaCollectionClass is None: @@ -245,6 +246,10 @@ # From there on, the GC is fully initialized and the code # below can use it newsize = base.read_from_env('PYPY_GC_NURSERY') + # PYPY_GC_NURSERY=1 forces a minor collect for every malloc. + # Useful to debug external factors, like trackgcroot or the + # handling of the write barrier. + self.debug_always_do_minor_collect = newsize == 1 if newsize <= 0: newsize = generation.estimate_best_nursery_size() if newsize <= 0: @@ -444,6 +449,10 @@ result = self.nursery_free self.nursery_free = result + totalsize ll_assert(self.nursery_free <= self.nursery_top, "nursery overflow") + # + if self.debug_always_do_minor_collect: + self.nursery_free = self.nursery_top + # return result collect_and_reserve._dont_inline_ = True Modified: pypy/branch/fast-forward/pypy/rpython/test/test_rdict.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/test/test_rdict.py (original) +++ pypy/branch/fast-forward/pypy/rpython/test/test_rdict.py Tue Nov 9 00:03:02 2010 @@ -682,6 +682,26 @@ # if it does not crash, we are fine. It crashes if you forget the hash field. self.interpret(func, []) + def test_dict_popitem(self): + def func(): + d = {} + d[5] = 2 + d[6] = 3 + k1, v1 = d.popitem() + assert len(d) == 1 + k2, v2 = d.popitem() + try: + d.popitem() + except KeyError: + pass + else: + assert 0, "should have raised KeyError" + assert len(d) == 0 + return k1*1000 + v1*100 + k2*10 + v2 + + res = self.interpret(func, []) + assert res in [5263, 6352] + # ____________________________________________________________ def test_opt_nullkeymarker(self): Modified: pypy/branch/fast-forward/pypy/tool/alarm.py ============================================================================== --- pypy/branch/fast-forward/pypy/tool/alarm.py (original) +++ pypy/branch/fast-forward/pypy/tool/alarm.py Tue Nov 9 00:03:02 2010 @@ -38,8 +38,9 @@ sys.path.insert(0, os.path.dirname(sys.argv[0])) return sys.argv[0] -finished = [] -try: - execfile(_main_with_alarm(finished)) -finally: - finished.append(True) +if __name__ == '__main__': + finished = [] + try: + execfile(_main_with_alarm(finished)) + finally: + finished.append(True) Modified: pypy/branch/fast-forward/pypy/tool/readdictinfo.py ============================================================================== --- pypy/branch/fast-forward/pypy/tool/readdictinfo.py (original) +++ pypy/branch/fast-forward/pypy/tool/readdictinfo.py Tue Nov 9 00:03:02 2010 @@ -7,37 +7,38 @@ import sys -infile = open(sys.argv[1]) +if __name__ == '__main__': + infile = open(sys.argv[1]) -curr = None -slots = [] -for line in infile: - if line == '------------------\n': - if curr: - break - curr = 1 - else: - attr, val = [s.strip() for s in line.split(':')] - slots.append(attr) + curr = None + slots = [] + for line in infile: + if line == '------------------\n': + if curr: + break + curr = 1 + else: + attr, val = [s.strip() for s in line.split(':')] + slots.append(attr) -class DictInfo(object): - __slots__ = slots + class DictInfo(object): + __slots__ = slots -infile = open(sys.argv[1]) + infile = open(sys.argv[1]) -infos = [] + infos = [] -for line in infile: - if line == '------------------\n': - curr = object.__new__(DictInfo) - infos.append(curr) - else: - attr, val = [s.strip() for s in line.split(':')] - if '.' in val: - val = float(val) + for line in infile: + if line == '------------------\n': + curr = object.__new__(DictInfo) + infos.append(curr) else: - val = int(val) - setattr(curr, attr, val) + attr, val = [s.strip() for s in line.split(':')] + if '.' in val: + val = float(val) + else: + val = int(val) + setattr(curr, attr, val) def histogram(infos, keyattr, *attrs): r = {} Modified: pypy/branch/fast-forward/pypy/tool/rundictbenchmarks.py ============================================================================== --- pypy/branch/fast-forward/pypy/tool/rundictbenchmarks.py (original) +++ pypy/branch/fast-forward/pypy/tool/rundictbenchmarks.py Tue Nov 9 00:03:02 2010 @@ -7,20 +7,21 @@ # need to hack a copy of rst2html for yourself (svn docutils # required). -try: - os.unlink("dictinfo.txt") -except os.error: - pass +if __name__ == '__main__': + try: + os.unlink("dictinfo.txt") + except os.error: + pass -progs = [('pystone', ['-c', 'from test import pystone; pystone.main()']), - ('richards', ['richards.py']), - ('docutils', ['rst2html.py', '../../doc/coding-guide.txt', 'foo.html']), - ('translate', ['translate.py', '--backendopt', '--no-compile', '--batch', - 'targetrpystonedalone.py']) - ] + progs = [('pystone', ['-c', 'from test import pystone; pystone.main()']), + ('richards', ['richards.py']), + ('docutils', ['rst2html.py', '../../doc/coding-guide.txt', 'foo.html']), + ('translate', ['translate.py', '--backendopt', '--no-compile', '--batch', + 'targetrpystonedalone.py']) + ] -EXE = sys.argv[1] + EXE = sys.argv[1] -for suffix, args in progs: - os.spawnv(os.P_WAIT, EXE, [EXE] + args) - os.rename('dictinfo.txt', 'dictinfo-%s.txt'%suffix) + for suffix, args in progs: + os.spawnv(os.P_WAIT, EXE, [EXE] + args) + os.rename('dictinfo.txt', 'dictinfo-%s.txt'%suffix) Modified: pypy/branch/fast-forward/pypy/tool/statistic_irc_log.py ============================================================================== --- pypy/branch/fast-forward/pypy/tool/statistic_irc_log.py (original) +++ pypy/branch/fast-forward/pypy/tool/statistic_irc_log.py Tue Nov 9 00:03:02 2010 @@ -2,53 +2,54 @@ from os import system, chdir from urllib import urlopen -log_URL = 'http://tismerysoft.de/pypy/irc-logs/' -archive_FILENAME = 'pypy.tar.gz' - -tempdir = py.test.ensuretemp("irc-log") - -# get compressed archive -chdir( str(tempdir)) -system('wget -q %s%s' % (log_URL, archive_FILENAME)) -system('tar xzf %s' % archive_FILENAME) -chdir('pypy') - -# get more recent daily logs -pypydir = tempdir.join('pypy') -for line in urlopen(log_URL + 'pypy/').readlines(): - i = line.find('%23pypy.log.') - if i == -1: - continue - filename = line[i:].split('"')[0] - system('wget -q %spypy/%s' % (log_URL, filename)) - -# rename to YYYYMMDD -for log_filename in pypydir.listdir('#pypy.log.*'): - rename_to = None - b = log_filename.basename - if '-' in b: - rename_to = log_filename.basename.replace('-', '') - elif len(b) == 19: - months= 'Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec'.split() - day = b[10:12] - month = months.index(b[12:15]) + 1 - year = b[15:20] - rename_to = '#pypy.log.%04s%02d%02s' % (year, month, day) - - if rename_to: - log_filename.rename(rename_to) - #print 'RENAMED', log_filename, 'TO', rename_to - -# print sorted list of filenames of daily logs -print 'irc://irc.freenode.org/pypy' -print 'date, messages, visitors' -for log_filename in pypydir.listdir('#pypy.log.*'): - n_messages, visitors = 0, {} - f = str(log_filename) - for s in file(f): - if '<' in s and '>' in s: - n_messages += 1 - elif ' joined #pypy' in s: - v = s.split()[1] - visitors[v] = True - print '%04s-%02s-%02s, %d, %d' % (f[-8:-4], f[-4:-2], f[-2:], n_messages, len(visitors.keys())) +if __name__ == '__main__': + log_URL = 'http://tismerysoft.de/pypy/irc-logs/' + archive_FILENAME = 'pypy.tar.gz' + + tempdir = py.test.ensuretemp("irc-log") + + # get compressed archive + chdir( str(tempdir)) + system('wget -q %s%s' % (log_URL, archive_FILENAME)) + system('tar xzf %s' % archive_FILENAME) + chdir('pypy') + + # get more recent daily logs + pypydir = tempdir.join('pypy') + for line in urlopen(log_URL + 'pypy/').readlines(): + i = line.find('%23pypy.log.') + if i == -1: + continue + filename = line[i:].split('"')[0] + system('wget -q %spypy/%s' % (log_URL, filename)) + + # rename to YYYYMMDD + for log_filename in pypydir.listdir('#pypy.log.*'): + rename_to = None + b = log_filename.basename + if '-' in b: + rename_to = log_filename.basename.replace('-', '') + elif len(b) == 19: + months= 'Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec'.split() + day = b[10:12] + month = months.index(b[12:15]) + 1 + year = b[15:20] + rename_to = '#pypy.log.%04s%02d%02s' % (year, month, day) + + if rename_to: + log_filename.rename(rename_to) + #print 'RENAMED', log_filename, 'TO', rename_to + + # print sorted list of filenames of daily logs + print 'irc://irc.freenode.org/pypy' + print 'date, messages, visitors' + for log_filename in pypydir.listdir('#pypy.log.*'): + n_messages, visitors = 0, {} + f = str(log_filename) + for s in file(f): + if '<' in s and '>' in s: + n_messages += 1 + elif ' joined #pypy' in s: + v = s.split()[1] + visitors[v] = True + print '%04s-%02s-%02s, %d, %d' % (f[-8:-4], f[-4:-2], f[-2:], n_messages, len(visitors.keys())) Modified: pypy/branch/fast-forward/pypy/tool/watchdog.py ============================================================================== --- pypy/branch/fast-forward/pypy/tool/watchdog.py (original) +++ pypy/branch/fast-forward/pypy/tool/watchdog.py Tue Nov 9 00:03:02 2010 @@ -7,9 +7,6 @@ return name return 'signal %d' % (n,) -timeout = float(sys.argv[1]) -timedout = False - def childkill(): global timedout timedout = True @@ -20,31 +17,35 @@ except OSError: pass -pid = os.fork() -if pid == 0: - os.execvp(sys.argv[2], sys.argv[2:]) -else: # parent - t = threading.Timer(timeout, childkill) - t.start() - while True: - try: - pid, status = os.waitpid(pid, 0) - except KeyboardInterrupt: - continue +if __name__ == '__main__': + timeout = float(sys.argv[1]) + timedout = False + + pid = os.fork() + if pid == 0: + os.execvp(sys.argv[2], sys.argv[2:]) + else: # parent + t = threading.Timer(timeout, childkill) + t.start() + while True: + try: + pid, status = os.waitpid(pid, 0) + except KeyboardInterrupt: + continue + else: + t.cancel() + break + if os.WIFEXITED(status): + sys.exit(os.WEXITSTATUS(status)) else: - t.cancel() - break - if os.WIFEXITED(status): - sys.exit(os.WEXITSTATUS(status)) - else: - assert os.WIFSIGNALED(status) - sign = os.WTERMSIG(status) - if timedout and sign == signal.SIGTERM: + assert os.WIFSIGNALED(status) + sign = os.WTERMSIG(status) + if timedout and sign == signal.SIGTERM: + sys.exit(1) + signame = getsignalname(sign) + sys.stderr.write("="*26 + "timedout" + "="*26 + "\n") + sys.stderr.write("="*25 + " %-08s " % signame + "="*25 + "\n") sys.exit(1) - signame = getsignalname(sign) - sys.stderr.write("="*26 + "timedout" + "="*26 + "\n") - sys.stderr.write("="*25 + " %-08s " % signame + "="*25 + "\n") - sys.exit(1) - - + + Modified: pypy/branch/fast-forward/pypy/tool/watchdog_nt.py ============================================================================== --- pypy/branch/fast-forward/pypy/tool/watchdog_nt.py (original) +++ pypy/branch/fast-forward/pypy/tool/watchdog_nt.py Tue Nov 9 00:03:02 2010 @@ -2,11 +2,6 @@ import threading import ctypes -PROCESS_TERMINATE = 0x1 - -timeout = float(sys.argv[1]) -timedout = False - def childkill(pid): global timedout timedout = True @@ -14,19 +9,25 @@ sys.stderr.write("="*26 + "timedout" + "="*26 + "\n") ctypes.windll.kernel32.TerminateProcess(pid, 1) -pid = os.spawnv(os.P_NOWAIT, sys.argv[2], sys.argv[2:]) +if __name__ == '__main__': + PROCESS_TERMINATE = 0x1 + + timeout = float(sys.argv[1]) + timedout = False + + pid = os.spawnv(os.P_NOWAIT, sys.argv[2], sys.argv[2:]) + + t = threading.Timer(timeout, childkill, (pid,)) + t.start() + while True: + try: + pid, status = os.waitpid(pid, 0) + except KeyboardInterrupt: + continue + else: + t.cancel() + break + + #print 'status ', status >> 8 + sys.exit(status >> 8) -t = threading.Timer(timeout, childkill, (pid,)) -t.start() -while True: - try: - pid, status = os.waitpid(pid, 0) - except KeyboardInterrupt: - continue - else: - t.cancel() - break - -#print 'status ', status >> 8 -sys.exit(status >> 8) - Modified: pypy/branch/fast-forward/pypy/translator/c/gcc/test/test_trackgcroot.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/gcc/test/test_trackgcroot.py (original) +++ pypy/branch/fast-forward/pypy/translator/c/gcc/test/test_trackgcroot.py Tue Nov 9 00:03:02 2010 @@ -109,7 +109,7 @@ def test_computegcmaptable(): tests = [] - for format in ('elf', 'darwin', 'msvc', 'elf64'): + for format in ('elf', 'elf64', 'darwin', 'darwin64', 'msvc'): for path in this_dir.join(format).listdir("track*.s"): n = path.purebasename[5:] try: Modified: pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py (original) +++ pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py Tue Nov 9 00:03:02 2010 @@ -455,7 +455,7 @@ 'inc', 'dec', 'not', 'neg', 'or', 'and', 'sbb', 'adc', 'shl', 'shr', 'sal', 'sar', 'rol', 'ror', 'mul', 'imul', 'div', 'idiv', 'bswap', 'bt', 'rdtsc', - 'punpck', 'pshufd', + 'punpck', 'pshufd', 'psll', # zero-extending moves should not produce GC pointers 'movz', ]) @@ -1090,7 +1090,7 @@ ElfFunctionGcRootTracker64.init_regexp() -class DarwinFunctionGcRootTracker(ElfFunctionGcRootTracker32): +class DarwinFunctionGcRootTracker32(ElfFunctionGcRootTracker32): format = 'darwin' function_names_prefix = '_' @@ -1102,7 +1102,22 @@ funcname = '_' + match.group(1) FunctionGcRootTracker32.__init__(self, funcname, lines, filetag) -class Mingw32FunctionGcRootTracker(DarwinFunctionGcRootTracker): +class DarwinFunctionGcRootTracker64(ElfFunctionGcRootTracker64): + format = 'darwin64' + function_names_prefix = '_' + + LABEL = ElfFunctionGcRootTracker32.LABEL + r_jmptable_item = re.compile(r"\t.(?:long|quad)\t"+LABEL+"(-\"?[A-Za-z0-9$]+\"?)?\s*$") + + r_functionstart = re.compile(r"_(\w+):\s*$") + OFFSET_LABELS = 0 + + def __init__(self, lines, filetag=0): + match = self.r_functionstart.match(lines[0]) + funcname = '_' + match.group(1) + FunctionGcRootTracker64.__init__(self, funcname, lines, filetag) + +class Mingw32FunctionGcRootTracker(DarwinFunctionGcRootTracker32): format = 'mingw32' function_names_prefix = '_' @@ -1373,7 +1388,7 @@ class DarwinAssemblerParser(AssemblerParser): format = "darwin" - FunctionGcRootTracker = DarwinFunctionGcRootTracker + FunctionGcRootTracker = DarwinFunctionGcRootTracker32 r_textstart = re.compile(r"\t.text\s*$") @@ -1419,6 +1434,10 @@ return super(DarwinAssemblerParser, self).process_function( lines, entrypoint, filename) +class DarwinAssemblerParser64(DarwinAssemblerParser): + format = "darwin64" + FunctionGcRootTracker = DarwinFunctionGcRootTracker64 + class Mingw32AssemblerParser(DarwinAssemblerParser): format = "mingw32" FunctionGcRootTracker = Mingw32FunctionGcRootTracker @@ -1542,6 +1561,7 @@ 'elf': ElfAssemblerParser, 'elf64': ElfAssemblerParser64, 'darwin': DarwinAssemblerParser, + 'darwin64': DarwinAssemblerParser64, 'mingw32': Mingw32AssemblerParser, 'msvc': MsvcAssemblerParser, } @@ -1579,7 +1599,7 @@ txt = kwargs[self.format] print >> output, "\t%s" % txt - if self.format == 'elf64': + if self.format in ('elf64', 'darwin64'): word_decl = '.quad' else: word_decl = '.long' @@ -1632,10 +1652,11 @@ } } """ - elif self.format == 'elf64': + elif self.format in ('elf64', 'darwin64'): print >> output, "\t.text" print >> output, "\t.globl %s" % _globalname('pypy_asm_stackwalk') - print >> output, "\t.type pypy_asm_stackwalk, @function" + _variant(elf64='.type pypy_asm_stackwalk, @function', + darwin64='') print >> output, "%s:" % _globalname('pypy_asm_stackwalk') print >> output, """\ @@ -1680,8 +1701,9 @@ /* the return value is the one of the 'call' above, */ /* because %rax (and possibly %rdx) are unmodified */ ret - .size pypy_asm_stackwalk, .-pypy_asm_stackwalk """ + _variant(elf64='.size pypy_asm_stackwalk, .-pypy_asm_stackwalk', + darwin64='') else: print >> output, "\t.text" print >> output, "\t.globl %s" % _globalname('pypy_asm_stackwalk') @@ -1808,6 +1830,7 @@ _variant(elf='.section\t.rodata', elf64='.section\t.rodata', darwin='.const', + darwin64='.const', mingw32='') print >> output, """\ @@ -1881,11 +1904,14 @@ if __name__ == '__main__': - verbose = 1 + verbose = 0 shuffle = False output_raw_table = False if sys.platform == 'darwin': - format = 'darwin' + if sys.maxint > 2147483647: + format = 'darwin64' + else: + format = 'darwin' elif sys.platform == 'win32': format = 'mingw32' else: Modified: pypy/branch/fast-forward/pypy/translator/c/genc.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/genc.py (original) +++ pypy/branch/fast-forward/pypy/translator/c/genc.py Tue Nov 9 00:03:02 2010 @@ -623,9 +623,16 @@ mk.definition('OBJECTS', '$(ASMLBLFILES) gcmaptable.s') mk.rule('%.s', '%.c', '$(CC) $(CFLAGS) $(CFLAGSEXTRA) -frandom-seed=$< -o $@ -S $< $(INCLUDEDIRS)') mk.rule('%.lbl.s %.gcmap', '%.s', - python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py -m$(PYPY_MAIN_FUNCTION) -t $< > $*.gcmap') + [python + + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py ' + '-m$(PYPY_MAIN_FUNCTION) -t $< > $*.gctmp', + 'mv $*.gctmp $*.gcmap']) mk.rule('gcmaptable.s', '$(GCMAPFILES)', - python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py $(GCMAPFILES) > $@') + [python + + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py ' + '$(GCMAPFILES) > $@.tmp', + 'mv $@.tmp $@']) + mk.rule('.PRECIOUS', '%.s', "# don't remove .s files if Ctrl-C'ed") else: mk.definition('DEBUGFLAGS', '-O1 -g') Modified: pypy/branch/fast-forward/pypy/translator/c/src/g_include.h ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/src/g_include.h (original) +++ pypy/branch/fast-forward/pypy/translator/c/src/g_include.h Tue Nov 9 00:03:02 2010 @@ -39,9 +39,10 @@ #include "src/instrument.h" /* optional assembler bits */ -#if defined(__GNUC__) && defined(__i386__) -# include "src/asm_gcc_x86.h" -#endif +// disabled: does not give any speed-up +//#if defined(__GNUC__) && defined(__i386__) +//# include "src/asm_gcc_x86.h" +//#endif #if defined(__GNUC__) && defined(__ppc__) # include "src/asm_ppc.h" Modified: pypy/branch/fast-forward/pypy/translator/c/src/int.h ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/src/int.h (original) +++ pypy/branch/fast-forward/pypy/translator/c/src/int.h Tue Nov 9 00:03:02 2010 @@ -2,40 +2,27 @@ /************************************************************/ /*** C header subsection: operations between ints ***/ -#ifndef LLONG_MAX -# if SIZEOF_LONG_LONG == 8 -# define LLONG_MAX 0X7FFFFFFFFFFFFFFFLL -# else -# error "fix LLONG_MAX" -# endif -#endif - -#ifndef LLONG_MIN -# define LLONG_MIN (-LLONG_MAX-1) -#endif /*** unary operations ***/ -#define OP_INT_IS_TRUE(x,r) OP_INT_NE(x,0,r) - -#define OP_INT_INVERT(x,r) r = ~((x)) - -#define OP_INT_NEG(x,r) r = -(x) +#define OP_INT_IS_TRUE(x,r) r = ((x) != 0) +#define OP_INT_INVERT(x,r) r = ~(x) +#define OP_INT_NEG(x,r) r = -(x) #define OP_INT_NEG_OVF(x,r) \ - if ((x) == LONG_MIN) FAIL_OVF("integer negate"); \ + if ((x) == LONG_MIN) FAIL_OVF("integer negate"); \ OP_INT_NEG(x,r) #define OP_LLONG_NEG_OVF(x,r) \ - if ((x) == LLONG_MIN) FAIL_OVF("integer negate"); \ + if ((x) == LLONG_MIN) FAIL_OVF("integer negate"); \ OP_LLONG_NEG(x,r) #define OP_INT_ABS(x,r) r = (x) >= 0 ? x : -(x) #define OP_INT_ABS_OVF(x,r) \ - if ((x) == LONG_MIN) FAIL_OVF("integer absolute"); \ + if ((x) == LONG_MIN) FAIL_OVF("integer absolute"); \ OP_INT_ABS(x,r) #define OP_LLONG_ABS_OVF(x,r) \ - if ((x) == LLONG_MIN) FAIL_OVF("integer absolute"); \ + if ((x) == LLONG_MIN) FAIL_OVF("integer absolute"); \ OP_LLONG_ABS(x,r) /*** binary operations ***/ @@ -59,50 +46,46 @@ #define OP_INT_ADD(x,y,r) r = (x) + (y) +/* cast to avoid undefined behaviour on overflow */ #define OP_INT_ADD_OVF(x,y,r) \ - OP_INT_ADD(x,y,r); \ - if ((r^(x)) >= 0 || (r^(y)) >= 0); \ - else FAIL_OVF("integer addition") + r = (long)((unsigned long)x + y); \ + if ((r^x) < 0 && (r^y) < 0) FAIL_OVF("integer addition") + +#define OP_LLONG_ADD_OVF(x,y,r) \ + r = (long long)((unsigned long long)x + y); \ + if ((r^x) < 0 && (r^y) < 0) FAIL_OVF("integer addition") #define OP_INT_ADD_NONNEG_OVF(x,y,r) /* y can be assumed >= 0 */ \ - r = (long)((unsigned long)x + (unsigned long)y); \ - if (r >= (x)); \ - else FAIL_OVF("integer addition") -/* Can a C compiler be too clever and think it can "prove" that - * r >= x always holds above? Yes. Hence the casting. */ + r = (long)((unsigned long)x + y); \ + if ((r&~x) < 0) FAIL_OVF("integer addition") #define OP_INT_SUB(x,y,r) r = (x) - (y) #define OP_INT_SUB_OVF(x,y,r) \ - OP_INT_SUB(x,y,r); \ - if ((r^(x)) >= 0 || (r^~(y)) >= 0); \ - else FAIL_OVF("integer subtraction") - -#define OP_INT_MUL(x,y,r) r = (x) * (y) - -#if defined(HAVE_LONG_LONG) && SIZE_OF_LONG_LONG < SIZE_OF_LONG -# define OP_INT_MUL_OVF_LL 1 -#lse -# define OP_INT_MUL_OVF_LL 0 -#endif + r = (long)((unsigned long)x - y); \ + if ((r^x) < 0 && (r^~y) < 0) FAIL_OVF("integer subtraction") -#if !OP_INT_MUL_OVF_LL +#define OP_LLONG_SUB_OVF(x,y,r) \ + r = (long long)((unsigned long long)x - y); \ + if ((r^x) < 0 && (r^~y) < 0) FAIL_OVF("integer subtraction") -#define OP_INT_MUL_OVF(x,y,r) \ - if (op_int_mul_ovf(x,y,&r)); \ - else FAIL_OVF("integer multiplication") - -#else +#define OP_INT_MUL(x,y,r) r = (x) * (y) +#if SIZEOF_LONG * 2 <= SIZEOF_LONG_LONG #define OP_INT_MUL_OVF(x,y,r) \ { \ - PY_LONG_LONG lr = (PY_LONG_LONG)(x) * (PY_LONG_LONG)(y); \ - r = (long)lr; \ - if ((PY_LONG_LONG)r == lr); \ - else FAIL_OVF("integer multiplication"); \ + long long _lr = (long long)x * y; \ + r = (long)_lr; \ + if (_lr != (long long)r) FAIL_OVF("integer multiplication"); \ } +#else +#define OP_INT_MUL_OVF(x,y,r) \ + r = op_llong_mul_ovf(x, y) /* long == long long */ #endif +#define OP_LLONG_MUL_OVF(x,y,r) \ + r = op_llong_mul_ovf(x, y) + /* shifting */ /* NB. shifting has same limitations as C: the shift count must be @@ -121,6 +104,10 @@ OP_INT_LSHIFT(x,y,r); \ if ((x) != Py_ARITHMETIC_RIGHT_SHIFT(long, r, (y))) \ FAIL_OVF("x< signum or -1 */ -/* When a signal is received, the bit 30 of pypysig_occurred is set. - After all signals are processed by pypysig_poll(), the bit 30 is - cleared again. The variable is exposed and RPython code is free to - use the other bits in any way. */ -#define PENDING_SIGNAL_BIT (1 << 30) +/* When a signal is received, pypysig_counter is set to -1. */ /* This is a struct for the JIT. See interp_signal.py. */ struct pypysig_long_struct { long value; }; -extern struct pypysig_long_struct pypysig_occurred; +extern struct pypysig_long_struct pypysig_counter; /* some C tricks to get/set the variable as efficiently as possible: use macros when compiling as a stand-alone program, but still @@ -65,18 +61,20 @@ #undef pypysig_getaddr_occurred void *pypysig_getaddr_occurred(void); #ifndef PYPY_NOT_MAIN_FILE -void *pypysig_getaddr_occurred(void) { return (void *)(&pypysig_occurred); } +void *pypysig_getaddr_occurred(void) { return (void *)(&pypysig_counter); } #endif -#define pypysig_getaddr_occurred() ((void *)(&pypysig_occurred)) +#define pypysig_getaddr_occurred() ((void *)(&pypysig_counter)) /************************************************************/ /* Implementation */ #ifndef PYPY_NOT_MAIN_FILE -struct pypysig_long_struct pypysig_occurred; -static volatile long *pypysig_occurred_v = (volatile long *)&pypysig_occurred.value; -static volatile int pypysig_flags[NSIG]; +struct pypysig_long_struct pypysig_counter = {0}; +static char volatile pypysig_flags[NSIG] = {0}; +static int volatile pypysig_occurred = 0; +/* pypysig_occurred is only an optimization: it tells if any + pypysig_flags could be set. */ static int wakeup_fd = -1; void pypysig_ignore(int signum) @@ -110,10 +108,11 @@ static void signal_setflag_handler(int signum) { if (0 <= signum && signum < NSIG) + { pypysig_flags[signum] = 1; - /* the point of "*pypysig_occurred_v" instead of just "pypysig_occurred" - is the volatile declaration */ - *pypysig_occurred_v |= PENDING_SIGNAL_BIT; + pypysig_occurred = 1; + pypysig_counter.value = -1; + } if (wakeup_fd != -1) write(wakeup_fd, "\0", 1); @@ -135,25 +134,19 @@ int pypysig_poll(void) { - /* the two commented out lines below are useful for performance in - normal usage of pypysig_poll(); however, pypy/module/signal/ is - not normal usage. It only calls pypysig_poll() if the - PENDING_SIGNAL_BIT is set, and it clears that bit first. */ - -/* if (pypysig_occurred & PENDING_SIGNAL_BIT) */ + if (pypysig_occurred) { - int i; -/* pypysig_occurred &= ~PENDING_SIGNAL_BIT; */ - for (i=0; i Author: wlav Date: Tue Nov 9 02:18:57 2010 New Revision: 78903 Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py Log: array support for all integer types Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/converter.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/converter.py Tue Nov 9 02:18:57 2010 @@ -306,8 +306,14 @@ _converters["int*"] = LongPtrConverter _converters["int[]"] = LongArrayConverter _converters["unsigned int"] = LongConverter +_converters["unsigned int*"] = LongPtrConverter +_converters["unsigned int[]"] = LongArrayConverter _converters["long int"] = LongConverter +_converters["long int*"] = LongPtrConverter +_converters["long int[]"] = LongArrayConverter _converters["unsigned long int"] = LongConverter +_converters["unsigned long int*"] = LongPtrConverter +_converters["unsigned long int[]"] = LongArrayConverter _converters["float"] = FloatConverter _converters["double"] = DoubleConverter _converters["const char*"] = CStringConverter Modified: pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py Tue Nov 9 02:18:57 2010 @@ -98,8 +98,8 @@ # integer arrays import array a = range(self.N) - atypes = ['h', 'H', 'i', 'I', 'l']#, 'L' ] - for j in range(len(atypes)):#names)): + atypes = ['h', 'H', 'i', 'I', 'l', 'L' ] + for j in range(len(names)): b = array.array(atypes[j], a) exec 'c.m_%s_array = b' % names[j] # buffer copies for i in range(self.N): From afa at codespeak.net Tue Nov 9 08:38:53 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 08:38:53 +0100 (CET) Subject: [pypy-svn] r78904 - pypy/branch/fast-forward/lib-python/2.7.0 Message-ID: <20101109073853.615E4282B90@codespeak.net> Author: afa Date: Tue Nov 9 08:38:49 2010 New Revision: 78904 Modified: pypy/branch/fast-forward/lib-python/2.7.0/collections.py Log: Pull a change I just made in CPython Modified: pypy/branch/fast-forward/lib-python/2.7.0/collections.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/collections.py (original) +++ pypy/branch/fast-forward/lib-python/2.7.0/collections.py Tue Nov 9 08:38:49 2010 @@ -14,7 +14,7 @@ ifilter as _ifilter, imap as _imap try: from thread import get_ident -except AttributeError: +except ImportError: from dummy_thread import get_ident def _recursive_repr(user_function): From micke at codespeak.net Tue Nov 9 09:16:50 2010 From: micke at codespeak.net (micke at codespeak.net) Date: Tue, 9 Nov 2010 09:16:50 +0100 (CET) Subject: [pypy-svn] r78905 - pypy/extradoc/planning/hg-migration Message-ID: <20101109081650.890AC282B90@codespeak.net> Author: micke Date: Tue Nov 9 09:16:46 2010 New Revision: 78905 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: Added my email address. Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Tue Nov 9 09:16:46 2010 @@ -97,7 +97,7 @@ haypo=Victor Stinner antoine=Antoine Pitrou atobe=Toby Watson -micke=Mikael Sch?nenberg +micke=Mikael Sch?nenberg nshepperd=Neil Shepperd stuart=Stuart Williams hruske=Gasper Zejn From afa at codespeak.net Tue Nov 9 10:10:49 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 10:10:49 +0100 (CET) Subject: [pypy-svn] r78906 - pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests Message-ID: <20101109091049.9F2B3282B90@codespeak.net> Author: afa Date: Tue Nov 9 10:10:48 2010 New Revision: 78906 Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_structures.py Log: Don't test c_longdouble in Structures: struct.calcsize cannot handle it :-( Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_structures.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_structures.py (original) +++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_structures.py Tue Nov 9 10:10:48 2010 @@ -57,7 +57,6 @@ "Q": c_ulonglong, "f": c_float, "d": c_double, - "g": c_longdouble, } def test_simple_structs(self): @@ -66,7 +65,7 @@ _fields_ = [("x", c_char), ("y", tp)] assert (sizeof(X), code) == ( - (calcsize("c%c0%c" % (code, code)), code)) + (calcsize("c%c" % (code,)), code)) def test_unions(self): for code, tp in self.formats.items(): From arigo at codespeak.net Tue Nov 9 10:11:13 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 9 Nov 2010 10:11:13 +0100 (CET) Subject: [pypy-svn] r78907 - in pypy/trunk/pypy: jit/backend/llsupport jit/backend/x86 jit/backend/x86/test rpython/memory/gc Message-ID: <20101109091113.4EDB7282B9E@codespeak.net> Author: arigo Date: Tue Nov 9 10:11:11 2010 New Revision: 78907 Modified: pypy/trunk/pypy/jit/backend/llsupport/gc.py pypy/trunk/pypy/jit/backend/x86/assembler.py pypy/trunk/pypy/jit/backend/x86/test/test_zrpy_gc.py pypy/trunk/pypy/rpython/memory/gc/minimark.py pypy/trunk/pypy/rpython/memory/gc/semispace.py Log: Test and fix for an obscure crash that shows up on the JIT with (preferencially) 64-bit versions. The issue is that the JIT did not respect the minimal allocation size in the nursery. This gives strange effects at the next minor collection. Modified: pypy/trunk/pypy/jit/backend/llsupport/gc.py ============================================================================== --- pypy/trunk/pypy/jit/backend/llsupport/gc.py (original) +++ pypy/trunk/pypy/jit/backend/llsupport/gc.py Tue Nov 9 10:11:11 2010 @@ -19,6 +19,7 @@ # ____________________________________________________________ class GcLLDescription(GcCache): + minimal_size_in_nursery = 0 def __init__(self, gcdescr, translator=None, rtyper=None): GcCache.__init__(self, translator is not None, rtyper) self.gcdescr = gcdescr @@ -386,6 +387,7 @@ (self.array_basesize, _, self.array_length_ofs) = \ symbolic.get_array_token(lltype.GcArray(lltype.Signed), True) self.max_size_of_young_obj = self.GCClass.JIT_max_size_of_young_obj() + self.minimal_size_in_nursery=self.GCClass.JIT_minimal_size_in_nursery() # make a malloc function, with three arguments def malloc_basic(size, tid): @@ -468,6 +470,7 @@ def malloc_fixedsize_slowpath(size): if self.DEBUG: random_usage_of_xmm_registers() + assert size >= self.minimal_size_in_nursery try: gcref = llop1.do_malloc_fixedsize_clear(llmemory.GCREF, 0, size, True, False, False) Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/assembler.py (original) +++ pypy/trunk/pypy/jit/backend/x86/assembler.py Tue Nov 9 10:11:11 2010 @@ -1853,6 +1853,7 @@ def malloc_cond_fixedsize(self, nursery_free_adr, nursery_top_adr, size, tid): + size = max(size, self.cpu.gc_ll_descr.minimal_size_in_nursery) self.mc.ensure_bytes_available(256) self.mc.MOV(eax, heap(nursery_free_adr)) self.mc.LEA_rm(edx.value, (eax.value, size)) Modified: pypy/trunk/pypy/jit/backend/x86/test/test_zrpy_gc.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/test/test_zrpy_gc.py (original) +++ pypy/trunk/pypy/jit/backend/x86/test/test_zrpy_gc.py Tue Nov 9 10:11:11 2010 @@ -550,3 +550,29 @@ def test_compile_framework_float(self): self.run('compile_framework_float') + + def define_compile_framework_minimal_size_in_nursery(self): + S = lltype.GcStruct('S') # no fields! + T = lltype.GcStruct('T', ('i', lltype.Signed)) + @unroll_safe + def f42(n, x, x0, x1, x2, x3, x4, x5, x6, x7, l, s): + lst1 = [] + lst2 = [] + i = 0 + while i < 42: + s1 = lltype.malloc(S) + t1 = lltype.malloc(T) + t1.i = 10000 + i + n + lst1.append(s1) + lst2.append(t1) + i += 1 + i = 0 + while i < 42: + check(lst2[i].i == 10000 + i + n) + i += 1 + n -= 1 + return n, x, x0, x1, x2, x3, x4, x5, x6, x7, l, s + return None, f42, None + + def test_compile_framework_minimal_size_in_nursery(self): + self.run('compile_framework_minimal_size_in_nursery') Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Tue Nov 9 10:11:11 2010 @@ -77,7 +77,7 @@ # During a minor collection, the objects in the nursery that are # moved outside are changed in-place: their header is replaced with - # the value -1, and the following word is set to the address of + # the value -42, and the following word is set to the address of # where the object was moved. This means that all objects in the # nursery need to be at least 2 words long, but objects outside the # nursery don't need to. @@ -656,10 +656,13 @@ def is_forwarded(self, obj): """Returns True if the nursery obj is marked as forwarded. Implemented a bit obscurely by checking an unrelated flag - that can never be set on a young object -- except if tid == -1. + that can never be set on a young object -- except if tid == -42. """ assert self.is_in_nursery(obj) - return self.header(obj).tid & GCFLAG_FINALIZATION_ORDERING + result = (self.header(obj).tid & GCFLAG_FINALIZATION_ORDERING != 0) + if result: + ll_assert(self.header(obj).tid == -42, "bogus header for young obj") + return result def get_forwarding_address(self, obj): return llmemory.cast_adr_to_ptr(obj, FORWARDSTUBPTR).forw @@ -743,6 +746,10 @@ def JIT_max_size_of_young_obj(cls): return cls.TRANSLATION_PARAMS['large_object'] + @classmethod + def JIT_minimal_size_in_nursery(cls): + return cls.minimal_size_in_nursery + def write_barrier(self, newvalue, addr_struct): if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS: self.remember_young_pointer(addr_struct, newvalue) @@ -1080,7 +1087,7 @@ llarena.arena_reset(obj - size_gc_header, totalsize, 0) llarena.arena_reserve(obj - size_gc_header, size_gc_header + llmemory.sizeof(FORWARDSTUB)) - self.header(obj).tid = -1 + self.header(obj).tid = -42 newobj = newhdr + size_gc_header llmemory.cast_adr_to_ptr(obj, FORWARDSTUBPTR).forw = newobj # Modified: pypy/trunk/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/semispace.py Tue Nov 9 10:11:11 2010 @@ -230,6 +230,10 @@ while self.max_space_size > size: self.max_space_size >>= 1 + @classmethod + def JIT_minimal_size_in_nursery(cls): + return cls.object_minimal_size + def collect(self, gen=0): self.debug_check_consistency() self.semispace_collect() From arigo at codespeak.net Tue Nov 9 10:20:32 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 9 Nov 2010 10:20:32 +0100 (CET) Subject: [pypy-svn] r78908 - pypy/trunk/pypy/jit/metainterp/test Message-ID: <20101109092032.660FF282BDA@codespeak.net> Author: arigo Date: Tue Nov 9 10:20:30 2010 New Revision: 78908 Modified: pypy/trunk/pypy/jit/metainterp/test/test_del.py Log: Fix the test. Modified: pypy/trunk/pypy/jit/metainterp/test/test_del.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_del.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_del.py Tue Nov 9 10:20:30 2010 @@ -85,6 +85,7 @@ def test_signal_action(self): from pypy.module.signal.interp_signal import SignalActionFlag action = SignalActionFlag() + action.has_bytecode_counter = True # myjitdriver = JitDriver(greens = [], reds = ['n', 'x']) class X: @@ -92,17 +93,17 @@ # def f(n): x = X() - while n > 0: + action.reset_ticker(n) + while True: myjitdriver.can_enter_jit(n=n, x=x) myjitdriver.jit_merge_point(n=n, x=x) x.foo = n n -= 1 - if action.get() != 0: + if action.decrement_ticker(1) < 0: break - action.set(0) return 42 self.meta_interp(f, [20]) - self.check_loops(getfield_raw=1, call=0, call_pure=0) + self.check_loops(getfield_raw=1, setfield_raw=1, call=0, call_pure=0) class TestOOtype(DelTests, OOJitMixin): def setup_class(cls): From arigo at codespeak.net Tue Nov 9 10:40:56 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 9 Nov 2010 10:40:56 +0100 (CET) Subject: [pypy-svn] r78909 - in pypy/trunk/lib_pypy/ctypes_config_cache: . test Message-ID: <20101109094056.B88DA282B90@codespeak.net> Author: arigo Date: Tue Nov 9 10:40:47 2010 New Revision: 78909 Modified: pypy/trunk/lib_pypy/ctypes_config_cache/dumpcache.py pypy/trunk/lib_pypy/ctypes_config_cache/test/test_cache.py Log: Fix when run as tests of pypy/module/test_lib_pypy/ (mostly by whacking until it works) Modified: pypy/trunk/lib_pypy/ctypes_config_cache/dumpcache.py ============================================================================== --- pypy/trunk/lib_pypy/ctypes_config_cache/dumpcache.py (original) +++ pypy/trunk/lib_pypy/ctypes_config_cache/dumpcache.py Tue Nov 9 10:40:47 2010 @@ -16,8 +16,10 @@ except ImportError: from pypy.jit.backend import detect_cpu cpumodel = detect_cpu.autodetect_main_model_and_size() -mod = __import__("ctypes_config_cache._%s_%%s_" %% (cpumodel,), - None, None, ["*"]) +# XXX relative import, should be removed together with +# XXX the relative imports done e.g. by lib_pypy/pypy_test/test_hashlib +mod = __import__("_%s_%%s_" %% (cpumodel,), + globals(), locals(), ["*"]) globals().update(mod.__dict__)\ ''' % (basename,) g.close() Modified: pypy/trunk/lib_pypy/ctypes_config_cache/test/test_cache.py ============================================================================== --- pypy/trunk/lib_pypy/ctypes_config_cache/test/test_cache.py (original) +++ pypy/trunk/lib_pypy/ctypes_config_cache/test/test_cache.py Tue Nov 9 10:40:47 2010 @@ -21,14 +21,16 @@ # outputpath = tmpdir.join(outputname) assert outputpath.check(exists=1) - d = {} + modname = os.path.splitext(outputname)[0] try: sys.path.insert(0, str(tmpdir2)) - execfile(str(outputpath), d) + d = {} + exec "from ctypes_config_cache import %s" % modname in d + mod = d[modname] finally: sys.path[:] = path sys.modules.pop('ctypes_config_cache', None) - return d + return mod.__dict__ def test_syslog(): From afa at codespeak.net Tue Nov 9 12:04:10 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 12:04:10 +0100 (CET) Subject: [pypy-svn] r78911 - in pypy/branch/fast-forward/pypy/module/_warnings: . test Message-ID: <20101109110410.925E1282B90@codespeak.net> Author: afa Date: Tue Nov 9 12:04:08 2010 New Revision: 78911 Modified: pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py pypy/branch/fast-forward/pypy/module/_warnings/test/test_warnings.py Log: Fix the lineno returned in a warning message Modified: pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py (original) +++ pypy/branch/fast-forward/pypy/module/_warnings/interp_warnings.py Tue Nov 9 12:04:08 2010 @@ -77,7 +77,7 @@ stacklevel -= 1 if frame: w_globals = frame.w_globals - lineno = frame.f_lineno + lineno = frame.get_last_lineno() else: w_globals = space.sys.w_dict lineno = 1 Modified: pypy/branch/fast-forward/pypy/module/_warnings/test/test_warnings.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_warnings/test/test_warnings.py (original) +++ pypy/branch/fast-forward/pypy/module/_warnings/test/test_warnings.py Tue Nov 9 12:04:08 2010 @@ -18,6 +18,13 @@ _warnings.warn("some message", DeprecationWarning) _warnings.warn("some message", Warning) + def test_lineno(self): + import warnings, _warnings, sys + with warnings.catch_warnings(record=True) as w: + _warnings.warn("some message", Warning) + lineno = sys._getframe().f_lineno - 1 # the line above + assert w[-1].lineno == lineno + def test_warn_explicit(self): import _warnings _warnings.warn_explicit("some message", DeprecationWarning, From antocuni at codespeak.net Tue Nov 9 12:07:10 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 9 Nov 2010 12:07:10 +0100 (CET) Subject: [pypy-svn] r78912 - in pypy/trunk/pypy/rlib: . test Message-ID: <20101109110710.AC0FD282BDB@codespeak.net> Author: antocuni Date: Tue Nov 9 12:07:09 2010 New Revision: 78912 Modified: pypy/trunk/pypy/rlib/rarithmetic.py pypy/trunk/pypy/rlib/test/test_rarithmetic.py Log: don't crash if you try to compare an r_singlefloat with == or !=. Some places in the translation rely on being able to compare random stuff without crashing. Make sure we still get a crash if we use == or != in rpython Modified: pypy/trunk/pypy/rlib/rarithmetic.py ============================================================================== --- pypy/trunk/pypy/rlib/rarithmetic.py (original) +++ pypy/trunk/pypy/rlib/rarithmetic.py Tue Nov 9 12:07:09 2010 @@ -501,6 +501,11 @@ def __cmp__(self, other): raise TypeError("not supported on r_singlefloat instances") + def __eq__(self, other): + return self.__class__ is other.__class__ and self._bytes == other._bytes + + def __ne__(self, other): + return not self.__eq__(other) class For_r_singlefloat_values_Entry(extregistry.ExtRegistryEntry): _type_ = r_singlefloat Modified: pypy/trunk/pypy/rlib/test/test_rarithmetic.py ============================================================================== --- pypy/trunk/pypy/rlib/test/test_rarithmetic.py (original) +++ pypy/trunk/pypy/rlib/test/test_rarithmetic.py Tue Nov 9 12:07:09 2010 @@ -325,6 +325,15 @@ assert float(x) != 2.1 assert abs(float(x) - 2.1) < 1E-6 +def test_r_singlefloat_eq(): + x = r_singlefloat(2.5) # exact number + y = r_singlefloat(2.5) + assert x == y + assert not x != y + assert not x == 2.5 + assert x != 2.5 + py.test.raises(TypeError, "x>y") + class BaseTestRarithmetic(BaseRtypingTest): def test_formatd(self): from pypy.rlib.rarithmetic import formatd @@ -358,7 +367,17 @@ assert res == 1.0 res = self.interpret(f, [1]) - assert res == 1e-100 + assert res == 1e-100 + + def test_compare_singlefloat_crashes(self): + from pypy.rlib.rarithmetic import r_singlefloat + from pypy.rpython.error import MissingRTypeOperation + def f(x): + a = r_singlefloat(x) + b = r_singlefloat(x+1) + return a == b + py.test.raises(MissingRTypeOperation, "self.interpret(f, [42.0])") + class TestLLtype(BaseTestRarithmetic, LLRtypeMixin): pass From fijal at codespeak.net Tue Nov 9 12:26:13 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 9 Nov 2010 12:26:13 +0100 (CET) Subject: [pypy-svn] r78913 - pypy/branch/fast-forward/lib-python Message-ID: <20101109112613.57A16282BDB@codespeak.net> Author: fijal Date: Tue Nov 9 12:26:11 2010 New Revision: 78913 Modified: pypy/branch/fast-forward/lib-python/TODO Log: add a task Modified: pypy/branch/fast-forward/lib-python/TODO ============================================================================== --- pypy/branch/fast-forward/lib-python/TODO (original) +++ pypy/branch/fast-forward/lib-python/TODO Tue Nov 9 12:26:11 2010 @@ -55,6 +55,8 @@ - add 'unicode' in ObjSpace.MethodTable + probably a default implementation that falls back to space.str(). +- socket module has a couple of changes (including AF_TIPC packet range) + Longer tasks ------------ From afa at codespeak.net Tue Nov 9 13:03:44 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 13:03:44 +0100 (CET) Subject: [pypy-svn] r78914 - pypy/branch/fast-forward/lib-python/modified-2.7.0/test Message-ID: <20101109120344.A685A282B9D@codespeak.net> Author: afa Date: Tue Nov 9 13:03:41 2010 New Revision: 78914 Added: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_struct.py - copied, changed from r78910, pypy/branch/fast-forward/lib-python/2.7.0/test/test_struct.py Log: Implementation detail: allow the struct module to "work" in this case. Copied: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_struct.py (from r78910, pypy/branch/fast-forward/lib-python/2.7.0/test/test_struct.py) ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/test/test_struct.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_struct.py Tue Nov 9 13:03:41 2010 @@ -535,7 +535,8 @@ @unittest.skipUnless(IS32BIT, "Specific to 32bit machines") def test_crasher(self): - self.assertRaises(MemoryError, struct.pack, "357913941c", "a") + self.assertRaises((MemoryError, struct.error), struct.pack, + "357913941c", "a") def test_count_overflow(self): hugecount = '{}b'.format(sys.maxsize+1) From fijal at codespeak.net Tue Nov 9 13:12:34 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 9 Nov 2010 13:12:34 +0100 (CET) Subject: [pypy-svn] r78915 - pypy/trunk/pypy/rpython Message-ID: <20101109121234.C5991282B90@codespeak.net> Author: fijal Date: Tue Nov 9 13:12:33 2010 New Revision: 78915 Modified: pypy/trunk/pypy/rpython/rmodel.py Log: Make setupstate a newstyle class and make Repr a more-obvious newstyle class Modified: pypy/trunk/pypy/rpython/rmodel.py ============================================================================== --- pypy/trunk/pypy/rpython/rmodel.py (original) +++ pypy/trunk/pypy/rpython/rmodel.py Tue Nov 9 13:12:33 2010 @@ -11,14 +11,14 @@ # initialization states for Repr instances -class setupstate: +class setupstate(object): NOTINITIALIZED = 0 INPROGRESS = 1 BROKEN = 2 FINISHED = 3 DELAYED = 4 -class Repr: +class Repr(object): """ An instance of Repr is associated with each instance of SomeXxx. It defines the chosen representation for the SomeXxx. The Repr subclasses generally follows the SomeXxx subclass hierarchy, but there are numerous From afa at codespeak.net Tue Nov 9 13:40:07 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 13:40:07 +0100 (CET) Subject: [pypy-svn] r78916 - pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests Message-ID: <20101109124007.0F4C5282B90@codespeak.net> Author: afa Date: Tue Nov 9 13:40:05 2010 New Revision: 78916 Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c Log: Fix test on Windows Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c ============================================================================== --- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c (original) +++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c Tue Nov 9 13:40:05 2010 @@ -533,7 +533,7 @@ return inp; } -int my_unused_function(void) +EXPORT(int) my_unused_function(void) { return 42; } From afa at codespeak.net Tue Nov 9 13:41:43 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 13:41:43 +0100 (CET) Subject: [pypy-svn] r78917 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101109124143.294CA36C222@codespeak.net> Author: afa Date: Tue Nov 9 13:41:40 2010 New Revision: 78917 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py Log: Make some ctypes exceptions more like CPython. Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/array.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/array.py Tue Nov 9 13:41:40 2010 @@ -91,6 +91,9 @@ if len(value) > self._length_: raise ValueError("Invalid length") value = self(*value) + elif not isinstance(value, self): + raise TypeError("expected string or Unicode object, %s found" + % (value.__class__.__name__,)) else: if isinstance(value, tuple): if len(value) > self._length_: Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py Tue Nov 9 13:41:40 2010 @@ -144,7 +144,11 @@ def from_param(self, value): if isinstance(value, tuple): - value = self(*value) + try: + value = self(*value) + except Exception, e: + # XXX CPython does not even respect the exception type + raise RuntimeError("(%s) %s: %s" % (self.__name__, type(e), e)) return _CDataMeta.from_param(self, value) def _CData_output(self, resarray, base=None, index=-1): @@ -174,7 +178,7 @@ def __init__(self, *args, **kwds): if len(args) > len(self._names): - raise TypeError("too many arguments") + raise TypeError("too many initializers") for name, arg in zip(self._names, args): if name in kwds: raise TypeError("duplicate value for argument %r" % ( From arigo at codespeak.net Tue Nov 9 14:17:07 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 9 Nov 2010 14:17:07 +0100 (CET) Subject: [pypy-svn] r78919 - pypy/branch/rlist-jit Message-ID: <20101109131707.B4A1F282B90@codespeak.net> Author: arigo Date: Tue Nov 9 14:17:06 2010 New Revision: 78919 Added: pypy/branch/rlist-jit/ - copied from r78918, pypy/trunk/ Log: A branch to try to improve the current delicate situation of oopspecs, multi-levels in rpython/rlist and rpython/lltypesystem/rlist, and func=dum_checkids. From afa at codespeak.net Tue Nov 9 15:02:30 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 15:02:30 +0100 (CET) Subject: [pypy-svn] r78920 - in pypy/branch/fast-forward: lib_pypy/_ctypes pypy/module/test_lib_pypy/ctypes_tests Message-ID: <20101109140230.85835282BDA@codespeak.net> Author: afa Date: Tue Nov 9 15:02:26 2010 New Revision: 78920 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_structures.py Log: ctypes: Fix all tests in test_struct_fields.py: # Structure/Union classes must get 'finalized' sooner or # later, when one of these things happen: # # 1. _fields_ is set. # 2. An instance is created. # 3. The type is used as field of another Structure/Union. # 4. The type is subclassed # # When they are finalized, assigning _fields_ is no longer allowed. Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py Tue Nov 9 15:02:26 2010 @@ -37,6 +37,8 @@ tp = f[1] if not isinstance(tp, _CDataMeta): raise TypeError("Expected CData subclass, got %s" % (tp,)) + if isinstance(tp, StructOrUnionMeta): + tp._make_final() import ctypes all_fields = _fields_[:] for cls in inspect.getmro(superclass): @@ -113,6 +115,8 @@ def __new__(self, name, cls, typedict): res = type.__new__(self, name, cls, typedict) + if isinstance(cls[0], StructOrUnionMeta): + cls[0]._make_final() if '_fields_' in typedict: if not hasattr(typedict.get('_anonymous_', []), '__iter__'): raise TypeError("Anonymous field must be iterable") @@ -126,6 +130,15 @@ return res + def _make_final(self): + if self is StructOrUnion: + return + if '_fields_' not in self.__dict__: + self._fields_ = [] + self._names = [] + self._fieldtypes = {} + _set_shape(self, [], self._is_union) + __getattr__ = struct_getattr __setattr__ = struct_setattr @@ -170,13 +183,13 @@ __metaclass__ = StructOrUnionMeta def __new__(cls, *args, **kwds): - if not hasattr(cls, '_ffistruct'): - raise TypeError("Cannot instantiate structure, has no _fields_") self = super(_CData, cls).__new__(cls, *args, **kwds) - self.__dict__['_buffer'] = self._ffistruct(autofree=True) + if hasattr(cls, '_ffistruct'): + self.__dict__['_buffer'] = self._ffistruct(autofree=True) return self def __init__(self, *args, **kwds): + type(self)._make_final() if len(args) > len(self._names): raise TypeError("too many initializers") for name, arg in zip(self._names, args): Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_structures.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_structures.py (original) +++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_structures.py Tue Nov 9 15:02:26 2010 @@ -495,6 +495,24 @@ else: raise AssertionError, "AttributeError not raised" + def test_nonfinal_struct(self): + class X(Structure): + pass + assert sizeof(X) == 0 + X._fields_ = [("a", c_int),] + raises(AttributeError, setattr, X, "_fields_", []) + + class X(Structure): + pass + X() + raises(AttributeError, setattr, X, "_fields_", []) + + class X(Structure): + pass + class Y(X): + pass + raises(AttributeError, setattr, X, "_fields_", []) + Y.__fields__ = [] class TestPatologicalCases(BaseCTypesTestChecker): def test_structure_overloading_getattr(self): From afa at codespeak.net Tue Nov 9 15:04:33 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 15:04:33 +0100 (CET) Subject: [pypy-svn] r78921 - pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests Message-ID: <20101109140433.A0C81282BDD@codespeak.net> Author: afa Date: Tue Nov 9 15:04:32 2010 New Revision: 78921 Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_struct_fields.py Log: These tests pass now... Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_struct_fields.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_struct_fields.py (original) +++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_struct_fields.py Tue Nov 9 15:04:32 2010 @@ -26,14 +26,12 @@ raises(AttributeError, setattr, X, "_fields_", []) def test_2(self): - py.test.skip("absent _fields_ semantics not implemented") class X(Structure): pass X() raises(AttributeError, setattr, X, "_fields_", []) def test_3(self): - py.test.skip("subclassing semantics not implemented") class X(Structure): pass class Y(Structure): @@ -41,7 +39,6 @@ raises(AttributeError, setattr, X, "_fields_", []) def test_4(self): - py.test.skip("subclassing semantics not implemented") class X(Structure): pass class Y(X): From afa at codespeak.net Tue Nov 9 15:09:21 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 15:09:21 +0100 (CET) Subject: [pypy-svn] r78922 - in pypy/branch/fast-forward: lib_pypy/_ctypes pypy/module/test_lib_pypy/ctypes_tests Message-ID: <20101109140921.4A596282B9E@codespeak.net> Author: afa Date: Tue Nov 9 15:09:19 2010 New Revision: 78922 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_structures.py Log: Base fields should appear before subclass fields Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py Tue Nov 9 15:09:19 2010 @@ -40,9 +40,10 @@ if isinstance(tp, StructOrUnionMeta): tp._make_final() import ctypes - all_fields = _fields_[:] + all_fields = [] for cls in inspect.getmro(superclass): all_fields += getattr(cls, '_fields_', []) + all_fields += _fields_ names = [f[0] for f in all_fields] rawfields = [(f[0], f[1]._ffishape) for f in all_fields] Modified: pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_structures.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_structures.py (original) +++ pypy/branch/fast-forward/pypy/module/test_lib_pypy/ctypes_tests/test_structures.py Tue Nov 9 15:09:19 2010 @@ -22,6 +22,8 @@ assert Y._fields_ == [("b", c_int)] assert Z._fields_ == [("a", c_int)] + assert Y._names == ['a', 'b'] + def test_subclass_delayed(self): class X(Structure): pass From afa at codespeak.net Tue Nov 9 15:35:17 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 15:35:17 +0100 (CET) Subject: [pypy-svn] r78923 - in pypy/branch/fast-forward/lib-python/modified-2.7.0/ctypes: . test Message-ID: <20101109143517.583FE282B9E@codespeak.net> Author: afa Date: Tue Nov 9 15:35:15 2010 New Revision: 78923 Added: pypy/branch/fast-forward/lib-python/modified-2.7.0/ctypes/ - copied from r78910, pypy/branch/fast-forward/lib-python/2.7.0/ctypes/ Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/ctypes/test/test_delattr.py Log: Let delattr raise AttributeError instead of TypeError. It really depends on the kind of attribute/member/slot/descriptor used to represent the attribute. Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/ctypes/test/test_delattr.py ============================================================================== --- pypy/branch/fast-forward/lib-python/2.7.0/ctypes/test/test_delattr.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/ctypes/test/test_delattr.py Tue Nov 9 15:35:15 2010 @@ -6,15 +6,15 @@ class TestCase(unittest.TestCase): def test_simple(self): - self.assertRaises(TypeError, + self.assertRaises((TypeError, AttributeError), delattr, c_int(42), "value") def test_chararray(self): - self.assertRaises(TypeError, + self.assertRaises((TypeError, AttributeError), delattr, (c_char * 5)(), "value") def test_struct(self): - self.assertRaises(TypeError, + self.assertRaises((TypeError, AttributeError), delattr, X(), "foo") if __name__ == "__main__": From arigo at codespeak.net Tue Nov 9 15:51:43 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 9 Nov 2010 15:51:43 +0100 (CET) Subject: [pypy-svn] r78924 - in pypy/branch/rlist-jit/pypy: jit/codewriter jit/codewriter/test jit/metainterp/test rpython rpython/test Message-ID: <20101109145143.918F1282B9E@codespeak.net> Author: arigo Date: Tue Nov 9 15:51:41 2010 New Revision: 78924 Modified: pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py pypy/branch/rlist-jit/pypy/jit/codewriter/test/test_list.py pypy/branch/rlist-jit/pypy/jit/codewriter/test/test_void_list.py pypy/branch/rlist-jit/pypy/jit/metainterp/test/test_list.py pypy/branch/rlist-jit/pypy/rpython/rlist.py pypy/branch/rlist-jit/pypy/rpython/test/test_rlist.py Log: Not really a complete clean-up, but good enough to make one new test in jit/metainterp/test/test_list.py pass. Modified: pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py (original) +++ pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py Tue Nov 9 15:51:41 2010 @@ -924,21 +924,19 @@ # base hints on the name of the ll function, which is a bit xxx-ish # but which is safe for now assert func.func_name.startswith('ll_') + # check that we have carefully placed the oopspec in + # pypy/rpython/rlist.py. There should not be an oopspec on + # a ll_getitem or ll_setitem that expects a 'func' argument. + # The idea is that a ll_getitem/ll_setitem with dum_checkidx + # should get inlined by the JIT, so that we see the potential + # 'raise IndexError'. + assert 'func' not in func.func_code.co_varnames non_negative = '_nonneg' in func.func_name fast = '_fast' in func.func_name - if fast: - can_raise = False - non_negative = True - else: - tag = op.args[1].value - assert tag in (rlist.dum_nocheck, rlist.dum_checkidx) - can_raise = tag != rlist.dum_nocheck - return non_negative, can_raise + return non_negative or fast def _prepare_list_getset(self, op, descr, args, checkname): - non_negative, can_raise = self._get_list_nonneg_canraise_flags(op) - if can_raise: - raise NotSupported("list operation can raise") + non_negative = self._get_list_nonneg_canraise_flags(op) if non_negative: return args[1], [] else: @@ -950,9 +948,8 @@ return v_posindex, [op0, op1] def _prepare_void_list_getset(self, op): - non_negative, can_raise = self._get_list_nonneg_canraise_flags(op) - if can_raise: - raise NotSupported("list operation can raise") + # sanity check: + self._get_list_nonneg_canraise_flags(op) def _get_initial_newlist_length(self, op, args): # normalize number of arguments to the 'newlist' function Modified: pypy/branch/rlist-jit/pypy/jit/codewriter/test/test_list.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/codewriter/test/test_list.py (original) +++ pypy/branch/rlist-jit/pypy/jit/codewriter/test/test_list.py Tue Nov 9 15:51:41 2010 @@ -56,9 +56,8 @@ if '/' in oopspec_name: oopspec_name, property = oopspec_name.split('/') def force_flags(op): - if property == 'NONNEG': return True, False - if property == 'NEG': return False, False - if property == 'CANRAISE': return False, True + if property == 'NONNEG': return True + if property == 'NEG': return False raise ValueError(property) tr._get_list_nonneg_canraise_flags = force_flags op = SpaceOperation('direct_call', @@ -122,9 +121,6 @@ check_neg_index %r0, , %i0 -> %i1 getarrayitem_gc_i %r0, , %i1 -> %i2 """) - builtin_test('list.getitem/CANRAISE', - [varoftype(FIXEDLIST), varoftype(lltype.Signed)], - lltype.Signed, NotSupported) def test_fixed_getitem_foldable(): builtin_test('list.getitem_foldable/NONNEG', @@ -139,9 +135,6 @@ check_neg_index %r0, , %i0 -> %i1 getarrayitem_gc_pure_i %r0, , %i1 -> %i2 """) - builtin_test('list.getitem_foldable/CANRAISE', - [varoftype(FIXEDLIST), varoftype(lltype.Signed)], - lltype.Signed, NotSupported) def test_fixed_setitem(): builtin_test('list.setitem/NONNEG', [varoftype(FIXEDLIST), @@ -158,10 +151,6 @@ check_neg_index %r0, , %i0 -> %i1 setarrayitem_gc_i %r0, , %i1, %i2 """) - builtin_test('list.setitem/CANRAISE', [varoftype(FIXEDLIST), - varoftype(lltype.Signed), - varoftype(lltype.Signed)], - lltype.Void, NotSupported) def test_fixed_len(): builtin_test('list.len', [varoftype(FIXEDLIST)], lltype.Signed, @@ -206,9 +195,6 @@ check_resizable_neg_index %r0, , %i0 -> %i1 getlistitem_gc_i %r0, , , %i1 -> %i2 """) - builtin_test('list.getitem/CANRAISE', - [varoftype(VARLIST), varoftype(lltype.Signed)], - lltype.Signed, NotSupported) def test_resizable_setitem(): builtin_test('list.setitem/NONNEG', [varoftype(VARLIST), @@ -225,10 +211,6 @@ check_resizable_neg_index %r0, , %i0 -> %i1 setlistitem_gc_i %r0, , , %i1, %i2 """) - builtin_test('list.setitem/CANRAISE', [varoftype(VARLIST), - varoftype(lltype.Signed), - varoftype(lltype.Signed)], - lltype.Void, NotSupported) def test_resizable_len(): builtin_test('list.len', [varoftype(VARLIST)], lltype.Signed, Modified: pypy/branch/rlist-jit/pypy/jit/codewriter/test/test_void_list.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/codewriter/test/test_void_list.py (original) +++ pypy/branch/rlist-jit/pypy/jit/codewriter/test/test_void_list.py Tue Nov 9 15:51:41 2010 @@ -51,9 +51,6 @@ builtin_test('list.getitem/NEG', [varoftype(FIXEDLIST), varoftype(lltype.Signed)], lltype.Void, "") - builtin_test('list.getitem/CANRAISE', - [varoftype(FIXEDLIST), varoftype(lltype.Signed)], - lltype.Void, NotSupported) def test_fixed_getitem_foldable(): builtin_test('list.getitem_foldable/NONNEG', @@ -62,9 +59,6 @@ builtin_test('list.getitem_foldable/NEG', [varoftype(FIXEDLIST), varoftype(lltype.Signed)], lltype.Void, "") - builtin_test('list.getitem_foldable/CANRAISE', - [varoftype(FIXEDLIST), varoftype(lltype.Signed)], - lltype.Void, NotSupported) def test_fixed_setitem(): builtin_test('list.setitem/NONNEG', [varoftype(FIXEDLIST), @@ -75,10 +69,6 @@ varoftype(lltype.Signed), varoftype(lltype.Void)], lltype.Void, "") - builtin_test('list.setitem/CANRAISE', [varoftype(FIXEDLIST), - varoftype(lltype.Signed), - varoftype(lltype.Void)], - lltype.Void, NotSupported) def test_fixed_len(): builtin_test('list.len', [varoftype(FIXEDLIST)], lltype.Signed, @@ -115,9 +105,6 @@ builtin_test('list.getitem/NEG', [varoftype(VARLIST), varoftype(lltype.Signed)], lltype.Void, "") - builtin_test('list.getitem/CANRAISE', - [varoftype(VARLIST), varoftype(lltype.Signed)], - lltype.Void, NotSupported) def test_resizable_setitem(): builtin_test('list.setitem/NONNEG', [varoftype(VARLIST), @@ -128,10 +115,6 @@ varoftype(lltype.Signed), varoftype(lltype.Void)], lltype.Void, "") - builtin_test('list.setitem/CANRAISE', [varoftype(VARLIST), - varoftype(lltype.Signed), - varoftype(lltype.Void)], - lltype.Void, NotSupported) def test_resizable_len(): builtin_test('list.len', [varoftype(VARLIST)], lltype.Signed, Modified: pypy/branch/rlist-jit/pypy/jit/metainterp/test/test_list.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/metainterp/test/test_list.py (original) +++ pypy/branch/rlist-jit/pypy/jit/metainterp/test/test_list.py Tue Nov 9 15:51:41 2010 @@ -188,6 +188,26 @@ assert res == f(4) self.check_loops(call=0, getfield_gc=0) + def test_fold_indexerror(self): + jitdriver = JitDriver(greens = [], reds = ['total', 'n', 'lst']) + def f(n): + lst = [] + total = 0 + while n > 0: + jitdriver.can_enter_jit(lst=lst, n=n, total=total) + jitdriver.jit_merge_point(lst=lst, n=n, total=total) + lst.append(n) + try: + total += lst[n] + except IndexError: + total += 1000 + n -= 1 + return total + + res = self.meta_interp(f, [15], listops=True) + assert res == f(15) + self.check_loops(guard_exception=0) + class TestOOtype(ListTests, OOJitMixin): pass Modified: pypy/branch/rlist-jit/pypy/rpython/rlist.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rpython/rlist.py (original) +++ pypy/branch/rlist-jit/pypy/rpython/rlist.py Tue Nov 9 15:51:41 2010 @@ -9,7 +9,7 @@ from pypy.rpython import robject from pypy.rlib.objectmodel import malloc_zero_filled from pypy.rlib.debug import ll_assert -from pypy.rlib.rarithmetic import ovfcheck, widen +from pypy.rlib.rarithmetic import ovfcheck, widen, r_uint from pypy.rpython.annlowlevel import ADTInterface from pypy.rlib import rgc @@ -241,17 +241,22 @@ class __extend__(pairtype(AbstractBaseListRepr, IntegerRepr)): def rtype_getitem((r_lst, r_int), hop, checkidx=False): + v_lst, v_index = hop.inputargs(r_lst, Signed) if checkidx: - spec = dum_checkidx + hop.exception_is_here() else: - spec = dum_nocheck - v_func = hop.inputconst(Void, spec) - v_lst, v_index = hop.inputargs(r_lst, Signed) + hop.exception_cannot_occur() if hop.args_s[0].listdef.listitem.mutated or checkidx: if hop.args_s[1].nonneg: llfn = ll_getitem_nonneg else: llfn = ll_getitem + if checkidx: + spec = dum_checkidx + else: + spec = dum_nocheck + c_func_marker = hop.inputconst(Void, spec) + v_res = hop.gendirectcall(llfn, c_func_marker, v_lst, v_index) else: # this is the 'foldable' version, which is not used when # we check for IndexError @@ -259,11 +264,7 @@ llfn = ll_getitem_foldable_nonneg else: llfn = ll_getitem_foldable - if checkidx: - hop.exception_is_here() - else: - hop.exception_cannot_occur() - v_res = hop.gendirectcall(llfn, v_func, v_lst, v_index) + v_res = hop.gendirectcall(llfn, v_lst, v_index) return r_lst.recast(hop.llops, v_res) rtype_getitem_key = rtype_getitem @@ -538,12 +539,14 @@ dest.ll_setitem_fast(dest_start + i, item) i += 1 ll_arraycopy._annenforceargs_ = [None, None, int, int, int] +# no oopspec -- the function is inlined by the JIT def ll_copy(RESLIST, l): length = l.ll_length() new_lst = RESLIST.ll_newlist(length) ll_arraycopy(l, new_lst, 0, 0, length) return new_lst +# no oopspec -- the function is inlined by the JIT def ll_len(l): return l.ll_length() @@ -551,6 +554,7 @@ def ll_list_is_true(l): # check if a list is True, allowing for None return bool(l) and l.ll_length() != 0 +# no oopspec -- the function is inlined by the JIT def ll_len_foldable(l): return l.ll_length() @@ -558,6 +562,7 @@ def ll_list_is_true_foldable(l): return bool(l) and ll_len_foldable(l) != 0 +# no oopspec -- the function is inlined by the JIT def ll_append(l, newitem): length = l.ll_length() @@ -588,6 +593,7 @@ ll_arraycopy(l1, l, 0, 0, len1) ll_arraycopy(l2, l, 0, len1, len2) return l +# no oopspec -- the function is inlined by the JIT def ll_insert_nonneg(l, index, newitem): length = l.ll_length() @@ -674,60 +680,69 @@ l.ll_setitem_fast(length_1_i, tmp) i += 1 length_1_i -= 1 +ll_reverse.oopspec = 'list.reverse(l)' def ll_getitem_nonneg(func, l, index): ll_assert(index >= 0, "unexpectedly negative list getitem index") if func is dum_checkidx: if index >= l.ll_length(): raise IndexError - else: - ll_assert(index < l.ll_length(), "list getitem index out of bound") return l.ll_getitem_fast(index) -ll_getitem_nonneg.oopspec = 'list.getitem(l, index)' +# no oopspec -- the function is inlined by the JIT def ll_getitem(func, l, index): - length = l.ll_length() - if index < 0: - index += length if func is dum_checkidx: - if index < 0 or index >= length: - raise IndexError + length = l.ll_length() # common case: 0 <= index < length + if r_uint(index) >= r_uint(length): + # Failed, so either (-length <= index < 0), or we have to raise + # IndexError. Using r_uint, the condition can be rewritten as + # (-length-1 < index), which is (~length < index). + if r_uint(~length) < r_uint(index): + index += length + else: + raise IndexError else: - ll_assert(index >= 0, "negative list getitem index out of bound") - ll_assert(index < length, "list getitem index out of bound") + # We don't want checking, but still want to support index < 0. + # Only call ll_length() if needed. + if index < 0: + index += l.ll_length() + ll_assert(index >= 0, "negative list getitem index out of bound") return l.ll_getitem_fast(index) -ll_getitem.oopspec = 'list.getitem(l, index)' +# no oopspec -- the function is inlined by the JIT -def ll_getitem_foldable_nonneg(func, l, index): - return ll_getitem_nonneg(func, l, index) +def ll_getitem_foldable_nonneg(l, index): + ll_assert(index >= 0, "unexpectedly negative list getitem index") + return l.ll_getitem_fast(index) ll_getitem_foldable_nonneg.oopspec = 'list.getitem_foldable(l, index)' -def ll_getitem_foldable(func, l, index): - return ll_getitem(func, l, index) -ll_getitem_foldable.oopspec = 'list.getitem_foldable(l, index)' +def ll_getitem_foldable(l, index): + if index < 0: + index += l.ll_length() + return ll_getitem_foldable_nonneg(l, index) +# no oopspec -- the function is inlined by the JIT def ll_setitem_nonneg(func, l, index, newitem): ll_assert(index >= 0, "unexpectedly negative list setitem index") if func is dum_checkidx: if index >= l.ll_length(): raise IndexError - else: - ll_assert(index < l.ll_length(), "list setitem index out of bound") l.ll_setitem_fast(index, newitem) -ll_setitem_nonneg.oopspec = 'list.setitem(l, index, newitem)' +# no oopspec -- the function is inlined by the JIT def ll_setitem(func, l, index, newitem): - length = l.ll_length() - if index < 0: - index += length if func is dum_checkidx: - if index < 0 or index >= length: - raise IndexError + length = l.ll_length() + if r_uint(index) >= r_uint(length): # see comments in ll_getitem(). + if r_uint(~length) < r_uint(index): + index += length + else: + raise IndexError else: - ll_assert(index >= 0, "negative list setitem index out of bound") - ll_assert(index < length, "list setitem index out of bound") + if index < 0: + index += l.ll_length() + ll_assert(index >= 0, "negative list getitem index out of bound") l.ll_setitem_fast(index, newitem) -ll_setitem.oopspec = 'list.setitem(l, index, newitem)' +# no oopspec -- the function is inlined by the JIT def ll_delitem_nonneg(func, l, index): ll_assert(index >= 0, "unexpectedly negative list delitem index") @@ -756,7 +771,7 @@ if i < 0: i += length if func is dum_checkidx: - if i < 0 or i >= length: + if r_uint(i) >= r_uint(length): raise IndexError else: ll_assert(i >= 0, "negative list delitem index out of bound") @@ -799,6 +814,7 @@ lst.ll_setitem_fast(j, c) i += 1 j += 1 +# not inlined by the JIT -- contains a loop def ll_extend_with_str_slice_startstop(lst, s, getstrlen, getstritem, start, stop): @@ -824,6 +840,7 @@ lst.ll_setitem_fast(j, c) i += 1 j += 1 +# not inlined by the JIT -- contains a loop def ll_extend_with_str_slice_minusone(lst, s, getstrlen, getstritem): len1 = lst.ll_length() @@ -843,6 +860,7 @@ lst.ll_setitem_fast(j, c) i += 1 j += 1 +# not inlined by the JIT -- contains a loop def ll_extend_with_char_count(lst, char, count): if count <= 0: @@ -859,6 +877,7 @@ while j < newlength: lst.ll_setitem_fast(j, char) j += 1 +# not inlined by the JIT -- contains a loop def ll_listslice_startonly(RESLIST, l1, start): len1 = l1.ll_length() @@ -869,6 +888,7 @@ ll_arraycopy(l1, l, start, 0, newlength) return l ll_listslice_startonly._annenforceargs_ = (None, None, int) +# no oopspec -- the function is inlined by the JIT def ll_listslice_startstop(RESLIST, l1, start, stop): length = l1.ll_length() @@ -881,6 +901,7 @@ l = RESLIST.ll_newlist(newlength) ll_arraycopy(l1, l, start, 0, newlength) return l +# no oopspec -- the function is inlined by the JIT def ll_listslice_minusone(RESLIST, l1): newlength = l1.ll_length() - 1 @@ -888,6 +909,7 @@ l = RESLIST.ll_newlist(newlength) ll_arraycopy(l1, l, 0, 0, newlength) return l +# no oopspec -- the function is inlined by the JIT def ll_listdelslice_startonly(l, start): ll_assert(start >= 0, "del l[start:] with unexpectedly negative start") @@ -958,6 +980,7 @@ return False j += 1 return True +# not inlined by the JIT -- contains a loop def ll_listcontains(lst, obj, eqfn): lng = lst.ll_length() @@ -971,6 +994,7 @@ return True j += 1 return False +# not inlined by the JIT -- contains a loop def ll_listindex(lst, obj, eqfn): lng = lst.ll_length() @@ -984,6 +1008,7 @@ return j j += 1 raise ValueError # can't say 'list.index(x): x not in list' +# not inlined by the JIT -- contains a loop def ll_listremove(lst, obj, eqfn): index = ll_listindex(lst, obj, eqfn) # raises ValueError if obj not in lst @@ -1030,3 +1055,4 @@ i += 1 j += length return res +# not inlined by the JIT -- contains a loop Modified: pypy/branch/rlist-jit/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rpython/test/test_rlist.py (original) +++ pypy/branch/rlist-jit/pypy/rpython/test/test_rlist.py Tue Nov 9 15:51:41 2010 @@ -1420,14 +1420,15 @@ lst2 = [i] lst2.append(42) # mutated list return lst1[i] + lst2[i] - _, _, graph = self.gengraph(f, [int]) + from pypy.annotation import model as annmodel + _, _, graph = self.gengraph(f, [annmodel.SomeInteger(nonneg=True)]) block = graph.startblock lst1_getitem_op = block.operations[-3] # XXX graph fishing lst2_getitem_op = block.operations[-2] func1 = lst1_getitem_op.args[0].value._obj._callable func2 = lst2_getitem_op.args[0].value._obj._callable assert func1.oopspec == 'list.getitem_foldable(l, index)' - assert func2.oopspec == 'list.getitem(l, index)' + assert not hasattr(func2, 'oopspec') class TestOOtype(BaseTestRlist, OORtypeMixin): rlist = oo_rlist From fijal at codespeak.net Tue Nov 9 16:33:51 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 9 Nov 2010 16:33:51 +0100 (CET) Subject: [pypy-svn] r78925 - pypy/release/1.4.x Message-ID: <20101109153351.32622282BE0@codespeak.net> Author: fijal Date: Tue Nov 9 16:33:50 2010 New Revision: 78925 Removed: pypy/release/1.4.x/ Log: Remove this (will branch again) From fijal at codespeak.net Tue Nov 9 16:35:01 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 9 Nov 2010 16:35:01 +0100 (CET) Subject: [pypy-svn] r78926 - pypy/release/1.4.x Message-ID: <20101109153501.E311D282BE3@codespeak.net> Author: fijal Date: Tue Nov 9 16:35:00 2010 New Revision: 78926 Added: pypy/release/1.4.x/ (props changed) - copied from r78925, pypy/trunk/ Log: Branch again for the release From antocuni at codespeak.net Tue Nov 9 18:28:22 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 9 Nov 2010 18:28:22 +0100 (CET) Subject: [pypy-svn] r78932 - pypy/extradoc/logo Message-ID: <20101109172822.A209A282BDD@codespeak.net> Author: antocuni Date: Tue Nov 9 18:28:19 2010 New Revision: 78932 Added: pypy/extradoc/logo/ pypy/extradoc/logo/pypy_fin.jpg (contents, props changed) pypy/extradoc/logo/pypy_fin.svg Log: the pypy logo, by Samuel Reis Added: pypy/extradoc/logo/pypy_fin.jpg ============================================================================== Binary file. No diff available. Added: pypy/extradoc/logo/pypy_fin.svg ============================================================================== --- (empty file) +++ pypy/extradoc/logo/pypy_fin.svg Tue Nov 9 18:28:19 2010 @@ -0,0 +1,108 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + From arigo at codespeak.net Tue Nov 9 18:30:37 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 9 Nov 2010 18:30:37 +0100 (CET) Subject: [pypy-svn] r78933 - in pypy/branch/rlist-jit/pypy: annotation rpython/test Message-ID: <20101109173037.C555F282BE0@codespeak.net> Author: arigo Date: Tue Nov 9 18:30:35 2010 New Revision: 78933 Modified: pypy/branch/rlist-jit/pypy/annotation/description.py pypy/branch/rlist-jit/pypy/annotation/unaryop.py pypy/branch/rlist-jit/pypy/rpython/test/test_rlist.py Log: Handle _immutable_fields=['lst[*]'] already at annotation level. This is done by simply returning, for 'object.lst', an annotation that is an immutable list. It should be enough for rtyping to automatically turn reads from that list into the '*_foldable' variants. (Which are the same as the regular variants, expect for the JIT.) Modified: pypy/branch/rlist-jit/pypy/annotation/description.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/description.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/description.py Tue Nov 9 18:30:35 2010 @@ -636,6 +636,21 @@ return self return None + def maybe_return_immutable_list(self, attr, s_result): + # hack: 'x.lst' where lst is listed in _immutable_fields_ as 'lst[*]' + # should really return an immutable list as a result. Implemented + # by changing the result's annotation (but not, of course, doing an + # actual copy in the rtyper). Tested in pypy.rpython.test.test_rlist, + # test_immutable_list_out_of_instance. + search = '%s[*]' % (attr,) + cdesc = self + while cdesc is not None: + if '_immutable_fields_' in cdesc.classdict: + if search in cdesc.classdict['_immutable_fields_'].value: + return s_result.listdef.offspring() + cdesc = cdesc.basedesc + return s_result # common case + def consider_call_site(bookkeeper, family, descs, args, s_result): from pypy.annotation.model import SomeInstance, SomePBC, s_None if len(descs) == 1: Modified: pypy/branch/rlist-jit/pypy/annotation/unaryop.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/unaryop.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/unaryop.py Tue Nov 9 18:30:35 2010 @@ -615,6 +615,9 @@ if basedef.classdesc.all_enforced_attrs is not None: if attr in basedef.classdesc.all_enforced_attrs: raise HarmlesslyBlocked("get enforced attr") + elif isinstance(s_result, SomeList): + s_result = ins.classdef.classdesc.maybe_return_immutable_list( + attr, s_result) return s_result return SomeObject() getattr.can_only_throw = [] Modified: pypy/branch/rlist-jit/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rpython/test/test_rlist.py (original) +++ pypy/branch/rlist-jit/pypy/rpython/test/test_rlist.py Tue Nov 9 18:30:35 2010 @@ -12,6 +12,7 @@ from pypy.rpython.rint import signed_repr from pypy.objspace.flow.model import Constant, Variable from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin +from pypy.rlib.debug import ll_assert # undo the specialization parameter for n1 in 'get set del'.split(): @@ -1076,7 +1077,13 @@ res = self.interpret(f, [0]) assert res == 1 - py.test.raises(AssertionError, self.interpret, f, [1]) + if self.rlist is ll_rlist: + # on lltype we always get an AssertionError + py.test.raises(AssertionError, self.interpret, f, [1]) + else: + # on ootype we happen to get through the ll_asserts and to + # hit the IndexError from ootype.py + self.interpret_raises(IndexError, f, [1]) def f(x): l = [1] @@ -1121,12 +1128,13 @@ res = self.interpret(f, [0]) assert res == 1 - try: - self.interpret_raises(IndexError, f, [1]) - except (AssertionError,), e: - pass + if self.rlist is ll_rlist: + # on lltype we always get an AssertionError + py.test.raises(AssertionError, self.interpret, f, [1]) else: - assert False + # on ootype we happen to get through the ll_asserts and to + # hit the IndexError from ootype.py + self.interpret_raises(IndexError, f, [1]) def f(x): l = [1] @@ -1163,12 +1171,13 @@ res = self.interpret(f, [0]) assert res == 1 - try: - self.interpret_raises(IndexError, f, [1]) - except (AssertionError,), e: - pass + if self.rlist is ll_rlist: + # on lltype we always get an AssertionError + py.test.raises(AssertionError, self.interpret, f, [1]) else: - assert False + # on ootype we happen to get through the ll_asserts and to + # hit the IndexError from ootype.py + self.interpret_raises(IndexError, f, [1]) def test_charlist_extension_1(self): def f(n): @@ -1327,6 +1336,29 @@ res = self.interpret(f, [2]) assert res == True + def test_immutable_list_out_of_instance(self): + from pypy.translator.simplify import get_funcobj + for immutable_fields in (["a", "b"], ["a", "b", "y[*]"]): + class A(object): + _immutable_fields_ = immutable_fields + class B(A): + pass + def f(i): + b = B() + lst = [i] + lst[0] += 1 + b.y = lst + ll_assert(b.y is lst, "copying when reading out the attr?") + return b.y[0] + res = self.interpret(f, [10]) + assert res == 11 + t, rtyper, graph = self.gengraph(f, [int]) + block = graph.startblock + op = block.operations[-1] + assert op.opname == 'direct_call' + func = get_funcobj(op.args[0].value)._callable + assert ('foldable' in func.func_name) == \ + ("y[*]" in immutable_fields) class TestLLtype(BaseTestRlist, LLRtypeMixin): rlist = ll_rlist From arigo at codespeak.net Tue Nov 9 18:32:54 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 9 Nov 2010 18:32:54 +0100 (CET) Subject: [pypy-svn] r78934 - pypy/branch/rlist-jit/pypy/jit/codewriter Message-ID: <20101109173254.930B5282BE8@codespeak.net> Author: arigo Date: Tue Nov 9 18:32:53 2010 New Revision: 78934 Modified: pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py Log: Kill the special-casing for 'object.lst[n]' where lst is listed in _immutable_fields_ as 'lst[*]'. It should be all nicely handled by annotation in r78933 now. Modified: pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py (original) +++ pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py Tue Nov 9 18:32:53 2010 @@ -38,7 +38,6 @@ def optimize_block(self, block): if block.operations == (): return - self.immutable_arrays = {} self.vable_array_vars = {} self.vable_flags = {} renamings = {} @@ -533,8 +532,6 @@ pure = '_greenfield' else: pure = '_pure' - if immut == "[*]": - self.immutable_arrays[op.result] = True else: pure = '' argname = getattr(v_inst.concretetype.TO, '_gckind', 'gc') @@ -994,7 +991,7 @@ v_index, extraop = self._prepare_list_getset(op, arraydescr, args, 'check_neg_index') extra = getkind(op.result.concretetype)[0] - if pure or args[0] in self.immutable_arrays: + if pure: extra = 'pure_' + extra op = SpaceOperation('getarrayitem_gc_%s' % extra, [args[0], arraydescr, v_index], op.result) From afa at codespeak.net Tue Nov 9 19:16:55 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 19:16:55 +0100 (CET) Subject: [pypy-svn] r78936 - pypy/branch/fast-forward/pypy/annotation/test Message-ID: <20101109181655.90DB35080B@codespeak.net> Author: afa Date: Tue Nov 9 19:16:53 2010 New Revision: 78936 Modified: pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py Log: A failing test about make_sure_not_modified() Modified: pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py (original) +++ pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py Tue Nov 9 19:16:53 2010 @@ -3357,6 +3357,24 @@ # not a constant: both __enter__ and __exit__ have been annotated assert not s.is_constant() + def test_make_sure_not_modified(self): + from pypy.rlib.debug import make_sure_not_modified + + def pycode(consts): + make_sure_not_modified(consts) + def build1(): + return pycode(consts=[1]) + def build2(): + return pycode(consts=[0]) + def fn(): + build1() + build2() + + a = self.RPythonAnnotator() + a.translator.config.translation.list_comprehension_operations = True + a.build_types(fn, []) + assert 0 + def g(n): return [0,1,2,n] From david at codespeak.net Tue Nov 9 19:43:11 2010 From: david at codespeak.net (david at codespeak.net) Date: Tue, 9 Nov 2010 19:43:11 +0100 (CET) Subject: [pypy-svn] r78937 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . test Message-ID: <20101109184311.D5D4D282BEA@codespeak.net> Author: david Date: Tue Nov 9 19:43:09 2010 New Revision: 78937 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Log: Generate block data transfer instruction encoding and refactor machine code generation tests Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Tue Nov 9 19:43:09 2010 @@ -49,11 +49,6 @@ instr = self._encode_reg_list(cond << 28 | 0x8BD << 16, regs) self.write32(instr) - def LDM(self, rn, regs, w=0, cond=cond.AL): - instr = cond << 28 | 0x89 << 20 | w << 21 | (rn & 0xFF) << 16 - instr = self._encode_reg_list(instr, regs) - self.write32(instr) - def BKPT(self, cond=cond.AL): self.write32(cond << 28 | 0x1200070) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Tue Nov 9 19:43:09 2010 @@ -255,8 +255,20 @@ return f -def define_long_mult_div_instructions(name, table): - pass +def define_block_data_func(name, table): + n = (table['op'] & 0x3F) << 20 + def f(self, rn, regs, w=0, cond=cond.AL): + # no R bit for now at bit 15 + instr = (n + | cond << 28 + | 0x1 << 27 + | (w & 0x1) << 21 + | (rn & 0xF) << 16) + instr = self._encode_reg_list(instr, regs) + self.write32(instr) + + return f + def imm_operation(rt, rn, imm): return ((rn & 0xFF) << 16 | (rt & 0xFF) << 12 @@ -282,7 +294,8 @@ (instructions.data_proc_imm, define_data_proc_imm), (instructions.supervisor_and_coproc, define_supervisor_and_coproc), (instructions.multiply, define_multiply_instructions), - (instructions.data_proc_reg_shift_reg, define_data_proc_register_shifted)] + (instructions.data_proc_reg_shift_reg, define_data_proc_register_shifted), + (instructions.block_data, define_block_data_func)] for inss, gen in i_g_map: for key, val in inss.iteritems(): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/instructions.py Tue Nov 9 19:43:09 2010 @@ -104,8 +104,8 @@ 'LDMDB': {'op': 0x11}, 'STMIB': {'op': 0x18}, 'LDMIB': {'op': 0x19}, - 'STM': {'op': 0x4}, - 'LDM': {'op': 0x5}, + #'STM': {'op': 0x4}, + #'LDM': {'op': 0x5}, } branch = { 'B': {'op': 0x20}, Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/support.py Tue Nov 9 19:43:09 2010 @@ -32,3 +32,46 @@ finally: if skip: py.test.skip(msg) + +# generators for asm tests + +def gen_test_function(name, asm, args, kwargs=None, asm_ext=None): + if kwargs is None: + kwargs = {} + if asm_ext is None: + asm_ext = '' + def f(self): + func = getattr(self.cb, name) + func(*args, **kwargs) + try: + f_name = name[:name.index('_')] + except ValueError, e: + f_name = name + self.assert_equal('%s%s %s' % (f_name, asm_ext, asm)) + return f + +def define_test(cls, name, test_case, base_name=None): + import types + if base_name is None: + base_name = '' + templ = 'test_generated_%s_%s' + test_name = templ % (base_name, name) + if hasattr(cls, test_name): + i = 1 + new_test_name = test_name + while hasattr(cls, new_test_name): + new_test_name = '%s_%d' % (test_name, i) + i += 1 + test_name = new_test_name + if not isinstance(test_case, types.FunctionType): + asm, sig = test_case[0:2] + kw_args = None + asm_ext = None + if len(test_case) > 2: + kw_args = test_case[2] + if len(test_case) > 3: + asm_ext = test_case[3] + f = gen_test_function(name, asm, sig, kw_args, asm_ext) + else: + f = test_case + setattr(cls, test_name, f) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_instr_codebuilder.py Tue Nov 9 19:43:09 2010 @@ -2,7 +2,7 @@ from pypy.jit.backend.arm import codebuilder from pypy.jit.backend.arm import conditions from pypy.jit.backend.arm import instructions -from pypy.jit.backend.arm.test.support import requires_arm_as +from pypy.jit.backend.arm.test.support import (requires_arm_as, define_test, gen_test_function) from gen import assemble import py @@ -91,18 +91,6 @@ self.cb.PUSH([reg.value for reg in [r.fp, r.ip, r.lr, r.pc]]) self.assert_equal('PUSH {fp, ip, lr, pc}') - def test_ldm_one_reg(self): - self.cb.LDM(r.sp.value, [r.fp.value]) - self.assert_equal('LDM sp, {fp}') - - def test_ldm_multiple_reg(self): - self.cb.LDM(r.sp.value, [reg.value for reg in [r.fp, r.ip, r.lr]]) - self.assert_equal('LDM sp, {fp, ip, lr}') - - def test_ldm_multiple_reg2(self): - self.cb.LDM(r.sp.value, [reg.value for reg in [r.fp, r.sp, r.pc]]) - self.assert_equal("LDM sp, {fp, sp, pc}") - def test_sub_ri(self): self.cb.SUB_ri(r.r2.value, r.r4.value, 123) self.assert_equal('SUB r2, r4, #123') @@ -149,24 +137,6 @@ def setup_method(self, ffuu_method): self.cb = CodeBuilder() -# XXX refactor this functions - -def build_test(builder, key, value, test_name): - test = builder(key, value) - setattr(TestInstrCodeBuilderForGeneratedInstr, test_name % key, test) - -def gen_test_function(name, asm, args, kwargs=None): - if kwargs is None: - kwargs = {} - def f(self): - func = getattr(self.cb, name) - func(*args, **kwargs) - try: - f_name = name[:name.index('_')] - except ValueError, e: - f_name = name - self.assert_equal(asm % f_name) - return f def gen_test_data_proc_imm_func(name, table): if table['result'] and table['base']: @@ -175,112 +145,97 @@ func(r.r3.value, r.r7.value, 23) self.assert_equal('%s r3, r7, #23' % name[:name.index('_')]) py.test.raises(ValueError, 'func(r.r3.value, r.r7.value, -12)') + return [f] else: - f = gen_test_function(name, '%s r3, #23', [r.r3.value, 23]) - return f + return [('r3, #23', [r.r3.value, 23])] -def gen_test_imm_func(name, table): - return gen_test_function(name, '%s r3, [r7, #23]', [r.r3.value, r.r7.value, 23]) - -def gen_test_reg_func(name, table): - return gen_test_function(name, '%s r3, [r7, r12]', [r.r3.value, r.r7.value, r.r12.value]) +def gen_test_load_store_func(name, table): + if table['imm']: + return [('r3, [r7, #23]', [r.r3.value, r.r7.value, 23]), + ('r3, [r7, #-23]', [r.r3.value, r.r7.value, -23]) + ] + else: + return [('r3, [r7, r12]', [r.r3.value, r.r7.value, r.r12.value])] def gen_test_extra_load_store_func(name, table): if name[-4] == 'D': if name[-2:] == 'rr': - f = gen_test_function(name, '%s r4, [r8, r12]', [r.r4.value, r.r5.value, r.r8.value, r.r12.value]) + return [('r4, [r8, r12]', [r.r4.value, r.r5.value, r.r8.value, r.r12.value])] else: - f = gen_test_function(name, '%s r4, [r8, #223]', [r.r4.value, r.r5.value, r.r8.value, 223]) + return [('r4, [r8, #223]', [r.r4.value, r.r5.value, r.r8.value, 223])] else: if name[-2:] == 'rr': - f = gen_test_function(name, '%s r4, [r5, r12]', [r.r4.value, r.r5.value, r.r12.value]) + return [('r4, [r5, r12]', [r.r4.value, r.r5.value, r.r12.value])] else: - f = gen_test_function(name, '%s r4, [r5, #223]', [r.r4.value, r.r5.value, 223]) + return [('r4, [r5, #223]', [r.r4.value, r.r5.value, 223])] return f -def gen_test_mul_func(name, table): + +def gen_test_multiply_func(name, table): if 'acc' in table and table['acc']: if 'update_flags' in table and table['update_flags']: - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, r.r7.value, r.r12.value, r.r13.value) - func(r.r3.value, r.r7.value, r.r12.value, r.r13.value, s=1) - self.assert_equal(""" - %(name)s r3, r7, r12, r13 - %(name)sS r3, r7, r12, r13 - """ % {'name':name}) + return [ + ('r3, r7, r12, r13', (r.r3.value, r.r7.value, r.r12.value, r.r13.value)), + ('r3, r7, r12, r13', (r.r3.value, r.r7.value, r.r12.value, r.r13.value), {'s':1}, 'S') + ] else: - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, r.r7.value, r.r12.value, r.r13.value) - self.assert_equal("""%(name)s r3, r7, r12, r13 """ % {'name':name}) + return [('r3, r7, r12, r13', (r.r3.value, r.r7.value, r.r12.value, + r.r13.value))] elif 'long' in table and table['long']: - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, r.r13.value, r.r7.value, r.r12.value) - self.assert_equal("""%(name)s r3, r13, r7, r12 """ % {'name':name}) + return [('r3, r13, r7, r12', (r.r3.value, r.r13.value, r.r7.value, r.r12.value))] else: - f = gen_test_function(name, '%s r3, r7, r12 ', [r.r3.value, r.r7.value, r.r12.value]) - return f + return [('r3, r7, r12', (r.r3.value, r.r7.value, r.r12.value))] -def gen_test_data_reg_shift_reg_func(name, table): +def gen_test_data_proc_reg_shift_reg_func(name, table): if name[-2:] == 'rr': - f = gen_test_function(name, '%s r3, r7, r12', [r.r3.value, r.r7.value, r.r12.value]) + return [('r3, r7, r12', [r.r3.value, r.r7.value, r.r12.value])] else: result = 'result' not in table or table['result'] if result: - f = gen_test_function(name, '%s r3, r7, r8, ASR r11', - [r.r3.value, r.r7.value, r.r8.value, r.r11.value], {'shifttype':0x2}) + return [('r3, r7, r8, ASR r11', [r.r3.value, r.r7.value, + r.r8.value, r.r11.value], {'shifttype':0x2})] else: - f = gen_test_function(name, '%s r3, r7, ASR r11', - [r.r3.value, r.r7.value, r.r11.value], {'shifttype':0x2}) - return f + return [('r3, r7, ASR r11', [r.r3.value, r.r7.value, + r.r11.value], {'shifttype':0x2})] -def gen_test_data_reg_func(name, table): +def gen_test_data_proc_func(name, table): op_name = name[:name.index('_')] if name[-2:] == 'ri': - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, r.r7.value, 12) - func(r.r3.value, r.r7.value, 12, s=1) - self.assert_equal(""" -%(name)s r3, r7, #12 -%(name)sS r3, r7, #12""" % {'name': op_name}) - + return [('r3, r7, #12', (r.r3.value, r.r7.value, 12)), + ('r3, r7, #12', (r.r3.value, r.r7.value, 12), {'s':1}, 'S')] elif table['base'] and table['result']: - def f(self): - func = getattr(self.cb, name) - func(r.r3.value, r.r7.value, r.r12.value) - func(r.r3.value, r.r7.value, r.r12.value, s=1) - self.assert_equal("""%(name)s r3, r7, r12 -%(name)sS r3, r7, r12 -""" % {'name':op_name}) + return [('r3, r7, r12', (r.r3.value, r.r7.value, r.r12.value)), + ('r3, r7, r12', (r.r3.value, r.r7.value, r.r12.value), {'s':1}, 'S')] else: - f = gen_test_function(name, '%s r3, r7', [r.r3.value, r.r7.value]) + return [('r3, r7', [r.r3.value, r.r7.value])] - return f +def gen_test_supervisor_and_coproc_func(name, table): + def f(self): + py.test.skip('not used at the moment') + return [f] + +def gen_test_branch_func(name, table): + def f(self): + py.test.skip('not used at the moment') + return [f] +def gen_test_block_data_func(name, table): + tests = [] + for c,v in [('EQ', conditions.EQ), ('LE', conditions.LE), ('AL', conditions.AL)]: + for regs in range(16): + asm = 'r3, {%s}' % ','.join(['r%d' % i for i in range(regs+1)]) + tests.append((asm, (r.r3.value, range(regs+1)))) + return tests def build_tests(): + cls = TestInstrCodeBuilderForGeneratedInstr test_name = 'test_generated_%s' - for key, value in instructions.load_store.iteritems(): - if value['imm']: - f = gen_test_imm_func - else: - f = gen_test_reg_func - build_test(f, key, value, test_name) - - for key, value in instructions.extra_load_store.iteritems(): - build_test(gen_test_extra_load_store_func, key, value, 'test_extra_load_store_%s' % test_name) - - for key, value, in instructions.data_proc.iteritems(): - build_test(gen_test_data_reg_func, key, value, test_name) - - for key, value, in instructions.data_proc_reg_shift_reg.iteritems(): - build_test(gen_test_data_reg_shift_reg_func, key, value, test_name) - - for key, value, in instructions.data_proc_imm.iteritems(): - build_test(gen_test_data_proc_imm_func, key, value, test_name) - - for key, value, in instructions.multiply.iteritems(): - build_test(gen_test_mul_func, key, value, test_name) - + ins = [k for k in instructions.__dict__.keys() if not k.startswith('__')] + for name in ins: + try: + func = globals()['gen_test_%s_func' % name] + except KeyError: + print 'No test generator for %s instructions' % name + continue + for key, value in getattr(instructions, name).iteritems(): + for test_case in func(key, value): + define_test(cls, key, test_case, name) build_tests() From afa at codespeak.net Tue Nov 9 19:43:54 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 19:43:54 +0100 (CET) Subject: [pypy-svn] r78938 - pypy/branch/fast-forward/pypy/module/cpyext Message-ID: <20101109184354.4AA09282BEB@codespeak.net> Author: afa Date: Tue Nov 9 19:43:52 2010 New Revision: 78938 Modified: pypy/branch/fast-forward/pypy/module/cpyext/stubs.py Log: This function is now implemented Modified: pypy/branch/fast-forward/pypy/module/cpyext/stubs.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/stubs.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/stubs.py Tue Nov 9 19:43:52 2010 @@ -1861,14 +1861,6 @@ """ raise NotImplementedError - at cpython_api([PyObject], rffi.LONGLONG, error=-1) -def PyInt_AsUnsignedLongLongMask(space, io): - """Will first attempt to cast the object to a PyIntObject or - PyLongObject, if it is not already one, and then return its value as - unsigned long long, without checking for overflow. - """ - raise NotImplementedError - @cpython_api([], lltype.Signed, error=CANNOT_FAIL) def PyInt_GetMax(space, ): """ From fijal at codespeak.net Tue Nov 9 20:36:21 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 9 Nov 2010 20:36:21 +0100 (CET) Subject: [pypy-svn] r78939 - pypy/trunk/pypy/rpython/memory/gc Message-ID: <20101109193621.4A27E282BE9@codespeak.net> Author: fijal Date: Tue Nov 9 20:36:19 2010 New Revision: 78939 Modified: pypy/trunk/pypy/rpython/memory/gc/inspector.py Log: add() was adding to self.pending, which was already something else Modified: pypy/trunk/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/inspector.py Tue Nov 9 20:36:19 2010 @@ -179,9 +179,9 @@ _hd_add_root) self.gc._heap_dumper = None pendingroots = self.pending - self.pending = AddressStack() self.walk(pendingroots) pendingroots.delete() + self.pending = AddressStack() self.write_marker() def walk(self, pending): From hakanardo at codespeak.net Tue Nov 9 22:00:10 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Tue, 9 Nov 2010 22:00:10 +0100 (CET) Subject: [pypy-svn] r78940 - in pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test Message-ID: <20101109210010.019A5282BE8@codespeak.net> Author: hakanardo Date: Tue Nov 9 22:00:06 2010 New Revision: 78940 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/fficall.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Log: Recreate the optimizer after the preamble to get full controll over what status information probagates fron the preamble to the loop. Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py Tue Nov 9 22:00:06 2010 @@ -5,7 +5,7 @@ from pypy.jit.metainterp.optimizeopt.heap import OptHeap from pypy.jit.metainterp.optimizeopt.fficall import OptFfiCall from pypy.jit.metainterp.optimizeopt.string import OptString -from pypy.jit.metainterp.optimizeopt.unroll import OptUnroll +from pypy.jit.metainterp.optimizeopt.unroll import optimize_unroll def optimize_loop_1(metainterp_sd, loop, unroll=True): """Optimize loop.operations to remove internal overheadish operations. @@ -19,11 +19,12 @@ OptFfiCall(), ] if unroll: - optimizations.insert(0, OptUnroll()) opt_str.enabled = False # FIXME: Workaround to disable string optimisation # during preamble but to keep it during the loop - optimizer = Optimizer(metainterp_sd, loop, optimizations) - optimizer.propagate_all_forward() + optimize_unroll(metainterp_sd, loop, optimizations) + else: + optimizer = Optimizer(metainterp_sd, loop, optimizations) + optimizer.propagate_all_forward() def optimize_bridge_1(metainterp_sd, bridge): """The same, but for a bridge. """ Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/fficall.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/fficall.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/fficall.py Tue Nov 9 22:00:06 2010 @@ -67,6 +67,10 @@ def __init__(self): self.funcinfo = None + def reconstruct_for_next_iteration(self): + return OptFfiCall() + # FIXME: Should any status be saved for next iteration? + def begin_optimization(self, funcval, op): self.rollback_maybe() self.funcinfo = FuncInfo(funcval, self.optimizer.cpu, op) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py Tue Nov 9 22:00:06 2010 @@ -23,6 +23,15 @@ self.lazy_setfields = {} self.lazy_setfields_descrs = [] # keys (at least) of previous dict + def reconstruct_for_next_iteration(self): + self.force_all_lazy_setfields() + assert not self.lazy_setfields_descrs + assert not self.lazy_setfields + new = OptHeap() + new.cached_fields = self.cached_fields + new.cached_arrayitems = self.cached_arrayitems + return new + def clean_caches(self): self.cached_fields.clear() self.cached_arrayitems.clear() @@ -149,9 +158,6 @@ self.clean_caches() - def force_at_end_of_preamble(self): - self.force_all_lazy_setfields() - def turned_constant(self, value): assert value.is_constant() newvalue = self.getvalue(value.box) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py Tue Nov 9 22:00:06 2010 @@ -13,6 +13,10 @@ self.posponedop = None self.nextop = None + def reconstruct_for_next_iteration(self): + assert self.posponedop is None + return self + def propagate_forward(self, op): if op.is_ovf(): self.posponedop = op Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Tue Nov 9 22:00:06 2010 @@ -199,6 +199,11 @@ def turned_constant(self, value): pass + def reconstruct_for_next_iteration(self): + #return self.__class__() + raise NotImplementedError + + class Optimizer(Optimization): def __init__(self, metainterp_sd, loop, optimizations=None): @@ -214,6 +219,8 @@ self.producer = {} self.pendingfields = [] self.posponedop = None + self.exception_might_have_happened = False + self.newoperations = [] if optimizations: self.first_optimization = optimizations[0] @@ -233,6 +240,28 @@ self.resumedata_memo = resume.ResumeDataLoopMemo(self.metainterp_sd) for o in self.optimizations: o.force_at_end_of_preamble() + + def reconstruct_for_next_iteration(self): + optimizations = [o.reconstruct_for_next_iteration() for o in + self.optimizations] + optimizations = self.optimizations + new = Optimizer(self.metainterp_sd, self.loop, optimizations) + new.values = self.values + new.interned_refs = self.interned_refs + new.bool_boxes = self.bool_boxes + new.loop_invariant_results = self.loop_invariant_results + new.pure_operations = self.pure_operations + new.producer = self.producer + assert self.posponedop is None + + # FIXME: HACK!! Add a reconstruct_for_next_iteration to + # the values instead and reconstruct them in the same manner. + # That should also give us a clean solution for enabling + # OptString in the preamble that forces it's virtuals before + # the loop + for v in new.values.values(): + v.optimizer = new + return new def turned_constant(self, value): for o in self.optimizations: @@ -322,8 +351,6 @@ return CVAL_ZERO def propagate_all_forward(self): - self.exception_might_have_happened = False - self.newoperations = [] self.i = 0 while self.i < len(self.loop.operations): op = self.loop.operations[self.i] Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py Tue Nov 9 22:00:06 2010 @@ -9,6 +9,9 @@ """Rewrite operations into equivalent, cheaper operations. This includes already executed operations and constants. """ + + def reconstruct_for_next_iteration(self): + return self def propagate_forward(self, op): args = self.optimizer.make_args_key(op) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py Tue Nov 9 22:00:06 2010 @@ -366,6 +366,10 @@ class OptString(optimizer.Optimization): "Handling of strings and unicodes." enabled = True + + def reconstruct_for_next_iteration(self): + self.enabled = True + return self def make_vstring_plain(self, box, source_op, mode): vvalue = VStringPlainValue(self.optimizer, box, source_op, mode) @@ -656,8 +660,6 @@ else: self.emit_operation(op) - def force_at_end_of_preamble(self): - self.enabled = True optimize_ops = _findall(OptString, 'optimize_') Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Tue Nov 9 22:00:06 2010 @@ -3,37 +3,51 @@ from pypy.jit.metainterp.compile import ResumeGuardDescr from pypy.jit.metainterp.resume import Snapshot -class OptUnroll(Optimization): +# FXIME: Introduce some VirtualOptimizer super class instead + +def optimize_unroll(metainterp_sd, loop, optimizations): + opt = UnrollOptimizer(metainterp_sd, loop, optimizations) + opt.propagate_all_forward() + +class UnrollOptimizer(Optimization): """Unroll the loop into two iterations. The first one will become the preamble or entry bridge (don't think there is a distinction anymore)""" - def setup(self): + def __init__(self, metainterp_sd, loop, optimizations): + self.optimizer = Optimizer(metainterp_sd, loop, optimizations) self.cloned_operations = [] for op in self.optimizer.loop.operations: self.cloned_operations.append(op.clone()) - def propagate_forward(self, op): + def propagate_all_forward(self): + loop = self.optimizer.loop + jumpop = loop.operations[-1] + if jumpop.getopnum() == rop.JUMP: + loop.operations = loop.operations[:-1] + else: + loopop = None + + self.optimizer.propagate_all_forward() + - if op.getopnum() == rop.JUMP: - self.optimizer.force_at_end_of_preamble() - loop = self.optimizer.loop - assert op.getdescr() is loop.token + if jumpop: + assert jumpop.getdescr() is loop.token loop.preamble.operations = self.optimizer.newoperations - self.optimizer.newoperations = [] - jump_args = op.getarglist() - op.initarglist([]) - # Exceptions not caught in one iteration should not propagate to the next - self.optimizer.exception_might_have_happened = False + + self.optimizer = self.optimizer.reconstruct_for_next_iteration() + + jump_args = jumpop.getarglist() + jumpop.initarglist([]) inputargs = self.inline(self.cloned_operations, loop.inputargs, jump_args) loop.inputargs = inputargs jmp = ResOperation(rop.JUMP, loop.inputargs[:], None) jmp.setdescr(loop.token) loop.preamble.operations.append(jmp) - else: - self.emit_operation(op) + + loop.operations = self.optimizer.newoperations def inline(self, loop_operations, loop_args, jump_args): self.argmap = argmap = {} @@ -62,6 +76,7 @@ else: args = newop.getarglist() newop.initarglist([self.inline_arg(a) for a in args]) + #print 'P:', newop if newop.result: old_result = newop.result @@ -71,8 +86,8 @@ descr = newop.getdescr() if isinstance(descr, ResumeGuardDescr): descr.rd_snapshot = self.inline_snapshot(descr.rd_snapshot) - - self.emit_operation(newop) + + self.optimizer.first_optimization.propagate_forward(newop) # Remove jump to make sure forced code are placed before it newoperations = self.optimizer.newoperations @@ -83,7 +98,7 @@ boxes_created_this_iteration = {} jumpargs = jmp.getarglist() - # FIXME: Should also loop over operations added by forcing things in this loop + # FIXME: Should also loop over operations added by forcing things in this loop for op in newoperations: #print 'E: ', str(op) boxes_created_this_iteration[op.result] = True Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Tue Nov 9 22:00:06 2010 @@ -228,6 +228,9 @@ class OptVirtualize(optimizer.Optimization): "Virtualize objects until they escape." + def reconstruct_for_next_iteration(self): + return self + def make_virtual(self, known_class, box, source_op=None): vvalue = VirtualValue(self.optimizer, known_class, box, source_op) self.make_equal_to(box, vvalue) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Tue Nov 9 22:00:06 2010 @@ -3994,6 +3994,15 @@ jump(p0, p1, p3, p5, p7, p8, p14, 1) """ self.optimize_loop(ops, expected) + + def test_inputargs_added_by_forcing_jumpargs(self): + # FXIME: Can this occur? + ops = """ + [p0, p1, pinv] + i1 = getfield_gc(pinv, descr=valuedescr) + p2 = new_with_vtable(ConstClass(node_vtable)) + setfield_gc(p2, i1, descr=nextdescr) + """ # ---------- def optimize_strunicode_loop(self, ops, optops, preamble=None): From afa at codespeak.net Tue Nov 9 22:10:07 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 22:10:07 +0100 (CET) Subject: [pypy-svn] r78941 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101109211007.32C6F282BE9@codespeak.net> Author: afa Date: Tue Nov 9 22:10:05 2010 New Revision: 78941 Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py pypy/branch/fast-forward/pypy/objspace/std/typetype.py Log: Hide the type.__abstractmethods__ descriptor. Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_typeobject.py Tue Nov 9 22:10:05 2010 @@ -110,6 +110,7 @@ raises(TypeError, X) del X.__abstractmethods__ X() + raises(AttributeError, getattr, type, "__abstractmethods__") def test_call_type(self): assert type(42) is int Modified: pypy/branch/fast-forward/pypy/objspace/std/typetype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/typetype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/typetype.py Tue Nov 9 22:10:05 2010 @@ -208,11 +208,14 @@ def descr_get___abstractmethods__(space, w_type): w_type = _check(space, w_type) - try: - return w_type.dict_w["__abstractmethods__"] - except KeyError: - raise OperationError(space.w_AttributeError, - space.wrap("__abstractmethods__")) + # type itself has an __abstractmethods__ descriptor (this). Don't return it + if not space.is_w(w_type, space.w_type): + try: + return w_type.dict_w["__abstractmethods__"] + except KeyError: + pass + raise OperationError(space.w_AttributeError, + space.wrap("__abstractmethods__")) def descr_set___abstractmethods__(space, w_type, w_new): w_type = _check(space, w_type) From afa at codespeak.net Tue Nov 9 22:18:16 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 22:18:16 +0100 (CET) Subject: [pypy-svn] r78942 - in pypy/branch/fast-forward/pypy/module/imp: . test Message-ID: <20101109211816.0C416282BEC@codespeak.net> Author: afa Date: Tue Nov 9 22:18:15 2010 New Revision: 78942 Modified: pypy/branch/fast-forward/pypy/module/imp/importing.py pypy/branch/fast-forward/pypy/module/imp/test/test_import.py Log: __import__ keyword arguments Modified: pypy/branch/fast-forward/pypy/module/imp/importing.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/imp/importing.py (original) +++ pypy/branch/fast-forward/pypy/module/imp/importing.py Tue Nov 9 22:18:15 2010 @@ -91,8 +91,9 @@ def check_sys_modules_w(space, modulename): return space.finditem_str(space.sys.get('modules'), modulename) -def importhook(space, modulename, w_globals=None, +def importhook(space, name, w_globals=None, w_locals=None, w_fromlist=None, level=-1): + modulename = name space.timer.start_name("importhook", modulename) if not modulename and level < 0: raise OperationError( Modified: pypy/branch/fast-forward/pypy/module/imp/test/test_import.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/imp/test/test_import.py (original) +++ pypy/branch/fast-forward/pypy/module/imp/test/test_import.py Tue Nov 9 22:18:15 2010 @@ -186,6 +186,9 @@ assert pkg == sys.modules.get('pkg') assert pkg.a == sys.modules.get('pkg.a') + def test_import_keywords(self): + __import__(name='sys', level=0) + def test_import_badcase(self): def missing(name): try: From fijal at codespeak.net Tue Nov 9 22:39:45 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 9 Nov 2010 22:39:45 +0100 (CET) Subject: [pypy-svn] r78943 - pypy/trunk/pypy/rpython/memory/gc Message-ID: <20101109213945.A6EF1282BE9@codespeak.net> Author: fijal Date: Tue Nov 9 22:39:44 2010 New Revision: 78943 Modified: pypy/trunk/pypy/rpython/memory/gc/inspector.py Log: Revert 78939, it still doesn't work. Add correct casts Modified: pypy/trunk/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/inspector.py Tue Nov 9 22:39:44 2010 @@ -101,7 +101,8 @@ # ---------- raw_os_write = rffi.llexternal(underscore_on_windows+'write', - [rffi.INT, llmemory.Address, rffi.SIZE_T], + [rffi.INT, rffi.CArrayPtr(lltype.Signed), + rffi.SIZE_T], rffi.SIZE_T, sandboxsafe=True, _nowrapper=True) @@ -130,7 +131,7 @@ if self.buf_count > 0: bytes = self.buf_count * rffi.sizeof(rffi.LONG) count = raw_os_write(self.fd, - rffi.cast(llmemory.Address, self.writebuffer), + self.writebuffer, rffi.cast(rffi.SIZE_T, bytes)) if rffi.cast(lltype.Signed, count) != bytes: raise OSError(rposix.get_errno(), "raw_os_write failed") @@ -139,7 +140,7 @@ def write(self, value): x = self.buf_count - self.writebuffer[x] = value + self.writebuffer[x] = llmemory.raw_malloc_usage(value) x += 1 self.buf_count = x if x == self.BUFSIZE: @@ -179,9 +180,9 @@ _hd_add_root) self.gc._heap_dumper = None pendingroots = self.pending + self.pending = AddressStack() self.walk(pendingroots) pendingroots.delete() - self.pending = AddressStack() self.write_marker() def walk(self, pending): From afa at codespeak.net Tue Nov 9 22:43:05 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 22:43:05 +0100 (CET) Subject: [pypy-svn] r78944 - in pypy/branch/fast-forward/pypy/module/__builtin__: . test Message-ID: <20101109214305.8ABF8282BEF@codespeak.net> Author: afa Date: Tue Nov 9 22:43:04 2010 New Revision: 78944 Modified: pypy/branch/fast-forward/pypy/module/__builtin__/app_inspect.py pypy/branch/fast-forward/pypy/module/__builtin__/test/test_builtin.py Log: dir() is supposed to fail on broken modules with invalid __dict__ Modified: pypy/branch/fast-forward/pypy/module/__builtin__/app_inspect.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/app_inspect.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/app_inspect.py Tue Nov 9 22:43:04 2010 @@ -64,7 +64,7 @@ if isinstance(obj, types.ModuleType): try: - result = list(obj.__dict__.keys()) + result = list(obj.__dict__) result.sort() return result except AttributeError: Modified: pypy/branch/fast-forward/pypy/module/__builtin__/test/test_builtin.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__builtin__/test/test_builtin.py (original) +++ pypy/branch/fast-forward/pypy/module/__builtin__/test/test_builtin.py Tue Nov 9 22:43:04 2010 @@ -104,6 +104,12 @@ return 'a' # not a list! raises(TypeError, eval, "dir()", {}, C()) + def test_dir_broken_module(self): + import types + class Foo(types.ModuleType): + __dict__ = 8 + raises(TypeError, dir, Foo("foo")) + def test_format(self): assert format(4) == "4" assert format(10, "o") == "12" From antocuni at codespeak.net Tue Nov 9 22:47:13 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 9 Nov 2010 22:47:13 +0100 (CET) Subject: [pypy-svn] r78945 - pypy/extradoc/planning/hg-migration Message-ID: <20101109214713.E4BB7282BF0@codespeak.net> Author: antocuni Date: Tue Nov 9 22:47:12 2010 New Revision: 78945 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: add bea Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Tue Nov 9 22:47:12 2010 @@ -1,4 +1,4 @@ -?# -*- encoding: utf-8 -*- +# -*- encoding: utf-8 -*- # # map codespeak usernames to mercurial usernames # @@ -35,7 +35,7 @@ guido=Guido Wesdorp ludal=Ludovic Aubry jlg=Jakub Gustak -bea=Beatrice During +bea=Beatrice During hakanardo=Hakan Ardo niko=Niko Matsakis alex=Alex Martelli From fijal at codespeak.net Tue Nov 9 22:55:22 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 9 Nov 2010 22:55:22 +0100 (CET) Subject: [pypy-svn] r78946 - pypy/trunk/pypy/translator/c/test Message-ID: <20101109215522.F3658282B9C@codespeak.net> Author: fijal Date: Tue Nov 9 22:55:21 2010 New Revision: 78946 Modified: pypy/trunk/pypy/translator/c/test/test_newgc.py Log: Make the test actually test *something* Modified: pypy/trunk/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/trunk/pypy/translator/c/test/test_newgc.py (original) +++ pypy/trunk/pypy/translator/c/test/test_newgc.py Tue Nov 9 22:55:21 2010 @@ -1066,7 +1066,9 @@ filename_dump = str(udir.join('test_dump_rpy_heap')) def define_dump_rpy_heap(self): - U = lltype.GcStruct('U', ('x', lltype.Signed)) + U = lltype.GcForwardReference() + U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)), + ('x', lltype.Signed))) S = lltype.GcStruct('S', ('u', lltype.Ptr(U))) A = lltype.GcArray(lltype.Ptr(S)) filename = self.filename_dump @@ -1074,11 +1076,16 @@ def fn(): s = lltype.malloc(S) s.u = lltype.malloc(U) + s.u.next = lltype.malloc(U) + s.u.next.next = lltype.malloc(U) a = lltype.malloc(A, 1000) s2 = lltype.malloc(S) # fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666) rgc.dump_rpy_heap(fd) + keepalive_until_here(s2) + keepalive_until_here(s) + keepalive_until_here(a) os.close(fd) return 0 @@ -1087,8 +1094,7 @@ def test_dump_rpy_heap(self): self.run("dump_rpy_heap") assert os.path.exists(self.filename_dump) - assert os.path.getsize(self.filename_dump) > 0 # minimal test - + assert os.path.getsize(self.filename_dump) > 64 class TestSemiSpaceGC(TestUsingFramework, snippet.SemiSpaceGCTestDefines): gcpolicy = "semispace" From afa at codespeak.net Tue Nov 9 23:36:50 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 23:36:50 +0100 (CET) Subject: [pypy-svn] r78947 - pypy/branch/fast-forward/lib-python Message-ID: <20101109223650.5578C282B9C@codespeak.net> Author: afa Date: Tue Nov 9 23:36:48 2010 New Revision: 78947 Modified: pypy/branch/fast-forward/lib-python/TODO Log: Another TODO item, not sure how to do it. Modified: pypy/branch/fast-forward/lib-python/TODO ============================================================================== --- pypy/branch/fast-forward/lib-python/TODO (original) +++ pypy/branch/fast-forward/lib-python/TODO Tue Nov 9 23:36:48 2010 @@ -75,6 +75,13 @@ - "Shorter float representation": copy dtoa.c from CPython and use it to format/parse floats. Enable this with a translation option. +- Fix usage of __cmp__ in subclasses:: + + class badint(int): + def __cmp__(self, other): + raise RuntimeError + raises(RuntimeError, cmp, 0, badint(1)) + More difficult issues --------------------- From afa at codespeak.net Tue Nov 9 23:43:50 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 9 Nov 2010 23:43:50 +0100 (CET) Subject: [pypy-svn] r78948 - in pypy/branch/fast-forward/pypy/module/pyexpat: . test Message-ID: <20101109224350.57F27282BEA@codespeak.net> Author: afa Date: Tue Nov 9 23:43:48 2010 New Revision: 78948 Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py Log: Raise TypeError instead of OverflowError Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py (original) +++ pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py Tue Nov 9 23:43:48 2010 @@ -530,7 +530,7 @@ def get_buffer_size(space, self): return space.wrap(self.buffer_size) def set_buffer_size(space, self, w_value): - value = space.int_w(w_value) + value = space.getindex_w(w_value, space.w_TypeError) if value <= 0: raise OperationError(space.w_ValueError, space.wrap( "buffer_size must be greater than zero")) Modified: pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py (original) +++ pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py Tue Nov 9 23:43:48 2010 @@ -17,7 +17,8 @@ raises(pyexpat.ExpatError, p.Parse, "3") def test_set_buffersize(self): - import pyexpat + import pyexpat, sys p = pyexpat.ParserCreate() p.buffer_size = 150 assert p.buffer_size == 150 + raises(TypeError, setattr, p, 'buffer_size', sys.maxint + 1) From fijal at codespeak.net Tue Nov 9 23:45:56 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 9 Nov 2010 23:45:56 +0100 (CET) Subject: [pypy-svn] r78949 - pypy/trunk/pypy/jit/backend/x86/test Message-ID: <20101109224556.8BDE1282BF1@codespeak.net> Author: fijal Date: Tue Nov 9 23:45:55 2010 New Revision: 78949 Modified: pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py Log: A day of wonders - make this test test anything Modified: pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py (original) +++ pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py Tue Nov 9 23:45:55 2010 @@ -2,6 +2,7 @@ from pypy.tool.udir import udir from pypy.rlib.jit import JitDriver, OPTIMIZER_FULL, unroll_parameters from pypy.rlib.jit import PARAMETERS, dont_look_inside +from pypy.rlib.jit import hint from pypy.jit.metainterp.jitprof import Profiler from pypy.jit.backend.detect_cpu import getcpuclass from pypy.jit.backend.test.support import CCompiledMixin @@ -9,6 +10,7 @@ from pypy.translator.translator import TranslationContext from pypy.jit.backend.x86.arch import IS_X86_32, IS_X86_64 from pypy.config.translationoption import DEFL_GC +from pypy.rlib import rgc class TestTranslationX86(CCompiledMixin): CPUClass = getcpuclass() @@ -87,7 +89,7 @@ def main(i, j): return f(i, j) + libffi_stuff(i, j) expected = f(40, -49) - res = self.meta_interp(f, [40, -49]) + res = self.meta_interp(main, [40, -49]) assert res == expected def test_direct_assembler_call_translates(self): From afa at codespeak.net Wed Nov 10 08:26:32 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 10 Nov 2010 08:26:32 +0100 (CET) Subject: [pypy-svn] r78952 - in pypy/branch/fast-forward/pypy/module/pyexpat: . test Message-ID: <20101110072632.C35B2282B90@codespeak.net> Author: afa Date: Wed Nov 10 08:26:29 2010 New Revision: 78952 Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py Log: Really implement interning of pyexpat strings Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py (original) +++ pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py Wed Nov 10 08:26:29 2010 @@ -62,32 +62,37 @@ kw['compilation_info'] = eci return rffi.llexternal(*a, **kw) +INTERNED_CCHARP = "INTERNED" + HANDLERS = dict( - StartElementHandler = [rffi.CCHARP, rffi.CCHARPP], - EndElementHandler = [rffi.CCHARP], - ProcessingInstructionHandler = [rffi.CCHARP, rffi.CCHARP], + StartElementHandler = [INTERNED_CCHARP, rffi.CCHARPP], + EndElementHandler = [INTERNED_CCHARP], + ProcessingInstructionHandler = [INTERNED_CCHARP, INTERNED_CCHARP], CharacterDataHandler = [rffi.CCHARP, rffi.INT], - UnparsedEntityDeclHandler = [rffi.CCHARP] * 5, - NotationDeclHandler = [rffi.CCHARP] * 4, - StartNamespaceDeclHandler = [rffi.CCHARP, rffi.CCHARP], - EndNamespaceDeclHandler = [rffi.CCHARP], + UnparsedEntityDeclHandler = [INTERNED_CCHARP] * 5, + NotationDeclHandler = [INTERNED_CCHARP] * 4, + StartNamespaceDeclHandler = [INTERNED_CCHARP, INTERNED_CCHARP], + EndNamespaceDeclHandler = [INTERNED_CCHARP], CommentHandler = [rffi.CCHARP], StartCdataSectionHandler = [], EndCdataSectionHandler = [], DefaultHandler = [rffi.CCHARP, rffi.INT], DefaultHandlerExpand = [rffi.CCHARP, rffi.INT], NotStandaloneHandler = [], - ExternalEntityRefHandler = [rffi.CCHARP] * 4, - StartDoctypeDeclHandler = [rffi.CCHARP, rffi.CCHARP, rffi.CCHARP, rffi.INT], + ExternalEntityRefHandler = [rffi.CCHARP] + [INTERNED_CCHARP] * 3, + StartDoctypeDeclHandler = [INTERNED_CCHARP, INTERNED_CCHARP, + INTERNED_CCHARP, rffi.INT], EndDoctypeDeclHandler = [], - EntityDeclHandler = [rffi.CCHARP, rffi.INT, rffi.CCHARP, rffi.INT, - rffi.CCHARP, rffi.CCHARP, rffi.CCHARP, rffi.CCHARP], + EntityDeclHandler = [INTERNED_CCHARP, rffi.INT, rffi.CCHARP, rffi.INT, + INTERNED_CCHARP, INTERNED_CCHARP, INTERNED_CCHARP, + INTERNED_CCHARP], XmlDeclHandler = [rffi.CCHARP, rffi.CCHARP, rffi.INT], - ElementDeclHandler = [rffi.CCHARP, lltype.Ptr(XML_Content)], - AttlistDeclHandler = [rffi.CCHARP] * 4 + [rffi.INT], + ElementDeclHandler = [INTERNED_CCHARP, lltype.Ptr(XML_Content)], + AttlistDeclHandler = [INTERNED_CCHARP, INTERNED_CCHARP, + rffi.CCHARP, rffi.CCHARP, rffi.INT], ) if XML_COMBINED_VERSION >= 19504: - HANDLERS['SkippedEntityHandler'] = [rffi.CCHARP, rffi.INT] + HANDLERS['SkippedEntityHandler'] = [INTERNED_CCHARP, rffi.INT] NB_HANDLERS = len(HANDLERS) class Storage: @@ -128,6 +133,8 @@ warg_names = ['w_arg%d' % (i,) for i in range(len(params))] converters = [] + real_params = [] + for i, ARG in enumerate(params): # Some custom argument conversions if name == "StartElementHandler" and i == 1: @@ -146,6 +153,10 @@ elif ARG == rffi.CCHARP: converters.append( 'w_arg%d = parser.w_convert_charp(space, arg%d)' % (i, i)) + elif ARG == INTERNED_CCHARP: + converters.append( + 'w_arg%d = parser.w_convert_interned(space, arg%d)' % (i, i)) + ARG = rffi.CCHARP elif ARG == lltype.Ptr(XML_Content): converters.append( 'w_arg%d = parser.w_convert_model(space, arg%d)' % (i, i)) @@ -154,6 +165,7 @@ else: converters.append( 'w_arg%d = space.wrap(arg%d)' % (i, i)) + real_params.append(ARG) converters = '; '.join(converters) args = ', '.join(arg_names) @@ -208,7 +220,7 @@ c_name = 'XML_Set' + name callback_type = lltype.Ptr(lltype.FuncType( - [rffi.VOIDP] + params, result_type)) + [rffi.VOIDP] + real_params, result_type)) func = expat_external(c_name, [XML_Parser, callback_type], lltype.Void) SETTERS[name] = (index, func, callback) @@ -340,6 +352,21 @@ else: return space.w_None + def w_convert_interned(self, space, data): + if not data: + return space.w_None + w_data = self.w_convert_charp(space, data) + if not self.w_intern: + return w_data + + try: + return space.getitem(self.w_intern, w_data) + except OperationError, e: + if not e.match(space, space.w_KeyError): + raise + space.setitem(self.w_intern, w_data, w_data) + return w_data + def w_convert_charp_n(self, space, data, length): ll_length = rffi.cast(lltype.Signed, length) if data: @@ -471,7 +498,7 @@ else: encoding = space.str_w(w_encoding) - parser = W_XMLParserType(encoding, 0, space.newdict(), + parser = W_XMLParserType(encoding, 0, self.w_intern, _from_external_entity=True) parser.itself = XML_ExternalEntityParserCreate(self.itself, context, encoding) @@ -548,7 +575,10 @@ self.buffer_w = None def get_intern(space, self): - return self.w_intern + if self.w_intern: + return self.w_intern + else: + return space.w_None def bool_property(name, cls, doc=None): @@ -626,8 +656,12 @@ space.wrap('ParserCreate() argument 2 must be string or None,' ' not %s' % (type_name,))) + # Explicitly passing None means no interning is desired. + # Not passing anything means that a new dictionary is used. if w_intern is None: w_intern = space.newdict() + elif space.is_w(w_intern, space.w_None): + w_intern = None parser = W_XMLParserType(encoding, namespace_separator, w_intern) if not parser.itself: Modified: pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py (original) +++ pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py Wed Nov 10 08:26:29 2010 @@ -16,6 +16,15 @@ raises(pyexpat.ExpatError, p.Parse, "3") + def test_intern(self): + import pyexpat + p = pyexpat.ParserCreate() + def f(*args): pass + p.StartElementHandler = f + p.EndElementHandler = f + p.Parse("") + assert len(p.intern) == 1 + def test_set_buffersize(self): import pyexpat, sys p = pyexpat.ParserCreate() From fijal at codespeak.net Wed Nov 10 09:48:40 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 10 Nov 2010 09:48:40 +0100 (CET) Subject: [pypy-svn] r78953 - pypy/trunk/pypy/doc Message-ID: <20101110084840.9578136C220@codespeak.net> Author: fijal Date: Wed Nov 10 09:48:38 2010 New Revision: 78953 Added: pypy/trunk/pypy/doc/release-1.4.0.txt Log: start drafting a release announcement Added: pypy/trunk/pypy/doc/release-1.4.0.txt ============================================================================== --- (empty file) +++ pypy/trunk/pypy/doc/release-1.4.0.txt Wed Nov 10 09:48:38 2010 @@ -0,0 +1,20 @@ +======================= +PyPy 1.3: Self hosting +======================= + +Hello. + +We're pleased to announce release of PyPy 1.4. This is a major breaktrhough +in our long journey, since PyPy 1.4 is the first PyPy release that can translate +itself faster than CPython. Since today, we're planning to be starting using +PyPy for our own development. + +Among other features, this release includes numerous performance improvements +(which made fast self-hosting possible), 64jit backend as well as serious +stabilization. As of now, we can consider 32bit version of PyPy stable enough +to run in production. + +More highlights +=============== + + From antocuni at codespeak.net Wed Nov 10 10:11:49 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 10 Nov 2010 10:11:49 +0100 (CET) Subject: [pypy-svn] r78954 - pypy/trunk/pypy/doc Message-ID: <20101110091149.C0165282B90@codespeak.net> Author: antocuni Date: Wed Nov 10 10:11:47 2010 New Revision: 78954 Modified: pypy/trunk/pypy/doc/release-1.4.0.txt Log: fix title, fix english, add two highlights Modified: pypy/trunk/pypy/doc/release-1.4.0.txt ============================================================================== --- pypy/trunk/pypy/doc/release-1.4.0.txt (original) +++ pypy/trunk/pypy/doc/release-1.4.0.txt Wed Nov 10 10:11:47 2010 @@ -1,12 +1,12 @@ -======================= -PyPy 1.3: Self hosting -======================= +=============================== +PyPy 1.4: Ouroboros in practice +=============================== Hello. We're pleased to announce release of PyPy 1.4. This is a major breaktrhough in our long journey, since PyPy 1.4 is the first PyPy release that can translate -itself faster than CPython. Since today, we're planning to be starting using +itself faster than CPython. Since today, we plan to start using PyPy for our own development. Among other features, this release includes numerous performance improvements @@ -17,4 +17,8 @@ More highlights =============== +Virtualenv support: now PyPy is fully compatible with virtualenv_: note that +to use it, you need a recent version of virtualenv (>= 1.5). + +XXX write me: better regular expressions From antocuni at codespeak.net Wed Nov 10 10:54:56 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 10 Nov 2010 10:54:56 +0100 (CET) Subject: [pypy-svn] r78955 - pypy/build/bot2/pypybuildbot Message-ID: <20101110095456.049EB36C224@codespeak.net> Author: antocuni Date: Wed Nov 10 10:54:54 2010 New Revision: 78955 Modified: pypy/build/bot2/pypybuildbot/master.py Log: add a builder for win32 own tests (without scheduling them nightly) Modified: pypy/build/bot2/pypybuildbot/master.py ============================================================================== --- pypy/build/bot2/pypybuildbot/master.py (original) +++ pypy/build/bot2/pypybuildbot/master.py Wed Nov 10 10:54:54 2010 @@ -167,7 +167,7 @@ LINUX32 = "own-linux-x86-32" LINUX64 = "own-linux-x86-64" MACOSX32 = "own-macosx-x86-32" -#WIN32 = "own-win-x86-32" +WIN32 = "own-win-x86-32" APPLVLLINUX32 = "pypy-c-app-level-linux-x86-32" APPLVLLINUX64 = "pypy-c-app-level-linux-x86-64" STACKLESSAPPLVLLINUX32 = "pypy-c-stackless-app-level-linux-x86-32" @@ -239,7 +239,13 @@ "builddir": MACOSX32, "factory": pypyOwnTestFactory, "category": 'mac' - }, + }, + {"name": WIN32, + "slavenames": ["bigboard"], + "builddir": WIN32, + "factory": pypyOwnTestFactoryWin, + "category": 'own' + }, {"name": APPLVLLINUX32, "slavenames": ["bigdogvm1", "tannit32"], "builddir": APPLVLLINUX32, From arigo at codespeak.net Wed Nov 10 11:46:18 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 11:46:18 +0100 (CET) Subject: [pypy-svn] r78956 - in pypy/branch/rlist-jit/pypy: jit/codewriter jit/metainterp jit/metainterp/test jit/tl rpython Message-ID: <20101110104618.AE627282B9D@codespeak.net> Author: arigo Date: Wed Nov 10 11:46:14 2010 New Revision: 78956 Modified: pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py pypy/branch/rlist-jit/pypy/jit/metainterp/pyjitpl.py pypy/branch/rlist-jit/pypy/jit/metainterp/test/test_recursive.py pypy/branch/rlist-jit/pypy/jit/metainterp/test/test_virtualizable.py pypy/branch/rlist-jit/pypy/jit/metainterp/warmspot.py pypy/branch/rlist-jit/pypy/jit/tl/tl.py pypy/branch/rlist-jit/pypy/rpython/rlist.py Log: Fixes a few failing tests. Adds a requirement: virtualizable2 arrays specified with 'lst[*]' must be accessed using non-negative, non-raising getitems and setitems. (This requirement was implicit before, in the sense that it mostly worked to do negative indexing except at one place in pyjitpl.) Modified: pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py (original) +++ pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py Wed Nov 10 11:46:14 2010 @@ -82,6 +82,7 @@ self.follow_constant_exit(block) self.optimize_goto_if_not(block) for link in block.exits: + self._check_no_vable_array(link.args) self._do_renaming_on_link(renamings, link) def _do_renaming(self, rename, op): @@ -99,6 +100,26 @@ op.args[i] = ListOfKind(v.kind, newlst) return op + def _check_no_vable_array(self, list): + for v in list: + if v in self.vable_array_vars: + raise AssertionError( + "A virtualizable array is passed around; it should\n" + "only be used immediately after being read. Note\n" + "that a possible cause is indexing with an index not\n" + "known non-negative, or catching IndexError, or\n" + "not inlining at all (for tests: use listops=True).\n" + "Occurred in: %r" % self.graph) + # extra expanation: with the way things are organized in + # rpython/rlist.py, the ll_getitem becomes a function call + # that is typically meant to be inlined by the JIT, but + # this does not work with vable arrays because + # jtransform.py expects the getfield and the getarrayitem + # to be in the same basic block. It works a bit as a hack + # for simple cases where we performed the backendopt + # inlining before (even with a very low threshold, because + # there is _always_inline_ on the relevant functions). + def _do_renaming_on_link(self, rename, link): for i, v in enumerate(link.args): if isinstance(v, Variable): @@ -169,7 +190,10 @@ else: return rewrite(self, op) - def rewrite_op_same_as(self, op): pass + def rewrite_op_same_as(self, op): + if op.args[0] in self.vable_array_vars: + self.vable_array_vars[op.result]= self.vable_array_vars[op.args[0]] + def rewrite_op_cast_pointer(self, op): pass def rewrite_op_cast_opaque_ptr(self, op): pass # rlib.rerased def rewrite_op_cast_primitive(self, op): pass @@ -256,6 +280,7 @@ The name is one of '{residual,direct}_call_{r,ir,irf}_{i,r,f,v}'.""" if args is None: args = op.args[1:] + self._check_no_vable_array(args) lst_i, lst_r, lst_f = self.make_three_lists(args) reskind = getkind(op.result.concretetype)[0] if lst_f or reskind == 'f': kinds = 'irf' @@ -523,9 +548,8 @@ arrayfielddescr, arraydescr) return [] - # check for deepfrozen structures that force constant-folding - immut = v_inst.concretetype.TO._immutable_field(c_fieldname.value) - if immut: + # check for _immutable_fields_ hints + if v_inst.concretetype.TO._immutable_field(c_fieldname.value): if (self.callcontrol is not None and self.callcontrol.could_be_green_field(v_inst.concretetype.TO, c_fieldname.value)): Modified: pypy/branch/rlist-jit/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/rlist-jit/pypy/jit/metainterp/pyjitpl.py Wed Nov 10 11:46:14 2010 @@ -601,8 +601,10 @@ virtualizable = vinfo.unwrap_virtualizable_box(virtualizable_box) arrayindex = vinfo.array_field_by_descrs[arrayfielddescr] index = indexbox.getint() - if index < 0: - index += vinfo.get_array_length(virtualizable, arrayindex) + # Support for negative index: disabled + # (see codewriter/jtransform.py, _check_no_vable_array). + #if index < 0: + # index += vinfo.get_array_length(virtualizable, arrayindex) assert 0 <= index < vinfo.get_array_length(virtualizable, arrayindex) return vinfo.get_index_in_array(virtualizable, arrayindex, index) Modified: pypy/branch/rlist-jit/pypy/jit/metainterp/test/test_recursive.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/metainterp/test/test_recursive.py (original) +++ pypy/branch/rlist-jit/pypy/jit/metainterp/test/test_recursive.py Wed Nov 10 11:46:14 2010 @@ -927,12 +927,16 @@ x=x) frame.s = hint(frame.s, promote=True) n -= 1 - x += frame.l[frame.s] + s = frame.s + assert s >= 0 + x += frame.l[s] frame.s += 1 if codeno == 0: subframe = Frame([n, n+1, n+2, n+3], 0) x += f(1, 10, 1, subframe) - x += frame.l[frame.s] + s = frame.s + assert s >= 0 + x += frame.l[s] x += len(frame.l) frame.s -= 1 return x Modified: pypy/branch/rlist-jit/pypy/jit/metainterp/test/test_virtualizable.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/metainterp/test/test_virtualizable.py (original) +++ pypy/branch/rlist-jit/pypy/jit/metainterp/test/test_virtualizable.py Wed Nov 10 11:46:14 2010 @@ -480,9 +480,13 @@ myjitdriver.jit_merge_point(frame=frame, n=n, x=x) frame.s = hint(frame.s, promote=True) n -= 1 - x += frame.l[frame.s] + s = frame.s + assert s >= 0 + x += frame.l[s] frame.s += 1 - x += frame.l[frame.s] + s = frame.s + assert s >= 0 + x += frame.l[s] x += len(frame.l) frame.s -= 1 return x @@ -994,7 +998,9 @@ jitdriver.can_enter_jit(frame=frame, n=n) jitdriver.jit_merge_point(frame=frame, n=n) popped = frame.stack[frame.stackpos] - frame.stackpos -= 1 + sp = frame.stackpos - 1 + assert sp >= 0 + frame.stackpos = sp to_push = intmask(popped * 3) frame.stack[frame.stackpos] = to_push frame.stackpos += 1 Modified: pypy/branch/rlist-jit/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/metainterp/warmspot.py (original) +++ pypy/branch/rlist-jit/pypy/jit/metainterp/warmspot.py Wed Nov 10 11:46:14 2010 @@ -163,6 +163,8 @@ self.check_access_directly_sanity(graphs) if backendopt: self.prejit_optimizations(policy, graphs) + elif self.opt.listops: + self.prejit_optimizations_minimal_inline(policy, graphs) self.build_meta_interp(ProfilerClass) self.make_args_specifications() @@ -260,6 +262,10 @@ remove_asserts=True, really_remove_asserts=True) + def prejit_optimizations_minimal_inline(self, policy, graphs): + from pypy.translator.backendopt.inline import auto_inline_graphs + auto_inline_graphs(self.translator, graphs, 0.01) + def build_cpu(self, CPUClass, translate_support_code=False, no_stats=False, **kwds): assert CPUClass is not None Modified: pypy/branch/rlist-jit/pypy/jit/tl/tl.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/tl/tl.py (original) +++ pypy/branch/rlist-jit/pypy/jit/tl/tl.py Wed Nov 10 11:46:14 2010 @@ -16,32 +16,40 @@ def __init__(self, size): self = hint(self, access_directly=True, fresh_virtualizable=True) self.stack = [0] * size - self.stackpos = 0 + self.stackpos = 0 # always store a known-nonneg integer here def append(self, elem): self.stack[self.stackpos] = elem self.stackpos += 1 def pop(self): - self.stackpos -= 1 - if self.stackpos < 0: + stackpos = self.stackpos - 1 + if stackpos < 0: raise IndexError - return self.stack[self.stackpos] + self.stackpos = stackpos # always store a known-nonneg integer here + return self.stack[stackpos] def pick(self, i): - self.append(self.stack[self.stackpos - i - 1]) + n = self.stackpos - i - 1 + assert n >= 0 + self.append(self.stack[n]) def put(self, i): elem = self.pop() - self.stack[self.stackpos - i - 1] = elem + n = self.stackpos - i - 1 + assert n >= 0 + self.stack[n] = elem def roll(self, r): if r < -1: i = self.stackpos + r if i < 0: raise IndexError - elem = self.stack[self.stackpos - 1] + n = self.stackpos - 1 + assert n >= 0 + elem = self.stack[n] for j in range(self.stackpos - 2, i - 1, -1): + assert j >= 0 self.stack[j + 1] = self.stack[j] self.stack[i] = elem elif r > 1: @@ -51,7 +59,9 @@ elem = self.stack[i] for j in range(i, self.stackpos - 1): self.stack[j] = self.stack[j + 1] - self.stack[self.stackpos - 1] = elem + n = self.stackpos - 1 + assert n >= 0 + self.stack[n] = elem def make_interp(supports_call): Modified: pypy/branch/rlist-jit/pypy/rpython/rlist.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rpython/rlist.py (original) +++ pypy/branch/rlist-jit/pypy/rpython/rlist.py Wed Nov 10 11:46:14 2010 @@ -688,6 +688,7 @@ if index >= l.ll_length(): raise IndexError return l.ll_getitem_fast(index) +ll_getitem_nonneg._always_inline_ = True # no oopspec -- the function is inlined by the JIT def ll_getitem(func, l, index): @@ -719,6 +720,7 @@ if index < 0: index += l.ll_length() return ll_getitem_foldable_nonneg(l, index) +ll_getitem_foldable._always_inline_ = True # no oopspec -- the function is inlined by the JIT def ll_setitem_nonneg(func, l, index, newitem): @@ -727,6 +729,7 @@ if index >= l.ll_length(): raise IndexError l.ll_setitem_fast(index, newitem) +ll_setitem_nonneg._always_inline_ = True # no oopspec -- the function is inlined by the JIT def ll_setitem(func, l, index, newitem): From arigo at codespeak.net Wed Nov 10 13:09:24 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 13:09:24 +0100 (CET) Subject: [pypy-svn] r78957 - in pypy/branch/fast-forward/pypy/annotation: . test Message-ID: <20101110120924.3F5EB5080B@codespeak.net> Author: arigo Date: Wed Nov 10 13:09:20 2010 New Revision: 78957 Modified: pypy/branch/fast-forward/pypy/annotation/listdef.py pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py Log: Fix make_sure_not_{modified,resized} by doing the proper thing in listdef.py -- i.e. custom flags -- instead of obscurely abusing existing flags. Fixes the new test. Modified: pypy/branch/fast-forward/pypy/annotation/listdef.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/listdef.py (original) +++ pypy/branch/fast-forward/pypy/annotation/listdef.py Wed Nov 10 13:09:20 2010 @@ -6,11 +6,16 @@ class TooLateForChange(Exception): pass +class ListChangeUnallowed(Exception): + pass + class ListItem(object): mutated = False # True for lists mutated after creation resized = False # True for lists resized after creation range_step = None # the step -- only for lists only created by a range() dont_change_any_more = False # set to True when too late for changes + must_not_mutate = False # make_sure_not_modified() + must_not_resize = False # make_sure_not_resized() # what to do if range_step is different in merge. # - if one is a list (range_step is None), unify to a list. @@ -26,7 +31,6 @@ self.bookkeeper = bookkeeper self.itemof = {} # set of all ListDefs using this ListItem self.read_locations = {} - self.dont_resize = False if bookkeeper is None: self.dont_change_any_more = True @@ -34,12 +38,16 @@ if not self.mutated: if self.dont_change_any_more: raise TooLateForChange + if self.must_not_mutate: + raise ListChangeUnallowed("mutating list") self.mutated = True def resize(self): if not self.resized: - if self.dont_change_any_more or self.dont_resize: + if self.dont_change_any_more: raise TooLateForChange + if self.must_not_resize: + raise ListChangeUnallowed("resizing list") self.resized = True def setrangestep(self, step): @@ -63,11 +71,16 @@ # things more general self, other = other, self - if other.dont_resize: - if self.resized: - raise TooLateForChange() - self.dont_resize = True - if other.mutated: self.mutate() + if other.must_not_mutate: + if self.mutated: + raise ListChangeUnallowed("list merge with a mutated") + self.must_not_mutate = True + if other.must_not_resize: + if self.resized: + raise ListChangeUnallowed("list merge with a resized") + self.must_not_resize = True + if other.mutated: + self.mutate() if other.resized: self.resize() if other.range_step != self.range_step: @@ -189,13 +202,14 @@ def never_resize(self): if self.listitem.resized: - raise TooLateForChange() - self.listitem.dont_resize = True + raise ListChangeUnallowed("list already resized") + self.listitem.must_not_resize = True def never_mutate(self): - if self.listitem.resized or self.listitem.mutated: - raise TooLateForChange() - self.listitem.dont_change_any_more = True + self.never_resize() + if self.listitem.mutated: + raise ListChangeUnallowed("list already mutated") + self.listitem.must_not_mutate = True MOST_GENERAL_LISTDEF = ListDef(None, SomeObject()) Modified: pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py (original) +++ pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py Wed Nov 10 13:09:20 2010 @@ -10,7 +10,7 @@ from pypy.translator.translator import graphof as tgraphof from pypy.annotation import policy from pypy.annotation import specialize -from pypy.annotation.listdef import ListDef, TooLateForChange +from pypy.annotation.listdef import ListDef, ListChangeUnallowed from pypy.annotation.dictdef import DictDef from pypy.objspace.flow.model import * from pypy.rlib.rarithmetic import r_uint, base_int, r_longlong, r_ulonglong @@ -3206,7 +3206,7 @@ l.append(4) a = self.RPythonAnnotator() - py.test.raises(TooLateForChange, a.build_types, g, []) + py.test.raises(ListChangeUnallowed, a.build_types, g, []) assert called def test_listitem_no_mutating2(self): @@ -3229,7 +3229,7 @@ a = self.RPythonAnnotator() a.translator.config.translation.list_comprehension_operations = True - py.test.raises(TooLateForChange, a.build_types, fn, [int]) + py.test.raises(ListChangeUnallowed, a.build_types, fn, [int]) def test_listitem_never_resize(self): from pypy.rlib.debug import check_annotation @@ -3243,7 +3243,7 @@ check_annotation(l, checker) a = self.RPythonAnnotator() - py.test.raises(TooLateForChange, a.build_types, f, []) + py.test.raises(ListChangeUnallowed, a.build_types, f, []) def test_len_of_empty_list(self): @@ -3373,7 +3373,7 @@ a = self.RPythonAnnotator() a.translator.config.translation.list_comprehension_operations = True a.build_types(fn, []) - assert 0 + # assert did not raise ListChangeUnallowed def g(n): From arigo at codespeak.net Wed Nov 10 13:24:57 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 13:24:57 +0100 (CET) Subject: [pypy-svn] r78958 - in pypy/branch/rlist-jit/pypy/annotation: . test Message-ID: <20101110122457.8571B282B9D@codespeak.net> Author: arigo Date: Wed Nov 10 13:24:55 2010 New Revision: 78958 Modified: pypy/branch/rlist-jit/pypy/annotation/description.py pypy/branch/rlist-jit/pypy/annotation/listdef.py pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py Log: * Merge r78957 from branch/fast-forward. * Mark the 'x.list' marked as _immutable_fields_=['list[*]'] with never_mutate(), to make sure. Modified: pypy/branch/rlist-jit/pypy/annotation/description.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/description.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/description.py Wed Nov 10 13:24:55 2010 @@ -647,7 +647,9 @@ while cdesc is not None: if '_immutable_fields_' in cdesc.classdict: if search in cdesc.classdict['_immutable_fields_'].value: - return s_result.listdef.offspring() + s_copy = s_result.listdef.offspring() + s_copy.listdef.never_mutate() + return s_copy cdesc = cdesc.basedesc return s_result # common case Modified: pypy/branch/rlist-jit/pypy/annotation/listdef.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/listdef.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/listdef.py Wed Nov 10 13:24:55 2010 @@ -6,11 +6,16 @@ class TooLateForChange(Exception): pass +class ListChangeUnallowed(Exception): + pass + class ListItem(object): mutated = False # True for lists mutated after creation resized = False # True for lists resized after creation range_step = None # the step -- only for lists only created by a range() dont_change_any_more = False # set to True when too late for changes + must_not_mutate = False # make_sure_not_modified() + must_not_resize = False # make_sure_not_resized() # what to do if range_step is different in merge. # - if one is a list (range_step is None), unify to a list. @@ -26,7 +31,6 @@ self.bookkeeper = bookkeeper self.itemof = {} # set of all ListDefs using this ListItem self.read_locations = {} - self.dont_resize = False if bookkeeper is None: self.dont_change_any_more = True @@ -34,12 +38,16 @@ if not self.mutated: if self.dont_change_any_more: raise TooLateForChange + if self.must_not_mutate: + raise ListChangeUnallowed("mutating list") self.mutated = True def resize(self): if not self.resized: - if self.dont_change_any_more or self.dont_resize: + if self.dont_change_any_more: raise TooLateForChange + if self.must_not_resize: + raise ListChangeUnallowed("resizing list") self.resized = True def setrangestep(self, step): @@ -63,11 +71,16 @@ # things more general self, other = other, self - if other.dont_resize: - if self.resized: - raise TooLateForChange() - self.dont_resize = True - if other.mutated: self.mutate() + if other.must_not_mutate: + if self.mutated: + raise ListChangeUnallowed("list merge with a mutated") + self.must_not_mutate = True + if other.must_not_resize: + if self.resized: + raise ListChangeUnallowed("list merge with a resized") + self.must_not_resize = True + if other.mutated: + self.mutate() if other.resized: self.resize() if other.range_step != self.range_step: @@ -189,13 +202,14 @@ def never_resize(self): if self.listitem.resized: - raise TooLateForChange() - self.listitem.dont_resize = True + raise ListChangeUnallowed("list already resized") + self.listitem.must_not_resize = True def never_mutate(self): - if self.listitem.resized or self.listitem.mutated: - raise TooLateForChange() - self.listitem.dont_change_any_more = True + self.never_resize() + if self.listitem.mutated: + raise ListChangeUnallowed("list already mutated") + self.listitem.must_not_mutate = True MOST_GENERAL_LISTDEF = ListDef(None, SomeObject()) Modified: pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py Wed Nov 10 13:24:55 2010 @@ -10,7 +10,7 @@ from pypy.translator.translator import graphof as tgraphof from pypy.annotation import policy from pypy.annotation import specialize -from pypy.annotation.listdef import ListDef, TooLateForChange +from pypy.annotation.listdef import ListDef, ListChangeUnallowed from pypy.annotation.dictdef import DictDef from pypy.objspace.flow.model import * from pypy.rlib.rarithmetic import r_uint, base_int, r_longlong, r_ulonglong @@ -3206,7 +3206,7 @@ l.append(4) a = self.RPythonAnnotator() - py.test.raises(TooLateForChange, a.build_types, g, []) + py.test.raises(ListChangeUnallowed, a.build_types, g, []) assert called def test_listitem_no_mutating2(self): @@ -3229,7 +3229,7 @@ a = self.RPythonAnnotator() a.translator.config.translation.list_comprehension_operations = True - py.test.raises(TooLateForChange, a.build_types, fn, [int]) + py.test.raises(ListChangeUnallowed, a.build_types, fn, [int]) def test_listitem_never_resize(self): from pypy.rlib.debug import check_annotation @@ -3243,7 +3243,7 @@ check_annotation(l, checker) a = self.RPythonAnnotator() - py.test.raises(TooLateForChange, a.build_types, f, []) + py.test.raises(ListChangeUnallowed, a.build_types, f, []) def test_len_of_empty_list(self): @@ -3357,6 +3357,38 @@ # not a constant: both __enter__ and __exit__ have been annotated assert not s.is_constant() + def test_make_sure_not_modified(self): + from pypy.rlib.debug import make_sure_not_modified + + def pycode(consts): + make_sure_not_modified(consts) + def build1(): + return pycode(consts=[1]) + def build2(): + return pycode(consts=[0]) + def fn(): + build1() + build2() + + a = self.RPythonAnnotator() + a.translator.config.translation.list_comprehension_operations = True + a.build_types(fn, []) + # assert did not raise ListChangeUnallowed + + def test_return_immutable_list(self): + class A: + _immutable_fields_ = 'lst[*]' + def f(n): + a = A() + l1 = [n] + l1.append(n+1) + a.lst = l1 + return a.lst + + a = self.RPythonAnnotator() + s = a.build_types(f, [int]) + assert s.listdef.listitem.must_not_mutate + def g(n): return [0,1,2,n] From antocuni at codespeak.net Wed Nov 10 13:27:13 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 10 Nov 2010 13:27:13 +0100 (CET) Subject: [pypy-svn] r78959 - pypy/trunk/pypy/jit/metainterp/optimizeopt Message-ID: <20101110122713.269945080B@codespeak.net> Author: antocuni Date: Wed Nov 10 13:27:11 2010 New Revision: 78959 Modified: pypy/trunk/pypy/jit/metainterp/optimizeopt/optimizer.py Log: don't rely on catching NotImplementedError, it's not rpython Modified: pypy/trunk/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/trunk/pypy/jit/metainterp/optimizeopt/optimizer.py Wed Nov 10 13:27:11 2010 @@ -66,10 +66,10 @@ assert isinstance(constbox, Const) self.box = constbox self.level = LEVEL_CONSTANT - try: - val = self.box.getint() + if isinstance(constbox, ConstInt): + val = constbox.getint() self.intbound = IntBound(val, val) - except NotImplementedError: + else: self.intbound = IntUnbounded() def get_constant_class(self, cpu): From antocuni at codespeak.net Wed Nov 10 13:57:04 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 10 Nov 2010 13:57:04 +0100 (CET) Subject: [pypy-svn] r78960 - in pypy/trunk/pypy/objspace/flow: . test Message-ID: <20101110125704.7A340282B90@codespeak.net> Author: antocuni Date: Wed Nov 10 13:57:01 2010 New Revision: 78960 Modified: pypy/trunk/pypy/objspace/flow/objspace.py pypy/trunk/pypy/objspace/flow/test/test_objspace.py Log: make "except {NotImplementedError,AssertionError}" explicitly not rpython (but still allow it for geninterp) Modified: pypy/trunk/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/trunk/pypy/objspace/flow/objspace.py (original) +++ pypy/trunk/pypy/objspace/flow/objspace.py Wed Nov 10 13:57:01 2010 @@ -210,6 +210,11 @@ check_class = self.unwrap(w_check_class) except UnwrapException: raise Exception, "non-constant except guard" + if check_class in (NotImplementedError, AssertionError): + # if we are in geninterp, we cannot catch these exceptions + if not self.config.translation.builtins_can_raise_exceptions: + raise error.FlowingError("Catching %s is not valid in RPython" % + check_class.__name__) if not isinstance(check_class, tuple): # the simple case return ObjSpace.exception_match(self, w_exc_type, w_check_class) Modified: pypy/trunk/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/trunk/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/trunk/pypy/objspace/flow/test/test_objspace.py Wed Nov 10 13:57:01 2010 @@ -5,7 +5,7 @@ from pypy.objspace.flow.model import flatten, mkentrymap, c_last_exception from pypy.interpreter.argument import Arguments from pypy.translator.simplify import simplify_graph -from pypy.objspace.flow.objspace import FlowObjSpace +from pypy.objspace.flow.objspace import FlowObjSpace, error from pypy.objspace.flow import objspace, flowcontext from pypy import conftest from pypy.tool.stdlib_opcode import bytecode_spec @@ -953,6 +953,22 @@ assert op.args[0] == Constant(g) + def test_cannot_catch_special_exceptions(self): + def f(): + try: + f() + except NotImplementedError: + pass + py.test.raises(error.FlowingError, "self.codetest(f)") + # + def f(): + try: + f() + except AssertionError: + pass + py.test.raises(error.FlowingError, "self.codetest(f)") + + class TestFlowObjSpaceDelay(Base): def setup_class(cls): cls.space = FlowObjSpace() @@ -1013,6 +1029,15 @@ expected.sort() assert excfound == expected + def test_can_catch_special_exceptions(self): + def f(): + try: + f() + except NotImplementedError: + pass + graph = self.codetest(f) + # assert did not crash + DATA = {'x': 5, 'y': 6} From arigo at codespeak.net Wed Nov 10 14:00:11 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 14:00:11 +0100 (CET) Subject: [pypy-svn] r78961 - in pypy/branch/rlist-jit/pypy: annotation annotation/test interpreter rlib rlib/rsre rlib/rsre/test rlib/test Message-ID: <20101110130011.134245080C@codespeak.net> Author: arigo Date: Wed Nov 10 14:00:10 2010 New Revision: 78961 Modified: pypy/branch/rlist-jit/pypy/annotation/listdef.py pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py pypy/branch/rlist-jit/pypy/interpreter/pycode.py pypy/branch/rlist-jit/pypy/rlib/debug.py pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py pypy/branch/rlist-jit/pypy/rlib/rsre/test/test_zjit.py pypy/branch/rlist-jit/pypy/rlib/test/test_debug.py Log: Rename make_sure_not_modified() to list_not_modified_any_more(), who has only part of the effect: the *returned* list is flagged as immutable, but the argument list's annotation is not changed. Modified: pypy/branch/rlist-jit/pypy/annotation/listdef.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/listdef.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/listdef.py Wed Nov 10 14:00:10 2010 @@ -14,7 +14,7 @@ resized = False # True for lists resized after creation range_step = None # the step -- only for lists only created by a range() dont_change_any_more = False # set to True when too late for changes - must_not_mutate = False # make_sure_not_modified() + must_not_mutate = False # list_not_modified_any_more() must_not_resize = False # make_sure_not_resized() # what to do if range_step is different in merge. Modified: pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py Wed Nov 10 14:00:10 2010 @@ -3357,11 +3357,29 @@ # not a constant: both __enter__ and __exit__ have been annotated assert not s.is_constant() - def test_make_sure_not_modified(self): - from pypy.rlib.debug import make_sure_not_modified + def test_make_sure_not_resized(self): + from pypy.rlib.debug import make_sure_not_resized def pycode(consts): - make_sure_not_modified(consts) + make_sure_not_resized(consts) + def build1(): + return pycode(consts=[1]) + def build2(): + return pycode(consts=[0]) + def fn(): + build1() + build2() + + a = self.RPythonAnnotator() + a.translator.config.translation.list_comprehension_operations = True + a.build_types(fn, []) + # assert did not raise ListChangeUnallowed + + def test_list_not_modified_any_more(self): + from pypy.rlib.debug import list_not_modified_any_more + + def pycode(consts): + return list_not_modified_any_more(consts) def build1(): return pycode(consts=[1]) def build2(): Modified: pypy/branch/rlist-jit/pypy/interpreter/pycode.py ============================================================================== --- pypy/branch/rlist-jit/pypy/interpreter/pycode.py (original) +++ pypy/branch/rlist-jit/pypy/interpreter/pycode.py Wed Nov 10 14:00:10 2010 @@ -15,7 +15,7 @@ CO_OPTIMIZED, CO_NEWLOCALS, CO_VARARGS, CO_VARKEYWORDS, CO_NESTED, CO_GENERATOR, CO_CONTAINSGLOBALS) from pypy.rlib.rarithmetic import intmask -from pypy.rlib.debug import make_sure_not_resized, make_sure_not_modified +from pypy.rlib.debug import make_sure_not_resized, list_not_modified_any_more from pypy.rlib import jit from pypy.rlib.objectmodel import compute_hash from pypy.tool.stdlib_opcode import opcodedesc, HAVE_ARGUMENT @@ -69,7 +69,7 @@ self.co_stacksize = stacksize self.co_flags = flags self.co_code = code - self.co_consts_w = make_sure_not_modified(consts) + self.co_consts_w = list_not_modified_any_more(consts) self.co_names_w = [space.new_interned_str(aname) for aname in names] self.co_varnames = varnames self.co_freevars = freevars Modified: pypy/branch/rlist-jit/pypy/rlib/debug.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rlib/debug.py (original) +++ pypy/branch/rlist-jit/pypy/rlib/debug.py Wed Nov 10 14:00:10 2010 @@ -226,14 +226,15 @@ hop.exception_cannot_occur() return hop.inputarg(hop.args_r[0], arg=0) -def make_sure_not_modified(arg): - """ Function checking whether annotation of SomeList is never resized - and never modified, useful for debugging. Does nothing when run directly +def list_not_modified_any_more(arg): + """ Returns an annotator-time copy of the list 'arg' which is + flagged as 'don't mutate me'. Actually implemented as just + returning 'arg'. This is useful for debugging only. """ return arg class Entry(ExtRegistryEntry): - _about_ = make_sure_not_modified + _about_ = list_not_modified_any_more def compute_result_annotation(self, s_arg): from pypy.annotation.model import SomeList @@ -241,10 +242,11 @@ # the logic behind it is that we try not to propagate # make_sure_not_resized, when list comprehension is not on if self.bookkeeper.annotator.translator.config.translation.list_comprehension_operations: + s_arg = s_arg.listdef.offspring() s_arg.listdef.never_mutate() else: from pypy.annotation.annrpython import log - log.WARNING('make_sure_not_modified called, but has no effect since list_comprehension is off') + log.WARNING('list_not_modified_any_more called, but has no effect since list_comprehension is off') return s_arg def specialize_call(self, hop): Modified: pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py (original) +++ pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py Wed Nov 10 14:00:10 2010 @@ -1,5 +1,5 @@ import sys -from pypy.rlib.debug import check_nonneg, make_sure_not_modified +from pypy.rlib.debug import check_nonneg, list_not_modified_any_more from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.rsre import rsre_char from pypy.tool.sourcetools import func_with_new_name @@ -91,7 +91,7 @@ # and they must not be more than len(string). check_nonneg(match_start) check_nonneg(end) - self.pattern = pattern + self.pattern = list_not_modified_any_more(pattern) self.match_start = match_start self.end = end self.flags = flags @@ -471,7 +471,6 @@ while True: op = ctx.pat(ppos) ppos += 1 - make_sure_not_modified(ctx.pattern) #jit.jit_debug("sre_match", op, ppos, ptr) # Modified: pypy/branch/rlist-jit/pypy/rlib/rsre/test/test_zjit.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rlib/rsre/test/test_zjit.py (original) +++ pypy/branch/rlist-jit/pypy/rlib/rsre/test/test_zjit.py Wed Nov 10 14:00:10 2010 @@ -1,6 +1,5 @@ from pypy.jit.metainterp.test import test_basic from pypy.rlib.nonconst import NonConstant -from pypy.rlib.debug import make_sure_not_modified from pypy.rlib.rsre.test.test_match import get_code from pypy.rlib.rsre import rsre_core from pypy.rpython.lltypesystem import lltype Modified: pypy/branch/rlist-jit/pypy/rlib/test/test_debug.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rlib/test/test_debug.py (original) +++ pypy/branch/rlist-jit/pypy/rlib/test/test_debug.py Wed Nov 10 14:00:10 2010 @@ -42,14 +42,14 @@ py.test.raises(IntegerCanBeNegative, interpret, g, [9]) def test_make_sure_not_resized(): - from pypy.annotation.listdef import TooLateForChange + from pypy.annotation.listdef import ListChangeUnallowed def f(): result = [1,2,3] make_sure_not_resized(result) result.append(4) return len(result) - py.test.raises(TooLateForChange, interpret, f, [], + py.test.raises(ListChangeUnallowed, interpret, f, [], list_comprehension_operations=True) From arigo at codespeak.net Wed Nov 10 14:02:17 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 14:02:17 +0100 (CET) Subject: [pypy-svn] r78962 - pypy/branch/rlist-jit/pypy/jit/codewriter Message-ID: <20101110130217.C3C9C282B90@codespeak.net> Author: arigo Date: Wed Nov 10 14:02:16 2010 New Revision: 78962 Modified: pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py Log: Fix for tests. Modified: pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py (original) +++ pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py Wed Nov 10 14:02:16 2010 @@ -24,6 +24,7 @@ class Transformer(object): + vable_array_vars = None def __init__(self, cpu=None, callcontrol=None, portal_jd=None): self.cpu = cpu @@ -101,6 +102,8 @@ return op def _check_no_vable_array(self, list): + if not self.vable_array_vars: + return for v in list: if v in self.vable_array_vars: raise AssertionError( From arigo at codespeak.net Wed Nov 10 14:08:59 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 14:08:59 +0100 (CET) Subject: [pypy-svn] r78963 - in pypy/branch/fast-forward/pypy: annotation annotation/test interpreter rlib rlib/rsre rlib/rsre/test rlib/test Message-ID: <20101110130859.9D5E25080C@codespeak.net> Author: arigo Date: Wed Nov 10 14:08:57 2010 New Revision: 78963 Modified: pypy/branch/fast-forward/pypy/annotation/listdef.py pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py pypy/branch/fast-forward/pypy/interpreter/pycode.py pypy/branch/fast-forward/pypy/rlib/debug.py pypy/branch/fast-forward/pypy/rlib/rsre/rsre_core.py pypy/branch/fast-forward/pypy/rlib/rsre/test/test_zjit.py pypy/branch/fast-forward/pypy/rlib/test/test_debug.py Log: Merge r78961 from branch/rlist-jit: rename make_sure_not_modified() to list_not_modified_any_more(). Modified: pypy/branch/fast-forward/pypy/annotation/listdef.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/listdef.py (original) +++ pypy/branch/fast-forward/pypy/annotation/listdef.py Wed Nov 10 14:08:57 2010 @@ -14,7 +14,7 @@ resized = False # True for lists resized after creation range_step = None # the step -- only for lists only created by a range() dont_change_any_more = False # set to True when too late for changes - must_not_mutate = False # make_sure_not_modified() + must_not_mutate = False # list_not_modified_any_more() must_not_resize = False # make_sure_not_resized() # what to do if range_step is different in merge. Modified: pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py (original) +++ pypy/branch/fast-forward/pypy/annotation/test/test_annrpython.py Wed Nov 10 14:08:57 2010 @@ -3357,11 +3357,11 @@ # not a constant: both __enter__ and __exit__ have been annotated assert not s.is_constant() - def test_make_sure_not_modified(self): - from pypy.rlib.debug import make_sure_not_modified + def test_make_sure_not_resized(self): + from pypy.rlib.debug import make_sure_not_resized def pycode(consts): - make_sure_not_modified(consts) + make_sure_not_resized(consts) def build1(): return pycode(consts=[1]) def build2(): @@ -3375,6 +3375,23 @@ a.build_types(fn, []) # assert did not raise ListChangeUnallowed + def test_list_not_modified_any_more(self): + from pypy.rlib.debug import list_not_modified_any_more + + def pycode(consts): + return list_not_modified_any_more(consts) + def build1(): + return pycode(consts=[1]) + def build2(): + return pycode(consts=[0]) + def fn(): + build1() + build2() + + a = self.RPythonAnnotator() + a.translator.config.translation.list_comprehension_operations = True + a.build_types(fn, []) + # assert did not raise ListChangeUnallowed def g(n): return [0,1,2,n] Modified: pypy/branch/fast-forward/pypy/interpreter/pycode.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/pycode.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/pycode.py Wed Nov 10 14:08:57 2010 @@ -15,7 +15,7 @@ CO_OPTIMIZED, CO_NEWLOCALS, CO_VARARGS, CO_VARKEYWORDS, CO_NESTED, CO_GENERATOR, CO_CONTAINSGLOBALS) from pypy.rlib.rarithmetic import intmask -from pypy.rlib.debug import make_sure_not_resized, make_sure_not_modified +from pypy.rlib.debug import make_sure_not_resized, list_not_modified_any_more from pypy.rlib import jit from pypy.rlib.objectmodel import compute_hash from pypy.tool.stdlib_opcode import opcodedesc, HAVE_ARGUMENT @@ -69,7 +69,7 @@ self.co_stacksize = stacksize self.co_flags = flags self.co_code = code - self.co_consts_w = make_sure_not_modified(consts) + self.co_consts_w = list_not_modified_any_more(consts) self.co_names_w = [space.new_interned_str(aname) for aname in names] self.co_varnames = varnames self.co_freevars = freevars Modified: pypy/branch/fast-forward/pypy/rlib/debug.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/debug.py (original) +++ pypy/branch/fast-forward/pypy/rlib/debug.py Wed Nov 10 14:08:57 2010 @@ -226,14 +226,15 @@ hop.exception_cannot_occur() return hop.inputarg(hop.args_r[0], arg=0) -def make_sure_not_modified(arg): - """ Function checking whether annotation of SomeList is never resized - and never modified, useful for debugging. Does nothing when run directly +def list_not_modified_any_more(arg): + """ Returns an annotator-time copy of the list 'arg' which is + flagged as 'don't mutate me'. Actually implemented as just + returning 'arg'. This is useful for debugging only. """ return arg class Entry(ExtRegistryEntry): - _about_ = make_sure_not_modified + _about_ = list_not_modified_any_more def compute_result_annotation(self, s_arg): from pypy.annotation.model import SomeList @@ -241,10 +242,11 @@ # the logic behind it is that we try not to propagate # make_sure_not_resized, when list comprehension is not on if self.bookkeeper.annotator.translator.config.translation.list_comprehension_operations: + s_arg = s_arg.listdef.offspring() s_arg.listdef.never_mutate() else: from pypy.annotation.annrpython import log - log.WARNING('make_sure_not_modified called, but has no effect since list_comprehension is off') + log.WARNING('list_not_modified_any_more called, but has no effect since list_comprehension is off') return s_arg def specialize_call(self, hop): Modified: pypy/branch/fast-forward/pypy/rlib/rsre/rsre_core.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rsre/rsre_core.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rsre/rsre_core.py Wed Nov 10 14:08:57 2010 @@ -1,5 +1,5 @@ import sys -from pypy.rlib.debug import check_nonneg, make_sure_not_modified +from pypy.rlib.debug import check_nonneg, list_not_modified_any_more from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.rsre import rsre_char from pypy.tool.sourcetools import func_with_new_name @@ -91,7 +91,7 @@ # and they must not be more than len(string). check_nonneg(match_start) check_nonneg(end) - self.pattern = pattern + self.pattern = list_not_modified_any_more(pattern) self.match_start = match_start self.end = end self.flags = flags @@ -471,7 +471,6 @@ while True: op = ctx.pat(ppos) ppos += 1 - make_sure_not_modified(ctx.pattern) #jit.jit_debug("sre_match", op, ppos, ptr) # Modified: pypy/branch/fast-forward/pypy/rlib/rsre/test/test_zjit.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rsre/test/test_zjit.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rsre/test/test_zjit.py Wed Nov 10 14:08:57 2010 @@ -1,6 +1,5 @@ from pypy.jit.metainterp.test import test_basic from pypy.rlib.nonconst import NonConstant -from pypy.rlib.debug import make_sure_not_modified from pypy.rlib.rsre.test.test_match import get_code from pypy.rlib.rsre import rsre_core from pypy.rpython.lltypesystem import lltype Modified: pypy/branch/fast-forward/pypy/rlib/test/test_debug.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/test/test_debug.py (original) +++ pypy/branch/fast-forward/pypy/rlib/test/test_debug.py Wed Nov 10 14:08:57 2010 @@ -42,14 +42,14 @@ py.test.raises(IntegerCanBeNegative, interpret, g, [9]) def test_make_sure_not_resized(): - from pypy.annotation.listdef import TooLateForChange + from pypy.annotation.listdef import ListChangeUnallowed def f(): result = [1,2,3] make_sure_not_resized(result) result.append(4) return len(result) - py.test.raises(TooLateForChange, interpret, f, [], + py.test.raises(ListChangeUnallowed, interpret, f, [], list_comprehension_operations=True) From arigo at codespeak.net Wed Nov 10 14:22:59 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 14:22:59 +0100 (CET) Subject: [pypy-svn] r78964 - in pypy/branch/rlist-jit/pypy: annotation annotation/test objspace/std Message-ID: <20101110132259.37884282B9D@codespeak.net> Author: arigo Date: Wed Nov 10 14:22:56 2010 New Revision: 78964 Modified: pypy/branch/rlist-jit/pypy/annotation/description.py pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py pypy/branch/rlist-jit/pypy/objspace/std/objspace.py pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py Log: Test and fix for _immutable_fields_=['lst[*]']: verify that the list is actually a non-resizable list. With this check done here, we can remove a few asserts from the std objspace. Modified: pypy/branch/rlist-jit/pypy/annotation/description.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/description.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/description.py Wed Nov 10 14:22:56 2010 @@ -647,6 +647,7 @@ while cdesc is not None: if '_immutable_fields_' in cdesc.classdict: if search in cdesc.classdict['_immutable_fields_'].value: + s_result.listdef.never_resize() s_copy = s_result.listdef.offspring() s_copy.listdef.never_mutate() return s_copy Modified: pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py Wed Nov 10 14:22:56 2010 @@ -3398,8 +3398,8 @@ _immutable_fields_ = 'lst[*]' def f(n): a = A() - l1 = [n] - l1.append(n+1) + l1 = [n, 0] + l1[1] = n+1 a.lst = l1 return a.lst @@ -3407,6 +3407,19 @@ s = a.build_types(f, [int]) assert s.listdef.listitem.must_not_mutate + def test_immutable_list_is_actually_resized(self): + class A: + _immutable_fields_ = 'lst[*]' + def f(n): + a = A() + l1 = [n] + l1.append(n+1) + a.lst = l1 + return a.lst + + a = self.RPythonAnnotator() + py.test.raises(ListChangeUnallowed, a.build_types, f, [int]) + def g(n): return [0,1,2,n] Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/objspace.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Wed Nov 10 14:22:56 2010 @@ -265,7 +265,6 @@ def newtuple(self, list_w): assert isinstance(list_w, list) - make_sure_not_resized(list_w) return W_TupleObject(list_w) def newlist(self, list_w): Modified: pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py Wed Nov 10 14:22:56 2010 @@ -6,14 +6,12 @@ from pypy.rlib.rarithmetic import intmask from pypy.objspace.std.sliceobject import W_SliceObject, normalize_simple_slice from pypy.interpreter import gateway -from pypy.rlib.debug import make_sure_not_resized class W_TupleObject(W_Object): from pypy.objspace.std.tupletype import tuple_typedef as typedef _immutable_fields_ = ['wrappeditems[*]'] def __init__(w_self, wrappeditems): - make_sure_not_resized(wrappeditems) w_self.wrappeditems = wrappeditems # a list of wrapped values def __repr__(w_self): From antocuni at codespeak.net Wed Nov 10 14:27:32 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 10 Nov 2010 14:27:32 +0100 (CET) Subject: [pypy-svn] r78965 - pypy/trunk/pypy/rlib Message-ID: <20101110132732.52E0A5080B@codespeak.net> Author: antocuni Date: Wed Nov 10 14:27:30 2010 New Revision: 78965 Modified: pypy/trunk/pypy/rlib/streamio.py Log: don't use NotImplementError for flow control, it's not fully rpython Modified: pypy/trunk/pypy/rlib/streamio.py ============================================================================== --- pypy/trunk/pypy/rlib/streamio.py (original) +++ pypy/trunk/pypy/rlib/streamio.py Wed Nov 10 14:27:30 2010 @@ -12,7 +12,7 @@ * some other methods also have no default parameters. * close() should be called exactly once and no further operations performed; there is no __del__() closing the stream for you. - * some methods may raise NotImplementedError. + * some methods may raise MyNotImplementedError. * peek() returns some (or no) characters that have already been read ahead. * flushable() returns True/False if flushing that stream is useful/pointless. @@ -54,6 +54,12 @@ ('a', True): O_RDWR | O_CREAT, } +class MyNotImplementedError(Exception): + """ + Catching NotImplementedError is not RPython, so we use this custom class + instead of it + """ + # ____________________________________________________________ @@ -209,16 +215,16 @@ some methods.""" def read(self, n): - raise NotImplementedError + raise MyNotImplementedError def write(self, data): - raise NotImplementedError + raise MyNotImplementedError def tell(self): - raise NotImplementedError + raise MyNotImplementedError def seek(self, offset, whence): - raise NotImplementedError + raise MyNotImplementedError def readall(self): bufsize = 8192 @@ -251,7 +257,7 @@ return ''.join(result) def truncate(self, size): - raise NotImplementedError + raise MyNotImplementedError def flush_buffers(self): pass @@ -487,7 +493,7 @@ if self.lines or self.buf: try: self.do_seek(self.tell(), 0) - except NotImplementedError: + except MyNotImplementedError: pass else: self.lines = [] @@ -535,14 +541,14 @@ self.buf = "" try: self.do_seek(offset, 1) - except NotImplementedError: + except MyNotImplementedError: intoffset = offset2int(offset) self.read(intoffset) return if whence == 2: try: self.do_seek(offset, 2) - except NotImplementedError: + except MyNotImplementedError: pass else: self.lines = [] @@ -1019,7 +1025,7 @@ if self.buf: try: self.base.seek(-len(self.buf), 1) - except NotImplementedError: + except MyNotImplementedError: pass else: self.buf = "" From antocuni at codespeak.net Wed Nov 10 14:44:35 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 10 Nov 2010 14:44:35 +0100 (CET) Subject: [pypy-svn] r78966 - pypy/trunk/pypy/jit/backend/x86/test Message-ID: <20101110134435.510BA282B9D@codespeak.net> Author: antocuni Date: Wed Nov 10 14:44:31 2010 New Revision: 78966 Modified: pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py Log: fix the test Modified: pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py (original) +++ pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py Wed Nov 10 14:44:31 2010 @@ -84,11 +84,11 @@ argchain.arg(x) res = func.call(argchain, rffi.DOUBLE) i -= 1 - return res + return int(res) # def main(i, j): return f(i, j) + libffi_stuff(i, j) - expected = f(40, -49) + expected = main(40, -49) res = self.meta_interp(main, [40, -49]) assert res == expected From arigo at codespeak.net Wed Nov 10 16:01:42 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 16:01:42 +0100 (CET) Subject: [pypy-svn] r78967 - in pypy/branch/rlist-jit/pypy: objspace/std rlib Message-ID: <20101110150142.E5408282B9C@codespeak.net> Author: arigo Date: Wed Nov 10 16:01:39 2010 New Revision: 78967 Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py pypy/branch/rlist-jit/pypy/rlib/debug.py Log: Add space.newtuple_imm(), to be called when the argument is an already-immutable list, instead of space.newtuple(). Add an explicit crash when doing two list_not_modified_any_more() in a row, which is used to find relatively quickly the places that still call space.newtuple() but must be corrected. Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/objspace.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Wed Nov 10 16:01:39 2010 @@ -8,7 +8,7 @@ transparent, callmethod, proxyobject) from pypy.objspace.descroperation import DescrOperation, raiseattrerror from pypy.rlib.objectmodel import instantiate, r_dict, specialize -from pypy.rlib.debug import make_sure_not_resized +from pypy.rlib.debug import make_sure_not_resized, list_not_modified_any_more from pypy.rlib.rarithmetic import base_int from pypy.rlib.objectmodel import we_are_translated from pypy.rlib.jit import hint @@ -265,6 +265,12 @@ def newtuple(self, list_w): assert isinstance(list_w, list) + return W_TupleObject(list_not_modified_any_more(list_w)) + + def newtuple_imm(self, list_w): + # call this instead of newtuple() if the list already has + # the annotation 'list_not_modified_any_more()' + assert isinstance(list_w, list) return W_TupleObject(list_w) def newlist(self, list_w): Modified: pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py Wed Nov 10 16:01:39 2010 @@ -53,7 +53,7 @@ for i in range(slicelength): subitems[i] = items[start] start += step - return W_TupleObject(subitems) + return space.newtuple(subitems) def getslice__Tuple_ANY_ANY(space, w_tuple, w_start, w_stop): length = len(w_tuple.wrappeditems) Modified: pypy/branch/rlist-jit/pypy/rlib/debug.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rlib/debug.py (original) +++ pypy/branch/rlist-jit/pypy/rlib/debug.py Wed Nov 10 16:01:39 2010 @@ -239,6 +239,14 @@ def compute_result_annotation(self, s_arg): from pypy.annotation.model import SomeList assert isinstance(s_arg, SomeList) + # check that we don't call list_not_modified_any_more() on a list that + # is already the result of a previous list_not_modified_any_more(). + # This helps locating issues in a pypy translation: if newtuple() is + # first annotated with a list that comes e.g. from fixedview(), it + # will immediately crash here instead of later when newtuple() is + # also annotated with a normal list. + assert not s_arg.listdef.listitem.must_not_mutate, ( + "argument already has the flag 'must_not_mutate'") # the logic behind it is that we try not to propagate # make_sure_not_resized, when list comprehension is not on if self.bookkeeper.annotator.translator.config.translation.list_comprehension_operations: From arigo at codespeak.net Wed Nov 10 16:01:50 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 16:01:50 +0100 (CET) Subject: [pypy-svn] r78968 - pypy/branch/rlist-jit/pypy/interpreter Message-ID: <20101110150150.0061D282B9D@codespeak.net> Author: arigo Date: Wed Nov 10 16:01:49 2010 New Revision: 78968 Modified: pypy/branch/rlist-jit/pypy/interpreter/argument.py Log: Fix. Modified: pypy/branch/rlist-jit/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/rlist-jit/pypy/interpreter/argument.py (original) +++ pypy/branch/rlist-jit/pypy/interpreter/argument.py Wed Nov 10 16:01:49 2010 @@ -3,7 +3,7 @@ """ from pypy.interpreter.error import OperationError, operationerrfmt -from pypy.rlib.debug import make_sure_not_resized +from pypy.rlib.debug import make_sure_not_resized, list_not_modified_any_more from pypy.rlib import jit @@ -134,7 +134,8 @@ def replace_arguments(self, args_w): "Return a new Arguments with a args_w as positional arguments." - return Arguments(self.space, args_w, self.keywords, self.keywords_w) + return Arguments(self.space, list_not_modified_any_more(args_w), + self.keywords, self.keywords_w) def prepend(self, w_firstarg): "Return a new Arguments with a new argument inserted first." From arigo at codespeak.net Wed Nov 10 16:41:50 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 16:41:50 +0100 (CET) Subject: [pypy-svn] r78969 - in pypy/extradoc/pypy.org: . source/_layouts Message-ID: <20101110154150.D531B282B9C@codespeak.net> Author: arigo Date: Wed Nov 10 16:41:49 2010 New Revision: 78969 Modified: pypy/extradoc/pypy.org/compat.html pypy/extradoc/pypy.org/contact.html pypy/extradoc/pypy.org/download.html pypy/extradoc/pypy.org/features.html pypy/extradoc/pypy.org/index.html pypy/extradoc/pypy.org/source/_layouts/page.genshi Log: Add the google checkout link to pypy.org. Modified: pypy/extradoc/pypy.org/compat.html ============================================================================== --- pypy/extradoc/pypy.org/compat.html (original) +++ pypy/extradoc/pypy.org/compat.html Wed Nov 10 16:41:49 2010 @@ -122,6 +122,40 @@
  • + +
    + + + + + + + + + + + + + +
    $ + + +
    +
    +
    +
    • +     Modified: pypy/extradoc/pypy.org/contact.html ============================================================================== --- pypy/extradoc/pypy.org/contact.html (original) +++ pypy/extradoc/pypy.org/contact.html Wed Nov 10 16:41:49 2010 @@ -68,6 +68,40 @@
  • + +
    + + + + + + + + + + + + + +
    $ + + +
    +
    +
    +
    • +
  • +
    Download PyPy
    • Modified: pypy/extradoc/pypy.org/download.html ============================================================================== --- pypy/extradoc/pypy.org/download.html (original) +++ pypy/extradoc/pypy.org/download.html Wed Nov 10 16:41:49 2010 @@ -241,6 +241,40 @@
  • + +
    + + + + + + + + + + + + + +
    $ + + +
    +
    +
    +
    • +
  • +
    Download PyPy
    • Modified: pypy/extradoc/pypy.org/features.html ============================================================================== --- pypy/extradoc/pypy.org/features.html (original) +++ pypy/extradoc/pypy.org/features.html Wed Nov 10 16:41:49 2010 @@ -134,6 +134,40 @@
  • + +
    + + + + + + + + + + + + + +
    $ + + +
    +
    +
    +
    • +
  • +
    Download PyPy
    • Modified: pypy/extradoc/pypy.org/index.html ============================================================================== --- pypy/extradoc/pypy.org/index.html (original) +++ pypy/extradoc/pypy.org/index.html Wed Nov 10 16:41:49 2010 @@ -81,6 +81,40 @@
  • + +
    + + + + + + + + + + + + + +
    $ + + +
    +
    +
    +
    • +
  • +
    Download PyPy
    • Modified: pypy/extradoc/pypy.org/source/_layouts/page.genshi ============================================================================== --- pypy/extradoc/pypy.org/source/_layouts/page.genshi (original) +++ pypy/extradoc/pypy.org/source/_layouts/page.genshi Wed Nov 10 16:41:49 2010 @@ -22,6 +22,40 @@
  • + +
    + + + + + + + + + + + + + +
    $ + + +
    +
    +
    +
    • +
  • +
    Download PyPy
    • From arigo at codespeak.net Wed Nov 10 16:44:50 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 16:44:50 +0100 (CET) Subject: [pypy-svn] r78970 - pypy/extradoc/pypy.org/source Message-ID: <20101110154450.0DF7B282B9D@codespeak.net> Author: arigo Date: Wed Nov 10 16:44:48 2010 New Revision: 78970 Modified: pypy/extradoc/pypy.org/source/README Log: Update the README. Modified: pypy/extradoc/pypy.org/source/README ============================================================================== --- pypy/extradoc/pypy.org/source/README (original) +++ pypy/extradoc/pypy.org/source/README Wed Nov 10 16:44:48 2010 @@ -4,8 +4,11 @@ by running -yatiblog -o you'll get output in -output directory that contains all necessary stuff to upload and deploy. +..../ampify/environ/yatiblog -o .. + +you'll get html output in the parent directory. +Then you can check it in, go to codespeak in /www/pypy.org/htdocs/ +and type "svn up". Other required dependencies: * "docutils" from "easy_install docutils" From arigo at codespeak.net Wed Nov 10 17:23:10 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 17:23:10 +0100 (CET) Subject: [pypy-svn] r78971 - in pypy/branch/rlist-jit/pypy: interpreter module/__builtin__ module/_sre module/exceptions objspace/std rlib rlib/rsre Message-ID: <20101110162310.51D545080E@codespeak.net> Author: arigo Date: Wed Nov 10 17:23:07 2010 New Revision: 78971 Modified: pypy/branch/rlist-jit/pypy/interpreter/argument.py pypy/branch/rlist-jit/pypy/interpreter/function.py pypy/branch/rlist-jit/pypy/interpreter/pycode.py pypy/branch/rlist-jit/pypy/module/__builtin__/interp_classobj.py pypy/branch/rlist-jit/pypy/module/_sre/interp_sre.py pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py pypy/branch/rlist-jit/pypy/objspace/std/typetype.py pypy/branch/rlist-jit/pypy/rlib/debug.py pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py Log: A random set of fixes. Mostly a bit annoying. Modified: pypy/branch/rlist-jit/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/rlist-jit/pypy/interpreter/argument.py (original) +++ pypy/branch/rlist-jit/pypy/interpreter/argument.py Wed Nov 10 17:23:07 2010 @@ -3,7 +3,7 @@ """ from pypy.interpreter.error import OperationError, operationerrfmt -from pypy.rlib.debug import make_sure_not_resized, list_not_modified_any_more +from pypy.rlib.debug import make_sure_not_resized from pypy.rlib import jit @@ -134,8 +134,7 @@ def replace_arguments(self, args_w): "Return a new Arguments with a args_w as positional arguments." - return Arguments(self.space, list_not_modified_any_more(args_w), - self.keywords, self.keywords_w) + return Arguments(self.space, args_w, self.keywords, self.keywords_w) def prepend(self, w_firstarg): "Return a new Arguments with a new argument inserted first." Modified: pypy/branch/rlist-jit/pypy/interpreter/function.py ============================================================================== --- pypy/branch/rlist-jit/pypy/interpreter/function.py (original) +++ pypy/branch/rlist-jit/pypy/interpreter/function.py Wed Nov 10 17:23:07 2010 @@ -198,7 +198,7 @@ else: name = None if not space.is_w(w_argdefs, space.w_None): - defs_w = space.fixedview(w_argdefs) + defs_w = space.fixedview(w_argdefs)[:] else: defs_w = [] nfreevars = 0 @@ -323,7 +323,7 @@ if space.is_w(w_func_dict, space.w_None): w_func_dict = None self.w_func_dict = w_func_dict - self.defs_w = space.fixedview(w_defs_w) + self.defs_w = space.fixedview(w_defs_w)[:] self.w_module = w_module def fget_func_defaults(space, self): @@ -338,7 +338,7 @@ return if not space.is_true(space.isinstance(w_defaults, space.w_tuple)): raise OperationError( space.w_TypeError, space.wrap("func_defaults must be set to a tuple object or None") ) - self.defs_w = space.fixedview(w_defaults) + self.defs_w = space.fixedview(w_defaults)[:] def fdel_func_defaults(space, self): self.defs_w = [] Modified: pypy/branch/rlist-jit/pypy/interpreter/pycode.py ============================================================================== --- pypy/branch/rlist-jit/pypy/interpreter/pycode.py (original) +++ pypy/branch/rlist-jit/pypy/interpreter/pycode.py Wed Nov 10 17:23:07 2010 @@ -269,10 +269,10 @@ dis.dis(co) def fget_co_consts(space, self): - return space.newtuple(self.co_consts_w[:]) + return space.newtuple_imm(self.co_consts_w) def fget_co_names(space, self): - return space.newtuple(self.co_names_w) + return space.newtuple_imm(self.co_names_w) def fget_co_varnames(space, self): return space.newtuple([space.wrap(name) for name in self.co_varnames]) @@ -383,8 +383,8 @@ w(self.co_stacksize), w(self.co_flags), w(self.co_code), - space.newtuple(self.co_consts_w[:]), - space.newtuple(self.co_names_w), + space.newtuple_imm(self.co_consts_w), + space.newtuple_imm(self.co_names_w), space.newtuple([w(v) for v in self.co_varnames]), w(self.co_filename), w(self.co_name), Modified: pypy/branch/rlist-jit/pypy/module/__builtin__/interp_classobj.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/__builtin__/interp_classobj.py (original) +++ pypy/branch/rlist-jit/pypy/module/__builtin__/interp_classobj.py Wed Nov 10 17:23:07 2010 @@ -40,7 +40,7 @@ # XXX missing: lengthy and obscure logic about "__module__" - bases_w = space.fixedview(w_bases) + bases_w = space.fixedview(w_bases)[:] for w_base in bases_w: if not isinstance(w_base, W_ClassObject): w_metaclass = space.type(w_base) @@ -92,7 +92,7 @@ raise OperationError( space.w_TypeError, space.wrap("__bases__ must be a tuple object")) - bases_w = space.fixedview(w_bases) + bases_w = space.fixedview(w_bases)[:] for w_base in bases_w: if not isinstance(w_base, W_ClassObject): raise OperationError(space.w_TypeError, Modified: pypy/branch/rlist-jit/pypy/module/_sre/interp_sre.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/_sre/interp_sre.py (original) +++ pypy/branch/rlist-jit/pypy/module/_sre/interp_sre.py Wed Nov 10 17:23:07 2010 @@ -6,6 +6,7 @@ from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root from pypy.interpreter.error import OperationError from pypy.rlib.rarithmetic import intmask +from pypy.rlib.debug import list_not_modified_any_more from pypy.tool.pairtype import extendabletype @@ -111,17 +112,18 @@ space = self.space if pos < 0: pos = 0 if endpos < pos: endpos = pos + pattern = list_not_modified_any_more(self.code) if space.is_true(space.isinstance(w_string, space.w_unicode)): unicodestr = space.unicode_w(w_string) if pos > len(unicodestr): pos = len(unicodestr) if endpos > len(unicodestr): endpos = len(unicodestr) - return rsre_core.UnicodeMatchContext(self.code, unicodestr, + return rsre_core.UnicodeMatchContext(pattern, unicodestr, pos, endpos, self.flags) else: str = space.bufferstr_w(w_string) if pos > len(str): pos = len(str) if endpos > len(str): endpos = len(str) - return rsre_core.StrMatchContext(self.code, str, + return rsre_core.StrMatchContext(pattern, str, pos, endpos, self.flags) def getmatch(self, ctx, found): Modified: pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py (original) +++ pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py Wed Nov 10 17:23:07 2010 @@ -113,12 +113,12 @@ elif lgt == 1: return space.str(self.args_w[0]) else: - return space.str(space.newtuple(self.args_w)) + return space.str(space.newtuple_imm(self.args_w)) descr_str.unwrap_spec = ['self', ObjSpace] def descr_repr(self, space): if self.args_w: - args_repr = space.str_w(space.repr(space.newtuple(self.args_w))) + args_repr = space.str_w(space.repr(space.newtuple_imm(self.args_w))) else: args_repr = "()" clsname = self.getclass(space).getname(space, '?') @@ -126,13 +126,13 @@ descr_repr.unwrap_spec = ['self', ObjSpace] def descr_getargs(space, self): - return space.newtuple(self.args_w) + return space.newtuple_imm(self.args_w) def descr_setargs(space, self, w_newargs): self.args_w = space.fixedview(w_newargs) def descr_getitem(self, space, w_index): - return space.getitem(space.newtuple(self.args_w), w_index) + return space.getitem(space.newtuple_imm(self.args_w), w_index) descr_getitem.unwrap_spec = ['self', ObjSpace, W_Root] def getdict(self): @@ -146,7 +146,7 @@ self.w_dict = w_dict def descr_reduce(self, space): - lst = [self.getclass(space), space.newtuple(self.args_w)] + lst = [self.getclass(space), space.newtuple_imm(self.args_w)] if self.w_dict is not None and space.is_true(self.w_dict): lst = lst + [self.w_dict] return space.newtuple(lst) @@ -294,7 +294,7 @@ elif len(self.args_w) == 1: return space.repr(self.args_w[0]) else: - return space.str(space.newtuple(self.args_w)) + return space.str(space.newtuple_imm(self.args_w)) key_error_str.unwrap_spec = ['self', ObjSpace] W_KeyError = _new_exception('KeyError', W_LookupError, @@ -336,7 +336,7 @@ lst = [self.getclass(space), space.newtuple( self.args_w + [self.w_filename])] else: - lst = [self.getclass(space), space.newtuple(self.args_w)] + lst = [self.getclass(space), space.newtuple_imm(self.args_w)] if self.w_dict is not None and space.is_true(self.w_dict): lst = lst + [self.w_dict] return space.newtuple(lst) @@ -520,7 +520,7 @@ if len(args_w) == 1: self.w_code = args_w[0] elif len(args_w) > 1: - self.w_code = space.newtuple(args_w) + self.w_code = space.newtuple_imm(args_w) W_BaseException.descr_init(self, space, args_w) descr_init.unwrap_spec = ['self', ObjSpace, 'args_w'] Modified: pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py Wed Nov 10 17:23:07 2010 @@ -11,6 +11,7 @@ from pypy.rlib.objectmodel import current_object_addr_as_int, compute_hash from pypy.rlib.jit import hint, purefunction_promote, we_are_jitted from pypy.rlib.jit import dont_look_inside, purefunction +from pypy.rlib.debug import list_not_modified_any_more from pypy.rlib.rarithmetic import intmask, r_uint from copy_reg import _HEAPTYPE @@ -618,7 +619,7 @@ ensure_doc_attr(w_self) if w_self.is_heaptype(): ensure_module_attr(w_self) - w_self.mro_w = [] # temporarily + w_self.mro_w = list_not_modified_any_more([]) # temporarily compute_mro(w_self) def ensure_static_new(w_self): Modified: pypy/branch/rlist-jit/pypy/objspace/std/typetype.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/typetype.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/typetype.py Wed Nov 10 17:23:07 2010 @@ -12,7 +12,7 @@ w_typetype = _precheck_for_new(space, w_typetype) - bases_w = space.fixedview(w_bases) + bases_w = space.fixedview(w_bases)[:] w_winner = w_typetype for base in bases_w: @@ -115,7 +115,7 @@ "can only assign tuple to %s.__bases__, not %s", w_type.name, space.type(w_value).getname(space, '?')) - newbases_w = space.fixedview(w_value) + newbases_w = space.fixedview(w_value)[:] if len(newbases_w) == 0: raise operationerrfmt(space.w_TypeError, "can only assign non-empty tuple to %s.__bases__, not ()", Modified: pypy/branch/rlist-jit/pypy/rlib/debug.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rlib/debug.py (original) +++ pypy/branch/rlist-jit/pypy/rlib/debug.py Wed Nov 10 17:23:07 2010 @@ -247,6 +247,7 @@ # also annotated with a normal list. assert not s_arg.listdef.listitem.must_not_mutate, ( "argument already has the flag 'must_not_mutate'") + s_arg.listdef.mutate() # the logic behind it is that we try not to propagate # make_sure_not_resized, when list comprehension is not on if self.bookkeeper.annotator.translator.config.translation.list_comprehension_operations: Modified: pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py (original) +++ pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py Wed Nov 10 17:23:07 2010 @@ -91,7 +91,7 @@ # and they must not be more than len(string). check_nonneg(match_start) check_nonneg(end) - self.pattern = list_not_modified_any_more(pattern) + self.pattern = pattern self.match_start = match_start self.end = end self.flags = flags @@ -946,6 +946,7 @@ def match(pattern, string, start=0, end=sys.maxint, flags=0): start, end = _adjust(start, end, len(string)) + pattern = list_not_modified_any_more(pattern) ctx = StrMatchContext(pattern, string, start, end, flags) if match_context(ctx): return ctx @@ -954,6 +955,7 @@ def search(pattern, string, start=0, end=sys.maxint, flags=0): start, end = _adjust(start, end, len(string)) + pattern = list_not_modified_any_more(pattern) ctx = StrMatchContext(pattern, string, start, end, flags) if search_context(ctx): return ctx From antocuni at codespeak.net Wed Nov 10 17:29:31 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 10 Nov 2010 17:29:31 +0100 (CET) Subject: [pypy-svn] r78973 - pypy/branch/jitypes Message-ID: <20101110162931.5E5F8282B90@codespeak.net> Author: antocuni Date: Wed Nov 10 17:29:29 2010 New Revision: 78973 Removed: pypy/branch/jitypes/ Log: kill branch, will branch again from a newer version of trunk From antocuni at codespeak.net Wed Nov 10 17:30:17 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 10 Nov 2010 17:30:17 +0100 (CET) Subject: [pypy-svn] r78974 - pypy/branch/jitypes2 Message-ID: <20101110163017.2D489282B90@codespeak.net> Author: antocuni Date: Wed Nov 10 17:30:15 2010 New Revision: 78974 Added: pypy/branch/jitypes2/ (props changed) - copied from r78973, pypy/trunk/ Log: a branch where to try to have fast ctypes based on the _ffi module From antocuni at codespeak.net Wed Nov 10 17:34:41 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 10 Nov 2010 17:34:41 +0100 (CET) Subject: [pypy-svn] r78975 - pypy/branch/jitypes2/pypy/config Message-ID: <20101110163441.27D79282BE3@codespeak.net> Author: antocuni Date: Wed Nov 10 17:34:37 2010 New Revision: 78975 Modified: pypy/branch/jitypes2/pypy/config/pypyoption.py Log: enable _ffi by default on this branch Modified: pypy/branch/jitypes2/pypy/config/pypyoption.py ============================================================================== --- pypy/branch/jitypes2/pypy/config/pypyoption.py (original) +++ pypy/branch/jitypes2/pypy/config/pypyoption.py Wed Nov 10 17:34:37 2010 @@ -31,7 +31,7 @@ "crypt", "signal", "_rawffi", "termios", "zlib", "struct", "md5", "sha", "bz2", "_minimal_curses", "cStringIO", "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array", - "_bisect"] + "_bisect", "_ffi"] )) working_oo_modules = default_modules.copy() From arigo at codespeak.net Wed Nov 10 17:37:45 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 17:37:45 +0100 (CET) Subject: [pypy-svn] r78976 - pypy/branch/rlist-jit/pypy/objspace/std Message-ID: <20101110163745.C5198282BE3@codespeak.net> Author: arigo Date: Wed Nov 10 17:37:43 2010 New Revision: 78976 Modified: pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py Log: Last fix. Annotation without all working modules work. Modified: pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py Wed Nov 10 17:37:43 2010 @@ -11,7 +11,6 @@ from pypy.rlib.objectmodel import current_object_addr_as_int, compute_hash from pypy.rlib.jit import hint, purefunction_promote, we_are_jitted from pypy.rlib.jit import dont_look_inside, purefunction -from pypy.rlib.debug import list_not_modified_any_more from pypy.rlib.rarithmetic import intmask, r_uint from copy_reg import _HEAPTYPE @@ -619,7 +618,7 @@ ensure_doc_attr(w_self) if w_self.is_heaptype(): ensure_module_attr(w_self) - w_self.mro_w = list_not_modified_any_more([]) # temporarily + w_self.mro_w = [] # temporarily compute_mro(w_self) def ensure_static_new(w_self): @@ -653,7 +652,7 @@ if w_mro_func is not None and not space.is_w(w_where, space.w_type): w_mro_meth = space.get(w_mro_func, w_self) w_mro = space.call_function(w_mro_meth) - mro_w = space.fixedview(w_mro) + mro_w = space.fixedview(w_mro)[:] w_self.mro_w = validate_custom_mro(space, mro_w) return # done w_self.mro_w = w_self.compute_default_mro()[:] From arigo at codespeak.net Wed Nov 10 17:55:03 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 10 Nov 2010 17:55:03 +0100 (CET) Subject: [pypy-svn] r78977 - in pypy/branch/rlist-jit/pypy: annotation interpreter module/exceptions objspace/std rlib Message-ID: <20101110165503.C6331282B9C@codespeak.net> Author: arigo Date: Wed Nov 10 17:55:01 2010 New Revision: 78977 Modified: pypy/branch/rlist-jit/pypy/annotation/policy.py pypy/branch/rlist-jit/pypy/annotation/specialize.py pypy/branch/rlist-jit/pypy/interpreter/pycode.py pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py pypy/branch/rlist-jit/pypy/objspace/std/objspace.py pypy/branch/rlist-jit/pypy/rlib/debug.py Log: Remove again space.newtuple_imm(), as its job can nicely be done with a specialization. Modified: pypy/branch/rlist-jit/pypy/annotation/policy.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/policy.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/policy.py Wed Nov 10 17:55:01 2010 @@ -1,6 +1,7 @@ # base annotation policy for specialization from pypy.annotation.specialize import default_specialize as default from pypy.annotation.specialize import specialize_argvalue, specialize_argtype, specialize_arglistitemtype +from pypy.annotation.specialize import specialize_list_not_modified_any_more from pypy.annotation.specialize import memo # for some reason, model must be imported first, # or we create a cycle. @@ -75,6 +76,7 @@ specialize__arg = staticmethod(specialize_argvalue) # specialize:arg(N) specialize__argtype = staticmethod(specialize_argtype) # specialize:argtype(N) specialize__arglistitemtype = staticmethod(specialize_arglistitemtype) + specialize__list_not_modified_any_more = staticmethod(specialize_list_not_modified_any_more) def specialize__ll(pol, *args): from pypy.rpython.annlowlevel import LowLevelAnnotatorPolicy Modified: pypy/branch/rlist-jit/pypy/annotation/specialize.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/specialize.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/specialize.py Wed Nov 10 17:55:01 2010 @@ -369,3 +369,8 @@ else: key = s.listdef.listitem.s_value.knowntype return maybe_star_args(funcdesc, key, args_s) + +def specialize_list_not_modified_any_more(funcdesc, args_s, i): + s = args_s[i] + key = (s.knowntype is list and s.listdef.listitem.must_not_mutate) + return maybe_star_args(funcdesc, key, args_s) Modified: pypy/branch/rlist-jit/pypy/interpreter/pycode.py ============================================================================== --- pypy/branch/rlist-jit/pypy/interpreter/pycode.py (original) +++ pypy/branch/rlist-jit/pypy/interpreter/pycode.py Wed Nov 10 17:55:01 2010 @@ -269,10 +269,10 @@ dis.dis(co) def fget_co_consts(space, self): - return space.newtuple_imm(self.co_consts_w) + return space.newtuple(self.co_consts_w) def fget_co_names(space, self): - return space.newtuple_imm(self.co_names_w) + return space.newtuple(self.co_names_w) def fget_co_varnames(space, self): return space.newtuple([space.wrap(name) for name in self.co_varnames]) @@ -383,8 +383,8 @@ w(self.co_stacksize), w(self.co_flags), w(self.co_code), - space.newtuple_imm(self.co_consts_w), - space.newtuple_imm(self.co_names_w), + space.newtuple(self.co_consts_w), + space.newtuple(self.co_names_w), space.newtuple([w(v) for v in self.co_varnames]), w(self.co_filename), w(self.co_name), Modified: pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py (original) +++ pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py Wed Nov 10 17:55:01 2010 @@ -113,12 +113,12 @@ elif lgt == 1: return space.str(self.args_w[0]) else: - return space.str(space.newtuple_imm(self.args_w)) + return space.str(space.newtuple(self.args_w)) descr_str.unwrap_spec = ['self', ObjSpace] def descr_repr(self, space): if self.args_w: - args_repr = space.str_w(space.repr(space.newtuple_imm(self.args_w))) + args_repr = space.str_w(space.repr(space.newtuple(self.args_w))) else: args_repr = "()" clsname = self.getclass(space).getname(space, '?') @@ -126,13 +126,13 @@ descr_repr.unwrap_spec = ['self', ObjSpace] def descr_getargs(space, self): - return space.newtuple_imm(self.args_w) + return space.newtuple(self.args_w) def descr_setargs(space, self, w_newargs): self.args_w = space.fixedview(w_newargs) def descr_getitem(self, space, w_index): - return space.getitem(space.newtuple_imm(self.args_w), w_index) + return space.getitem(space.newtuple(self.args_w), w_index) descr_getitem.unwrap_spec = ['self', ObjSpace, W_Root] def getdict(self): @@ -146,7 +146,7 @@ self.w_dict = w_dict def descr_reduce(self, space): - lst = [self.getclass(space), space.newtuple_imm(self.args_w)] + lst = [self.getclass(space), space.newtuple(self.args_w)] if self.w_dict is not None and space.is_true(self.w_dict): lst = lst + [self.w_dict] return space.newtuple(lst) @@ -294,7 +294,7 @@ elif len(self.args_w) == 1: return space.repr(self.args_w[0]) else: - return space.str(space.newtuple_imm(self.args_w)) + return space.str(space.newtuple(self.args_w)) key_error_str.unwrap_spec = ['self', ObjSpace] W_KeyError = _new_exception('KeyError', W_LookupError, @@ -336,7 +336,7 @@ lst = [self.getclass(space), space.newtuple( self.args_w + [self.w_filename])] else: - lst = [self.getclass(space), space.newtuple_imm(self.args_w)] + lst = [self.getclass(space), space.newtuple(self.args_w)] if self.w_dict is not None and space.is_true(self.w_dict): lst = lst + [self.w_dict] return space.newtuple(lst) @@ -520,7 +520,7 @@ if len(args_w) == 1: self.w_code = args_w[0] elif len(args_w) > 1: - self.w_code = space.newtuple_imm(args_w) + self.w_code = space.newtuple(args_w) W_BaseException.descr_init(self, space, args_w) descr_init.unwrap_spec = ['self', ObjSpace, 'args_w'] Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/objspace.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Wed Nov 10 17:55:01 2010 @@ -266,12 +266,7 @@ def newtuple(self, list_w): assert isinstance(list_w, list) return W_TupleObject(list_not_modified_any_more(list_w)) - - def newtuple_imm(self, list_w): - # call this instead of newtuple() if the list already has - # the annotation 'list_not_modified_any_more()' - assert isinstance(list_w, list) - return W_TupleObject(list_w) + newtuple._annspecialcase_ = 'specialize:list_not_modified_any_more(1)' def newlist(self, list_w): return W_ListObject(list_w) Modified: pypy/branch/rlist-jit/pypy/rlib/debug.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rlib/debug.py (original) +++ pypy/branch/rlist-jit/pypy/rlib/debug.py Wed Nov 10 17:55:01 2010 @@ -239,15 +239,6 @@ def compute_result_annotation(self, s_arg): from pypy.annotation.model import SomeList assert isinstance(s_arg, SomeList) - # check that we don't call list_not_modified_any_more() on a list that - # is already the result of a previous list_not_modified_any_more(). - # This helps locating issues in a pypy translation: if newtuple() is - # first annotated with a list that comes e.g. from fixedview(), it - # will immediately crash here instead of later when newtuple() is - # also annotated with a normal list. - assert not s_arg.listdef.listitem.must_not_mutate, ( - "argument already has the flag 'must_not_mutate'") - s_arg.listdef.mutate() # the logic behind it is that we try not to propagate # make_sure_not_resized, when list comprehension is not on if self.bookkeeper.annotator.translator.config.translation.list_comprehension_operations: From afa at codespeak.net Wed Nov 10 20:08:14 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 10 Nov 2010 20:08:14 +0100 (CET) Subject: [pypy-svn] r78979 - pypy/trunk/pypy/module/_stackless/test Message-ID: <20101110190814.E2D71282B90@codespeak.net> Author: afa Date: Wed Nov 10 20:08:12 2010 New Revision: 78979 Added: pypy/trunk/pypy/module/_stackless/test/conftest.py (contents, props changed) Log: Skip stackless tests on Windows: they segfault :-( Added: pypy/trunk/pypy/module/_stackless/test/conftest.py ============================================================================== --- (empty file) +++ pypy/trunk/pypy/module/_stackless/test/conftest.py Wed Nov 10 20:08:12 2010 @@ -0,0 +1,7 @@ +import sys +import py.test + +def pytest_runtest_setup(item): + if sys.platform == 'win32': + py.test.skip("stackless tests segfault on Windows") + From pedronis at codespeak.net Wed Nov 10 20:15:14 2010 From: pedronis at codespeak.net (pedronis at codespeak.net) Date: Wed, 10 Nov 2010 20:15:14 +0100 (CET) Subject: [pypy-svn] r78980 - pypy/extradoc/planning/hg-migration Message-ID: <20101110191514.98D9C36C224@codespeak.net> Author: pedronis Date: Wed Nov 10 20:15:13 2010 New Revision: 78980 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: add my bitbucket mail address Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Wed Nov 10 20:15:13 2010 @@ -8,7 +8,7 @@ arigo=Armin Rigo fijal=Maciej Fijalkowski cfbolz=Carl Friedrich Bolz -pedronis=Samuele Pedroni +pedronis=Samuele Pedroni antocuni=Antonio Cuni hpk=Holger Krekel mwh=Michael Hudson From fijal at codespeak.net Wed Nov 10 20:32:42 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 10 Nov 2010 20:32:42 +0100 (CET) Subject: [pypy-svn] r78981 - pypy/trunk/pypy/rpython/memory/gc Message-ID: <20101110193242.14154282B90@codespeak.net> Author: fijal Date: Wed Nov 10 20:32:40 2010 New Revision: 78981 Modified: pypy/trunk/pypy/rpython/memory/gc/inspector.py Log: prebuilt_gc_objects is also a source of roots Modified: pypy/trunk/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/inspector.py Wed Nov 10 20:32:40 2010 @@ -174,6 +174,9 @@ def add_roots(self): self.gc._heap_dumper = self + if not self.gc.prebuilt_gc_objects_are_static_roots: + self.gc.prebuilt_root_objects.foreach(self.gc._collect_obj, + self.pending) self.gc.root_walker.walk_roots( _hd_add_root, _hd_add_root, From fijal at codespeak.net Wed Nov 10 20:33:05 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 10 Nov 2010 20:33:05 +0100 (CET) Subject: [pypy-svn] r78982 - pypy/trunk/pypy/rpython/memory/gctransform Message-ID: <20101110193305.60ECE5080B@codespeak.net> Author: fijal Date: Wed Nov 10 20:33:03 2010 New Revision: 78982 Modified: pypy/trunk/pypy/rpython/memory/gctransform/framework.py Log: Make BaseRootWalker a new style class Modified: pypy/trunk/pypy/rpython/memory/gctransform/framework.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gctransform/framework.py (original) +++ pypy/trunk/pypy/rpython/memory/gctransform/framework.py Wed Nov 10 20:33:03 2010 @@ -1210,7 +1210,7 @@ sizeofaddr = llmemory.sizeof(llmemory.Address) -class BaseRootWalker: +class BaseRootWalker(object): need_root_stack = False def __init__(self, gctransformer): From fijal at codespeak.net Wed Nov 10 20:43:28 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 10 Nov 2010 20:43:28 +0100 (CET) Subject: [pypy-svn] r78983 - pypy/trunk/pypy/doc Message-ID: <20101110194328.8E290282B9C@codespeak.net> Author: fijal Date: Wed Nov 10 20:43:27 2010 New Revision: 78983 Modified: pypy/trunk/pypy/doc/release-1.4.0.txt Log: (kirma) typo Modified: pypy/trunk/pypy/doc/release-1.4.0.txt ============================================================================== --- pypy/trunk/pypy/doc/release-1.4.0.txt (original) +++ pypy/trunk/pypy/doc/release-1.4.0.txt Wed Nov 10 20:43:27 2010 @@ -4,7 +4,7 @@ Hello. -We're pleased to announce release of PyPy 1.4. This is a major breaktrhough +We're pleased to announce release of PyPy 1.4. This is a major breakthrough in our long journey, since PyPy 1.4 is the first PyPy release that can translate itself faster than CPython. Since today, we plan to start using PyPy for our own development. From agaynor at codespeak.net Wed Nov 10 21:02:03 2010 From: agaynor at codespeak.net (agaynor at codespeak.net) Date: Wed, 10 Nov 2010 21:02:03 +0100 (CET) Subject: [pypy-svn] r78984 - pypy/trunk/pypy/doc Message-ID: <20101110200203.BF1D8282B9D@codespeak.net> Author: agaynor Date: Wed Nov 10 21:01:45 2010 New Revision: 78984 Modified: pypy/trunk/pypy/doc/release-1.4.0.txt Log: Various grammar fixes. Modified: pypy/trunk/pypy/doc/release-1.4.0.txt ============================================================================== --- pypy/trunk/pypy/doc/release-1.4.0.txt (original) +++ pypy/trunk/pypy/doc/release-1.4.0.txt Wed Nov 10 21:01:45 2010 @@ -4,14 +4,14 @@ Hello. -We're pleased to announce release of PyPy 1.4. This is a major breakthrough -in our long journey, since PyPy 1.4 is the first PyPy release that can translate -itself faster than CPython. Since today, we plan to start using -PyPy for our own development. +We're pleased to announce the 1.4 release of PyPy. This is a major breakthrough +in our long journey, as PyPy 1.4 is the first PyPy release that can translate +itself faster than CPython. Starting today, we plan to start using PyPy for our +own development. Among other features, this release includes numerous performance improvements -(which made fast self-hosting possible), 64jit backend as well as serious -stabilization. As of now, we can consider 32bit version of PyPy stable enough +(which made fast self-hosting possible), 64-bit JIT backend, as well as serious +stabilization. As of now, we can consider 32-bit version of PyPy stable enough to run in production. More highlights @@ -21,4 +21,3 @@ to use it, you need a recent version of virtualenv (>= 1.5). XXX write me: better regular expressions - From benjamin at codespeak.net Thu Nov 11 02:47:53 2010 From: benjamin at codespeak.net (benjamin at codespeak.net) Date: Thu, 11 Nov 2010 02:47:53 +0100 (CET) Subject: [pypy-svn] r78985 - pypy/trunk/pypy/jit/metainterp Message-ID: <20101111014753.56E8936C222@codespeak.net> Author: benjamin Date: Thu Nov 11 02:47:49 2010 New Revision: 78985 Modified: pypy/trunk/pypy/jit/metainterp/pyjitpl.py Log: add space Modified: pypy/trunk/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/trunk/pypy/jit/metainterp/pyjitpl.py Thu Nov 11 02:47:49 2010 @@ -1624,7 +1624,7 @@ assert jitdriver_sd is self.jitdriver_sd self.create_empty_history() try: - original_boxes = self.initialize_original_boxes(jitdriver_sd,*args) + original_boxes = self.initialize_original_boxes(jitdriver_sd, *args) return self._compile_and_run_once(original_boxes) finally: self.staticdata.profiler.end_tracing() From afa at codespeak.net Thu Nov 11 08:46:42 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 11 Nov 2010 08:46:42 +0100 (CET) Subject: [pypy-svn] r78986 - pypy/trunk/pypy/module/posix Message-ID: <20101111074642.7949936C222@codespeak.net> Author: afa Date: Thu Nov 11 08:46:39 2010 New Revision: 78986 Modified: pypy/trunk/pypy/module/posix/interp_posix.py Log: Fix a MallocMismatch on Windows Modified: pypy/trunk/pypy/module/posix/interp_posix.py ============================================================================== --- pypy/trunk/pypy/module/posix/interp_posix.py (original) +++ pypy/trunk/pypy/module/posix/interp_posix.py Thu Nov 11 08:46:39 2010 @@ -454,7 +454,8 @@ self.w_environ = space.newdict() if _WIN: self.cryptProviderPtr = lltype.malloc( - rffi.CArray(HCRYPTPROV), 1, zero=True, flavor='raw') + rffi.CArray(HCRYPTPROV), 1, zero=True, + flavor='raw', immortal=True) def startup(self, space): _convertenviron(space, self.w_environ) def _freeze_(self): From arigo at codespeak.net Thu Nov 11 10:09:52 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 10:09:52 +0100 (CET) Subject: [pypy-svn] r78987 - in pypy/branch/rlist-jit/pypy/module/cpyext: . include test Message-ID: <20101111090952.0CEB1282B90@codespeak.net> Author: arigo Date: Thu Nov 11 10:09:49 2010 New Revision: 78987 Modified: pypy/branch/rlist-jit/pypy/module/cpyext/api.py pypy/branch/rlist-jit/pypy/module/cpyext/include/tupleobject.h pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py Log: Implement PyTupleObject similarly to PyStringObject to support PyTuple_SetItem(). Intermediate check-in: segfaults. Modified: pypy/branch/rlist-jit/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/api.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/api.py Thu Nov 11 10:09:49 2010 @@ -367,7 +367,7 @@ }.items(): GLOBALS['Py%s_Type#' % (cpyname, )] = ('PyTypeObject*', pypyexpr) - for cpyname in 'Method List Int Long Dict Tuple Class'.split(): + for cpyname in 'Method List Int Long Dict Class'.split(): FORWARD_DECLS.append('typedef struct { PyObject_HEAD } ' 'Py%sObject' % (cpyname, )) build_exported_objects() Modified: pypy/branch/rlist-jit/pypy/module/cpyext/include/tupleobject.h ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/include/tupleobject.h (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/include/tupleobject.h Thu Nov 11 10:09:49 2010 @@ -10,9 +10,18 @@ /* defined in varargswrapper.c */ PyObject * PyTuple_Pack(Py_ssize_t, ...); -#define PyTuple_SET_ITEM PyTuple_SetItem -#define PyTuple_GET_ITEM PyTuple_GetItem +typedef struct { + PyObject_HEAD + PyObject **items; + Py_ssize_t size; +} PyTupleObject; +/* Macro, trading safety for speed */ +#define PyTuple_GET_ITEM(op, i) (((PyTupleObject *)(op))->items[i]) +#define PyTuple_GET_SIZE(op) (((PyTupleObject *)(op))->size) + +/* Macro, *only* to be used to fill in brand new tuples */ +#define PyTuple_SET_ITEM(op, i, v) (((PyTupleObject *)(op))->items[i] = v) #ifdef __cplusplus } Modified: pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py Thu Nov 11 10:09:49 2010 @@ -6,25 +6,28 @@ class TestTupleObject(BaseApiTest): def test_tupleobject(self, space, api): + XXX assert not api.PyTuple_Check(space.w_None) - assert api.PyTuple_SetItem(space.w_None, 0, space.w_None) == -1 + #assert api.PyTuple_SetItem(space.w_None, 0, space.w_None) == -1 XXX atuple = space.newtuple([0, 1, 'yay']) assert api.PyTuple_Size(atuple) == 3 assert api.PyTuple_GET_SIZE(atuple) == 3 - raises(TypeError, api.PyTuple_Size(space.newlist([]))) + #raises(TypeError, api.PyTuple_Size(space.newlist([]))) XXX api.PyErr_Clear() def test_tuple_resize(self, space, api): - py_tuple = api.PyTuple_New(3) + XXX + ref_tup = api.PyTuple_New(3) ar = lltype.malloc(PyObjectP.TO, 1, flavor='raw') - ar[0] = rffi.cast(PyObject, make_ref(space, py_tuple)) + ar[0] = rffi.cast(PyObject, ref_tup) api._PyTuple_Resize(ar, 2) - py_tuple = from_ref(space, ar[0]) - assert len(py_tuple.wrappeditems) == 2 + assert ar[0] == rffi.cast(PyObject, ref_tup) + # ^^^ our _PyTuple_Resize does not actually need to change the ptr so far + assert api.PyTuple_Size(ar[0]) == 2 + assert api.PyTuple_GET_SIZE(ar[0]) == 2 api._PyTuple_Resize(ar, 10) - py_tuple = from_ref(space, ar[0]) - assert len(py_tuple.wrappeditems) == 10 + assert api.PyTuple_Size(ar[0]) == 10 api.Py_DecRef(ar[0]) lltype.free(ar, flavor='raw') Modified: pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py Thu Nov 11 10:09:49 2010 @@ -1,55 +1,129 @@ from pypy.interpreter.error import OperationError from pypy.rpython.lltypesystem import rffi, lltype from pypy.module.cpyext.api import (cpython_api, Py_ssize_t, CANNOT_FAIL, - build_type_checkers) + build_type_checkers, PyObjectFields, + cpython_struct, bootstrap_function) from pypy.module.cpyext.pyobject import (PyObject, PyObjectP, Py_DecRef, - borrow_from, make_ref, from_ref) + borrow_from, make_ref, from_ref, make_typedescr) from pypy.module.cpyext.pyerrors import PyErr_BadInternalCall -from pypy.objspace.std.tupleobject import W_TupleObject +## +## Implementation of PyTupleObject +## =============================== +## +## We have the same problem as PyStringObject: a PyTupleObject can be +## initially used in a read-write way with PyTuple_New(), PyTuple_SetItem() +## and _PyTuple_Resize(). +## +## The 'size' and 'items' fields of a PyTupleObject are always valid. +## Apart from that detail, see the big comment in stringobject.py for +## more information. +## + +ARRAY_OF_PYOBJ = rffi.CArrayPtr(PyObject) +PyTupleObjectStruct = lltype.ForwardReference() +PyTupleObject = lltype.Ptr(PyTupleObjectStruct) +PyTupleObjectFields = PyObjectFields + \ + (("items", ARRAY_OF_PYOBJ), ("size", Py_ssize_t)) +cpython_struct("PyTupleObject", PyTupleObjectFields, PyTupleObjectStruct) + + at bootstrap_function +def init_tupleobject(space): + "Type description of PyTupleObject" + make_typedescr(space.w_tuple.instancetypedef, + basestruct=PyTupleObject.TO, + attach=tuple_attach, + dealloc=tuple_dealloc, + realize=tuple_realize) PyTuple_Check, PyTuple_CheckExact = build_type_checkers("Tuple") +def new_empty_tuple(space, length): + """ + Allocate a PyTupleObject and its array, but without a corresponding + interpreter object. The array items may be mutated, until + tuple_realize() is called. + """ + typedescr = get_typedescr(space.w_tuple.instancetypedef) + py_obj = typedescr.allocate(space, space.w_tuple) + py_tup = rffi.cast(PyTupleObject, py_obj) + + py_tup.c_items = lltype.malloc(ARRAY_OF_PYOBJ.TO, length, + flavor='raw', zero=True) + py_tup.c_size = length + return py_tup + +def tuple_attach(space, py_obj, w_obj): + """ + Fills a newly allocated PyTupleObject with the given tuple object. + """ + items_w = space.fixedview(w_obj) + py_tup = rffi.cast(PyTupleObject, py_obj) + py_tup.c_items = lltype.malloc(ARRAY_OF_PYOBJ.TO, len(items_w), + flavor='raw') + py_tup.c_size = len(items_w) + for i in range(len(items_w)): + py_tup.c_items[i] = make_ref(space, items_w[i]) + +def tuple_realize(space, py_obj): + """ + Creates the tuple in the interpreter. The PyTupleObject items array + must not be modified after this call. + """ + py_tup = rffi.cast(PyTupleObject, py_obj) + # If your CPython extension creates a self-referential tuple + # with PyTuple_SetItem(), you loose. + c_items = py_tup.c_items + items_w = [from_ref(space, c_items[i]) for i in range(py_tup.c_size)] + w_obj = space.newtuple(items_w) + track_reference(space, py_obj, w_obj) + return w_obj + + at cpython_api([PyObject], lltype.Void, external=False) +def tuple_dealloc(space, py_obj): + """Frees allocated PyTupleObject resources. + """ + py_tup = rffi.cast(PyTupleObject, py_obj) + for i in range(py_tup.c_size): + Py_DecRef(space, py_tup.c_items[i]) + lltype.free(py_tup.c_items, flavor="raw") + from pypy.module.cpyext.object import PyObject_dealloc + PyObject_dealloc(space, py_obj) + +#_______________________________________________________________________ + @cpython_api([Py_ssize_t], PyObject) def PyTuple_New(space, size): - return space.newtuple([space.w_None] * size) + return rffi.cast(PyObject, new_empty_tuple(space, size)) @cpython_api([PyObject, Py_ssize_t, PyObject], rffi.INT_real, error=-1) -def PyTuple_SetItem(space, w_t, pos, w_obj): - if not PyTuple_Check(space, w_t): - # XXX this should also steal a reference, test it!!! - PyErr_BadInternalCall(space) - assert isinstance(w_t, W_TupleObject) - w_t.wrappeditems[pos] = w_obj - Py_DecRef(space, w_obj) # SetItem steals a reference! +def PyTuple_SetItem(space, ref, pos, ref_item): + # XXX do PyTuple_Check, without forcing ref as an interpreter object + # XXX -- then if it fails it should also steal a reference, test it!!! + ref_tup = rffi.cast(PyTupleObject, ref) + # XXX raise some exception if PyTuple_SetItem() is called on an + # tuple that already has a corresponding interpreter object + ref_old = ref_tup.c_items[pos] + ref_tup.c_items[pos] = ref # SetItem steals a reference! + Py_DecRef(space, ref_old) return 0 @cpython_api([PyObject, Py_ssize_t], PyObject) -def PyTuple_GetItem(space, w_t, pos): - if not PyTuple_Check(space, w_t): - PyErr_BadInternalCall(space) - assert isinstance(w_t, W_TupleObject) - w_obj = w_t.wrappeditems[pos] - return borrow_from(w_t, w_obj) - - at cpython_api([PyObject], Py_ssize_t, error=CANNOT_FAIL) -def PyTuple_GET_SIZE(space, w_t): - """Return the size of the tuple p, which must be non-NULL and point to a tuple; - no error checking is performed. """ - assert isinstance(w_t, W_TupleObject) - return len(w_t.wrappeditems) +def PyTuple_GetItem(space, ref, pos): + # XXX do PyTuple_Check, without forcing ref as an interpreter object + ref_tup = rffi.cast(PyTupleObject, ref) + return ref_tup.c_items[pos] # borrowed reference @cpython_api([PyObject], Py_ssize_t, error=-1) def PyTuple_Size(space, ref): """Take a pointer to a tuple object, and return the size of that tuple.""" - if not PyTuple_Check(space, ref): - raise OperationError(space.w_TypeError, - space.wrap("expected tuple object")) - return PyTuple_GET_SIZE(space, ref) + # XXX do PyTuple_Check, without forcing ref as an interpreter object + ref_tup = rffi.cast(PyTupleObject, ref) + return ref_tup.c_size @cpython_api([PyObjectP, Py_ssize_t], rffi.INT_real, error=-1) -def _PyTuple_Resize(space, ref, newsize): +def _PyTuple_Resize(space, refp, newsize): """Can be used to resize a tuple. newsize will be the new length of the tuple. Because tuples are supposed to be immutable, this should only be used if there is only one reference to the object. Do not use this if the tuple may already @@ -60,18 +134,17 @@ this function. If the object referenced by *p is replaced, the original *p is destroyed. On failure, returns -1 and sets *p to NULL, and raises MemoryError or SystemError.""" - py_tuple = from_ref(space, ref[0]) - if not PyTuple_Check(space, py_tuple): - PyErr_BadInternalCall(space) - assert isinstance(py_tuple, W_TupleObject) - py_newtuple = PyTuple_New(space, newsize) - - to_cp = newsize - oldsize = len(py_tuple.wrappeditems) - if oldsize < newsize: - to_cp = oldsize - for i in range(to_cp): - py_newtuple.wrappeditems[i] = py_tuple.wrappeditems[i] - Py_DecRef(space, ref[0]) - ref[0] = make_ref(space, py_newtuple) + # XXX do PyTuple_Check, without forcing ref as an interpreter object + # XXX -- then if it fails it should reset refp[0] to null + ref_tup = rffi.cast(PyTupleObject, refp[0]) + c_newitems = lltype.malloc(ARRAY_OF_PYOBJ.TO, newsize, + flavor='raw', zero=True) + c_olditems = ref_tup.c_items + for i in range(min(oldsize, newsize)): + c_newitems[i] = c_olditems[i] + # decref items deleted by shrinkage + for i in range(newsize, oldsize): + Py_DecRef(space, c_olditems[i]) + ref_tup.c_items = c_newitems + lltype.free(c_olditems, flavor='raw') return 0 From arigo at codespeak.net Thu Nov 11 10:39:20 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 10:39:20 +0100 (CET) Subject: [pypy-svn] r78988 - pypy/trunk/lib_pypy/ctypes_config_cache/test Message-ID: <20101111093920.BA90A5080B@codespeak.net> Author: arigo Date: Thu Nov 11 10:39:17 2010 New Revision: 78988 Modified: pypy/trunk/lib_pypy/ctypes_config_cache/test/test_cache.py Log: Whack at test_cache until it really really seems to work... :-/ Modified: pypy/trunk/lib_pypy/ctypes_config_cache/test/test_cache.py ============================================================================== --- pypy/trunk/lib_pypy/ctypes_config_cache/test/test_cache.py (original) +++ pypy/trunk/lib_pypy/ctypes_config_cache/test/test_cache.py Thu Nov 11 10:39:17 2010 @@ -7,9 +7,8 @@ def run(filename, outputname): filepath = dirpath.join(filename) - tmpdir2 = udir.ensure('testcache-' + filename, dir=True) - tmpdir = tmpdir2.ensure('ctypes_config_cache', dir=True) - tmpdir.join('__init__.py').write('\n') + tmpdir = udir.ensure('testcache-' + os.path.splitext(filename)[0], + dir=True) tmpdir.join('dumpcache.py').write(dirpath.join('dumpcache.py').read()) path = sys.path[:] try: @@ -23,14 +22,12 @@ assert outputpath.check(exists=1) modname = os.path.splitext(outputname)[0] try: - sys.path.insert(0, str(tmpdir2)) + sys.path.insert(0, str(tmpdir)) d = {} - exec "from ctypes_config_cache import %s" % modname in d - mod = d[modname] + execfile(str(outputpath), d) finally: sys.path[:] = path - sys.modules.pop('ctypes_config_cache', None) - return mod.__dict__ + return d def test_syslog(): From david at codespeak.net Thu Nov 11 10:42:27 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 11 Nov 2010 10:42:27 +0100 (CET) Subject: [pypy-svn] r78989 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101111094227.20FB036C220@codespeak.net> Author: david Date: Thu Nov 11 10:42:25 2010 New Revision: 78989 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Log: Refactor the generation of the functions to write machine instructions Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/instruction_builder.py Thu Nov 11 10:42:25 2010 @@ -101,7 +101,7 @@ | (imm & 0xF)) return f -def define_data_proc_imm(name, table): +def define_data_proc_imm_func(name, table): n = (0x1 << 25 | (table['op'] & 0x1F) << 20) if table['result'] and table['base']: @@ -129,7 +129,7 @@ | imm_operation(0, rn, imm)) return imm_func -def define_data_proc(name, table): +def define_data_proc_func(name, table): n = ((table['op1'] & 0x1F) << 20 | (table['op2'] & 0x1F) << 7 | (table['op3'] & 0x3) << 5) @@ -165,7 +165,7 @@ | reg_operation(rd, rn, rm, imm, s, shifttype)) return f -def define_data_proc_register_shifted(name, table): +def define_data_proc_reg_shift_reg_func(name, table): n = ((0x1 << 4) | (table['op1'] & 0x1F) << 20 | (table['op2'] & 0x3) << 5) if 'result' in table and not table['result']: result = False @@ -201,7 +201,7 @@ | (rn & 0xF)) return f -def define_supervisor_and_coproc(name, table): +def define_supervisor_and_coproc_func(name, table): n = (0x3 << 26 | (table['op1'] & 0x3F) << 20 | (table['op'] & 0x1) << 4) def f(self, coproc, opc1, rt, crn, crm, opc2=0, cond=cond.AL): self.write32(n @@ -214,7 +214,7 @@ | (crm & 0xF)) return f -def define_multiply_instructions(name, table): +def define_multiply_func(name, table): n = (table['op'] & 0xF) << 20 | 0x9 << 4 if 'acc' in table and table['acc']: if 'update_flags' in table and table['update_flags']: @@ -288,15 +288,14 @@ setattr(target, key, f) def define_instructions(target): - i_g_map = [(instructions.load_store, define_load_store_func), - (instructions.extra_load_store, define_extra_load_store_func), - (instructions.data_proc, define_data_proc), - (instructions.data_proc_imm, define_data_proc_imm), - (instructions.supervisor_and_coproc, define_supervisor_and_coproc), - (instructions.multiply, define_multiply_instructions), - (instructions.data_proc_reg_shift_reg, define_data_proc_register_shifted), - (instructions.block_data, define_block_data_func)] - - for inss, gen in i_g_map: - for key, val in inss.iteritems(): - define_instruction(gen, key, val, target) + inss = [k for k in instructions.__dict__.keys() if not k.startswith('__')] + for name in inss: + if name == 'branch': + continue + try: + func = globals()['define_%s_func' % name] + except KeyError: + print 'No instr generator for %s instructions' % name + continue + for key, value in getattr(instructions, name).iteritems(): + define_instruction(func, key, value, target) From david at codespeak.net Thu Nov 11 10:50:38 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 11 Nov 2010 10:50:38 +0100 (CET) Subject: [pypy-svn] r78990 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101111095038.70B365080C@codespeak.net> Author: david Date: Thu Nov 11 10:50:37 2010 New Revision: 78990 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Temporary hack to check equality flags for mul_ovf, because arm does not set the overflow flags for this operation Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Thu Nov 11 10:50:37 2010 @@ -95,9 +95,9 @@ reg2 = regalloc.make_sure_var_in_reg(a1, [a0], imm_fine=False) res = regalloc.force_allocate_reg(op.result, [a0, a1]) self.mc.SMULL(res.value, r.ip.value, reg1.value, reg2.value, cond=fcond) - self.mc.CMP_rr(r.ip.value, res.value, shifttype=shift.ASR, s=31, cond=fcond) + self.mc.CMP_rr(r.ip.value, res.value, shifttype=shift.ASR, imm=31, cond=fcond) regalloc.possibly_free_vars_for_op(op) - return fcond + return 0xF # XXX Remove: hack to show that the prev operation was a mul_ovf emit_op_int_floordiv = gen_emit_op_by_helper_call('DIV') emit_op_int_mod = gen_emit_op_by_helper_call('MOD') @@ -176,9 +176,13 @@ return self._emit_guard(op, regalloc, fcond) def emit_op_guard_no_overflow(self, op, regalloc, fcond): + if fcond == 0xF: # XXX: hack to check if the prev op was a mul_ovf + return self._emit_guard(op, regalloc, c.NE) return self._emit_guard(op, regalloc, c.VS) def emit_op_guard_overflow(self, op, regalloc, fcond): + if fcond == 0xF: # XXX: hack to check if the prev op was a mul_ovf + return self._emit_guard(op, regalloc, c.EQ) return self._emit_guard(op, regalloc, c.VC) class OpAssembler(object): From arigo at codespeak.net Thu Nov 11 11:21:18 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 11:21:18 +0100 (CET) Subject: [pypy-svn] r78991 - pypy/build/bot2/pypybuildbot Message-ID: <20101111102118.3A245282B90@codespeak.net> Author: arigo Date: Thu Nov 11 11:21:17 2010 New Revision: 78991 Modified: pypy/build/bot2/pypybuildbot/master.py Log: Tentative: change the category names to match the platform. Modified: pypy/build/bot2/pypybuildbot/master.py ============================================================================== --- pypy/build/bot2/pypybuildbot/master.py (original) +++ pypy/build/bot2/pypybuildbot/master.py Thu Nov 11 11:21:17 2010 @@ -226,91 +226,91 @@ "slavenames": ["cobra", "bigdogvm1", "tannit32"], "builddir": LINUX32, "factory": pypyOwnTestFactory, - "category": 'own' + "category": 'linux32' }, {"name": LINUX64, "slavenames": ["tannit64"], "builddir": LINUX64, "factory": pypyOwnTestFactory, - "category": 'own64' + "category": 'linux64' }, {"name": MACOSX32, "slavenames": ["minime"], "builddir": MACOSX32, "factory": pypyOwnTestFactory, - "category": 'mac' + "category": 'mac32' }, {"name": WIN32, "slavenames": ["bigboard"], "builddir": WIN32, "factory": pypyOwnTestFactoryWin, - "category": 'own' + "category": 'win32' }, {"name": APPLVLLINUX32, "slavenames": ["bigdogvm1", "tannit32"], "builddir": APPLVLLINUX32, "factory": pypyTranslatedAppLevelTestFactory, - 'category': 'applevel' + 'category': 'linux32' }, {"name": APPLVLLINUX64, "slavenames": ["tannit64"], "builddir": APPLVLLINUX64, "factory": pypyTranslatedAppLevelTestFactory64, - "category": "applevel64" + "category": "linux64" }, {"name": STACKLESSAPPLVLLINUX32, "slavenames": ["bigdogvm1", "tannit32"], "builddir": STACKLESSAPPLVLLINUX32, "factory": pypyStacklessTranslatedAppLevelTestFactory, - "category": 'stackless' + "category": 'linux32-stackless' }, {"name": OJITLINUX32, "slavenames": ["bigdogvm1", "tannit32"], "builddir": OJITLINUX32, "factory": pypy_OjitTranslatedTestFactory, - "category": 'applevel' + "category": 'linux32' }, {"name": APPLVLWIN32, "slavenames": ["bigboard"], "builddir": APPLVLWIN32, "factory": pypyTranslatedAppLevelTestFactoryWin, - "category": "windows" + "category": "win32" }, {"name" : STACKLESSAPPLVLFREEBSD64, "slavenames": ['headless'], 'builddir' : STACKLESSAPPLVLFREEBSD64, 'factory' : pypyStacklessTranslatedAppLevelTestFactory, - "category": 'other' + "category": 'freebsd64-stackless' }, {"name" : JITLINUX32, "slavenames": ["bigdogvm1", "tannit32"], 'builddir' : JITLINUX32, 'factory' : pypyJITTranslatedTestFactory, - 'category' : 'jit', + 'category' : 'linux32', }, {'name': JITLINUX64, 'slavenames': ['tannit64'], 'builddir': JITLINUX64, 'factory': pypyJITTranslatedTestFactory64, - 'category': 'jit64', + 'category': 'linux64', }, {"name" : JITMACOSX64, "slavenames": ["macmini-mvt"], 'builddir' : JITMACOSX64, 'factory' : pypyJITTranslatedTestFactoryOSX, - 'category' : 'jit', + 'category' : 'mac64', }, {"name" : JITWIN32, "slavenames": ["bigboard"], 'builddir' : JITWIN32, 'factory' : pypyJITTranslatedTestFactoryWin, - 'category' : 'jit', + 'category' : 'win32', }, {"name": JITONLYLINUX32, "slavenames": ["tannit32", "bigdogvm1"], "builddir": JITONLYLINUX32, "factory": pypyJitOnlyOwnTestFactory, - "category": 'own' + "category": 'linux32' }, {"name": JITBENCH, "slavenames": ["tannit32"], From arigo at codespeak.net Thu Nov 11 11:28:58 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 11:28:58 +0100 (CET) Subject: [pypy-svn] r78992 - pypy/build/bot2/pypybuildbot Message-ID: <20101111102858.68B59282B90@codespeak.net> Author: arigo Date: Thu Nov 11 11:28:56 2010 New Revision: 78992 Modified: pypy/build/bot2/pypybuildbot/master.py Log: Adapt the branch_order_prefixes. Modified: pypy/build/bot2/pypybuildbot/master.py ============================================================================== --- pypy/build/bot2/pypybuildbot/master.py (original) +++ pypy/build/bot2/pypybuildbot/master.py Thu Nov 11 11:28:56 2010 @@ -86,12 +86,7 @@ # pypy test summary page summary = load('pypybuildbot.summary') -status.putChild('summary', summary.Summary(['own', 'applevel', - 'lib-python', 'jit', - 'stackless', - 'windows', 'mac', - 'benchmark-run', - 'other'])) +status.putChild('summary', summary.Summary(['linux', 'mac', 'win', 'freebsd'])) status.putChild('nightly', PyPyList(os.path.expanduser('~/nightly'), defaultType='application/octet-stream')) From david at codespeak.net Thu Nov 11 11:32:45 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 11 Nov 2010 11:32:45 +0100 (CET) Subject: [pypy-svn] r78993 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101111103245.4CA1A282B9C@codespeak.net> Author: david Date: Thu Nov 11 11:32:43 2010 New Revision: 78993 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Improve guard_true and guard_false and implement guard_value, guard_nonnull and guard_isnull Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Thu Nov 11 11:32:43 2010 @@ -166,14 +166,33 @@ return c.AL def emit_op_guard_true(self, op, regalloc, fcond): - assert fcond == c.LE - cond = c.get_opposite_of(fcond) - assert cond == c.GT - return self._emit_guard(op, regalloc, cond) + a0 = op.getarg(0) + l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + self.mc.CMP_ri(l0.value, 0) + regalloc.possibly_free_var(l0) + return self._emit_guard(op, regalloc, c.EQ) def emit_op_guard_false(self, op, regalloc, fcond): - assert fcond == c.EQ - return self._emit_guard(op, regalloc, fcond) + a0 = op.getarg(0) + l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + self.mc.CMP_ri(l0.value, 0) + regalloc.possibly_free_var(l0) + return self._emit_guard(op, regalloc, c.NE) + + def emit_op_guard_value(self, op, regalloc, fcond): + a0 = op.getarg(0) + a1 = op.getarg(1) + l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(a1) + if l1.is_imm(): + self.mc.CMP_rr(l0.value, l1.getint()) + else: + self.mc.CMP_rr(l0.value, l1.value) + regalloc.possibly_free_vars_for_op(op) + return self._emit_guard(op, regalloc, c.NE) + + emit_op_guard_nonnull = emit_op_guard_true + emit_op_guard_isnull = emit_op_guard_false def emit_op_guard_no_overflow(self, op, regalloc, fcond): if fcond == 0xF: # XXX: hack to check if the prev op was a mul_ovf From arigo at codespeak.net Thu Nov 11 11:40:40 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 11:40:40 +0100 (CET) Subject: [pypy-svn] r78994 - pypy/build/bot2/pypybuildbot Message-ID: <20101111104040.98A75282B9D@codespeak.net> Author: arigo Date: Thu Nov 11 11:40:39 2010 New Revision: 78994 Modified: pypy/build/bot2/pypybuildbot/master.py pypy/build/bot2/pypybuildbot/summary.py Log: Allow category prefixes too. Modified: pypy/build/bot2/pypybuildbot/master.py ============================================================================== --- pypy/build/bot2/pypybuildbot/master.py (original) +++ pypy/build/bot2/pypybuildbot/master.py Thu Nov 11 11:40:39 2010 @@ -86,7 +86,10 @@ # pypy test summary page summary = load('pypybuildbot.summary') -status.putChild('summary', summary.Summary(['linux', 'mac', 'win', 'freebsd'])) +status.putChild('summary', summary.Summary(categories=['linux', + 'mac', + 'win', + 'freebsd'])) status.putChild('nightly', PyPyList(os.path.expanduser('~/nightly'), defaultType='application/octet-stream')) Modified: pypy/build/bot2/pypybuildbot/summary.py ============================================================================== --- pypy/build/bot2/pypybuildbot/summary.py (original) +++ pypy/build/bot2/pypybuildbot/summary.py Thu Nov 11 11:40:39 2010 @@ -771,12 +771,12 @@ break else: branch_key = (len(self.branch_order_prefixes)+1, branch) - try: - i = self.categories.index(category) - cat_key = (0, i) - except ValueError: - cat_key = (1, category) - + for i, catprefix in enumerate(self.categories): + if category.startswith(catprefix): + break + else: + i = len(self.categories) + cat_key = (i, category) return cat_key + branch_key def body(self, request): From david at codespeak.net Thu Nov 11 13:54:15 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 11 Nov 2010 13:54:15 +0100 (CET) Subject: [pypy-svn] r78995 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101111125415.321D0282B90@codespeak.net> Author: david Date: Thu Nov 11 13:54:13 2010 New Revision: 78995 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Implement string operations strlen, strgetitem and strsetitem Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Thu Nov 11 13:54:13 2010 @@ -11,19 +11,13 @@ from pypy.rlib import rgc from pypy.rpython.annlowlevel import llhelper from pypy.rpython.lltypesystem import lltype, rffi, llmemory -from pypy.jit.backend.arm.opassembler import (GuardOpAssembler, - IntOpAsslember, - OpAssembler, - UnaryIntOpAssembler, - FieldOpAssembler, - ArrayOpAssember) +from pypy.jit.backend.arm.opassembler import ResOpAssembler + # XXX Move to llsupport from pypy.jit.backend.x86.support import values_array -class AssemblerARM(GuardOpAssembler, IntOpAsslember, - OpAssembler, UnaryIntOpAssembler, - FieldOpAssembler, ArrayOpAssember): +class AssemblerARM(ResOpAssembler): def __init__(self, cpu, failargs_limit=1000): self.mc = ARMv7Builder() Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Thu Nov 11 13:54:13 2010 @@ -10,15 +10,17 @@ gen_emit_op_ri, gen_emit_cmp_op) from pypy.jit.backend.arm.codebuilder import ARMv7Builder, ARMv7InMemoryBuilder from pypy.jit.backend.arm.regalloc import ARMRegisterManager +from pypy.jit.backend.llsupport import symbolic from pypy.jit.backend.llsupport.descr import BaseFieldDescr, BaseArrayDescr from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity, TempBox from pypy.jit.metainterp.history import ConstInt, BoxInt, Box, BasicFailDescr from pypy.jit.metainterp.resoperation import rop from pypy.rlib import rgc from pypy.rpython.annlowlevel import llhelper -from pypy.rpython.lltypesystem import lltype, rffi, llmemory +from pypy.rpython.lltypesystem import lltype, rffi, rstr, llmemory class IntOpAsslember(object): + _mixin_ = True def emit_op_int_add(self, op, regalloc, fcond): @@ -129,6 +131,9 @@ class UnaryIntOpAssembler(object): + + _mixin_ = True + emit_op_int_is_true = gen_emit_op_unary_cmp(c.NE, c.EQ) emit_op_int_is_zero = gen_emit_op_unary_cmp(c.EQ, c.NE) @@ -152,6 +157,7 @@ return fcond class GuardOpAssembler(object): + _mixin_ = True def _emit_guard(self, op, regalloc, fcond): @@ -205,6 +211,7 @@ return self._emit_guard(op, regalloc, c.VC) class OpAssembler(object): + _mixin_ = True def emit_op_jump(self, op, regalloc, fcond): @@ -252,6 +259,8 @@ class FieldOpAssembler(object): + _mixin_ = True + def emit_op_setfield_gc(self, op, regalloc, fcond): a0 = op.getarg(0) a1 = op.getarg(1) @@ -306,6 +315,8 @@ class ArrayOpAssember(object): + _mixin_ = True + def emit_op_arraylen_gc(self, op, regalloc, fcond): arraydescr = op.getdescr() assert isinstance(arraydescr, BaseArrayDescr) @@ -377,3 +388,66 @@ scale += 1 assert (1 << scale) == size return scale, ofs, ofs_length, ptr + +class StrOpAssembler(object): + _mixin_ = True + + def emit_op_strlen(self, op, regalloc, fcond): + l0 = regalloc.make_sure_var_in_reg(op.getarg(0)) + regalloc.possibly_free_vars_for_op(op) + res = regalloc.force_allocate_reg(op.result) + basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.STR, + self.cpu.translate_support_code) + l1 = regalloc.make_sure_var_in_reg(ConstInt(ofs_length)) + if l1.is_imm(): + self.mc.LDR_ri(res.value, l0.value, l1.getint(), cond=fcond) + else: + self.mc.LDR_rr(res.value, l0.value, l1.value, cond=fcond) + return fcond + + def emit_op_strgetitem(self, op, regalloc, fcond): + base_loc = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=True) + ofs_loc = regalloc.make_sure_var_in_reg(op.getarg(1)) + t = TempBox() + temp = regalloc.force_allocate_reg(t) + res = regalloc.force_allocate_reg(op.result) + regalloc.possibly_free_vars_for_op(op) + regalloc.possibly_free_var(t) + + basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.STR, + self.cpu.translate_support_code) + assert itemsize == 1 + if ofs_loc.is_imm(): + self.mc.ADD_ri(temp.value, base_loc.value, ofs_loc.getint(), cond=fcond) + else: + self.mc.ADD_rr(temp.value, base_loc.value, ofs_loc.value, cond=fcond) + + self.mc.LDRB_ri(res.value, temp.value, basesize, cond=fcond) + return fcond + + def emit_op_strsetitem(self, op, regalloc, fcond): + base_loc = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=True) + ofs_loc = regalloc.make_sure_var_in_reg(op.getarg(1)) + value_loc = regalloc.make_sure_var_in_reg(op.getarg(2)) + t = TempBox() + temp = regalloc.force_allocate_reg(t) + regalloc.possibly_free_vars_for_op(op) + regalloc.possibly_free_var(t) + + basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.STR, + self.cpu.translate_support_code) + assert itemsize == 1 + if ofs_loc.is_imm(): + self.mc.ADD_ri(temp.value, base_loc.value, ofs_loc.getint(), cond=fcond) + else: + self.mc.ADD_rr(temp.value, base_loc.value, ofs_loc.value, cond=fcond) + + self.mc.STRB_ri(value_loc.value, temp.value, basesize, cond=fcond) + return fcond + +class ResOpAssembler(GuardOpAssembler, IntOpAsslember, + OpAssembler, UnaryIntOpAssembler, + FieldOpAssembler, ArrayOpAssember, + StrOpAssembler): + pass + From david at codespeak.net Thu Nov 11 14:21:54 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 11 Nov 2010 14:21:54 +0100 (CET) Subject: [pypy-svn] r78997 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101111132154.F004E5080B@codespeak.net> Author: david Date: Thu Nov 11 14:21:53 2010 New Revision: 78997 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Implement unicode operations unicodelen, unicodegetitem and unicodesetitem Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Thu Nov 11 14:21:53 2010 @@ -393,7 +393,7 @@ _mixin_ = True def emit_op_strlen(self, op, regalloc, fcond): - l0 = regalloc.make_sure_var_in_reg(op.getarg(0)) + l0 = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) regalloc.possibly_free_vars_for_op(op) res = regalloc.force_allocate_reg(op.result) basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.STR, @@ -406,7 +406,7 @@ return fcond def emit_op_strgetitem(self, op, regalloc, fcond): - base_loc = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=True) + base_loc = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) ofs_loc = regalloc.make_sure_var_in_reg(op.getarg(1)) t = TempBox() temp = regalloc.force_allocate_reg(t) @@ -426,7 +426,7 @@ return fcond def emit_op_strsetitem(self, op, regalloc, fcond): - base_loc = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=True) + base_loc = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) ofs_loc = regalloc.make_sure_var_in_reg(op.getarg(1)) value_loc = regalloc.make_sure_var_in_reg(op.getarg(2)) t = TempBox() @@ -445,9 +445,72 @@ self.mc.STRB_ri(value_loc.value, temp.value, basesize, cond=fcond) return fcond +class UnicodeOpAssembler(object): + _mixin_ = True + + def emit_op_unicodelen(self, op, regalloc, fcond): + l0 = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) + regalloc.possibly_free_vars_for_op(op) + res = regalloc.force_allocate_reg(op.result) + basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.UNICODE, + self.cpu.translate_support_code) + l1 = regalloc.make_sure_var_in_reg(ConstInt(ofs_length)) + if l1.is_imm(): + self.mc.LDR_ri(res.value, l0.value, l1.getint(), cond=fcond) + else: + self.mc.LDR_rr(res.value, l0.value, l1.value, cond=fcond) + return fcond + + def emit_op_unicodegetitem(self, op, regalloc, fcond): + base_loc = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) + ofs_loc = regalloc.make_sure_var_in_reg(op.getarg(1), imm_fine=False) + t = TempBox() + temp = regalloc.force_allocate_reg(t) + res = regalloc.force_allocate_reg(op.result) + regalloc.possibly_free_vars_for_op(op) + regalloc.possibly_free_var(t) + + basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.UNICODE, + self.cpu.translate_support_code) + scale = itemsize/2 + if scale == 2: + f = self.mc.LDR_ri + elif scale == 1: + f = self.mc.LDRH_ri + else: + assert 0, itemsize + self.mc.ADD_rr(temp.value, base_loc.value, ofs_loc.value, cond=fcond, + imm=scale, shifttype=shift.LSL) + f(res.value, temp.value, basesize, cond=fcond) + return fcond + + def emit_op_unicodesetitem(self, op, regalloc, fcond): + base_loc = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) + ofs_loc = regalloc.make_sure_var_in_reg(op.getarg(1)) + value_loc = regalloc.make_sure_var_in_reg(op.getarg(2)) + t = TempBox() + temp = regalloc.force_allocate_reg(t) + regalloc.possibly_free_vars_for_op(op) + regalloc.possibly_free_var(t) + + basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.UNICODE, + self.cpu.translate_support_code) + scale = itemsize/2 + if scale == 2: + f = self.mc.STR_ri + elif scale == 1: + f = self.mc.STRH_ri + else: + assert 0, itemsize + + self.mc.ADD_rr(temp.value, base_loc.value, ofs_loc.value, cond=fcond, + imm=scale, shifttype=shift.LSL) + f(value_loc.value, temp.value, basesize, cond=fcond) + return fcond + class ResOpAssembler(GuardOpAssembler, IntOpAsslember, OpAssembler, UnaryIntOpAssembler, FieldOpAssembler, ArrayOpAssember, - StrOpAssembler): + StrOpAssembler, UnicodeOpAssembler): pass From david at codespeak.net Thu Nov 11 15:12:42 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 11 Nov 2010 15:12:42 +0100 (CET) Subject: [pypy-svn] r78999 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . helper Message-ID: <20101111141242.670A3282B90@codespeak.net> Author: david Date: Thu Nov 11 15:12:40 2010 New Revision: 78999 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Some register allocation related fixes Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Thu Nov 11 15:12:40 2010 @@ -4,8 +4,8 @@ from pypy.jit.backend.arm.arch import WORD, FUNC_ALIGN from pypy.jit.backend.arm.codebuilder import ARMv7Builder, ARMv7InMemoryBuilder from pypy.jit.backend.arm.regalloc import ARMRegisterManager, ARMFrameManager -from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity -from pypy.jit.metainterp.history import (ConstInt, BoxInt, Box, BasicFailDescr, +from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity, TempBox +from pypy.jit.metainterp.history import (ConstInt, BoxInt, BasicFailDescr, INT, REF, FLOAT) from pypy.jit.metainterp.resoperation import rop from pypy.rlib import rgc @@ -135,7 +135,7 @@ \xFE = Empty arg """ - box = Box() + box = TempBox() reg = regalloc.force_allocate_reg(box) # XXX free this memory # XXX allocate correct amount of memory @@ -185,8 +185,7 @@ # guards going to be patched are allways conditional if fcond != c.AL: op.getdescr()._arm_guard_reg = reg - else: - regalloc.possibly_free_var(reg) + regalloc.possibly_free_var(box) return memaddr def align(self): @@ -259,19 +258,19 @@ if regalloc.frame_manager.frame_depth == 1: return n = (regalloc.frame_manager.frame_depth)*WORD - self._adjust_sp(n, cb) + self._adjust_sp(n, regalloc, cb) - def _adjust_sp(self, n, cb=None, fcond=c.AL): + def _adjust_sp(self, n, regalloc, cb=None, fcond=c.AL): if cb is None: cb = self.mc if n <= 0xFF and fcond == c.AL: cb.SUB_ri(r.sp.value, r.sp.value, n) else: - b = Box() + b = TempBox() reg = regalloc.force_allocate_reg(b) cb.gen_load_int(reg.value, n, cond=fcond) cb.SUB_rr(r.sp.value, r.sp.value, reg.value, cond=fcond) - regalloc.possibly_free_var(reg) + regalloc.possibly_free_var(b) def assemble_bridge(self, faildescr, inputargs, operations): enc = rffi.cast(rffi.CCHARP, faildescr._failure_recovery_code) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Thu Nov 11 15:12:40 2010 @@ -7,7 +7,7 @@ from pypy.rlib.rmmap import alloc, PTR from pypy.rpython.annlowlevel import llhelper from pypy.rpython.lltypesystem import lltype, rffi -from pypy.jit.metainterp.history import ConstInt, BoxInt, Box, BasicFailDescr +from pypy.jit.metainterp.history import ConstInt, BoxInt, BasicFailDescr def binary_helper_call(name): signature = getattr(arch, 'arm_%s_sign' % name) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Thu Nov 11 15:12:40 2010 @@ -1,6 +1,6 @@ from pypy.jit.backend.arm import conditions as c from pypy.jit.backend.arm import registers as r -from pypy.jit.metainterp.history import ConstInt, BoxInt, Box +from pypy.jit.metainterp.history import ConstInt, BoxInt def gen_emit_op_unary_cmp(true_cond, false_cond): def f(self, op, regalloc, fcond): @@ -58,7 +58,6 @@ def gen_emit_cmp_op(condition, inverse=False): def f(self, op, regalloc, fcond): - assert fcond == c.AL if not inverse: arg0 = op.getarg(0) arg1 = op.getarg(1) @@ -77,11 +76,11 @@ l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) l1 = regalloc.make_sure_var_in_reg(arg1, [l0], imm_fine=False) res = regalloc.force_allocate_reg(op.result) - self.mc.CMP_rr(l0.value, l1.value) + self.mc.CMP_rr(l0.value, l1.value, cond=fcond) inv = c.get_opposite_of(condition) self.mc.MOV_ri(res.value, 1, cond=condition) self.mc.MOV_ri(res.value, 0, cond=inv) regalloc.possibly_free_vars_for_op(op) - return condition + return fcond return f Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Thu Nov 11 15:12:40 2010 @@ -13,7 +13,7 @@ from pypy.jit.backend.llsupport import symbolic from pypy.jit.backend.llsupport.descr import BaseFieldDescr, BaseArrayDescr from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity, TempBox -from pypy.jit.metainterp.history import ConstInt, BoxInt, Box, BasicFailDescr +from pypy.jit.metainterp.history import ConstInt, BoxInt, BasicFailDescr from pypy.jit.metainterp.resoperation import rop from pypy.rlib import rgc from pypy.rpython.annlowlevel import llhelper From afa at codespeak.net Thu Nov 11 15:27:11 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 11 Nov 2010 15:27:11 +0100 (CET) Subject: [pypy-svn] r79000 - pypy/trunk/pypy/module/cpyext Message-ID: <20101111142711.5468F5080B@codespeak.net> Author: afa Date: Thu Nov 11 15:27:09 2010 New Revision: 79000 Modified: pypy/trunk/pypy/module/cpyext/api.py pypy/trunk/pypy/module/cpyext/pyobject.py Log: cpyext: introduce "Reference", to serve as a common base between raw PyObject references and BorrowPair(container, borrowed) objects. Modified: pypy/trunk/pypy/module/cpyext/api.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/api.py (original) +++ pypy/trunk/pypy/module/cpyext/api.py Thu Nov 11 15:27:09 2010 @@ -226,7 +226,7 @@ def unwrapper(space, *args): from pypy.module.cpyext.pyobject import Py_DecRef from pypy.module.cpyext.pyobject import make_ref, from_ref - from pypy.module.cpyext.pyobject import BorrowPair + from pypy.module.cpyext.pyobject import Reference newargs = () to_decref = [] assert len(args) == len(api_function.argtypes) @@ -270,8 +270,8 @@ return api_function.error_value if res is None: return None - elif isinstance(res, BorrowPair): - return res.w_borrowed + elif isinstance(res, Reference): + return res.get_wrapped(space) else: return res finally: @@ -473,7 +473,7 @@ @specialize.ll() def wrapper(*args): from pypy.module.cpyext.pyobject import make_ref, from_ref - from pypy.module.cpyext.pyobject import BorrowPair + from pypy.module.cpyext.pyobject import Reference # we hope that malloc removal removes the newtuple() that is # inserted exactly here by the varargs specializer llop.gc_stack_bottom(lltype.Void) # marker for trackgcroot.py @@ -525,7 +525,7 @@ elif is_PyObject(callable.api_func.restype): if result is None: retval = make_ref(space, None) - elif isinstance(result, BorrowPair): + elif isinstance(result, Reference): retval = result.get_ref(space) elif not rffi._isllptr(result): retval = rffi.cast(callable.api_func.restype, Modified: pypy/trunk/pypy/module/cpyext/pyobject.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/pyobject.py (original) +++ pypy/trunk/pypy/module/cpyext/pyobject.py Thu Nov 11 15:27:09 2010 @@ -424,7 +424,18 @@ state = space.fromcache(RefcountState) return state.make_borrowed(w_container, w_borrowed) -class BorrowPair: +class Reference: + def __init__(self, pyobj): + assert not isinstance(pyobj, W_Root) + self.pyobj = pyobj + + def get_ref(self, space): + return self.pyobj + + def get_wrapped(self, space): + return from_ref(space, self.pyobj) + +class BorrowPair(Reference): """ Delays the creation of a borrowed reference. """ @@ -435,6 +446,9 @@ def get_ref(self, space): return make_borrowed_ref(space, self.w_container, self.w_borrowed) + def get_wrapped(self, space): + return self.w_borrowed + def borrow_from(container, borrowed): return BorrowPair(container, borrowed) From antocuni at codespeak.net Thu Nov 11 16:48:21 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 11 Nov 2010 16:48:21 +0100 (CET) Subject: [pypy-svn] r79001 - pypy/trunk/pypy/jit/tool Message-ID: <20101111154821.82B7E282B9D@codespeak.net> Author: antocuni Date: Thu Nov 11 16:48:19 2010 New Revision: 79001 Modified: pypy/trunk/pypy/jit/tool/pypytrace-mode.el Log: fix this regexp, it seems that the format of debug_merge_points is slightly different nowadays Modified: pypy/trunk/pypy/jit/tool/pypytrace-mode.el ============================================================================== --- pypy/trunk/pypy/jit/tool/pypytrace-mode.el (original) +++ pypy/trunk/pypy/jit/tool/pypytrace-mode.el Thu Nov 11 16:48:19 2010 @@ -26,7 +26,7 @@ ("<.*FieldDescr \\([^ ]*\\)" (1 'font-lock-variable-name-face)) ;; comment out debug_merge_point, but then highlight specific part of it ("^debug_merge_point.*" . font-lock-comment-face) - ("^\\(debug_merge_point\\).*code object\\(.*\\), file \\('.*'\\), \\(line .*\\)> \\(.*\\)')" + ("^\\(debug_merge_point\\).*code object\\(.*\\), file \\('.*'\\), \\(line .*\\)> \\(.*\\)" (1 'compilation-warning t) (2 'escape-glyph t) (3 'font-lock-string-face t) From arigo at codespeak.net Thu Nov 11 17:13:35 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 17:13:35 +0100 (CET) Subject: [pypy-svn] r79002 - pypy/branch/rlist-jit/pypy/module/cpyext Message-ID: <20101111161335.F02745080B@codespeak.net> Author: arigo Date: Thu Nov 11 17:13:33 2010 New Revision: 79002 Modified: pypy/branch/rlist-jit/pypy/module/cpyext/api.py pypy/branch/rlist-jit/pypy/module/cpyext/pyobject.py Log: Merge r79000 from trunk. Modified: pypy/branch/rlist-jit/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/api.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/api.py Thu Nov 11 17:13:33 2010 @@ -226,7 +226,7 @@ def unwrapper(space, *args): from pypy.module.cpyext.pyobject import Py_DecRef from pypy.module.cpyext.pyobject import make_ref, from_ref - from pypy.module.cpyext.pyobject import BorrowPair + from pypy.module.cpyext.pyobject import Reference newargs = () to_decref = [] assert len(args) == len(api_function.argtypes) @@ -270,8 +270,8 @@ return api_function.error_value if res is None: return None - elif isinstance(res, BorrowPair): - return res.w_borrowed + elif isinstance(res, Reference): + return res.get_wrapped(space) else: return res finally: @@ -473,7 +473,7 @@ @specialize.ll() def wrapper(*args): from pypy.module.cpyext.pyobject import make_ref, from_ref - from pypy.module.cpyext.pyobject import BorrowPair + from pypy.module.cpyext.pyobject import Reference # we hope that malloc removal removes the newtuple() that is # inserted exactly here by the varargs specializer llop.gc_stack_bottom(lltype.Void) # marker for trackgcroot.py @@ -525,7 +525,7 @@ elif is_PyObject(callable.api_func.restype): if result is None: retval = make_ref(space, None) - elif isinstance(result, BorrowPair): + elif isinstance(result, Reference): retval = result.get_ref(space) elif not rffi._isllptr(result): retval = rffi.cast(callable.api_func.restype, Modified: pypy/branch/rlist-jit/pypy/module/cpyext/pyobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/pyobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/pyobject.py Thu Nov 11 17:13:33 2010 @@ -424,7 +424,18 @@ state = space.fromcache(RefcountState) return state.make_borrowed(w_container, w_borrowed) -class BorrowPair: +class Reference: + def __init__(self, pyobj): + assert not isinstance(pyobj, W_Root) + self.pyobj = pyobj + + def get_ref(self, space): + return self.pyobj + + def get_wrapped(self, space): + return from_ref(space, self.pyobj) + +class BorrowPair(Reference): """ Delays the creation of a borrowed reference. """ @@ -435,6 +446,9 @@ def get_ref(self, space): return make_borrowed_ref(space, self.w_container, self.w_borrowed) + def get_wrapped(self, space): + return self.w_borrowed + def borrow_from(container, borrowed): return BorrowPair(container, borrowed) From arigo at codespeak.net Thu Nov 11 17:23:16 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 17:23:16 +0100 (CET) Subject: [pypy-svn] r79003 - in pypy/branch/rlist-jit/pypy/module/cpyext: . include test Message-ID: <20101111162316.49523282B9D@codespeak.net> Author: arigo Date: Thu Nov 11 17:23:14 2010 New Revision: 79003 Modified: pypy/branch/rlist-jit/pypy/module/cpyext/include/tupleobject.h pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py Log: Fix the problem with tupleobject.py: copying eagerly the objects from the interpreter world to the PyObject world does not work (bootstrapping issues when creating types). Modified: pypy/branch/rlist-jit/pypy/module/cpyext/include/tupleobject.h ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/include/tupleobject.h (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/include/tupleobject.h Thu Nov 11 17:23:14 2010 @@ -16,8 +16,9 @@ Py_ssize_t size; } PyTupleObject; +#define PyTuple_GET_ITEM PyTuple_GetItem + /* Macro, trading safety for speed */ -#define PyTuple_GET_ITEM(op, i) (((PyTupleObject *)(op))->items[i]) #define PyTuple_GET_SIZE(op) (((PyTupleObject *)(op))->size) /* Macro, *only* to be used to fill in brand new tuples */ Modified: pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py Thu Nov 11 17:23:14 2010 @@ -6,17 +6,14 @@ class TestTupleObject(BaseApiTest): def test_tupleobject(self, space, api): - XXX assert not api.PyTuple_Check(space.w_None) #assert api.PyTuple_SetItem(space.w_None, 0, space.w_None) == -1 XXX atuple = space.newtuple([0, 1, 'yay']) assert api.PyTuple_Size(atuple) == 3 - assert api.PyTuple_GET_SIZE(atuple) == 3 #raises(TypeError, api.PyTuple_Size(space.newlist([]))) XXX api.PyErr_Clear() def test_tuple_resize(self, space, api): - XXX ref_tup = api.PyTuple_New(3) ar = lltype.malloc(PyObjectP.TO, 1, flavor='raw') ar[0] = rffi.cast(PyObject, ref_tup) @@ -24,7 +21,6 @@ assert ar[0] == rffi.cast(PyObject, ref_tup) # ^^^ our _PyTuple_Resize does not actually need to change the ptr so far assert api.PyTuple_Size(ar[0]) == 2 - assert api.PyTuple_GET_SIZE(ar[0]) == 2 api._PyTuple_Resize(ar, 10) assert api.PyTuple_Size(ar[0]) == 10 Modified: pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py Thu Nov 11 17:23:14 2010 @@ -4,7 +4,7 @@ build_type_checkers, PyObjectFields, cpython_struct, bootstrap_function) from pypy.module.cpyext.pyobject import (PyObject, PyObjectP, Py_DecRef, - borrow_from, make_ref, from_ref, make_typedescr) + borrow_from, make_ref, from_ref, make_typedescr, get_typedescr) from pypy.module.cpyext.pyerrors import PyErr_BadInternalCall ## @@ -59,11 +59,8 @@ """ items_w = space.fixedview(w_obj) py_tup = rffi.cast(PyTupleObject, py_obj) - py_tup.c_items = lltype.malloc(ARRAY_OF_PYOBJ.TO, len(items_w), - flavor='raw') + py_tup.c_items = lltype.nullptr(ARRAY_OF_PYOBJ.TO) py_tup.c_size = len(items_w) - for i in range(len(items_w)): - py_tup.c_items[i] = make_ref(space, items_w[i]) def tuple_realize(space, py_obj): """ @@ -84,9 +81,10 @@ """Frees allocated PyTupleObject resources. """ py_tup = rffi.cast(PyTupleObject, py_obj) - for i in range(py_tup.c_size): - Py_DecRef(space, py_tup.c_items[i]) - lltype.free(py_tup.c_items, flavor="raw") + if py_tup.c_items: + for i in range(py_tup.c_size): + Py_DecRef(space, py_tup.c_items[i]) + lltype.free(py_tup.c_items, flavor="raw") from pypy.module.cpyext.object import PyObject_dealloc PyObject_dealloc(space, py_obj) @@ -101,8 +99,9 @@ # XXX do PyTuple_Check, without forcing ref as an interpreter object # XXX -- then if it fails it should also steal a reference, test it!!! ref_tup = rffi.cast(PyTupleObject, ref) - # XXX raise some exception if PyTuple_SetItem() is called on an - # tuple that already has a corresponding interpreter object + if not ref_tup.c_items: + msg = "PyTuple_SetItem() called on an already-escaped tuple object" + raise OperationError(space.w_SystemError, space.wrap(msg)) ref_old = ref_tup.c_items[pos] ref_tup.c_items[pos] = ref # SetItem steals a reference! Py_DecRef(space, ref_old) @@ -112,7 +111,12 @@ def PyTuple_GetItem(space, ref, pos): # XXX do PyTuple_Check, without forcing ref as an interpreter object ref_tup = rffi.cast(PyTupleObject, ref) - return ref_tup.c_items[pos] # borrowed reference + if ref_tup.c_items: + return Reference(ref_tup.c_items[pos]) # borrowed reference + else: + w_t = from_ref(space, ref) + w_obj = space.getitem(w_t, space.wrap(pos)) + return borrow_from(w_t, w_obj) @cpython_api([PyObject], Py_ssize_t, error=-1) def PyTuple_Size(space, ref): @@ -140,11 +144,16 @@ c_newitems = lltype.malloc(ARRAY_OF_PYOBJ.TO, newsize, flavor='raw', zero=True) c_olditems = ref_tup.c_items + if not c_olditems: + msg = "_PyTuple_Resize() called on an already-escaped tuple object" + raise OperationError(space.w_SystemError, space.wrap(msg)) + oldsize = ref_tup.c_size for i in range(min(oldsize, newsize)): c_newitems[i] = c_olditems[i] # decref items deleted by shrinkage for i in range(newsize, oldsize): Py_DecRef(space, c_olditems[i]) ref_tup.c_items = c_newitems + ref_tup.c_size = newsize lltype.free(c_olditems, flavor='raw') return 0 From arigo at codespeak.net Thu Nov 11 17:32:13 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 17:32:13 +0100 (CET) Subject: [pypy-svn] r79004 - in pypy/trunk/pypy/jit: codewriter codewriter/test metainterp metainterp/optimizeopt metainterp/test Message-ID: <20101111163213.698135080C@codespeak.net> Author: arigo Date: Thu Nov 11 17:32:11 2010 New Revision: 79004 Modified: pypy/trunk/pypy/jit/codewriter/assembler.py pypy/trunk/pypy/jit/codewriter/call.py pypy/trunk/pypy/jit/codewriter/codewriter.py pypy/trunk/pypy/jit/codewriter/effectinfo.py pypy/trunk/pypy/jit/codewriter/jtransform.py pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py pypy/trunk/pypy/jit/metainterp/optimizeopt/string.py pypy/trunk/pypy/jit/metainterp/pyjitpl.py pypy/trunk/pypy/jit/metainterp/resume.py pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py pypy/trunk/pypy/jit/metainterp/test/test_resume.py pypy/trunk/pypy/jit/metainterp/test/test_virtualref.py Log: Remove the (only?) global data that we have, because it contains addresses of functions, and you cannot send to the C backend the address of a function whose graph was translated with another exceptiontransformer. General Rule: Global Data Is Really Bad. Fixes jit/backend/x86/test/test_ztranslation. Modified: pypy/trunk/pypy/jit/codewriter/assembler.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/assembler.py (original) +++ pypy/trunk/pypy/jit/codewriter/assembler.py Thu Nov 11 17:32:11 2010 @@ -233,10 +233,9 @@ addr = llmemory.cast_ptr_to_adr(value) self.list_of_addr2name.append((addr, name)) - def finished(self): + def finished(self, callinfocollection): # Helper called at the end of assembling. Registers the extra # functions shown in _callinfo_for_oopspec. - from pypy.jit.codewriter.effectinfo import _callinfo_for_oopspec - for _, func in _callinfo_for_oopspec.values(): + for func in callinfocollection.all_function_addresses_as_int(): func = heaptracker.int2adr(func) self.see_raw_object(func.ptr) Modified: pypy/trunk/pypy/jit/codewriter/call.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/call.py (original) +++ pypy/trunk/pypy/jit/codewriter/call.py Thu Nov 11 17:32:11 2010 @@ -7,7 +7,7 @@ from pypy.jit.codewriter.jitcode import JitCode from pypy.jit.codewriter.effectinfo import VirtualizableAnalyzer from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze -from pypy.jit.codewriter.effectinfo import EffectInfo +from pypy.jit.codewriter.effectinfo import EffectInfo, CallInfoCollection from pypy.translator.simplify import get_funcobj, get_functype from pypy.rpython.lltypesystem import lltype, llmemory from pypy.translator.backendopt.canraise import RaiseAnalyzer @@ -23,6 +23,7 @@ self.jitdrivers_sd = jitdrivers_sd self.jitcodes = {} # map {graph: jitcode} self.unfinished_graphs = [] # list of graphs with pending jitcodes + self.callinfocollection = CallInfoCollection() if hasattr(cpu, 'rtyper'): # for tests self.rtyper = cpu.rtyper translator = self.rtyper.annotator.translator Modified: pypy/trunk/pypy/jit/codewriter/codewriter.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/codewriter.py (original) +++ pypy/trunk/pypy/jit/codewriter/codewriter.py Thu Nov 11 17:32:11 2010 @@ -73,7 +73,7 @@ count += 1 if not count % 500: log.info("Produced %d jitcodes" % count) - self.assembler.finished() + self.assembler.finished(self.callcontrol.callinfocollection) heaptracker.finish_registering(self.cpu) log.info("there are %d JitCode instances." % count) Modified: pypy/trunk/pypy/jit/codewriter/effectinfo.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/effectinfo.py (original) +++ pypy/trunk/pypy/jit/codewriter/effectinfo.py Thu Nov 11 17:32:11 2010 @@ -144,30 +144,44 @@ # ____________________________________________________________ -_callinfo_for_oopspec = {} # {oopspecindex: (calldescr, func_as_int)} - -def callinfo_for_oopspec(oopspecindex): - """A function that returns the calldescr and the function - address (as an int) of one of the OS_XYZ functions defined above. - Don't use this if there might be several implementations of the same - OS_XYZ specialized by type, e.g. OS_ARRAYCOPY.""" - try: - return _callinfo_for_oopspec[oopspecindex] - except KeyError: - return (None, 0) - - -def _funcptr_for_oopspec_memo(oopspecindex): - from pypy.jit.codewriter import heaptracker - _, func_as_int = callinfo_for_oopspec(oopspecindex) - funcadr = heaptracker.int2adr(func_as_int) - return funcadr.ptr -_funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo' - -def funcptr_for_oopspec(oopspecindex): - """A memo function that returns a pointer to the function described - by OS_XYZ (as a real low-level function pointer).""" - funcptr = _funcptr_for_oopspec_memo(oopspecindex) - assert funcptr - return funcptr -funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(0)' +class CallInfoCollection(object): + def __init__(self): + # {oopspecindex: (calldescr, func_as_int)} + self._callinfo_for_oopspec = {} + + def _freeze_(self): + return True + + def add(self, oopspecindex, calldescr, func_as_int): + self._callinfo_for_oopspec[oopspecindex] = calldescr, func_as_int + + def has_oopspec(self, oopspecindex): + return oopspecindex in self._callinfo_for_oopspec + + def all_function_addresses_as_int(self): + return [func for (_, func) in self._callinfo_for_oopspec.values()] + + def callinfo_for_oopspec(self, oopspecindex): + """A function that returns the calldescr and the function + address (as an int) of one of the OS_XYZ functions defined above. + Don't use this if there might be several implementations of the same + OS_XYZ specialized by type, e.g. OS_ARRAYCOPY.""" + try: + return self._callinfo_for_oopspec[oopspecindex] + except KeyError: + return (None, 0) + + def _funcptr_for_oopspec_memo(self, oopspecindex): + from pypy.jit.codewriter import heaptracker + _, func_as_int = self.callinfo_for_oopspec(oopspecindex) + funcadr = heaptracker.int2adr(func_as_int) + return funcadr.ptr + _funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo' + + def funcptr_for_oopspec(self, oopspecindex): + """A memo function that returns a pointer to the function described + by OS_XYZ (as a real low-level function pointer).""" + funcptr = self._funcptr_for_oopspec_memo(oopspecindex) + assert funcptr + return funcptr + funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(1)' Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/jtransform.py (original) +++ pypy/trunk/pypy/jit/codewriter/jtransform.py Thu Nov 11 17:32:11 2010 @@ -6,7 +6,7 @@ from pypy.objspace.flow.model import Block, Link, c_last_exception from pypy.jit.codewriter.flatten import ListOfKind, IndirectCallTargets from pypy.jit.codewriter import support, heaptracker -from pypy.jit.codewriter.effectinfo import EffectInfo, _callinfo_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.jit.codewriter.policy import log from pypy.jit.metainterp.typesystem import deref, arrayItem from pypy.rlib import objectmodel @@ -1084,7 +1084,8 @@ else: func = heaptracker.adr2int( llmemory.cast_ptr_to_adr(op.args[0].value)) - _callinfo_for_oopspec[oopspecindex] = calldescr, func + self.callcontrol.callinfocollection.add(oopspecindex, + calldescr, func) op1 = self.rewrite_call(op, 'residual_call', [op.args[0], calldescr], args=args) @@ -1095,7 +1096,7 @@ def _register_extra_helper(self, oopspecindex, oopspec_name, argtypes, resulttype): # a bit hackish - if oopspecindex in _callinfo_for_oopspec: + if self.callcontrol.callinfocollection.has_oopspec(oopspecindex): return c_func, TP = support.builtin_func_for_spec(self.cpu.rtyper, oopspec_name, argtypes, @@ -1109,7 +1110,7 @@ else: func = heaptracker.adr2int( llmemory.cast_ptr_to_adr(c_func.value)) - _callinfo_for_oopspec[oopspecindex] = calldescr, func + self.callcontrol.callinfocollection.add(oopspecindex, calldescr, func) def _handle_stroruni_call(self, op, oopspec_name, args): SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE) Modified: pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py (original) +++ pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py Thu Nov 11 17:32:11 2010 @@ -74,7 +74,20 @@ def calldescr_canraise(self, calldescr): return False +class FakeCallInfoCollection: + def __init__(self): + self.seen = [] + def add(self, oopspecindex, calldescr, func): + self.seen.append((oopspecindex, calldescr, func)) + def has_oopspec(self, oopspecindex): + for i, c, f in self.seen: + if i == oopspecindex: + return True + return False + class FakeBuiltinCallControl: + def __init__(self): + self.callinfocollection = FakeCallInfoCollection() def guess_call_kind(self, op): return 'builtin' def getcalldescr(self, op, oopspecindex=None): @@ -810,7 +823,8 @@ v2 = varoftype(PSTR) v3 = varoftype(PSTR) op = SpaceOperation('direct_call', [const(func), v1, v2], v3) - tr = Transformer(FakeCPU(), FakeBuiltinCallControl()) + cc = FakeBuiltinCallControl() + tr = Transformer(FakeCPU(), cc) op1 = tr.rewrite_operation(op) assert op1.opname == 'residual_call_r_r' assert op1.args[0].value == func @@ -819,9 +833,10 @@ assert op1.result == v3 # # check the callinfo_for_oopspec - got = effectinfo.callinfo_for_oopspec(effectinfo.EffectInfo.OS_UNI_CONCAT) - assert got[0] == op1.args[1] # the calldescr - assert heaptracker.int2adr(got[1]) == llmemory.cast_ptr_to_adr(func) + got = cc.callinfocollection.seen[0] + assert got[0] == effectinfo.EffectInfo.OS_UNI_CONCAT + assert got[1] == op1.args[1] # the calldescr + assert heaptracker.int2adr(got[2]) == llmemory.cast_ptr_to_adr(func) def test_str_slice(): # test that the oopspec is present and correctly transformed @@ -893,7 +908,8 @@ v2 = varoftype(PUNICODE) v3 = varoftype(lltype.Bool) op = SpaceOperation('direct_call', [const(func), v1, v2], v3) - tr = Transformer(FakeCPU(), FakeBuiltinCallControl()) + cc = FakeBuiltinCallControl() + tr = Transformer(FakeCPU(), cc) op1 = tr.rewrite_operation(op) assert op1.opname == 'residual_call_r_i' assert op1.args[0].value == func @@ -901,9 +917,9 @@ assert op1.args[2] == ListOfKind('ref', [v1, v2]) assert op1.result == v3 # test that the OS_UNIEQ_* functions are registered - cifo = effectinfo._callinfo_for_oopspec - assert effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL in cifo - assert effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR in cifo + cic = cc.callinfocollection + assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL) + assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR) def test_list_ll_arraycopy(): from pypy.rlib.rgc import ll_arraycopy Modified: pypy/trunk/pypy/jit/metainterp/optimizeopt/string.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/optimizeopt/string.py (original) +++ pypy/trunk/pypy/jit/metainterp/optimizeopt/string.py Thu Nov 11 17:32:11 2010 @@ -9,7 +9,7 @@ from pypy.jit.metainterp.optimizeopt.optimizer import CONST_0, CONST_1 from pypy.jit.metainterp.optimizeopt.optimizer import llhelper from pypy.jit.metainterp.optimizeutil import _findall -from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.jit.codewriter import heaptracker from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.objectmodel import specialize, we_are_translated @@ -593,7 +593,8 @@ def generate_modified_call(self, oopspecindex, args, result, mode): oopspecindex += mode.OS_offset - calldescr, func = callinfo_for_oopspec(oopspecindex) + cic = self.optimizer.metainterp_sd.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(oopspecindex) op = ResOperation(rop.CALL, [ConstInt(func)] + args, result, descr=calldescr) self.optimizer.newoperations.append(op) Modified: pypy/trunk/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/trunk/pypy/jit/metainterp/pyjitpl.py Thu Nov 11 17:32:11 2010 @@ -1261,6 +1261,7 @@ # self.jitdrivers_sd = codewriter.callcontrol.jitdrivers_sd self.virtualref_info = codewriter.callcontrol.virtualref_info + self.callinfocollection = codewriter.callcontrol.callinfocollection self.setup_jitdrivers_sd(optimizer) # # store this information for fastpath of call_assembler Modified: pypy/trunk/pypy/jit/metainterp/resume.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/resume.py (original) +++ pypy/trunk/pypy/jit/metainterp/resume.py Thu Nov 11 17:32:11 2010 @@ -4,8 +4,7 @@ from pypy.jit.metainterp.history import INT, REF, FLOAT, HOLE from pypy.jit.metainterp.resoperation import rop from pypy.jit.metainterp import jitprof -from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec -from pypy.jit.codewriter.effectinfo import funcptr_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.rpython.lltypesystem import lltype, llmemory, rffi, rstr from pypy.rlib import rarithmetic from pypy.rlib.objectmodel import we_are_translated, specialize @@ -774,14 +773,16 @@ strbox, ConstInt(index), charbox) def concat_strings(self, str1num, str2num): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT) str1box = self.decode_box(str1num, REF) str2box = self.decode_box(str2num, REF) return self.metainterp.execute_and_record_varargs( rop.CALL, [ConstInt(func), str1box, str2box], calldescr) def slice_string(self, strnum, startnum, lengthnum): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_SLICE) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_SLICE) strbox = self.decode_box(strnum, REF) startbox = self.decode_box(startnum, INT) lengthbox = self.decode_box(lengthnum, INT) @@ -800,14 +801,16 @@ strbox, ConstInt(index), charbox) def concat_unicodes(self, str1num, str2num): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT) str1box = self.decode_box(str1num, REF) str2box = self.decode_box(str2num, REF) return self.metainterp.execute_and_record_varargs( rop.CALL, [ConstInt(func), str1box, str2box], calldescr) def slice_unicode(self, strnum, startnum, lengthnum): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE) strbox = self.decode_box(strnum, REF) startbox = self.decode_box(startnum, INT) lengthbox = self.decode_box(lengthnum, INT) @@ -903,8 +906,8 @@ def blackhole_from_resumedata(blackholeinterpbuilder, jitdriver_sd, storage, all_virtuals=None): - resumereader = ResumeDataDirectReader(blackholeinterpbuilder.cpu, storage, - all_virtuals) + resumereader = ResumeDataDirectReader(blackholeinterpbuilder.metainterp_sd, + storage, all_virtuals) vinfo = jitdriver_sd.virtualizable_info ginfo = jitdriver_sd.greenfield_info vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info @@ -939,7 +942,7 @@ return firstbh def force_from_resumedata(metainterp_sd, storage, vinfo, ginfo): - resumereader = ResumeDataDirectReader(metainterp_sd.cpu, storage) + resumereader = ResumeDataDirectReader(metainterp_sd, storage) resumereader.handling_async_forcing() vrefinfo = metainterp_sd.virtualref_info resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo) @@ -953,8 +956,9 @@ # 1: in handle_async_forcing # 2: resuming from the GUARD_NOT_FORCED - def __init__(self, cpu, storage, all_virtuals=None): - self._init(cpu, storage) + def __init__(self, metainterp_sd, storage, all_virtuals=None): + self._init(metainterp_sd.cpu, storage) + self.callinfocollection = metainterp_sd.callinfocollection if all_virtuals is None: # common case self._prepare(storage) else: @@ -1047,7 +1051,8 @@ str2 = self.decode_ref(str2num) str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str1) str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str2) - funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT) result = funcptr(str1, str2) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1056,7 +1061,8 @@ start = self.decode_int(startnum) length = self.decode_int(lengthnum) str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str) - funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_SLICE) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_SLICE) result = funcptr(str, start, start + length) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1072,7 +1078,8 @@ str2 = self.decode_ref(str2num) str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str1) str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str2) - funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT) result = funcptr(str1, str2) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1081,7 +1088,8 @@ start = self.decode_int(startnum) length = self.decode_int(lengthnum) str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str) - funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE) result = funcptr(str, start, start + length) return lltype.cast_opaque_ptr(llmemory.GCREF, result) Modified: pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py Thu Nov 11 17:32:11 2010 @@ -264,6 +264,8 @@ metainterp_sd = FakeMetaInterpStaticData(self.cpu) if hasattr(self, 'vrefinfo'): metainterp_sd.virtualref_info = self.vrefinfo + if hasattr(self, 'callinfocollection'): + metainterp_sd.callinfocollection = self.callinfocollection optimize_loop_1(metainterp_sd, loop) # expected = self.parse(optops) @@ -4180,22 +4182,20 @@ # ---------- def optimize_strunicode_loop_extradescrs(self, ops, spectext, optops): from pypy.jit.metainterp.optimizeopt import string - def my_callinfo_for_oopspec(oopspecindex): - calldescrtype = type(LLtypeMixin.strequaldescr) - for value in LLtypeMixin.__dict__.values(): - if isinstance(value, calldescrtype): - if (value.get_extra_info() and - value.get_extra_info().oopspecindex == oopspecindex): - # returns 0 for 'func' in this test - return value, 0 - raise AssertionError("not found: oopspecindex=%d" % oopspecindex) + class FakeCallInfoCollection: + def callinfo_for_oopspec(self, oopspecindex): + calldescrtype = type(LLtypeMixin.strequaldescr) + for value in LLtypeMixin.__dict__.values(): + if isinstance(value, calldescrtype): + extra = value.get_extra_info() + if extra and extra.oopspecindex == oopspecindex: + # returns 0 for 'func' in this test + return value, 0 + raise AssertionError("not found: oopspecindex=%d" % + oopspecindex) # - saved = string.callinfo_for_oopspec - try: - string.callinfo_for_oopspec = my_callinfo_for_oopspec - self.optimize_strunicode_loop(ops, spectext, optops) - finally: - string.callinfo_for_oopspec = saved + self.callinfocollection = FakeCallInfoCollection() + self.optimize_strunicode_loop(ops, spectext, optops) def test_str_equal_noop1(self): ops = """ Modified: pypy/trunk/pypy/jit/metainterp/test/test_resume.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_resume.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_resume.py Thu Nov 11 17:32:11 2010 @@ -51,6 +51,7 @@ class MyMetaInterp: _already_allocated_resume_virtuals = None + callinfocollection = None def __init__(self, cpu=None): if cpu is None: @@ -156,12 +157,12 @@ storage.rd_numb = numb # cpu = MyCPU([42, gcref1, -66]) - reader = ResumeDataDirectReader(cpu, storage) + metainterp = MyMetaInterp(cpu) + reader = ResumeDataDirectReader(metainterp, storage) _next_section(reader, 42, 111, gcrefnull, 42, gcref1) _next_section(reader, 222, 333) _next_section(reader, 42, gcref1, -66) # - metainterp = MyMetaInterp(cpu) reader = ResumeDataBoxReader(storage, metainterp) bi, br, bf = [None]*3, [None]*2, [None]*0 info = MyBlackholeInterp([lltype.Signed, lltype.Signed, @@ -193,7 +194,7 @@ storage.rd_numb = numb # cpu = MyCPU([]) - reader = ResumeDataDirectReader(cpu, storage) + reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage) _next_section(reader, 100) @@ -211,7 +212,7 @@ class FakeMetainterp(object): _already_allocated_resume_virtuals = None cpu = None - reader = ResumeDataDirectReader(None, FakeStorage()) + reader = ResumeDataDirectReader(MyMetaInterp(None), FakeStorage()) assert reader.force_all_virtuals() == ["allocated", reader.virtual_default] # ____________________________________________________________ @@ -925,7 +926,7 @@ liveboxes = modifier.finish({}) assert storage.rd_snapshot is None cpu = MyCPU([]) - reader = ResumeDataDirectReader(cpu, storage) + reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage) _next_section(reader, sys.maxint, 2**16, -65) _next_section(reader, 2, 3) _next_section(reader, sys.maxint, 1, sys.maxint, 2**16) Modified: pypy/trunk/pypy/jit/metainterp/test/test_virtualref.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_virtualref.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_virtualref.py Thu Nov 11 17:32:11 2010 @@ -88,7 +88,11 @@ cpu.get_latest_value_int = lambda i:guard_op.getfailargs()[i].getint() cpu.get_latest_value_ref = lambda i:guard_op.getfailargs()[i].getref_base() cpu.clear_latest_values = lambda count: None - resumereader = ResumeDataDirectReader(cpu, guard_op.getdescr()) + class FakeMetaInterpSd: + callinfocollection = None + FakeMetaInterpSd.cpu = cpu + resumereader = ResumeDataDirectReader(FakeMetaInterpSd(), + guard_op.getdescr()) vrefinfo = self.metainterp.staticdata.virtualref_info lst = [] vrefinfo.continue_tracing = lambda vref, virtual: \ From arigo at codespeak.net Thu Nov 11 17:41:16 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 17:41:16 +0100 (CET) Subject: [pypy-svn] r79005 - pypy/branch/rlist-jit/pypy/module/cpyext/test Message-ID: <20101111164116.7A182282BD6@codespeak.net> Author: arigo Date: Thu Nov 11 17:41:15 2010 New Revision: 79005 Modified: pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py Log: A failing test. Modified: pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py Thu Nov 11 17:41:15 2010 @@ -27,3 +27,13 @@ api.Py_DecRef(ar[0]) lltype.free(ar, flavor='raw') + + def test_tuple_setup(self, space, api): + ref_tup = api.PyTuple_New(2) + ref0 = api.PyInt_FromLong(123) + ref1 = api.PyInt_FromLong(456) + api.PyTuple_SetItem(ref_tup, 0, ref0) + api.PyTuple_SetItem(ref_tup, 1, ref1) + + w_tup = from_ref(space, ref_tup) + assert space.is_true(space.eq(w_tup, space.wrap((123, 456)))) From arigo at codespeak.net Thu Nov 11 17:52:18 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 17:52:18 +0100 (CET) Subject: [pypy-svn] r79006 - in pypy/branch/rlist-jit/pypy/module/cpyext: . test Message-ID: <20101111165218.BC6815080E@codespeak.net> Author: arigo Date: Thu Nov 11 17:52:17 2010 New Revision: 79006 Modified: pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py Log: Fix. Not enough though. Modified: pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py Thu Nov 11 17:52:17 2010 @@ -31,9 +31,10 @@ def test_tuple_setup(self, space, api): ref_tup = api.PyTuple_New(2) ref0 = api.PyInt_FromLong(123) - ref1 = api.PyInt_FromLong(456) api.PyTuple_SetItem(ref_tup, 0, ref0) + ref1 = api.PyInt_FromLong(456) api.PyTuple_SetItem(ref_tup, 1, ref1) w_tup = from_ref(space, ref_tup) assert space.is_true(space.eq(w_tup, space.wrap((123, 456)))) + api.DecRef(ref_tup) Modified: pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py Thu Nov 11 17:52:17 2010 @@ -103,7 +103,7 @@ msg = "PyTuple_SetItem() called on an already-escaped tuple object" raise OperationError(space.w_SystemError, space.wrap(msg)) ref_old = ref_tup.c_items[pos] - ref_tup.c_items[pos] = ref # SetItem steals a reference! + ref_tup.c_items[pos] = ref_item # SetItem steals a reference! Py_DecRef(space, ref_old) return 0 From fijal at codespeak.net Thu Nov 11 17:54:12 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Thu, 11 Nov 2010 17:54:12 +0100 (CET) Subject: [pypy-svn] r79007 - pypy/trunk/pypy/translator/c Message-ID: <20101111165412.AF14F282BDC@codespeak.net> Author: fijal Date: Thu Nov 11 17:54:11 2010 New Revision: 79007 Modified: pypy/trunk/pypy/translator/c/genc.py Log: in case someone wondered, we can't use -O1 with asmgcc Modified: pypy/trunk/pypy/translator/c/genc.py ============================================================================== --- pypy/trunk/pypy/translator/c/genc.py (original) +++ pypy/trunk/pypy/translator/c/genc.py Thu Nov 11 17:54:11 2010 @@ -553,7 +553,7 @@ ('debug', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT" $(TARGET)'), ('debug_exc', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DDO_LOG_EXC" $(TARGET)'), ('debug_mem', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DTRIVIAL_MALLOC_DEBUG" $(TARGET)'), - ('no_obmalloc', '', '$(MAKE) CFLAGS="-g -O1 -DRPY_ASSERT -DNO_OBMALLOC" $(TARGET)'), + ('no_obmalloc', '', '$(MAKE) CFLAGS="-g -O2 -DRPY_ASSERT -DNO_OBMALLOC" $(TARGET)'), ('linuxmemchk', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DLINUXMEMCHK" $(TARGET)'), ('llsafer', '', '$(MAKE) CFLAGS="-O2 -DRPY_LL_ASSERT" $(TARGET)'), ('lldebug', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DRPY_LL_ASSERT" $(TARGET)'), From agaynor at codespeak.net Thu Nov 11 18:03:23 2010 From: agaynor at codespeak.net (agaynor at codespeak.net) Date: Thu, 11 Nov 2010 18:03:23 +0100 (CET) Subject: [pypy-svn] r79008 - pypy/trunk/pypy/translator/goal/test2 Message-ID: <20101111170323.5B3B5282B9E@codespeak.net> Author: agaynor Date: Thu Nov 11 18:03:21 2010 New Revision: 79008 Modified: pypy/trunk/pypy/translator/goal/test2/test_app_main.py Log: Allow running the app_main tests with no $PYTHONPATH. Modified: pypy/trunk/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/trunk/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/trunk/pypy/translator/goal/test2/test_app_main.py Thu Nov 11 18:03:21 2010 @@ -463,8 +463,10 @@ yield finally: old_cwd.chdir() - os.putenv('PYTHONPATH', old_pythonpath) - + # Can't call putenv with a None argument. + if old_pythonpath is not None: + os.putenv('PYTHONPATH', old_pythonpath) + tmpdir.join('site.py').write('print "SHOULD NOT RUN"') runme_py = tmpdir.join('runme.py') runme_py.write('print "some text"') From afa at codespeak.net Thu Nov 11 18:05:48 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 11 Nov 2010 18:05:48 +0100 (CET) Subject: [pypy-svn] r79009 - pypy/branch/rlist-jit/pypy/module/cpyext Message-ID: <20101111170548.B7CF0282BE3@codespeak.net> Author: afa Date: Thu Nov 11 18:05:47 2010 New Revision: 79009 Modified: pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py Log: Fix a NameError Modified: pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py Thu Nov 11 18:05:47 2010 @@ -4,7 +4,7 @@ build_type_checkers, PyObjectFields, cpython_struct, bootstrap_function) from pypy.module.cpyext.pyobject import (PyObject, PyObjectP, Py_DecRef, - borrow_from, make_ref, from_ref, make_typedescr, get_typedescr) + borrow_from, make_ref, from_ref, make_typedescr, get_typedescr, Reference) from pypy.module.cpyext.pyerrors import PyErr_BadInternalCall ## From arigo at codespeak.net Thu Nov 11 18:09:23 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 18:09:23 +0100 (CET) Subject: [pypy-svn] r79010 - in pypy/branch/rlist-jit/pypy/module/cpyext: . test Message-ID: <20101111170923.0B74D282BEA@codespeak.net> Author: arigo Date: Thu Nov 11 18:09:22 2010 New Revision: 79010 Modified: pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py Log: Fix the test, add a comment in PyTuple_SetItem() about why it should not be called with an interpreter object. Modified: pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/test/test_tupleobject.py Thu Nov 11 18:09:22 2010 @@ -30,11 +30,11 @@ def test_tuple_setup(self, space, api): ref_tup = api.PyTuple_New(2) - ref0 = api.PyInt_FromLong(123) + ref0 = make_ref(space, space.wrap(123)) api.PyTuple_SetItem(ref_tup, 0, ref0) - ref1 = api.PyInt_FromLong(456) + ref1 = make_ref(space, space.wrap(456)) api.PyTuple_SetItem(ref_tup, 1, ref1) w_tup = from_ref(space, ref_tup) assert space.is_true(space.eq(w_tup, space.wrap((123, 456)))) - api.DecRef(ref_tup) + api.Py_DecRef(ref_tup) Modified: pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py Thu Nov 11 18:09:22 2010 @@ -4,7 +4,8 @@ build_type_checkers, PyObjectFields, cpython_struct, bootstrap_function) from pypy.module.cpyext.pyobject import (PyObject, PyObjectP, Py_DecRef, - borrow_from, make_ref, from_ref, make_typedescr, get_typedescr, Reference) + borrow_from, make_ref, from_ref, make_typedescr, get_typedescr, Reference, + track_reference) from pypy.module.cpyext.pyerrors import PyErr_BadInternalCall ## @@ -96,6 +97,9 @@ @cpython_api([PyObject, Py_ssize_t, PyObject], rffi.INT_real, error=-1) def PyTuple_SetItem(space, ref, pos, ref_item): + # XXX steals a reference at the level of PyObjects. Don't try to + # XXX call this function with an interpreter object as ref_item! + # XXX do PyTuple_Check, without forcing ref as an interpreter object # XXX -- then if it fails it should also steal a reference, test it!!! ref_tup = rffi.cast(PyTupleObject, ref) From agaynor at codespeak.net Thu Nov 11 18:16:15 2010 From: agaynor at codespeak.net (agaynor at codespeak.net) Date: Thu, 11 Nov 2010 18:16:15 +0100 (CET) Subject: [pypy-svn] r79011 - in pypy/trunk/pypy/translator/goal: . test2 Message-ID: <20101111171615.BB8C7282BEB@codespeak.net> Author: agaynor Date: Thu Nov 11 18:16:14 2010 New Revision: 79011 Modified: pypy/trunk/pypy/translator/goal/app_main.py pypy/trunk/pypy/translator/goal/test2/test_app_main.py Log: Don't put '' on sys.path when executing a script, only when running the REPL, this behavior now matches CPython. Modified: pypy/trunk/pypy/translator/goal/app_main.py ============================================================================== --- pypy/trunk/pypy/translator/goal/app_main.py (original) +++ pypy/trunk/pypy/translator/goal/app_main.py Thu Nov 11 18:16:14 2010 @@ -326,10 +326,6 @@ except: print >> sys.stderr, "'import site' failed" - # update sys.path *after* loading site.py, in case there is a - # "site.py" file in the script's directory. - sys.path.insert(0, '') - if warnoptions: sys.warnoptions.append(warnoptions) from warnings import _processoptions @@ -378,6 +374,13 @@ elif run_stdin: # handle the case where no command/filename/module is specified # on the command-line. + + # update sys.path *after* loading site.py, in case there is a + # "site.py" file in the script's directory. Only run this if we're + # executing the interactive prompt, if we're running a script we + # put it's directory on sys.path + sys.path.insert(0, '') + if go_interactive or sys.stdin.isatty(): # If stdin is a tty or if "-i" is specified, we print # a banner and run $PYTHONSTARTUP. Modified: pypy/trunk/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/trunk/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/trunk/pypy/translator/goal/test2/test_app_main.py Thu Nov 11 18:16:14 2010 @@ -95,6 +95,11 @@ child.expect('>>> ') child.sendline('__name__') child.expect("'__main__'") + child.expect('>>> ') + child.sendline('import sys') + child.expect('>>> ') + child.sendline("'' in sys.path") + child.expect("True") def test_run_script(self): child = self.spawn([demo_script]) @@ -487,9 +492,9 @@ with chdir_and_unset_pythonpath(tmpdir): data = self.run(cmdline2, python_flags='-S') - assert data.startswith("some new text\n") assert repr(str(tmpdir.join('otherpath'))) in data + assert "''" not in data class AppTestAppMain: From cfbolz at codespeak.net Thu Nov 11 18:24:38 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 11 Nov 2010 18:24:38 +0100 (CET) Subject: [pypy-svn] r79012 - pypy/branch/mapdict-without-jit/pypy/objspace/std/test Message-ID: <20101111172438.37E37282BEC@codespeak.net> Author: cfbolz Date: Thu Nov 11 18:24:36 2010 New Revision: 79012 Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py Log: more precise name Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py ============================================================================== --- pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py (original) +++ pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py Thu Nov 11 18:24:36 2010 @@ -778,7 +778,7 @@ res = self.check(f, 'x') assert res == (0, 0, 1) -class AppTestCaching(AppTestWithMapDict): +class AppTestGlobalCaching(AppTestWithMapDict): def setup_class(cls): cls.space = gettestobjspace( **{"objspace.std.withmethodcachecounter": True, From antocuni at codespeak.net Thu Nov 11 18:24:43 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 11 Nov 2010 18:24:43 +0100 (CET) Subject: [pypy-svn] r79013 - in pypy/branch/jitypes2: lib-python/modified-2.5.2/ctypes lib_pypy/_ctypes Message-ID: <20101111172443.71C55282BF1@codespeak.net> Author: antocuni Date: Thu Nov 11 18:24:41 2010 New Revision: 79013 Modified: pypy/branch/jitypes2/lib-python/modified-2.5.2/ctypes/__init__.py pypy/branch/jitypes2/lib_pypy/_ctypes/function.py Log: veeeery IN-PROGRESS: use _ffi instead of _rawffi to call C functions; it's already much faster, but there is still an issue with calling funcptr(*args), as it forces a frame Modified: pypy/branch/jitypes2/lib-python/modified-2.5.2/ctypes/__init__.py ============================================================================== --- pypy/branch/jitypes2/lib-python/modified-2.5.2/ctypes/__init__.py (original) +++ pypy/branch/jitypes2/lib-python/modified-2.5.2/ctypes/__init__.py Thu Nov 11 18:24:41 2010 @@ -345,9 +345,11 @@ _restype_ = c_int # default, can be overridden in instances def __init__(self, name, mode=DEFAULT_MODE, handle=None): + import _ffi self._name = name if handle is None: - self._handle = _dlopen(self._name, mode) + #self._handle = _dlopen(self._name, mode) + self._handle = _ffi.CDLL(name) else: self._handle = handle Modified: pypy/branch/jitypes2/lib_pypy/_ctypes/function.py ============================================================================== --- pypy/branch/jitypes2/lib_pypy/_ctypes/function.py (original) +++ pypy/branch/jitypes2/lib_pypy/_ctypes/function.py Thu Nov 11 18:24:41 2010 @@ -110,7 +110,8 @@ self.dll = ctypes.CDLL(self.dll) # we need to check dll anyway ptr = self._getfuncptr([], ctypes.c_int) - self._buffer = ptr.byptr() + #self._buffer = ptr.byptr() + self._buffer = None elif (sys.platform == 'win32' and len(args) >= 2 and isinstance(args[0], (int, long))): @@ -140,6 +141,7 @@ def __call__(self, *args): if self.callable is not None: + assert False, 'TODO' try: res = self.callable(*args) except: @@ -153,6 +155,7 @@ argtypes = self._argtypes_ if self._com_index: + assert False, 'TODO' from ctypes import cast, c_void_p, POINTER thisarg = cast(args[0], POINTER(POINTER(c_void_p))).contents argtypes = [c_void_p] + list(argtypes) @@ -162,13 +165,29 @@ thisarg = None if argtypes is None: + assert False, 'TODO' argtypes = self._guess_argtypes(args) - argtypes, argsandobjs = self._wrap_args(argtypes, args) + + # XXX + #assert False, 'TODO' + #argtypes, argsandobjs = self._wrap_args(argtypes, args) restype = self._restype_ funcptr = self._getfuncptr(argtypes, restype, thisarg) - resbuffer = funcptr(*[arg._buffer for _, arg in argsandobjs]) - return self._build_result(restype, resbuffer, argtypes, argsandobjs) + return funcptr(*args) + #return funcptr(args[0]) + #resbuffer = funcptr(*[arg._buffer for _, arg in argsandobjs]) + #return self._build_result(restype, resbuffer, argtypes, argsandobjs) + + def _shape_to_ffi_type(self, shape): + from _ffi import types + if shape == 'l': + return types.slong + elif shape == 'd': + return types.double + else: + print 'unknown shape %s' % shape + assert False, 'TODO' def _getfuncptr(self, argtypes, restype, thisarg=None): if self._ptr is not None and argtypes is self._argtypes_: @@ -194,7 +213,11 @@ cdll = self.dll._handle try: - return cdll.ptr(self.name, argshapes, resshape, self._flags_) + #return cdll.ptr(self.name, argshapes, resshape, self._flags_) + ffi_argtypes = [self._shape_to_ffi_type(shape) for shape in argshapes] + ffi_restype = self._shape_to_ffi_type(resshape) + self._ptr = cdll.getfunc(self.name, ffi_argtypes, ffi_restype) + return self._ptr except AttributeError: if self._flags_ & _rawffi.FUNCFLAG_CDECL: raise From arigo at codespeak.net Thu Nov 11 18:26:39 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 18:26:39 +0100 (CET) Subject: [pypy-svn] r79014 - pypy/branch/rlist-jit/pypy/module/cpyext Message-ID: <20101111172639.B6351282BEF@codespeak.net> Author: arigo Date: Thu Nov 11 18:26:38 2010 New Revision: 79014 Modified: pypy/branch/rlist-jit/pypy/module/cpyext/slotdefs.py pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py Log: Fix. Modified: pypy/branch/rlist-jit/pypy/module/cpyext/slotdefs.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/slotdefs.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/slotdefs.py Thu Nov 11 18:26:38 2010 @@ -28,14 +28,14 @@ def check_num_args(space, ob, n): from pypy.module.cpyext.tupleobject import PyTuple_CheckExact, \ - PyTuple_GET_SIZE + _PyTuple_Size_Fast if not PyTuple_CheckExact(space, ob): raise OperationError(space.w_SystemError, space.wrap("PyArg_UnpackTuple() argument list is not a tuple")) - if n == PyTuple_GET_SIZE(space, ob): + if n == _PyTuple_Size_Fast(space, ob): return raise operationerrfmt(space.w_TypeError, - "expected %d arguments, got %d", n, PyTuple_GET_SIZE(space, ob)) + "expected %d arguments, got %d", n, _PyTuple_Size_Fast(space, ob)) def wrap_init(space, w_self, w_args, func, w_kwargs): func_init = rffi.cast(initproc, func) Modified: pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/tupleobject.py Thu Nov 11 18:26:38 2010 @@ -123,6 +123,13 @@ return borrow_from(w_t, w_obj) @cpython_api([PyObject], Py_ssize_t, error=-1) +def _PyTuple_Size_Fast(space, ref): + # custom version: it's not a macro, so it can be called from other .py + # files; but it doesn't include PyTuple_Check() + ref_tup = rffi.cast(PyTupleObject, ref) + return ref_tup.c_size + + at cpython_api([PyObject], Py_ssize_t, error=-1) def PyTuple_Size(space, ref): """Take a pointer to a tuple object, and return the size of that tuple.""" # XXX do PyTuple_Check, without forcing ref as an interpreter object From afa at codespeak.net Thu Nov 11 18:31:16 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 11 Nov 2010 18:31:16 +0100 (CET) Subject: [pypy-svn] r79015 - pypy/trunk/pypy/translator/cli/src Message-ID: <20101111173116.35878282BE3@codespeak.net> Author: afa Date: Thu Nov 11 18:31:14 2010 New Revision: 79015 Modified: pypy/trunk/pypy/translator/cli/src/pypylib.cs Log: Fix str->double conversion in cli backend on Windows, which apparently depends on the locale... Modified: pypy/trunk/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/trunk/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/trunk/pypy/translator/cli/src/pypylib.cs Thu Nov 11 18:31:14 2010 @@ -83,8 +83,12 @@ return Double.NegativeInfinity; else if (s == "nan") return Double.NaN; - else - return System.Convert.ToDouble(s); + else { + System.Globalization.NumberFormatInfo formatter; + formatter = new System.Globalization.NumberFormatInfo(); + formatter.NumberDecimalSeparator = "."; + return System.Convert.ToDouble(s, formatter); + } } } From arigo at codespeak.net Thu Nov 11 18:38:46 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 18:38:46 +0100 (CET) Subject: [pypy-svn] r79016 - pypy/branch/rlist-jit/pypy/module/cpyext/test Message-ID: <20101111173846.9C119282BE3@codespeak.net> Author: arigo Date: Thu Nov 11 18:38:44 2010 New Revision: 79016 Modified: pypy/branch/rlist-jit/pypy/module/cpyext/test/test_cpyext.py Log: Fix test. Modified: pypy/branch/rlist-jit/pypy/module/cpyext/test/test_cpyext.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/test/test_cpyext.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/test/test_cpyext.py Thu Nov 11 18:38:44 2010 @@ -534,16 +534,17 @@ PyObject *true = Py_True; PyObject *tup = NULL; int refcnt = true->ob_refcnt; - int refcnt_after; + int refcnt_middle, refcnt_after; tup = PyTuple_New(1); Py_INCREF(true); if (PyTuple_SetItem(tup, 0, true) < 0) return NULL; - refcnt_after = true->ob_refcnt; + refcnt_middle = true->ob_refcnt; Py_DECREF(tup); - fprintf(stderr, "REFCNT2 %i %i\\n", refcnt, refcnt_after); - return PyBool_FromLong(refcnt_after == refcnt); + refcnt_after = true->ob_refcnt; + fprintf(stderr, "REFCNT2 %i %i %i\\n", refcnt, refcnt_middle, refcnt_after); + return PyBool_FromLong(refcnt_after == refcnt && refcnt_middle == refcnt+1); } static PyMethodDef methods[] = { From arigo at codespeak.net Thu Nov 11 19:28:48 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 11 Nov 2010 19:28:48 +0100 (CET) Subject: [pypy-svn] r79017 - in pypy/branch/rlist-jit/pypy: interpreter module/__builtin__ module/cpyext module/exceptions objspace/std Message-ID: <20101111182848.AFF12282BDC@codespeak.net> Author: arigo Date: Thu Nov 11 19:28:46 2010 New Revision: 79017 Modified: pypy/branch/rlist-jit/pypy/interpreter/baseobjspace.py pypy/branch/rlist-jit/pypy/interpreter/function.py pypy/branch/rlist-jit/pypy/module/__builtin__/interp_classobj.py pypy/branch/rlist-jit/pypy/module/cpyext/typeobject.py pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py pypy/branch/rlist-jit/pypy/objspace/std/objspace.py pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py pypy/branch/rlist-jit/pypy/objspace/std/typetype.py Log: Replace the expressions 'space.fixedview(x)[:]' with yet another method, 'space.fixedunpack(x)'. Modified: pypy/branch/rlist-jit/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/rlist-jit/pypy/interpreter/baseobjspace.py Thu Nov 11 19:28:46 2010 @@ -8,7 +8,7 @@ from pypy.tool.cache import Cache from pypy.tool.uid import HUGEVAL_BYTES from pypy.rlib.objectmodel import we_are_translated -from pypy.rlib.debug import make_sure_not_resized +from pypy.rlib.debug import make_sure_not_resized, list_not_modified_any_more from pypy.rlib.timer import DummyTimer, Timer from pypy.rlib.rarithmetic import r_uint from pypy.rlib import jit @@ -764,13 +764,21 @@ 'unpackiterable_unroll')) def fixedview(self, w_iterable, expected_length=-1): - """ A fixed list view of w_iterable. Don't modify the result + """ A fixed list view of w_iterable. The result is supposed to be + used temporarily: it is a list with the annotation flag 'do not mutate'. """ - return make_sure_not_resized(self.unpackiterable(w_iterable, + return list_not_modified_any_more(self.unpackiterable(w_iterable, expected_length)[:]) fixedview_unroll = fixedview + def fixedunpack(self, w_iterable, expected_length=-1): + """ A fixed list with the content of w_iterable. + For most cases, use fixedview() instead. + """ + return make_sure_not_resized(self.unpackiterable(w_iterable, + expected_length)[:]) + def listview(self, w_iterable, expected_length=-1): """ A non-fixed view of w_iterable. Don't modify the result """ Modified: pypy/branch/rlist-jit/pypy/interpreter/function.py ============================================================================== --- pypy/branch/rlist-jit/pypy/interpreter/function.py (original) +++ pypy/branch/rlist-jit/pypy/interpreter/function.py Thu Nov 11 19:28:46 2010 @@ -198,7 +198,7 @@ else: name = None if not space.is_w(w_argdefs, space.w_None): - defs_w = space.fixedview(w_argdefs)[:] + defs_w = space.fixedunpack(w_argdefs) else: defs_w = [] nfreevars = 0 @@ -323,7 +323,7 @@ if space.is_w(w_func_dict, space.w_None): w_func_dict = None self.w_func_dict = w_func_dict - self.defs_w = space.fixedview(w_defs_w)[:] + self.defs_w = space.fixedunpack(w_defs_w) self.w_module = w_module def fget_func_defaults(space, self): @@ -338,7 +338,7 @@ return if not space.is_true(space.isinstance(w_defaults, space.w_tuple)): raise OperationError( space.w_TypeError, space.wrap("func_defaults must be set to a tuple object or None") ) - self.defs_w = space.fixedview(w_defaults)[:] + self.defs_w = space.fixedunpack(w_defaults) def fdel_func_defaults(space, self): self.defs_w = [] Modified: pypy/branch/rlist-jit/pypy/module/__builtin__/interp_classobj.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/__builtin__/interp_classobj.py (original) +++ pypy/branch/rlist-jit/pypy/module/__builtin__/interp_classobj.py Thu Nov 11 19:28:46 2010 @@ -40,7 +40,7 @@ # XXX missing: lengthy and obscure logic about "__module__" - bases_w = space.fixedview(w_bases)[:] + bases_w = space.fixedunpack(w_bases) for w_base in bases_w: if not isinstance(w_base, W_ClassObject): w_metaclass = space.type(w_base) @@ -92,7 +92,7 @@ raise OperationError( space.w_TypeError, space.wrap("__bases__ must be a tuple object")) - bases_w = space.fixedview(w_bases)[:] + bases_w = space.fixedunpack(w_bases) for w_base in bases_w: if not isinstance(w_base, W_ClassObject): raise OperationError(space.w_TypeError, Modified: pypy/branch/rlist-jit/pypy/module/cpyext/typeobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/typeobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/typeobject.py Thu Nov 11 19:28:46 2010 @@ -262,7 +262,7 @@ class W_PyCTypeObject(W_TypeObject): def __init__(self, space, pto): - bases_w = space.fixedview(from_ref(space, pto.c_tp_bases)) + bases_w = space.fixedunpack(from_ref(space, pto.c_tp_bases)) dict_w = {} add_operators(space, dict_w, pto) Modified: pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py (original) +++ pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py Thu Nov 11 19:28:46 2010 @@ -77,6 +77,7 @@ from pypy.interpreter.gateway import interp2app, Arguments from pypy.interpreter.error import OperationError from pypy.rlib import rwin32 +from pypy.rlib.debug import list_not_modified_any_more def readwrite_attrproperty_w(name, cls): def fget(space, obj): @@ -92,11 +93,15 @@ and will be deprecated at some point. """ w_dict = None - args_w = [] + _empty_args_w = [] def __init__(self, space): self.space = space self.w_message = space.w_None + self.args_w = list_not_modified_any_more(W_BaseException._empty_args_w) + # Note that 'self.args_w' is annotated as a list-that-is-not-modified, + # which cannot easily be mixed together with a general list annotation. + # That's why we use 'list_not_modified_any_more()'. def descr_init(self, space, args_w): self.args_w = args_w @@ -327,7 +332,7 @@ self.w_strerror = args_w[1] if len(args_w) == 3: self.w_filename = args_w[2] - self.args_w = [args_w[0], args_w[1]] + self.args_w = list_not_modified_any_more([args_w[0], args_w[1]]) descr_init.unwrap_spec = ['self', ObjSpace, 'args_w'] # since we rebind args_w, we need special reduce, grump Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/objspace.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Thu Nov 11 19:28:46 2010 @@ -362,14 +362,15 @@ if isinstance(w_obj, W_TupleObject): t = w_obj.wrappeditems elif isinstance(w_obj, W_ListObject): - t = w_obj.wrappeditems[:] + t = list_not_modified_any_more(w_obj.wrappeditems[:]) else: if unroll: - return make_sure_not_resized(ObjSpace.unpackiterable_unroll( - self, w_obj, expected_length)[:]) + r = ObjSpace.unpackiterable_unroll( + self, w_obj, expected_length) else: - return make_sure_not_resized(ObjSpace.unpackiterable( - self, w_obj, expected_length)[:]) + r = ObjSpace.unpackiterable( + self, w_obj, expected_length) + return list_not_modified_any_more(r[:]) if expected_length != -1 and len(t) != expected_length: raise self._wrap_expected_length(expected_length, len(t)) return t @@ -377,6 +378,17 @@ def fixedview_unroll(self, w_obj, expected_length=-1): return self.fixedview(w_obj, expected_length, unroll=True) + def fixedunpack(self, w_obj, expected_length=-1): + if isinstance(w_obj, W_TupleObject): + t = w_obj.wrappeditems[:] + elif isinstance(w_obj, W_ListObject): + t = w_obj.wrappeditems[:] + else: + return ObjSpace.fixedunpack(self, w_obj, expected_length) + if expected_length != -1 and len(t) != expected_length: + raise self._wrap_expected_length(expected_length, len(t)) + return t + def listview(self, w_obj, expected_length=-1): if isinstance(w_obj, W_ListObject): t = w_obj.wrappeditems Modified: pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py Thu Nov 11 19:28:46 2010 @@ -652,7 +652,7 @@ if w_mro_func is not None and not space.is_w(w_where, space.w_type): w_mro_meth = space.get(w_mro_func, w_self) w_mro = space.call_function(w_mro_meth) - mro_w = space.fixedview(w_mro)[:] + mro_w = space.fixedunpack(w_mro) w_self.mro_w = validate_custom_mro(space, mro_w) return # done w_self.mro_w = w_self.compute_default_mro()[:] Modified: pypy/branch/rlist-jit/pypy/objspace/std/typetype.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/typetype.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/typetype.py Thu Nov 11 19:28:46 2010 @@ -12,7 +12,7 @@ w_typetype = _precheck_for_new(space, w_typetype) - bases_w = space.fixedview(w_bases)[:] + bases_w = space.fixedunpack(w_bases) w_winner = w_typetype for base in bases_w: @@ -115,7 +115,7 @@ "can only assign tuple to %s.__bases__, not %s", w_type.name, space.type(w_value).getname(space, '?')) - newbases_w = space.fixedview(w_value)[:] + newbases_w = space.fixedunpack(w_value) if len(newbases_w) == 0: raise operationerrfmt(space.w_TypeError, "can only assign non-empty tuple to %s.__bases__, not ()", From cfbolz at codespeak.net Thu Nov 11 23:52:10 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 11 Nov 2010 23:52:10 +0100 (CET) Subject: [pypy-svn] r79018 - in pypy/branch/mapdict-without-jit/pypy/objspace/std: . test Message-ID: <20101111225210.0AAB4282B90@codespeak.net> Author: cfbolz Date: Thu Nov 11 23:52:08 2010 New Revision: 79018 Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/callmethod.py pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py Log: Also use the local mapdict cache in the CALLMETHOD bytecode. I think it all can be generalized a bit more, but I first want to see if this helps. Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/callmethod.py ============================================================================== --- pypy/branch/mapdict-without-jit/pypy/objspace/std/callmethod.py (original) +++ pypy/branch/mapdict-without-jit/pypy/objspace/std/callmethod.py Thu Nov 11 23:52:08 2010 @@ -13,6 +13,9 @@ from pypy.interpreter import function from pypy.objspace.descroperation import object_getattribute from pypy.rlib import jit, rstack # for resume points +from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict, \ + LOOKUP_METHOD_mapdict_fill_cache_method + # This module exports two extra methods for StdObjSpaceFrame implementing # the LOOKUP_METHOD and CALL_METHOD opcodes in an efficient way, as well @@ -30,6 +33,13 @@ # space = f.space w_obj = f.popvalue() + + if space.config.objspace.std.withmapdict: + # mapdict has an extra-fast version of this function + from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict + if LOOKUP_METHOD_mapdict(f, nameindex, w_obj): + return + w_name = f.getname_w(nameindex) w_value = None @@ -50,6 +60,10 @@ # nothing in the instance f.pushvalue(w_descr) f.pushvalue(w_obj) + if space.config.objspace.std.withmapdict: + # let mapdict cache stuff + LOOKUP_METHOD_mapdict_fill_cache_method( + f.getcode(), nameindex, w_obj, w_type, w_descr) return if w_value is None: w_value = space.getattr(w_obj, w_name) Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py ============================================================================== --- pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py (original) +++ pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py Thu Nov 11 23:52:08 2010 @@ -659,30 +659,54 @@ map = None version_tag = None index = 0 + w_method = None # for callmethod success_counter = 0 failure_counter = 0 + def is_valid_for_obj(self, w_obj): + map = w_obj._get_mapdict_map() + return self.is_valid_for_map(map) + + def is_valid_for_map(self, map): + if map is self.map: + version_tag = map.terminator.w_cls.version_tag() + if version_tag is self.version_tag: + # everything matches, it's incredibly fast + if map.space.config.objspace.std.withmethodcachecounter: + self.success_counter += 1 + return True + return False + INVALID_CACHE_ENTRY = CacheEntry() INVALID_CACHE_ENTRY.map = objectmodel.instantiate(AbstractAttribute) # different from any real map ^^^ INVALID_CACHE_ENTRY.map.terminator = None + def init_mapdict_cache(pycode): num_entries = len(pycode.co_names_w) pycode._mapdict_caches = [INVALID_CACHE_ENTRY] * num_entries +def _fill_cache(pycode, nameindex, map, version_tag, index, w_method=None): + entry = pycode._mapdict_caches[nameindex] + if entry is INVALID_CACHE_ENTRY: + entry = CacheEntry() + pycode._mapdict_caches[nameindex] = entry + entry.map = map + entry.version_tag = version_tag + entry.index = index + entry.w_method = w_method + if pycode.space.config.objspace.std.withmethodcachecounter: + entry.failure_counter += 1 + def LOAD_ATTR_caching(pycode, w_obj, nameindex): # this whole mess is to make the interpreter quite a bit faster; it's not # used if we_are_jitted(). entry = pycode._mapdict_caches[nameindex] map = w_obj._get_mapdict_map() - if map is entry.map: - version_tag = map.terminator.w_cls.version_tag() - if version_tag is entry.version_tag: - # everything matches, it's incredibly fast - if pycode.space.config.objspace.std.withmethodcachecounter: - entry.success_counter += 1 - return w_obj._mapdict_read_storage(entry.index) + if entry.is_valid_for_map(map): + # everything matches, it's incredibly fast + return w_obj._mapdict_read_storage(entry.index) return LOAD_ATTR_slowpath(pycode, w_obj, nameindex, map) LOAD_ATTR_caching._always_inline_ = True @@ -710,17 +734,30 @@ if selector[1] != INVALID: index = map.index(selector) if index >= 0: - entry = pycode._mapdict_caches[nameindex] - if entry is INVALID_CACHE_ENTRY: - entry = CacheEntry() - pycode._mapdict_caches[nameindex] = entry - entry.map = map - entry.version_tag = version_tag - entry.index = index - if space.config.objspace.std.withmethodcachecounter: - entry.failure_counter += 1 + _fill_cache(pycode, nameindex, map, version_tag, index) return w_obj._mapdict_read_storage(index) if space.config.objspace.std.withmethodcachecounter: INVALID_CACHE_ENTRY.failure_counter += 1 return space.getattr(w_obj, w_name) LOAD_ATTR_slowpath._dont_inline_ = True + +def LOOKUP_METHOD_mapdict(f, nameindex, w_obj): + space = f.space + pycode = f.getcode() + entry = pycode._mapdict_caches[nameindex] + if entry.is_valid_for_obj(w_obj): + w_method = entry.w_method + if w_method is not None: + f.pushvalue(w_method) + f.pushvalue(w_obj) + return True + return False + +def LOOKUP_METHOD_mapdict_fill_cache_method(pycode, nameindex, w_obj, w_type, w_method): + version_tag = w_type.version_tag() + if version_tag is None: + return + map = w_obj._get_mapdict_map() + if map is None: + return + _fill_cache(pycode, nameindex, map, version_tag, -1, w_method) Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py ============================================================================== --- pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py (original) +++ pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py Thu Nov 11 23:52:08 2010 @@ -597,7 +597,8 @@ from pypy.interpreter import gateway cls.space = gettestobjspace( **{"objspace.std.withmapdict": True, - "objspace.std.withmethodcachecounter": True}) + "objspace.std.withmethodcachecounter": True, + "objspace.opcodes.CALL_METHOD": True}) # def check(space, w_func, name): w_code = space.getattr(w_func, space.wrap('func_code')) @@ -778,6 +779,58 @@ res = self.check(f, 'x') assert res == (0, 0, 1) + def test_call_method_uses_cache(self): + # bit sucky + global C + + class C(object): + def m(*args): + return args + C.sm = staticmethod(C.m.im_func) + C.cm = classmethod(C.m.im_func) + + exec """if 1: + + def f(): + c = C() + res = c.m(1) + assert res == (c, 1) + return 42 + + def g(): + c = C() + res = c.sm(1) + assert res == (1, ) + return 42 + + def h(): + c = C() + res = c.cm(1) + assert res == (C, 1) + return 42 + """ + res = self.check(f, 'm') + assert res == (1, 0, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + + # static methods are not cached + res = self.check(g, 'sm') + assert res == (0, 0, 0) + res = self.check(g, 'sm') + assert res == (0, 0, 0) + + # neither are class methods + res = self.check(h, 'cm') + assert res == (0, 0, 0) + res = self.check(h, 'cm') + assert res == (0, 0, 0) + + class AppTestGlobalCaching(AppTestWithMapDict): def setup_class(cls): cls.space = gettestobjspace( From afa at codespeak.net Thu Nov 11 23:53:21 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 11 Nov 2010 23:53:21 +0100 (CET) Subject: [pypy-svn] r79019 - in pypy/branch/fast-forward/pypy/module/pyexpat: . test Message-ID: <20101111225321.861BA5080E@codespeak.net> Author: afa Date: Thu Nov 11 23:53:20 2010 New Revision: 79019 Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py Log: pyexpat: correctly parse strings with non-utf8 encoding. Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py (original) +++ pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py Thu Nov 11 23:53:20 2010 @@ -278,10 +278,7 @@ def __init__(self, encoding, namespace_separator, w_intern, _from_external_entity=False): - if encoding: - self.encoding = encoding - else: - self.encoding = 'utf-8' + self.encoding = encoding self.namespace_separator = namespace_separator self.w_intern = w_intern @@ -339,10 +336,9 @@ def w_convert(self, space, s): if self.returns_unicode: - return space.call_function( - space.getattr(space.wrap(s), space.wrap("decode")), - space.wrap(self.encoding), - space.wrap("strict")) + from pypy.rlib.runicode import str_decode_utf_8 + return space.wrap(str_decode_utf_8( + s, len(s), "strict")[0]) else: return space.wrap(s) Modified: pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py (original) +++ pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py Thu Nov 11 23:53:20 2010 @@ -31,3 +31,25 @@ p.buffer_size = 150 assert p.buffer_size == 150 raises(TypeError, setattr, p, 'buffer_size', sys.maxint + 1) + + def test_encoding(self): + # use one of the few encodings built-in in expat + xml = "caf\xe9" + import pyexpat + p = pyexpat.ParserCreate() + def gotText(text): + assert text == u"caf\xe9" + p.CharacterDataHandler = gotText + assert p.returns_unicode + p.Parse(xml) + + def test_explicit_encoding(self): + xml = "caf\xe9" + import pyexpat + p = pyexpat.ParserCreate(encoding='iso-8859-1') + def gotText(text): + assert text == u"caf\xe9" + p.CharacterDataHandler = gotText + assert p.returns_unicode + p.Parse(xml) + From afa at codespeak.net Fri Nov 12 00:15:20 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 12 Nov 2010 00:15:20 +0100 (CET) Subject: [pypy-svn] r79020 - pypy/branch/fast-forward/pypy/module/pyexpat Message-ID: <20101111231520.9D44A282B90@codespeak.net> Author: afa Date: Fri Nov 12 00:15:18 2010 New Revision: 79020 Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py Log: Refactor a bit, reduce stored attributes Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py (original) +++ pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py Fri Nov 12 00:15:18 2010 @@ -276,10 +276,8 @@ class W_XMLParserType(Wrappable): - def __init__(self, encoding, namespace_separator, w_intern, - _from_external_entity=False): - self.encoding = encoding - self.namespace_separator = namespace_separator + def __init__(self, space, parser, w_intern): + self.itself = parser self.w_intern = w_intern @@ -287,14 +285,6 @@ self.ordered_attributes = False self.specified_attributes = False - if not _from_external_entity: - if namespace_separator: - self.itself = XML_ParserCreateNS( - self.encoding, - rffi.cast(rffi.CHAR, namespace_separator)) - else: - self.itself = XML_ParserCreate(self.encoding) - self.handlers = [None] * NB_HANDLERS self.buffer_w = None @@ -304,6 +294,12 @@ self._exc_info = None + # Set user data for callback function + global_storage.get_nonmoving_id( + CallbackData(space, self), + id=rffi.cast(lltype.Signed, self.itself)) + XML_SetUserData(self.itself, rffi.cast(rffi.VOIDP, self.itself)) + def __del__(self): if XML_ParserFree: # careful with CPython interpreter shutdown XML_ParserFree(self.itself) @@ -494,14 +490,9 @@ else: encoding = space.str_w(w_encoding) - parser = W_XMLParserType(encoding, 0, self.w_intern, - _from_external_entity=True) - parser.itself = XML_ExternalEntityParserCreate(self.itself, - context, encoding) - global_storage.get_nonmoving_id( - CallbackData(space, parser), - id=rffi.cast(lltype.Signed, parser.itself)) - XML_SetUserData(parser.itself, rffi.cast(rffi.VOIDP, parser.itself)) + xmlparser = XML_ExternalEntityParserCreate( + self.itself, context, encoding) + parser = W_XMLParserType(space, xmlparser, self.w_intern) # copy handlers from self for i in range(NB_HANDLERS): @@ -659,16 +650,18 @@ elif space.is_w(w_intern, space.w_None): w_intern = None - parser = W_XMLParserType(encoding, namespace_separator, w_intern) + if namespace_separator: + xmlparser = XML_ParserCreateNS( + encoding, + rffi.cast(rffi.CHAR, namespace_separator)) + else: + xmlparser = XML_ParserCreate(encoding) + + parser = W_XMLParserType(space, xmlparser, w_intern) if not parser.itself: raise OperationError(space.w_RuntimeError, space.wrap('XML_ParserCreate failed')) - global_storage.get_nonmoving_id( - CallbackData(space, parser), - id=rffi.cast(lltype.Signed, parser.itself)) - XML_SetUserData(parser.itself, rffi.cast(rffi.VOIDP, parser.itself)) - return space.wrap(parser) ParserCreate.unwrap_spec = [ObjSpace, W_Root, W_Root, W_Root] From afa at codespeak.net Fri Nov 12 00:18:29 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 12 Nov 2010 00:18:29 +0100 (CET) Subject: [pypy-svn] r79021 - pypy/branch/fast-forward/pypy/module/pyexpat Message-ID: <20101111231829.02E7F36C220@codespeak.net> Author: afa Date: Fri Nov 12 00:18:28 2010 New Revision: 79021 Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py Log: Handle errors a bit sooner Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py (original) +++ pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py Fri Nov 12 00:18:28 2010 @@ -492,6 +492,9 @@ xmlparser = XML_ExternalEntityParserCreate( self.itself, context, encoding) + if not xmlparser: + raise MemoryError + parser = W_XMLParserType(space, xmlparser, self.w_intern) # copy handlers from self @@ -657,11 +660,11 @@ else: xmlparser = XML_ParserCreate(encoding) - parser = W_XMLParserType(space, xmlparser, w_intern) - if not parser.itself: + if not xmlparser: raise OperationError(space.w_RuntimeError, space.wrap('XML_ParserCreate failed')) + parser = W_XMLParserType(space, xmlparser, w_intern) return space.wrap(parser) ParserCreate.unwrap_spec = [ObjSpace, W_Root, W_Root, W_Root] From cfbolz at codespeak.net Fri Nov 12 00:20:14 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 12 Nov 2010 00:20:14 +0100 (CET) Subject: [pypy-svn] r79022 - pypy/branch/mapdict-without-jit/pypy/objspace/std Message-ID: <20101111232014.7604F36C224@codespeak.net> Author: cfbolz Date: Fri Nov 12 00:20:13 2010 New Revision: 79022 Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/callmethod.py Log: the JIT should not see this stuff Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/callmethod.py ============================================================================== --- pypy/branch/mapdict-without-jit/pypy/objspace/std/callmethod.py (original) +++ pypy/branch/mapdict-without-jit/pypy/objspace/std/callmethod.py Fri Nov 12 00:20:13 2010 @@ -34,7 +34,7 @@ space = f.space w_obj = f.popvalue() - if space.config.objspace.std.withmapdict: + if space.config.objspace.std.withmapdict and not jit.we_are_jitted(): # mapdict has an extra-fast version of this function from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict if LOOKUP_METHOD_mapdict(f, nameindex, w_obj): @@ -60,7 +60,8 @@ # nothing in the instance f.pushvalue(w_descr) f.pushvalue(w_obj) - if space.config.objspace.std.withmapdict: + if (space.config.objspace.std.withmapdict and + not jit.we_are_jitted(): # let mapdict cache stuff LOOKUP_METHOD_mapdict_fill_cache_method( f.getcode(), nameindex, w_obj, w_type, w_descr) From cfbolz at codespeak.net Fri Nov 12 00:21:48 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 12 Nov 2010 00:21:48 +0100 (CET) Subject: [pypy-svn] r79023 - pypy/branch/mapdict-without-jit/pypy/objspace/std Message-ID: <20101111232148.285CE282B90@codespeak.net> Author: cfbolz Date: Fri Nov 12 00:21:46 2010 New Revision: 79023 Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/callmethod.py Log: grumble Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/callmethod.py ============================================================================== --- pypy/branch/mapdict-without-jit/pypy/objspace/std/callmethod.py (original) +++ pypy/branch/mapdict-without-jit/pypy/objspace/std/callmethod.py Fri Nov 12 00:21:46 2010 @@ -61,7 +61,7 @@ f.pushvalue(w_descr) f.pushvalue(w_obj) if (space.config.objspace.std.withmapdict and - not jit.we_are_jitted(): + not jit.we_are_jitted()): # let mapdict cache stuff LOOKUP_METHOD_mapdict_fill_cache_method( f.getcode(), nameindex, w_obj, w_type, w_descr) From arigo at codespeak.net Fri Nov 12 10:48:47 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 12 Nov 2010 10:48:47 +0100 (CET) Subject: [pypy-svn] r79024 - pypy/trunk/pypy/rpython/memory/gc Message-ID: <20101112094847.13D85282B9C@codespeak.net> Author: arigo Date: Fri Nov 12 10:48:44 2010 New Revision: 79024 Modified: pypy/trunk/pypy/rpython/memory/gc/inspector.py Log: Revert all checkins that were attempts to fix an issue with inspector.py not seeing all roots. Will be solved more generally. (r78981, r78943, r78939) Modified: pypy/trunk/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/inspector.py Fri Nov 12 10:48:44 2010 @@ -101,8 +101,7 @@ # ---------- raw_os_write = rffi.llexternal(underscore_on_windows+'write', - [rffi.INT, rffi.CArrayPtr(lltype.Signed), - rffi.SIZE_T], + [rffi.INT, llmemory.Address, rffi.SIZE_T], rffi.SIZE_T, sandboxsafe=True, _nowrapper=True) @@ -131,7 +130,7 @@ if self.buf_count > 0: bytes = self.buf_count * rffi.sizeof(rffi.LONG) count = raw_os_write(self.fd, - self.writebuffer, + rffi.cast(llmemory.Address, self.writebuffer), rffi.cast(rffi.SIZE_T, bytes)) if rffi.cast(lltype.Signed, count) != bytes: raise OSError(rposix.get_errno(), "raw_os_write failed") @@ -140,7 +139,7 @@ def write(self, value): x = self.buf_count - self.writebuffer[x] = llmemory.raw_malloc_usage(value) + self.writebuffer[x] = value x += 1 self.buf_count = x if x == self.BUFSIZE: @@ -174,9 +173,6 @@ def add_roots(self): self.gc._heap_dumper = self - if not self.gc.prebuilt_gc_objects_are_static_roots: - self.gc.prebuilt_root_objects.foreach(self.gc._collect_obj, - self.pending) self.gc.root_walker.walk_roots( _hd_add_root, _hd_add_root, From arigo at codespeak.net Fri Nov 12 11:13:03 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 12 Nov 2010 11:13:03 +0100 (CET) Subject: [pypy-svn] r79025 - pypy/trunk/pypy/doc Message-ID: <20101112101303.5E7BB5080B@codespeak.net> Author: arigo Date: Fri Nov 12 11:13:02 2010 New Revision: 79025 Modified: pypy/trunk/pypy/doc/release-1.4.0.txt Log: Fix test failure by missing link. Modified: pypy/trunk/pypy/doc/release-1.4.0.txt ============================================================================== --- pypy/trunk/pypy/doc/release-1.4.0.txt (original) +++ pypy/trunk/pypy/doc/release-1.4.0.txt Fri Nov 12 11:13:02 2010 @@ -21,3 +21,6 @@ to use it, you need a recent version of virtualenv (>= 1.5). XXX write me: better regular expressions + + +.. _virtualenv: http://pypi.python.org/pypi/virtualenv From david at codespeak.net Fri Nov 12 11:38:53 2010 From: david at codespeak.net (david at codespeak.net) Date: Fri, 12 Nov 2010 11:38:53 +0100 (CET) Subject: [pypy-svn] r79026 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101112103853.C10BF282B9C@codespeak.net> Author: david Date: Fri Nov 12 11:38:52 2010 New Revision: 79026 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Fix passing a forgotten argument used to adjust the sp Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Fri Nov 12 11:38:52 2010 @@ -237,7 +237,7 @@ if op.numargs() > 5: stack_args = op.numargs() - 5 n = stack_args*WORD - self._adjust_sp(n, fcond=fcond) + self._adjust_sp(n, regalloc, fcond=fcond) for i in range(5, op.numargs()): reg = regalloc.make_sure_var_in_reg(op.getarg(i)) self.mc.STR_ri(reg.value, r.sp.value, (i-5)*WORD) From antocuni at codespeak.net Fri Nov 12 11:48:48 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 12 Nov 2010 11:48:48 +0100 (CET) Subject: [pypy-svn] r79027 - pypy/extradoc/planning Message-ID: <20101112104848.7D95E5080B@codespeak.net> Author: antocuni Date: Fri Nov 12 11:48:47 2010 New Revision: 79027 Modified: pypy/extradoc/planning/jit.txt Log: add a task Modified: pypy/extradoc/planning/jit.txt ============================================================================== --- pypy/extradoc/planning/jit.txt (original) +++ pypy/extradoc/planning/jit.txt Fri Nov 12 11:48:47 2010 @@ -91,6 +91,9 @@ - 64-bit issues: in test_pypy_c, test_array_intimg, the STORE_SUBSCR turns into strange code (truncate the int to 32 bits). +- foo(*args) generates sub-optimal code even if foo takes a fixed number of + arguments, because we always allocate the Arguments class in that case + - xxx (find more examples :-) BACKEND TASKS From afa at codespeak.net Fri Nov 12 11:57:03 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 12 Nov 2010 11:57:03 +0100 (CET) Subject: [pypy-svn] r79028 - in pypy/branch/fast-forward/pypy: interpreter module/_rawffi module/cpyext module/cpyext/test module/sys rlib Message-ID: <20101112105703.9FBF5282B9D@codespeak.net> Author: afa Date: Fri Nov 12 11:57:01 2010 New Revision: 79028 Modified: pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py pypy/branch/fast-forward/pypy/module/cpyext/__init__.py pypy/branch/fast-forward/pypy/module/cpyext/api.py pypy/branch/fast-forward/pypy/module/cpyext/state.py pypy/branch/fast-forward/pypy/module/cpyext/test/conftest.py pypy/branch/fast-forward/pypy/module/cpyext/test/test_api.py pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py pypy/branch/fast-forward/pypy/module/sys/__init__.py pypy/branch/fast-forward/pypy/module/sys/vm.py pypy/branch/fast-forward/pypy/rlib/clibffi.py pypy/branch/fast-forward/pypy/rlib/rdynload.py pypy/branch/fast-forward/pypy/rlib/rwin32.py Log: On Windows, set sys.dllhandle to a working DLL handle if cpyext is enabled. Now ctypes.pythonapi works, except that: - the 'O' ctypes code does not work yet (python object <-> PyObject* conversion) - ctypes.pythonapi functions will crash the interpreter if they raise an exception which means that PyUnicode_GetDefaultEncoding() is the only usable function at the moment :-/ Modified: pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py Fri Nov 12 11:57:01 2010 @@ -554,6 +554,9 @@ def setup_builtin_modules(self): "NOT_RPYTHON: only for initializing the space." + if self.config.objspace.usemodules.cpyext: + from pypy.module.cpyext.state import State + self.fromcache(State).build_api(self) self.getbuiltinmodule('sys') self.getbuiltinmodule('imp') self.getbuiltinmodule('__builtin__') Modified: pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py (original) +++ pypy/branch/fast-forward/pypy/module/_rawffi/interp_rawffi.py Fri Nov 12 11:57:01 2010 @@ -136,12 +136,8 @@ for w_arg in space.unpackiterable(w_argtypes)] class W_CDLL(Wrappable): - def __init__(self, space, name): - try: - self.cdll = CDLL(name) - except DLOpenError, e: - raise operationerrfmt(space.w_OSError, '%s: %s', name, - e.msg or 'unspecified error') + def __init__(self, space, name, cdll): + self.cdll = cdll self.name = name self.w_cache = space.newdict() self.space = space @@ -211,9 +207,13 @@ def descr_new_cdll(space, w_type, name): try: - return space.wrap(W_CDLL(space, name)) + cdll = CDLL(name) + except DLOpenError, e: + raise operationerrfmt(space.w_OSError, '%s: %s', name, + e.msg or 'unspecified error') except OSError, e: raise wrap_oserror(space, e) + return space.wrap(W_CDLL(space, name, cdll)) descr_new_cdll.unwrap_spec = [ObjSpace, W_Root, str] W_CDLL.typedef = TypeDef( @@ -520,10 +520,12 @@ check_HRESULT.unwrap_spec = [ObjSpace, int] def get_libc(space): + name = get_libc_name() try: - return space.wrap(W_CDLL(space, get_libc_name())) + cdll = CDLL(name) except OSError, e: raise wrap_oserror(space, e) + return space.wrap(W_CDLL(space, name, cdll)) def get_errno(space): return space.wrap(rposix.get_errno()) Modified: pypy/branch/fast-forward/pypy/module/cpyext/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/__init__.py Fri Nov 12 11:57:01 2010 @@ -12,30 +12,8 @@ appleveldefs = { } - def setup_after_space_initialization(self): - """NOT_RPYTHON""" - state = self.space.fromcache(State) - if not self.space.config.translating: - state.api_lib = str(api.build_bridge(self.space)) - else: - api.setup_library(self.space) - def startup(self, space): - state = space.fromcache(State) - from pypy.module.cpyext.typeobject import setup_new_method_def - from pypy.module.cpyext.pyobject import RefcountState - setup_new_method_def(space) - if not we_are_translated(): - space.setattr(space.wrap(self), - space.wrap('api_lib'), - space.wrap(state.api_lib)) - else: - refcountstate = space.fromcache(RefcountState) - refcountstate.init_r2w_from_w2r() - - for func in api.INIT_FUNCTIONS: - func(space) - state.check_and_raise_exception() + space.fromcache(State).startup(space) # import these modules to register api functions by side-effect import pypy.module.cpyext.thread Modified: pypy/branch/fast-forward/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/api.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/api.py Fri Nov 12 11:57:01 2010 @@ -662,6 +662,8 @@ import ctypes bridge = ctypes.CDLL(str(modulename), mode=ctypes.RTLD_GLOBAL) + space.fromcache(State).install_dll(eci) + # populate static data for name, (typ, expr) in GLOBALS.iteritems(): from pypy.module import cpyext @@ -841,6 +843,25 @@ structs.append('%s %s = NULL;' % (typ, name)) struct_source = '\n'.join(structs) + separate_module_sources = [code, struct_source] + + if sys.platform == 'win32': + get_pythonapi_source = ''' + #include + HANDLE pypy_get_pythonapi_handle() { + MEMORY_BASIC_INFORMATION mi; + memset(&mi, 0, sizeof(mi)); + + if( !VirtualQueryEx(GetCurrentProcess(), &pypy_get_pythonapi_handle, + &mi, sizeof(mi)) ) + return 0; + + return (HMODULE)mi.AllocationBase; + } + ''' + separate_module_sources.append(get_pythonapi_source) + export_symbols_eci.append('pypy_get_pythonapi_handle') + eci = ExternalCompilationInfo( include_dirs=include_dirs, separate_module_files=[source_dir / "varargwrapper.c", @@ -855,11 +876,12 @@ source_dir / "cobject.c", source_dir / "capsule.c", ], - separate_module_sources = [code, struct_source], + separate_module_sources=separate_module_sources, export_symbols=export_symbols_eci, compile_extra=compile_extra, **kwds ) + return eci @@ -878,6 +900,8 @@ eci = build_eci(False, export_symbols, code) + space.fromcache(State).install_dll(eci) + run_bootstrap_functions(space) setup_va_functions(eci) Modified: pypy/branch/fast-forward/pypy/module/cpyext/state.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/state.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/state.py Fri Nov 12 11:57:01 2010 @@ -2,7 +2,8 @@ from pypy.rpython.lltypesystem import rffi, lltype from pypy.interpreter.error import OperationError from pypy.rpython.lltypesystem import lltype - +from pypy.rlib.rdynload import DLLHANDLE +import sys class State: datetimeAPI = None # used in tests @@ -43,3 +44,43 @@ if always: raise OperationError(self.space.w_SystemError, self.space.wrap( "Function returned an error result without setting an exception")) + + def build_api(self, space): + """NOT_RPYTHON + This function is called when at object space creation, + and drives the compilation of the cpyext library + """ + from pypy.module.cpyext import api + state = self.space.fromcache(State) + if not self.space.config.translating: + state.api_lib = str(api.build_bridge(self.space)) + else: + api.setup_library(self.space) + + def install_dll(self, eci): + """NOT_RPYTHON + Called when the dll has been compiled""" + if sys.platform == 'win32': + self.get_pythonapi_handle = rffi.llexternal( + 'pypy_get_pythonapi_handle', [], DLLHANDLE, + compilation_info=eci) + + def startup(self, space): + "This function is called when the program really starts" + + from pypy.module.cpyext.typeobject import setup_new_method_def + from pypy.module.cpyext.pyobject import RefcountState + from pypy.module.cpyext.api import INIT_FUNCTIONS + + setup_new_method_def(space) + if not we_are_translated(): + space.setattr(space.wrap(self), + space.wrap('api_lib'), + space.wrap(self.api_lib)) + else: + refcountstate = space.fromcache(RefcountState) + refcountstate.init_r2w_from_w2r() + + for func in INIT_FUNCTIONS: + func(space) + self.check_and_raise_exception() Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/conftest.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/test/conftest.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/conftest.py Fri Nov 12 11:57:01 2010 @@ -9,7 +9,7 @@ import pypy.module.cpyext.test.test_cpyext def pytest_funcarg__space(request): - return gettestobjspace(usemodules=['cpyext', 'thread']) + return gettestobjspace(usemodules=['cpyext', 'thread', '_rawffi']) def pytest_funcarg__api(request): return request.cls.api Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_api.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/test/test_api.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_api.py Fri Nov 12 11:57:01 2010 @@ -19,7 +19,7 @@ class BaseApiTest(LeakCheckingTest): def setup_class(cls): - cls.space = space = gettestobjspace(usemodules=['cpyext', 'thread']) + cls.space = space = gettestobjspace(usemodules=['cpyext', 'thread', '_rawffi']) # warm up reference counts: # - the posix module allocates a HCRYPTPROV on Windows Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py Fri Nov 12 11:57:01 2010 @@ -33,7 +33,7 @@ class AppTestApi: def setup_class(cls): - cls.space = gettestobjspace(usemodules=['cpyext', 'thread']) + cls.space = gettestobjspace(usemodules=['cpyext', 'thread', '_rawffi']) from pypy.rlib.libffi import get_libc_name cls.w_libc = cls.space.wrap(get_libc_name()) @@ -42,6 +42,15 @@ raises(ImportError, cpyext.load_module, "missing.file", "foo") raises(ImportError, cpyext.load_module, self.libc, "invalid.function") + def test_dllhandle(self): + import sys + assert sys.dllhandle + assert sys.dllhandle.getaddressindll('PyPyErr_NewException') + import ctypes # slow + PyUnicode_GetDefaultEncoding = ctypes.pythonapi.PyPyUnicode_GetDefaultEncoding + PyUnicode_GetDefaultEncoding.restype = ctypes.c_char_p + assert PyUnicode_GetDefaultEncoding() == 'ascii' + def compile_module(modname, **kwds): """ Build an extension module and return the filename of the resulting native @@ -132,7 +141,7 @@ class AppTestCpythonExtensionBase(LeakCheckingTest): def setup_class(cls): - cls.space = gettestobjspace(usemodules=['cpyext', 'thread']) + cls.space = gettestobjspace(usemodules=['cpyext', 'thread', '_rawffi']) cls.space.getbuiltinmodule("cpyext") from pypy.module.imp.importing import importhook importhook(cls.space, "os") # warm up reference counts Modified: pypy/branch/fast-forward/pypy/module/sys/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/sys/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/sys/__init__.py Fri Nov 12 11:57:01 2010 @@ -3,6 +3,8 @@ from pypy.rlib.objectmodel import we_are_translated import sys +_WIN = sys.platform == 'win32' + class Module(MixedModule): """Sys Builtin Module. """ def __init__(self, space, w_name): @@ -81,7 +83,6 @@ } if sys.platform == 'win32': - interpleveldefs['dllhandle'] = 'space.wrap(0)' # XXX for the moment interpleveldefs['winver'] = 'version.get_winver(space)' interpleveldefs['getwindowsversion'] = 'vm.getwindowsversion' @@ -105,6 +106,12 @@ # don't get the filesystemencoding at translation time assert self.filesystemencoding is None + else: + if _WIN: + from pypy.module.sys import vm + w_handle = vm.get_dllhandle(space) + space.setattr(self, space.wrap("dllhandle"), w_handle) + def getmodule(self, name): space = self.space w_modules = self.get('modules') Modified: pypy/branch/fast-forward/pypy/module/sys/vm.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/sys/vm.py (original) +++ pypy/branch/fast-forward/pypy/module/sys/vm.py Fri Nov 12 11:57:01 2010 @@ -135,3 +135,18 @@ space.wrap(info[2]), space.wrap(info[3]), space.wrap(info[4])]) + +def get_dllhandle(space): + if not space.config.objspace.usemodules.cpyext: + return space.wrap(0) + if not space.config.objspace.usemodules._rawffi: + return space.wrap(0) + + # Retrieve cpyext api handle + from pypy.module.cpyext.api import State + handle = space.fromcache(State).get_pythonapi_handle() + + # Make a dll object with it + from pypy.module._rawffi.interp_rawffi import W_CDLL, RawCDLL + cdll = RawCDLL(handle) + return space.wrap(W_CDLL(space, "python api", cdll)) Modified: pypy/branch/fast-forward/pypy/rlib/clibffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/clibffi.py (original) +++ pypy/branch/fast-forward/pypy/rlib/clibffi.py Fri Nov 12 11:57:01 2010 @@ -8,7 +8,7 @@ from pypy.rlib.objectmodel import we_are_translated from pypy.rlib.rmmap import alloc from pypy.rlib.rdynload import dlopen, dlclose, dlsym, dlsym_byordinal -from pypy.rlib.rdynload import DLOpenError +from pypy.rlib.rdynload import DLOpenError, DLLHANDLE from pypy.tool.autopath import pypydir from pypy.translator.tool.cbuild import ExternalCompilationInfo from pypy.translator.platform import platform @@ -32,7 +32,7 @@ #include /* Get the module where the "fopen" function resides in */ - HANDLE get_libc_handle() { + HANDLE pypy_get_libc_handle() { MEMORY_BASIC_INFORMATION mi; char buf[1000]; memset(&mi, 0, sizeof(mi)); @@ -102,7 +102,7 @@ libffidir.join('pypy_ffi.c'), ], export_symbols = ['ffi_call', 'ffi_prep_cif', 'ffi_prep_closure', - 'get_libc_handle'], + 'pypy_get_libc_handle'], ) FFI_TYPE_P = lltype.Ptr(lltype.ForwardReference()) @@ -246,7 +246,7 @@ LoadLibrary = rwin32.LoadLibrary - get_libc_handle = external('get_libc_handle', [], rwin32.HANDLE) + get_libc_handle = external('pypy_get_libc_handle', [], DLLHANDLE) def get_libc_name(): return rwin32.GetModuleFileName(get_libc_handle()) @@ -555,20 +555,9 @@ self.ll_result = lltype.nullptr(rffi.VOIDP.TO) AbstractFuncPtr.__del__(self) -class CDLL(object): - def __init__(self, libname): - """Load the library, or raises DLOpenError.""" - self.lib = lltype.nullptr(rffi.CCHARP.TO) - ll_libname = rffi.str2charp(libname) - try: - self.lib = dlopen(ll_libname) - finally: - lltype.free(ll_libname, flavor='raw') - - def __del__(self): - if self.lib: - dlclose(self.lib) - self.lib = lltype.nullptr(rffi.CCHARP.TO) +class RawCDLL(object): + def __init__(self, handle): + self.lib = handle def getpointer(self, name, argtypes, restype, flags=FUNCFLAG_CDECL): # these arguments are already casted to proper ffi @@ -593,3 +582,20 @@ def getaddressindll(self, name): return dlsym(self.lib, name) +class CDLL(RawCDLL): + _default = rffi.cast(DLLHANDLE, -1) + + def __init__(self, libname): + """Load the library, or raises DLOpenError.""" + RawCDLL.__init__(self, self._default) + ll_libname = rffi.str2charp(libname) + try: + self.lib = dlopen(ll_libname) + finally: + lltype.free(ll_libname, flavor='raw') + + def __del__(self): + if self.lib != self._default: + dlclose(self.lib) + self.lib = self._default + Modified: pypy/branch/fast-forward/pypy/rlib/rdynload.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rdynload.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rdynload.py Fri Nov 12 11:57:01 2010 @@ -67,6 +67,8 @@ c_dlerror = external('dlerror', [], rffi.CCHARP) c_dlsym = external('dlsym', [rffi.VOIDP, rffi.CCHARP], rffi.VOIDP) + DLLHANDLE = rffi.VOIDP + RTLD_LOCAL = cConfig.RTLD_LOCAL RTLD_GLOBAL = cConfig.RTLD_GLOBAL RTLD_NOW = cConfig.RTLD_NOW @@ -110,6 +112,8 @@ raise KeyError(index) if _WIN32: + DLLHANDLE = rwin32.HMODULE + def dlopen(name): res = rwin32.LoadLibrary(name) if not res: Modified: pypy/branch/fast-forward/pypy/rlib/rwin32.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rwin32.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rwin32.py Fri Nov 12 11:57:01 2010 @@ -98,11 +98,11 @@ # is hidden by operations in ll2ctypes. Call it now. GetLastError() - LoadLibrary = winexternal('LoadLibraryA', [rffi.CCHARP], rffi.VOIDP) + LoadLibrary = winexternal('LoadLibraryA', [rffi.CCHARP], HMODULE) GetProcAddress = winexternal('GetProcAddress', - [rffi.VOIDP, rffi.CCHARP], + [HMODULE, rffi.CCHARP], rffi.VOIDP) - FreeLibrary = winexternal('FreeLibrary', [rffi.VOIDP], BOOL) + FreeLibrary = winexternal('FreeLibrary', [HMODULE], BOOL) LocalFree = winexternal('LocalFree', [HLOCAL], DWORD) CloseHandle = winexternal('CloseHandle', [HANDLE], BOOL) From arigo at codespeak.net Fri Nov 12 12:03:32 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 12 Nov 2010 12:03:32 +0100 (CET) Subject: [pypy-svn] r79029 - in pypy/trunk/pypy: rpython/memory/gc translator/c/test Message-ID: <20101112110332.7E3B45080C@codespeak.net> Author: arigo Date: Fri Nov 12 12:03:30 2010 New Revision: 79029 Modified: pypy/trunk/pypy/rpython/memory/gc/base.py pypy/trunk/pypy/rpython/memory/gc/generation.py pypy/trunk/pypy/rpython/memory/gc/inspector.py pypy/trunk/pypy/rpython/memory/gc/minimark.py pypy/trunk/pypy/rpython/memory/gc/semispace.py pypy/trunk/pypy/translator/c/test/test_newgc.py Log: Hopefully, this is a more general fix for the issue of not finding all GC roots in inspector.py. Adds a method enumerate_all_roots() on the GCBase class, which is overridden in some subclasses. Apart from doing collection, all other usages of walk_roots() have been replaced by calls to enumerate_all_roots() Modified: pypy/trunk/pypy/rpython/memory/gc/base.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/base.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/base.py Fri Nov 12 12:03:30 2010 @@ -247,6 +247,21 @@ (not self.config.taggedpointers or llmemory.cast_adr_to_int(addr) & 1 == 0)) + def enumerate_all_roots(self, callback, arg): + """For each root object, invoke callback(obj, arg). + 'callback' should not be a bound method. + Note that this method is not suitable for actually doing the + collection in a moving GC, because you cannot write back a + modified address. It is there only for inspection. + """ + # overridden in some subclasses, for GCs which have an additional + # list of last generation roots + callback2, attrname = _convert_callback_formats(callback) # :-/ + setattr(self, attrname, arg) + self.root_walker.walk_roots(callback2, callback2, callback2) + self.run_finalizers.foreach(callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' + def debug_check_consistency(self): """To use after a collection. If self.DEBUG is set, this enumerates all roots and traces all objects to check if we didn't @@ -260,8 +275,7 @@ self._debug_pending = self.AddressStack() if not we_are_translated(): self.root_walker._walk_prebuilt_gc(self._debug_record) - callback = GCBase._debug_callback - self.root_walker.walk_roots(callback, callback, callback) + self.enumerate_all_roots(GCBase._debug_callback, self) pending = self._debug_pending while pending.non_empty(): obj = pending.pop() @@ -275,9 +289,8 @@ seen.add(obj) self.debug_check_object(obj) self._debug_pending.append(obj) - def _debug_callback(self, root): - obj = root.address[0] - ll_assert(bool(obj), "NULL address from walk_roots()") + @staticmethod + def _debug_callback(obj, self): self._debug_record(obj) def _debug_callback2(self, pointer, ignored): obj = pointer.address[0] @@ -432,3 +445,17 @@ if factor != 1: return 0.0 return value + +def _convert_callback_formats(callback): + callback = getattr(callback, 'im_func', callback) + if callback not in _converted_callback_formats: + def callback2(gc, root): + obj = root.address[0] + ll_assert(bool(obj), "NULL address from walk_roots()") + callback(obj, getattr(gc, attrname)) + attrname = '_callback2_arg%d' % len(_converted_callback_formats) + _converted_callback_formats[callback] = callback2, attrname + return _converted_callback_formats[callback] + +_convert_callback_formats._annspecialcase_ = 'specialize:memo' +_converted_callback_formats = {} Modified: pypy/trunk/pypy/rpython/memory/gc/generation.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/generation.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/generation.py Fri Nov 12 12:03:30 2010 @@ -572,16 +572,10 @@ def _compute_current_nursery_hash(self, obj): return intmask(llmemory.cast_adr_to_int(obj) + self.nursery_hash_base) - def heap_stats_walk_roots(self): - self.last_generation_root_objects.foreach( - self._track_heap_ext, None) - self.root_walker.walk_roots( - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root) - - def _track_heap_ext(self, adr, ignored): - self.trace(adr, self.track_heap_parent, adr) + def enumerate_all_roots(self, callback, arg): + self.last_generation_root_objects.foreach(callback, arg) + SemiSpaceGC.enumerate_all_roots(self, callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' def debug_check_object(self, obj): """Check the invariants about 'obj' that should be true Modified: pypy/trunk/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/inspector.py Fri Nov 12 12:03:30 2010 @@ -11,18 +11,15 @@ # ---------- implementation of pypy.rlib.rgc.get_rpy_roots() ---------- -def _counting_rpy_root(gc, root): +def _counting_rpy_root(obj, gc): gc._count_rpy += 1 def _do_count_rpy_roots(gc): gc._count_rpy = 0 - gc.root_walker.walk_roots( - _counting_rpy_root, - _counting_rpy_root, - _counting_rpy_root) + gc.enumerate_all_roots(_counting_rpy_root, gc) return gc._count_rpy -def _append_rpy_root(gc, root): +def _append_rpy_root(obj, gc): # Can use the gc list, but should not allocate! # It is essential that the list is not resizable! lst = gc._list_rpy @@ -30,15 +27,12 @@ if index >= len(lst): raise ValueError gc._count_rpy = index + 1 - lst[index] = llmemory.cast_adr_to_ptr(root.address[0], llmemory.GCREF) + lst[index] = llmemory.cast_adr_to_ptr(obj, llmemory.GCREF) def _do_append_rpy_roots(gc, lst): gc._count_rpy = 0 gc._list_rpy = lst - gc.root_walker.walk_roots( - _append_rpy_root, - _append_rpy_root, - _append_rpy_root) + gc.enumerate_all_roots(_append_rpy_root, gc) gc._list_rpy = None def get_rpy_roots(gc): @@ -172,12 +166,7 @@ self.pending.append(obj) def add_roots(self): - self.gc._heap_dumper = self - self.gc.root_walker.walk_roots( - _hd_add_root, - _hd_add_root, - _hd_add_root) - self.gc._heap_dumper = None + self.gc.enumerate_all_roots(_hd_add_root, self) pendingroots = self.pending self.pending = AddressStack() self.walk(pendingroots) @@ -188,8 +177,8 @@ while pending.non_empty(): self.writeobj(pending.pop()) -def _hd_add_root(gc, root): - gc._heap_dumper.add(root.address[0]) +def _hd_add_root(obj, heap_dumper): + heap_dumper.add(obj) def dump_rpy_heap(gc, fd): heapdumper = HeapDumper(gc, fd) Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Fri Nov 12 12:03:30 2010 @@ -1288,6 +1288,11 @@ self.run_finalizers.foreach(self._collect_obj, self.objects_to_trace) + def enumerate_all_roots(self, callback, arg): + self.prebuilt_root_objects.foreach(callback, arg) + MovingGCBase.enumerate_all_roots(self, callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' + @staticmethod def _collect_obj(obj, objects_to_trace): objects_to_trace.append(obj) Modified: pypy/trunk/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/semispace.py Fri Nov 12 12:03:30 2010 @@ -693,15 +693,10 @@ self._ll_typeid_map[idx].size += llmemory.raw_malloc_usage(totsize) self.trace(adr, self.track_heap_parent, adr) - def _track_heap_root(self, root): - self.track_heap(root.address[0]) + @staticmethod + def _track_heap_root(obj, self): + self.track_heap(obj) - def heap_stats_walk_roots(self): - self.root_walker.walk_roots( - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root) - def heap_stats(self): self._tracked_dict = self.AddressDict() max_tid = self.root_walker.gcdata.max_type_id @@ -714,7 +709,7 @@ while i < max_tid: self._tracked_dict.add(llmemory.cast_ptr_to_adr(ll_typeid_map[i])) i += 1 - self.heap_stats_walk_roots() + self.enumerate_all_roots(SemiSpaceGC._track_heap_root, self) self._ll_typeid_map = lltype.nullptr(ARRAY_TYPEID_MAP) self._tracked_dict.delete() return ll_typeid_map Modified: pypy/trunk/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/trunk/pypy/translator/c/test/test_newgc.py (original) +++ pypy/trunk/pypy/translator/c/test/test_newgc.py Fri Nov 12 12:03:30 2010 @@ -1179,21 +1179,22 @@ b = 0 c = 0 for i in range(len(tb)): - if tb[i].count == 10: + if tb[i].count == 10: # the type of S a += 1 nr = i for i in range(len(tb)): - if tb[i].count == 3: + if tb[i].count == 3: # the type GcArray(Ptr(S)) b += 1 c += tb[i].links[nr] - # we don't count b here since there can be more singletons, + # b can be 1 or 2 here since _heap_stats() is free to return or + # ignore the three GcStructs that point to the GcArray(Ptr(S)). # important one is c, a is for check return c * 100 + b * 10 + a return f def test_gc_heap_stats(self): res = self.run("gc_heap_stats") - assert res == 3011 + assert res == 3011 or res == 3021 def definestr_string_builder(cls): def fn(_): From afa at codespeak.net Fri Nov 12 12:55:13 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 12 Nov 2010 12:55:13 +0100 (CET) Subject: [pypy-svn] r79030 - pypy/trunk/pypy/jit/backend/test Message-ID: <20101112115513.2E9B036C222@codespeak.net> Author: afa Date: Fri Nov 12 12:55:11 2010 New Revision: 79030 Modified: pypy/trunk/pypy/jit/backend/test/runner_test.py Log: Export symbols to fix tests on Windows Modified: pypy/trunk/pypy/jit/backend/test/runner_test.py ============================================================================== --- pypy/trunk/pypy/jit/backend/test/runner_test.py (original) +++ pypy/trunk/pypy/jit/backend/test/runner_test.py Fri Nov 12 12:55:11 2010 @@ -2168,12 +2168,14 @@ # Tested with a function that intentionally does not cast the # result to RESTYPE, but makes sure that we return the whole # value in eax or rax. - eci = ExternalCompilationInfo(separate_module_sources=[""" + eci = ExternalCompilationInfo( + separate_module_sources=[""" long fn_test_result_of_call(long x) { return x + 1; } - """]) + """], + export_symbols=['fn_test_result_of_call']) f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed], RESTYPE, compilation_info=eci, _nowrapper=True) value = intmask(0xFFEEDDCCBBAA9988) @@ -2199,12 +2201,14 @@ # Tested with a function that intentionally does not cast the # result to RESTYPE, but makes sure that we return the whole # value in eax or rax. - eci = ExternalCompilationInfo(separate_module_sources=[""" + eci = ExternalCompilationInfo( + separate_module_sources=[""" long fn_test_result_of_call(long x) { return x + 1; } - """]) + """], + export_symbols=['fn_test_result_of_call']) f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed], RESTYPE, compilation_info=eci, _nowrapper=True) value = intmask(0xFFEEDDCCBBAA9988) From arigo at codespeak.net Fri Nov 12 13:02:50 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 12 Nov 2010 13:02:50 +0100 (CET) Subject: [pypy-svn] r79031 - pypy/trunk/pypy/rpython/test Message-ID: <20101112120250.EEFF65080B@codespeak.net> Author: arigo Date: Fri Nov 12 13:02:49 2010 New Revision: 79031 Modified: pypy/trunk/pypy/rpython/test/test_rpbc.py Log: Fix: don't catch NotImplementedError. Modified: pypy/trunk/pypy/rpython/test/test_rpbc.py ============================================================================== --- pypy/trunk/pypy/rpython/test/test_rpbc.py (original) +++ pypy/trunk/pypy/rpython/test/test_rpbc.py Fri Nov 12 13:02:49 2010 @@ -1547,7 +1547,7 @@ def test_always_raising_methods(self): class Base: def m(self): - raise NotImplementedError + raise KeyError class A(Base): def m(self): return 42 @@ -1560,11 +1560,11 @@ o = B() try: o.m() - except NotImplementedError: - pass + except KeyError: + assert 0 return B().m() - self.interpret_raises(NotImplementedError, f, [7]) + self.interpret_raises(KeyError, f, [7]) def test_possible_missing_attribute_access(self): py.test.skip("Should explode or give some warning") From arigo at codespeak.net Fri Nov 12 14:04:49 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 12 Nov 2010 14:04:49 +0100 (CET) Subject: [pypy-svn] r79036 - pypy/trunk/pypy/jit/backend/llsupport Message-ID: <20101112130449.37486282B9D@codespeak.net> Author: arigo Date: Fri Nov 12 14:04:47 2010 New Revision: 79036 Modified: pypy/trunk/pypy/jit/backend/llsupport/descr.py Log: Special-case array of floats. May fix tests on Windows, where the default alignment is 8 bytes. Modified: pypy/trunk/pypy/jit/backend/llsupport/descr.py ============================================================================== --- pypy/trunk/pypy/jit/backend/llsupport/descr.py (original) +++ pypy/trunk/pypy/jit/backend/llsupport/descr.py Fri Nov 12 14:04:47 2010 @@ -130,6 +130,7 @@ # ArrayDescrs _A = lltype.GcArray(lltype.Signed) # a random gcarray +_AF = lltype.GcArray(lltype.Float) # an array of C doubles class BaseArrayDescr(AbstractDescr): @@ -171,16 +172,21 @@ _clsname = 'GcPtrArrayDescr' _is_array_of_pointers = True -_CA = rffi.CArray(lltype.Signed) +class FloatArrayDescr(BaseArrayDescr): + _clsname = 'FloatArrayDescr' + _is_array_of_floats = True + def get_base_size(self, translate_support_code): + basesize, _, _ = symbolic.get_array_token(_AF, translate_support_code) + return basesize + def get_item_size(self, translate_support_code): + return symbolic.get_size(lltype.Float, translate_support_code) class BaseArrayNoLengthDescr(BaseArrayDescr): def get_base_size(self, translate_support_code): - basesize, _, _ = symbolic.get_array_token(_CA, translate_support_code) - return basesize + return 0 def get_ofs_length(self, translate_support_code): - _, _, ofslength = symbolic.get_array_token(_CA, translate_support_code) - return ofslength + return -1 class NonGcPtrArrayNoLengthDescr(BaseArrayNoLengthDescr): _clsname = 'NonGcPtrArrayNoLengthDescr' @@ -192,6 +198,8 @@ _is_array_of_pointers = True def getArrayDescrClass(ARRAY): + if ARRAY.OF is lltype.Float: + return FloatArrayDescr return getDescrClass(ARRAY.OF, BaseArrayDescr, GcPtrArrayDescr, NonGcPtrArrayDescr, 'Array', 'get_item_size', '_is_array_of_floats', '_is_item_signed') @@ -219,7 +227,8 @@ basesize, itemsize, ofslength = symbolic.get_array_token(ARRAY, False) assert basesize == arraydescr.get_base_size(False) assert itemsize == arraydescr.get_item_size(False) - assert ofslength == arraydescr.get_ofs_length(False) + if not ARRAY._hints.get('nolength', False): + assert ofslength == arraydescr.get_ofs_length(False) if isinstance(ARRAY, lltype.GcArray): gccache.init_array_descr(ARRAY, arraydescr) cache[ARRAY] = arraydescr From afa at codespeak.net Fri Nov 12 14:16:25 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 12 Nov 2010 14:16:25 +0100 (CET) Subject: [pypy-svn] r79037 - pypy/trunk/pypy/jit/backend/llsupport/test Message-ID: <20101112131625.A95FE5080E@codespeak.net> Author: afa Date: Fri Nov 12 14:16:24 2010 New Revision: 79037 Modified: pypy/trunk/pypy/jit/backend/llsupport/test/test_descr.py Log: Fix the test on Windows, where doubles are aligned on 8-bytes boundary. Modified: pypy/trunk/pypy/jit/backend/llsupport/test/test_descr.py ============================================================================== --- pypy/trunk/pypy/jit/backend/llsupport/test/test_descr.py (original) +++ pypy/trunk/pypy/jit/backend/llsupport/test/test_descr.py Fri Nov 12 14:16:24 2010 @@ -5,6 +5,7 @@ from pypy.rpython.annlowlevel import llhelper from pypy.jit.metainterp.history import BoxInt, BoxFloat, BoxPtr from pypy.jit.metainterp import history +import struct def test_get_size_descr(): c0 = GcCache(False) @@ -130,11 +131,13 @@ assert not descr3.is_array_of_floats() assert descr4.is_array_of_floats() # - WORD = rffi.sizeof(lltype.Signed) - assert descr1.get_base_size(False) == WORD - assert descr2.get_base_size(False) == WORD - assert descr3.get_base_size(False) == WORD - assert descr4.get_base_size(False) == WORD + def get_alignment(code): + # Retrieve default alignment for the compiler/platform + return struct.calcsize('l' + code) - struct.calcsize(code) + assert descr1.get_base_size(False) == get_alignment('c') + assert descr2.get_base_size(False) == get_alignment('p') + assert descr3.get_base_size(False) == get_alignment('p') + assert descr4.get_base_size(False) == get_alignment('d') assert descr1.get_ofs_length(False) == 0 assert descr2.get_ofs_length(False) == 0 assert descr3.get_ofs_length(False) == 0 From afa at codespeak.net Fri Nov 12 14:25:42 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 12 Nov 2010 14:25:42 +0100 (CET) Subject: [pypy-svn] r79039 - pypy/branch/fast-forward/lib-python/modified-2.7.0/test Message-ID: <20101112132542.86427282BDC@codespeak.net> Author: afa Date: Fri Nov 12 14:25:40 2010 New Revision: 79039 Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py Log: Try to release mmap resources during the test. Modified: pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.7.0/test/test_multiprocessing.py Fri Nov 12 14:25:40 2010 @@ -1585,6 +1585,10 @@ if len(blocks) > maxblocks: i = random.randrange(maxblocks) del blocks[i] + # XXX There should be a better way to release resources for a + # single block + if i % maxblocks == 0: + import gc; gc.collect() # get the heap object heap = multiprocessing.heap.BufferWrapper._heap From afa at codespeak.net Fri Nov 12 14:26:46 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 12 Nov 2010 14:26:46 +0100 (CET) Subject: [pypy-svn] r79040 - pypy/branch/fast-forward/lib_pypy Message-ID: <20101112132646.6B306282BE3@codespeak.net> Author: afa Date: Fri Nov 12 14:26:44 2010 New Revision: 79040 Modified: pypy/branch/fast-forward/lib_pypy/_testcapi.py Log: Fix import of testcapi on Windows Modified: pypy/branch/fast-forward/lib_pypy/_testcapi.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_testcapi.py (original) +++ pypy/branch/fast-forward/lib_pypy/_testcapi.py Fri Nov 12 14:26:44 2010 @@ -28,7 +28,8 @@ # XXX libpypy-c.lib is currently not installed automatically library = os.path.join(thisdir, '..', 'include', 'libpypy-c') libraries = [library, 'oleaut32'] - extra_ldargs = ['/MANIFEST'] # needed for VC10 + extra_ldargs = ['/MANIFEST', # needed for VC10 + '/EXPORT:init_testcapi'] else: output_filename = '_testcapi.so' libraries = [] From cfbolz at codespeak.net Fri Nov 12 15:31:41 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 12 Nov 2010 15:31:41 +0100 (CET) Subject: [pypy-svn] r79043 - in pypy/branch/mapdict-without-jit/pypy/objspace/std: . test Message-ID: <20101112143141.DE605282BDC@codespeak.net> Author: cfbolz Date: Fri Nov 12 15:31:40 2010 New Revision: 79043 Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py Log: Add a test and a fix for the case when the two uses of the cache disturb each other. Can be improved, but probably rare. Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py ============================================================================== --- pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py (original) +++ pypy/branch/mapdict-without-jit/pypy/objspace/std/mapdict.py Fri Nov 12 15:31:40 2010 @@ -428,6 +428,7 @@ self.storage = make_sure_not_resized([None] * map.size_estimate()) def _mapdict_read_storage(self, index): + assert index >= 0 return self.storage[index] def _mapdict_write_storage(self, index, value): self.storage[index] = value @@ -495,6 +496,7 @@ return rerased.unerase_fixedsizelist(erased, W_Root) def _mapdict_read_storage(self, index): + assert index >= 0 for i in rangenmin1: if index == i: erased = getattr(self, "_value%s" % i) @@ -704,7 +706,7 @@ # used if we_are_jitted(). entry = pycode._mapdict_caches[nameindex] map = w_obj._get_mapdict_map() - if entry.is_valid_for_map(map): + if entry.is_valid_for_map(map) and entry.w_method is None: # everything matches, it's incredibly fast return w_obj._mapdict_read_storage(entry.index) return LOAD_ATTR_slowpath(pycode, w_obj, nameindex, map) Modified: pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py ============================================================================== --- pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py (original) +++ pypy/branch/mapdict-without-jit/pypy/objspace/std/test/test_mapdict.py Fri Nov 12 15:31:40 2010 @@ -830,6 +830,34 @@ res = self.check(h, 'cm') assert res == (0, 0, 0) + def test_mix_cache_bug(self): + # bit sucky + global C + + class C(object): + def m(*args): + return args + + exec """if 1: + + def f(): + c = C() + res = c.m(1) + assert res == (c, 1) + bm = c.m + res = bm(1) + assert res == (c, 1) + return 42 + + """ + res = self.check(f, 'm') + assert res == (1, 1, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) class AppTestGlobalCaching(AppTestWithMapDict): def setup_class(cls): From afa at codespeak.net Fri Nov 12 16:25:03 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 12 Nov 2010 16:25:03 +0100 (CET) Subject: [pypy-svn] r79044 - pypy/branch/fast-forward/pypy/module/sys Message-ID: <20101112152503.9367C282BEB@codespeak.net> Author: afa Date: Fri Nov 12 16:25:01 2010 New Revision: 79044 Modified: pypy/branch/fast-forward/pypy/module/sys/vm.py Log: When a llexternal function is defined in a "separate_module_source", chances are that it's not declared in any header. This is not a major problem for the C translation, (the compiler will emit a warning for the function call), but the jitcode cannot compile because it needs the address of the function. In short: mark the calling code with @jit.dont_look_inside Modified: pypy/branch/fast-forward/pypy/module/sys/vm.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/sys/vm.py (original) +++ pypy/branch/fast-forward/pypy/module/sys/vm.py Fri Nov 12 16:25:01 2010 @@ -4,6 +4,7 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.gateway import ObjSpace from pypy.rlib.runicode import MAXUNICODE +from pypy.rlib import jit import sys # ____________________________________________________________ @@ -136,6 +137,7 @@ space.wrap(info[3]), space.wrap(info[4])]) + at jit.dont_look_inside def get_dllhandle(space): if not space.config.objspace.usemodules.cpyext: return space.wrap(0) From afa at codespeak.net Fri Nov 12 16:29:28 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 12 Nov 2010 16:29:28 +0100 (CET) Subject: [pypy-svn] r79045 - in pypy/branch/fast-forward: lib_pypy/ctypes_config_cache lib_pypy/ctypes_config_cache/test pypy/doc pypy/jit/backend/llsupport pypy/jit/backend/llsupport/test pypy/jit/backend/test pypy/jit/backend/x86 pypy/jit/backend/x86/test pypy/jit/codewriter pypy/jit/codewriter/test pypy/jit/metainterp pypy/jit/metainterp/optimizeopt pypy/jit/metainterp/test pypy/jit/tool pypy/module/_stackless/test pypy/module/cpyext pypy/module/posix pypy/objspace/flow pypy/objspace/flow/test pypy/rlib pypy/rlib/test pypy/rpython pypy/rpython/memory/gc pypy/rpython/memory/gctransform pypy/rpython/test pypy/translator/c pypy/translator/c/test pypy/translator/cli/src pypy/translator/goal pypy/translator/goal/test2 Message-ID: <20101112152928.D194E282BE3@codespeak.net> Author: afa Date: Fri Nov 12 16:29:24 2010 New Revision: 79045 Added: pypy/branch/fast-forward/pypy/doc/release-1.4.0.txt - copied unchanged from r79038, pypy/trunk/pypy/doc/release-1.4.0.txt pypy/branch/fast-forward/pypy/module/_stackless/test/conftest.py - copied unchanged from r79038, pypy/trunk/pypy/module/_stackless/test/conftest.py Modified: pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/dumpcache.py pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/test/test_cache.py pypy/branch/fast-forward/pypy/jit/backend/llsupport/descr.py pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_descr.py pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_zrpy_gc.py pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py pypy/branch/fast-forward/pypy/jit/codewriter/assembler.py pypy/branch/fast-forward/pypy/jit/codewriter/call.py pypy/branch/fast-forward/pypy/jit/codewriter/codewriter.py pypy/branch/fast-forward/pypy/jit/codewriter/effectinfo.py pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/optimizer.py pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/string.py pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py pypy/branch/fast-forward/pypy/jit/metainterp/resume.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_del.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_virtualref.py pypy/branch/fast-forward/pypy/jit/tool/pypytrace-mode.el pypy/branch/fast-forward/pypy/module/cpyext/api.py pypy/branch/fast-forward/pypy/module/cpyext/pyobject.py pypy/branch/fast-forward/pypy/module/posix/interp_posix.py pypy/branch/fast-forward/pypy/objspace/flow/objspace.py pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py pypy/branch/fast-forward/pypy/rlib/rarithmetic.py pypy/branch/fast-forward/pypy/rlib/streamio.py pypy/branch/fast-forward/pypy/rlib/test/test_rarithmetic.py pypy/branch/fast-forward/pypy/rpython/memory/gc/base.py pypy/branch/fast-forward/pypy/rpython/memory/gc/generation.py pypy/branch/fast-forward/pypy/rpython/memory/gc/inspector.py pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py pypy/branch/fast-forward/pypy/rpython/memory/gc/semispace.py pypy/branch/fast-forward/pypy/rpython/memory/gctransform/framework.py pypy/branch/fast-forward/pypy/rpython/rmodel.py pypy/branch/fast-forward/pypy/rpython/test/test_rpbc.py pypy/branch/fast-forward/pypy/translator/c/genc.py pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py pypy/branch/fast-forward/pypy/translator/cli/src/pypylib.cs pypy/branch/fast-forward/pypy/translator/goal/app_main.py pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py Log: Merge from trunk svn merge -r78901:79038 ../trunk Modified: pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/dumpcache.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/dumpcache.py (original) +++ pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/dumpcache.py Fri Nov 12 16:29:24 2010 @@ -16,8 +16,10 @@ except ImportError: from pypy.jit.backend import detect_cpu cpumodel = detect_cpu.autodetect_main_model_and_size() -mod = __import__("ctypes_config_cache._%s_%%s_" %% (cpumodel,), - None, None, ["*"]) +# XXX relative import, should be removed together with +# XXX the relative imports done e.g. by lib_pypy/pypy_test/test_hashlib +mod = __import__("_%s_%%s_" %% (cpumodel,), + globals(), locals(), ["*"]) globals().update(mod.__dict__)\ ''' % (basename,) g.close() Modified: pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/test/test_cache.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/test/test_cache.py (original) +++ pypy/branch/fast-forward/lib_pypy/ctypes_config_cache/test/test_cache.py Fri Nov 12 16:29:24 2010 @@ -7,9 +7,8 @@ def run(filename, outputname): filepath = dirpath.join(filename) - tmpdir2 = udir.ensure('testcache-' + filename, dir=True) - tmpdir = tmpdir2.ensure('ctypes_config_cache', dir=True) - tmpdir.join('__init__.py').write('\n') + tmpdir = udir.ensure('testcache-' + os.path.splitext(filename)[0], + dir=True) tmpdir.join('dumpcache.py').write(dirpath.join('dumpcache.py').read()) path = sys.path[:] try: @@ -21,13 +20,13 @@ # outputpath = tmpdir.join(outputname) assert outputpath.check(exists=1) - d = {} + modname = os.path.splitext(outputname)[0] try: - sys.path.insert(0, str(tmpdir2)) + sys.path.insert(0, str(tmpdir)) + d = {} execfile(str(outputpath), d) finally: sys.path[:] = path - sys.modules.pop('ctypes_config_cache', None) return d Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/descr.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/llsupport/descr.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/descr.py Fri Nov 12 16:29:24 2010 @@ -130,6 +130,7 @@ # ArrayDescrs _A = lltype.GcArray(lltype.Signed) # a random gcarray +_AF = lltype.GcArray(lltype.Float) # an array of C doubles class BaseArrayDescr(AbstractDescr): @@ -171,16 +172,21 @@ _clsname = 'GcPtrArrayDescr' _is_array_of_pointers = True -_CA = rffi.CArray(lltype.Signed) +class FloatArrayDescr(BaseArrayDescr): + _clsname = 'FloatArrayDescr' + _is_array_of_floats = True + def get_base_size(self, translate_support_code): + basesize, _, _ = symbolic.get_array_token(_AF, translate_support_code) + return basesize + def get_item_size(self, translate_support_code): + return symbolic.get_size(lltype.Float, translate_support_code) class BaseArrayNoLengthDescr(BaseArrayDescr): def get_base_size(self, translate_support_code): - basesize, _, _ = symbolic.get_array_token(_CA, translate_support_code) - return basesize + return 0 def get_ofs_length(self, translate_support_code): - _, _, ofslength = symbolic.get_array_token(_CA, translate_support_code) - return ofslength + return -1 class NonGcPtrArrayNoLengthDescr(BaseArrayNoLengthDescr): _clsname = 'NonGcPtrArrayNoLengthDescr' @@ -192,6 +198,8 @@ _is_array_of_pointers = True def getArrayDescrClass(ARRAY): + if ARRAY.OF is lltype.Float: + return FloatArrayDescr return getDescrClass(ARRAY.OF, BaseArrayDescr, GcPtrArrayDescr, NonGcPtrArrayDescr, 'Array', 'get_item_size', '_is_array_of_floats', '_is_item_signed') @@ -219,7 +227,8 @@ basesize, itemsize, ofslength = symbolic.get_array_token(ARRAY, False) assert basesize == arraydescr.get_base_size(False) assert itemsize == arraydescr.get_item_size(False) - assert ofslength == arraydescr.get_ofs_length(False) + if not ARRAY._hints.get('nolength', False): + assert ofslength == arraydescr.get_ofs_length(False) if isinstance(ARRAY, lltype.GcArray): gccache.init_array_descr(ARRAY, arraydescr) cache[ARRAY] = arraydescr Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py Fri Nov 12 16:29:24 2010 @@ -19,6 +19,7 @@ # ____________________________________________________________ class GcLLDescription(GcCache): + minimal_size_in_nursery = 0 def __init__(self, gcdescr, translator=None, rtyper=None): GcCache.__init__(self, translator is not None, rtyper) self.gcdescr = gcdescr @@ -386,6 +387,7 @@ (self.array_basesize, _, self.array_length_ofs) = \ symbolic.get_array_token(lltype.GcArray(lltype.Signed), True) self.max_size_of_young_obj = self.GCClass.JIT_max_size_of_young_obj() + self.minimal_size_in_nursery=self.GCClass.JIT_minimal_size_in_nursery() # make a malloc function, with three arguments def malloc_basic(size, tid): @@ -468,6 +470,7 @@ def malloc_fixedsize_slowpath(size): if self.DEBUG: random_usage_of_xmm_registers() + assert size >= self.minimal_size_in_nursery try: gcref = llop1.do_malloc_fixedsize_clear(llmemory.GCREF, 0, size, True, False, False) Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_descr.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_descr.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_descr.py Fri Nov 12 16:29:24 2010 @@ -5,6 +5,7 @@ from pypy.rpython.annlowlevel import llhelper from pypy.jit.metainterp.history import BoxInt, BoxFloat, BoxPtr from pypy.jit.metainterp import history +import struct def test_get_size_descr(): c0 = GcCache(False) @@ -130,11 +131,13 @@ assert not descr3.is_array_of_floats() assert descr4.is_array_of_floats() # - WORD = rffi.sizeof(lltype.Signed) - assert descr1.get_base_size(False) == WORD - assert descr2.get_base_size(False) == WORD - assert descr3.get_base_size(False) == WORD - assert descr4.get_base_size(False) == WORD + def get_alignment(code): + # Retrieve default alignment for the compiler/platform + return struct.calcsize('l' + code) - struct.calcsize(code) + assert descr1.get_base_size(False) == get_alignment('c') + assert descr2.get_base_size(False) == get_alignment('p') + assert descr3.get_base_size(False) == get_alignment('p') + assert descr4.get_base_size(False) == get_alignment('d') assert descr1.get_ofs_length(False) == 0 assert descr2.get_ofs_length(False) == 0 assert descr3.get_ofs_length(False) == 0 Modified: pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py Fri Nov 12 16:29:24 2010 @@ -2168,12 +2168,14 @@ # Tested with a function that intentionally does not cast the # result to RESTYPE, but makes sure that we return the whole # value in eax or rax. - eci = ExternalCompilationInfo(separate_module_sources=[""" + eci = ExternalCompilationInfo( + separate_module_sources=[""" long fn_test_result_of_call(long x) { return x + 1; } - """]) + """], + export_symbols=['fn_test_result_of_call']) f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed], RESTYPE, compilation_info=eci, _nowrapper=True) value = intmask(0xFFEEDDCCBBAA9988) @@ -2199,12 +2201,14 @@ # Tested with a function that intentionally does not cast the # result to RESTYPE, but makes sure that we return the whole # value in eax or rax. - eci = ExternalCompilationInfo(separate_module_sources=[""" + eci = ExternalCompilationInfo( + separate_module_sources=[""" long fn_test_result_of_call(long x) { return x + 1; } - """]) + """], + export_symbols=['fn_test_result_of_call']) f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed], RESTYPE, compilation_info=eci, _nowrapper=True) value = intmask(0xFFEEDDCCBBAA9988) Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py Fri Nov 12 16:29:24 2010 @@ -1853,6 +1853,7 @@ def malloc_cond_fixedsize(self, nursery_free_adr, nursery_top_adr, size, tid): + size = max(size, self.cpu.gc_ll_descr.minimal_size_in_nursery) self.mc.ensure_bytes_available(256) self.mc.MOV(eax, heap(nursery_free_adr)) self.mc.LEA_rm(edx.value, (eax.value, size)) Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_zrpy_gc.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_zrpy_gc.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_zrpy_gc.py Fri Nov 12 16:29:24 2010 @@ -550,3 +550,29 @@ def test_compile_framework_float(self): self.run('compile_framework_float') + + def define_compile_framework_minimal_size_in_nursery(self): + S = lltype.GcStruct('S') # no fields! + T = lltype.GcStruct('T', ('i', lltype.Signed)) + @unroll_safe + def f42(n, x, x0, x1, x2, x3, x4, x5, x6, x7, l, s): + lst1 = [] + lst2 = [] + i = 0 + while i < 42: + s1 = lltype.malloc(S) + t1 = lltype.malloc(T) + t1.i = 10000 + i + n + lst1.append(s1) + lst2.append(t1) + i += 1 + i = 0 + while i < 42: + check(lst2[i].i == 10000 + i + n) + i += 1 + n -= 1 + return n, x, x0, x1, x2, x3, x4, x5, x6, x7, l, s + return None, f42, None + + def test_compile_framework_minimal_size_in_nursery(self): + self.run('compile_framework_minimal_size_in_nursery') Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py Fri Nov 12 16:29:24 2010 @@ -2,6 +2,7 @@ from pypy.tool.udir import udir from pypy.rlib.jit import JitDriver, OPTIMIZER_FULL, unroll_parameters from pypy.rlib.jit import PARAMETERS, dont_look_inside +from pypy.rlib.jit import hint from pypy.jit.metainterp.jitprof import Profiler from pypy.jit.backend.detect_cpu import getcpuclass from pypy.jit.backend.test.support import CCompiledMixin @@ -9,6 +10,7 @@ from pypy.translator.translator import TranslationContext from pypy.jit.backend.x86.arch import IS_X86_32, IS_X86_64 from pypy.config.translationoption import DEFL_GC +from pypy.rlib import rgc class TestTranslationX86(CCompiledMixin): CPUClass = getcpuclass() @@ -82,12 +84,12 @@ argchain.arg(x) res = func.call(argchain, rffi.DOUBLE) i -= 1 - return res + return int(res) # def main(i, j): return f(i, j) + libffi_stuff(i, j) - expected = f(40, -49) - res = self.meta_interp(f, [40, -49]) + expected = main(40, -49) + res = self.meta_interp(main, [40, -49]) assert res == expected def test_direct_assembler_call_translates(self): Modified: pypy/branch/fast-forward/pypy/jit/codewriter/assembler.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/codewriter/assembler.py (original) +++ pypy/branch/fast-forward/pypy/jit/codewriter/assembler.py Fri Nov 12 16:29:24 2010 @@ -233,10 +233,9 @@ addr = llmemory.cast_ptr_to_adr(value) self.list_of_addr2name.append((addr, name)) - def finished(self): + def finished(self, callinfocollection): # Helper called at the end of assembling. Registers the extra # functions shown in _callinfo_for_oopspec. - from pypy.jit.codewriter.effectinfo import _callinfo_for_oopspec - for _, func in _callinfo_for_oopspec.values(): + for func in callinfocollection.all_function_addresses_as_int(): func = heaptracker.int2adr(func) self.see_raw_object(func.ptr) Modified: pypy/branch/fast-forward/pypy/jit/codewriter/call.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/codewriter/call.py (original) +++ pypy/branch/fast-forward/pypy/jit/codewriter/call.py Fri Nov 12 16:29:24 2010 @@ -7,7 +7,7 @@ from pypy.jit.codewriter.jitcode import JitCode from pypy.jit.codewriter.effectinfo import VirtualizableAnalyzer from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze -from pypy.jit.codewriter.effectinfo import EffectInfo +from pypy.jit.codewriter.effectinfo import EffectInfo, CallInfoCollection from pypy.translator.simplify import get_funcobj, get_functype from pypy.rpython.lltypesystem import lltype, llmemory from pypy.translator.backendopt.canraise import RaiseAnalyzer @@ -23,6 +23,7 @@ self.jitdrivers_sd = jitdrivers_sd self.jitcodes = {} # map {graph: jitcode} self.unfinished_graphs = [] # list of graphs with pending jitcodes + self.callinfocollection = CallInfoCollection() if hasattr(cpu, 'rtyper'): # for tests self.rtyper = cpu.rtyper translator = self.rtyper.annotator.translator Modified: pypy/branch/fast-forward/pypy/jit/codewriter/codewriter.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/codewriter/codewriter.py (original) +++ pypy/branch/fast-forward/pypy/jit/codewriter/codewriter.py Fri Nov 12 16:29:24 2010 @@ -73,7 +73,7 @@ count += 1 if not count % 500: log.info("Produced %d jitcodes" % count) - self.assembler.finished() + self.assembler.finished(self.callcontrol.callinfocollection) heaptracker.finish_registering(self.cpu) log.info("there are %d JitCode instances." % count) Modified: pypy/branch/fast-forward/pypy/jit/codewriter/effectinfo.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/codewriter/effectinfo.py (original) +++ pypy/branch/fast-forward/pypy/jit/codewriter/effectinfo.py Fri Nov 12 16:29:24 2010 @@ -144,30 +144,44 @@ # ____________________________________________________________ -_callinfo_for_oopspec = {} # {oopspecindex: (calldescr, func_as_int)} - -def callinfo_for_oopspec(oopspecindex): - """A function that returns the calldescr and the function - address (as an int) of one of the OS_XYZ functions defined above. - Don't use this if there might be several implementations of the same - OS_XYZ specialized by type, e.g. OS_ARRAYCOPY.""" - try: - return _callinfo_for_oopspec[oopspecindex] - except KeyError: - return (None, 0) - - -def _funcptr_for_oopspec_memo(oopspecindex): - from pypy.jit.codewriter import heaptracker - _, func_as_int = callinfo_for_oopspec(oopspecindex) - funcadr = heaptracker.int2adr(func_as_int) - return funcadr.ptr -_funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo' - -def funcptr_for_oopspec(oopspecindex): - """A memo function that returns a pointer to the function described - by OS_XYZ (as a real low-level function pointer).""" - funcptr = _funcptr_for_oopspec_memo(oopspecindex) - assert funcptr - return funcptr -funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(0)' +class CallInfoCollection(object): + def __init__(self): + # {oopspecindex: (calldescr, func_as_int)} + self._callinfo_for_oopspec = {} + + def _freeze_(self): + return True + + def add(self, oopspecindex, calldescr, func_as_int): + self._callinfo_for_oopspec[oopspecindex] = calldescr, func_as_int + + def has_oopspec(self, oopspecindex): + return oopspecindex in self._callinfo_for_oopspec + + def all_function_addresses_as_int(self): + return [func for (_, func) in self._callinfo_for_oopspec.values()] + + def callinfo_for_oopspec(self, oopspecindex): + """A function that returns the calldescr and the function + address (as an int) of one of the OS_XYZ functions defined above. + Don't use this if there might be several implementations of the same + OS_XYZ specialized by type, e.g. OS_ARRAYCOPY.""" + try: + return self._callinfo_for_oopspec[oopspecindex] + except KeyError: + return (None, 0) + + def _funcptr_for_oopspec_memo(self, oopspecindex): + from pypy.jit.codewriter import heaptracker + _, func_as_int = self.callinfo_for_oopspec(oopspecindex) + funcadr = heaptracker.int2adr(func_as_int) + return funcadr.ptr + _funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo' + + def funcptr_for_oopspec(self, oopspecindex): + """A memo function that returns a pointer to the function described + by OS_XYZ (as a real low-level function pointer).""" + funcptr = self._funcptr_for_oopspec_memo(oopspecindex) + assert funcptr + return funcptr + funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(1)' Modified: pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py (original) +++ pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py Fri Nov 12 16:29:24 2010 @@ -6,7 +6,7 @@ from pypy.objspace.flow.model import Block, Link, c_last_exception from pypy.jit.codewriter.flatten import ListOfKind, IndirectCallTargets from pypy.jit.codewriter import support, heaptracker -from pypy.jit.codewriter.effectinfo import EffectInfo, _callinfo_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.jit.codewriter.policy import log from pypy.jit.metainterp.typesystem import deref, arrayItem from pypy.rlib import objectmodel @@ -1084,7 +1084,8 @@ else: func = heaptracker.adr2int( llmemory.cast_ptr_to_adr(op.args[0].value)) - _callinfo_for_oopspec[oopspecindex] = calldescr, func + self.callcontrol.callinfocollection.add(oopspecindex, + calldescr, func) op1 = self.rewrite_call(op, 'residual_call', [op.args[0], calldescr], args=args) @@ -1095,7 +1096,7 @@ def _register_extra_helper(self, oopspecindex, oopspec_name, argtypes, resulttype): # a bit hackish - if oopspecindex in _callinfo_for_oopspec: + if self.callcontrol.callinfocollection.has_oopspec(oopspecindex): return c_func, TP = support.builtin_func_for_spec(self.cpu.rtyper, oopspec_name, argtypes, @@ -1109,7 +1110,7 @@ else: func = heaptracker.adr2int( llmemory.cast_ptr_to_adr(c_func.value)) - _callinfo_for_oopspec[oopspecindex] = calldescr, func + self.callcontrol.callinfocollection.add(oopspecindex, calldescr, func) def _handle_stroruni_call(self, op, oopspec_name, args): SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE) Modified: pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py (original) +++ pypy/branch/fast-forward/pypy/jit/codewriter/test/test_jtransform.py Fri Nov 12 16:29:24 2010 @@ -74,7 +74,20 @@ def calldescr_canraise(self, calldescr): return False +class FakeCallInfoCollection: + def __init__(self): + self.seen = [] + def add(self, oopspecindex, calldescr, func): + self.seen.append((oopspecindex, calldescr, func)) + def has_oopspec(self, oopspecindex): + for i, c, f in self.seen: + if i == oopspecindex: + return True + return False + class FakeBuiltinCallControl: + def __init__(self): + self.callinfocollection = FakeCallInfoCollection() def guess_call_kind(self, op): return 'builtin' def getcalldescr(self, op, oopspecindex=None): @@ -810,7 +823,8 @@ v2 = varoftype(PSTR) v3 = varoftype(PSTR) op = SpaceOperation('direct_call', [const(func), v1, v2], v3) - tr = Transformer(FakeCPU(), FakeBuiltinCallControl()) + cc = FakeBuiltinCallControl() + tr = Transformer(FakeCPU(), cc) op1 = tr.rewrite_operation(op) assert op1.opname == 'residual_call_r_r' assert op1.args[0].value == func @@ -819,9 +833,10 @@ assert op1.result == v3 # # check the callinfo_for_oopspec - got = effectinfo.callinfo_for_oopspec(effectinfo.EffectInfo.OS_UNI_CONCAT) - assert got[0] == op1.args[1] # the calldescr - assert heaptracker.int2adr(got[1]) == llmemory.cast_ptr_to_adr(func) + got = cc.callinfocollection.seen[0] + assert got[0] == effectinfo.EffectInfo.OS_UNI_CONCAT + assert got[1] == op1.args[1] # the calldescr + assert heaptracker.int2adr(got[2]) == llmemory.cast_ptr_to_adr(func) def test_str_slice(): # test that the oopspec is present and correctly transformed @@ -893,7 +908,8 @@ v2 = varoftype(PUNICODE) v3 = varoftype(lltype.Bool) op = SpaceOperation('direct_call', [const(func), v1, v2], v3) - tr = Transformer(FakeCPU(), FakeBuiltinCallControl()) + cc = FakeBuiltinCallControl() + tr = Transformer(FakeCPU(), cc) op1 = tr.rewrite_operation(op) assert op1.opname == 'residual_call_r_i' assert op1.args[0].value == func @@ -901,9 +917,9 @@ assert op1.args[2] == ListOfKind('ref', [v1, v2]) assert op1.result == v3 # test that the OS_UNIEQ_* functions are registered - cifo = effectinfo._callinfo_for_oopspec - assert effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL in cifo - assert effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR in cifo + cic = cc.callinfocollection + assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL) + assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR) def test_list_ll_arraycopy(): from pypy.rlib.rgc import ll_arraycopy Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/optimizer.py Fri Nov 12 16:29:24 2010 @@ -66,10 +66,10 @@ assert isinstance(constbox, Const) self.box = constbox self.level = LEVEL_CONSTANT - try: - val = self.box.getint() + if isinstance(constbox, ConstInt): + val = constbox.getint() self.intbound = IntBound(val, val) - except NotImplementedError: + else: self.intbound = IntUnbounded() def get_constant_class(self, cpu): Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/string.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/string.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/string.py Fri Nov 12 16:29:24 2010 @@ -9,7 +9,7 @@ from pypy.jit.metainterp.optimizeopt.optimizer import CONST_0, CONST_1 from pypy.jit.metainterp.optimizeopt.optimizer import llhelper from pypy.jit.metainterp.optimizeutil import _findall -from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.jit.codewriter import heaptracker from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.objectmodel import specialize, we_are_translated @@ -593,7 +593,8 @@ def generate_modified_call(self, oopspecindex, args, result, mode): oopspecindex += mode.OS_offset - calldescr, func = callinfo_for_oopspec(oopspecindex) + cic = self.optimizer.metainterp_sd.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(oopspecindex) op = ResOperation(rop.CALL, [ConstInt(func)] + args, result, descr=calldescr) self.optimizer.newoperations.append(op) Modified: pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py Fri Nov 12 16:29:24 2010 @@ -1261,6 +1261,7 @@ # self.jitdrivers_sd = codewriter.callcontrol.jitdrivers_sd self.virtualref_info = codewriter.callcontrol.virtualref_info + self.callinfocollection = codewriter.callcontrol.callinfocollection self.setup_jitdrivers_sd(optimizer) # # store this information for fastpath of call_assembler @@ -1624,7 +1625,7 @@ assert jitdriver_sd is self.jitdriver_sd self.create_empty_history() try: - original_boxes = self.initialize_original_boxes(jitdriver_sd,*args) + original_boxes = self.initialize_original_boxes(jitdriver_sd, *args) return self._compile_and_run_once(original_boxes) finally: self.staticdata.profiler.end_tracing() Modified: pypy/branch/fast-forward/pypy/jit/metainterp/resume.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/resume.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/resume.py Fri Nov 12 16:29:24 2010 @@ -4,8 +4,7 @@ from pypy.jit.metainterp.history import INT, REF, FLOAT, HOLE from pypy.jit.metainterp.resoperation import rop from pypy.jit.metainterp import jitprof -from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec -from pypy.jit.codewriter.effectinfo import funcptr_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.rpython.lltypesystem import lltype, llmemory, rffi, rstr from pypy.rlib import rarithmetic from pypy.rlib.objectmodel import we_are_translated, specialize @@ -774,14 +773,16 @@ strbox, ConstInt(index), charbox) def concat_strings(self, str1num, str2num): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT) str1box = self.decode_box(str1num, REF) str2box = self.decode_box(str2num, REF) return self.metainterp.execute_and_record_varargs( rop.CALL, [ConstInt(func), str1box, str2box], calldescr) def slice_string(self, strnum, startnum, lengthnum): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_SLICE) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_SLICE) strbox = self.decode_box(strnum, REF) startbox = self.decode_box(startnum, INT) lengthbox = self.decode_box(lengthnum, INT) @@ -800,14 +801,16 @@ strbox, ConstInt(index), charbox) def concat_unicodes(self, str1num, str2num): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT) str1box = self.decode_box(str1num, REF) str2box = self.decode_box(str2num, REF) return self.metainterp.execute_and_record_varargs( rop.CALL, [ConstInt(func), str1box, str2box], calldescr) def slice_unicode(self, strnum, startnum, lengthnum): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE) strbox = self.decode_box(strnum, REF) startbox = self.decode_box(startnum, INT) lengthbox = self.decode_box(lengthnum, INT) @@ -903,8 +906,8 @@ def blackhole_from_resumedata(blackholeinterpbuilder, jitdriver_sd, storage, all_virtuals=None): - resumereader = ResumeDataDirectReader(blackholeinterpbuilder.cpu, storage, - all_virtuals) + resumereader = ResumeDataDirectReader(blackholeinterpbuilder.metainterp_sd, + storage, all_virtuals) vinfo = jitdriver_sd.virtualizable_info ginfo = jitdriver_sd.greenfield_info vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info @@ -939,7 +942,7 @@ return firstbh def force_from_resumedata(metainterp_sd, storage, vinfo, ginfo): - resumereader = ResumeDataDirectReader(metainterp_sd.cpu, storage) + resumereader = ResumeDataDirectReader(metainterp_sd, storage) resumereader.handling_async_forcing() vrefinfo = metainterp_sd.virtualref_info resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo) @@ -953,8 +956,9 @@ # 1: in handle_async_forcing # 2: resuming from the GUARD_NOT_FORCED - def __init__(self, cpu, storage, all_virtuals=None): - self._init(cpu, storage) + def __init__(self, metainterp_sd, storage, all_virtuals=None): + self._init(metainterp_sd.cpu, storage) + self.callinfocollection = metainterp_sd.callinfocollection if all_virtuals is None: # common case self._prepare(storage) else: @@ -1047,7 +1051,8 @@ str2 = self.decode_ref(str2num) str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str1) str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str2) - funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT) result = funcptr(str1, str2) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1056,7 +1061,8 @@ start = self.decode_int(startnum) length = self.decode_int(lengthnum) str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str) - funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_SLICE) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_SLICE) result = funcptr(str, start, start + length) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1072,7 +1078,8 @@ str2 = self.decode_ref(str2num) str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str1) str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str2) - funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT) result = funcptr(str1, str2) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1081,7 +1088,8 @@ start = self.decode_int(startnum) length = self.decode_int(lengthnum) str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str) - funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE) result = funcptr(str, start, start + length) return lltype.cast_opaque_ptr(llmemory.GCREF, result) Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_del.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_del.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_del.py Fri Nov 12 16:29:24 2010 @@ -85,6 +85,7 @@ def test_signal_action(self): from pypy.module.signal.interp_signal import SignalActionFlag action = SignalActionFlag() + action.has_bytecode_counter = True # myjitdriver = JitDriver(greens = [], reds = ['n', 'x']) class X: @@ -92,17 +93,17 @@ # def f(n): x = X() - while n > 0: + action.reset_ticker(n) + while True: myjitdriver.can_enter_jit(n=n, x=x) myjitdriver.jit_merge_point(n=n, x=x) x.foo = n n -= 1 - if action.get() != 0: + if action.decrement_ticker(1) < 0: break - action.set(0) return 42 self.meta_interp(f, [20]) - self.check_loops(getfield_raw=1, call=0, call_pure=0) + self.check_loops(getfield_raw=1, setfield_raw=1, call=0, call_pure=0) class TestOOtype(DelTests, OOJitMixin): def setup_class(cls): Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py Fri Nov 12 16:29:24 2010 @@ -264,6 +264,8 @@ metainterp_sd = FakeMetaInterpStaticData(self.cpu) if hasattr(self, 'vrefinfo'): metainterp_sd.virtualref_info = self.vrefinfo + if hasattr(self, 'callinfocollection'): + metainterp_sd.callinfocollection = self.callinfocollection optimize_loop_1(metainterp_sd, loop) # expected = self.parse(optops) @@ -4180,22 +4182,20 @@ # ---------- def optimize_strunicode_loop_extradescrs(self, ops, spectext, optops): from pypy.jit.metainterp.optimizeopt import string - def my_callinfo_for_oopspec(oopspecindex): - calldescrtype = type(LLtypeMixin.strequaldescr) - for value in LLtypeMixin.__dict__.values(): - if isinstance(value, calldescrtype): - if (value.get_extra_info() and - value.get_extra_info().oopspecindex == oopspecindex): - # returns 0 for 'func' in this test - return value, 0 - raise AssertionError("not found: oopspecindex=%d" % oopspecindex) + class FakeCallInfoCollection: + def callinfo_for_oopspec(self, oopspecindex): + calldescrtype = type(LLtypeMixin.strequaldescr) + for value in LLtypeMixin.__dict__.values(): + if isinstance(value, calldescrtype): + extra = value.get_extra_info() + if extra and extra.oopspecindex == oopspecindex: + # returns 0 for 'func' in this test + return value, 0 + raise AssertionError("not found: oopspecindex=%d" % + oopspecindex) # - saved = string.callinfo_for_oopspec - try: - string.callinfo_for_oopspec = my_callinfo_for_oopspec - self.optimize_strunicode_loop(ops, spectext, optops) - finally: - string.callinfo_for_oopspec = saved + self.callinfocollection = FakeCallInfoCollection() + self.optimize_strunicode_loop(ops, spectext, optops) def test_str_equal_noop1(self): ops = """ Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py Fri Nov 12 16:29:24 2010 @@ -51,6 +51,7 @@ class MyMetaInterp: _already_allocated_resume_virtuals = None + callinfocollection = None def __init__(self, cpu=None): if cpu is None: @@ -156,12 +157,12 @@ storage.rd_numb = numb # cpu = MyCPU([42, gcref1, -66]) - reader = ResumeDataDirectReader(cpu, storage) + metainterp = MyMetaInterp(cpu) + reader = ResumeDataDirectReader(metainterp, storage) _next_section(reader, 42, 111, gcrefnull, 42, gcref1) _next_section(reader, 222, 333) _next_section(reader, 42, gcref1, -66) # - metainterp = MyMetaInterp(cpu) reader = ResumeDataBoxReader(storage, metainterp) bi, br, bf = [None]*3, [None]*2, [None]*0 info = MyBlackholeInterp([lltype.Signed, lltype.Signed, @@ -193,7 +194,7 @@ storage.rd_numb = numb # cpu = MyCPU([]) - reader = ResumeDataDirectReader(cpu, storage) + reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage) _next_section(reader, 100) @@ -211,7 +212,7 @@ class FakeMetainterp(object): _already_allocated_resume_virtuals = None cpu = None - reader = ResumeDataDirectReader(None, FakeStorage()) + reader = ResumeDataDirectReader(MyMetaInterp(None), FakeStorage()) assert reader.force_all_virtuals() == ["allocated", reader.virtual_default] # ____________________________________________________________ @@ -925,7 +926,7 @@ liveboxes = modifier.finish({}) assert storage.rd_snapshot is None cpu = MyCPU([]) - reader = ResumeDataDirectReader(cpu, storage) + reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage) _next_section(reader, sys.maxint, 2**16, -65) _next_section(reader, 2, 3) _next_section(reader, sys.maxint, 1, sys.maxint, 2**16) Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_virtualref.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_virtualref.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_virtualref.py Fri Nov 12 16:29:24 2010 @@ -88,7 +88,11 @@ cpu.get_latest_value_int = lambda i:guard_op.getfailargs()[i].getint() cpu.get_latest_value_ref = lambda i:guard_op.getfailargs()[i].getref_base() cpu.clear_latest_values = lambda count: None - resumereader = ResumeDataDirectReader(cpu, guard_op.getdescr()) + class FakeMetaInterpSd: + callinfocollection = None + FakeMetaInterpSd.cpu = cpu + resumereader = ResumeDataDirectReader(FakeMetaInterpSd(), + guard_op.getdescr()) vrefinfo = self.metainterp.staticdata.virtualref_info lst = [] vrefinfo.continue_tracing = lambda vref, virtual: \ Modified: pypy/branch/fast-forward/pypy/jit/tool/pypytrace-mode.el ============================================================================== --- pypy/branch/fast-forward/pypy/jit/tool/pypytrace-mode.el (original) +++ pypy/branch/fast-forward/pypy/jit/tool/pypytrace-mode.el Fri Nov 12 16:29:24 2010 @@ -26,7 +26,7 @@ ("<.*FieldDescr \\([^ ]*\\)" (1 'font-lock-variable-name-face)) ;; comment out debug_merge_point, but then highlight specific part of it ("^debug_merge_point.*" . font-lock-comment-face) - ("^\\(debug_merge_point\\).*code object\\(.*\\), file \\('.*'\\), \\(line .*\\)> \\(.*\\)')" + ("^\\(debug_merge_point\\).*code object\\(.*\\), file \\('.*'\\), \\(line .*\\)> \\(.*\\)" (1 'compilation-warning t) (2 'escape-glyph t) (3 'font-lock-string-face t) Modified: pypy/branch/fast-forward/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/api.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/api.py Fri Nov 12 16:29:24 2010 @@ -226,7 +226,7 @@ def unwrapper(space, *args): from pypy.module.cpyext.pyobject import Py_DecRef from pypy.module.cpyext.pyobject import make_ref, from_ref - from pypy.module.cpyext.pyobject import BorrowPair + from pypy.module.cpyext.pyobject import Reference newargs = () to_decref = [] assert len(args) == len(api_function.argtypes) @@ -270,8 +270,8 @@ return api_function.error_value if res is None: return None - elif isinstance(res, BorrowPair): - return res.w_borrowed + elif isinstance(res, Reference): + return res.get_wrapped(space) else: return res finally: @@ -478,7 +478,7 @@ @specialize.ll() def wrapper(*args): from pypy.module.cpyext.pyobject import make_ref, from_ref - from pypy.module.cpyext.pyobject import BorrowPair + from pypy.module.cpyext.pyobject import Reference # we hope that malloc removal removes the newtuple() that is # inserted exactly here by the varargs specializer llop.gc_stack_bottom(lltype.Void) # marker for trackgcroot.py @@ -530,7 +530,7 @@ elif is_PyObject(callable.api_func.restype): if result is None: retval = make_ref(space, None) - elif isinstance(result, BorrowPair): + elif isinstance(result, Reference): retval = result.get_ref(space) elif not rffi._isllptr(result): retval = rffi.cast(callable.api_func.restype, Modified: pypy/branch/fast-forward/pypy/module/cpyext/pyobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/pyobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/pyobject.py Fri Nov 12 16:29:24 2010 @@ -424,7 +424,18 @@ state = space.fromcache(RefcountState) return state.make_borrowed(w_container, w_borrowed) -class BorrowPair: +class Reference: + def __init__(self, pyobj): + assert not isinstance(pyobj, W_Root) + self.pyobj = pyobj + + def get_ref(self, space): + return self.pyobj + + def get_wrapped(self, space): + return from_ref(space, self.pyobj) + +class BorrowPair(Reference): """ Delays the creation of a borrowed reference. """ @@ -435,6 +446,9 @@ def get_ref(self, space): return make_borrowed_ref(space, self.w_container, self.w_borrowed) + def get_wrapped(self, space): + return self.w_borrowed + def borrow_from(container, borrowed): return BorrowPair(container, borrowed) Modified: pypy/branch/fast-forward/pypy/module/posix/interp_posix.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/posix/interp_posix.py (original) +++ pypy/branch/fast-forward/pypy/module/posix/interp_posix.py Fri Nov 12 16:29:24 2010 @@ -462,7 +462,8 @@ self.w_environ = space.newdict() if _WIN: self.cryptProviderPtr = lltype.malloc( - rffi.CArray(HCRYPTPROV), 1, zero=True, flavor='raw') + rffi.CArray(HCRYPTPROV), 1, zero=True, + flavor='raw', immortal=True) def startup(self, space): _convertenviron(space, self.w_environ) def _freeze_(self): Modified: pypy/branch/fast-forward/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/flow/objspace.py (original) +++ pypy/branch/fast-forward/pypy/objspace/flow/objspace.py Fri Nov 12 16:29:24 2010 @@ -213,6 +213,11 @@ check_class = self.unwrap(w_check_class) except UnwrapException: raise Exception, "non-constant except guard" + if check_class in (NotImplementedError, AssertionError): + # if we are in geninterp, we cannot catch these exceptions + if not self.config.translation.builtins_can_raise_exceptions: + raise error.FlowingError("Catching %s is not valid in RPython" % + check_class.__name__) if not isinstance(check_class, tuple): # the simple case return ObjSpace.exception_match(self, w_exc_type, w_check_class) Modified: pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/branch/fast-forward/pypy/objspace/flow/test/test_objspace.py Fri Nov 12 16:29:24 2010 @@ -5,7 +5,7 @@ from pypy.objspace.flow.model import flatten, mkentrymap, c_last_exception from pypy.interpreter.argument import Arguments from pypy.translator.simplify import simplify_graph -from pypy.objspace.flow.objspace import FlowObjSpace +from pypy.objspace.flow.objspace import FlowObjSpace, error from pypy.objspace.flow import objspace, flowcontext from pypy import conftest from pypy.tool.stdlib_opcode import bytecode_spec @@ -954,6 +954,22 @@ assert op.args[0] == Constant(g) + def test_cannot_catch_special_exceptions(self): + def f(): + try: + f() + except NotImplementedError: + pass + py.test.raises(error.FlowingError, "self.codetest(f)") + # + def f(): + try: + f() + except AssertionError: + pass + py.test.raises(error.FlowingError, "self.codetest(f)") + + class TestFlowObjSpaceDelay(Base): def setup_class(cls): cls.space = FlowObjSpace() @@ -1014,6 +1030,15 @@ expected.sort() assert excfound == expected + def test_can_catch_special_exceptions(self): + def f(): + try: + f() + except NotImplementedError: + pass + graph = self.codetest(f) + # assert did not crash + DATA = {'x': 5, 'y': 6} Modified: pypy/branch/fast-forward/pypy/rlib/rarithmetic.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rarithmetic.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rarithmetic.py Fri Nov 12 16:29:24 2010 @@ -614,6 +614,12 @@ def __cmp__(self, other): raise TypeError("not supported on r_singlefloat instances") + def __eq__(self, other): + return self.__class__ is other.__class__ and self._bytes == other._bytes + + def __ne__(self, other): + return not self.__eq__(other) + class r_longfloat(object): """A value of the C type 'long double'. @@ -632,6 +638,11 @@ def __cmp__(self, other): raise TypeError("not supported on r_longfloat instances") + def __eq__(self, other): + return self.__class__ is other.__class__ and self.value == other.value + + def __ne__(self, other): + return not self.__eq__(other) class For_r_singlefloat_values_Entry(extregistry.ExtRegistryEntry): Modified: pypy/branch/fast-forward/pypy/rlib/streamio.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/streamio.py (original) +++ pypy/branch/fast-forward/pypy/rlib/streamio.py Fri Nov 12 16:29:24 2010 @@ -12,7 +12,7 @@ * some other methods also have no default parameters. * close() should be called exactly once and no further operations performed; there is no __del__() closing the stream for you. - * some methods may raise NotImplementedError. + * some methods may raise MyNotImplementedError. * peek() returns some (or no) characters that have already been read ahead. * flushable() returns True/False if flushing that stream is useful/pointless. @@ -54,6 +54,12 @@ ('a', True): O_RDWR | O_CREAT, } +class MyNotImplementedError(Exception): + """ + Catching NotImplementedError is not RPython, so we use this custom class + instead of it + """ + # ____________________________________________________________ @@ -210,16 +216,16 @@ some methods.""" def read(self, n): - raise NotImplementedError + raise MyNotImplementedError def write(self, data): - raise NotImplementedError + raise MyNotImplementedError def tell(self): - raise NotImplementedError + raise MyNotImplementedError def seek(self, offset, whence): - raise NotImplementedError + raise MyNotImplementedError def readall(self): bufsize = 8192 @@ -252,7 +258,7 @@ return ''.join(result) def truncate(self, size): - raise NotImplementedError + raise MyNotImplementedError def flush_buffers(self): pass @@ -488,7 +494,7 @@ if self.lines or self.buf: try: self.do_seek(self.tell(), 0) - except NotImplementedError: + except MyNotImplementedError: pass else: self.lines = [] @@ -536,14 +542,14 @@ self.buf = "" try: self.do_seek(offset, 1) - except NotImplementedError: + except MyNotImplementedError: intoffset = offset2int(offset) self.read(intoffset) return if whence == 2: try: self.do_seek(offset, 2) - except NotImplementedError: + except MyNotImplementedError: pass else: self.lines = [] @@ -1020,7 +1026,7 @@ if self.buf: try: self.base.seek(-len(self.buf), 1) - except NotImplementedError: + except MyNotImplementedError: pass else: self.buf = "" Modified: pypy/branch/fast-forward/pypy/rlib/test/test_rarithmetic.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/test/test_rarithmetic.py (original) +++ pypy/branch/fast-forward/pypy/rlib/test/test_rarithmetic.py Fri Nov 12 16:29:24 2010 @@ -325,6 +325,15 @@ assert float(x) != 2.1 assert abs(float(x) - 2.1) < 1E-6 +def test_r_singlefloat_eq(): + x = r_singlefloat(2.5) # exact number + y = r_singlefloat(2.5) + assert x == y + assert not x != y + assert not x == 2.5 + assert x != 2.5 + py.test.raises(TypeError, "x>y") + class BaseTestRarithmetic(BaseRtypingTest): def test_formatd(self): from pypy.rlib.rarithmetic import formatd @@ -358,7 +367,17 @@ assert res == 1.0 res = self.interpret(f, [1]) - assert res == 1e-100 + assert res == 1e-100 + + def test_compare_singlefloat_crashes(self): + from pypy.rlib.rarithmetic import r_singlefloat + from pypy.rpython.error import MissingRTypeOperation + def f(x): + a = r_singlefloat(x) + b = r_singlefloat(x+1) + return a == b + py.test.raises(MissingRTypeOperation, "self.interpret(f, [42.0])") + class TestLLtype(BaseTestRarithmetic, LLRtypeMixin): pass Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/base.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gc/base.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gc/base.py Fri Nov 12 16:29:24 2010 @@ -247,6 +247,21 @@ (not self.config.taggedpointers or llmemory.cast_adr_to_int(addr) & 1 == 0)) + def enumerate_all_roots(self, callback, arg): + """For each root object, invoke callback(obj, arg). + 'callback' should not be a bound method. + Note that this method is not suitable for actually doing the + collection in a moving GC, because you cannot write back a + modified address. It is there only for inspection. + """ + # overridden in some subclasses, for GCs which have an additional + # list of last generation roots + callback2, attrname = _convert_callback_formats(callback) # :-/ + setattr(self, attrname, arg) + self.root_walker.walk_roots(callback2, callback2, callback2) + self.run_finalizers.foreach(callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' + def debug_check_consistency(self): """To use after a collection. If self.DEBUG is set, this enumerates all roots and traces all objects to check if we didn't @@ -260,8 +275,7 @@ self._debug_pending = self.AddressStack() if not we_are_translated(): self.root_walker._walk_prebuilt_gc(self._debug_record) - callback = GCBase._debug_callback - self.root_walker.walk_roots(callback, callback, callback) + self.enumerate_all_roots(GCBase._debug_callback, self) pending = self._debug_pending while pending.non_empty(): obj = pending.pop() @@ -275,9 +289,8 @@ seen.add(obj) self.debug_check_object(obj) self._debug_pending.append(obj) - def _debug_callback(self, root): - obj = root.address[0] - ll_assert(bool(obj), "NULL address from walk_roots()") + @staticmethod + def _debug_callback(obj, self): self._debug_record(obj) def _debug_callback2(self, pointer, ignored): obj = pointer.address[0] @@ -432,3 +445,17 @@ if factor != 1: return 0.0 return value + +def _convert_callback_formats(callback): + callback = getattr(callback, 'im_func', callback) + if callback not in _converted_callback_formats: + def callback2(gc, root): + obj = root.address[0] + ll_assert(bool(obj), "NULL address from walk_roots()") + callback(obj, getattr(gc, attrname)) + attrname = '_callback2_arg%d' % len(_converted_callback_formats) + _converted_callback_formats[callback] = callback2, attrname + return _converted_callback_formats[callback] + +_convert_callback_formats._annspecialcase_ = 'specialize:memo' +_converted_callback_formats = {} Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/generation.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gc/generation.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gc/generation.py Fri Nov 12 16:29:24 2010 @@ -572,16 +572,10 @@ def _compute_current_nursery_hash(self, obj): return intmask(llmemory.cast_adr_to_int(obj) + self.nursery_hash_base) - def heap_stats_walk_roots(self): - self.last_generation_root_objects.foreach( - self._track_heap_ext, None) - self.root_walker.walk_roots( - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root) - - def _track_heap_ext(self, adr, ignored): - self.trace(adr, self.track_heap_parent, adr) + def enumerate_all_roots(self, callback, arg): + self.last_generation_root_objects.foreach(callback, arg) + SemiSpaceGC.enumerate_all_roots(self, callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' def debug_check_object(self, obj): """Check the invariants about 'obj' that should be true Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gc/inspector.py Fri Nov 12 16:29:24 2010 @@ -11,18 +11,15 @@ # ---------- implementation of pypy.rlib.rgc.get_rpy_roots() ---------- -def _counting_rpy_root(gc, root): +def _counting_rpy_root(obj, gc): gc._count_rpy += 1 def _do_count_rpy_roots(gc): gc._count_rpy = 0 - gc.root_walker.walk_roots( - _counting_rpy_root, - _counting_rpy_root, - _counting_rpy_root) + gc.enumerate_all_roots(_counting_rpy_root, gc) return gc._count_rpy -def _append_rpy_root(gc, root): +def _append_rpy_root(obj, gc): # Can use the gc list, but should not allocate! # It is essential that the list is not resizable! lst = gc._list_rpy @@ -30,15 +27,12 @@ if index >= len(lst): raise ValueError gc._count_rpy = index + 1 - lst[index] = llmemory.cast_adr_to_ptr(root.address[0], llmemory.GCREF) + lst[index] = llmemory.cast_adr_to_ptr(obj, llmemory.GCREF) def _do_append_rpy_roots(gc, lst): gc._count_rpy = 0 gc._list_rpy = lst - gc.root_walker.walk_roots( - _append_rpy_root, - _append_rpy_root, - _append_rpy_root) + gc.enumerate_all_roots(_append_rpy_root, gc) gc._list_rpy = None def get_rpy_roots(gc): @@ -172,12 +166,7 @@ self.pending.append(obj) def add_roots(self): - self.gc._heap_dumper = self - self.gc.root_walker.walk_roots( - _hd_add_root, - _hd_add_root, - _hd_add_root) - self.gc._heap_dumper = None + self.gc.enumerate_all_roots(_hd_add_root, self) pendingroots = self.pending self.pending = AddressStack() self.walk(pendingroots) @@ -188,8 +177,8 @@ while pending.non_empty(): self.writeobj(pending.pop()) -def _hd_add_root(gc, root): - gc._heap_dumper.add(root.address[0]) +def _hd_add_root(obj, heap_dumper): + heap_dumper.add(obj) def dump_rpy_heap(gc, fd): heapdumper = HeapDumper(gc, fd) Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py Fri Nov 12 16:29:24 2010 @@ -77,7 +77,7 @@ # During a minor collection, the objects in the nursery that are # moved outside are changed in-place: their header is replaced with - # the value -1, and the following word is set to the address of + # the value -42, and the following word is set to the address of # where the object was moved. This means that all objects in the # nursery need to be at least 2 words long, but objects outside the # nursery don't need to. @@ -656,10 +656,13 @@ def is_forwarded(self, obj): """Returns True if the nursery obj is marked as forwarded. Implemented a bit obscurely by checking an unrelated flag - that can never be set on a young object -- except if tid == -1. + that can never be set on a young object -- except if tid == -42. """ assert self.is_in_nursery(obj) - return self.header(obj).tid & GCFLAG_FINALIZATION_ORDERING + result = (self.header(obj).tid & GCFLAG_FINALIZATION_ORDERING != 0) + if result: + ll_assert(self.header(obj).tid == -42, "bogus header for young obj") + return result def get_forwarding_address(self, obj): return llmemory.cast_adr_to_ptr(obj, FORWARDSTUBPTR).forw @@ -743,6 +746,10 @@ def JIT_max_size_of_young_obj(cls): return cls.TRANSLATION_PARAMS['large_object'] + @classmethod + def JIT_minimal_size_in_nursery(cls): + return cls.minimal_size_in_nursery + def write_barrier(self, newvalue, addr_struct): if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS: self.remember_young_pointer(addr_struct, newvalue) @@ -1080,7 +1087,7 @@ llarena.arena_reset(obj - size_gc_header, totalsize, 0) llarena.arena_reserve(obj - size_gc_header, size_gc_header + llmemory.sizeof(FORWARDSTUB)) - self.header(obj).tid = -1 + self.header(obj).tid = -42 newobj = newhdr + size_gc_header llmemory.cast_adr_to_ptr(obj, FORWARDSTUBPTR).forw = newobj # @@ -1281,6 +1288,11 @@ self.run_finalizers.foreach(self._collect_obj, self.objects_to_trace) + def enumerate_all_roots(self, callback, arg): + self.prebuilt_root_objects.foreach(callback, arg) + MovingGCBase.enumerate_all_roots(self, callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' + @staticmethod def _collect_obj(obj, objects_to_trace): objects_to_trace.append(obj) Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gc/semispace.py Fri Nov 12 16:29:24 2010 @@ -230,6 +230,10 @@ while self.max_space_size > size: self.max_space_size >>= 1 + @classmethod + def JIT_minimal_size_in_nursery(cls): + return cls.object_minimal_size + def collect(self, gen=0): self.debug_check_consistency() self.semispace_collect() @@ -689,15 +693,10 @@ self._ll_typeid_map[idx].size += llmemory.raw_malloc_usage(totsize) self.trace(adr, self.track_heap_parent, adr) - def _track_heap_root(self, root): - self.track_heap(root.address[0]) + @staticmethod + def _track_heap_root(obj, self): + self.track_heap(obj) - def heap_stats_walk_roots(self): - self.root_walker.walk_roots( - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root) - def heap_stats(self): self._tracked_dict = self.AddressDict() max_tid = self.root_walker.gcdata.max_type_id @@ -710,7 +709,7 @@ while i < max_tid: self._tracked_dict.add(llmemory.cast_ptr_to_adr(ll_typeid_map[i])) i += 1 - self.heap_stats_walk_roots() + self.enumerate_all_roots(SemiSpaceGC._track_heap_root, self) self._ll_typeid_map = lltype.nullptr(ARRAY_TYPEID_MAP) self._tracked_dict.delete() return ll_typeid_map Modified: pypy/branch/fast-forward/pypy/rpython/memory/gctransform/framework.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gctransform/framework.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gctransform/framework.py Fri Nov 12 16:29:24 2010 @@ -1210,7 +1210,7 @@ sizeofaddr = llmemory.sizeof(llmemory.Address) -class BaseRootWalker: +class BaseRootWalker(object): need_root_stack = False def __init__(self, gctransformer): Modified: pypy/branch/fast-forward/pypy/rpython/rmodel.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/rmodel.py (original) +++ pypy/branch/fast-forward/pypy/rpython/rmodel.py Fri Nov 12 16:29:24 2010 @@ -11,14 +11,14 @@ # initialization states for Repr instances -class setupstate: +class setupstate(object): NOTINITIALIZED = 0 INPROGRESS = 1 BROKEN = 2 FINISHED = 3 DELAYED = 4 -class Repr: +class Repr(object): """ An instance of Repr is associated with each instance of SomeXxx. It defines the chosen representation for the SomeXxx. The Repr subclasses generally follows the SomeXxx subclass hierarchy, but there are numerous Modified: pypy/branch/fast-forward/pypy/rpython/test/test_rpbc.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/test/test_rpbc.py (original) +++ pypy/branch/fast-forward/pypy/rpython/test/test_rpbc.py Fri Nov 12 16:29:24 2010 @@ -1547,7 +1547,7 @@ def test_always_raising_methods(self): class Base: def m(self): - raise NotImplementedError + raise KeyError class A(Base): def m(self): return 42 @@ -1560,11 +1560,11 @@ o = B() try: o.m() - except NotImplementedError: - pass + except KeyError: + assert 0 return B().m() - self.interpret_raises(NotImplementedError, f, [7]) + self.interpret_raises(KeyError, f, [7]) def test_possible_missing_attribute_access(self): py.test.skip("Should explode or give some warning") Modified: pypy/branch/fast-forward/pypy/translator/c/genc.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/genc.py (original) +++ pypy/branch/fast-forward/pypy/translator/c/genc.py Fri Nov 12 16:29:24 2010 @@ -553,7 +553,7 @@ ('debug', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT" $(TARGET)'), ('debug_exc', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DDO_LOG_EXC" $(TARGET)'), ('debug_mem', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DTRIVIAL_MALLOC_DEBUG" $(TARGET)'), - ('no_obmalloc', '', '$(MAKE) CFLAGS="-g -O1 -DRPY_ASSERT -DNO_OBMALLOC" $(TARGET)'), + ('no_obmalloc', '', '$(MAKE) CFLAGS="-g -O2 -DRPY_ASSERT -DNO_OBMALLOC" $(TARGET)'), ('linuxmemchk', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DLINUXMEMCHK" $(TARGET)'), ('llsafer', '', '$(MAKE) CFLAGS="-O2 -DRPY_LL_ASSERT" $(TARGET)'), ('lldebug', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DRPY_LL_ASSERT" $(TARGET)'), Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py (original) +++ pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py Fri Nov 12 16:29:24 2010 @@ -1066,7 +1066,9 @@ filename_dump = str(udir.join('test_dump_rpy_heap')) def define_dump_rpy_heap(self): - U = lltype.GcStruct('U', ('x', lltype.Signed)) + U = lltype.GcForwardReference() + U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)), + ('x', lltype.Signed))) S = lltype.GcStruct('S', ('u', lltype.Ptr(U))) A = lltype.GcArray(lltype.Ptr(S)) filename = self.filename_dump @@ -1074,11 +1076,16 @@ def fn(): s = lltype.malloc(S) s.u = lltype.malloc(U) + s.u.next = lltype.malloc(U) + s.u.next.next = lltype.malloc(U) a = lltype.malloc(A, 1000) s2 = lltype.malloc(S) # fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666) rgc.dump_rpy_heap(fd) + keepalive_until_here(s2) + keepalive_until_here(s) + keepalive_until_here(a) os.close(fd) return 0 @@ -1087,8 +1094,7 @@ def test_dump_rpy_heap(self): self.run("dump_rpy_heap") assert os.path.exists(self.filename_dump) - assert os.path.getsize(self.filename_dump) > 0 # minimal test - + assert os.path.getsize(self.filename_dump) > 64 class TestSemiSpaceGC(TestUsingFramework, snippet.SemiSpaceGCTestDefines): gcpolicy = "semispace" @@ -1173,21 +1179,22 @@ b = 0 c = 0 for i in range(len(tb)): - if tb[i].count == 10: + if tb[i].count == 10: # the type of S a += 1 nr = i for i in range(len(tb)): - if tb[i].count == 3: + if tb[i].count == 3: # the type GcArray(Ptr(S)) b += 1 c += tb[i].links[nr] - # we don't count b here since there can be more singletons, + # b can be 1 or 2 here since _heap_stats() is free to return or + # ignore the three GcStructs that point to the GcArray(Ptr(S)). # important one is c, a is for check return c * 100 + b * 10 + a return f def test_gc_heap_stats(self): res = self.run("gc_heap_stats") - assert res == 3011 + assert res == 3011 or res == 3021 def definestr_string_builder(cls): def fn(_): Modified: pypy/branch/fast-forward/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/branch/fast-forward/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/branch/fast-forward/pypy/translator/cli/src/pypylib.cs Fri Nov 12 16:29:24 2010 @@ -83,8 +83,12 @@ return Double.NegativeInfinity; else if (s == "nan") return Double.NaN; - else - return System.Convert.ToDouble(s); + else { + System.Globalization.NumberFormatInfo formatter; + formatter = new System.Globalization.NumberFormatInfo(); + formatter.NumberDecimalSeparator = "."; + return System.Convert.ToDouble(s, formatter); + } } } Modified: pypy/branch/fast-forward/pypy/translator/goal/app_main.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/goal/app_main.py (original) +++ pypy/branch/fast-forward/pypy/translator/goal/app_main.py Fri Nov 12 16:29:24 2010 @@ -399,10 +399,6 @@ except: print >> sys.stderr, "'import site' failed" - # update sys.path *after* loading site.py, in case there is a - # "site.py" file in the script's directory. - sys.path.insert(0, '') - pythonwarnings = os.getenv('PYTHONWARNINGS') if pythonwarnings: warnoptions.extend(pythonwarnings.split(',')) @@ -454,6 +450,13 @@ elif run_stdin: # handle the case where no command/filename/module is specified # on the command-line. + + # update sys.path *after* loading site.py, in case there is a + # "site.py" file in the script's directory. Only run this if we're + # executing the interactive prompt, if we're running a script we + # put it's directory on sys.path + sys.path.insert(0, '') + if interactive or sys.stdin.isatty(): # If stdin is a tty or if "-i" is specified, we print # a banner and run $PYTHONSTARTUP. Modified: pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py Fri Nov 12 16:29:24 2010 @@ -95,6 +95,11 @@ child.expect('>>> ') child.sendline('__name__') child.expect("'__main__'") + child.expect('>>> ') + child.sendline('import sys') + child.expect('>>> ') + child.sendline("'' in sys.path") + child.expect("True") def test_run_script(self): child = self.spawn([demo_script]) @@ -504,9 +509,10 @@ yield finally: old_cwd.chdir() + # Can't call putenv with a None argument. if old_pythonpath is not None: os.putenv('PYTHONPATH', old_pythonpath) - + tmpdir.join('site.py').write('print "SHOULD NOT RUN"') runme_py = tmpdir.join('runme.py') runme_py.write('print "some text"') @@ -527,9 +533,9 @@ with chdir_and_unset_pythonpath(tmpdir): data = self.run(cmdline2, python_flags='-S') - assert data.startswith("some new text\n") assert repr(str(tmpdir.join('otherpath'))) in data + assert "''" not in data class AppTestAppMain: From cfbolz at codespeak.net Fri Nov 12 17:48:08 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 12 Nov 2010 17:48:08 +0100 (CET) Subject: [pypy-svn] r79046 - pypy/branch/mapdict-without-jit/pypy/module/gc/test Message-ID: <20101112164808.F1433282BDC@codespeak.net> Author: cfbolz Date: Fri Nov 12 17:48:06 2010 New Revision: 79046 Modified: pypy/branch/mapdict-without-jit/pypy/module/gc/test/test_gc.py Log: Fix a test when using mapdict with local caches. Add a more precise one about the index cache. Modified: pypy/branch/mapdict-without-jit/pypy/module/gc/test/test_gc.py ============================================================================== --- pypy/branch/mapdict-without-jit/pypy/module/gc/test/test_gc.py (original) +++ pypy/branch/mapdict-without-jit/pypy/module/gc/test/test_gc.py Fri Nov 12 17:48:06 2010 @@ -117,9 +117,12 @@ pass C().f() # Fill the method cache rlist.append(weakref.ref(C)) - for i in range(5): + for i in range(10): f() gc.collect() # the classes C should all go away here + # the last class won't go in mapdict, as long as the code object of f + # is around + rlist.pop() for r in rlist: assert r() is None @@ -127,3 +130,22 @@ def setup_class(cls): cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True, "objspace.std.withmapdict": True}) + + + def test_clear_index_cache(self): + import gc, weakref + rlist = [] + def f(): + class C(object): + def f(self): + pass + c = C() + c.x = 1 + getattr(c, "x") # fill the index cache without using the local cache + getattr(c, "x") + rlist.append(weakref.ref(C)) + for i in range(5): + f() + gc.collect() # the classes C should all go away here + for r in rlist: + assert r() is None From arigo at codespeak.net Sat Nov 13 15:48:11 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 13 Nov 2010 15:48:11 +0100 (CET) Subject: [pypy-svn] r79052 - in pypy/trunk/pypy/translator/c: . test Message-ID: <20101113144811.ED5CF282B9D@codespeak.net> Author: arigo Date: Sat Nov 13 15:48:09 2010 New Revision: 79052 Modified: pypy/trunk/pypy/translator/c/node.py pypy/trunk/pypy/translator/c/test/test_lltyped.py Log: Test and fix for generating non-zero-terminated arrays of chars without length. Modified: pypy/trunk/pypy/translator/c/node.py ============================================================================== --- pypy/trunk/pypy/translator/c/node.py (original) +++ pypy/trunk/pypy/translator/c/node.py Sat Nov 13 15:48:09 2010 @@ -714,7 +714,11 @@ s = ''.join([self.obj.getitem(i) for i in range(len(self.obj.items))]) else: s = ''.join(self.obj.items) - yield '\t%s%s' % (length, c_char_array_constant(s)) + array_constant = c_char_array_constant(s) + if array_constant.startswith('{') and barebonearray(T): + assert array_constant.endswith('}') + array_constant = array_constant[1:-1].strip() + yield '\t%s%s' % (length, array_constant) yield '}' else: barebone = barebonearray(T) Modified: pypy/trunk/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/trunk/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/trunk/pypy/translator/c/test/test_lltyped.py Sat Nov 13 15:48:09 2010 @@ -1,6 +1,7 @@ import py from pypy.rpython.lltypesystem.lltype import * from pypy.translator.c.test import test_typed +from pypy.tool.sourcetools import func_with_new_name class TestLowLevelType(test_typed.CompilationTestCase): @@ -655,6 +656,45 @@ fn = self.getcompiled(llf) fn() + def test_prebuilt_raw_arrays(self): + from pypy.rpython.lltypesystem import rffi, ll2ctypes + # + def make_test_function(cast, haslength, length): + a = malloc(A, length, flavor='raw', immortal=True) + # two cases: a zero-terminated array if length == 6 or 1030, + # a non-zero-terminated array if length == 557 or 1031 + for i in range(length): + a[i] = cast(256 - 5 + i) + def llf(): + for i in range(length): + if a[i] != cast(256 - 5 + i): + return False + if haslength and len(a) != length: + return False + return True + return func_with_new_name(llf, repr((A, haslength, length))) + # + testfns = [] + records = [] + for OF, cast in [(Void, lambda n: None), + (Char, lambda n: chr(n & 0xFF)), + (Signed, lambda n: n)]: + for A, haslength in [(rffi.CArray(OF), False), + (Array(OF), True)]: + for length in [0, 6, 557, 1030, 1031]: + testfns.append(make_test_function(cast, haslength, length)) + records.append((A, haslength, length)) + def llf(): + i = 0 + for fn in testfns: + if not fn(): + return i # returns the index of the failing function + i += 1 + return -42 + fn = self.getcompiled(llf) + res = fn() + assert res == -42, "failing function: %r" % (records[res],) + def test_prebuilt_ll2ctypes_array(self): from pypy.rpython.lltypesystem import rffi, ll2ctypes A = rffi.CArray(Char) From cfbolz at codespeak.net Sat Nov 13 16:18:28 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Sat, 13 Nov 2010 16:18:28 +0100 (CET) Subject: [pypy-svn] r79053 - in pypy/trunk/pypy: module/__pypy__ module/gc module/gc/test objspace/std objspace/std/test Message-ID: <20101113151828.64AE9282BD6@codespeak.net> Author: cfbolz Date: Sat Nov 13 16:18:26 2010 New Revision: 79053 Modified: pypy/trunk/pypy/module/__pypy__/__init__.py pypy/trunk/pypy/module/__pypy__/interp_magic.py pypy/trunk/pypy/module/gc/interp_gc.py pypy/trunk/pypy/module/gc/test/test_gc.py pypy/trunk/pypy/objspace/std/callmethod.py pypy/trunk/pypy/objspace/std/mapdict.py pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py pypy/trunk/pypy/objspace/std/test/test_mapdict.py Log: svn merge -r78362:HEAD http://codespeak.net/svn/pypy/branch/mapdict-without-jit (arigo, cfbolz): merge mapdict-without-jit branch try to make it more practical to use mapdicts without the JIT for that purpose two new optimizations are introduced: - a cache that is used when looking up an index in a map, either for reading or for writing an attribute - integrating the local map cache with the LOOKUP_METHOD bytecode particularly the latter gives nice speedups. Modified: pypy/trunk/pypy/module/__pypy__/__init__.py ============================================================================== --- pypy/trunk/pypy/module/__pypy__/__init__.py (original) +++ pypy/trunk/pypy/module/__pypy__/__init__.py Sat Nov 13 16:18:26 2010 @@ -23,6 +23,9 @@ 'interp_magic.method_cache_counter') self.extra_interpdef('reset_method_cache_counter', 'interp_magic.reset_method_cache_counter') + if self.space.config.objspace.std.withmapdict: + self.extra_interpdef('mapdict_cache_counter', + 'interp_magic.mapdict_cache_counter') PYC_MAGIC = get_pyc_magic(self.space) self.extra_interpdef('PYC_MAGIC', 'space.wrap(%d)' % PYC_MAGIC) # Modified: pypy/trunk/pypy/module/__pypy__/interp_magic.py ============================================================================== --- pypy/trunk/pypy/module/__pypy__/interp_magic.py (original) +++ pypy/trunk/pypy/module/__pypy__/interp_magic.py Sat Nov 13 16:18:26 2010 @@ -2,6 +2,7 @@ from pypy.interpreter.gateway import ObjSpace from pypy.rlib.objectmodel import we_are_translated from pypy.objspace.std.typeobject import MethodCache +from pypy.objspace.std.mapdict import IndexCache def internal_repr(space, w_object): return space.wrap('%r' % (w_object,)) @@ -36,4 +37,17 @@ cache = space.fromcache(MethodCache) cache.misses = {} cache.hits = {} - + if space.config.objspace.std.withmapdict: + cache = space.fromcache(IndexCache) + cache.misses = {} + cache.hits = {} + +def mapdict_cache_counter(space, name): + """Return a tuple (index_cache_hits, index_cache_misses) for lookups + in the mapdict cache with the given attribute name.""" + assert space.config.objspace.std.withmethodcachecounter + assert space.config.objspace.std.withmapdict + cache = space.fromcache(IndexCache) + return space.newtuple([space.newint(cache.hits.get(name, 0)), + space.newint(cache.misses.get(name, 0))]) +mapdict_cache_counter.unwrap_spec = [ObjSpace, str] Modified: pypy/trunk/pypy/module/gc/interp_gc.py ============================================================================== --- pypy/trunk/pypy/module/gc/interp_gc.py (original) +++ pypy/trunk/pypy/module/gc/interp_gc.py Sat Nov 13 16:18:26 2010 @@ -10,6 +10,10 @@ from pypy.objspace.std.typeobject import MethodCache cache = space.fromcache(MethodCache) cache.clear() + if space.config.objspace.std.withmapdict: + from pypy.objspace.std.mapdict import IndexCache + cache = space.fromcache(IndexCache) + cache.clear() rgc.collect() return space.wrap(0) Modified: pypy/trunk/pypy/module/gc/test/test_gc.py ============================================================================== --- pypy/trunk/pypy/module/gc/test/test_gc.py (original) +++ pypy/trunk/pypy/module/gc/test/test_gc.py Sat Nov 13 16:18:26 2010 @@ -117,6 +117,33 @@ pass C().f() # Fill the method cache rlist.append(weakref.ref(C)) + for i in range(10): + f() + gc.collect() # the classes C should all go away here + # the last class won't go in mapdict, as long as the code object of f + # is around + rlist.pop() + for r in rlist: + assert r() is None + +class AppTestGcMapDictIndexCache(AppTestGcMethodCache): + def setup_class(cls): + cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True, + "objspace.std.withmapdict": True}) + + + def test_clear_index_cache(self): + import gc, weakref + rlist = [] + def f(): + class C(object): + def f(self): + pass + c = C() + c.x = 1 + getattr(c, "x") # fill the index cache without using the local cache + getattr(c, "x") + rlist.append(weakref.ref(C)) for i in range(5): f() gc.collect() # the classes C should all go away here Modified: pypy/trunk/pypy/objspace/std/callmethod.py ============================================================================== --- pypy/trunk/pypy/objspace/std/callmethod.py (original) +++ pypy/trunk/pypy/objspace/std/callmethod.py Sat Nov 13 16:18:26 2010 @@ -13,6 +13,9 @@ from pypy.interpreter import function from pypy.objspace.descroperation import object_getattribute from pypy.rlib import jit, rstack # for resume points +from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict, \ + LOOKUP_METHOD_mapdict_fill_cache_method + # This module exports two extra methods for StdObjSpaceFrame implementing # the LOOKUP_METHOD and CALL_METHOD opcodes in an efficient way, as well @@ -30,6 +33,13 @@ # space = f.space w_obj = f.popvalue() + + if space.config.objspace.std.withmapdict and not jit.we_are_jitted(): + # mapdict has an extra-fast version of this function + from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict + if LOOKUP_METHOD_mapdict(f, nameindex, w_obj): + return + w_name = f.getname_w(nameindex) w_value = None @@ -50,6 +60,11 @@ # nothing in the instance f.pushvalue(w_descr) f.pushvalue(w_obj) + if (space.config.objspace.std.withmapdict and + not jit.we_are_jitted()): + # let mapdict cache stuff + LOOKUP_METHOD_mapdict_fill_cache_method( + f.getcode(), nameindex, w_obj, w_type, w_descr) return if w_value is None: w_value = space.getattr(w_obj, w_name) Modified: pypy/trunk/pypy/objspace/std/mapdict.py ============================================================================== --- pypy/trunk/pypy/objspace/std/mapdict.py (original) +++ pypy/trunk/pypy/objspace/std/mapdict.py Sat Nov 13 16:18:26 2010 @@ -1,4 +1,5 @@ from pypy.rlib import jit, objectmodel, debug +from pypy.rlib.rarithmetic import intmask, r_uint from pypy.interpreter.baseobjspace import W_Root from pypy.objspace.std.dictmultiobject import W_DictMultiObject @@ -15,24 +16,70 @@ # we want to propagate knowledge that the result cannot be negative class AbstractAttribute(object): - _immutable_fields_ = ['w_cls'] + _immutable_fields_ = ['terminator'] cache_attrs = None _size_estimate = 0 - def __init__(self, space, w_cls): + def __init__(self, space, terminator): self.space = space - self.w_cls = w_cls + assert isinstance(terminator, Terminator) + self.terminator = terminator def read(self, obj, selector): - raise NotImplementedError("abstract base class") + index = self.index(selector) + if index < 0: + return self.terminator._read_terminator(obj, selector) + return obj._mapdict_read_storage(index) def write(self, obj, selector, w_value): - raise NotImplementedError("abstract base class") + index = self.index(selector) + if index < 0: + return self.terminator._write_terminator(obj, selector, w_value) + obj._mapdict_write_storage(index, w_value) + return True def delete(self, obj, selector): return None def index(self, selector): + if (self.space.config.objspace.std.withmethodcache and + not jit.we_are_jitted()): + return self._index_cache(selector) + else: + return self._index(selector) + + @jit.dont_look_inside + def _index_cache(self, selector): + space = self.space + cache = space.fromcache(IndexCache) + SHIFT2 = r_uint.BITS - space.config.objspace.std.methodcachesizeexp + SHIFT1 = SHIFT2 - 5 + attrs_as_int = objectmodel.current_object_addr_as_int(self) + # ^^^Note: see comment in typeobject.py for + # _pure_lookup_where_with_method_cache() + hash_selector = objectmodel.compute_hash(selector) + product = intmask(attrs_as_int * hash_selector) + index_hash = (r_uint(product) ^ (r_uint(product) << SHIFT1)) >> SHIFT2 + # ^^^Note2: same comment too + cached_attr = cache.attrs[index_hash] + if cached_attr is self: + cached_selector = cache.selectors[index_hash] + if cached_selector == selector: + index = cache.indices[index_hash] + if space.config.objspace.std.withmethodcachecounter: + name = selector[0] + cache.hits[name] = cache.hits.get(name, 0) + 1 + return index + index = self._index(selector) + cache.attrs[index_hash] = self + cache.selectors[index_hash] = selector + cache.indices[index_hash] = index + if space.config.objspace.std.withmethodcachecounter: + name = selector[0] + cache.misses[name] = cache.misses.get(name, 0) + 1 + return index + + def _index(self, selector): return -1 def copy(self, obj): @@ -42,7 +89,7 @@ raise NotImplementedError("abstract base class") def get_terminator(self): - raise NotImplementedError("abstract base class") + return self.terminator def set_terminator(self, obj, terminator): raise NotImplementedError("abstract base class") @@ -95,15 +142,20 @@ raise NotImplementedError("abstract base class") def __repr__(self): - return "<%s w_cls=%s>" % (self.__class__.__name__, self.w_cls) + return "<%s>" % (self.__class__.__name__,) class Terminator(AbstractAttribute): + _immutable_fields_ = ['w_cls'] - def read(self, obj, selector): + def __init__(self, space, w_cls): + AbstractAttribute.__init__(self, space, self) + self.w_cls = w_cls + + def _read_terminator(self, obj, selector): return None - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): obj._get_mapdict_map().add_attr(obj, selector, w_value) return True @@ -116,9 +168,6 @@ def length(self): return 0 - def get_terminator(self): - return self - def set_terminator(self, obj, terminator): result = Object() result.space = self.space @@ -128,6 +177,9 @@ def remove_dict_entries(self, obj): return self.copy(obj) + def __repr__(self): + return "<%s w_cls=%s>" % (self.__class__.__name__, self.w_cls) + class DictTerminator(Terminator): _immutable_fields_ = ['devolved_dict_terminator'] def __init__(self, space, w_cls): @@ -142,27 +194,27 @@ class NoDictTerminator(Terminator): - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): if selector[1] == DICT: return False - return Terminator.write(self, obj, selector, w_value) + return Terminator._write_terminator(self, obj, selector, w_value) class DevolvedDictTerminator(Terminator): - def read(self, obj, selector): + def _read_terminator(self, obj, selector): if selector[1] == DICT: w_dict = obj.getdict() space = self.space return space.finditem_str(w_dict, selector[0]) - return Terminator.read(self, obj, selector) + return Terminator._read_terminator(self, obj, selector) - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): if selector[1] == DICT: w_dict = obj.getdict() space = self.space space.setitem_str(w_dict, selector[0], w_value) return True - return Terminator.write(self, obj, selector, w_value) + return Terminator._write_terminator(self, obj, selector, w_value) def delete(self, obj, selector): from pypy.interpreter.error import OperationError @@ -189,7 +241,7 @@ class PlainAttribute(AbstractAttribute): _immutable_fields_ = ['selector', 'position', 'back'] def __init__(self, selector, back): - AbstractAttribute.__init__(self, back.space, back.w_cls) + AbstractAttribute.__init__(self, back.space, back.terminator) self.selector = selector self.position = back.length() self.back = back @@ -199,17 +251,6 @@ w_value = self.read(obj, self.selector) new_obj._get_mapdict_map().add_attr(new_obj, self.selector, w_value) - def read(self, obj, selector): - if selector == self.selector: - return obj._mapdict_read_storage(self.position) - return self.back.read(obj, selector) - - def write(self, obj, selector, w_value): - if selector == self.selector: - obj._mapdict_write_storage(self.position, w_value) - return True - return self.back.write(obj, selector, w_value) - def delete(self, obj, selector): if selector == self.selector: # ok, attribute is deleted @@ -219,10 +260,10 @@ self._copy_attr(obj, new_obj) return new_obj - def index(self, selector): + def _index(self, selector): if selector == self.selector: return self.position - return self.back.index(selector) + return self.back._index(selector) def copy(self, obj): new_obj = self.back.copy(obj) @@ -232,9 +273,6 @@ def length(self): return self.position + 1 - def get_terminator(self): - return self.back.get_terminator() - def set_terminator(self, obj, terminator): new_obj = self.back.set_terminator(obj, terminator) self._copy_attr(obj, new_obj) @@ -268,6 +306,24 @@ # RPython reasons w_obj._set_mapdict_storage_and_map(new_obj.storage, new_obj.map) +class IndexCache(object): + def __init__(self, space): + assert space.config.objspace.std.withmethodcache + SIZE = 1 << space.config.objspace.std.methodcachesizeexp + self.attrs = [None] * SIZE + self._empty_selector = (None, INVALID) + self.selectors = [self._empty_selector] * SIZE + self.indices = [0] * SIZE + if space.config.objspace.std.withmethodcachecounter: + self.hits = {} + self.misses = {} + + def clear(self): + for i in range(len(self.attrs)): + self.attrs[i] = None + for i in range(len(self.selectors)): + self.selectors[i] = self._empty_selector + # ____________________________________________________________ # object implementation @@ -328,7 +384,7 @@ assert flag def getclass(self, space): - return self._get_mapdict_map().w_cls + return self._get_mapdict_map().terminator.w_cls def setclass(self, space, w_cls): new_obj = self._get_mapdict_map().set_terminator(self, w_cls.terminator) @@ -373,6 +429,7 @@ self.storage = make_sure_not_resized([None] * map.size_estimate()) def _mapdict_read_storage(self, index): + assert index >= 0 return self.storage[index] def _mapdict_write_storage(self, index, value): self.storage[index] = value @@ -440,6 +497,7 @@ return rerased.unerase_fixedsizelist(erased, W_Root) def _mapdict_read_storage(self, index): + assert index >= 0 for i in rangenmin1: if index == i: erased = getattr(self, "_value%s" % i) @@ -604,30 +662,54 @@ map = None version_tag = None index = 0 + w_method = None # for callmethod success_counter = 0 failure_counter = 0 + def is_valid_for_obj(self, w_obj): + map = w_obj._get_mapdict_map() + return self.is_valid_for_map(map) + + def is_valid_for_map(self, map): + if map is self.map: + version_tag = map.terminator.w_cls.version_tag() + if version_tag is self.version_tag: + # everything matches, it's incredibly fast + if map.space.config.objspace.std.withmethodcachecounter: + self.success_counter += 1 + return True + return False + INVALID_CACHE_ENTRY = CacheEntry() INVALID_CACHE_ENTRY.map = objectmodel.instantiate(AbstractAttribute) # different from any real map ^^^ -INVALID_CACHE_ENTRY.map.w_cls = None +INVALID_CACHE_ENTRY.map.terminator = None + def init_mapdict_cache(pycode): num_entries = len(pycode.co_names_w) pycode._mapdict_caches = [INVALID_CACHE_ENTRY] * num_entries +def _fill_cache(pycode, nameindex, map, version_tag, index, w_method=None): + entry = pycode._mapdict_caches[nameindex] + if entry is INVALID_CACHE_ENTRY: + entry = CacheEntry() + pycode._mapdict_caches[nameindex] = entry + entry.map = map + entry.version_tag = version_tag + entry.index = index + entry.w_method = w_method + if pycode.space.config.objspace.std.withmethodcachecounter: + entry.failure_counter += 1 + def LOAD_ATTR_caching(pycode, w_obj, nameindex): # this whole mess is to make the interpreter quite a bit faster; it's not # used if we_are_jitted(). entry = pycode._mapdict_caches[nameindex] map = w_obj._get_mapdict_map() - if map is entry.map: - version_tag = map.w_cls.version_tag() - if version_tag is entry.version_tag: - # everything matches, it's incredibly fast - if pycode.space.config.objspace.std.withmethodcachecounter: - entry.success_counter += 1 - return w_obj._mapdict_read_storage(entry.index) + if entry.is_valid_for_map(map) and entry.w_method is None: + # everything matches, it's incredibly fast + return w_obj._mapdict_read_storage(entry.index) return LOAD_ATTR_slowpath(pycode, w_obj, nameindex, map) LOAD_ATTR_caching._always_inline_ = True @@ -635,7 +717,7 @@ space = pycode.space w_name = pycode.co_names_w[nameindex] if map is not None: - w_type = map.w_cls + w_type = map.terminator.w_cls w_descr = w_type.getattribute_if_not_from_object() if w_descr is not None: return space._handle_getattribute(w_descr, w_obj, w_name) @@ -655,17 +737,30 @@ if selector[1] != INVALID: index = map.index(selector) if index >= 0: - entry = pycode._mapdict_caches[nameindex] - if entry is INVALID_CACHE_ENTRY: - entry = CacheEntry() - pycode._mapdict_caches[nameindex] = entry - entry.map = map - entry.version_tag = version_tag - entry.index = index - if space.config.objspace.std.withmethodcachecounter: - entry.failure_counter += 1 + _fill_cache(pycode, nameindex, map, version_tag, index) return w_obj._mapdict_read_storage(index) if space.config.objspace.std.withmethodcachecounter: INVALID_CACHE_ENTRY.failure_counter += 1 return space.getattr(w_obj, w_name) LOAD_ATTR_slowpath._dont_inline_ = True + +def LOOKUP_METHOD_mapdict(f, nameindex, w_obj): + space = f.space + pycode = f.getcode() + entry = pycode._mapdict_caches[nameindex] + if entry.is_valid_for_obj(w_obj): + w_method = entry.w_method + if w_method is not None: + f.pushvalue(w_method) + f.pushvalue(w_obj) + return True + return False + +def LOOKUP_METHOD_mapdict_fill_cache_method(pycode, nameindex, w_obj, w_type, w_method): + version_tag = w_type.version_tag() + if version_tag is None: + return + map = w_obj._get_mapdict_map() + if map is None: + return + _fill_cache(pycode, nameindex, map, version_tag, -1, w_method) Modified: pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py ============================================================================== --- pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py (original) +++ pypy/trunk/pypy/objspace/std/test/test_dictmultiobject.py Sat Nov 13 16:18:26 2010 @@ -616,6 +616,7 @@ withdictmeasurement = False withsmalldicts = False withcelldict = False + withmethodcache = False class opcodes: CALL_LIKELY_BUILTIN = False Modified: pypy/trunk/pypy/objspace/std/test/test_mapdict.py ============================================================================== --- pypy/trunk/pypy/objspace/std/test/test_mapdict.py (original) +++ pypy/trunk/pypy/objspace/std/test/test_mapdict.py Sat Nov 13 16:18:26 2010 @@ -24,13 +24,13 @@ hasdict = False def test_plain_attribute(): - space = " " w_cls = "class" aa = PlainAttribute(("b", DICT), PlainAttribute(("a", DICT), Terminator(space, w_cls))) assert aa.space is space - assert aa.w_cls is w_cls + assert aa.terminator.w_cls is w_cls + assert aa.get_terminator() is aa.terminator obj = Object() obj.map, obj.storage = aa, [10, 20] @@ -604,7 +604,8 @@ from pypy.interpreter import gateway cls.space = gettestobjspace( **{"objspace.std.withmapdict": True, - "objspace.std.withmethodcachecounter": True}) + "objspace.std.withmethodcachecounter": True, + "objspace.opcodes.CALL_METHOD": True}) # def check(space, w_func, name): w_code = space.getattr(w_func, space.wrap('func_code')) @@ -785,6 +786,135 @@ res = self.check(f, 'x') assert res == (0, 0, 1) + def test_call_method_uses_cache(self): + # bit sucky + global C + + class C(object): + def m(*args): + return args + C.sm = staticmethod(C.m.im_func) + C.cm = classmethod(C.m.im_func) + + exec """if 1: + + def f(): + c = C() + res = c.m(1) + assert res == (c, 1) + return 42 + + def g(): + c = C() + res = c.sm(1) + assert res == (1, ) + return 42 + + def h(): + c = C() + res = c.cm(1) + assert res == (C, 1) + return 42 + """ + res = self.check(f, 'm') + assert res == (1, 0, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + + # static methods are not cached + res = self.check(g, 'sm') + assert res == (0, 0, 0) + res = self.check(g, 'sm') + assert res == (0, 0, 0) + + # neither are class methods + res = self.check(h, 'cm') + assert res == (0, 0, 0) + res = self.check(h, 'cm') + assert res == (0, 0, 0) + + def test_mix_cache_bug(self): + # bit sucky + global C + + class C(object): + def m(*args): + return args + + exec """if 1: + + def f(): + c = C() + res = c.m(1) + assert res == (c, 1) + bm = c.m + res = bm(1) + assert res == (c, 1) + return 42 + + """ + res = self.check(f, 'm') + assert res == (1, 1, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + +class AppTestGlobalCaching(AppTestWithMapDict): + def setup_class(cls): + cls.space = gettestobjspace( + **{"objspace.std.withmethodcachecounter": True, + "objspace.std.withmapdict": True, + "objspace.opcodes.CALL_METHOD": True}) + + def test_mix_classes(self): + import __pypy__ + class A(object): + def f(self): + return 42 + class B(object): + def f(self): + return 43 + class C(object): + def f(self): + return 44 + l = [A(), B(), C()] * 10 + __pypy__.reset_method_cache_counter() + # 'exec' to make sure that a.f() is compiled with CALL_METHOD + exec """for i, a in enumerate(l): + assert a.f() == 42 + i % 3 +""" + cache_counter = __pypy__.mapdict_cache_counter("f") + assert cache_counter[0] >= 15 + assert cache_counter[1] >= 3 # should be (27, 3) + assert sum(cache_counter) == 30 + + def test_mix_classes_attribute(self): + import __pypy__ + class A(object): + def __init__(self): + self.x = 42 + class B(object): + def __init__(self): + self.x = 43 + class C(object): + def __init__(self): + self.x = 44 + l = [A(), B(), C()] * 10 + __pypy__.reset_method_cache_counter() + for i, a in enumerate(l): + assert a.x == 42 + i % 3 + cache_counter = __pypy__.mapdict_cache_counter("x") + assert cache_counter[0] >= 15 + assert cache_counter[1] >= 3 # should be (27, 3) + assert sum(cache_counter) == 30 + class TestDictSubclassShortcutBug(object): def setup_class(cls): cls.space = gettestobjspace( From afa at codespeak.net Sat Nov 13 16:34:21 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sat, 13 Nov 2010 16:34:21 +0100 (CET) Subject: [pypy-svn] r79054 - pypy/branch/fast-forward/pypy/rlib Message-ID: <20101113153421.47922282B9D@codespeak.net> Author: afa Date: Sat Nov 13 16:34:19 2010 New Revision: 79054 Modified: pypy/branch/fast-forward/pypy/rlib/clibffi.py Log: translate.py *segfaulted* in node.py where trying to render the CDLL.__del__ function. This seems to be related to the VOIDP prebuilt constant, avoid it. Modified: pypy/branch/fast-forward/pypy/rlib/clibffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/clibffi.py (original) +++ pypy/branch/fast-forward/pypy/rlib/clibffi.py Sat Nov 13 16:34:19 2010 @@ -583,11 +583,9 @@ return dlsym(self.lib, name) class CDLL(RawCDLL): - _default = rffi.cast(DLLHANDLE, -1) - def __init__(self, libname): """Load the library, or raises DLOpenError.""" - RawCDLL.__init__(self, self._default) + RawCDLL.__init__(self, rffi.cast(DLLHANDLE, -1)) ll_libname = rffi.str2charp(libname) try: self.lib = dlopen(ll_libname) @@ -595,7 +593,7 @@ lltype.free(ll_libname, flavor='raw') def __del__(self): - if self.lib != self._default: + if self.lib != rffi.cast(DLLHANDLE, -1): dlclose(self.lib) - self.lib = self._default + self.lib = rffi.cast(DLLHANDLE, -1) From arigo at codespeak.net Sat Nov 13 17:07:03 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 13 Nov 2010 17:07:03 +0100 (CET) Subject: [pypy-svn] r79055 - in pypy/trunk/pypy: module/gc rlib rpython rpython/lltypesystem rpython/memory/gc rpython/memory/gctransform translator/c/test Message-ID: <20101113160703.223E6282BDC@codespeak.net> Author: arigo Date: Sat Nov 13 17:07:01 2010 New Revision: 79055 Modified: pypy/trunk/pypy/module/gc/__init__.py pypy/trunk/pypy/module/gc/app_referents.py pypy/trunk/pypy/module/gc/referents.py pypy/trunk/pypy/rlib/rgc.py pypy/trunk/pypy/rpython/llinterp.py pypy/trunk/pypy/rpython/lltypesystem/lloperation.py pypy/trunk/pypy/rpython/memory/gc/inspector.py pypy/trunk/pypy/rpython/memory/gc/minimark.py pypy/trunk/pypy/rpython/memory/gctransform/framework.py pypy/trunk/pypy/translator/c/test/test_newgc.py Log: Implement a convenience: typeids.txt is compressed and stored in the pypy-c itself. It is available via gc.get_typeids_z(), and automatically extracted to 'typeids.txt' by gc.dump_rpy_heap(). Modified: pypy/trunk/pypy/module/gc/__init__.py ============================================================================== --- pypy/trunk/pypy/module/gc/__init__.py (original) +++ pypy/trunk/pypy/module/gc/__init__.py Sat Nov 13 17:07:01 2010 @@ -29,6 +29,7 @@ 'get_referents': 'referents.get_referents', 'get_referrers': 'referents.get_referrers', '_dump_rpy_heap': 'referents._dump_rpy_heap', + 'get_typeids_z': 'referents.get_typeids_z', 'GcRef': 'referents.W_GcRef', }) MixedModule.__init__(self, space, w_name) Modified: pypy/trunk/pypy/module/gc/app_referents.py ============================================================================== --- pypy/trunk/pypy/module/gc/app_referents.py (original) +++ pypy/trunk/pypy/module/gc/app_referents.py Sat Nov 13 17:07:01 2010 @@ -14,11 +14,25 @@ and [addr1]..[addrn] are addresses of other objects that this object points to. The full dump is a list of such objects, with a marker [0][0][0][-1] inserted after all GC roots, before all non-roots. + + If the argument is a filename and the 'zlib' module is available, + we also write a 'typeids.txt' in the same directory, if none exists. """ if isinstance(file, str): f = open(file, 'wb') gc._dump_rpy_heap(f.fileno()) f.close() + try: + import zlib, os + except ImportError: + pass + else: + filename2 = os.path.join(os.path.dirname(file), 'typeids.txt') + if not os.path.exists(filename2): + data = zlib.decompress(gc.get_typeids_z()) + f = open(filename2, 'wb') + f.write(data) + f.close() else: if isinstance(file, int): fd = file Modified: pypy/trunk/pypy/module/gc/referents.py ============================================================================== --- pypy/trunk/pypy/module/gc/referents.py (original) +++ pypy/trunk/pypy/module/gc/referents.py Sat Nov 13 17:07:01 2010 @@ -177,3 +177,9 @@ if not ok: raise missing_operation(space) _dump_rpy_heap.unwrap_spec = [ObjSpace, int] + +def get_typeids_z(space): + a = rgc.get_typeids_z() + s = ''.join([a[i] for i in range(len(a))]) + return space.wrap(s) +get_typeids_z.unwrap_spec = [ObjSpace] Modified: pypy/trunk/pypy/rlib/rgc.py ============================================================================== --- pypy/trunk/pypy/rlib/rgc.py (original) +++ pypy/trunk/pypy/rlib/rgc.py Sat Nov 13 17:07:01 2010 @@ -379,6 +379,11 @@ "NOT_RPYTHON" raise NotImplementedError +def get_typeids_z(): + "NOT_RPYTHON" + raise NotImplementedError + +ARRAY_OF_CHAR = lltype.Array(lltype.Char) NULL_GCREF = lltype.nullptr(llmemory.GCREF.TO) class _GcRef(object): @@ -530,3 +535,12 @@ vlist = hop.inputargs(lltype.Signed) hop.exception_is_here() return hop.genop('gc_dump_rpy_heap', vlist, resulttype = hop.r_result) + +class Entry(ExtRegistryEntry): + _about_ = get_typeids_z + def compute_result_annotation(self): + from pypy.annotation.model import SomePtr + return SomePtr(lltype.Ptr(ARRAY_OF_CHAR)) + def specialize_call(self, hop): + hop.exception_is_here() + return hop.genop('gc_typeids_z', [], resulttype = hop.r_result) Modified: pypy/trunk/pypy/rpython/llinterp.py ============================================================================== --- pypy/trunk/pypy/rpython/llinterp.py (original) +++ pypy/trunk/pypy/rpython/llinterp.py Sat Nov 13 17:07:01 2010 @@ -912,6 +912,9 @@ def op_gc_dump_rpy_heap(self): raise NotImplementedError("gc_dump_rpy_heap") + def op_gc_typeids_z(self): + raise NotImplementedError("gc_typeids_z") + def op_do_malloc_fixedsize_clear(self): raise NotImplementedError("do_malloc_fixedsize_clear") Modified: pypy/trunk/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/trunk/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/trunk/pypy/rpython/lltypesystem/lloperation.py Sat Nov 13 17:07:01 2010 @@ -476,6 +476,7 @@ 'gc_get_rpy_type_index': LLOp(), 'gc_is_rpy_instance' : LLOp(), 'gc_dump_rpy_heap' : LLOp(), + 'gc_typeids_z' : LLOp(), # ------- JIT & GC interaction, only for some GCs ---------- Modified: pypy/trunk/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/inspector.py Sat Nov 13 17:07:01 2010 @@ -4,7 +4,7 @@ from pypy.rpython.lltypesystem import lltype, llmemory, rffi from pypy.rlib.objectmodel import free_non_gc_object from pypy.rpython.module.ll_os import underscore_on_windows -from pypy.rlib import rposix +from pypy.rlib import rposix, rgc from pypy.rpython.memory.support import AddressDict, get_address_stack @@ -187,3 +187,7 @@ heapdumper.flush() heapdumper.delete() return True + +def get_typeids_z(gc): + srcaddress = gc.root_walker.gcdata.typeids_z + return llmemory.cast_adr_to_ptr(srcaddress, lltype.Ptr(rgc.ARRAY_OF_CHAR)) Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Sat Nov 13 17:07:01 2010 @@ -1016,7 +1016,7 @@ obj = oldlist.pop() # # Add the flag GCFLAG_NO_YOUNG_PTRS. All live objects should have - # this flag after a nursery collection. + # this flag set after a nursery collection. self.header(obj).tid |= GCFLAG_NO_YOUNG_PTRS # # Trace the 'obj' to replace pointers to nursery with pointers @@ -1079,7 +1079,7 @@ # nursery are kept unchanged in this step. llmemory.raw_memcopy(obj - size_gc_header, newhdr, totalsize) # - # Set the old object's tid to -1 (containing all flags) and + # Set the old object's tid to -42 (containing all flags) and # replace the old object's content with the target address. # A bit of no-ops to convince llarena that we are changing # the layout, in non-translated versions. @@ -1344,7 +1344,7 @@ if self.is_valid_gc_object(obj): if self.is_in_nursery(obj): # - # The object not a tagged pointer, and is it still in the + # The object is not a tagged pointer, and it is still in the # nursery. Find or allocate a "shadow" object, which is # where the object will be moved by the next minor # collection Modified: pypy/trunk/pypy/rpython/memory/gctransform/framework.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gctransform/framework.py (original) +++ pypy/trunk/pypy/rpython/memory/gctransform/framework.py Sat Nov 13 17:07:01 2010 @@ -172,6 +172,7 @@ gcdata.static_root_nongcend = a_random_address # patched in finish() gcdata.static_root_end = a_random_address # patched in finish() gcdata.max_type_id = 13 # patched in finish() + gcdata.typeids_z = a_random_address # patched in finish() self.gcdata = gcdata self.malloc_fnptr_cache = {} @@ -212,6 +213,9 @@ data_classdef.generalize_attr( 'max_type_id', annmodel.SomeInteger()) + data_classdef.generalize_attr( + 'typeids_z', + annmodel.SomeAddress()) annhelper = annlowlevel.MixLevelHelperAnnotator(self.translator.rtyper) @@ -415,6 +419,11 @@ [s_gc, annmodel.SomeInteger()], annmodel.s_Bool, minimal_transform=False) + self.get_typeids_z_ptr = getfn(inspector.get_typeids_z, + [s_gc], + annmodel.SomePtr( + lltype.Ptr(rgc.ARRAY_OF_CHAR)), + minimal_transform=False) self.set_max_heap_size_ptr = getfn(GCClass.set_max_heap_size.im_func, [s_gc, @@ -572,7 +581,14 @@ newgcdependencies = [] newgcdependencies.append(ll_static_roots_inside) ll_instance.inst_max_type_id = len(group.members) - self.write_typeid_list() + typeids_z = self.write_typeid_list() + ll_typeids_z = lltype.malloc(rgc.ARRAY_OF_CHAR, + len(typeids_z), + immortal=True) + for i in range(len(typeids_z)): + ll_typeids_z[i] = typeids_z[i] + ll_instance.inst_typeids_z = llmemory.cast_ptr_to_adr(ll_typeids_z) + newgcdependencies.append(ll_typeids_z) return newgcdependencies def get_finish_tables(self): @@ -599,6 +615,11 @@ for index in range(len(self.layoutbuilder.type_info_group.members)): f.write("member%-4d %s\n" % (index, all_ids.get(index, '?'))) f.close() + try: + import zlib + return zlib.compress(udir.join("typeids.txt").read(), 9) + except ImportError: + return '' def transform_graph(self, graph): func = getattr(graph, 'func', None) @@ -988,6 +1009,13 @@ resultvar=hop.spaceop.result) self.pop_roots(hop, livevars) + def gct_gc_typeids_z(self, hop): + livevars = self.push_roots(hop) + hop.genop("direct_call", + [self.get_typeids_z_ptr, self.c_const_gc], + resultvar=hop.spaceop.result) + self.pop_roots(hop, livevars) + def gct_malloc_nonmovable_varsize(self, hop): TYPE = hop.spaceop.result.concretetype if self.gcdata.gc.can_malloc_nonmovable(): Modified: pypy/trunk/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/trunk/pypy/translator/c/test/test_newgc.py (original) +++ pypy/trunk/pypy/translator/c/test/test_newgc.py Sat Nov 13 17:07:01 2010 @@ -1096,6 +1096,42 @@ assert os.path.exists(self.filename_dump) assert os.path.getsize(self.filename_dump) > 64 + filename_dump_typeids_z = str(udir.join('test_typeids_z')) + def define_write_typeids_z(self): + U = lltype.GcForwardReference() + U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)), + ('x', lltype.Signed))) + S = lltype.GcStruct('S', ('u', lltype.Ptr(U))) + A = lltype.GcArray(lltype.Ptr(S)) + filename = self.filename_dump_typeids_z + + def fn(): + s = lltype.malloc(S) + s.u = lltype.malloc(U) + s.u.next = lltype.malloc(U) + s.u.next.next = lltype.malloc(U) + a = lltype.malloc(A, 1000) + s2 = lltype.malloc(S) + # + p = rgc.get_typeids_z() + s = ''.join([p[i] for i in range(len(p))]) + fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666) + os.write(fd, s) + os.close(fd) + return 0 + + return fn + + def test_write_typeids_z(self): + self.run("write_typeids_z") + f = open(self.filename_dump_typeids_z) + data_z = f.read() + f.close() + import zlib + data = zlib.decompress(data_z) + assert data.startswith('member0') + assert 'GcArray of * GcStruct S {' in data + class TestSemiSpaceGC(TestUsingFramework, snippet.SemiSpaceGCTestDefines): gcpolicy = "semispace" should_be_moving = True From david at codespeak.net Sat Nov 13 17:21:21 2010 From: david at codespeak.net (david at codespeak.net) Date: Sat, 13 Nov 2010 17:21:21 +0100 (CET) Subject: [pypy-svn] r79060 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . helper Message-ID: <20101113162121.24AA7282BE3@codespeak.net> Author: david Date: Sat Nov 13 17:21:19 2010 New Revision: 79060 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Log: Implement forcing with force_token, call_may_force and guard_no_force operations. Add some logic to possibly merge operations with following guards Additionaly implement some fixes for procedure calling related to the manipulation of frame and stack pointers Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Sat Nov 13 17:21:19 2010 @@ -35,23 +35,25 @@ def setup_failure_recovery(self): @rgc.no_collect - def failure_recovery_func(mem_loc, stackloc): + def failure_recovery_func(mem_loc, frame_loc): """mem_loc is a structure in memory describing where the values for - the failargs are stored. stacklock is the address of the stack - section where the registers were saved.""" - enc = rffi.cast(rffi.CCHARP, mem_loc) - stack = rffi.cast(rffi.CCHARP, stackloc) - return self.decode_registers_and_descr(enc, stack) + the failargs are stored. + frame loc is the address of the frame pointer for the frame to be + decoded frame """ + return self.decode_registers_and_descr(mem_loc, frame_loc) self.failure_recovery_func = failure_recovery_func @rgc.no_collect - def decode_registers_and_descr(self, enc, stack): - """Decode locations encoded in memory at enc and write the values to + def decode_registers_and_descr(self, mem_loc, frame_loc): + """Decode locations encoded in memory at mem_loc and write the values to the failboxes. - Registers are saved on the stack - XXX Rest to follow""" + Values for spilled vars and registers are stored on stack at frame_loc + """ + enc = rffi.cast(rffi.CCHARP, mem_loc) frame_depth = self.decode32(enc, 0) + stack = rffi.cast(rffi.CCHARP, frame_loc - (frame_depth)*WORD) + regs = rffi.cast(rffi.CCHARP, frame_loc - (frame_depth + len(r.all_regs))*WORD) i = 3 fail_index = -1 while(True): @@ -73,13 +75,11 @@ i += 4 elif res == '\xFC': # stack location stack_loc = self.decode32(enc, i+1) - #XXX ffuu use propper calculation here - value = self.decode32(stack, - (len(r.all_regs)+frame_depth-stack_loc)*WORD) + value = self.decode32(stack, (frame_depth - stack_loc)*WORD) i += 4 else: # an int for now reg = ord(enc[i]) - value = self.decode32(stack, reg*WORD) + value = self.decode32(regs, reg*WORD) if group == '\xEF': # INT self.fail_boxes_int.setitem(fail_index, value) @@ -113,15 +113,17 @@ self.setup_failure_recovery() functype = lltype.Ptr(lltype.FuncType([lltype.Signed, lltype.Signed], lltype.Signed)) decode_registers_addr = llhelper(functype, self.failure_recovery_func) + self.mc.PUSH([reg.value for reg in r.all_regs]) # registers r0 .. r10 self.mc.MOV_rr(r.r0.value, r.lr.value) # move mem block address, to r0 to pass as # parameter to next procedure call - self.mc.MOV_rr(r.r1.value, r.sp.value) # pass the current stack pointer as second param + self.mc.MOV_rr(r.r1.value, r.fp.value) # pass the current frame pointer as second param self.mc.BL(rffi.cast(lltype.Signed, decode_registers_addr)) self.mc.MOV_rr(r.ip.value, r.r0.value) self.mc.POP([reg.value for reg in r.all_regs]) self.mc.MOV_rr(r.r0.value, r.ip.value) + self.mc.ensure_can_fit(self.epilog_size) self.gen_func_epilog() def _gen_path_to_exit_path(self, op, args, regalloc, fcond=c.AL): @@ -135,12 +137,15 @@ \xFE = Empty arg """ + descr = op.getdescr() box = TempBox() reg = regalloc.force_allocate_reg(box) # XXX free this memory # XXX allocate correct amount of memory mem = lltype.malloc(rffi.CArray(lltype.Char), (len(args)+5)*4, flavor='raw') - self.encode32(mem, 0, regalloc.frame_manager.frame_depth) + # Note, the actual frame depth is one less than the value stored in + # regalloc.frame_manager.frame_depth + self.encode32(mem, 0, regalloc.frame_manager.frame_depth - 1) i = 0 j = 4 while(i < len(args)): @@ -175,8 +180,7 @@ mem[j] = chr(0xFF) memaddr = rffi.cast(lltype.Signed, mem) - - n = self.cpu.get_fail_descr_number(op.getdescr()) + n = self.cpu.get_fail_descr_number(descr) self.encode32(mem, j+1, n) self.mc.gen_load_int(r.lr.value, memaddr, cond=fcond) # use lr to pass an argument self.mc.B(self._exit_code_addr, fcond, reg) @@ -184,7 +188,7 @@ # This register is used for patching when assembling a bridge # guards going to be patched are allways conditional if fcond != c.AL: - op.getdescr()._arm_guard_reg = reg + descr._arm_guard_reg = reg regalloc.possibly_free_var(box) return memaddr @@ -192,13 +196,16 @@ while(self.mc.curraddr() % FUNC_ALIGN != 0): self.mc.writechar(chr(0)) - epilog_size = 2*WORD + epilog_size = 3*WORD def gen_func_epilog(self,cond=c.AL): self.mc.MOV_rr(r.sp.value, r.fp.value) + self.mc.POP([r.r4.value], cond=cond) # Pop value used as forcething self.mc.POP([reg.value for reg in r.callee_restored_registers], cond=cond) def gen_func_prolog(self): self.mc.PUSH([reg.value for reg in r.callee_saved_registers]) + self.mc.MOV_ri(r.r4.value, 0xCC) + self.mc.PUSH([r.r4.value]) # Push some reg to use as force thing which is restored when popping from stack self.mc.MOV_rr(r.fp.value, r.sp.value) def gen_bootstrap_code(self, inputargs, regalloc, looptoken): @@ -232,12 +239,8 @@ loop_head=self.mc.curraddr() looptoken._arm_bootstrap_code = loop_start looptoken._arm_loop_code = loop_head - fcond=c.AL print inputargs, operations - for op in operations: - # XXX consider merging ops with next one if it is an adecuate guard - opnum = op.getopnum() - fcond = self.operations[opnum](self, op, regalloc, fcond) + self._walk_operations(operations, regalloc) self._patch_sp_offset(sp_patch_location, regalloc) @@ -255,23 +258,59 @@ def _patch_sp_offset(self, addr, regalloc): cb = ARMv7InMemoryBuilder(addr, ARMv7InMemoryBuilder.size_of_gen_load_int) + # Note: the frame_depth is one less than the value stored in the frame + # manager if regalloc.frame_manager.frame_depth == 1: return - n = (regalloc.frame_manager.frame_depth)*WORD + n = (regalloc.frame_manager.frame_depth-1)*WORD self._adjust_sp(n, regalloc, cb) def _adjust_sp(self, n, regalloc, cb=None, fcond=c.AL): if cb is None: cb = self.mc + if n < 0: + n = -n + rev = True + else: + rev = False if n <= 0xFF and fcond == c.AL: - cb.SUB_ri(r.sp.value, r.sp.value, n) + if rev: + op = cb.ADD_ri + else: + op = cb.SUB_ri + op(r.sp.value, r.sp.value, n) else: b = TempBox() reg = regalloc.force_allocate_reg(b) cb.gen_load_int(reg.value, n, cond=fcond) - cb.SUB_rr(r.sp.value, r.sp.value, reg.value, cond=fcond) + if rev: + op = cb.ADD_rr + else: + op = cb.SUB_rr + op(r.sp.value, r.sp.value, reg.value, cond=fcond) regalloc.possibly_free_var(b) + def _walk_operations(self, operations, regalloc): + fcond=c.AL + i = 0 + while i < len(operations): + op = operations[i] + # XXX consider merging ops with next one if it is an adecuate guard + opnum = op.getopnum() + if self.can_merge_with_next_guard(op, i, operations): + fcond = self.operations_with_guard[opnum](self, op, + operations[i+1], regalloc, fcond) + i += 1 + else: + fcond = self.operations[opnum](self, op, regalloc, fcond) + i += 1 + + def can_merge_with_next_guard(self, op, i, operations): + if op.getopnum() == rop.CALL_MAY_FORCE or op.getopnum() == rop.CALL_ASSEMBLER: + assert operations[i + 1].getopnum() == rop.GUARD_NOT_FORCED + return True + return False + def assemble_bridge(self, faildescr, inputargs, operations): enc = rffi.cast(rffi.CCHARP, faildescr._failure_recovery_code) longevity = compute_vars_longevity(inputargs, operations) @@ -280,10 +319,7 @@ regalloc.update_bindings(enc, inputargs) bridge_head = self.mc.curraddr() - fcond = c.AL - for op in operations: - opnum = op.getopnum() - fcond = self.operations[opnum](self, op, regalloc, fcond) + self._walk_operations(operations, regalloc) self.gen_func_epilog() print 'Done building bridges' self.patch_trace(faildescr, bridge_head) @@ -322,6 +358,9 @@ self.mc.MOV_rr(loc.value, prev_loc.value) mov_loc_loc = regalloc_mov + def leave_jitted_hook(self): + pass + def make_operation_list(): def notimplemented(self, op, regalloc, fcond): raise NotImplementedError, op @@ -339,4 +378,19 @@ operations[value] = func return operations +def make_guard_operation_list(): + def notimplemented(self, op, guard_op, regalloc, fcond): + raise NotImplementedError, op + guard_operations = [notimplemented] * rop._LAST + for key, value in rop.__dict__.items(): + key = key.lower() + if key.startswith('_'): + continue + methname = 'emit_guard_%s' % key + if hasattr(AssemblerARM, methname): + func = getattr(AssemblerARM, methname).im_func + guard_operations[value] = func + return guard_operations + AssemblerARM.operations = make_operation_list() +AssemblerARM.operations_with_guard = make_guard_operation_list() Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Sat Nov 13 17:21:19 2010 @@ -50,8 +50,10 @@ arg2 = regalloc.make_sure_var_in_reg(a1, [a0], selected_reg=r.r1, imm_fine=False) assert arg1 == r.r0 assert arg2 == r.r1 + regalloc.before_call() res = regalloc.force_allocate_reg(op.result, selected_reg=r.r0) getattr(self.mc, opname)(fcond) + regalloc.after_call(op.result) regalloc.possibly_free_vars_for_op(op) return fcond return f Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Sat Nov 13 17:21:19 2010 @@ -230,7 +230,7 @@ self._gen_path_to_exit_path(op, op.getarglist(), regalloc, c.AL) return fcond - def emit_op_call(self, op, regalloc, fcond): + def emit_op_call(self, op, regalloc, fcond, save_all_regs=False): locs = [] # all arguments past the 4th go on the stack # XXX support types other than int (one word types) @@ -244,17 +244,23 @@ regalloc.possibly_free_var(reg) adr = self.cpu.cast_adr_to_int(op.getarg(0).getint()) - # XXX use PUSH here instead of spilling every reg for itself - regalloc.before_call() reg_args = min(op.numargs()-1, 4) for i in range(1, reg_args+1): l = regalloc.make_sure_var_in_reg(op.getarg(i), selected_reg=r.all_regs[i-1]) locs.append(l) - self.mc.BL(adr) + # XXX use PUSH here instead of spilling every reg for itself + if save_all_regs: + regalloc.before_call(r.all_regs, save_all_regs) + else: + regalloc.before_call() regalloc.force_allocate_reg(op.result, selected_reg=r.r0) + self.mc.BL(adr) regalloc.after_call(op.result) + # readjust the sp in case we passed some args on the stack + if op.numargs() > 5: + self._adjust_sp(-n, regalloc, fcond=fcond) regalloc.possibly_free_vars(locs) class FieldOpAssembler(object): @@ -508,9 +514,35 @@ f(value_loc.value, temp.value, basesize, cond=fcond) return fcond +class ForceOpAssembler(object): + def emit_op_force_token(self, op, regalloc, fcond): + res_loc = regalloc.force_allocate_reg(op.result) + self.mc.MOV_rr(res_loc.value, r.fp.value) + return fcond + + def emit_guard_call_may_force(self, op, guard_op, regalloc, fcond): + faildescr = guard_op.getdescr() + fail_index = self.cpu.get_fail_descr_number(faildescr) + t = TempBox() + l0 = regalloc.force_allocate_reg(t) + self.mc.gen_load_int(l0.value, fail_index) + self.mc.STR_ri(l0.value, r.fp.value) + + # force all reg values to be spilled when calling + fcond = self.emit_op_call(op, regalloc, fcond, save_all_regs=True) + + self.mc.LDR_ri(l0.value, r.fp.value) + self.mc.CMP_ri(l0.value, 0) + + regalloc.possibly_free_var(t) + + self._emit_guard(guard_op, regalloc, c.LT) + return fcond + class ResOpAssembler(GuardOpAssembler, IntOpAsslember, OpAssembler, UnaryIntOpAssembler, FieldOpAssembler, ArrayOpAssember, - StrOpAssembler, UnicodeOpAssembler): + StrOpAssembler, UnicodeOpAssembler, + ForceOpAssembler): pass Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Sat Nov 13 17:21:19 2010 @@ -61,3 +61,18 @@ def cast_ptr_to_int(x): adr = llmemory.cast_ptr_to_adr(x) return self.cast_adr_to_int(adr) + + def force(self, addr_of_force_index): + TP = rffi.CArrayPtr(lltype.Signed) + fail_index = rffi.cast(TP, addr_of_force_index)[0] + assert fail_index >= 0, "already forced!" + faildescr = self.get_fail_descr_from_number(fail_index) + rffi.cast(TP, addr_of_force_index)[0] = -1 + # start of "no gc operation!" block + fail_index_2 = self.assembler.failure_recovery_func( + faildescr._failure_recovery_code, + addr_of_force_index) + self.assembler.leave_jitted_hook() + # end of "no gc operation!" block + #assert fail_index == fail_index_2 + return faildescr From arigo at codespeak.net Sat Nov 13 18:03:45 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 13 Nov 2010 18:03:45 +0100 (CET) Subject: [pypy-svn] r79066 - pypy/trunk/pypy/tool Message-ID: <20101113170345.05CD1282B90@codespeak.net> Author: arigo Date: Sat Nov 13 18:03:44 2010 New Revision: 79066 Added: pypy/trunk/pypy/tool/gcdump.py - copied, changed from r79065, user/arigo/hack/pypy-hack/heapstats/dump.py Log: Remove the old, not-working-any-more version of gcdump.py. Replace it with the up-to-date version from my user dir, simplified to only keep the part that gives useful output. Copied: pypy/trunk/pypy/tool/gcdump.py (from r79065, user/arigo/hack/pypy-hack/heapstats/dump.py) ============================================================================== --- user/arigo/hack/pypy-hack/heapstats/dump.py (original) +++ pypy/trunk/pypy/tool/gcdump.py Sat Nov 13 18:03:44 2010 @@ -1,13 +1,13 @@ #! /usr/bin/env python """ -Syntax: dump.py +Prints a human-readable total out of a dumpfile produced +by gc.dump_rpy_heap(), and optionally a typeids.txt. -where the first argument can be: - -r raw summary - -d summary after reduction to dominators +Syntax: dump.py [] + +By default, typeids.txt is loaded from the same dir as dumpfile. """ import sys, array, struct, os -import dominator class Stat(object): @@ -20,24 +20,6 @@ for obj in self.walk(a): self.add_object_summary(obj[2], obj[3]) - def reduce_and_summarize(self, filename): - a = self.load_dump_file(filename) - fg = FlowGraph(self, a) - dom = fg.compute_dominator_tree() - del fg - # - self.summary = {} # {typenum: [count, totalsize]} - typenums = {ROOT: 0} - for obj in self.walk(a): - typenums[obj[1]] = obj[2] - for obj in self.walk(a): - try: - w = dom[obj[1]] - except KeyError: - assert obj[1] == 0 - continue - self.add_object_summary(typenums[w], obj[3]) - def load_typeids(self, filename): self.typeids = Stat.typeids.copy() for num, line in enumerate(open(filename)): @@ -97,52 +79,17 @@ print >> sys.stderr, 'done' -class ROOT(object): - def __repr__(self): - return 'ROOT' -ROOT = ROOT() - - -class FlowGraph(object): - - def __init__(self, stat, a): - self.stat = stat - self.a = a - # - succ = {} - roots = [] - for obj in self.stat.walk(self.a): - if roots is not None: - if obj[2] == 0: # root end marker - succ[ROOT] = tuple(roots) - roots = None - continue - else: - roots.append(obj[1]) - succ[obj[1]] = tuple(obj[-1]) - self.succ = succ - - def compute_dominator_tree(self): - print >> sys.stderr, 'computing dominator tree...', - dom = dominator.dominator_tree(self.succ, ROOT) - print >> sys.stderr, 'done' - return dom - - if __name__ == '__main__': - kind = sys.argv[1] + if len(sys.argv) <= 1: + print >> sys.stderr, __doc__ + sys.exit(2) stat = Stat() - if kind == '-r': - stat.summarize(sys.argv[2]) - elif kind == '-d': - stat.reduce_and_summarize(sys.argv[2]) - else: - raise ValueError(kind) + stat.summarize(sys.argv[1]) # - if len(sys.argv) > 3: - typeid_name = sys.argv[3] + if len(sys.argv) > 2: + typeid_name = sys.argv[2] else: - typeid_name = os.path.join(os.path.dirname(sys.argv[2]), 'typeids.txt') + typeid_name = os.path.join(os.path.dirname(sys.argv[1]), 'typeids.txt') if os.path.isfile(typeid_name): stat.load_typeids(typeid_name) # From arigo at codespeak.net Sat Nov 13 18:18:42 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 13 Nov 2010 18:18:42 +0100 (CET) Subject: [pypy-svn] r79067 - pypy/trunk/pypy/tool Message-ID: <20101113171842.05D70282BE3@codespeak.net> Author: arigo Date: Sat Nov 13 18:18:41 2010 New Revision: 79067 Modified: pypy/trunk/pypy/tool/gcdump.py Log: Weaken this assert. This weaker check is enough to detect mixes between 32-bits and 64-bits, and the stricter version might actually be wrong. Modified: pypy/trunk/pypy/tool/gcdump.py ============================================================================== --- pypy/trunk/pypy/tool/gcdump.py (original) +++ pypy/trunk/pypy/tool/gcdump.py Sat Nov 13 18:18:41 2010 @@ -65,7 +65,7 @@ def walk(self, a, start=0, stop=None): assert a[-1] == -1, "invalid or truncated dump file (or 32/64-bit mix)" - assert a[-2] > 0, "invalid or truncated dump file (or 32/64-bit mix)" + assert a[-2] != -1, "invalid or truncated dump file (or 32/64-bit mix)" print >> sys.stderr, 'walking...', i = start if stop is None: From arigo at codespeak.net Sat Nov 13 19:11:39 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 13 Nov 2010 19:11:39 +0100 (CET) Subject: [pypy-svn] r79068 - in pypy/trunk/pypy/jit/metainterp: . test Message-ID: <20101113181139.98A6C282B9C@codespeak.net> Author: arigo Date: Sat Nov 13 19:11:37 2010 New Revision: 79068 Modified: pypy/trunk/pypy/jit/metainterp/compile.py pypy/trunk/pypy/jit/metainterp/resume.py pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py pypy/trunk/pypy/jit/metainterp/test/test_resume.py pypy/trunk/pypy/jit/metainterp/virtualizable.py Log: Rewrite the class Numbering as a ll GcStruct containing an Array part. This is a bit of a hack, but it's very useful; it can save 10% of memory in agaynor's example. Modified: pypy/trunk/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/compile.py (original) +++ pypy/trunk/pypy/jit/metainterp/compile.py Sat Nov 13 19:11:37 2010 @@ -1,4 +1,5 @@ +from pypy.rpython.lltypesystem import lltype from pypy.rpython.ootypesystem import ootype from pypy.objspace.flow.model import Constant, Variable from pypy.rlib.objectmodel import we_are_translated @@ -14,6 +15,7 @@ from pypy.jit.metainterp.specnode import NotSpecNode, more_general_specnodes from pypy.jit.metainterp.typesystem import llhelper, oohelper from pypy.jit.metainterp.optimizeutil import InvalidLoop +from pypy.jit.metainterp.resume import NUMBERING from pypy.jit.codewriter import heaptracker def giveup(): @@ -217,7 +219,7 @@ # this class also gets the following attributes stored by resume.py code rd_snapshot = None rd_frame_info_list = None - rd_numb = None + rd_numb = lltype.nullptr(NUMBERING) rd_consts = None rd_virtuals = None rd_pendingfields = None Modified: pypy/trunk/pypy/jit/metainterp/resume.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/resume.py (original) +++ pypy/trunk/pypy/jit/metainterp/resume.py Sat Nov 13 19:11:37 2010 @@ -65,12 +65,21 @@ snapshot = Snapshot(snapshot, boxes) storage.rd_snapshot = snapshot -class Numbering(object): - __slots__ = ('prev', 'nums') - - def __init__(self, prev, nums): - self.prev = prev - self.nums = nums +# +# The following is equivalent to the RPython-level declaration: +# +# class Numbering: __slots__ = ['prev', 'nums'] +# +# except that it is more compact in translated programs, because the +# array 'nums' is inlined in the single NUMBERING object. This is +# important because this is often the biggest single consumer of memory +# in a pypy-c-jit. +# +NUMBERINGP = lltype.Ptr(lltype.GcForwardReference()) +NUMBERING = lltype.GcStruct('Numbering', + ('prev', NUMBERINGP), + ('nums', lltype.Array(rffi.SHORT))) +NUMBERINGP.TO.become(NUMBERING) TAGMASK = 3 @@ -162,7 +171,7 @@ def number(self, values, snapshot): if snapshot is None: - return None, {}, 0 + return lltype.nullptr(NUMBERING), {}, 0 if snapshot in self.numberings: numb, liveboxes, v = self.numberings[snapshot] return numb, liveboxes.copy(), v @@ -171,7 +180,7 @@ n = len(liveboxes)-v boxes = snapshot.boxes length = len(boxes) - nums = [UNASSIGNED] * length + numb = lltype.malloc(NUMBERING, length) for i in range(length): box = boxes[i] value = values.get(box, None) @@ -190,9 +199,9 @@ tagged = tag(n, TAGBOX) n += 1 liveboxes[box] = tagged - nums[i] = tagged + numb.nums[i] = tagged # - numb = Numbering(numb1, nums) + numb.prev = numb1 self.numberings[snapshot] = numb, liveboxes, v return numb, liveboxes.copy(), v @@ -297,7 +306,7 @@ # compute the numbering storage = self.storage # make sure that nobody attached resume data to this guard yet - assert storage.rd_numb is None + assert not storage.rd_numb numb, liveboxes_from_env, v = self.memo.number(values, storage.rd_snapshot) self.liveboxes_from_env = liveboxes_from_env @@ -722,34 +731,36 @@ self.boxes_f = boxes_f self._prepare_next_section(info) - def consume_virtualizable_boxes(self, vinfo, nums): + def consume_virtualizable_boxes(self, vinfo, numb): # we have to ignore the initial part of 'nums' (containing vrefs), # find the virtualizable from nums[-1], and use it to know how many # boxes of which type we have to return. This does not write # anything into the virtualizable. - virtualizablebox = self.decode_ref(nums[-1]) + index = len(numb.nums) - 1 + virtualizablebox = self.decode_ref(numb.nums[index]) virtualizable = vinfo.unwrap_virtualizable_box(virtualizablebox) - return vinfo.load_list_of_boxes(virtualizable, self, nums) + return vinfo.load_list_of_boxes(virtualizable, self, numb) - def consume_virtualref_boxes(self, nums, end): + def consume_virtualref_boxes(self, numb, end): # Returns a list of boxes, assumed to be all BoxPtrs. # We leave up to the caller to call vrefinfo.continue_tracing(). assert (end & 1) == 0 - return [self.decode_ref(nums[i]) for i in range(end)] + return [self.decode_ref(numb.nums[i]) for i in range(end)] def consume_vref_and_vable_boxes(self, vinfo, ginfo): - nums = self.cur_numb.nums - self.cur_numb = self.cur_numb.prev + numb = self.cur_numb + self.cur_numb = numb.prev if vinfo is not None: - virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, nums) - end = len(nums) - len(virtualizable_boxes) + virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, numb) + end = len(numb.nums) - len(virtualizable_boxes) elif ginfo is not None: - virtualizable_boxes = [self.decode_ref(nums[-1])] - end = len(nums) - 1 + index = len(numb.nums) - 1 + virtualizable_boxes = [self.decode_ref(numb.nums[index])] + end = len(numb.nums) - 1 else: virtualizable_boxes = None - end = len(nums) - virtualref_boxes = self.consume_virtualref_boxes(nums, end) + end = len(numb.nums) + virtualref_boxes = self.consume_virtualref_boxes(numb, end) return virtualizable_boxes, virtualref_boxes def allocate_with_vtable(self, known_class): @@ -977,23 +988,24 @@ info = blackholeinterp.get_current_position_info() self._prepare_next_section(info) - def consume_virtualref_info(self, vrefinfo, nums, end): + def consume_virtualref_info(self, vrefinfo, numb, end): # we have to decode a list of references containing pairs # [..., virtual, vref, ...] stopping at 'end' assert (end & 1) == 0 for i in range(0, end, 2): - virtual = self.decode_ref(nums[i]) - vref = self.decode_ref(nums[i+1]) + virtual = self.decode_ref(numb.nums[i]) + vref = self.decode_ref(numb.nums[i+1]) # For each pair, we store the virtual inside the vref. vrefinfo.continue_tracing(vref, virtual) - def consume_vable_info(self, vinfo, nums): + def consume_vable_info(self, vinfo, numb): # we have to ignore the initial part of 'nums' (containing vrefs), # find the virtualizable from nums[-1], load all other values # from the CPU stack, and copy them into the virtualizable if vinfo is None: - return len(nums) - virtualizable = self.decode_ref(nums[-1]) + return len(numb.nums) + index = len(numb.nums) - 1 + virtualizable = self.decode_ref(numb.nums[index]) virtualizable = vinfo.cast_gcref_to_vtype(virtualizable) if self.resume_after_guard_not_forced == 1: # in the middle of handle_async_forcing() @@ -1005,7 +1017,7 @@ # is and stays 0. Note the call to reset_vable_token() in # warmstate.py. assert not virtualizable.vable_token - return vinfo.write_from_resume_data_partial(virtualizable, self, nums) + return vinfo.write_from_resume_data_partial(virtualizable, self, numb) def load_value_of_type(self, TYPE, tagged): from pypy.jit.metainterp.warmstate import specialize_value @@ -1022,12 +1034,12 @@ load_value_of_type._annspecialcase_ = 'specialize:arg(1)' def consume_vref_and_vable(self, vrefinfo, vinfo, ginfo): - nums = self.cur_numb.nums - self.cur_numb = self.cur_numb.prev + numb = self.cur_numb + self.cur_numb = numb.prev if self.resume_after_guard_not_forced != 2: - end_vref = self.consume_vable_info(vinfo, nums) + end_vref = self.consume_vable_info(vinfo, numb) if ginfo is not None: end_vref -= 1 - self.consume_virtualref_info(vrefinfo, nums, end_vref) + self.consume_virtualref_info(vrefinfo, numb, end_vref) def allocate_with_vtable(self, known_class): from pypy.jit.metainterp.executor import exec_new_with_vtable @@ -1180,8 +1192,9 @@ 'at', compute_unique_id(frameinfo)) frameinfo = frameinfo.prev numb = storage.rd_numb - while numb is not None: - debug_print('\tnumb', str([untag(i) for i in numb.nums]), + while numb: + debug_print('\tnumb', str([untag(numb.nums[i]) + for i in range(len(numb.nums))]), 'at', compute_unique_id(numb)) numb = numb.prev for const in storage.rd_consts: Modified: pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py Sat Nov 13 19:11:37 2010 @@ -51,12 +51,12 @@ # opt.store_final_boxes_in_guard(op) if op.getfailargs() == [b0, b1]: - assert fdescr.rd_numb.nums == [tag(1, TAGBOX)] - assert fdescr.rd_numb.prev.nums == [tag(0, TAGBOX)] + assert list(fdescr.rd_numb.nums) == [tag(1, TAGBOX)] + assert list(fdescr.rd_numb.prev.nums) == [tag(0, TAGBOX)] else: assert op.getfailargs() == [b1, b0] - assert fdescr.rd_numb.nums == [tag(0, TAGBOX)] - assert fdescr.rd_numb.prev.nums == [tag(1, TAGBOX)] + assert list(fdescr.rd_numb.nums) == [tag(0, TAGBOX)] + assert list(fdescr.rd_numb.prev.nums) == [tag(1, TAGBOX)] assert fdescr.rd_virtuals is None assert fdescr.rd_consts == [] Modified: pypy/trunk/pypy/jit/metainterp/test/test_resume.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_resume.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_resume.py Sat Nov 13 19:11:37 2010 @@ -142,6 +142,13 @@ assert bh.written_f == expected_f +def Numbering(prev, nums): + numb = lltype.malloc(NUMBERING, len(nums)) + numb.prev = prev or lltype.nullptr(NUMBERING) + for i in range(len(nums)): + numb.nums[i] = nums[i] + return numb + def test_simple_read(): #b1, b2, b3 = [BoxInt(), BoxPtr(), BoxInt()] c1, c2, c3 = [ConstInt(111), ConstInt(222), ConstInt(333)] @@ -391,15 +398,15 @@ assert fi1.pc == 3 def test_Numbering_create(): - l = [1, 2] + l = [rffi.r_short(1), rffi.r_short(2)] numb = Numbering(None, l) - assert numb.prev is None - assert numb.nums is l + assert not numb.prev + assert list(numb.nums) == l - l1 = ['b3'] + l1 = [rffi.r_short(3)] numb1 = Numbering(numb, l1) - assert numb1.prev is numb - assert numb1.nums is l1 + assert numb1.prev == numb + assert list(numb1.nums) == l1 def test_capture_resumedata(): b1, b2, b3 = [BoxInt(), BoxPtr(), BoxInt()] @@ -765,11 +772,12 @@ assert liveboxes == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b3: tag(2, TAGBOX)} - assert numb.nums == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), - tag(1, TAGINT)] - assert numb.prev.nums == [tag(0, TAGBOX), tag(1, TAGINT), tag(1, TAGBOX), - tag(0, TAGBOX), tag(2, TAGINT)] - assert numb.prev.prev is None + assert list(numb.nums) == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), + tag(1, TAGINT)] + assert list(numb.prev.nums) == [tag(0, TAGBOX), tag(1, TAGINT), + tag(1, TAGBOX), + tag(0, TAGBOX), tag(2, TAGINT)] + assert not numb.prev.prev numb2, liveboxes2, v = memo.number({}, snap2) assert v == 0 @@ -777,9 +785,9 @@ assert liveboxes2 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b3: tag(2, TAGBOX)} assert liveboxes2 is not liveboxes - assert numb2.nums == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb2.prev is numb.prev + assert list(numb2.nums) == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), + tag(3, TAGINT)] + assert numb2.prev == numb.prev env3 = [c3, b3, b1, c3] snap3 = Snapshot(snap, env3) @@ -800,9 +808,9 @@ assert v == 0 assert liveboxes3 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX)} - assert numb3.nums == [tag(3, TAGINT), tag(4, TAGINT), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb3.prev is numb.prev + assert list(numb3.nums) == [tag(3, TAGINT), tag(4, TAGINT), tag(0, TAGBOX), + tag(3, TAGINT)] + assert numb3.prev == numb.prev # virtual env4 = [c3, b4, b1, c3] @@ -813,9 +821,9 @@ assert liveboxes4 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b4: tag(0, TAGVIRTUAL)} - assert numb4.nums == [tag(3, TAGINT), tag(0, TAGVIRTUAL), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb4.prev is numb.prev + assert list(numb4.nums) == [tag(3, TAGINT), tag(0, TAGVIRTUAL), + tag(0, TAGBOX), tag(3, TAGINT)] + assert numb4.prev == numb.prev env5 = [b1, b4, b5] snap5 = Snapshot(snap4, env5) @@ -826,9 +834,9 @@ assert liveboxes5 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b4: tag(0, TAGVIRTUAL), b5: tag(1, TAGVIRTUAL)} - assert numb5.nums == [tag(0, TAGBOX), tag(0, TAGVIRTUAL), - tag(1, TAGVIRTUAL)] - assert numb5.prev is numb4 + assert list(numb5.nums) == [tag(0, TAGBOX), tag(0, TAGVIRTUAL), + tag(1, TAGVIRTUAL)] + assert numb5.prev == numb4 def test_ResumeDataLoopMemo_number_boxes(): memo = ResumeDataLoopMemo(FakeMetaInterpStaticData()) Modified: pypy/trunk/pypy/jit/metainterp/virtualizable.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/virtualizable.py (original) +++ pypy/trunk/pypy/jit/metainterp/virtualizable.py Sat Nov 13 19:11:37 2010 @@ -100,48 +100,48 @@ i = i + 1 assert len(boxes) == i + 1 # - def write_from_resume_data_partial(virtualizable, reader, nums): + def write_from_resume_data_partial(virtualizable, reader, numb): # Load values from the reader (see resume.py) described by # the list of numbers 'nums', and write them in their proper # place in the 'virtualizable'. This works from the end of # the list and returns the index in 'nums' of the start of # the virtualizable data found, allowing the caller to do # further processing with the start of the list. - i = len(nums) - 1 + i = len(numb.nums) - 1 assert i >= 0 for ARRAYITEMTYPE, fieldname in unroll_array_fields_rev: lst = getattr(virtualizable, fieldname) for j in range(getlength(lst)-1, -1, -1): i -= 1 assert i >= 0 - x = reader.load_value_of_type(ARRAYITEMTYPE, nums[i]) + x = reader.load_value_of_type(ARRAYITEMTYPE, numb.nums[i]) setarrayitem(lst, j, x) for FIELDTYPE, fieldname in unroll_static_fields_rev: i -= 1 assert i >= 0 - x = reader.load_value_of_type(FIELDTYPE, nums[i]) + x = reader.load_value_of_type(FIELDTYPE, numb.nums[i]) setattr(virtualizable, fieldname, x) return i # - def load_list_of_boxes(virtualizable, reader, nums): + def load_list_of_boxes(virtualizable, reader, numb): # Uses 'virtualizable' only to know the length of the arrays; # does not write anything into it. The returned list is in # the format expected of virtualizable_boxes, so it ends in # the virtualizable itself. - i = len(nums) - 1 + i = len(numb.nums) - 1 assert i >= 0 - boxes = [reader.decode_box_of_type(self.VTYPEPTR, nums[i])] + boxes = [reader.decode_box_of_type(self.VTYPEPTR, numb.nums[i])] for ARRAYITEMTYPE, fieldname in unroll_array_fields_rev: lst = getattr(virtualizable, fieldname) for j in range(getlength(lst)-1, -1, -1): i -= 1 assert i >= 0 - box = reader.decode_box_of_type(ARRAYITEMTYPE, nums[i]) + box = reader.decode_box_of_type(ARRAYITEMTYPE,numb.nums[i]) boxes.append(box) for FIELDTYPE, fieldname in unroll_static_fields_rev: i -= 1 assert i >= 0 - box = reader.decode_box_of_type(FIELDTYPE, nums[i]) + box = reader.decode_box_of_type(FIELDTYPE, numb.nums[i]) boxes.append(box) boxes.reverse() return boxes From fijal at codespeak.net Sun Nov 14 10:56:17 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 14 Nov 2010 10:56:17 +0100 (CET) Subject: [pypy-svn] r79069 - pypy/trunk/pypy/objspace/flow Message-ID: <20101114095617.2302A282B90@codespeak.net> Author: fijal Date: Sun Nov 14 10:56:15 2010 New Revision: 79069 Modified: pypy/trunk/pypy/objspace/flow/model.py Log: a slight newstylization Modified: pypy/trunk/pypy/objspace/flow/model.py ============================================================================== --- pypy/trunk/pypy/objspace/flow/model.py (original) +++ pypy/trunk/pypy/objspace/flow/model.py Sun Nov 14 10:56:15 2010 @@ -355,7 +355,7 @@ return "%r = %s(%s)" % (self.result, self.opname, ", ".join(map(repr, self.args))) -class Atom: +class Atom(object): def __init__(self, name): self.__name__ = name # make save_global happy def __repr__(self): From fijal at codespeak.net Sun Nov 14 11:04:27 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 14 Nov 2010 11:04:27 +0100 (CET) Subject: [pypy-svn] r79070 - pypy/trunk/pypy/rpython Message-ID: <20101114100427.79148282B90@codespeak.net> Author: fijal Date: Sun Nov 14 11:04:26 2010 New Revision: 79070 Modified: pypy/trunk/pypy/rpython/annlowlevel.py Log: make MixLevelHelperAnnotator a newstyle class Modified: pypy/trunk/pypy/rpython/annlowlevel.py ============================================================================== --- pypy/trunk/pypy/rpython/annlowlevel.py (original) +++ pypy/trunk/pypy/rpython/annlowlevel.py Sun Nov 14 11:04:26 2010 @@ -136,7 +136,7 @@ return funcdesc.cachedgraph(TYPE, alt_name=valid_identifier(alt_name)) -class MixLevelHelperAnnotator: +class MixLevelHelperAnnotator(object): def __init__(self, rtyper): self.rtyper = rtyper From fijal at codespeak.net Sun Nov 14 11:05:19 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 14 Nov 2010 11:05:19 +0100 (CET) Subject: [pypy-svn] r79071 - pypy/trunk/pypy/jit/backend/llsupport/test Message-ID: <20101114100519.53EFB282B9C@codespeak.net> Author: fijal Date: Sun Nov 14 11:05:17 2010 New Revision: 79071 Modified: pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py Log: Make all classes here newstyle Modified: pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py ============================================================================== --- pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py (original) +++ pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py Sun Nov 14 11:05:17 2010 @@ -114,7 +114,7 @@ assert gcrootmap._gcmap[i*2+1] == expected_shapeaddr[i] -class FakeLLOp: +class FakeLLOp(object): def __init__(self): self.record = [] @@ -148,19 +148,19 @@ return llhelper(FPTRTYPE, self._write_barrier_failing_case) -class TestFramework: +class TestFramework(object): gc = 'hybrid' def setup_method(self, meth): - class config_: - class translation: + class config_(object): + class translation(object): gc = self.gc gcrootfinder = 'asmgcc' gctransformer = 'framework' gcremovetypeptr = False - class FakeTranslator: + class FakeTranslator(object): config = config_ - class FakeCPU: + class FakeCPU(object): def cast_adr_to_int(self, adr): ptr = llmemory.cast_adr_to_ptr(adr, gc_ll_descr.WB_FUNCPTR) assert ptr._obj._callable == llop1._write_barrier_failing_case @@ -278,11 +278,11 @@ def test_rewrite_assembler_1(self): # check rewriting of ConstPtrs - class MyFakeCPU: + class MyFakeCPU(object): def cast_adr_to_int(self, adr): assert adr == "some fake address" return 43 - class MyFakeGCRefList: + class MyFakeGCRefList(object): def get_address_of_gcref(self, s_gcref1): assert s_gcref1 == s_gcref return "some fake address" @@ -311,10 +311,10 @@ def test_rewrite_assembler_1_cannot_move(self): # check rewriting of ConstPtrs - class MyFakeCPU: + class MyFakeCPU(object): def cast_adr_to_int(self, adr): xxx # should not be called - class MyFakeGCRefList: + class MyFakeGCRefList(object): def get_address_of_gcref(self, s_gcref1): seen.append(s_gcref1) assert s_gcref1 == s_gcref From fijal at codespeak.net Sun Nov 14 11:44:40 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 14 Nov 2010 11:44:40 +0100 (CET) Subject: [pypy-svn] r79072 - pypy/extradoc/planning Message-ID: <20101114104440.7852D282B90@codespeak.net> Author: fijal Date: Sun Nov 14 11:44:39 2010 New Revision: 79072 Modified: pypy/extradoc/planning/jit.txt Log: another task Modified: pypy/extradoc/planning/jit.txt ============================================================================== --- pypy/extradoc/planning/jit.txt (original) +++ pypy/extradoc/planning/jit.txt Sun Nov 14 11:44:39 2010 @@ -41,6 +41,7 @@ there is also manipulating of valuestackdepth and such. +- consider how much old style classes in stdlib hurt us. OPTIMIZATIONS ------------- From fijal at codespeak.net Sun Nov 14 11:45:52 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 14 Nov 2010 11:45:52 +0100 (CET) Subject: [pypy-svn] r79073 - pypy/trunk/pypy/jit/metainterp Message-ID: <20101114104552.0F5D05080B@codespeak.net> Author: fijal Date: Sun Nov 14 11:45:50 2010 New Revision: 79073 Modified: pypy/trunk/pypy/jit/metainterp/resoperation.py Log: I think NEWSTR and NEWUNICODE are as effect-free as NEWARRAY. Please prove me wrong. Modified: pypy/trunk/pypy/jit/metainterp/resoperation.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/resoperation.py (original) +++ pypy/trunk/pypy/jit/metainterp/resoperation.py Sun Nov 14 11:45:50 2010 @@ -444,6 +444,8 @@ 'NEW/0d', 'NEW_WITH_VTABLE/1', 'NEW_ARRAY/1d', + 'NEWSTR/1', + 'NEWUNICODE/1', 'FORCE_TOKEN/0', 'VIRTUAL_REF/2', # removed before it's passed to the backend '_NOSIDEEFFECT_LAST', # ----- end of no_side_effect operations ----- @@ -452,10 +454,8 @@ 'SETARRAYITEM_RAW/3d', 'SETFIELD_GC/2d', 'SETFIELD_RAW/2d', - 'NEWSTR/1', 'STRSETITEM/3', 'UNICODESETITEM/3', - 'NEWUNICODE/1', #'RUNTIMENEW/1', # ootype operation 'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier) 'DEBUG_MERGE_POINT/2', # debugging only From fijal at codespeak.net Sun Nov 14 11:58:54 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 14 Nov 2010 11:58:54 +0100 (CET) Subject: [pypy-svn] r79074 - in pypy/trunk/pypy/jit/metainterp: . test Message-ID: <20101114105854.1FF9C282B90@codespeak.net> Author: fijal Date: Sun Nov 14 11:58:53 2010 New Revision: 79074 Modified: pypy/trunk/pypy/jit/metainterp/resoperation.py pypy/trunk/pypy/jit/metainterp/test/test_resoperation.py Log: can_malloc hook. Also have is_call which is right now the same as can_raise, but let's be on the safe side Modified: pypy/trunk/pypy/jit/metainterp/resoperation.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/resoperation.py (original) +++ pypy/trunk/pypy/jit/metainterp/resoperation.py Sun Nov 14 11:58:53 2010 @@ -143,6 +143,13 @@ def can_raise(self): return rop._CANRAISE_FIRST <= self.getopnum() <= rop._CANRAISE_LAST + def can_malloc(self): + return (self.is_call() or + rop._MALLOC_FIRST <= self.getopnum() <= rop._MALLOC_LAST) + + def is_call(self): + return rop._CALL_FIRST <= self.getopnum() <= rop._CALL_LAST + def is_ovf(self): return rop._OVF_FIRST <= self.getopnum() <= rop._OVF_LAST @@ -441,11 +448,13 @@ 'GETARRAYITEM_RAW/2d', 'GETFIELD_GC/1d', 'GETFIELD_RAW/1d', + '_MALLOC_FIRST', 'NEW/0d', 'NEW_WITH_VTABLE/1', 'NEW_ARRAY/1d', 'NEWSTR/1', 'NEWUNICODE/1', + '_MALLOC_LAST', 'FORCE_TOKEN/0', 'VIRTUAL_REF/2', # removed before it's passed to the backend '_NOSIDEEFFECT_LAST', # ----- end of no_side_effect operations ----- @@ -465,6 +474,7 @@ 'COPYUNICODECONTENT/5', '_CANRAISE_FIRST', # ----- start of can_raise operations ----- + '_CALL_FIRST', 'CALL/*d', 'CALL_ASSEMBLER/*d', # call already compiled assembler 'CALL_MAY_FORCE/*d', @@ -473,6 +483,7 @@ #'OOSEND_PURE', # ootype operation 'CALL_PURE/*d', # removed before it's passed to the backend # CALL_PURE(result, func, arg_1,..,arg_n) + '_CALL_LAST', '_CANRAISE_LAST', # ----- end of can_raise operations ----- '_OVF_FIRST', # ----- start of is_ovf operations ----- Modified: pypy/trunk/pypy/jit/metainterp/test/test_resoperation.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_resoperation.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_resoperation.py Sun Nov 14 11:58:53 2010 @@ -61,3 +61,10 @@ assert op.getarglist() == ['a', 'b'] assert op.result == 'c' assert op.getdescr() is mydescr + +def test_can_malloc(): + mydescr = AbstractDescr() + assert rop.ResOperation(rop.rop.NEW, [], 'b').can_malloc() + call = rop.ResOperation(rop.rop.CALL, ['a', 'b'], 'c', descr=mydescr) + assert call.can_malloc() + assert not rop.ResOperation(rop.rop.INT_ADD, ['a', 'b'], 'c').can_malloc() From fijal at codespeak.net Sun Nov 14 12:02:25 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 14 Nov 2010 12:02:25 +0100 (CET) Subject: [pypy-svn] r79075 - pypy/trunk/pypy/jit/metainterp Message-ID: <20101114110225.608A05080B@codespeak.net> Author: fijal Date: Sun Nov 14 12:02:23 2010 New Revision: 79075 Modified: pypy/trunk/pypy/jit/metainterp/resoperation.py Log: refactor slightly Modified: pypy/trunk/pypy/jit/metainterp/resoperation.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/resoperation.py (original) +++ pypy/trunk/pypy/jit/metainterp/resoperation.py Sun Nov 14 12:02:23 2010 @@ -143,9 +143,12 @@ def can_raise(self): return rop._CANRAISE_FIRST <= self.getopnum() <= rop._CANRAISE_LAST + def is_malloc(self): + # a slightly different meaning from can_malloc + return rop._MALLOC_FIRST <= self.getopnum() <= rop._MALLOC_LAST + def can_malloc(self): - return (self.is_call() or - rop._MALLOC_FIRST <= self.getopnum() <= rop._MALLOC_LAST) + return self.is_call() or self.is_malloc() def is_call(self): return rop._CALL_FIRST <= self.getopnum() <= rop._CALL_LAST From hakanardo at codespeak.net Sun Nov 14 12:06:29 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Sun, 14 Nov 2010 12:06:29 +0100 (CET) Subject: [pypy-svn] r79076 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt Message-ID: <20101114110629.ACA02282B90@codespeak.net> Author: hakanardo Date: Sun Nov 14 12:06:28 2010 New Revision: 79076 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Log: no longer used Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Sun Nov 14 12:06:28 2010 @@ -25,14 +25,12 @@ class OptValue(object): __metaclass__ = extendabletype - _attrs_ = ('box', 'known_class', 'last_guard_index', 'level', 'intbound', - 'fromstart') + _attrs_ = ('box', 'known_class', 'last_guard_index', 'level', 'intbound') last_guard_index = -1 level = LEVEL_UNKNOWN known_class = None intbound = None - fromstart = False def __init__(self, box): self.box = box From fijal at codespeak.net Sun Nov 14 12:12:56 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 14 Nov 2010 12:12:56 +0100 (CET) Subject: [pypy-svn] r79077 - in pypy/trunk/pypy/jit/backend/llsupport: . test Message-ID: <20101114111256.F0C1D282B90@codespeak.net> Author: fijal Date: Sun Nov 14 12:12:55 2010 New Revision: 79077 Modified: pypy/trunk/pypy/jit/backend/llsupport/gc.py pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py Log: Implement an optimization - when we initialize the store don't call write barrier (this is for examples where new is followed immediately by setfield/arrayitem set) Modified: pypy/trunk/pypy/jit/backend/llsupport/gc.py ============================================================================== --- pypy/trunk/pypy/jit/backend/llsupport/gc.py (original) +++ pypy/trunk/pypy/jit/backend/llsupport/gc.py Sun Nov 14 12:12:55 2010 @@ -573,6 +573,9 @@ # GETFIELD_RAW from the array 'gcrefs.list'. # newops = [] + # we can only remember one malloc since the next malloc can possibly + # collect + last_malloc = None for op in operations: if op.getopnum() == rop.DEBUG_MERGE_POINT: continue @@ -590,22 +593,32 @@ [ConstInt(addr)], box, self.single_gcref_descr)) op.setarg(i, box) + if op.is_malloc(): + last_malloc = op.result + elif op.can_malloc(): + last_malloc = None # ---------- write barrier for SETFIELD_GC ---------- if op.getopnum() == rop.SETFIELD_GC: - v = op.getarg(1) - if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and - bool(v.value)): # store a non-NULL - self._gen_write_barrier(newops, op.getarg(0), v) - op = op.copy_and_change(rop.SETFIELD_RAW) + val = op.getarg(0) + # no need for a write barrier in the case of previous malloc + if val is not last_malloc: + v = op.getarg(1) + if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and + bool(v.value)): # store a non-NULL + self._gen_write_barrier(newops, op.getarg(0), v) + op = op.copy_and_change(rop.SETFIELD_RAW) # ---------- write barrier for SETARRAYITEM_GC ---------- if op.getopnum() == rop.SETARRAYITEM_GC: - v = op.getarg(2) - if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and - bool(v.value)): # store a non-NULL - # XXX detect when we should produce a - # write_barrier_from_array - self._gen_write_barrier(newops, op.getarg(0), v) - op = op.copy_and_change(rop.SETARRAYITEM_RAW) + val = op.getarg(0) + # no need for a write barrier in the case of previous malloc + if val is not last_malloc: + v = op.getarg(2) + if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and + bool(v.value)): # store a non-NULL + # XXX detect when we should produce a + # write_barrier_from_array + self._gen_write_barrier(newops, op.getarg(0), v) + op = op.copy_and_change(rop.SETARRAYITEM_RAW) # ---------- newops.append(op) del operations[:] Modified: pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py ============================================================================== --- pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py (original) +++ pypy/trunk/pypy/jit/backend/llsupport/test/test_gc.py Sun Nov 14 12:12:55 2010 @@ -6,7 +6,9 @@ from pypy.jit.backend.llsupport.gc import * from pypy.jit.backend.llsupport import symbolic from pypy.jit.metainterp.gc import get_description - +from pypy.jit.tool.oparser import parse +from pypy.rpython.lltypesystem.rclass import OBJECT, OBJECT_VTABLE +from pypy.jit.metainterp.test.test_optimizeopt import equaloplists def test_boehm(): gc_ll_descr = GcLLDescr_boehm(None, None, None) @@ -394,6 +396,68 @@ assert operations[1].getarg(2) == v_value assert operations[1].getdescr() == array_descr + def test_rewrite_assembler_initialization_store(self): + S = lltype.GcStruct('S', ('parent', OBJECT), + ('x', lltype.Signed)) + s_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True) + xdescr = get_field_descr(self.gc_ll_descr, S, 'x') + ops = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + # no write barrier + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) + + def test_rewrite_assembler_initialization_store_2(self): + S = lltype.GcStruct('S', ('parent', OBJECT), + ('x', lltype.Signed)) + s_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True) + wbdescr = self.gc_ll_descr.write_barrier_descr + xdescr = get_field_descr(self.gc_ll_descr, S, 'x') + ops = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + p3 = new_with_vtable(ConstClass(s_vtable)) + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + p3 = new_with_vtable(ConstClass(s_vtable)) + cond_call_gc_wb(p0, p1, descr=wbdescr) + setfield_raw(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) + + def test_rewrite_assembler_initialization_store_3(self): + A = lltype.GcArray(lltype.Ptr(lltype.GcStruct('S'))) + arraydescr = get_array_descr(self.gc_ll_descr, A) + ops = parse(""" + [p1] + p0 = new_array(3, descr=arraydescr) + setarrayitem_gc(p0, 0, p1, descr=arraydescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_array(3, descr=arraydescr) + setarrayitem_gc(p0, 0, p1, descr=arraydescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) class TestFrameworkMiniMark(TestFramework): gc = 'minimark' From fijal at codespeak.net Sun Nov 14 12:46:37 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 14 Nov 2010 12:46:37 +0100 (CET) Subject: [pypy-svn] r79078 - pypy/trunk/pypy/annotation Message-ID: <20101114114637.1BD19282B9C@codespeak.net> Author: fijal Date: Sun Nov 14 12:46:35 2010 New Revision: 79078 Modified: pypy/trunk/pypy/annotation/bookkeeper.py Log: Kill an apparently dead function Modified: pypy/trunk/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/trunk/pypy/annotation/bookkeeper.py (original) +++ pypy/trunk/pypy/annotation/bookkeeper.py Sun Nov 14 12:46:35 2010 @@ -87,13 +87,6 @@ else: return obj.knowntype.__name__ - def consider_tuple_iter(self, tup): - ctxt = "[%s]" % sys._getframe(4).f_code.co_name - if tup.is_constant(): - return ctxt, tup.const - else: - return ctxt, tuple([self.typerepr(x) for x in tup.items]) - def consider_tuple_random_getitem(self, tup): return tuple([self.typerepr(x) for x in tup.items]) From hakanardo at codespeak.net Sun Nov 14 15:46:47 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Sun, 14 Nov 2010 15:46:47 +0100 (CET) Subject: [pypy-svn] r79079 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt Message-ID: <20101114144647.920985080F@codespeak.net> Author: hakanardo Date: Sun Nov 14 15:46:45 2010 New Revision: 79079 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/fficall.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Log: Started to reconstruct values aswell (this is work in progress) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/fficall.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/fficall.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/fficall.py Sun Nov 14 15:46:45 2010 @@ -67,7 +67,7 @@ def __init__(self): self.funcinfo = None - def reconstruct_for_next_iteration(self): + def reconstruct_for_next_iteration(self, valuemap): return OptFfiCall() # FIXME: Should any status be saved for next iteration? Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py Sun Nov 14 15:46:45 2010 @@ -23,7 +23,7 @@ self.lazy_setfields = {} self.lazy_setfields_descrs = [] # keys (at least) of previous dict - def reconstruct_for_next_iteration(self): + def reconstruct_for_next_iteration(self, valuemap): self.force_all_lazy_setfields() assert not self.lazy_setfields_descrs assert not self.lazy_setfields Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py Sun Nov 14 15:46:45 2010 @@ -13,7 +13,7 @@ self.posponedop = None self.nextop = None - def reconstruct_for_next_iteration(self): + def reconstruct_for_next_iteration(self, valuemap): assert self.posponedop is None return self Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Sun Nov 14 15:46:45 2010 @@ -51,6 +51,16 @@ boxes.append(self.force_box()) already_seen[self.get_key_box()] = None + def get_reconstructed(self, optimizer, valuemap): + if self in valuemap: + return valuemap[self] + new = self.reconstruct_for_next_iteration(optimizer, valuemap) + valuemap[self] = new + return new + + def reconstruct_for_next_iteration(self, optimizer, valuemap): + return self + def get_args_for_fail(self, modifier): pass @@ -197,7 +207,7 @@ def turned_constant(self, value): pass - def reconstruct_for_next_iteration(self): + def reconstruct_for_next_iteration(self, valuemap): #return self.__class__() raise NotImplementedError @@ -240,25 +250,29 @@ o.force_at_end_of_preamble() def reconstruct_for_next_iteration(self): - optimizations = [o.reconstruct_for_next_iteration() for o in + valuemap = {} + optimizations = [o.reconstruct_for_next_iteration(valuemap) for o in self.optimizations] optimizations = self.optimizations new = Optimizer(self.metainterp_sd, self.loop, optimizations) - new.values = self.values + new.values = {} + for box, value in self.values.items(): + new.values[box] = value.get_reconstructed(new, valuemap) new.interned_refs = self.interned_refs - new.bool_boxes = self.bool_boxes - new.loop_invariant_results = self.loop_invariant_results + new.bool_boxes = {} + for value in new.bool_boxes.keys(): + new.bool_boxes[value.get_reconstructed(new, valuemap)] = None + + # FIXME: Move to rewrite.py + new.loop_invariant_results = {} + for key, value in self.loop_invariant_results.items(): + new.loop_invariant_results[key] = \ + value.get_reconstructed(new, valuemap) + new.pure_operations = self.pure_operations new.producer = self.producer assert self.posponedop is None - # FIXME: HACK!! Add a reconstruct_for_next_iteration to - # the values instead and reconstruct them in the same manner. - # That should also give us a clean solution for enabling - # OptString in the preamble that forces it's virtuals before - # the loop - for v in new.values.values(): - v.optimizer = new return new def turned_constant(self, value): Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py Sun Nov 14 15:46:45 2010 @@ -10,7 +10,7 @@ This includes already executed operations and constants. """ - def reconstruct_for_next_iteration(self): + def reconstruct_for_next_iteration(self, valuemap): return self def propagate_forward(self, op): Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py Sun Nov 14 15:46:45 2010 @@ -367,7 +367,7 @@ "Handling of strings and unicodes." enabled = True - def reconstruct_for_next_iteration(self): + def reconstruct_for_next_iteration(self, valuemap): self.enabled = True return self Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Sun Nov 14 15:46:45 2010 @@ -44,6 +44,9 @@ def _really_force(self): raise NotImplementedError("abstract base") + def reconstruct_for_next_iteration(self, optimizer, valuemap): + return optimizer.OptValue(self.force_box()) + def get_fielddescrlist_cache(cpu): if not hasattr(cpu, '_optimizeopt_fielddescrlist_cache'): result = descrlist_dict() @@ -132,6 +135,15 @@ else: boxes.append(self.box) + def reconstruct_for_next_iteration(self, optimizer, valuemap): + self.optimizer = optimizer + if self.box is None: + lst = self._get_field_descr_list() + for ofs in lst: + self._fields[ofs] = \ + self._fields[ofs].get_reconstructed(optimizer, valuemap) + return self + class VirtualValue(AbstractVirtualStructValue): level = optimizer.LEVEL_KNOWNCLASS @@ -224,11 +236,18 @@ else: boxes.append(self.box) + def reconstruct_for_next_iteration(self, optimizer, valuemap): + self.optimizer = optimizer + if self.box is None: + for i in range(len(self._items)): + self._items[i] = self._items[i].get_reconstructed(optimizer, + valuemap) + return self class OptVirtualize(optimizer.Optimization): "Virtualize objects until they escape." - def reconstruct_for_next_iteration(self): + def reconstruct_for_next_iteration(self, valuemap): return self def make_virtual(self, known_class, box, source_op=None): From afa at codespeak.net Sun Nov 14 23:49:41 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Sun, 14 Nov 2010 23:49:41 +0100 (CET) Subject: [pypy-svn] r79080 - in pypy/branch/fast-forward/pypy/module/pyexpat: . test Message-ID: <20101114224941.DBF0A282B90@codespeak.net> Author: afa Date: Sun Nov 14 23:49:39 2010 New Revision: 79080 Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py Log: Add the default UnknownEncodingHandler to pyexpat Modified: pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py (original) +++ pypy/branch/fast-forward/pypy/module/pyexpat/interp_pyexpat.py Sun Nov 14 23:49:39 2010 @@ -44,6 +44,12 @@ ('type', rffi.INT), ('quant', rffi.INT), ]) + XML_Encoding = rffi_platform.Struct('XML_Encoding', [ + ('map', rffi.CFixedArray(rffi.INT, 1)), + ('data', rffi.VOIDP), + ('convert', rffi.VOIDP), + ('release', rffi.VOIDP), + ]) for name in ['XML_PARAM_ENTITY_PARSING_NEVER', 'XML_PARAM_ENTITY_PARSING_UNLESS_STANDALONE', 'XML_PARAM_ENTITY_PARSING_ALWAYS']: @@ -56,6 +62,7 @@ globals()[k] = v XML_Content_Ptr.TO.become(rffi.CArray(XML_Content)) +XML_Encoding_Ptr = lltype.Ptr(XML_Encoding) def expat_external(*a, **kw): @@ -171,8 +178,7 @@ args = ', '.join(arg_names) wargs = ', '.join(warg_names) - if name in ['UnknownEncodingHandler', - 'ExternalEntityRefHandler', + if name in ['ExternalEntityRefHandler', 'NotStandaloneHandler']: result_type = rffi.INT result_converter = "space.int_w(w_result)" @@ -225,6 +231,30 @@ [XML_Parser, callback_type], lltype.Void) SETTERS[name] = (index, func, callback) +# special case for UnknownEncodingHandlerData: +# XML_SetUnknownEncodingHandler() needs an additional argument, +# and it's not modifiable via user code anyway +def UnknownEncodingHandlerData_callback(ll_userdata, name, info): + id = rffi.cast(lltype.Signed, ll_userdata) + userdata = global_storage.get_object(id) + space = userdata.space + parser = userdata.parser + + name = rffi.charp2str(name) + + try: + parser.UnknownEncodingHandler(space, name, info) + except OperationError, e: + parser._exc_info = e + XML_StopParser(parser.itself, XML_FALSE) + return 0 + return 1 +callback_type = lltype.Ptr(lltype.FuncType( + [rffi.VOIDP, rffi.CCHARP, XML_Encoding_Ptr], rffi.INT)) +XML_SetUnknownEncodingHandler = expat_external( + 'XML_SetUnknownEncodingHandler', + [XML_Parser, callback_type, rffi.VOIDP], lltype.Void) + ENUMERATE_SETTERS = unrolling_iterable(SETTERS.items()) # Declarations of external functions @@ -295,10 +325,9 @@ self._exc_info = None # Set user data for callback function - global_storage.get_nonmoving_id( - CallbackData(space, self), - id=rffi.cast(lltype.Signed, self.itself)) - XML_SetUserData(self.itself, rffi.cast(rffi.VOIDP, self.itself)) + self.id = global_storage.get_nonmoving_id( + CallbackData(space, self)) + XML_SetUserData(self.itself, rffi.cast(rffi.VOIDP, self.id)) def __del__(self): if XML_ParserFree: # careful with CPython interpreter shutdown @@ -429,6 +458,27 @@ sethandler._annspecialcase_ = 'specialize:arg(2)' + all_chars = ''.join(chr(i) for i in range(256)) + + def UnknownEncodingHandler(self, space, name, info): + # Yes, supports only 8bit encodings + translationmap = space.unicode_w( + space.call_method( + space.wrap(self.all_chars), "decode", + space.wrap(name), space.wrap("replace"))) + + for i in range(256): + c = translationmap[i] + if c == u'\ufffd': + info.c_map[i] = rffi.cast(rffi.INT, -1) + else: + info.c_map[i] = rffi.cast(rffi.INT, c) + info.c_data = lltype.nullptr(rffi.VOIDP.TO) + info.c_convert = lltype.nullptr(rffi.VOIDP.TO) + info.c_release = lltype.nullptr(rffi.VOIDP.TO) + return True + + def setattr(self, space, name, w_value): if name == "namespace_prefixes": XML_SetReturnNSTriplet(self.itself, space.int_w(w_value)) @@ -665,6 +715,9 @@ space.wrap('XML_ParserCreate failed')) parser = W_XMLParserType(space, xmlparser, w_intern) + XML_SetUnknownEncodingHandler( + parser.itself, UnknownEncodingHandlerData_callback, + rffi.cast(rffi.VOIDP, parser.id)) return space.wrap(parser) ParserCreate.unwrap_spec = [ObjSpace, W_Root, W_Root, W_Root] Modified: pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py (original) +++ pypy/branch/fast-forward/pypy/module/pyexpat/test/test_parser.py Sun Nov 14 23:49:39 2010 @@ -50,6 +50,14 @@ def gotText(text): assert text == u"caf\xe9" p.CharacterDataHandler = gotText - assert p.returns_unicode p.Parse(xml) + def test_python_encoding(self): + # This name is not knonwn by expat + xml = "caf\xe9" + import pyexpat + p = pyexpat.ParserCreate() + def gotText(text): + assert text == u"caf\xe9" + p.CharacterDataHandler = gotText + p.Parse(xml) From afa at codespeak.net Mon Nov 15 00:24:31 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 00:24:31 +0100 (CET) Subject: [pypy-svn] r79081 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101114232431.D0BC25080F@codespeak.net> Author: afa Date: Mon Nov 15 00:24:30 2010 New Revision: 79081 Modified: pypy/branch/fast-forward/pypy/objspace/std/newformat.py pypy/branch/fast-forward/pypy/objspace/std/test/test_newformat.py Log: format(s, '.3') should truncate the string. Test and fix. Modified: pypy/branch/fast-forward/pypy/objspace/std/newformat.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/newformat.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/newformat.py Mon Nov 15 00:24:30 2010 @@ -443,8 +443,11 @@ msg = "'=' alignment not allowed in string format specifier" raise OperationError(space.w_ValueError, space.wrap(msg)) length = len(string) - if self._precision != -1 and length >= self._precision: - length = self._precision + precision = self._precision + if precision != -1 and length >= precision: + assert precision >= 0 + length = precision + string = string[:precision] if self._fill_char == "\0": self._fill_char = self._lit(" ")[0] self._calc_padding(string, length) Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_newformat.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_newformat.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_newformat.py Mon Nov 15 00:24:30 2010 @@ -95,6 +95,9 @@ assert format(self.s("h"), "c<3") == self.s("hcc") raises(ValueError, format, self.s("blah"), "=12") + def test_precision(self): + assert format(self.s("abcdef"), ".3") == self.s("abc") + def test_non_ascii_presentation(self): raises(ValueError, format, self.s(""), "\x234") From afa at codespeak.net Mon Nov 15 00:46:02 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 00:46:02 +0100 (CET) Subject: [pypy-svn] r79082 - in pypy/branch/fast-forward/pypy/interpreter: astcompiler test Message-ID: <20101114234602.CCFEE282B90@codespeak.net> Author: afa Date: Mon Nov 15 00:46:01 2010 New Revision: 79082 Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py pypy/branch/fast-forward/pypy/interpreter/test/test_compiler.py Log: UnicodeError in literal should be turned into a SyntaxError Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py Mon Nov 15 00:46:01 2010 @@ -1087,9 +1087,15 @@ encoding = self.compile_info.encoding flags = self.compile_info.flags unicode_literals = flags & consts.CO_FUTURE_UNICODE_LITERALS - sub_strings_w = [parsestring.parsestr(space, encoding, s.value, - unicode_literals) - for s in atom_node.children] + try: + sub_strings_w = [parsestring.parsestr(space, encoding, s.value, + unicode_literals) + for s in atom_node.children] + except error.OperationError, e: + if not e.match(space, space.w_UnicodeError): + raise + # UnicodeError in literal: turn into SyntaxError + self.error(e.errorstr(space), atom_node) # This implements implicit string concatenation. if len(sub_strings_w) > 1: w_sub_strings = space.newlist(sub_strings_w) Modified: pypy/branch/fast-forward/pypy/interpreter/test/test_compiler.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/test/test_compiler.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/test/test_compiler.py Mon Nov 15 00:46:01 2010 @@ -206,25 +206,21 @@ assert not space.eq_w(w_const, space.wrap("b")) assert not space.eq_w(w_const, space.wrap("c")) - _unicode_error_kind = "w_UnicodeError" - def test_unicodeliterals(self): - w_error = getattr(self.space, self._unicode_error_kind) - e = py.test.raises(OperationError, self.eval_string, "u'\\Ufffffffe'") ex = e.value ex.normalize_exception(self.space) - assert ex.match(self.space, w_error) + assert ex.match(self.space, self.space.w_SyntaxError) e = py.test.raises(OperationError, self.eval_string, "u'\\Uffffffff'") ex = e.value ex.normalize_exception(self.space) - assert ex.match(self.space, w_error) + assert ex.match(self.space, self.space.w_SyntaxError) e = py.test.raises(OperationError, self.eval_string, "u'\\U%08x'" % 0x110000) ex = e.value ex.normalize_exception(self.space) - assert ex.match(self.space, w_error) + assert ex.match(self.space, self.space.w_SyntaxError) def test_unicode_docstring(self): space = self.space From hakanardo at codespeak.net Mon Nov 15 09:36:41 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Mon, 15 Nov 2010 09:36:41 +0100 (CET) Subject: [pypy-svn] r79084 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt Message-ID: <20101115083641.8248E5080F@codespeak.net> Author: hakanardo Date: Mon Nov 15 09:36:39 2010 New Revision: 79084 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/fficall.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Log: Reconstructs all the OptValus after the preamble Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/fficall.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/fficall.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/fficall.py Mon Nov 15 09:36:39 2010 @@ -67,7 +67,7 @@ def __init__(self): self.funcinfo = None - def reconstruct_for_next_iteration(self, valuemap): + def reconstruct_for_next_iteration(self, optimizer, valuemap): return OptFfiCall() # FIXME: Should any status be saved for next iteration? Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/heap.py Mon Nov 15 09:36:39 2010 @@ -23,13 +23,35 @@ self.lazy_setfields = {} self.lazy_setfields_descrs = [] # keys (at least) of previous dict - def reconstruct_for_next_iteration(self, valuemap): + def reconstruct_for_next_iteration(self, optimizer, valuemap): self.force_all_lazy_setfields() assert not self.lazy_setfields_descrs assert not self.lazy_setfields new = OptHeap() - new.cached_fields = self.cached_fields - new.cached_arrayitems = self.cached_arrayitems + + for descr, d in self.cached_fields.items(): + newd = {} + new.cached_fields[descr] = newd + for value, fieldvalue in d.items(): + newd[value.get_reconstructed(optimizer, valuemap)] = \ + fieldvalue.get_reconstructed(optimizer, valuemap) + + new.cached_arrayitems = {} + for descr, d in self.cached_arrayitems.items(): + newd = {} + new.cached_arrayitems[descr] = newd + for value, cache in d.items(): + newcache = CachedArrayItems() + newd[value.get_reconstructed(optimizer, valuemap)] = newcache + if cache.var_index_item: + newcache.var_index_item = \ + cache.var_index_item.get_reconstructed(optimizer, valuemap) + if newcache.var_index_indexvalue: + newcache.var_index_indexvalue = \ + cache.var_index_indexvalue.get_reconstructed(optimizer, valuemap) + for index, fieldvalue in cache.fixed_index_items.items(): + newcache.fixed_index_items[index] = \ + fieldvalue.get_reconstructed(optimizer, valuemap) return new def clean_caches(self): Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/intbounds.py Mon Nov 15 09:36:39 2010 @@ -13,7 +13,7 @@ self.posponedop = None self.nextop = None - def reconstruct_for_next_iteration(self, valuemap): + def reconstruct_for_next_iteration(self, optimizer, valuemap): assert self.posponedop is None return self Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Mon Nov 15 09:36:39 2010 @@ -54,13 +54,17 @@ def get_reconstructed(self, optimizer, valuemap): if self in valuemap: return valuemap[self] - new = self.reconstruct_for_next_iteration(optimizer, valuemap) + new = self.reconstruct_for_next_iteration(optimizer) valuemap[self] = new + self.reconstruct_childs(new, valuemap) return new - def reconstruct_for_next_iteration(self, optimizer, valuemap): + def reconstruct_for_next_iteration(self, optimizer): return self + def reconstruct_childs(self, new, valuemap): + pass + def get_args_for_fail(self, modifier): pass @@ -207,7 +211,7 @@ def turned_constant(self, value): pass - def reconstruct_for_next_iteration(self, valuemap): + def reconstruct_for_next_iteration(self, optimizer=None, valuemap=None): #return self.__class__() raise NotImplementedError @@ -230,6 +234,9 @@ self.exception_might_have_happened = False self.newoperations = [] + self.set_optimizations(optimizations) + + def set_optimizations(self, optimizations): if optimizations: self.first_optimization = optimizations[0] for i in range(1, len(optimizations)): @@ -249,12 +256,15 @@ for o in self.optimizations: o.force_at_end_of_preamble() - def reconstruct_for_next_iteration(self): + def reconstruct_for_next_iteration(self, optimizer=None, valuemap=None): + assert optimizer is None + assert valuemap is None valuemap = {} - optimizations = [o.reconstruct_for_next_iteration(valuemap) for o in + new = Optimizer(self.metainterp_sd, self.loop) + optimizations = [o.reconstruct_for_next_iteration(new, valuemap) for o in self.optimizations] - optimizations = self.optimizations - new = Optimizer(self.metainterp_sd, self.loop, optimizations) + new.set_optimizations(optimizations) + new.values = {} for box, value in self.values.items(): new.values[box] = value.get_reconstructed(new, valuemap) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py Mon Nov 15 09:36:39 2010 @@ -10,7 +10,7 @@ This includes already executed operations and constants. """ - def reconstruct_for_next_iteration(self, valuemap): + def reconstruct_for_next_iteration(self, optimizer, valuemap): return self def propagate_forward(self, op): Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py Mon Nov 15 09:36:39 2010 @@ -367,7 +367,7 @@ "Handling of strings and unicodes." enabled = True - def reconstruct_for_next_iteration(self, valuemap): + def reconstruct_for_next_iteration(self, optimizer, valuemap): self.enabled = True return self Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/virtualize.py Mon Nov 15 09:36:39 2010 @@ -44,7 +44,7 @@ def _really_force(self): raise NotImplementedError("abstract base") - def reconstruct_for_next_iteration(self, optimizer, valuemap): + def reconstruct_for_next_iteration(self, _optimizer): return optimizer.OptValue(self.force_box()) def get_fielddescrlist_cache(cpu): @@ -135,15 +135,17 @@ else: boxes.append(self.box) - def reconstruct_for_next_iteration(self, optimizer, valuemap): + def reconstruct_for_next_iteration(self, optimizer): self.optimizer = optimizer - if self.box is None: - lst = self._get_field_descr_list() - for ofs in lst: - self._fields[ofs] = \ - self._fields[ofs].get_reconstructed(optimizer, valuemap) return self + def reconstruct_childs(self, new, valuemap): + assert isinstance(new, AbstractVirtualStructValue) + if new.box is None: + lst = self._get_field_descr_list() + for ofs in lst: + new._fields[ofs] = \ + self._fields[ofs].get_reconstructed(new.optimizer, valuemap) class VirtualValue(AbstractVirtualStructValue): level = optimizer.LEVEL_KNOWNCLASS @@ -236,18 +238,21 @@ else: boxes.append(self.box) - def reconstruct_for_next_iteration(self, optimizer, valuemap): + def reconstruct_for_next_iteration(self, optimizer): self.optimizer = optimizer - if self.box is None: - for i in range(len(self._items)): - self._items[i] = self._items[i].get_reconstructed(optimizer, - valuemap) return self + def reconstruct_childs(self, new, valuemap): + assert isinstance(new, VArrayValue) + if new.box is None: + for i in range(len(self._items)): + new._items[i] = self._items[i].get_reconstructed(new.optimizer, + valuemap) + class OptVirtualize(optimizer.Optimization): "Virtualize objects until they escape." - def reconstruct_for_next_iteration(self, valuemap): + def reconstruct_for_next_iteration(self, optimizer, valuemap): return self def make_virtual(self, known_class, box, source_op=None): From arigo at codespeak.net Mon Nov 15 13:12:59 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 13:12:59 +0100 (CET) Subject: [pypy-svn] r79086 - pypy/branch/jit-free Message-ID: <20101115121259.944E4282B9C@codespeak.net> Author: arigo Date: Mon Nov 15 13:12:57 2010 New Revision: 79086 Added: pypy/branch/jit-free/ - copied from r79085, pypy/trunk/ Log: A branch in which to implement freeing of old machine code. From arigo at codespeak.net Mon Nov 15 13:14:54 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 13:14:54 +0100 (CET) Subject: [pypy-svn] r79087 - in pypy/trunk/pypy/jit/metainterp: . test Message-ID: <20101115121454.A2C615080F@codespeak.net> Author: arigo Date: Mon Nov 15 13:14:53 2010 New Revision: 79087 Modified: pypy/trunk/pypy/jit/metainterp/test/test_ztranslation.py pypy/trunk/pypy/jit/metainterp/warmspot.py Log: Remove the argument not used any more, for this function only called from test_ztranslation. Modified: pypy/trunk/pypy/jit/metainterp/test/test_ztranslation.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_ztranslation.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_ztranslation.py Mon Nov 15 13:14:53 2010 @@ -79,7 +79,7 @@ res = ll_meta_interp(main, [40, 5], CPUClass=self.CPUClass, type_system=self.type_system) assert res == main(40, 5) - res = rpython_ll_meta_interp(main, [40, 5], loops=2, + res = rpython_ll_meta_interp(main, [40, 5], CPUClass=self.CPUClass, type_system=self.type_system, optimizer=OPTIMIZER_FULL, @@ -120,7 +120,7 @@ res = ll_meta_interp(main, [40], CPUClass=self.CPUClass, type_system=self.type_system) assert res == main(40) - res = rpython_ll_meta_interp(main, [40], loops=2, CPUClass=self.CPUClass, + res = rpython_ll_meta_interp(main, [40], CPUClass=self.CPUClass, type_system=self.type_system, optimizer=OPTIMIZER_FULL, ProfilerClass=Profiler) Modified: pypy/trunk/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/warmspot.py (original) +++ pypy/trunk/pypy/jit/metainterp/warmspot.py Mon Nov 15 13:14:53 2010 @@ -98,8 +98,7 @@ repeat -= 1 return res -def rpython_ll_meta_interp(function, args, backendopt=True, - loops='not used right now', **kwds): +def rpython_ll_meta_interp(function, args, backendopt=True, **kwds): return ll_meta_interp(function, args, backendopt=backendopt, translate_support_code=True, **kwds) From arigo at codespeak.net Mon Nov 15 13:16:51 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 13:16:51 +0100 (CET) Subject: [pypy-svn] r79088 - in pypy/trunk/pypy/rpython/memory/gc: . test Message-ID: <20101115121651.6AA25282B9C@codespeak.net> Author: arigo Date: Mon Nov 15 13:16:49 2010 New Revision: 79088 Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py pypy/trunk/pypy/rpython/memory/gc/test/test_minimark.py Log: Write a big docstring about the env vars at the start of the file. Implement and test the new PYPY_GC_GROWTH. See docstring for what it does :-) Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Mon Nov 15 13:16:49 2010 @@ -1,3 +1,36 @@ +""" MiniMark GC. + +Environment variables can be used to fine-tune the following parameters: + + PYPY_GC_NURSERY The nursery size. Defaults to half the size of + the L2 cache. Try values like '1.2MB'. + + PYPY_GC_MAJOR_COLLECT Major collection memory factor. Default is '1.82', + which means trigger a major collection when the + memory consumed equals 1.82 times the memory + really used at the end of the previous major + collection. + + PYPY_GC_GROWTH Major collection threshold's max growth rate. + Default is '1.3'. Useful to collect more often + than normally on sudden memory growth, e.g. when + there is a temporary peak in memory usage. + + PYPY_GC_MAX The max heap size. If coming near this limit, it + will first collect more often, then raise an + RPython MemoryError, and if that is not enough, + crash the program with a fatal error. Try values + like '1.6GB'. + + PYPY_GC_MIN Don't collect while the memory size is below this + limit. Useful to avoid spending all the time in + the GC in very small programs. Defaults to 8 + times the nursery. +""" +# XXX Should find a way to bound the major collection threshold by the +# XXX total addressable size. Maybe by keeping some minimarkpage arenas +# XXX pre-reserved, enough for a few nursery collections? What about +# XXX raw-malloced memory? import sys from pypy.rpython.lltypesystem import lltype, llmemory, llarena, llgroup from pypy.rpython.lltypesystem.lloperation import llop @@ -87,13 +120,8 @@ TRANSLATION_PARAMS = { # Automatically adjust the size of the nursery and the - # 'major_collection_threshold' from the environment. For - # 'nursery_size' it will look it up in the env var - # PYPY_GC_NURSERY and fall back to half the size of - # the L2 cache. For 'major_collection_threshold' it will look - # it up in the env var PYPY_GC_MAJOR_COLLECT. It also sets - # 'max_heap_size' to PYPY_GC_MAX. Finally, PYPY_GC_MIN sets - # the minimal value of 'next_major_collection_threshold'. + # 'major_collection_threshold' from the environment. + # See docstring at the start of the file. "read_from_env": True, # The size of the nursery. Note that this is only used as a @@ -121,6 +149,13 @@ # we trigger the next major collection. "major_collection_threshold": 1.82, + # Threshold to avoid that the total heap size grows by a factor of + # major_collection_threshold at every collection: it can only + # grow at most by the following factor from one collection to the + # next. Used e.g. when there is a sudden, temporary peak in memory + # usage; this avoids that the upper bound grows too fast. + "growth_rate_max": 1.3, + # The number of array indices that are mapped to a single bit in # write_barrier_from_array(). Must be a power of two. The default # value of 128 means that card pages are 512 bytes (1024 on 64-bits) @@ -146,6 +181,7 @@ arena_size=64*WORD, small_request_threshold=5*WORD, major_collection_threshold=2.5, + growth_rate_max=2.5, # for tests card_page_indices=0, large_object=8*WORD, large_object_gcptrs=10*WORD, @@ -157,6 +193,7 @@ self.nursery_size = nursery_size self.small_request_threshold = small_request_threshold self.major_collection_threshold = major_collection_threshold + self.growth_rate_max = growth_rate_max self.num_major_collects = 0 self.min_heap_size = 0.0 self.max_heap_size = 0.0 @@ -257,9 +294,13 @@ newsize = max(newsize, minsize) # major_coll = base.read_float_from_env('PYPY_GC_MAJOR_COLLECT') - if major_coll >= 1.0: + if major_coll > 1.0: self.major_collection_threshold = major_coll # + growth = base.read_float_from_env('PYPY_GC_GROWTH') + if growth > 1.0: + self.growth_rate_max = growth + # min_heap_size = base.read_uint_from_env('PYPY_GC_MIN') if min_heap_size > 0: self.min_heap_size = float(min_heap_size) @@ -295,11 +336,19 @@ # initialize the threshold self.min_heap_size = max(self.min_heap_size, self.nursery_size * self.major_collection_threshold) + self.next_major_collection_threshold = self.min_heap_size self.set_major_threshold_from(0.0) debug_stop("gc-set-nursery-size") - def set_major_threshold_from(self, threshold): + + def set_major_threshold_from(self, threshold, reserving_size=0): # Set the next_major_collection_threshold. + threshold_max = (self.next_major_collection_threshold * + self.growth_rate_max) + if threshold > threshold_max: + threshold = threshold_max + # + threshold += reserving_size if threshold < self.min_heap_size: threshold = self.min_heap_size # @@ -1198,8 +1247,8 @@ # have allocated 'major_collection_threshold' times more than # we currently have. bounded = self.set_major_threshold_from( - (self.get_total_memory_used() * self.major_collection_threshold) - + reserving_size) + self.get_total_memory_used() * self.major_collection_threshold, + reserving_size) # # Max heap size: gives an upper bound on the threshold. If we # already have at least this much allocated, raise MemoryError. Modified: pypy/trunk/pypy/rpython/memory/gc/test/test_minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/test/test_minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/test/test_minimark.py Mon Nov 15 13:16:49 2010 @@ -28,3 +28,42 @@ assert gc.card_marking_bytes_for_length(P+P+1) == 3 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P) == 8 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P+1) == 9 + +def test_set_major_threshold(): + gc = MiniMarkGC(None, major_collection_threshold=2.0, + growth_rate_max=1.5) + gc.min_heap_size = 100.0 + gc.max_heap_size = 300.0 + gc.next_major_collection_threshold = 0.0 + # first, we don't grow past min_heap_size + for i in range(5): + gc.set_major_threshold_from(100.0) + assert gc.next_major_collection_threshold == 100.0 + # then we grow a lot + b = gc.set_major_threshold_from(100 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 150.0 + b = gc.set_major_threshold_from(150 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 225.0 + b = gc.set_major_threshold_from(225 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 # max reached + b = gc.set_major_threshold_from(300 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 + # then we shrink instantly + b = gc.set_major_threshold_from(100.0) + assert b is False + assert gc.next_major_collection_threshold == 100.0 + # then we grow a bit + b = gc.set_major_threshold_from(100 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 125.0 + b = gc.set_major_threshold_from(125 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 156.25 + # check that we cannot shrink below min_heap_size + b = gc.set_major_threshold_from(42.7) + assert b is False + assert gc.next_major_collection_threshold == 100.0 From arigo at codespeak.net Mon Nov 15 13:21:37 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 13:21:37 +0100 (CET) Subject: [pypy-svn] r79089 - pypy/trunk/pypy/jit/backend/x86 Message-ID: <20101115122137.DC3DE5080F@codespeak.net> Author: arigo Date: Mon Nov 15 13:21:36 2010 New Revision: 79089 Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py Log: Improve a bit the code generated for write barriers. (Tiny improvement on 32-bit, bigger on 64-bit.) Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/assembler.py (original) +++ pypy/trunk/pypy/jit/backend/x86/assembler.py Mon Nov 15 13:21:36 2010 @@ -1773,13 +1773,18 @@ self.implement_guard(guard_token, 'L') def genop_discard_cond_call_gc_wb(self, op, arglocs): - # use 'mc._mc' directly instead of 'mc', to avoid - # bad surprizes if the code buffer is mostly full + # Write code equivalent to write_barrier() in the GC: it checks + # a flag in the object at arglocs[0], and if set, it calls the + # function remember_young_pointer() from the GC. The two arguments + # to the call are in arglocs[:2]. The rest, arglocs[2:], contains + # registers that need to be saved and restored across the call. descr = op.getdescr() if we_are_translated(): cls = self.cpu.gc_ll_descr.has_write_barrier_class() assert cls is not None and isinstance(descr, cls) loc_base = arglocs[0] + # ensure that enough bytes are available to write the whole + # following piece of code atomically (for the JZ) self.mc.ensure_bytes_available(256) self.mc.TEST8_mi((loc_base.value, descr.jit_wb_if_flag_byteofs), descr.jit_wb_if_flag_singlebyte) @@ -1787,36 +1792,39 @@ jz_location = self.mc.get_relative_pos() # the following is supposed to be the slow path, so whenever possible # we choose the most compact encoding over the most efficient one. - # XXX improve a bit, particularly for IS_X86_64. - for i in range(len(arglocs)-1, -1, -1): + if IS_X86_32: + limit = -1 # push all arglocs on the stack + elif IS_X86_64: + limit = 1 # push only arglocs[2:] on the stack + for i in range(len(arglocs)-1, limit, -1): loc = arglocs[i] if isinstance(loc, RegLoc): self.mc.PUSH_r(loc.value) else: - if IS_X86_64: - self.mc.MOV_ri(X86_64_SCRATCH_REG.value, loc.getint()) - self.mc.PUSH_r(X86_64_SCRATCH_REG.value) - else: - self.mc.PUSH_i32(loc.getint()) - + assert not IS_X86_64 # there should only be regs in arglocs[2:] + self.mc.PUSH_i32(loc.getint()) if IS_X86_64: # We clobber these registers to pass the arguments, but that's # okay, because consider_cond_call_gc_wb makes sure that any - # caller-save registers with values in them are present in arglocs, - # so they are saved on the stack above and restored below - self.mc.MOV_rs(edi.value, 0) - self.mc.MOV_rs(esi.value, 8) + # caller-save registers with values in them are present in + # arglocs[2:] too, so they are saved on the stack above and + # restored below. + remap_frame_layout(self, arglocs[:2], [edi, esi], + X86_64_SCRATCH_REG) # misaligned stack in the call, but it's ok because the write barrier # is not going to call anything more. Also, this assumes that the - # write barrier does not touch the xmm registers. + # write barrier does not touch the xmm registers. (Slightly delicate + # assumption, given that the write barrier can end up calling the + # platform's malloc() from AddressStack.append(). XXX may need to + # be done properly) self.mc.CALL(imm(descr.get_write_barrier_fn(self.cpu))) - for i in range(len(arglocs)): + if IS_X86_32: + self.mc.ADD_ri(esp.value, 2*WORD) + for i in range(2, len(arglocs)): loc = arglocs[i] - if isinstance(loc, RegLoc): - self.mc.POP_r(loc.value) - else: - self.mc.ADD_ri(esp.value, WORD) # ignore the pushed constant + assert isinstance(loc, RegLoc) + self.mc.POP_r(loc.value) # patch the JZ above offset = self.mc.get_relative_pos() - jz_location assert 0 < offset <= 127 From afa at codespeak.net Mon Nov 15 13:23:18 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 13:23:18 +0100 (CET) Subject: [pypy-svn] r79090 - pypy/branch/fast-forward/pypy/interpreter/astcompiler Message-ID: <20101115122318.67A7C282B9C@codespeak.net> Author: afa Date: Mon Nov 15 13:23:16 2010 New Revision: 79090 Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py Log: Fix translation Modified: pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/astcompiler/astbuilder.py Mon Nov 15 13:23:16 2010 @@ -1096,6 +1096,7 @@ raise # UnicodeError in literal: turn into SyntaxError self.error(e.errorstr(space), atom_node) + sub_strings_w = [] # please annotator # This implements implicit string concatenation. if len(sub_strings_w) > 1: w_sub_strings = space.newlist(sub_strings_w) From arigo at codespeak.net Mon Nov 15 13:32:58 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 13:32:58 +0100 (CET) Subject: [pypy-svn] r79091 - in pypy/branch/jit-free/pypy/jit/backend: llsupport x86 Message-ID: <20101115123258.4F2635080F@codespeak.net> Author: arigo Date: Mon Nov 15 13:32:56 2010 New Revision: 79091 Modified: pypy/branch/jit-free/pypy/jit/backend/llsupport/gc.py pypy/branch/jit-free/pypy/jit/backend/x86/assembler.py pypy/branch/jit-free/pypy/jit/backend/x86/regalloc.py Log: For future reference: record places that leak even with loop freeing, and that we need to fix, with YYY. Modified: pypy/branch/jit-free/pypy/jit/backend/llsupport/gc.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/llsupport/gc.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/llsupport/gc.py Mon Nov 15 13:32:56 2010 @@ -254,7 +254,7 @@ def _enlarge_gcmap(self): newlength = 250 + self._gcmap_maxlength * 2 newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw', - track_allocation=False) + track_allocation=False) # YYY leak oldgcmap = self._gcmap for i in range(self._gcmap_curlength): newgcmap[i] = oldgcmap[i] @@ -311,7 +311,7 @@ length = len(shape) compressed = lltype.malloc(self.CALLSHAPE_ARRAY, length, flavor='raw', - track_allocation=False) # memory leak + track_allocation=False) # YYY leak for i in range(length): compressed[length-1-i] = rffi.cast(rffi.UCHAR, shape[i]) return llmemory.cast_ptr_to_adr(compressed) Modified: pypy/branch/jit-free/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/x86/assembler.py Mon Nov 15 13:32:56 2010 @@ -409,8 +409,9 @@ def _register_counter(self): if self._debug: + # YYY leak -- just put it in self.mc instead struct = lltype.malloc(DEBUG_COUNTER, flavor='raw', - track_allocation=False) # known to leak + track_allocation=False) struct.i = 0 self.loop_run_counters.append((len(self.loop_run_counters), struct)) Modified: pypy/branch/jit-free/pypy/jit/backend/x86/regalloc.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/x86/regalloc.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/x86/regalloc.py Mon Nov 15 13:32:56 2010 @@ -70,7 +70,7 @@ def _get_new_array(self): n = self.BASE_CONSTANT_SIZE # known to leak - self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n, + self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n, # YYY leak flavor='raw', track_allocation=False) self.cur_array_free = n _get_new_array._dont_inline_ = True From cfbolz at codespeak.net Mon Nov 15 13:33:40 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 15 Nov 2010 13:33:40 +0100 (CET) Subject: [pypy-svn] r79092 - pypy/branch/jit-starargs Message-ID: <20101115123340.010FB282B9C@codespeak.net> Author: cfbolz Date: Mon Nov 15 13:33:39 2010 New Revision: 79092 Added: pypy/branch/jit-starargs/ (props changed) - copied from r79091, pypy/trunk/ Log: a probably short-lived branch for making the JIT work with *args better. From arigo at codespeak.net Mon Nov 15 13:33:51 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 13:33:51 +0100 (CET) Subject: [pypy-svn] r79093 - pypy/branch/jit-free/pypy/jit/backend Message-ID: <20101115123351.8CA7150810@codespeak.net> Author: arigo Date: Mon Nov 15 13:33:49 2010 New Revision: 79093 Modified: pypy/branch/jit-free/pypy/jit/backend/model.py Log: Improve the backend interface to contain free_loop_and_bridges(). Modified: pypy/branch/jit-free/pypy/jit/backend/model.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/model.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/model.py Mon Nov 15 13:33:49 2010 @@ -10,15 +10,24 @@ def __init__(self): self.fail_descr_list = [] + self.fail_descr_free_list = [] def get_fail_descr_number(self, descr): assert isinstance(descr, history.AbstractFailDescr) n = descr.index if n < 0: lst = self.fail_descr_list - n = len(lst) - lst.append(descr) + if len(self.fail_descr_free_list) > 0: + n = self.fail_descr_free_list.pop() + assert lst[n] is None + lst[n] = descr + else: + n = len(lst) + lst.append(descr) descr.index = n + looptoken = descr.original_loop_token + if looptoken is not None: + looptoken.faildescr_indices.append(n) return n def get_fail_descr_from_number(self, n): @@ -113,6 +122,24 @@ oldlooptoken so that from now own they will call newlooptoken.""" raise NotImplementedError + def free_loop_and_bridges(self, looptoken): + """This method is called to free resources (machine code, + references to resume guards, etc.) allocated by the compilation + of a loop and all bridges attached to it. After this call, the + frontend cannot use this compiled loop any more; in fact, it + guarantees that at the point of the call to free_code_group(), + none of the corresponding assembler is currently running. + """ + # The base class provides a limited implementation: freeing the + # resume descrs. This is already quite helpful, because the + # resume descrs are the largest consumers of memory (about 3x + # more than the assembler, in the case of the x86 backend). + lst = self.fail_descr_list + for n in looptoken.faildescr_indices: + lst[n] = None + self.fail_descr_free_list.extend(looptoken.faildescr_indices) + # We expect 'looptoken' to be itself garbage-collected soon. + @staticmethod def sizeof(S): raise NotImplementedError From arigo at codespeak.net Mon Nov 15 13:39:56 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 13:39:56 +0100 (CET) Subject: [pypy-svn] r79094 - in pypy/branch/jit-free/pypy: jit/metainterp jit/metainterp/test rlib Message-ID: <20101115123956.16502282B9C@codespeak.net> Author: arigo Date: Mon Nov 15 13:39:54 2010 New Revision: 79094 Added: pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Modified: pypy/branch/jit-free/pypy/jit/metainterp/blackhole.py pypy/branch/jit-free/pypy/jit/metainterp/compile.py pypy/branch/jit-free/pypy/jit/metainterp/history.py pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py pypy/branch/jit-free/pypy/jit/metainterp/resume.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_compile.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_optimizeopt.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmspot.py pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py pypy/branch/jit-free/pypy/rlib/jit.py Log: In-progress: implement the new memmgr.py. Contains some kind of clean-up: now the ResumeDescrs contain, instead of 'original_greenkey', just 'original_loop_token' that points to the loop token corresponding to the root of the loop in which the guard is. Move 'greenkey' there too. Not done: see comment in test_memmgr.py. Modified: pypy/branch/jit-free/pypy/jit/metainterp/blackhole.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/blackhole.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/blackhole.py Mon Nov 15 13:39:54 2010 @@ -1362,14 +1362,12 @@ # We will continue to loop in _run_forever() from the parent level. return blackholeinterp, lle -def resume_in_blackhole(metainterp_sd, jitdriver_sd, resumedescr, - all_virtuals=None): +def resume_in_blackhole(metainterp_sd, resumedescr, all_virtuals=None): from pypy.jit.metainterp.resume import blackhole_from_resumedata debug_start('jit-blackhole') metainterp_sd.profiler.start_blackhole() blackholeinterp = blackhole_from_resumedata( metainterp_sd.blackholeinterpbuilder, - jitdriver_sd, resumedescr, all_virtuals) current_exc = blackholeinterp._prepare_resume_from_failure( Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/compile.py Mon Nov 15 13:39:54 2010 @@ -42,10 +42,11 @@ name = metainterp.staticdata.stats.name_for_new_loop() return TreeLoop(name) -def make_loop_token(nb_args, jitdriver_sd): +def make_loop_token(nb_args, jitdriver_sd, greenkey): loop_token = LoopToken() loop_token.specnodes = [prebuiltNotSpecNode] * nb_args loop_token.outermost_jitdriver_sd = jitdriver_sd + loop_token.outermost_greenkey = greenkey return loop_token # ____________________________________________________________ @@ -56,7 +57,6 @@ """ history = metainterp.history loop = create_empty_loop(metainterp) - loop.greenkey = greenkey loop.inputargs = history.inputargs for box in loop.inputargs: assert isinstance(box, Box) @@ -65,7 +65,7 @@ loop.operations = [h_ops[i].clone() for i in range(start, len(h_ops))] metainterp_sd = metainterp.staticdata jitdriver_sd = metainterp.jitdriver_sd - loop_token = make_loop_token(len(loop.inputargs), jitdriver_sd) + loop_token = make_loop_token(len(loop.inputargs), jitdriver_sd, greenkey) loop.token = loop_token loop.operations[-1].setdescr(loop_token) # patch the target of the JUMP try: @@ -98,6 +98,9 @@ loop_token = loop.token loop_token.number = n = globaldata.loopnumbering globaldata.loopnumbering += 1 + desc = metainterp_sd.warmrunnerdesc + if desc is not None: # for tests + desc.memory_manager.record_loop(loop_token) metainterp_sd.logger_ops.log_loop(loop.inputargs, loop.operations, n, type) if not we_are_translated(): @@ -142,32 +145,32 @@ pass class DoneWithThisFrameDescrVoid(_DoneWithThisFrameDescr): - def handle_fail(self, metainterp_sd, jitdriver_sd): - assert jitdriver_sd.result_type == history.VOID + def handle_fail(self, metainterp_sd): + #assert jitdriver_sd.result_type == history.VOID raise metainterp_sd.DoneWithThisFrameVoid() class DoneWithThisFrameDescrInt(_DoneWithThisFrameDescr): - def handle_fail(self, metainterp_sd, jitdriver_sd): - assert jitdriver_sd.result_type == history.INT + def handle_fail(self, metainterp_sd): + #assert jitdriver_sd.result_type == history.INT result = metainterp_sd.cpu.get_latest_value_int(0) raise metainterp_sd.DoneWithThisFrameInt(result) class DoneWithThisFrameDescrRef(_DoneWithThisFrameDescr): - def handle_fail(self, metainterp_sd, jitdriver_sd): - assert jitdriver_sd.result_type == history.REF + def handle_fail(self, metainterp_sd): + #assert jitdriver_sd.result_type == history.REF cpu = metainterp_sd.cpu result = cpu.get_latest_value_ref(0) cpu.clear_latest_values(1) raise metainterp_sd.DoneWithThisFrameRef(cpu, result) class DoneWithThisFrameDescrFloat(_DoneWithThisFrameDescr): - def handle_fail(self, metainterp_sd, jitdriver_sd): - assert jitdriver_sd.result_type == history.FLOAT + def handle_fail(self, metainterp_sd): + #assert jitdriver_sd.result_type == history.FLOAT result = metainterp_sd.cpu.get_latest_value_float(0) raise metainterp_sd.DoneWithThisFrameFloat(result) class ExitFrameWithExceptionDescrRef(_DoneWithThisFrameDescr): - def handle_fail(self, metainterp_sd, jitdriver_sd): + def handle_fail(self, metainterp_sd): cpu = metainterp_sd.cpu value = cpu.get_latest_value_ref(0) cpu.clear_latest_values(1) @@ -209,8 +212,9 @@ } class ResumeDescr(AbstractFailDescr): - def __init__(self, original_greenkey): - self.original_greenkey = original_greenkey + def __init__(self, original_loop_token): + assert isinstance(original_loop_token, history.LoopToken) + self.original_loop_token = original_loop_token class ResumeGuardDescr(ResumeDescr): _counter = 0 # if < 0, there is one counter per value; @@ -229,10 +233,6 @@ CNT_FLOAT = -0x60000000 CNT_MASK = 0x1FFFFFFF - def __init__(self, metainterp_sd, original_greenkey): - ResumeDescr.__init__(self, original_greenkey) - self.metainterp_sd = metainterp_sd - def store_final_boxes(self, guard_op, boxes): guard_op.setfailargs(boxes) self.guard_opnum = guard_op.getopnum() @@ -257,26 +257,29 @@ # a negative value self._counter = cnt | i - def handle_fail(self, metainterp_sd, jitdriver_sd): - if self.must_compile(metainterp_sd, jitdriver_sd): - return self._trace_and_compile_from_bridge(metainterp_sd, - jitdriver_sd) + def handle_fail(self, metainterp_sd): + if self.must_compile(metainterp_sd): + return self._trace_and_compile_from_bridge(metainterp_sd) else: from pypy.jit.metainterp.blackhole import resume_in_blackhole - resume_in_blackhole(metainterp_sd, jitdriver_sd, self) + resume_in_blackhole(metainterp_sd, self) assert 0, "unreachable" - def _trace_and_compile_from_bridge(self, metainterp_sd, jitdriver_sd): - # 'jitdriver_sd' corresponds to the outermost one, i.e. the one - # of the jit_merge_point where we started the loop, even if the - # loop itself may contain temporarily recursion into other - # jitdrivers. + def _trace_and_compile_from_bridge(self, metainterp_sd): from pypy.jit.metainterp.pyjitpl import MetaInterp - metainterp = MetaInterp(metainterp_sd, jitdriver_sd) + metainterp = MetaInterp(metainterp_sd, self.get_jitdriver_sd()) return metainterp.handle_guard_failure(self) _trace_and_compile_from_bridge._dont_inline_ = True - def must_compile(self, metainterp_sd, jitdriver_sd): + def get_jitdriver_sd(self): + # Returns the jitdriver_sd that corresponds to the outermost + # level, i.e. the level of the jit_merge_point where we started + # the loop, even if the loop itself may contain some recursion + # into other jitdrivers. + return self.original_loop_token.outermost_jitdriver_sd + + def must_compile(self, metainterp_sd): + jitdriver_sd = self.get_jitdriver_sd() trace_eagerness = jitdriver_sd.warmstate.trace_eagerness if self._counter >= 0: self._counter += 1 @@ -334,17 +337,18 @@ res.rd_pendingfields = self.rd_pendingfields def _clone_if_mutable(self): - res = ResumeGuardDescr(self.metainterp_sd, self.original_greenkey) + res = ResumeGuardDescr(self.original_loop_token) self.copy_all_attrbutes_into(res) return res class ResumeGuardForcedDescr(ResumeGuardDescr): - def __init__(self, metainterp_sd, original_greenkey, jitdriver_sd): - ResumeGuardDescr.__init__(self, metainterp_sd, original_greenkey) + def __init__(self, metainterp_sd, original_loop_token, jitdriver_sd): + ResumeGuardDescr.__init__(self, original_loop_token) + self.metainterp_sd = metainterp_sd self.jitdriver_sd = jitdriver_sd - def handle_fail(self, metainterp_sd, jitdriver_sd): + def handle_fail(self, metainterp_sd): # Failures of a GUARD_NOT_FORCED are never compiled, but # always just blackholed. First fish for the data saved when # the virtualrefs and virtualizable have been forced by @@ -354,8 +358,7 @@ all_virtuals = self.fetch_data(token) if all_virtuals is None: all_virtuals = [] - assert jitdriver_sd is self.jitdriver_sd - resume_in_blackhole(metainterp_sd, jitdriver_sd, self, all_virtuals) + resume_in_blackhole(metainterp_sd, self, all_virtuals) assert 0, "unreachable" @staticmethod @@ -408,7 +411,7 @@ def _clone_if_mutable(self): res = ResumeGuardForcedDescr(self.metainterp_sd, - self.original_greenkey, + self.original_loop_token, self.jitdriver_sd) self.copy_all_attrbutes_into(res) return res @@ -475,8 +478,11 @@ class ResumeFromInterpDescr(ResumeDescr): - def __init__(self, original_greenkey, redkey): - ResumeDescr.__init__(self, original_greenkey) + def __init__(self, metainterp, greenkey, redkey): + original_loop_token = make_loop_token(len(redkey), + metainterp.jitdriver_sd, + greenkey) + ResumeDescr.__init__(self, original_loop_token) self.redkey = redkey def compile_and_attach(self, metainterp, new_loop): @@ -487,18 +493,18 @@ metainterp_sd = metainterp.staticdata jitdriver_sd = metainterp.jitdriver_sd metainterp.history.inputargs = self.redkey - new_loop_token = make_loop_token(len(self.redkey), jitdriver_sd) - new_loop.greenkey = self.original_greenkey + new_loop_token = self.original_loop_token + original_greenkey = new_loop_token.outermost_greenkey new_loop.inputargs = self.redkey new_loop.token = new_loop_token send_loop_to_backend(metainterp_sd, new_loop, "entry bridge") # send the new_loop to warmspot.py, to be called directly the next time jitdriver_sd.warmstate.attach_unoptimized_bridge_from_interp( - self.original_greenkey, + original_greenkey, new_loop_token) # store the new loop in compiled_merge_points too old_loop_tokens = metainterp.get_compiled_merge_points( - self.original_greenkey) + original_greenkey) # it always goes at the end of the list, as it is the most # general loop token old_loop_tokens.append(new_loop_token) @@ -555,7 +561,7 @@ # ____________________________________________________________ class PropagateExceptionDescr(AbstractFailDescr): - def handle_fail(self, metainterp_sd, jitdriver_sd): + def handle_fail(self, metainterp_sd): cpu = metainterp_sd.cpu exception = cpu.grab_exc_value() raise metainterp_sd.ExitFrameWithExceptionRef(cpu, exception) @@ -569,7 +575,7 @@ """ # 'redboxes' is only used to know the types of red arguments. inputargs = [box.clonebox() for box in redboxes] - loop_token = make_loop_token(len(inputargs), jitdriver_sd) + loop_token = make_loop_token(len(inputargs), jitdriver_sd, greenboxes) # 'nb_red_args' might be smaller than len(redboxes), # because it doesn't include the virtualizable boxes. nb_red_args = jitdriver_sd.num_red_args Modified: pypy/branch/jit-free/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/history.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/history.py Mon Nov 15 13:39:54 2010 @@ -4,7 +4,7 @@ from pypy.rpython.ootypesystem import ootype from pypy.rlib.objectmodel import we_are_translated, r_dict, Symbolic from pypy.rlib.objectmodel import compute_hash, compute_unique_id -from pypy.rlib.rarithmetic import intmask +from pypy.rlib.rarithmetic import intmask, r_longlong from pypy.tool.uid import uid from pypy.conftest import option @@ -178,8 +178,9 @@ class AbstractFailDescr(AbstractDescr): index = -1 + original_loop_token = None - def handle_fail(self, metainterp_sd, jitdriver_sd): + def handle_fail(self, metainterp_sd): raise NotImplementedError def compile_and_attach(self, metainterp, new_loop): raise NotImplementedError @@ -727,13 +728,21 @@ generated assembler. """ terminating = False # see TerminatingLoopToken in compile.py + has_been_freed = False outermost_jitdriver_sd = None + outermost_greenkey = None # specnodes = ... # and more data specified by the backend when the loop is compiled - number = 0 def __init__(self, number=0): self.number = number + # See get_fail_descr_number() in backend/model.py: this growing + # list gives the 'descr_number' of all fail descrs that belong to + # this loop or to a bridge attached to it. + self.faildescr_indices = [] + # For memory management of assembled loops + self.contains_jumps_to = {} # set of other LoopTokens + self.generation = r_longlong(0) def repr_of_descr(self): return '' % self.number Added: pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py ============================================================================== --- (empty file) +++ pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py Mon Nov 15 13:39:54 2010 @@ -0,0 +1,106 @@ +import math +from pypy.rlib.rarithmetic import r_longlong +from pypy.rlib.objectmodel import we_are_translated + +# +# Logic to decide which loops are old and not used any more. +# +# Idea: We use the notion of a global 'current generation' which +# is, in practice, the total number of loops and bridges produced +# so far. When executing a loop: +# (1) we set 'generation' to -1 +# (2) we execute it +# (3) we set 'generation' to the latest generation +# (with a bit extra work to handle nested calls and to guarantee +# that 'generation' is always < 0 on a loop that is currently +# executing). +# +# A loop is said "old" if its generation is >= 0 but much smaller +# than the current generation. If a loop L is old, and if all +# other loops from which we can reach L through +# 'contains_jumps_to' are also old, then we can free L. +# + +class MemoryManager(object): + + def __init__(self, cpu): + self.cpu = cpu + self.check_frequency = -1 + # NB. use of r_longlong to be extremely far on the safe side: + # this is increasing by one after each loop or bridge is + # compiled, and it must not overflow. If the backend implements + # complete freeing in cpu.free_loop_and_bridges(), then it may + # be possible to get arbitrary many of them just by waiting long + # enough. But in this day and age, you'd still never have the + # patience of waiting for a slowly-increasing 64-bit number to + # overflow :-) + self.current_generation = r_longlong(0) + self.next_check = r_longlong(-1) + self.looptokens = [] + + def set_max_age(self, max_age, check_frequency=0): + if max_age <= 0: + self.next_check = r_longlong(-1) + else: + self.max_age = max_age + if check_frequency <= 0: + check_frequency = int(math.sqrt(max_age)) + self.check_frequency = check_frequency + self.next_check = self.current_generation + 1 + + def next_generation(self): + self.current_generation += 1 + if self.current_generation == self.next_check: + self._free_old_loops_now() + self.next_check = self.current_generation + self.check_frequency + + def enter_loop(self, looptoken): + if not we_are_translated(): + assert looptoken in self.looptokens + assert not looptoken.has_been_freed + if looptoken.generation >= 0: + looptoken.generation = -1 + else: + looptoken.generation -= 1 # nested enter_loop() + + def leave_loop(self, looptoken): + assert looptoken.generation < 0 + if looptoken.generation == -1: + looptoken.generation = self.current_generation + else: + looptoken.generation += 1 # nested leave_loop() + + def record_loop(self, looptoken): + looptoken.generation = self.current_generation + self.looptokens.append(looptoken) + + def _free_old_loops_now(self): + # + # Initialize '_is_young' on all loop tokens + max_generation = self.current_generation - self.max_age + youngloops = [] + for looptoken in self.looptokens: + if 0 <= looptoken.generation < max_generation: + looptoken._is_young = False # but may be turned to True later + else: + looptoken._is_young = True + youngloops.append(looptoken) + # + # Propagate forward the knowledge of "is a young loop" + while len(youngloops) > 0: + looptoken = youngloops.pop() + for jumptargettok in looptoken.contains_jumps_to: + if not jumptargettok._is_young: + jumptargettok._is_young = True + youngloops.append(jumptargettok) + # + # Now free all looptokens that still have _is_young == False. + i = 0 + while i < len(self.looptokens): + looptoken = self.looptokens[i] + if looptoken._is_young: + i += 1 + else: + self.looptokens[i] = self.looptokens[-1] + del self.looptokens[-1] + self.cpu.free_loop_and_bridges(looptoken) Modified: pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py Mon Nov 15 13:39:54 2010 @@ -1048,14 +1048,13 @@ else: moreargs = list(extraargs) metainterp_sd = metainterp.staticdata - original_greenkey = metainterp.resumekey.original_greenkey + original_loop_token = metainterp.resumekey.original_loop_token if opnum == rop.GUARD_NOT_FORCED: - resumedescr = compile.ResumeGuardForcedDescr(metainterp_sd, - original_greenkey, - metainterp.jitdriver_sd) + resumedescr = compile.ResumeGuardForcedDescr(metainterp.staticdata, + original_loop_token, + metainterp.jitdriver_sd) else: - resumedescr = compile.ResumeGuardDescr(metainterp_sd, - original_greenkey) + resumedescr = compile.ResumeGuardDescr(original_loop_token) guard_op = metainterp.history.record(opnum, moreargs, None, descr=resumedescr) virtualizable_boxes = None @@ -1370,6 +1369,9 @@ self.portal_trace_positions = [] self.free_frames_list = [] self.last_exc_value_box = None + # Increase here the generation recorded by the memory manager. + if self.staticdata.warmrunnerdesc is not None: # for tests + self.staticdata.warmrunnerdesc.memory_manager.next_generation() def perform_call(self, jitcode, boxes, greenkey=None): # causes the metainterp to enter the given subfunction @@ -1637,7 +1639,7 @@ num_green_args = self.jitdriver_sd.num_green_args original_greenkey = original_boxes[:num_green_args] redkey = original_boxes[num_green_args:] - self.resumekey = compile.ResumeFromInterpDescr(original_greenkey, + self.resumekey = compile.ResumeFromInterpDescr(self, original_greenkey, redkey) self.seen_loop_header_for_jdindex = -1 try: @@ -1660,7 +1662,7 @@ debug_stop('jit-tracing') def _handle_guard_failure(self, key): - original_greenkey = key.original_greenkey + original_greenkey = key.original_loop_token.outermost_greenkey # notice that here we just put the greenkey # use -1 to mark that we will have to give up # because we cannot reconstruct the beginning of the proper loop Modified: pypy/branch/jit-free/pypy/jit/metainterp/resume.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/resume.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/resume.py Mon Nov 15 13:39:54 2010 @@ -915,10 +915,11 @@ # ---------- when resuming for blackholing, get direct values ---------- -def blackhole_from_resumedata(blackholeinterpbuilder, jitdriver_sd, storage, +def blackhole_from_resumedata(blackholeinterpbuilder, storage, all_virtuals=None): resumereader = ResumeDataDirectReader(blackholeinterpbuilder.metainterp_sd, storage, all_virtuals) + jitdriver_sd = storage.get_jitdriver_sd() vinfo = jitdriver_sd.virtualizable_info ginfo = jitdriver_sd.greenfield_info vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_compile.py Mon Nov 15 13:39:54 2010 @@ -5,6 +5,7 @@ from pypy.jit.metainterp.compile import ResumeGuardDescr from pypy.jit.metainterp.compile import ResumeGuardCountersInt from pypy.jit.metainterp.compile import compile_tmp_callback +from pypy.jit.metainterp.compile import send_loop_to_backend from pypy.jit.metainterp import optimize, jitprof, typesystem, compile from pypy.jit.metainterp.test.test_optimizefindnode import LLtypeMixin from pypy.jit.tool.oparser import parse @@ -207,15 +208,62 @@ class ExitFrameWithExceptionRef(Exception): pass FakeMetaInterpSD.cpu = cpu - class FakeJitDriverSD: - pass cpu.set_future_value_int(0, -156) cpu.set_future_value_int(1, -178) cpu.set_future_value_int(2, -190) fail_descr = cpu.execute_token(loop_token) try: - fail_descr.handle_fail(FakeMetaInterpSD(), FakeJitDriverSD()) + fail_descr.handle_fail(FakeMetaInterpSD()) except FakeMetaInterpSD.ExitFrameWithExceptionRef, e: assert lltype.cast_opaque_ptr(lltype.Ptr(EXC), e.args[1]) == llexc else: assert 0, "should have raised" + + +def test_send_loop_to_backend(): + class FakeMetaInterpSD: + class globaldata: + loopnumbering = 17 + class warmrunnerdesc: + class memory_manager: + @staticmethod + def record_loop(token): + assert token is FakeLoop.token + token._recorded = True + class logger_ops: + @staticmethod + def log_loop(*args, **kwds): + pass + class profiler: + @staticmethod + def start_backend(): + pass + @staticmethod + def end_backend(): + pass + class cpu: + @staticmethod + def compile_loop(inputargs, operations, token): + assert inputargs is FakeLoop.inputargs + assert operations is FakeLoop.operations + assert token is FakeLoop.token + token._compiled = True + class stats: + @staticmethod + def add_new_loop(loop): + pass + def log(self, text): + pass + class FakeLoop: + inputargs = [] + operations = [] + class token: + number = 0 + _recorded = _compiled = False + def check_consistency(self): + pass + send_loop_to_backend(FakeMetaInterpSD(), FakeLoop(), "entry bridge") + assert FakeMetaInterpSD.globaldata.loopnumbering == 18 + assert FakeLoop.token.number == 17 + assert FakeLoop.token._recorded is True + assert FakeLoop.token._compiled is True Added: pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py ============================================================================== --- (empty file) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Mon Nov 15 13:39:54 2010 @@ -0,0 +1,158 @@ +from pypy.jit.metainterp.memmgr import MemoryManager +from pypy.jit.metainterp.history import LoopToken + +##missing: + +## in _free_old_loops_now(), remove looptoken from everywhere +## or mark it as freed + +## contains_jumps_to needs to be filled + + +class FakeCPU: + def free_loop_and_bridges(self, looptoken): + looptoken.has_been_freed = True +cpu = FakeCPU() + + +class TestMemoryManager: + + def test_disabled(self): + memmgr = MemoryManager(cpu) + memmgr.set_max_age(0) + tokens = [LoopToken() for i in range(10)] + for token in tokens: + memmgr.record_loop(token) + memmgr.next_generation() + for token in tokens: + assert not token.has_been_freed + + def test_basic(self): + memmgr = MemoryManager(cpu) + memmgr.set_max_age(3, 1) + tokens = [LoopToken() for i in range(10)] + for token in tokens: + memmgr.record_loop(token) + memmgr.next_generation() + for i in range(len(tokens)): + assert tokens[i].has_been_freed == (i < 7) + + def test_basic_2(self): + memmgr = MemoryManager(cpu) + memmgr.set_max_age(3, 1) + token = LoopToken() + memmgr.record_loop(token) + for i in range(10): + memmgr.next_generation() + assert token.has_been_freed == (i >= 3) + + def test_enter_loop_1(self): + memmgr = MemoryManager(cpu) + memmgr.set_max_age(3, 1) + tokens = [LoopToken() for i in range(10)] + for i in range(len(tokens)): + print 'record tokens[%d]' % i + memmgr.record_loop(tokens[i]) + memmgr.next_generation() + for j in range(0, i, 2): + assert not tokens[j].has_been_freed + print 'enter and leave tokens[%d]' % j + memmgr.enter_loop(tokens[j]) + memmgr.leave_loop(tokens[j]) + for i in range(len(tokens)): + assert tokens[i].has_been_freed == (i < 7 and (i%2) != 0) + + def test_enter_loop_2(self): + memmgr = MemoryManager(cpu) + memmgr.set_max_age(3, 1) + tokens = [LoopToken() for i in range(10)] + for i in range(len(tokens)): + print 'record tokens[%d]' % i + memmgr.record_loop(tokens[i]) + memmgr.next_generation() + for j in range(i-2, i+1): + if j >= 0: + assert not tokens[j].has_been_freed + print 'enter and leave tokens[%d]' % j + memmgr.enter_loop(tokens[j]) + memmgr.leave_loop(tokens[j]) + for i in range(len(tokens)): + assert tokens[i].has_been_freed == (i < 4) + + def test_loop_is_running(self): + memmgr = MemoryManager(cpu) + memmgr.set_max_age(3, 1) + token = LoopToken() + memmgr.record_loop(token) + memmgr.enter_loop(token) + for i in range(10): + memmgr.next_generation() + assert token.has_been_freed == False + memmgr.leave_loop(token) + for i in range(10): + memmgr.next_generation() + assert token.has_been_freed == (i >= 3) + + def test_nested_enter_loop(self): + memmgr = MemoryManager(cpu) + memmgr.set_max_age(3, 1) + token = LoopToken() + memmgr.record_loop(token) + memmgr.enter_loop(token) + # here we recursively end up seeing the same token again + memmgr.enter_loop(token) + for i in range(10): + memmgr.next_generation() + assert token.has_been_freed == False + memmgr.leave_loop(token) + # out of the recursive call, but the loop is still "locked" + for i in range(10): + memmgr.next_generation() + assert token.has_been_freed == False + memmgr.leave_loop(token) + for i in range(10): + memmgr.next_generation() + assert token.has_been_freed == (i >= 3) + + def test_contains_jumps_to(self): + memmgr = MemoryManager(cpu) + memmgr.set_max_age(3, 1) + token1 = LoopToken() + token2 = LoopToken() + token1.contains_jumps_to[token2] = None + memmgr.record_loop(token1) + memmgr.record_loop(token2) + memmgr.enter_loop(token1) + for i in range(10): + memmgr.next_generation() + assert token1.has_been_freed == False + assert token2.has_been_freed == False + memmgr.leave_loop(token1) + for i in range(10): + memmgr.next_generation() + assert token1.has_been_freed == (i >= 3) + assert token2.has_been_freed == (i >= 3) + + def test_contains_jumps_to_2(self): + memmgr = MemoryManager(cpu) + memmgr.set_max_age(3, 1) + token1 = LoopToken() + token2 = LoopToken() + token3 = LoopToken() + token1.contains_jumps_to[token2] = None + token2.contains_jumps_to[token3] = None + memmgr.record_loop(token1) + memmgr.record_loop(token2) + memmgr.record_loop(token3) + memmgr.enter_loop(token1) + for i in range(10): + memmgr.next_generation() + assert token1.has_been_freed == False + assert token2.has_been_freed == False + assert token3.has_been_freed == False + memmgr.leave_loop(token1) + for i in range(10): + memmgr.next_generation() + assert token1.has_been_freed == (i >= 3) + assert token2.has_been_freed == (i >= 3) + assert token3.has_been_freed == (i >= 3) Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_optimizeopt.py Mon Nov 15 13:39:54 2010 @@ -41,7 +41,7 @@ b1 = BoxInt() opt = optimizeopt.Optimizer(FakeMetaInterpStaticData(LLtypeMixin.cpu), None) - fdescr = ResumeGuardDescr(None, None) + fdescr = ResumeGuardDescr(history.LoopToken()) op = ResOperation(rop.GUARD_TRUE, ['dummy'], None, descr=fdescr) # setup rd data fi0 = resume.FrameInfo(None, "code0", 11) Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmspot.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmspot.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmspot.py Mon Nov 15 13:39:54 2010 @@ -382,7 +382,7 @@ def __init__(self, no): self.no = no - def handle_fail(self, metainterp_sd, jitdrivers_sd): + def handle_fail(self, metainterp_sd): if self.no == 0: raise metainterp_sd.warmrunnerdesc.DoneWithThisFrameInt(3) if self.no == 1: Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py Mon Nov 15 13:39:54 2010 @@ -12,11 +12,12 @@ from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.rarithmetic import r_uint, intmask from pypy.rlib.debug import debug_print, fatalerror +from pypy.rlib.debug import debug_start, debug_stop from pypy.rpython.lltypesystem.lloperation import llop from pypy.translator.simplify import get_funcobj, get_functype from pypy.translator.unsimplify import call_final_function -from pypy.jit.metainterp import history, pyjitpl, gc +from pypy.jit.metainterp import history, pyjitpl, gc, memmgr from pypy.jit.metainterp.pyjitpl import MetaInterpStaticData, MetaInterp from pypy.jit.metainterp.typesystem import LLTypeHelper, OOTypeHelper from pypy.jit.metainterp.jitprof import Profiler, EmptyProfiler @@ -183,6 +184,7 @@ self.rewrite_set_param() self.rewrite_force_virtual(vrefinfo) self.add_finish() + self.memory_manager = memmgr.MemoryManager(self.cpu) self.metainterp_sd.finish_setup(self.codewriter, optimizer=optimizer) def finish(self): @@ -711,10 +713,10 @@ vinfo.VTYPEPTR, virtualizableref) vinfo.reset_vable_token(virtualizable) try: - loop_token = fail_descr.handle_fail(self.metainterp_sd, jd) + loop_token = fail_descr.handle_fail(self.metainterp_sd) except JitException, e: return handle_jitexception(e) - fail_descr = self.cpu.execute_token(loop_token) + fail_descr = self.execute_token(loop_token) jd._assembler_call_helper = assembler_call_helper # for debugging jd._assembler_helper_ptr = self.helper_func( @@ -808,3 +810,15 @@ py.test.skip("rewrite_force_virtual: port it to ootype") all_graphs = self.translator.graphs vrefinfo.replace_force_virtual_with_call(all_graphs) + + # ____________________________________________________________ + + def execute_token(self, loop_token): + self.metainterp_sd.profiler.start_running() + debug_start("jit-running") + self.memory_manager.enter_loop(loop_token) + fail_descr = self.cpu.execute_token(loop_token) + self.memory_manager.leave_loop(loop_token) + debug_stop("jit-running") + self.metainterp_sd.profiler.end_running() + return fail_descr Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Mon Nov 15 13:39:54 2010 @@ -214,6 +214,10 @@ if self.profiler is not None: self.profiler.set_printing(value >= DEBUG_PROFILE) + def set_param_loop_longevity(self, value): + # note: it's a global parameter, not a per-jitdriver one + self.warmrunnerdesc.memory_manager.set_max_age(value) + def disable_noninlinable_function(self, greenkey): cell = self.jit_cell_at_key(greenkey) cell.dont_trace_here = True @@ -239,7 +243,8 @@ if hasattr(self, 'maybe_compile_and_run'): return self.maybe_compile_and_run - metainterp_sd = self.warmrunnerdesc.metainterp_sd + warmrunnerdesc = self.warmrunnerdesc + metainterp_sd = warmrunnerdesc.metainterp_sd jitdriver_sd = self.jitdriver_sd vinfo = jitdriver_sd.virtualizable_info index_of_virtualizable = jitdriver_sd.index_of_virtualizable @@ -304,16 +309,11 @@ # ---------- execute assembler ---------- while True: # until interrupted by an exception - metainterp_sd.profiler.start_running() - debug_start("jit-running") - fail_descr = metainterp_sd.cpu.execute_token(loop_token) - debug_stop("jit-running") - metainterp_sd.profiler.end_running() + fail_descr = warmrunnerdesc.execute_token(loop_token) if vinfo is not None: vinfo.reset_vable_token(virtualizable) - loop_token = fail_descr.handle_fail(metainterp_sd, - jitdriver_sd) - + loop_token = fail_descr.handle_fail(metainterp_sd) + maybe_compile_and_run._dont_inline_ = True self.maybe_compile_and_run = maybe_compile_and_run return maybe_compile_and_run Modified: pypy/branch/jit-free/pypy/rlib/jit.py ============================================================================== --- pypy/branch/jit-free/pypy/rlib/jit.py (original) +++ pypy/branch/jit-free/pypy/rlib/jit.py Mon Nov 15 13:39:54 2010 @@ -271,6 +271,7 @@ 'inlining': False, 'optimizer': OPTIMIZER_FULL, 'debug' : DEBUG_STEPS, + 'loop_longevity': 1000, } unroll_parameters = unrolling_iterable(PARAMETERS.keys()) From jacob at codespeak.net Mon Nov 15 13:44:42 2010 From: jacob at codespeak.net (jacob at codespeak.net) Date: Mon, 15 Nov 2010 13:44:42 +0100 (CET) Subject: [pypy-svn] r79095 - pypy/extradoc/planning/hg-migration Message-ID: <20101115124442.9126250811@codespeak.net> Author: jacob Date: Mon Nov 15 13:44:41 2010 New Revision: 79095 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: Updated me and Laura. Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Mon Nov 15 13:44:41 2010 @@ -24,14 +24,14 @@ getxsick=Bartosz Skowron nik=Niklaus Haldimann cami=Camillo Bruni -lac=Laura Creighton +lac=Laura Creighton david=David Schneider sanxiyn=Seo Sanghyeon santagada=Leonardo Santagada tverwaes=Toon Verwaest adim=Adrien Di Mascio rhymes=Lawrence Oluyede -jacob=Jacob Hallen +jacob=Jacob Hallen guido=Guido Wesdorp ludal=Ludovic Aubry jlg=Jakub Gustak From cfbolz at codespeak.net Mon Nov 15 13:50:06 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 15 Nov 2010 13:50:06 +0100 (CET) Subject: [pypy-svn] r79096 - in pypy/branch/jit-starargs/pypy/interpreter: . test Message-ID: <20101115125006.9A826282B9D@codespeak.net> Author: cfbolz Date: Mon Nov 15 13:50:05 2010 New Revision: 79096 Modified: pypy/branch/jit-starargs/pypy/interpreter/argument.py pypy/branch/jit-starargs/pypy/interpreter/test/test_argument.py Log: Make the JIT look into calls that use *args. Tests in test_pypy_c.py are forthcoming. Modified: pypy/branch/jit-starargs/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/jit-starargs/pypy/interpreter/argument.py (original) +++ pypy/branch/jit-starargs/pypy/interpreter/argument.py Mon Nov 15 13:50:05 2010 @@ -64,7 +64,7 @@ return not self == other - # make it look tuply for the annotator + # make it look tuply for its use in the annotator def __len__(self): return 3 @@ -103,10 +103,11 @@ make_sure_not_resized(self.keywords_w) make_sure_not_resized(self.arguments_w) - if ((w_stararg is not None and w_stararg) or - (w_starstararg is not None and w_starstararg)): - self._combine_wrapped(w_stararg, w_starstararg) - # if we have a call where * or ** args are used at the callsite + if w_stararg is not None and space.is_true(w_stararg): + self._combine_starargs_wrapped(w_stararg) + if w_starstararg is not None and space.is_true(w_starstararg): + self._combine_starstarargs_wrapped(w_starstararg) + # if we have a call where **args are used at the callsite # we shouldn't let the JIT see the argument matching self._dont_jit = True else: @@ -142,42 +143,48 @@ def _combine_wrapped(self, w_stararg, w_starstararg): "unpack the *arg and **kwd into arguments_w and keywords_w" - # unpack the * arguments if w_stararg is not None: - self.arguments_w = (self.arguments_w + - self.space.fixedview(w_stararg)) - # unpack the ** arguments + self._combine_starargs_wrapped(w_stararg) if w_starstararg is not None: - space = self.space - if not space.is_true(space.isinstance(w_starstararg, space.w_dict)): + self._combine_starstarargs_wrapped(w_starstararg) + + def _combine_starargs_wrapped(self, w_stararg): + # unpack the * arguments + self.arguments_w = (self.arguments_w + + self.space.fixedview(w_stararg)) + + def _combine_starstarargs_wrapped(self, w_starstararg): + # unpack the ** arguments + space = self.space + if not space.is_true(space.isinstance(w_starstararg, space.w_dict)): + raise OperationError(space.w_TypeError, + space.wrap("argument after ** must be " + "a dictionary")) + keywords_w = [None] * space.int_w(space.len(w_starstararg)) + keywords = [None] * space.int_w(space.len(w_starstararg)) + i = 0 + for w_key in space.unpackiterable(w_starstararg): + try: + key = space.str_w(w_key) + except OperationError, e: + if not e.match(space, space.w_TypeError): + raise raise OperationError(space.w_TypeError, - space.wrap("argument after ** must be " - "a dictionary")) - keywords_w = [None] * space.int_w(space.len(w_starstararg)) - keywords = [None] * space.int_w(space.len(w_starstararg)) - i = 0 - for w_key in space.unpackiterable(w_starstararg): - try: - key = space.str_w(w_key) - except OperationError, e: - if not e.match(space, space.w_TypeError): - raise - raise OperationError(space.w_TypeError, - space.wrap("keywords must be strings")) - if self.keywords and key in self.keywords: - raise operationerrfmt(self.space.w_TypeError, - "got multiple values " - "for keyword argument " - "'%s'", key) - keywords[i] = key - keywords_w[i] = space.getitem(w_starstararg, w_key) - i += 1 - if self.keywords is None: - self.keywords = keywords - self.keywords_w = keywords_w - else: - self.keywords = self.keywords + keywords - self.keywords_w = self.keywords_w + keywords_w + space.wrap("keywords must be strings")) + if self.keywords and key in self.keywords: + raise operationerrfmt(self.space.w_TypeError, + "got multiple values " + "for keyword argument " + "'%s'", key) + keywords[i] = key + keywords_w[i] = space.getitem(w_starstararg, w_key) + i += 1 + if self.keywords is None: + self.keywords = keywords + self.keywords_w = keywords_w + else: + self.keywords = self.keywords + keywords + self.keywords_w = self.keywords_w + keywords_w def fixedunpack(self, argcount): """The simplest argument parsing: get the 'argcount' arguments, @@ -226,6 +233,10 @@ # argnames = list of formal parameter names # scope_w = resulting list of wrapped values # + + # some comments about the JIT: it assumes that signature is a constant, + # so all values coming from there can be assumed constant. It assumes + # that the length of the defaults_w does not vary too much. co_argcount = signature.num_argnames() # expected formal arguments, without */** has_vararg = signature.has_vararg() has_kwarg = signature.has_kwarg() @@ -245,12 +256,6 @@ args_w = self.arguments_w num_args = len(args_w) - keywords = self.keywords - keywords_w = self.keywords_w - num_kwds = 0 - if keywords is not None: - num_kwds = len(keywords) - avail = num_args + upfront if input_argcount < co_argcount: @@ -260,15 +265,24 @@ else: take = num_args + # letting the JIT unroll this loop is safe, because take is always + # smaller than co_argcount for i in range(take): scope_w[i + input_argcount] = args_w[i] input_argcount += take + keywords = self.keywords + keywords_w = self.keywords_w + num_kwds = 0 + if keywords is not None: + num_kwds = len(keywords) # the code assumes that keywords can potentially be large, but that # argnames is typically not too large num_remainingkwds = num_kwds used_keywords = None if keywords: + # letting JIT unroll the loop is *only* safe if the callsite didn't + # use **args because num_kwds can be arbitrarily large otherwise. used_keywords = [False] * num_kwds for i in range(num_kwds): name = keywords[i] @@ -276,7 +290,7 @@ if j < 0: continue elif j < input_argcount: - # check that no keyword argument conflicts with these note + # check that no keyword argument conflicts with these. note # that for this purpose we ignore the first blindargs, # which were put into place by prepend(). This way, # keywords do not conflict with the hidden extra argument Modified: pypy/branch/jit-starargs/pypy/interpreter/test/test_argument.py ============================================================================== --- pypy/branch/jit-starargs/pypy/interpreter/test/test_argument.py (original) +++ pypy/branch/jit-starargs/pypy/interpreter/test/test_argument.py Mon Nov 15 13:50:05 2010 @@ -52,12 +52,17 @@ assert y == "d" assert z == "e" +class dummy_wrapped_dict(dict): + def __nonzero__(self): + raise NotImplementedError class DummySpace(object): def newtuple(self, items): return tuple(items) def is_true(self, obj): + if isinstance(obj, dummy_wrapped_dict): + return bool(dict(obj)) return bool(obj) def fixedview(self, it): @@ -229,7 +234,7 @@ kwds_w = dict(kwds[:i]) keywords = kwds_w.keys() keywords_w = kwds_w.values() - w_kwds = dict(kwds[i:]) + w_kwds = dummy_wrapped_dict(kwds[i:]) if i == 2: w_kwds = None assert len(keywords) == len(keywords_w) @@ -265,7 +270,7 @@ kwds_w = dict(kwds[:i]) keywords = kwds_w.keys() keywords_w = kwds_w.values() - w_kwds = dict(kwds[i:]) + w_kwds = dummy_wrapped_dict(kwds[i:]) if i == 3: w_kwds = None args = Arguments(space, [1, 2], keywords, keywords_w, w_starstararg=w_kwds) From arigo at codespeak.net Mon Nov 15 13:58:45 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 13:58:45 +0100 (CET) Subject: [pypy-svn] r79097 - pypy/extradoc/planning/hg-migration Message-ID: <20101115125845.7132D5080F@codespeak.net> Author: arigo Date: Mon Nov 15 13:58:43 2010 New Revision: 79097 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: By Eric's request, fill in his mail address too. Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Mon Nov 15 13:58:43 2010 @@ -15,7 +15,7 @@ afa=Amaury Forgeot d'Arc tismer=Christian Tismer benjamin=Benjamin Peterson -ericvrp=Eric van Riet Paap +ericvrp=Eric van Riet Paap ac=Anders Chrigstrom xoraxax=Alexander Schremmer rxe=Richard Emslie From afa at codespeak.net Mon Nov 15 14:11:37 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 14:11:37 +0100 (CET) Subject: [pypy-svn] r79098 - in pypy/branch/fast-forward/pypy/module/_winreg: . test Message-ID: <20101115131137.7C3BE36C220@codespeak.net> Author: afa Date: Mon Nov 15 14:11:35 2010 New Revision: 79098 Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py Log: _winreg: add context manager and the .handle attribute to registry keys. Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py Mon Nov 15 14:11:35 2010 @@ -1,12 +1,12 @@ from __future__ import with_statement from pypy.interpreter.baseobjspace import Wrappable from pypy.interpreter.baseobjspace import ObjSpace, W_Root -from pypy.interpreter.gateway import interp2app -from pypy.interpreter.typedef import TypeDef +from pypy.interpreter.gateway import interp2app, Arguments +from pypy.interpreter.typedef import TypeDef, GetSetProperty from pypy.interpreter.error import OperationError from pypy.rpython.lltypesystem import rffi, lltype from pypy.rlib import rwinreg, rwin32 -from pypy.rlib.rarithmetic import r_uint +from pypy.rlib.rarithmetic import r_uint, intmask def raiseWindowsError(space, errcode, context): message = rwin32.FormatError(errcode) @@ -29,6 +29,9 @@ return space.wrap(self.as_int() != 0) descr_nonzero.unwrap_spec = ['self', ObjSpace] + def descr_handle_get(space, self): + return space.wrap(self.as_int()) + def descr_repr(self, space): return space.wrap("" % (self.as_int(),)) descr_repr.unwrap_spec = ['self', ObjSpace] @@ -37,6 +40,14 @@ return space.wrap(self.as_int()) descr_int.unwrap_spec = ['self', ObjSpace] + def descr__enter__(self, space): + return self + descr__enter__.unwrap_spec = ['self', ObjSpace] + + def descr__exit__(self, space, __args__): + CloseKey(space, self) + descr__exit__.unwrap_spec = ['self', ObjSpace, Arguments] + def Close(self, space): """key.Close() - Closes the underlying Windows handle. If the handle is already closed, no error is raised.""" @@ -93,6 +104,9 @@ __repr__ = interp2app(W_HKEY.descr_repr), __int__ = interp2app(W_HKEY.descr_int), __nonzero__ = interp2app(W_HKEY.descr_nonzero), + __enter__ = interp2app(W_HKEY.descr__enter__), + __exit__ = interp2app(W_HKEY.descr__exit__), + handle = GetSetProperty(W_HKEY.descr_handle_get), Close = interp2app(W_HKEY.Close), Detach = interp2app(W_HKEY.Detach), ) @@ -123,6 +137,8 @@ ret = rwinreg.RegCloseKey(hkey) if ret != 0: raiseWindowsError(space, ret, 'RegCloseKey') + if isinstance(w_hkey, W_HKEY): + space.interp_w(W_HKEY, w_hkey).hkey = rwin32.NULL_HANDLE CloseKey.unwrap_spec = [ObjSpace, W_Root] def FlushKey(space, w_hkey): @@ -409,7 +425,8 @@ if ret != 0: raiseWindowsError(space, ret, 'RegQueryValueEx') - with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf: + with lltype.scoped_alloc(rffi.CCHARP.TO, + intmask(retDataSize[0])) as databuf: with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType: ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword, @@ -521,8 +538,10 @@ retValueSize[0] += 1 retDataSize[0] += 1 - with lltype.scoped_alloc(rffi.CCHARP.TO, retValueSize[0]) as valuebuf: - with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf: + with lltype.scoped_alloc(rffi.CCHARP.TO, + intmask(retValueSize[0])) as valuebuf: + with lltype.scoped_alloc(rffi.CCHARP.TO, + intmask(retDataSize[0])) as databuf: with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType: ret = rwinreg.RegEnumValue( hkey, index, valuebuf, retValueSize, Modified: pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py Mon Nov 15 14:11:35 2010 @@ -97,6 +97,15 @@ CloseKey(int_key) raises(EnvironmentError, QueryInfoKey, int_key) # now closed + def test_with(self): + from _winreg import OpenKey + with OpenKey(self.root_key, self.test_key_name) as key: + with OpenKey(key, "sub_key") as sub_key: + assert key.handle != 0 + assert sub_key.handle != 0 + assert key.handle == 0 + assert sub_key.handle == 0 + def test_exception(self): from _winreg import QueryInfoKey import errno From afa at codespeak.net Mon Nov 15 14:15:16 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 14:15:16 +0100 (CET) Subject: [pypy-svn] r79099 - in pypy/branch/fast-forward/pypy: module/_winreg/test rlib Message-ID: <20101115131516.157405080F@codespeak.net> Author: afa Date: Mon Nov 15 14:15:14 2010 New Revision: 79099 Modified: pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py pypy/branch/fast-forward/pypy/rlib/rwinreg.py Log: Typo in constant: test and fix Modified: pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py Mon Nov 15 14:15:14 2010 @@ -58,6 +58,12 @@ except WindowsError: pass + def test_constants(self): + from _winreg import ( + HKEY_LOCAL_MACHINE, HKEY_CLASSES_ROOT, HKEY_CURRENT_CONFIG, + HKEY_CURRENT_USER, HKEY_DYN_DATA, HKEY_LOCAL_MACHINE, + HKEY_PERFORMANCE_DATA, HKEY_USERS) + def test_simple_write(self): from _winreg import SetValue, QueryValue, REG_SZ value = "Some Default value" Modified: pypy/branch/fast-forward/pypy/rlib/rwinreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rwinreg.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rwinreg.py Mon Nov 15 14:15:14 2010 @@ -26,7 +26,7 @@ REG_FULL_RESOURCE_DESCRIPTOR REG_RESOURCE_REQUIREMENTS_LIST HKEY_LOCAL_MACHINE HKEY_CLASSES_ROOT HKEY_CURRENT_CONFIG HKEY_CURRENT_USER -HKEY_DYN_DATA HKEY_LOCAL_MACHINE HKEY_PERFORMANCE_DATATA HKEY_USERS +HKEY_DYN_DATA HKEY_LOCAL_MACHINE HKEY_PERFORMANCE_DATA HKEY_USERS '''.split() for name in constant_names: setattr(CConfig, name, platform.DefinedConstantInteger(name)) From afa at codespeak.net Mon Nov 15 14:46:08 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 14:46:08 +0100 (CET) Subject: [pypy-svn] r79100 - in pypy/branch/fast-forward/pypy: module/_winreg module/_winreg/test rlib rpython/lltypesystem Message-ID: <20101115134608.3F7E0282B90@codespeak.net> Author: afa Date: Mon Nov 15 14:46:06 2010 New Revision: 79100 Modified: pypy/branch/fast-forward/pypy/module/_winreg/__init__.py pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py pypy/branch/fast-forward/pypy/rlib/rwinreg.py pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py Log: Add _winreg.ExpandEnvironmentStrings() Modified: pypy/branch/fast-forward/pypy/module/_winreg/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/__init__.py Mon Nov 15 14:46:06 2010 @@ -61,6 +61,8 @@ 'LoadKey' : 'interp_winreg.LoadKey', 'SaveKey' : 'interp_winreg.SaveKey', 'ConnectRegistry': 'interp_winreg.ConnectRegistry', + + 'ExpandEnvironmentStrings': 'interp_winreg.ExpandEnvironmentStrings', } for name, value in constants.iteritems(): Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py Mon Nov 15 14:46:06 2010 @@ -636,3 +636,11 @@ raiseWindowsError(space, ret, 'RegConnectRegistry') return space.wrap(W_HKEY(rethkey[0])) ConnectRegistry.unwrap_spec = [ObjSpace, W_Root, W_Root] + +def ExpandEnvironmentStrings(space, source): + "string = ExpandEnvironmentStrings(string) - Expand environment vars." + try: + return space.wrap(rwinreg.ExpandEnvironmentStrings(source)) + except WindowsError, e: + raise wrap_windowserror(space, e) +ExpandEnvironmentStrings.unwrap_spec = [ObjSpace, unicode] Modified: pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py Mon Nov 15 14:46:06 2010 @@ -180,3 +180,10 @@ key = OpenKey(self.root_key, self.test_key_name, 0, KEY_ALL_ACCESS) SaveKey(key, self.tmpfilename) + + def test_expand_environment_string(self): + from _winreg import ExpandEnvironmentStrings + import nt + r = ExpandEnvironmentStrings(u"%windir%\\test") + assert isinstance(r, unicode) + assert r == nt.environ["WINDIR"] + "\\test" Modified: pypy/branch/fast-forward/pypy/rlib/rwinreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rwinreg.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rwinreg.py Mon Nov 15 14:46:06 2010 @@ -1,12 +1,13 @@ from pypy.rpython.lltypesystem import lltype, rffi from pypy.rpython.tool import rffi_platform as platform from pypy.translator.tool.cbuild import ExternalCompilationInfo +from pypy.rlib.rarithmetic import intmask from pypy.rlib import rwin32 eci = ExternalCompilationInfo( includes = ['windows.h', ], - libraries = ('Advapi32',) + libraries = ('Advapi32', 'kernel32') ) class CConfig: _compilation_info_ = eci @@ -132,3 +133,21 @@ 'RegConnectRegistryA', [rffi.CCHARP, HKEY, PHKEY], rffi.LONG) + +_ExpandEnvironmentStringsW = external( + 'ExpandEnvironmentStringsW', + [rffi.CWCHARP, rffi.CWCHARP, rwin32.DWORD], + rwin32.DWORD) + +def ExpandEnvironmentStrings(source): + with rffi.scoped_unicode2wcharp(source) as src_buf: + size = _ExpandEnvironmentStringsW(src_buf, + lltype.nullptr(rffi.CWCHARP.TO), 0) + if size == 0: + raise rwin32.lastWindowsError("ExpandEnvironmentStrings") + size = intmask(size) + with rffi.scoped_alloc_unicodebuffer(size) as dest_buf: + if _ExpandEnvironmentStringsW(src_buf, + dest_buf.raw, size) == 0: + raise rwin32.lastWindowsError("ExpandEnvironmentStrings") + return dest_buf.str(size - 1) # remove trailing \0 Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py (original) +++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py Mon Nov 15 14:46:06 2010 @@ -945,6 +945,15 @@ free_charp(self.buf) +class scoped_unicode2wcharp: + def __init__(self, value): + self.buf = unicode2wcharp(value) + def __enter__(self): + return self.buf + def __exit__(self, *args): + free_wcharp(self.buf) + + class scoped_nonmovingbuffer: def __init__(self, data): self.data = data @@ -955,6 +964,15 @@ free_nonmovingbuffer(self.data, self.buf) +class scoped_nonmoving_unicodebuffer: + def __init__(self, data): + self.data = data + def __enter__(self): + self.buf = get_nonmoving_unicodebuffer(self.data) + return self.buf + def __exit__(self, *args): + free_nonmoving_unicodebuffer(self.data, self.buf) + class scoped_alloc_buffer: def __init__(self, size): self.size = size @@ -965,3 +983,14 @@ keep_buffer_alive_until_here(self.raw, self.gc_buf) def str(self, length): return str_from_buffer(self.raw, self.gc_buf, self.size, length) + +class scoped_alloc_unicodebuffer: + def __init__(self, size): + self.size = size + def __enter__(self): + self.raw, self.gc_buf = alloc_unicodebuffer(self.size) + return self + def __exit__(self, *args): + keep_unicodebuffer_alive_until_here(self.raw, self.gc_buf) + def str(self, length): + return unicode_from_buffer(self.raw, self.gc_buf, self.size, length) From cfbolz at codespeak.net Mon Nov 15 15:00:24 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 15 Nov 2010 15:00:24 +0100 (CET) Subject: [pypy-svn] r79101 - pypy/branch/jit-starargs/pypy/module/pypyjit/test Message-ID: <20101115140024.7070B36C224@codespeak.net> Author: cfbolz Date: Mon Nov 15 15:00:22 2010 New Revision: 79101 Modified: pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py Log: test number one Modified: pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py Mon Nov 15 15:00:22 2010 @@ -382,6 +382,28 @@ # XXX a bit too many guards, but better than before assert len(op.get_opnames("guard")) <= 10 + def test_stararg(self): + self.run_source(''' + d = {} + + def g(*args): + return len(args) + + def main(x): + s = 0 + for i in range(x): + l = [i, x, 2] + s += g(*l) + return s + ''', 100000, ([100], 300), + ([1000], 3000), + ([10000], 30000), + ([100000], 300000)) + assert len(self.loops) == 1 + op, = self.get_by_bytecode("CALL_FUNCTION_VAR") + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 + def test_virtual_instance(self): self.run_source(''' class A(object): From cfbolz at codespeak.net Mon Nov 15 15:08:48 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 15 Nov 2010 15:08:48 +0100 (CET) Subject: [pypy-svn] r79102 - pypy/branch/jit-starargs/pypy/module/pypyjit/test Message-ID: <20101115140848.5966B36C224@codespeak.net> Author: cfbolz Date: Mon Nov 15 15:08:46 2010 New Revision: 79102 Modified: pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py Log: add another case Modified: pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py Mon Nov 15 15:08:46 2010 @@ -394,15 +394,19 @@ for i in range(x): l = [i, x, 2] s += g(*l) + s += g(i, x, 2) return s - ''', 100000, ([100], 300), - ([1000], 3000), - ([10000], 30000), - ([100000], 300000)) + ''', 100000, ([100], 600), + ([1000], 6000), + ([10000], 60000), + ([100000], 600000)) assert len(self.loops) == 1 op, = self.get_by_bytecode("CALL_FUNCTION_VAR") assert len(op.get_opnames("new")) == 0 assert len(op.get_opnames("call_may_force")) == 0 + oplen, op, oplen = self.get_by_bytecode("CALL_FUNCTION") + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 def test_virtual_instance(self): self.run_source(''' From cfbolz at codespeak.net Mon Nov 15 15:14:50 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 15 Nov 2010 15:14:50 +0100 (CET) Subject: [pypy-svn] r79103 - pypy/branch/jit-starargs/pypy/module/pypyjit/test Message-ID: <20101115141450.4D5BD282B90@codespeak.net> Author: cfbolz Date: Mon Nov 15 15:14:48 2010 New Revision: 79103 Modified: pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py Log: yet another case Modified: pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py Mon Nov 15 15:14:48 2010 @@ -388,25 +388,32 @@ def g(*args): return len(args) + def h(a, b, c): + return c def main(x): s = 0 for i in range(x): l = [i, x, 2] s += g(*l) + s += h(*l) s += g(i, x, 2) return s - ''', 100000, ([100], 600), - ([1000], 6000), - ([10000], 60000), - ([100000], 600000)) + ''', 100000, ([100], 800), + ([1000], 8000), + ([10000], 80000), + ([100000], 800000)) assert len(self.loops) == 1 - op, = self.get_by_bytecode("CALL_FUNCTION_VAR") - assert len(op.get_opnames("new")) == 0 - assert len(op.get_opnames("call_may_force")) == 0 - oplen, op, oplen = self.get_by_bytecode("CALL_FUNCTION") - assert len(op.get_opnames("new")) == 0 - assert len(op.get_opnames("call_may_force")) == 0 + ops = self.get_by_bytecode("CALL_FUNCTION_VAR") + assert len(ops) == 2 + for op in ops: + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 + + ops = self.get_by_bytecode("CALL_FUNCTION") + for op in ops: + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 def test_virtual_instance(self): self.run_source(''' From cfbolz at codespeak.net Mon Nov 15 15:21:12 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 15 Nov 2010 15:21:12 +0100 (CET) Subject: [pypy-svn] r79104 - pypy/branch/jit-starargs/pypy/module/pypyjit/test Message-ID: <20101115142112.99DA0282B90@codespeak.net> Author: cfbolz Date: Mon Nov 15 15:21:11 2010 New Revision: 79104 Modified: pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py Log: another case: mix *args and normal positional args Modified: pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py Mon Nov 15 15:21:11 2010 @@ -398,14 +398,18 @@ s += g(*l) s += h(*l) s += g(i, x, 2) + for i in range(x): + l = [x, 2] + s += g(i, *l) + s += h(i, *l) return s - ''', 100000, ([100], 800), - ([1000], 8000), - ([10000], 80000), - ([100000], 800000)) - assert len(self.loops) == 1 + ''', 100000, ([100], 1300), + ([1000], 13000), + ([10000], 130000), + ([100000], 1300000)) + assert len(self.loops) == 2 ops = self.get_by_bytecode("CALL_FUNCTION_VAR") - assert len(ops) == 2 + assert len(ops) == 4 for op in ops: assert len(op.get_opnames("new")) == 0 assert len(op.get_opnames("call_may_force")) == 0 From arigo at codespeak.net Mon Nov 15 15:34:27 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 15:34:27 +0100 (CET) Subject: [pypy-svn] r79105 - in pypy/branch/jit-free/pypy/jit/metainterp: . test Message-ID: <20101115143427.78285282B9C@codespeak.net> Author: arigo Date: Mon Nov 15 15:34:25 2010 New Revision: 79105 Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py pypy/branch/jit-free/pypy/jit/metainterp/history.py pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_compile.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Log: (antocuni, arigo) In-progress: the plan is to use weakrefs, giving much saner code. See the comment in memmgr.py. Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/compile.py Mon Nov 15 15:34:25 2010 @@ -42,8 +42,8 @@ name = metainterp.staticdata.stats.name_for_new_loop() return TreeLoop(name) -def make_loop_token(nb_args, jitdriver_sd, greenkey): - loop_token = LoopToken() +def make_loop_token(cpu, nb_args, jitdriver_sd, greenkey): + loop_token = LoopToken(cpu) loop_token.specnodes = [prebuiltNotSpecNode] * nb_args loop_token.outermost_jitdriver_sd = jitdriver_sd loop_token.outermost_greenkey = greenkey @@ -65,7 +65,8 @@ loop.operations = [h_ops[i].clone() for i in range(start, len(h_ops))] metainterp_sd = metainterp.staticdata jitdriver_sd = metainterp.jitdriver_sd - loop_token = make_loop_token(len(loop.inputargs), jitdriver_sd, greenkey) + loop_token = make_loop_token(metainterp.cpu, len(loop.inputargs), + jitdriver_sd, greenkey) loop.token = loop_token loop.operations[-1].setdescr(loop_token) # patch the target of the JUMP try: @@ -98,9 +99,6 @@ loop_token = loop.token loop_token.number = n = globaldata.loopnumbering globaldata.loopnumbering += 1 - desc = metainterp_sd.warmrunnerdesc - if desc is not None: # for tests - desc.memory_manager.record_loop(loop_token) metainterp_sd.logger_ops.log_loop(loop.inputargs, loop.operations, n, type) if not we_are_translated(): @@ -479,7 +477,7 @@ class ResumeFromInterpDescr(ResumeDescr): def __init__(self, metainterp, greenkey, redkey): - original_loop_token = make_loop_token(len(redkey), + original_loop_token = make_loop_token(metainterp.cpu, len(redkey), metainterp.jitdriver_sd, greenkey) ResumeDescr.__init__(self, original_loop_token) @@ -575,7 +573,7 @@ """ # 'redboxes' is only used to know the types of red arguments. inputargs = [box.clonebox() for box in redboxes] - loop_token = make_loop_token(len(inputargs), jitdriver_sd, greenboxes) + loop_token = make_loop_token(cpu, len(inputargs), jitdriver_sd, greenboxes) # 'nb_red_args' might be smaller than len(redboxes), # because it doesn't include the virtualizable boxes. nb_red_args = jitdriver_sd.num_red_args Modified: pypy/branch/jit-free/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/history.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/history.py Mon Nov 15 15:34:25 2010 @@ -728,14 +728,16 @@ generated assembler. """ terminating = False # see TerminatingLoopToken in compile.py - has_been_freed = False outermost_jitdriver_sd = None outermost_greenkey = None # specnodes = ... # and more data specified by the backend when the loop is compiled + cpu = None + number = 0 - def __init__(self, number=0): - self.number = number + def __init__(self, cpu=None): + assert not isinstance(cpu, int) # xxx temporary + self.cpu = cpu # See get_fail_descr_number() in backend/model.py: this growing # list gives the 'descr_number' of all fail descrs that belong to # this loop or to a bridge attached to it. @@ -744,6 +746,15 @@ self.contains_jumps_to = {} # set of other LoopTokens self.generation = r_longlong(0) + def __del__(self): + if self.cpu is None: + return + if self.generation > r_longlong(0): + # MemoryManager.keep_loop_alive() has been called on this + # loop token, which means that it has been successfully + # compiled by the backend. Free it now. + self.cpu.free_loop_and_bridges(self) + def repr_of_descr(self): return '' % self.number Modified: pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py Mon Nov 15 15:34:25 2010 @@ -1,30 +1,25 @@ import math from pypy.rlib.rarithmetic import r_longlong -from pypy.rlib.objectmodel import we_are_translated # # Logic to decide which loops are old and not used any more. # -# Idea: We use the notion of a global 'current generation' which -# is, in practice, the total number of loops and bridges produced -# so far. When executing a loop: -# (1) we set 'generation' to -1 -# (2) we execute it -# (3) we set 'generation' to the latest generation -# (with a bit extra work to handle nested calls and to guarantee -# that 'generation' is always < 0 on a loop that is currently -# executing). +# All the long-lived references to LoopToken are weakrefs, apart from +# the 'alive_loops' set in MemoryManager, which is the only (long-living) +# place that keeps them alive. If a loop was not called for long enough, +# then it is removed from 'alive_loops'. It will soon be freed by the +# GC. LoopToken.__del__ calls the method cpu.free_loop_and_bridges(). # -# A loop is said "old" if its generation is >= 0 but much smaller -# than the current generation. If a loop L is old, and if all -# other loops from which we can reach L through -# 'contains_jumps_to' are also old, then we can free L. +# The alive_loops set is maintained using the notion of a global +# 'current generation' which is, in practice, the total number of loops +# and bridges produced so far. A LoopToken is declared "old" if its +# 'generation' field is much smaller than the current generation, and +# removed from the set. # class MemoryManager(object): - def __init__(self, cpu): - self.cpu = cpu + def __init__(self): self.check_frequency = -1 # NB. use of r_longlong to be extremely far on the safe side: # this is increasing by one after each loop or bridge is @@ -34,9 +29,9 @@ # enough. But in this day and age, you'd still never have the # patience of waiting for a slowly-increasing 64-bit number to # overflow :-) - self.current_generation = r_longlong(0) + self.current_generation = r_longlong(1) self.next_check = r_longlong(-1) - self.looptokens = [] + self.alive_loops = {} def set_max_age(self, max_age, check_frequency=0): if max_age <= 0: @@ -51,56 +46,16 @@ def next_generation(self): self.current_generation += 1 if self.current_generation == self.next_check: - self._free_old_loops_now() + self._kill_old_loops_now() self.next_check = self.current_generation + self.check_frequency - def enter_loop(self, looptoken): - if not we_are_translated(): - assert looptoken in self.looptokens - assert not looptoken.has_been_freed - if looptoken.generation >= 0: - looptoken.generation = -1 - else: - looptoken.generation -= 1 # nested enter_loop() - - def leave_loop(self, looptoken): - assert looptoken.generation < 0 - if looptoken.generation == -1: + def keep_loop_alive(self, looptoken): + if looptoken.generation != self.current_generation: looptoken.generation = self.current_generation - else: - looptoken.generation += 1 # nested leave_loop() + self.alive_loops[looptoken] = None - def record_loop(self, looptoken): - looptoken.generation = self.current_generation - self.looptokens.append(looptoken) - - def _free_old_loops_now(self): - # - # Initialize '_is_young' on all loop tokens + def _kill_old_loops_now(self): max_generation = self.current_generation - self.max_age - youngloops = [] - for looptoken in self.looptokens: + for looptoken in self.alive_loops.keys(): if 0 <= looptoken.generation < max_generation: - looptoken._is_young = False # but may be turned to True later - else: - looptoken._is_young = True - youngloops.append(looptoken) - # - # Propagate forward the knowledge of "is a young loop" - while len(youngloops) > 0: - looptoken = youngloops.pop() - for jumptargettok in looptoken.contains_jumps_to: - if not jumptargettok._is_young: - jumptargettok._is_young = True - youngloops.append(jumptargettok) - # - # Now free all looptokens that still have _is_young == False. - i = 0 - while i < len(self.looptokens): - looptoken = self.looptokens[i] - if looptoken._is_young: - i += 1 - else: - self.looptokens[i] = self.looptokens[-1] - del self.looptokens[-1] - self.cpu.free_loop_and_bridges(looptoken) + del self.alive_loops[looptoken] Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_compile.py Mon Nov 15 15:34:25 2010 @@ -218,52 +218,3 @@ assert lltype.cast_opaque_ptr(lltype.Ptr(EXC), e.args[1]) == llexc else: assert 0, "should have raised" - - -def test_send_loop_to_backend(): - class FakeMetaInterpSD: - class globaldata: - loopnumbering = 17 - class warmrunnerdesc: - class memory_manager: - @staticmethod - def record_loop(token): - assert token is FakeLoop.token - token._recorded = True - class logger_ops: - @staticmethod - def log_loop(*args, **kwds): - pass - class profiler: - @staticmethod - def start_backend(): - pass - @staticmethod - def end_backend(): - pass - class cpu: - @staticmethod - def compile_loop(inputargs, operations, token): - assert inputargs is FakeLoop.inputargs - assert operations is FakeLoop.operations - assert token is FakeLoop.token - token._compiled = True - class stats: - @staticmethod - def add_new_loop(loop): - pass - def log(self, text): - pass - class FakeLoop: - inputargs = [] - operations = [] - class token: - number = 0 - _recorded = _compiled = False - def check_consistency(self): - pass - send_loop_to_backend(FakeMetaInterpSD(), FakeLoop(), "entry bridge") - assert FakeMetaInterpSD.globaldata.loopnumbering == 18 - assert FakeLoop.token.number == 17 - assert FakeLoop.token._recorded is True - assert FakeLoop.token._compiled is True Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Mon Nov 15 15:34:25 2010 @@ -1,158 +1,59 @@ from pypy.jit.metainterp.memmgr import MemoryManager -from pypy.jit.metainterp.history import LoopToken ##missing: - -## in _free_old_loops_now(), remove looptoken from everywhere -## or mark it as freed - ## contains_jumps_to needs to be filled -class FakeCPU: - def free_loop_and_bridges(self, looptoken): - looptoken.has_been_freed = True -cpu = FakeCPU() +class FakeLoopToken: + generation = 0 class TestMemoryManager: def test_disabled(self): - memmgr = MemoryManager(cpu) + memmgr = MemoryManager() memmgr.set_max_age(0) - tokens = [LoopToken() for i in range(10)] + tokens = [FakeLoopToken() for i in range(10)] for token in tokens: - memmgr.record_loop(token) + memmgr.keep_loop_alive(token) memmgr.next_generation() - for token in tokens: - assert not token.has_been_freed + assert memmgr.alive_loops == dict.fromkeys(tokens) def test_basic(self): - memmgr = MemoryManager(cpu) + memmgr = MemoryManager() memmgr.set_max_age(3, 1) - tokens = [LoopToken() for i in range(10)] + tokens = [FakeLoopToken() for i in range(10)] for token in tokens: - memmgr.record_loop(token) + memmgr.keep_loop_alive(token) memmgr.next_generation() - for i in range(len(tokens)): - assert tokens[i].has_been_freed == (i < 7) + assert memmgr.alive_loops == dict.fromkeys(tokens[7:]) def test_basic_2(self): - memmgr = MemoryManager(cpu) + memmgr = MemoryManager() memmgr.set_max_age(3, 1) - token = LoopToken() - memmgr.record_loop(token) + token = FakeLoopToken() + memmgr.keep_loop_alive(token) for i in range(10): memmgr.next_generation() - assert token.has_been_freed == (i >= 3) + if i < 3: + assert memmgr.alive_loops == {token: None} + else: + assert memmgr.alive_loops == {} - def test_enter_loop_1(self): - memmgr = MemoryManager(cpu) + def test_basic_3(self): + memmgr = MemoryManager() memmgr.set_max_age(3, 1) - tokens = [LoopToken() for i in range(10)] + tokens = [FakeLoopToken() for i in range(10)] for i in range(len(tokens)): print 'record tokens[%d]' % i - memmgr.record_loop(tokens[i]) + memmgr.keep_loop_alive(tokens[i]) memmgr.next_generation() for j in range(0, i, 2): - assert not tokens[j].has_been_freed - print 'enter and leave tokens[%d]' % j - memmgr.enter_loop(tokens[j]) - memmgr.leave_loop(tokens[j]) - for i in range(len(tokens)): - assert tokens[i].has_been_freed == (i < 7 and (i%2) != 0) - - def test_enter_loop_2(self): - memmgr = MemoryManager(cpu) - memmgr.set_max_age(3, 1) - tokens = [LoopToken() for i in range(10)] - for i in range(len(tokens)): - print 'record tokens[%d]' % i - memmgr.record_loop(tokens[i]) - memmgr.next_generation() - for j in range(i-2, i+1): - if j >= 0: - assert not tokens[j].has_been_freed - print 'enter and leave tokens[%d]' % j - memmgr.enter_loop(tokens[j]) - memmgr.leave_loop(tokens[j]) + assert tokens[j] in memmgr.alive_loops + print 'also keep alive tokens[%d]' % j + memmgr.keep_loop_alive(tokens[j]) for i in range(len(tokens)): - assert tokens[i].has_been_freed == (i < 4) - - def test_loop_is_running(self): - memmgr = MemoryManager(cpu) - memmgr.set_max_age(3, 1) - token = LoopToken() - memmgr.record_loop(token) - memmgr.enter_loop(token) - for i in range(10): - memmgr.next_generation() - assert token.has_been_freed == False - memmgr.leave_loop(token) - for i in range(10): - memmgr.next_generation() - assert token.has_been_freed == (i >= 3) - - def test_nested_enter_loop(self): - memmgr = MemoryManager(cpu) - memmgr.set_max_age(3, 1) - token = LoopToken() - memmgr.record_loop(token) - memmgr.enter_loop(token) - # here we recursively end up seeing the same token again - memmgr.enter_loop(token) - for i in range(10): - memmgr.next_generation() - assert token.has_been_freed == False - memmgr.leave_loop(token) - # out of the recursive call, but the loop is still "locked" - for i in range(10): - memmgr.next_generation() - assert token.has_been_freed == False - memmgr.leave_loop(token) - for i in range(10): - memmgr.next_generation() - assert token.has_been_freed == (i >= 3) - - def test_contains_jumps_to(self): - memmgr = MemoryManager(cpu) - memmgr.set_max_age(3, 1) - token1 = LoopToken() - token2 = LoopToken() - token1.contains_jumps_to[token2] = None - memmgr.record_loop(token1) - memmgr.record_loop(token2) - memmgr.enter_loop(token1) - for i in range(10): - memmgr.next_generation() - assert token1.has_been_freed == False - assert token2.has_been_freed == False - memmgr.leave_loop(token1) - for i in range(10): - memmgr.next_generation() - assert token1.has_been_freed == (i >= 3) - assert token2.has_been_freed == (i >= 3) - - def test_contains_jumps_to_2(self): - memmgr = MemoryManager(cpu) - memmgr.set_max_age(3, 1) - token1 = LoopToken() - token2 = LoopToken() - token3 = LoopToken() - token1.contains_jumps_to[token2] = None - token2.contains_jumps_to[token3] = None - memmgr.record_loop(token1) - memmgr.record_loop(token2) - memmgr.record_loop(token3) - memmgr.enter_loop(token1) - for i in range(10): - memmgr.next_generation() - assert token1.has_been_freed == False - assert token2.has_been_freed == False - assert token3.has_been_freed == False - memmgr.leave_loop(token1) - for i in range(10): - memmgr.next_generation() - assert token1.has_been_freed == (i >= 3) - assert token2.has_been_freed == (i >= 3) - assert token3.has_been_freed == (i >= 3) + if i < 7 and (i%2) != 0: + assert tokens[i] not in memmgr.alive_loops + else: + assert tokens[i] in memmgr.alive_loops Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py Mon Nov 15 15:34:25 2010 @@ -153,6 +153,7 @@ optimizer=None, ProfilerClass=EmptyProfiler, **kwds): pyjitpl._warmrunnerdesc = self # this is a global for debugging only! self.set_translator(translator) + self.memory_manager = memmgr.MemoryManager() self.build_cpu(CPUClass, **kwds) self.find_portals() self.codewriter = codewriter.CodeWriter(self.cpu, self.jitdrivers_sd) @@ -184,7 +185,6 @@ self.rewrite_set_param() self.rewrite_force_virtual(vrefinfo) self.add_finish() - self.memory_manager = memmgr.MemoryManager(self.cpu) self.metainterp_sd.finish_setup(self.codewriter, optimizer=optimizer) def finish(self): @@ -816,9 +816,8 @@ def execute_token(self, loop_token): self.metainterp_sd.profiler.start_running() debug_start("jit-running") - self.memory_manager.enter_loop(loop_token) fail_descr = self.cpu.execute_token(loop_token) - self.memory_manager.leave_loop(loop_token) debug_stop("jit-running") self.metainterp_sd.profiler.end_running() + self.memory_manager.keep_loop_alive(loop_token) return fail_descr Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Mon Nov 15 15:34:25 2010 @@ -165,6 +165,7 @@ "NOT_RPYTHON" self.warmrunnerdesc = warmrunnerdesc self.jitdriver_sd = jitdriver_sd + self.cpu = warmrunnerdesc.cpu try: self.profiler = warmrunnerdesc.metainterp_sd.profiler except AttributeError: # for tests @@ -233,8 +234,7 @@ old_token = cell.entry_loop_token cell.entry_loop_token = entry_loop_token if old_token is not None: - cpu = self.warmrunnerdesc.cpu - cpu.redirect_call_assembler(old_token, entry_loop_token) + self.cpu.redirect_call_assembler(old_token, entry_loop_token) # ---------- @@ -459,7 +459,7 @@ warmrunnerdesc = self.warmrunnerdesc jitdriver_sd = self.jitdriver_sd - cpu = warmrunnerdesc.cpu + cpu = self.cpu vinfo = jitdriver_sd.virtualizable_info red_args_types = unrolling_iterable(jitdriver_sd._red_args_types) # @@ -511,7 +511,7 @@ unwrap_greenkey = self.make_unwrap_greenkey() jit_getter = self.make_jitcell_getter() jd = self.jitdriver_sd - cpu = self.warmrunnerdesc.cpu + cpu = self.cpu def can_inline_greenargs(*greenargs): if can_never_inline(*greenargs): From arigo at codespeak.net Mon Nov 15 15:39:06 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 15:39:06 +0100 (CET) Subject: [pypy-svn] r79106 - in pypy/trunk/pypy/jit: metainterp/test tool/test Message-ID: <20101115143906.2590550810@codespeak.net> Author: arigo Date: Mon Nov 15 15:39:05 2010 New Revision: 79106 Added: pypy/trunk/pypy/jit/tool/test/test_oparser.py - copied unchanged from r79094, pypy/trunk/pypy/jit/metainterp/test/test_oparser.py Removed: pypy/trunk/pypy/jit/metainterp/test/test_oparser.py Log: (antocuni, arigo) Move test_oparser together with oparser. From cfbolz at codespeak.net Mon Nov 15 15:44:18 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 15 Nov 2010 15:44:18 +0100 (CET) Subject: [pypy-svn] r79107 - pypy/branch/jit-starargs/pypy/module/pypyjit/test Message-ID: <20101115144418.CA956282B9C@codespeak.net> Author: cfbolz Date: Mon Nov 15 15:44:17 2010 New Revision: 79107 Modified: pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py Log: also a test where the list is prebuilt. in this case we get some array copying, but the Argument object is still not forced. Modified: pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py Mon Nov 15 15:44:17 2010 @@ -382,7 +382,7 @@ # XXX a bit too many guards, but better than before assert len(op.get_opnames("guard")) <= 10 - def test_stararg(self): + def test_stararg_virtual(self): self.run_source(''' d = {} @@ -419,6 +419,34 @@ assert len(op.get_opnames("new")) == 0 assert len(op.get_opnames("call_may_force")) == 0 + def test_stararg(self): + self.run_source(''' + d = {} + + def g(*args): + return args[-1] + def h(*args): + return len(args) + + def main(x): + s = 0 + l = [] + i = 0 + while i < x: + l.append(1) + s += g(*l) + i = h(*l) + return s + ''', 100000, ([100], 100), + ([1000], 1000), + ([2000], 2000), + ([4000], 4000)) + assert len(self.loops) == 1 + ops = self.get_by_bytecode("CALL_FUNCTION_VAR") + for op in ops: + assert len(op.get_opnames("new_with_vtable")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 + def test_virtual_instance(self): self.run_source(''' class A(object): From arigo at codespeak.net Mon Nov 15 15:49:48 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 15:49:48 +0100 (CET) Subject: [pypy-svn] r79108 - in pypy/branch/jit-free/pypy/jit/metainterp: . test Message-ID: <20101115144948.1FBC0282B9C@codespeak.net> Author: arigo Date: Mon Nov 15 15:49:45 2010 New Revision: 79108 Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_logger.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_pyjitpl.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmspot.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmstate.py pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Log: (antocuni, arigo) Fix tests. Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_logger.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_logger.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_logger.py Mon Nov 15 15:49:45 2010 @@ -112,7 +112,8 @@ equaloplists(loop.operations, oloop.operations) def test_jump(self): - namespace = {'target': LoopToken(3)} + namespace = {'target': LoopToken()} + namespace['target'].number = 3 inp = ''' [i0] jump(i0, descr=target) Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_pyjitpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_pyjitpl.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_pyjitpl.py Mon Nov 15 15:49:45 2010 @@ -17,6 +17,7 @@ portal.setup(None) class FakeStaticData: cpu = None + warmrunnerdesc = None metainterp = pyjitpl.MetaInterp(FakeStaticData(), None) metainterp.framestack = [] @@ -53,6 +54,7 @@ def test_remove_consts_and_duplicates(): class FakeStaticData: cpu = None + warmrunnerdesc = None def is_another_box_like(box, referencebox): assert box is not referencebox assert isinstance(box, referencebox.clonebox().__class__) Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmspot.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmspot.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmspot.py Mon Nov 15 15:49:45 2010 @@ -378,23 +378,30 @@ exc_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True) cls.exc_vtable = exc_vtable - class FakeFailDescr(object): + class FakeLoopToken: def __init__(self, no): self.no = no + self.generation = 0 + + class FakeFailDescr(object): + def __init__(self, looptoken): + assert isinstance(looptoken, FakeLoopToken) + self.looptoken = looptoken def handle_fail(self, metainterp_sd): - if self.no == 0: + no = self.looptoken.no + if no == 0: raise metainterp_sd.warmrunnerdesc.DoneWithThisFrameInt(3) - if self.no == 1: + if no == 1: raise metainterp_sd.warmrunnerdesc.ContinueRunningNormally( [0], [], [], [1], [], []) - if self.no == 3: + if no == 3: exc = lltype.malloc(OBJECT) exc.typeptr = exc_vtable raise metainterp_sd.warmrunnerdesc.ExitFrameWithExceptionRef( metainterp_sd.cpu, lltype.cast_opaque_ptr(llmemory.GCREF, exc)) - return self.no + return self.looptoken class FakeDescr: def as_vtable_size_descr(self): @@ -419,11 +426,11 @@ sizeof = nodescr def get_fail_descr_from_number(self, no): - return FakeFailDescr(no) + return FakeFailDescr(FakeLoopToken(no)) def execute_token(self, token): - assert token == 2 - return FakeFailDescr(1) + assert token.no == 2 + return FakeFailDescr(FakeLoopToken(1)) driver = JitDriver(reds = ['red'], greens = ['green']) Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmstate.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmstate.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmstate.py Mon Nov 15 15:49:45 2010 @@ -99,6 +99,8 @@ lltype.Float], lltype.Void)) class FakeWarmRunnerDesc: rtyper = FakeRTyper() + cpu = None + memory_manager = None class FakeJitDriverSD: _get_jitcell_at_ptr = llhelper(GETTER, getter) _set_jitcell_at_ptr = llhelper(SETTER, setter) @@ -126,6 +128,7 @@ future_values[j] = "float", value class FakeWarmRunnerDesc: cpu = FakeCPU() + memory_manager = None class FakeJitDriverSD: _red_args_types = ["int", "float"] virtualizable_info = None @@ -164,6 +167,7 @@ def test_make_jitdriver_callbacks_1(): class FakeWarmRunnerDesc: cpu = None + memory_manager = None class FakeJitDriverSD: _green_args_spec = [lltype.Signed, lltype.Float] _get_printable_location_ptr = None @@ -189,6 +193,7 @@ class FakeWarmRunnerDesc: rtyper = None cpu = None + memory_manager = None class FakeJitDriverSD: _green_args_spec = [lltype.Signed, lltype.Float] _get_printable_location_ptr = llhelper(GET_LOCATION, get_location) @@ -211,6 +216,7 @@ class FakeWarmRunnerDesc: rtyper = None cpu = None + memory_manager = None class FakeJitDriverSD: _green_args_spec = [lltype.Signed, lltype.Float] _get_printable_location_ptr = None @@ -233,6 +239,7 @@ class FakeWarmRunnerDesc: rtyper = None cpu = None + memory_manager = None class FakeJitDriverSD: _green_args_spec = [lltype.Signed, lltype.Float] _get_printable_location_ptr = None Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Mon Nov 15 15:49:45 2010 @@ -165,7 +165,8 @@ "NOT_RPYTHON" self.warmrunnerdesc = warmrunnerdesc self.jitdriver_sd = jitdriver_sd - self.cpu = warmrunnerdesc.cpu + if warmrunnerdesc is not None: # for tests + self.cpu = warmrunnerdesc.cpu try: self.profiler = warmrunnerdesc.metainterp_sd.profiler except AttributeError: # for tests @@ -217,7 +218,9 @@ def set_param_loop_longevity(self, value): # note: it's a global parameter, not a per-jitdriver one - self.warmrunnerdesc.memory_manager.set_max_age(value) + if (self.warmrunnerdesc is not None and + self.warmrunnerdesc.memory_manager is not None): # all for tests + self.warmrunnerdesc.memory_manager.set_max_age(value) def disable_noninlinable_function(self, greenkey): cell = self.jit_cell_at_key(greenkey) From afa at codespeak.net Mon Nov 15 16:29:18 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 16:29:18 +0100 (CET) Subject: [pypy-svn] r79109 - in pypy/branch/fast-forward/pypy: module/_winreg rlib Message-ID: <20101115152918.2E0C7282B9C@codespeak.net> Author: afa Date: Mon Nov 15 16:29:16 2010 New Revision: 79109 Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py pypy/branch/fast-forward/pypy/rlib/rwinreg.py Log: Prevent race conditions and retry when ERROR_MORE_DATA is returned. Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py Mon Nov 15 16:29:16 2010 @@ -3,7 +3,7 @@ from pypy.interpreter.baseobjspace import ObjSpace, W_Root from pypy.interpreter.gateway import interp2app, Arguments from pypy.interpreter.typedef import TypeDef, GetSetProperty -from pypy.interpreter.error import OperationError +from pypy.interpreter.error import OperationError, wrap_windowserror from pypy.rpython.lltypesystem import rffi, lltype from pypy.rlib import rwinreg, rwin32 from pypy.rlib.rarithmetic import r_uint, intmask @@ -254,13 +254,24 @@ subkey = space.str_w(w_subkey) with lltype.scoped_alloc(rwin32.PLONG.TO, 1) as bufsize_p: ret = rwinreg.RegQueryValue(hkey, subkey, None, bufsize_p) - if ret != 0: + bufSize = intmask(bufsize_p[0]) + if ret == rwinreg.ERROR_MORE_DATA: + bufSize = 256 + elif ret != 0: raiseWindowsError(space, ret, 'RegQueryValue') - with lltype.scoped_alloc(rffi.CCHARP.TO, bufsize_p[0]) as buf: - ret = rwinreg.RegQueryValue(hkey, subkey, buf, bufsize_p) - if ret != 0: - raiseWindowsError(space, ret, 'RegQueryValue') - return space.wrap(rffi.charp2strn(buf, bufsize_p[0] - 1)) + + while True: + with lltype.scoped_alloc(rffi.CCHARP.TO, bufSize) as buf: + ret = rwinreg.RegQueryValue(hkey, subkey, buf, bufsize_p) + if ret == rwinreg.ERROR_MORE_DATA: + # Resize and retry + bufSize *= 2 + bufsize_p[0] = bufSize + continue + + if ret != 0: + raiseWindowsError(space, ret, 'RegQueryValue') + return space.wrap(rffi.charp2strn(buf, bufsize_p[0] - 1)) QueryValue.unwrap_spec = [ObjSpace, W_Root, W_Root] def convert_to_regdata(space, w_value, typ): @@ -537,24 +548,38 @@ # include null terminators retValueSize[0] += 1 retDataSize[0] += 1 + bufDataSize = intmask(retDataSize[0]) + bufValueSize = intmask(retValueSize[0]) with lltype.scoped_alloc(rffi.CCHARP.TO, intmask(retValueSize[0])) as valuebuf: - with lltype.scoped_alloc(rffi.CCHARP.TO, - intmask(retDataSize[0])) as databuf: - with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType: - ret = rwinreg.RegEnumValue( - hkey, index, valuebuf, retValueSize, - null_dword, retType, databuf, retDataSize) - if ret != 0: - raiseWindowsError(space, ret, 'RegEnumValue') - - return space.newtuple([ - space.wrap(rffi.charp2str(valuebuf)), - convert_from_regdata(space, databuf, - retDataSize[0], retType[0]), - space.wrap(retType[0]), - ]) + while True: + with lltype.scoped_alloc(rffi.CCHARP.TO, + bufDataSize) as databuf: + with lltype.scoped_alloc(rwin32.LPDWORD.TO, + 1) as retType: + ret = rwinreg.RegEnumValue( + hkey, index, valuebuf, retValueSize, + null_dword, retType, databuf, retDataSize) + if ret == rwinreg.ERROR_MORE_DATA: + # Resize and retry + bufDataSize *= 2 + retDataSize[0] = rffi.cast(rwin32.DWORD, + bufDataSize) + retValueSize[0] = rffi.cast(rwin32.DWORD, + bufValueSize) + continue + + if ret != 0: + raiseWindowsError(space, ret, 'RegEnumValue') + + return space.newtuple([ + space.wrap(rffi.charp2str(valuebuf)), + convert_from_regdata(space, databuf, + retDataSize[0], + retType[0]), + space.wrap(retType[0]), + ]) EnumValue.unwrap_spec = [ObjSpace, W_Root, int] Modified: pypy/branch/fast-forward/pypy/rlib/rwinreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rwinreg.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rwinreg.py Mon Nov 15 16:29:16 2010 @@ -28,6 +28,8 @@ HKEY_LOCAL_MACHINE HKEY_CLASSES_ROOT HKEY_CURRENT_CONFIG HKEY_CURRENT_USER HKEY_DYN_DATA HKEY_LOCAL_MACHINE HKEY_PERFORMANCE_DATA HKEY_USERS + +ERROR_MORE_DATA '''.split() for name in constant_names: setattr(CConfig, name, platform.DefinedConstantInteger(name)) From afa at codespeak.net Mon Nov 15 17:02:54 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 17:02:54 +0100 (CET) Subject: [pypy-svn] r79111 - in pypy/branch/fast-forward/pypy/module/_winreg: . test Message-ID: <20101115160254.BD868282B9C@codespeak.net> Author: afa Date: Mon Nov 15 17:02:53 2010 New Revision: 79111 Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py Log: Same thing for _winreg.QueryValuEx() Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py Mon Nov 15 17:02:53 2010 @@ -423,34 +423,48 @@ raiseWindowsError(space, ret, 'RegSetValueEx') SetValueEx.unwrap_spec = [ObjSpace, W_Root, str, W_Root, int, W_Root] -def QueryValueEx(space, w_hkey, subkey): +def QueryValueEx(space, w_hkey, w_subkey): """value,type_id = QueryValueEx(key, value_name) - Retrieves the type and data for a specified value name associated with an open registry key. key is an already open key, or any one of the predefined HKEY_* constants. value_name is a string indicating the value to query""" hkey = hkey_w(w_hkey, space) + if space.is_w(w_subkey, space.w_None): + subkey = None + else: + subkey = space.str_w(w_subkey) null_dword = lltype.nullptr(rwin32.LPDWORD.TO) with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retDataSize: ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword, null_dword, None, retDataSize) - if ret != 0: + bufSize = intmask(retDataSize[0]) + if ret == rwinreg.ERROR_MORE_DATA: + bufSize = 256 + elif ret != 0: raiseWindowsError(space, ret, 'RegQueryValueEx') - with lltype.scoped_alloc(rffi.CCHARP.TO, - intmask(retDataSize[0])) as databuf: - with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType: - - ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword, - retType, databuf, retDataSize) - if ret != 0: - raiseWindowsError(space, ret, 'RegQueryValueEx') - return space.newtuple([ - convert_from_regdata(space, databuf, - retDataSize[0], retType[0]), - space.wrap(retType[0]), - ]) + while True: + with lltype.scoped_alloc(rffi.CCHARP.TO, bufSize) as databuf: + with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType: + + ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword, + retType, databuf, retDataSize) + if ret == rwinreg.ERROR_MORE_DATA: + # Resize and retry + bufSize *= 2 + print "AFA CONTINUE", bufSize, retDataSize[0] + retDataSize[0] = rffi.cast(rwin32.DWORD, bufSize) + continue + if ret != 0: + raiseWindowsError(space, ret, 'RegQueryValueEx') + print "AFA OK", bufSize, retDataSize[0] + return space.newtuple([ + convert_from_regdata(space, databuf, + retDataSize[0], retType[0]), + space.wrap(retType[0]), + ]) -QueryValueEx.unwrap_spec = [ObjSpace, W_Root, str] +QueryValueEx.unwrap_spec = [ObjSpace, W_Root, W_Root] def CreateKey(space, w_hkey, subkey): """key = CreateKey(key, sub_key) - Creates or opens the specified key. Modified: pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py Mon Nov 15 17:02:53 2010 @@ -187,3 +187,9 @@ r = ExpandEnvironmentStrings(u"%windir%\\test") assert isinstance(r, unicode) assert r == nt.environ["WINDIR"] + "\\test" + + def test_dynamic_key(self): + from _winreg import EnumValue, QueryValueEx, HKEY_PERFORMANCE_DATA + EnumValue(HKEY_PERFORMANCE_DATA, 0) + QueryValueEx(HKEY_PERFORMANCE_DATA, None) + From arigo at codespeak.net Mon Nov 15 17:10:29 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 17:10:29 +0100 (CET) Subject: [pypy-svn] r79112 - in pypy/branch/jit-free/pypy/jit/metainterp: . test Message-ID: <20101115161029.AB3EC5080F@codespeak.net> Author: arigo Date: Mon Nov 15 17:10:27 2010 New Revision: 79112 Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py pypy/branch/jit-free/pypy/jit/metainterp/history.py pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_basic.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmstate.py pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Log: (antocuni, arigo) Turn the storage of looptokens into nice methods on JitCell, and internally store weakrefs. Some debug_start/debug_print/debug_stops. Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/compile.py Mon Nov 15 17:10:27 2010 @@ -500,12 +500,14 @@ jitdriver_sd.warmstate.attach_unoptimized_bridge_from_interp( original_greenkey, new_loop_token) - # store the new loop in compiled_merge_points too + # store the new loop in compiled_merge_points_wref too old_loop_tokens = metainterp.get_compiled_merge_points( original_greenkey) # it always goes at the end of the list, as it is the most # general loop token old_loop_tokens.append(new_loop_token) + metainterp.set_compiled_merge_points(original_greenkey, + old_loop_tokens) def reset_counter_from_failure(self): pass Modified: pypy/branch/jit-free/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/history.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/history.py Mon Nov 15 17:10:27 2010 @@ -5,6 +5,7 @@ from pypy.rlib.objectmodel import we_are_translated, r_dict, Symbolic from pypy.rlib.objectmodel import compute_hash, compute_unique_id from pypy.rlib.rarithmetic import intmask, r_longlong +from pypy.rlib.debug import debug_start, debug_print, debug_stop from pypy.tool.uid import uid from pypy.conftest import option @@ -753,7 +754,10 @@ # MemoryManager.keep_loop_alive() has been called on this # loop token, which means that it has been successfully # compiled by the backend. Free it now. + debug_start("jit-free-looptoken") + debug_print("Freeing loop #", self.number) self.cpu.free_loop_and_bridges(self) + debug_stop("jit-free-looptoken") def repr_of_descr(self): return '' % self.number Modified: pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py Mon Nov 15 17:10:27 2010 @@ -1,14 +1,16 @@ import math from pypy.rlib.rarithmetic import r_longlong +from pypy.rlib.debug import debug_start, debug_print, debug_stop # # Logic to decide which loops are old and not used any more. # -# All the long-lived references to LoopToken are weakrefs, apart from -# the 'alive_loops' set in MemoryManager, which is the only (long-living) -# place that keeps them alive. If a loop was not called for long enough, -# then it is removed from 'alive_loops'. It will soon be freed by the -# GC. LoopToken.__del__ calls the method cpu.free_loop_and_bridges(). +# All the long-lived references to LoopToken are weakrefs (see JitCell +# in warmstate.py), apart from the 'alive_loops' set in MemoryManager, +# which is the only (long-living) place that keeps them alive. If a +# loop was not called for long enough, then it is removed from +# 'alive_loops'. It will soon be freed by the GC. LoopToken.__del__ +# calls the method cpu.free_loop_and_bridges(). # # The alive_loops set is maintained using the notion of a global # 'current generation' which is, in practice, the total number of loops @@ -55,7 +57,14 @@ self.alive_loops[looptoken] = None def _kill_old_loops_now(self): + debug_start("jit-free-memmgr") + oldtotal = len(self.alive_loops) + debug_print("Loop tokens before:", oldtotal) max_generation = self.current_generation - self.max_age for looptoken in self.alive_loops.keys(): if 0 <= looptoken.generation < max_generation: del self.alive_loops[looptoken] + newtotal = len(self.alive_loops) + debug_print("Loop tokens freed: ", oldtotal - newtotal) + debug_print("Loop tokens left: ", newtotal) + debug_stop("jit-free-memmgr") Modified: pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py Mon Nov 15 17:10:27 2010 @@ -1796,10 +1796,15 @@ raise NotImplementedError(opname[opnum]) def get_compiled_merge_points(self, greenkey): + """Get the list of looptokens corresponding to the greenkey. + Turns the (internal) list of weakrefs into regular refs. + """ + cell = self.jitdriver_sd.warmstate.jit_cell_at_key(greenkey) + return cell.get_compiled_merge_points() + + def set_compiled_merge_points(self, greenkey, looptokens): cell = self.jitdriver_sd.warmstate.jit_cell_at_key(greenkey) - if cell.compiled_merge_points is None: - cell.compiled_merge_points = [] - return cell.compiled_merge_points + cell.set_compiled_merge_points(looptokens) def compile(self, original_boxes, live_arg_boxes, start): num_green_args = self.jitdriver_sd.num_green_args @@ -1810,6 +1815,7 @@ loop_token = compile.compile_new_loop(self, old_loop_tokens, greenkey, start) if loop_token is not None: # raise if it *worked* correctly + self.set_compiled_merge_points(greenkey, old_loop_tokens) raise GenerateMergePoint(live_arg_boxes, loop_token) self.history.operations.pop() # remove the JUMP Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_basic.py Mon Nov 15 17:10:27 2010 @@ -19,7 +19,11 @@ from pypy.jit.metainterp import simple_optimize class FakeJitCell: - compiled_merge_points = None + __compiled_merge_points = [] + def get_compiled_merge_points(self): + return self.__compiled_merge_points[:] + def set_compiled_merge_points(self, lst): + self.__compiled_merge_points = lst class FakeWarmRunnerState: def attach_unoptimized_bridge_from_interp(self, greenkey, newloop): Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmstate.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmstate.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmstate.py Mon Nov 15 17:10:27 2010 @@ -157,12 +157,15 @@ _get_jitcell_at_ptr = None state = WarmEnterState(None, FakeJitDriverSD()) get_jitcell = state.make_jitcell_getter() + class FakeLoopToken(object): + pass + looptoken = FakeLoopToken() state.attach_unoptimized_bridge_from_interp([ConstInt(5), ConstFloat(2.25)], - "entry loop token") + looptoken) cell1 = get_jitcell(True, 5, 2.25) assert cell1.counter < 0 - assert cell1.entry_loop_token == "entry loop token" + assert cell1.get_entry_loop_token() is looptoken def test_make_jitdriver_callbacks_1(): class FakeWarmRunnerDesc: Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Mon Nov 15 17:10:27 2010 @@ -1,4 +1,4 @@ -import sys +import sys, weakref from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi from pypy.rpython.ootypesystem import ootype from pypy.rpython.annlowlevel import hlstr, llstr, cast_base_ptr_to_instance @@ -150,9 +150,34 @@ # counter == -1: there is an entry bridge for this cell # counter == -2: tracing is currently going on for this cell counter = 0 - compiled_merge_points = None + compiled_merge_points_wref = None # list of weakrefs to LoopToken dont_trace_here = False - entry_loop_token = None + wref_entry_loop_token = None # (possibly) one weakref to LoopToken + + def get_compiled_merge_points(self): + result = [] + if self.compiled_merge_points_wref is not None: + for wref in self.compiled_merge_points_wref: + looptoken = wref() + if looptoken is not None: + result.append(looptoken) + return result + + def set_compiled_merge_points(self, looptokens): + self.compiled_merge_points_wref = [self._makeref(token) + for token in looptokens] + + def get_entry_loop_token(self): + if self.wref_entry_loop_token is not None: + return self.wref_entry_loop_token() + return None + + def set_entry_loop_token(self, looptoken): + self.wref_entry_loop_token = self._makeref(looptoken) + + def _makeref(self, looptoken): + assert looptoken is not None + return weakref.ref(looptoken) # ____________________________________________________________ @@ -234,8 +259,8 @@ entry_loop_token): cell = self.jit_cell_at_key(greenkey) cell.counter = -1 - old_token = cell.entry_loop_token - cell.entry_loop_token = entry_loop_token + old_token = cell.get_entry_loop_token() + cell.set_entry_loop_token(entry_loop_token) if old_token is not None: self.cpu.redirect_call_assembler(old_token, entry_loop_token) @@ -308,7 +333,7 @@ # machine code was already compiled for these greenargs # get the assembler and fill in the boxes set_future_values(*args[num_green_args:]) - loop_token = cell.entry_loop_token + loop_token = cell.get_entry_loop_token() # ---------- execute assembler ---------- while True: # until interrupted by an exception @@ -532,11 +557,13 @@ def get_assembler_token(greenkey, redboxes): # 'redboxes' is only used to know the types of red arguments cell = self.jit_cell_at_key(greenkey) - if cell.entry_loop_token is None: + entry_loop_token = cell.get_entry_loop_token() + if entry_loop_token is None: from pypy.jit.metainterp.compile import compile_tmp_callback - cell.entry_loop_token = compile_tmp_callback(cpu, jd, greenkey, - redboxes) - return cell.entry_loop_token + entry_loop_token = compile_tmp_callback(cpu, jd, greenkey, + redboxes) + cell.set_entry_loop_token(entry_loop_token) + return entry_loop_token self.get_assembler_token = get_assembler_token # From afa at codespeak.net Mon Nov 15 17:26:37 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 17:26:37 +0100 (CET) Subject: [pypy-svn] r79113 - in pypy/branch/fast-forward/pypy/module/_winreg: . test Message-ID: <20101115162637.D9C32282B9C@codespeak.net> Author: afa Date: Mon Nov 15 17:26:36 2010 New Revision: 79113 Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py Log: Implement issue2810 for pypy. Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py Mon Nov 15 17:26:36 2010 @@ -609,10 +609,15 @@ hkey = hkey_w(w_hkey, space) null_dword = lltype.nullptr(rwin32.LPDWORD.TO) - # max key name length is 255 - with lltype.scoped_alloc(rffi.CCHARP.TO, 256) as buf: + # The Windows docs claim that the max key name length is 255 + # characters, plus a terminating nul character. However, + # empirical testing demonstrates that it is possible to + # create a 256 character key that is missing the terminating + # nul. RegEnumKeyEx requires a 257 character buffer to + # retrieve such a key name. + with lltype.scoped_alloc(rffi.CCHARP.TO, 257) as buf: with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retValueSize: - retValueSize[0] = r_uint(256) # includes NULL terminator + retValueSize[0] = r_uint(257) # includes NULL terminator ret = rwinreg.RegEnumKeyEx(hkey, index, buf, retValueSize, null_dword, None, null_dword, lltype.nullptr(rwin32.PFILETIME.TO)) Modified: pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py Mon Nov 15 17:26:36 2010 @@ -188,6 +188,22 @@ assert isinstance(r, unicode) assert r == nt.environ["WINDIR"] + "\\test" + def test_long_key(self): + from _winreg import ( + HKEY_CURRENT_USER, KEY_ALL_ACCESS, CreateKey, SetValue, EnumKey, + REG_SZ, QueryInfoKey, OpenKey, DeleteKey) + name = 'x'*256 + try: + with CreateKey(HKEY_CURRENT_USER, self.test_key_name) as key: + SetValue(key, name, REG_SZ, 'x') + num_subkeys, num_values, t = QueryInfoKey(key) + EnumKey(key, 0) + finally: + with OpenKey(HKEY_CURRENT_USER, self.test_key_name, 0, + KEY_ALL_ACCESS) as key: + DeleteKey(key, name) + DeleteKey(HKEY_CURRENT_USER, self.test_key_name) + def test_dynamic_key(self): from _winreg import EnumValue, QueryValueEx, HKEY_PERFORMANCE_DATA EnumValue(HKEY_PERFORMANCE_DATA, 0) From afa at codespeak.net Mon Nov 15 17:54:04 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 17:54:04 +0100 (CET) Subject: [pypy-svn] r79114 - in pypy/branch/fast-forward/pypy/module/_winreg: . test Message-ID: <20101115165404.8CADA282B90@codespeak.net> Author: afa Date: Mon Nov 15 17:54:03 2010 New Revision: 79114 Modified: pypy/branch/fast-forward/pypy/module/_winreg/__init__.py pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py Log: Add "ReflectionKey" functions, they always raise NotImplemented: they only work on win64 anyway. Modified: pypy/branch/fast-forward/pypy/module/_winreg/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/__init__.py Mon Nov 15 17:54:03 2010 @@ -63,6 +63,11 @@ 'ConnectRegistry': 'interp_winreg.ConnectRegistry', 'ExpandEnvironmentStrings': 'interp_winreg.ExpandEnvironmentStrings', + + 'DisableReflectionKey': 'interp_winreg.DisableReflectionKey', + 'EnableReflectionKey': 'interp_winreg.EnableReflectionKey', + 'QueryReflectionKey': 'interp_winreg.QueryReflectionKey', + 'DeleteKeyEx': 'interp_winreg.DeleteKeyEx', } for name, value in constants.iteritems(): Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py Mon Nov 15 17:54:03 2010 @@ -688,3 +688,46 @@ except WindowsError, e: raise wrap_windowserror(space, e) ExpandEnvironmentStrings.unwrap_spec = [ObjSpace, unicode] + +def DisableReflectionKey(space, w_key): + """Disables registry reflection for 32-bit processes running on a 64-bit + Operating System. Will generally raise NotImplemented if executed on + a 32-bit Operating System. + If the key is not on the reflection list, the function succeeds but has no effect. + Disabling reflection for a key does not affect reflection of any subkeys.""" + raise OperationError(space.w_NotImplementedError, space.wrap( + "not implemented on this platform")) +DisableReflectionKey.unwrap_spec = [ObjSpace, W_Root] + +def EnableReflectionKey(space, w_key): + """Restores registry reflection for the specified disabled key. + Will generally raise NotImplemented if executed on a 32-bit Operating System. + Restoring reflection for a key does not affect reflection of any subkeys.""" + raise OperationError(space.w_NotImplementedError, space.wrap( + "not implemented on this platform")) +EnableReflectionKey.unwrap_spec = [ObjSpace, W_Root] + +def QueryReflectionKey(space, w_key): + """bool = QueryReflectionKey(hkey) - Determines the reflection state for the specified key. + Will generally raise NotImplemented if executed on a 32-bit Operating System.""" + raise OperationError(space.w_NotImplementedError, space.wrap( + "not implemented on this platform")) +QueryReflectionKey.unwrap_spec = [ObjSpace, W_Root] + +def DeleteKeyEx(space, w_key, subkey): + """DeleteKeyEx(key, sub_key, sam, res) - Deletes the specified key. + + key is an already open key, or any one of the predefined HKEY_* constants. + sub_key is a string that must be a subkey of the key identified by the key parameter. + res is a reserved integer, and must be zero. Default is zero. + sam is an integer that specifies an access mask that describes the desired + This value must not be None, and the key may not have subkeys. + + This method can not delete keys with subkeys. + + If the method succeeds, the entire key, including all of its values, + is removed. If the method fails, a WindowsError exception is raised. + On unsupported Windows versions, NotImplementedError is raised.""" + raise OperationError(space.w_NotImplementedError, space.wrap( + "not implemented on this platform")) +DeleteKeyEx.unwrap_spec = [ObjSpace, W_Root, str] Modified: pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py Mon Nov 15 17:54:03 2010 @@ -209,3 +209,17 @@ EnumValue(HKEY_PERFORMANCE_DATA, 0) QueryValueEx(HKEY_PERFORMANCE_DATA, None) + def test_reflection_unsupported(self): + import sys + if sys.getwindowsversion() >= (5, 2): + skip("Requires Windows XP") + from _winreg import ( + CreateKey, DisableReflectionKey, EnableReflectionKey, + QueryReflectionKey, DeleteKeyEx) + with CreateKey(self.root_key, self.test_key_name) as key: + raises(NotImplementedError, DisableReflectionKey, key) + raises(NotImplementedError, EnableReflectionKey, key) + raises(NotImplementedError, QueryReflectionKey, key) + raises(NotImplementedError, DeleteKeyEx, self.root_key, + self.test_key_name) + From afa at codespeak.net Mon Nov 15 18:18:22 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 18:18:22 +0100 (CET) Subject: [pypy-svn] r79116 - in pypy/branch/fast-forward/pypy: module/_winreg module/_winreg/test rlib Message-ID: <20101115171822.867EE282B90@codespeak.net> Author: afa Date: Mon Nov 15 18:18:21 2010 New Revision: 79116 Modified: pypy/branch/fast-forward/pypy/module/_winreg/__init__.py pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py pypy/branch/fast-forward/pypy/rlib/rwinreg.py Log: add CreateKeyEx; this fixes the last test in 2.7's test_winreg.py Modified: pypy/branch/fast-forward/pypy/module/_winreg/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/__init__.py Mon Nov 15 18:18:21 2010 @@ -49,6 +49,7 @@ 'QueryValue' : 'interp_winreg.QueryValue', 'QueryValueEx' : 'interp_winreg.QueryValueEx', 'CreateKey' : 'interp_winreg.CreateKey', + 'CreateKeyEx' : 'interp_winreg.CreateKeyEx', 'DeleteKey' : 'interp_winreg.DeleteKey', 'DeleteValue' : 'interp_winreg.DeleteValue', 'OpenKey' : 'interp_winreg.OpenKey', Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py Mon Nov 15 18:18:21 2010 @@ -486,6 +486,28 @@ return space.wrap(W_HKEY(rethkey[0])) CreateKey.unwrap_spec = [ObjSpace, W_Root, str] +def CreateKeyEx(space, w_hkey, subkey, res=0, sam=rwinreg.KEY_WRITE): + """key = CreateKey(key, sub_key) - Creates or opens the specified key. + +key is an already open key, or one of the predefined HKEY_* constants +sub_key is a string that names the key this method opens or creates. + If key is one of the predefined keys, sub_key may be None. In that case, + the handle returned is the same key handle passed in to the function. + +If the key already exists, this function opens the existing key + +The return value is the handle of the opened key. +If the function fails, an exception is raised.""" + hkey = hkey_w(w_hkey, space) + with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey: + ret = rwinreg.RegCreateKeyEx(hkey, subkey, res, None, 0, + sam, None, rethkey, + lltype.nullptr(rwin32.LPDWORD.TO)) + if ret != 0: + raiseWindowsError(space, ret, 'CreateKeyEx') + return space.wrap(W_HKEY(rethkey[0])) +CreateKeyEx.unwrap_spec = [ObjSpace, W_Root, str, int, rffi.r_uint] + def DeleteKey(space, w_hkey, subkey): """DeleteKey(key, sub_key) - Deletes the specified key. Modified: pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/test/test_winreg.py Mon Nov 15 18:18:21 2010 @@ -81,6 +81,18 @@ nkeys, nvalues, since_mod = QueryInfoKey(sub_key) assert nkeys == 0 + def test_CreateKeyEx(self): + from _winreg import CreateKeyEx, QueryInfoKey + from _winreg import KEY_ALL_ACCESS, KEY_READ + key = CreateKeyEx(self.root_key, self.test_key_name, 0, KEY_ALL_ACCESS) + sub_key = CreateKeyEx(key, "sub_key", 0, KEY_READ) + + nkeys, nvalues, since_mod = QueryInfoKey(key) + assert nkeys == 1 + + nkeys, nvalues, since_mod = QueryInfoKey(sub_key) + assert nkeys == 0 + def test_close(self): from _winreg import OpenKey, CloseKey, FlushKey, QueryInfoKey key = OpenKey(self.root_key, self.test_key_name) Modified: pypy/branch/fast-forward/pypy/rlib/rwinreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rwinreg.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rwinreg.py Mon Nov 15 18:18:21 2010 @@ -74,6 +74,12 @@ [HKEY, rffi.CCHARP, PHKEY], rffi.LONG) +RegCreateKeyEx = external( + 'RegCreateKeyExA', + [HKEY, rffi.CCHARP, rwin32.DWORD, rffi.CCHARP, rwin32.DWORD, + REGSAM, rffi.VOIDP, PHKEY, rwin32.LPDWORD], + rffi.LONG) + RegDeleteValue = external( 'RegDeleteValueA', [HKEY, rffi.CCHARP], From arigo at codespeak.net Mon Nov 15 18:32:53 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 18:32:53 +0100 (CET) Subject: [pypy-svn] r79117 - in pypy/trunk/pypy: config translator/goal Message-ID: <20101115173253.547D0282B90@codespeak.net> Author: arigo Date: Mon Nov 15 18:32:51 2010 New Revision: 79117 Modified: pypy/trunk/pypy/config/pypyoption.py pypy/trunk/pypy/translator/goal/targetpypystandalone.py Log: Add the translation option --translationmodules to pypy. Enables precisely those modules that are needed to run a translation of pypy on top of the generated pypy. Modified: pypy/trunk/pypy/config/pypyoption.py ============================================================================== --- pypy/trunk/pypy/config/pypyoption.py (original) +++ pypy/trunk/pypy/config/pypyoption.py Mon Nov 15 18:32:51 2010 @@ -34,6 +34,11 @@ "_bisect"] )) +translation_modules = default_modules.copy() +translation_modules.update(dict.fromkeys( + ["fcntl", "rctime", "select", "signal", "_rawffi", "zlib", + "struct", "md5", "cStringIO", "array"])) + working_oo_modules = default_modules.copy() working_oo_modules.update(dict.fromkeys( ["md5", "sha", "cStringIO", "itertools"] @@ -149,6 +154,12 @@ cmdline="--allworkingmodules", negation=True), + BoolOption("translationmodules", + "use only those modules that are needed to run translate.py on pypy", + default=False, + cmdline="--translationmodules", + suggests=[("objspace.allworkingmodules", False)]), + BoolOption("geninterp", "specify whether geninterp should be used", cmdline=None, default=True), @@ -369,6 +380,11 @@ modules = [name for name in modules if name not in essential_modules] config.objspace.usemodules.suggest(**dict.fromkeys(modules, True)) +def enable_translationmodules(config): + modules = translation_modules + modules = [name for name in modules if name not in essential_modules] + config.objspace.usemodules.suggest(**dict.fromkeys(modules, True)) + if __name__ == '__main__': config = get_pypy_config() Modified: pypy/trunk/pypy/translator/goal/targetpypystandalone.py ============================================================================== --- pypy/trunk/pypy/translator/goal/targetpypystandalone.py (original) +++ pypy/trunk/pypy/translator/goal/targetpypystandalone.py Mon Nov 15 18:32:51 2010 @@ -150,6 +150,9 @@ if config.objspace.allworkingmodules: from pypy.config.pypyoption import enable_allworkingmodules enable_allworkingmodules(config) + if config.objspace.translationmodules: + from pypy.config.pypyoption import enable_translationmodules + enable_translationmodules(config) if config.translation.type_system == 'ootype': config.objspace.usemodules.suggest(rbench=True) From arigo at codespeak.net Mon Nov 15 19:06:46 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 19:06:46 +0100 (CET) Subject: [pypy-svn] r79118 - in pypy/branch/jit-free/pypy/jit/metainterp: . test Message-ID: <20101115180646.9B215282B90@codespeak.net> Author: arigo Date: Mon Nov 15 19:06:44 2010 New Revision: 79118 Modified: pypy/branch/jit-free/pypy/jit/metainterp/blackhole.py pypy/branch/jit-free/pypy/jit/metainterp/compile.py pypy/branch/jit-free/pypy/jit/metainterp/history.py pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py pypy/branch/jit-free/pypy/jit/metainterp/resume.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Log: In-progress: revert the "cleanup" part of r79094. It was a really bad idea after all to put a strong reference from the ResumeDescrs to the LoopToken. Indeed, the LoopTokens are supposed to be only stored in weak lists on JitCell and be only be kept alive carefully by memmgr.py. Modified: pypy/branch/jit-free/pypy/jit/metainterp/blackhole.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/blackhole.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/blackhole.py Mon Nov 15 19:06:44 2010 @@ -1362,12 +1362,14 @@ # We will continue to loop in _run_forever() from the parent level. return blackholeinterp, lle -def resume_in_blackhole(metainterp_sd, resumedescr, all_virtuals=None): +def resume_in_blackhole(metainterp_sd, jitdriver_sd, resumedescr, + all_virtuals=None): from pypy.jit.metainterp.resume import blackhole_from_resumedata debug_start('jit-blackhole') metainterp_sd.profiler.start_blackhole() blackholeinterp = blackhole_from_resumedata( metainterp_sd.blackholeinterpbuilder, + jitdriver_sd, resumedescr, all_virtuals) current_exc = blackholeinterp._prepare_resume_from_failure( Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/compile.py Mon Nov 15 19:06:44 2010 @@ -42,11 +42,10 @@ name = metainterp.staticdata.stats.name_for_new_loop() return TreeLoop(name) -def make_loop_token(cpu, nb_args, jitdriver_sd, greenkey): +def make_loop_token(cpu, nb_args, jitdriver_sd): loop_token = LoopToken(cpu) loop_token.specnodes = [prebuiltNotSpecNode] * nb_args loop_token.outermost_jitdriver_sd = jitdriver_sd - loop_token.outermost_greenkey = greenkey return loop_token # ____________________________________________________________ @@ -66,7 +65,7 @@ metainterp_sd = metainterp.staticdata jitdriver_sd = metainterp.jitdriver_sd loop_token = make_loop_token(metainterp.cpu, len(loop.inputargs), - jitdriver_sd, greenkey) + jitdriver_sd) loop.token = loop_token loop.operations[-1].setdescr(loop_token) # patch the target of the JUMP try: @@ -143,32 +142,32 @@ pass class DoneWithThisFrameDescrVoid(_DoneWithThisFrameDescr): - def handle_fail(self, metainterp_sd): - #assert jitdriver_sd.result_type == history.VOID + def handle_fail(self, metainterp_sd, jitdriver_sd): + assert jitdriver_sd.result_type == history.VOID raise metainterp_sd.DoneWithThisFrameVoid() class DoneWithThisFrameDescrInt(_DoneWithThisFrameDescr): - def handle_fail(self, metainterp_sd): - #assert jitdriver_sd.result_type == history.INT + def handle_fail(self, metainterp_sd, jitdriver_sd): + assert jitdriver_sd.result_type == history.INT result = metainterp_sd.cpu.get_latest_value_int(0) raise metainterp_sd.DoneWithThisFrameInt(result) class DoneWithThisFrameDescrRef(_DoneWithThisFrameDescr): - def handle_fail(self, metainterp_sd): - #assert jitdriver_sd.result_type == history.REF + def handle_fail(self, metainterp_sd, jitdriver_sd): + assert jitdriver_sd.result_type == history.REF cpu = metainterp_sd.cpu result = cpu.get_latest_value_ref(0) cpu.clear_latest_values(1) raise metainterp_sd.DoneWithThisFrameRef(cpu, result) class DoneWithThisFrameDescrFloat(_DoneWithThisFrameDescr): - def handle_fail(self, metainterp_sd): - #assert jitdriver_sd.result_type == history.FLOAT + def handle_fail(self, metainterp_sd, jitdriver_sd): + assert jitdriver_sd.result_type == history.FLOAT result = metainterp_sd.cpu.get_latest_value_float(0) raise metainterp_sd.DoneWithThisFrameFloat(result) class ExitFrameWithExceptionDescrRef(_DoneWithThisFrameDescr): - def handle_fail(self, metainterp_sd): + def handle_fail(self, metainterp_sd, jitdriver_sd): cpu = metainterp_sd.cpu value = cpu.get_latest_value_ref(0) cpu.clear_latest_values(1) @@ -210,9 +209,8 @@ } class ResumeDescr(AbstractFailDescr): - def __init__(self, original_loop_token): - assert isinstance(original_loop_token, history.LoopToken) - self.original_loop_token = original_loop_token + def __init__(self, original_greenkey): + self.original_greenkey = original_greenkey class ResumeGuardDescr(ResumeDescr): _counter = 0 # if < 0, there is one counter per value; @@ -255,29 +253,26 @@ # a negative value self._counter = cnt | i - def handle_fail(self, metainterp_sd): - if self.must_compile(metainterp_sd): - return self._trace_and_compile_from_bridge(metainterp_sd) + def handle_fail(self, metainterp_sd, jitdriver_sd): + if self.must_compile(metainterp_sd, jitdriver_sd): + return self._trace_and_compile_from_bridge(metainterp_sd, + jitdriver_sd) else: from pypy.jit.metainterp.blackhole import resume_in_blackhole - resume_in_blackhole(metainterp_sd, self) + resume_in_blackhole(metainterp_sd, jitdriver_sd, self) assert 0, "unreachable" - def _trace_and_compile_from_bridge(self, metainterp_sd): + def _trace_and_compile_from_bridge(self, metainterp_sd, jitdriver_sd): + # 'jitdriver_sd' corresponds to the outermost one, i.e. the one + # of the jit_merge_point where we started the loop, even if the + # loop itself may contain temporarily recursion into other + # jitdrivers. from pypy.jit.metainterp.pyjitpl import MetaInterp - metainterp = MetaInterp(metainterp_sd, self.get_jitdriver_sd()) + metainterp = MetaInterp(metainterp_sd, jitdriver_sd) return metainterp.handle_guard_failure(self) _trace_and_compile_from_bridge._dont_inline_ = True - def get_jitdriver_sd(self): - # Returns the jitdriver_sd that corresponds to the outermost - # level, i.e. the level of the jit_merge_point where we started - # the loop, even if the loop itself may contain some recursion - # into other jitdrivers. - return self.original_loop_token.outermost_jitdriver_sd - - def must_compile(self, metainterp_sd): - jitdriver_sd = self.get_jitdriver_sd() + def must_compile(self, metainterp_sd, jitdriver_sd): trace_eagerness = jitdriver_sd.warmstate.trace_eagerness if self._counter >= 0: self._counter += 1 @@ -335,18 +330,18 @@ res.rd_pendingfields = self.rd_pendingfields def _clone_if_mutable(self): - res = ResumeGuardDescr(self.original_loop_token) + res = ResumeGuardDescr(self.original_greenkey) self.copy_all_attrbutes_into(res) return res class ResumeGuardForcedDescr(ResumeGuardDescr): - def __init__(self, metainterp_sd, original_loop_token, jitdriver_sd): - ResumeGuardDescr.__init__(self, original_loop_token) + def __init__(self, metainterp_sd, original_greenkey, jitdriver_sd): + ResumeGuardDescr.__init__(self, original_greenkey) self.metainterp_sd = metainterp_sd self.jitdriver_sd = jitdriver_sd - def handle_fail(self, metainterp_sd): + def handle_fail(self, metainterp_sd, jitdriver_sd): # Failures of a GUARD_NOT_FORCED are never compiled, but # always just blackholed. First fish for the data saved when # the virtualrefs and virtualizable have been forced by @@ -356,7 +351,8 @@ all_virtuals = self.fetch_data(token) if all_virtuals is None: all_virtuals = [] - resume_in_blackhole(metainterp_sd, self, all_virtuals) + assert jitdriver_sd is self.jitdriver_sd + resume_in_blackhole(metainterp_sd, jitdriver_sd, self, all_virtuals) assert 0, "unreachable" @staticmethod @@ -409,7 +405,7 @@ def _clone_if_mutable(self): res = ResumeGuardForcedDescr(self.metainterp_sd, - self.original_loop_token, + self.original_greenkey, self.jitdriver_sd) self.copy_all_attrbutes_into(res) return res @@ -476,11 +472,8 @@ class ResumeFromInterpDescr(ResumeDescr): - def __init__(self, metainterp, greenkey, redkey): - original_loop_token = make_loop_token(metainterp.cpu, len(redkey), - metainterp.jitdriver_sd, - greenkey) - ResumeDescr.__init__(self, original_loop_token) + def __init__(self, original_greenkey, redkey): + ResumeDescr.__init__(self, original_greenkey) self.redkey = redkey def compile_and_attach(self, metainterp, new_loop): @@ -491,22 +484,22 @@ metainterp_sd = metainterp.staticdata jitdriver_sd = metainterp.jitdriver_sd metainterp.history.inputargs = self.redkey - new_loop_token = self.original_loop_token - original_greenkey = new_loop_token.outermost_greenkey + new_loop_token = make_loop_token(metainterp.cpu, len(self.redkey), + metainterp.jitdriver_sd) new_loop.inputargs = self.redkey new_loop.token = new_loop_token send_loop_to_backend(metainterp_sd, new_loop, "entry bridge") # send the new_loop to warmspot.py, to be called directly the next time jitdriver_sd.warmstate.attach_unoptimized_bridge_from_interp( - original_greenkey, + self.original_greenkey, new_loop_token) # store the new loop in compiled_merge_points_wref too old_loop_tokens = metainterp.get_compiled_merge_points( - original_greenkey) + self.original_greenkey) # it always goes at the end of the list, as it is the most # general loop token old_loop_tokens.append(new_loop_token) - metainterp.set_compiled_merge_points(original_greenkey, + metainterp.set_compiled_merge_points(self.original_greenkey, old_loop_tokens) def reset_counter_from_failure(self): @@ -561,7 +554,7 @@ # ____________________________________________________________ class PropagateExceptionDescr(AbstractFailDescr): - def handle_fail(self, metainterp_sd): + def handle_fail(self, metainterp_sd, jitdriver_sd): cpu = metainterp_sd.cpu exception = cpu.grab_exc_value() raise metainterp_sd.ExitFrameWithExceptionRef(cpu, exception) @@ -575,7 +568,7 @@ """ # 'redboxes' is only used to know the types of red arguments. inputargs = [box.clonebox() for box in redboxes] - loop_token = make_loop_token(cpu, len(inputargs), jitdriver_sd, greenboxes) + loop_token = make_loop_token(cpu, len(inputargs), jitdriver_sd) # 'nb_red_args' might be smaller than len(redboxes), # because it doesn't include the virtualizable boxes. nb_red_args = jitdriver_sd.num_red_args Modified: pypy/branch/jit-free/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/history.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/history.py Mon Nov 15 19:06:44 2010 @@ -181,7 +181,7 @@ index = -1 original_loop_token = None - def handle_fail(self, metainterp_sd): + def handle_fail(self, metainterp_sd, jitdriver_sd): raise NotImplementedError def compile_and_attach(self, metainterp, new_loop): raise NotImplementedError @@ -730,7 +730,6 @@ """ terminating = False # see TerminatingLoopToken in compile.py outermost_jitdriver_sd = None - outermost_greenkey = None # specnodes = ... # and more data specified by the backend when the loop is compiled cpu = None @@ -759,6 +758,9 @@ self.cpu.free_loop_and_bridges(self) debug_stop("jit-free-looptoken") + def __repr__(self): + return '' % (self.number, self.generation) + def repr_of_descr(self): return '' % self.number Modified: pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py Mon Nov 15 19:06:44 2010 @@ -59,6 +59,8 @@ def _kill_old_loops_now(self): debug_start("jit-free-memmgr") oldtotal = len(self.alive_loops) + print self.alive_loops.keys() + debug_print("Current generation:", self.current_generation) debug_print("Loop tokens before:", oldtotal) max_generation = self.current_generation - self.max_age for looptoken in self.alive_loops.keys(): @@ -67,4 +69,5 @@ newtotal = len(self.alive_loops) debug_print("Loop tokens freed: ", oldtotal - newtotal) debug_print("Loop tokens left: ", newtotal) + print self.alive_loops.keys() debug_stop("jit-free-memmgr") Modified: pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py Mon Nov 15 19:06:44 2010 @@ -1048,13 +1048,13 @@ else: moreargs = list(extraargs) metainterp_sd = metainterp.staticdata - original_loop_token = metainterp.resumekey.original_loop_token + original_greenkey = metainterp.resumekey.original_greenkey if opnum == rop.GUARD_NOT_FORCED: - resumedescr = compile.ResumeGuardForcedDescr(metainterp.staticdata, - original_loop_token, - metainterp.jitdriver_sd) + resumedescr = compile.ResumeGuardForcedDescr(metainterp_sd, + original_greenkey, + metainterp.jitdriver_sd) else: - resumedescr = compile.ResumeGuardDescr(original_loop_token) + resumedescr = compile.ResumeGuardDescr(original_greenkey) guard_op = metainterp.history.record(opnum, moreargs, None, descr=resumedescr) virtualizable_boxes = None @@ -1326,6 +1326,11 @@ return jitcode return None + def try_to_free_some_loops(self): + # Increase here the generation recorded by the memory manager. + if self.warmrunnerdesc is not None: # for tests + self.warmrunnerdesc.memory_manager.next_generation() + # ---------------- logging ------------------------ def log(self, msg): @@ -1369,9 +1374,6 @@ self.portal_trace_positions = [] self.free_frames_list = [] self.last_exc_value_box = None - # Increase here the generation recorded by the memory manager. - if self.staticdata.warmrunnerdesc is not None: # for tests - self.staticdata.warmrunnerdesc.memory_manager.next_generation() def perform_call(self, jitcode, boxes, greenkey=None): # causes the metainterp to enter the given subfunction @@ -1622,6 +1624,7 @@ # is also available as 'self.jitdriver_sd', because we need to # specialize this function and a few other ones for the '*args'. debug_start('jit-tracing') + self.staticdata.try_to_free_some_loops() self.staticdata._setup_once() self.staticdata.profiler.start_tracing() assert jitdriver_sd is self.jitdriver_sd @@ -1639,7 +1642,7 @@ num_green_args = self.jitdriver_sd.num_green_args original_greenkey = original_boxes[:num_green_args] redkey = original_boxes[num_green_args:] - self.resumekey = compile.ResumeFromInterpDescr(self, original_greenkey, + self.resumekey = compile.ResumeFromInterpDescr(original_greenkey, redkey) self.seen_loop_header_for_jdindex = -1 try: @@ -1652,6 +1655,7 @@ def handle_guard_failure(self, key): debug_start('jit-tracing') + self.staticdata.try_to_free_some_loops() self.staticdata.profiler.start_tracing() assert isinstance(key, compile.ResumeGuardDescr) self.initialize_state_from_guard_failure(key) @@ -1662,7 +1666,7 @@ debug_stop('jit-tracing') def _handle_guard_failure(self, key): - original_greenkey = key.original_loop_token.outermost_greenkey + original_greenkey = key.original_greenkey # notice that here we just put the greenkey # use -1 to mark that we will have to give up # because we cannot reconstruct the beginning of the proper loop Modified: pypy/branch/jit-free/pypy/jit/metainterp/resume.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/resume.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/resume.py Mon Nov 15 19:06:44 2010 @@ -915,11 +915,10 @@ # ---------- when resuming for blackholing, get direct values ---------- -def blackhole_from_resumedata(blackholeinterpbuilder, storage, +def blackhole_from_resumedata(blackholeinterpbuilder, jitdriver_sd, storage, all_virtuals=None): resumereader = ResumeDataDirectReader(blackholeinterpbuilder.metainterp_sd, storage, all_virtuals) - jitdriver_sd = storage.get_jitdriver_sd() vinfo = jitdriver_sd.virtualizable_info ginfo = jitdriver_sd.greenfield_info vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Mon Nov 15 19:06:44 2010 @@ -1,7 +1,6 @@ from pypy.jit.metainterp.memmgr import MemoryManager - -##missing: -## contains_jumps_to needs to be filled +from pypy.jit.metainterp.test.test_basic import LLJitMixin +from pypy.rlib.jit import JitDriver class FakeLoopToken: @@ -57,3 +56,23 @@ assert tokens[i] not in memmgr.alive_loops else: assert tokens[i] in memmgr.alive_loops + + +class TestIntegration(LLJitMixin): + + def test_loop_kept_alive(self): + myjitdriver = JitDriver(greens=[], reds=['n']) + def g(): + n = 10 + while n > 0: + myjitdriver.can_enter_jit(n=n) + myjitdriver.jit_merge_point(n=n) + n = n - 1 + return 21 + def f(): + for i in range(6): + g() + return 42 + + res = self.meta_interp(f, [], loop_longevity=1) + assert res == 42 Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py Mon Nov 15 19:06:44 2010 @@ -67,7 +67,8 @@ def jittify_and_run(interp, graph, args, repeat=1, backendopt=False, trace_limit=sys.maxint, - debug_level=DEBUG_STEPS, inline=False, **kwds): + debug_level=DEBUG_STEPS, inline=False, + loop_longevity=0, **kwds): from pypy.config.config import ConfigError translator = interp.typer.annotator.translator try: @@ -85,6 +86,7 @@ jd.warmstate.set_param_trace_limit(trace_limit) jd.warmstate.set_param_inlining(inline) jd.warmstate.set_param_debug(debug_level) + jd.warmstate.set_param_loop_longevity(loop_longevity) warmrunnerdesc.finish() res = interp.eval_graph(graph, args) if not kwds.get('translate_support_code', False): @@ -713,7 +715,7 @@ vinfo.VTYPEPTR, virtualizableref) vinfo.reset_vable_token(virtualizable) try: - loop_token = fail_descr.handle_fail(self.metainterp_sd) + loop_token = fail_descr.handle_fail(self.metainterp_sd, jd) except JitException, e: return handle_jitexception(e) fail_descr = self.execute_token(loop_token) @@ -820,4 +822,5 @@ debug_stop("jit-running") self.metainterp_sd.profiler.end_running() self.memory_manager.keep_loop_alive(loop_token) + print loop_token return fail_descr Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Mon Nov 15 19:06:44 2010 @@ -340,7 +340,8 @@ fail_descr = warmrunnerdesc.execute_token(loop_token) if vinfo is not None: vinfo.reset_vable_token(virtualizable) - loop_token = fail_descr.handle_fail(metainterp_sd) + loop_token = fail_descr.handle_fail(metainterp_sd, + jitdriver_sd) maybe_compile_and_run._dont_inline_ = True self.maybe_compile_and_run = maybe_compile_and_run From arigo at codespeak.net Mon Nov 15 19:37:31 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 15 Nov 2010 19:37:31 +0100 (CET) Subject: [pypy-svn] r79122 - in pypy/branch/jit-free/pypy/jit/metainterp: . test Message-ID: <20101115183731.4E782282B90@codespeak.net> Author: arigo Date: Mon Nov 15 19:37:29 2010 New Revision: 79122 Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py pypy/branch/jit-free/pypy/jit/metainterp/history.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Log: (antocuni, arigo) Argh. Lots of efforts to finally manage to have this test failing :-/ Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/compile.py Mon Nov 15 19:37:29 2010 @@ -78,6 +78,7 @@ return old_loop_token send_loop_to_backend(metainterp_sd, loop, "loop") insert_loop_token(old_loop_tokens, loop_token) + loop.token = None return loop_token def insert_loop_token(old_loop_tokens, loop_token): @@ -316,7 +317,7 @@ # to the corrsponding guard_op and compile from there inputargs = metainterp.history.inputargs if not we_are_translated(): - self._debug_suboperations = new_loop.operations + pass #self._debug_suboperations = new_loop.operations send_bridge_to_backend(metainterp.staticdata, self, inputargs, new_loop.operations) @@ -501,6 +502,7 @@ old_loop_tokens.append(new_loop_token) metainterp.set_compiled_merge_points(self.original_greenkey, old_loop_tokens) + new_loop.token = None def reset_counter_from_failure(self): pass Modified: pypy/branch/jit-free/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/history.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/history.py Mon Nov 15 19:37:29 2010 @@ -747,6 +747,9 @@ self.generation = r_longlong(0) def __del__(self): + for i in range(160): + print '#', + print if self.cpu is None: return if self.generation > r_longlong(0): Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Mon Nov 15 19:37:29 2010 @@ -70,9 +70,12 @@ n = n - 1 return 21 def f(): - for i in range(6): + for i in range(15): g() return 42 res = self.meta_interp(f, [], loop_longevity=1) assert res == 42 + + # we should see only the loop and the entry bridge + self.check_tree_loop_count(2) Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Mon Nov 15 19:37:29 2010 @@ -330,10 +330,13 @@ assert cell.counter == -1 if not confirm_enter_jit(*args): return + loop_token = cell.get_entry_loop_token() + if loop_token is None: # it was a weakref that has been freed + cell.counter = 0 + return # machine code was already compiled for these greenargs # get the assembler and fill in the boxes set_future_values(*args[num_green_args:]) - loop_token = cell.get_entry_loop_token() # ---------- execute assembler ---------- while True: # until interrupted by an exception From fijal at codespeak.net Mon Nov 15 20:04:03 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 15 Nov 2010 20:04:03 +0100 (CET) Subject: [pypy-svn] r79124 - in pypy/release/1.4.x: . pypy pypy/jit/metainterp/optimizeopt pypy/module/array/benchmark pypy/module/array/test pypy/module/posix pypy/rlib pypy/rlib/test Message-ID: <20101115190403.15876282B90@codespeak.net> Author: fijal Date: Mon Nov 15 20:04:02 2010 New Revision: 79124 Modified: pypy/release/1.4.x/ (props changed) pypy/release/1.4.x/pypy/ (props changed) pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/Makefile (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimg.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/loop.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sum.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.py (props changed) pypy/release/1.4.x/pypy/module/array/test/test_array_old.py (props changed) pypy/release/1.4.x/pypy/module/posix/interp_posix.py pypy/release/1.4.x/pypy/rlib/rerased.py (props changed) pypy/release/1.4.x/pypy/rlib/test/test_rerased.py (props changed) Log: merge 78986 from trunk Modified: pypy/release/1.4.x/pypy/module/posix/interp_posix.py ============================================================================== --- pypy/release/1.4.x/pypy/module/posix/interp_posix.py (original) +++ pypy/release/1.4.x/pypy/module/posix/interp_posix.py Mon Nov 15 20:04:02 2010 @@ -454,7 +454,8 @@ self.w_environ = space.newdict() if _WIN: self.cryptProviderPtr = lltype.malloc( - rffi.CArray(HCRYPTPROV), 1, zero=True, flavor='raw') + rffi.CArray(HCRYPTPROV), 1, zero=True, + flavor='raw', immortal=True) def startup(self, space): _convertenviron(space, self.w_environ) def _freeze_(self): From fijal at codespeak.net Mon Nov 15 20:06:43 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 15 Nov 2010 20:06:43 +0100 (CET) Subject: [pypy-svn] r79125 - in pypy/release/1.4.x: . lib_pypy/ctypes_config_cache/test pypy pypy/jit/metainterp/optimizeopt pypy/module/array/benchmark pypy/module/array/test pypy/rlib pypy/rlib/test Message-ID: <20101115190643.7A8D1282B90@codespeak.net> Author: fijal Date: Mon Nov 15 20:06:41 2010 New Revision: 79125 Modified: pypy/release/1.4.x/ (props changed) pypy/release/1.4.x/lib_pypy/ctypes_config_cache/test/test_cache.py pypy/release/1.4.x/pypy/ (props changed) pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/Makefile (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimg.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/loop.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sum.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.py (props changed) pypy/release/1.4.x/pypy/module/array/test/test_array_old.py (props changed) pypy/release/1.4.x/pypy/rlib/rerased.py (props changed) pypy/release/1.4.x/pypy/rlib/test/test_rerased.py (props changed) Log: merge 78988 from trunk Modified: pypy/release/1.4.x/lib_pypy/ctypes_config_cache/test/test_cache.py ============================================================================== --- pypy/release/1.4.x/lib_pypy/ctypes_config_cache/test/test_cache.py (original) +++ pypy/release/1.4.x/lib_pypy/ctypes_config_cache/test/test_cache.py Mon Nov 15 20:06:41 2010 @@ -7,9 +7,8 @@ def run(filename, outputname): filepath = dirpath.join(filename) - tmpdir2 = udir.ensure('testcache-' + filename, dir=True) - tmpdir = tmpdir2.ensure('ctypes_config_cache', dir=True) - tmpdir.join('__init__.py').write('\n') + tmpdir = udir.ensure('testcache-' + os.path.splitext(filename)[0], + dir=True) tmpdir.join('dumpcache.py').write(dirpath.join('dumpcache.py').read()) path = sys.path[:] try: @@ -23,14 +22,12 @@ assert outputpath.check(exists=1) modname = os.path.splitext(outputname)[0] try: - sys.path.insert(0, str(tmpdir2)) + sys.path.insert(0, str(tmpdir)) d = {} - exec "from ctypes_config_cache import %s" % modname in d - mod = d[modname] + execfile(str(outputpath), d) finally: sys.path[:] = path - sys.modules.pop('ctypes_config_cache', None) - return mod.__dict__ + return d def test_syslog(): From fijal at codespeak.net Mon Nov 15 20:08:33 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 15 Nov 2010 20:08:33 +0100 (CET) Subject: [pypy-svn] r79126 - in pypy/release/1.4.x: . pypy pypy/jit/codewriter pypy/jit/codewriter/test pypy/jit/metainterp pypy/jit/metainterp/optimizeopt pypy/jit/metainterp/test pypy/module/array/benchmark pypy/module/array/test pypy/rlib pypy/rlib/test Message-ID: <20101115190833.C05E6282B90@codespeak.net> Author: fijal Date: Mon Nov 15 20:08:31 2010 New Revision: 79126 Modified: pypy/release/1.4.x/ (props changed) pypy/release/1.4.x/pypy/ (props changed) pypy/release/1.4.x/pypy/jit/codewriter/assembler.py pypy/release/1.4.x/pypy/jit/codewriter/call.py pypy/release/1.4.x/pypy/jit/codewriter/codewriter.py pypy/release/1.4.x/pypy/jit/codewriter/effectinfo.py pypy/release/1.4.x/pypy/jit/codewriter/jtransform.py pypy/release/1.4.x/pypy/jit/codewriter/test/test_jtransform.py pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py (props changed) pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/string.py pypy/release/1.4.x/pypy/jit/metainterp/pyjitpl.py pypy/release/1.4.x/pypy/jit/metainterp/resume.py pypy/release/1.4.x/pypy/jit/metainterp/test/test_optimizeopt.py pypy/release/1.4.x/pypy/jit/metainterp/test/test_resume.py pypy/release/1.4.x/pypy/jit/metainterp/test/test_virtualref.py pypy/release/1.4.x/pypy/module/array/benchmark/Makefile (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimg.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/loop.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sum.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.py (props changed) pypy/release/1.4.x/pypy/module/array/test/test_array_old.py (props changed) pypy/release/1.4.x/pypy/rlib/rerased.py (props changed) pypy/release/1.4.x/pypy/rlib/test/test_rerased.py (props changed) Log: merge 79004 from trunk Modified: pypy/release/1.4.x/pypy/jit/codewriter/assembler.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/codewriter/assembler.py (original) +++ pypy/release/1.4.x/pypy/jit/codewriter/assembler.py Mon Nov 15 20:08:31 2010 @@ -233,10 +233,9 @@ addr = llmemory.cast_ptr_to_adr(value) self.list_of_addr2name.append((addr, name)) - def finished(self): + def finished(self, callinfocollection): # Helper called at the end of assembling. Registers the extra # functions shown in _callinfo_for_oopspec. - from pypy.jit.codewriter.effectinfo import _callinfo_for_oopspec - for _, func in _callinfo_for_oopspec.values(): + for func in callinfocollection.all_function_addresses_as_int(): func = heaptracker.int2adr(func) self.see_raw_object(func.ptr) Modified: pypy/release/1.4.x/pypy/jit/codewriter/call.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/codewriter/call.py (original) +++ pypy/release/1.4.x/pypy/jit/codewriter/call.py Mon Nov 15 20:08:31 2010 @@ -7,7 +7,7 @@ from pypy.jit.codewriter.jitcode import JitCode from pypy.jit.codewriter.effectinfo import VirtualizableAnalyzer from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze -from pypy.jit.codewriter.effectinfo import EffectInfo +from pypy.jit.codewriter.effectinfo import EffectInfo, CallInfoCollection from pypy.translator.simplify import get_funcobj, get_functype from pypy.rpython.lltypesystem import lltype, llmemory from pypy.translator.backendopt.canraise import RaiseAnalyzer @@ -23,6 +23,7 @@ self.jitdrivers_sd = jitdrivers_sd self.jitcodes = {} # map {graph: jitcode} self.unfinished_graphs = [] # list of graphs with pending jitcodes + self.callinfocollection = CallInfoCollection() if hasattr(cpu, 'rtyper'): # for tests self.rtyper = cpu.rtyper translator = self.rtyper.annotator.translator Modified: pypy/release/1.4.x/pypy/jit/codewriter/codewriter.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/codewriter/codewriter.py (original) +++ pypy/release/1.4.x/pypy/jit/codewriter/codewriter.py Mon Nov 15 20:08:31 2010 @@ -73,7 +73,7 @@ count += 1 if not count % 500: log.info("Produced %d jitcodes" % count) - self.assembler.finished() + self.assembler.finished(self.callcontrol.callinfocollection) heaptracker.finish_registering(self.cpu) log.info("there are %d JitCode instances." % count) Modified: pypy/release/1.4.x/pypy/jit/codewriter/effectinfo.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/codewriter/effectinfo.py (original) +++ pypy/release/1.4.x/pypy/jit/codewriter/effectinfo.py Mon Nov 15 20:08:31 2010 @@ -144,30 +144,44 @@ # ____________________________________________________________ -_callinfo_for_oopspec = {} # {oopspecindex: (calldescr, func_as_int)} - -def callinfo_for_oopspec(oopspecindex): - """A function that returns the calldescr and the function - address (as an int) of one of the OS_XYZ functions defined above. - Don't use this if there might be several implementations of the same - OS_XYZ specialized by type, e.g. OS_ARRAYCOPY.""" - try: - return _callinfo_for_oopspec[oopspecindex] - except KeyError: - return (None, 0) - - -def _funcptr_for_oopspec_memo(oopspecindex): - from pypy.jit.codewriter import heaptracker - _, func_as_int = callinfo_for_oopspec(oopspecindex) - funcadr = heaptracker.int2adr(func_as_int) - return funcadr.ptr -_funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo' - -def funcptr_for_oopspec(oopspecindex): - """A memo function that returns a pointer to the function described - by OS_XYZ (as a real low-level function pointer).""" - funcptr = _funcptr_for_oopspec_memo(oopspecindex) - assert funcptr - return funcptr -funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(0)' +class CallInfoCollection(object): + def __init__(self): + # {oopspecindex: (calldescr, func_as_int)} + self._callinfo_for_oopspec = {} + + def _freeze_(self): + return True + + def add(self, oopspecindex, calldescr, func_as_int): + self._callinfo_for_oopspec[oopspecindex] = calldescr, func_as_int + + def has_oopspec(self, oopspecindex): + return oopspecindex in self._callinfo_for_oopspec + + def all_function_addresses_as_int(self): + return [func for (_, func) in self._callinfo_for_oopspec.values()] + + def callinfo_for_oopspec(self, oopspecindex): + """A function that returns the calldescr and the function + address (as an int) of one of the OS_XYZ functions defined above. + Don't use this if there might be several implementations of the same + OS_XYZ specialized by type, e.g. OS_ARRAYCOPY.""" + try: + return self._callinfo_for_oopspec[oopspecindex] + except KeyError: + return (None, 0) + + def _funcptr_for_oopspec_memo(self, oopspecindex): + from pypy.jit.codewriter import heaptracker + _, func_as_int = self.callinfo_for_oopspec(oopspecindex) + funcadr = heaptracker.int2adr(func_as_int) + return funcadr.ptr + _funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo' + + def funcptr_for_oopspec(self, oopspecindex): + """A memo function that returns a pointer to the function described + by OS_XYZ (as a real low-level function pointer).""" + funcptr = self._funcptr_for_oopspec_memo(oopspecindex) + assert funcptr + return funcptr + funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(1)' Modified: pypy/release/1.4.x/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/codewriter/jtransform.py (original) +++ pypy/release/1.4.x/pypy/jit/codewriter/jtransform.py Mon Nov 15 20:08:31 2010 @@ -6,7 +6,7 @@ from pypy.objspace.flow.model import Block, Link, c_last_exception from pypy.jit.codewriter.flatten import ListOfKind, IndirectCallTargets from pypy.jit.codewriter import support, heaptracker -from pypy.jit.codewriter.effectinfo import EffectInfo, _callinfo_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.jit.codewriter.policy import log from pypy.jit.metainterp.typesystem import deref, arrayItem from pypy.rlib import objectmodel @@ -1084,7 +1084,8 @@ else: func = heaptracker.adr2int( llmemory.cast_ptr_to_adr(op.args[0].value)) - _callinfo_for_oopspec[oopspecindex] = calldescr, func + self.callcontrol.callinfocollection.add(oopspecindex, + calldescr, func) op1 = self.rewrite_call(op, 'residual_call', [op.args[0], calldescr], args=args) @@ -1095,7 +1096,7 @@ def _register_extra_helper(self, oopspecindex, oopspec_name, argtypes, resulttype): # a bit hackish - if oopspecindex in _callinfo_for_oopspec: + if self.callcontrol.callinfocollection.has_oopspec(oopspecindex): return c_func, TP = support.builtin_func_for_spec(self.cpu.rtyper, oopspec_name, argtypes, @@ -1109,7 +1110,7 @@ else: func = heaptracker.adr2int( llmemory.cast_ptr_to_adr(c_func.value)) - _callinfo_for_oopspec[oopspecindex] = calldescr, func + self.callcontrol.callinfocollection.add(oopspecindex, calldescr, func) def _handle_stroruni_call(self, op, oopspec_name, args): SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE) Modified: pypy/release/1.4.x/pypy/jit/codewriter/test/test_jtransform.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/codewriter/test/test_jtransform.py (original) +++ pypy/release/1.4.x/pypy/jit/codewriter/test/test_jtransform.py Mon Nov 15 20:08:31 2010 @@ -74,7 +74,20 @@ def calldescr_canraise(self, calldescr): return False +class FakeCallInfoCollection: + def __init__(self): + self.seen = [] + def add(self, oopspecindex, calldescr, func): + self.seen.append((oopspecindex, calldescr, func)) + def has_oopspec(self, oopspecindex): + for i, c, f in self.seen: + if i == oopspecindex: + return True + return False + class FakeBuiltinCallControl: + def __init__(self): + self.callinfocollection = FakeCallInfoCollection() def guess_call_kind(self, op): return 'builtin' def getcalldescr(self, op, oopspecindex=None): @@ -810,7 +823,8 @@ v2 = varoftype(PSTR) v3 = varoftype(PSTR) op = SpaceOperation('direct_call', [const(func), v1, v2], v3) - tr = Transformer(FakeCPU(), FakeBuiltinCallControl()) + cc = FakeBuiltinCallControl() + tr = Transformer(FakeCPU(), cc) op1 = tr.rewrite_operation(op) assert op1.opname == 'residual_call_r_r' assert op1.args[0].value == func @@ -819,9 +833,10 @@ assert op1.result == v3 # # check the callinfo_for_oopspec - got = effectinfo.callinfo_for_oopspec(effectinfo.EffectInfo.OS_UNI_CONCAT) - assert got[0] == op1.args[1] # the calldescr - assert heaptracker.int2adr(got[1]) == llmemory.cast_ptr_to_adr(func) + got = cc.callinfocollection.seen[0] + assert got[0] == effectinfo.EffectInfo.OS_UNI_CONCAT + assert got[1] == op1.args[1] # the calldescr + assert heaptracker.int2adr(got[2]) == llmemory.cast_ptr_to_adr(func) def test_str_slice(): # test that the oopspec is present and correctly transformed @@ -893,7 +908,8 @@ v2 = varoftype(PUNICODE) v3 = varoftype(lltype.Bool) op = SpaceOperation('direct_call', [const(func), v1, v2], v3) - tr = Transformer(FakeCPU(), FakeBuiltinCallControl()) + cc = FakeBuiltinCallControl() + tr = Transformer(FakeCPU(), cc) op1 = tr.rewrite_operation(op) assert op1.opname == 'residual_call_r_i' assert op1.args[0].value == func @@ -901,9 +917,9 @@ assert op1.args[2] == ListOfKind('ref', [v1, v2]) assert op1.result == v3 # test that the OS_UNIEQ_* functions are registered - cifo = effectinfo._callinfo_for_oopspec - assert effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL in cifo - assert effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR in cifo + cic = cc.callinfocollection + assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL) + assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR) def test_list_ll_arraycopy(): from pypy.rlib.rgc import ll_arraycopy Modified: pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/string.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/string.py (original) +++ pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/string.py Mon Nov 15 20:08:31 2010 @@ -9,7 +9,7 @@ from pypy.jit.metainterp.optimizeopt.optimizer import CONST_0, CONST_1 from pypy.jit.metainterp.optimizeopt.optimizer import llhelper from pypy.jit.metainterp.optimizeutil import _findall -from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.jit.codewriter import heaptracker from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.objectmodel import specialize, we_are_translated @@ -593,7 +593,8 @@ def generate_modified_call(self, oopspecindex, args, result, mode): oopspecindex += mode.OS_offset - calldescr, func = callinfo_for_oopspec(oopspecindex) + cic = self.optimizer.metainterp_sd.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(oopspecindex) op = ResOperation(rop.CALL, [ConstInt(func)] + args, result, descr=calldescr) self.optimizer.newoperations.append(op) Modified: pypy/release/1.4.x/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/release/1.4.x/pypy/jit/metainterp/pyjitpl.py Mon Nov 15 20:08:31 2010 @@ -1261,6 +1261,7 @@ # self.jitdrivers_sd = codewriter.callcontrol.jitdrivers_sd self.virtualref_info = codewriter.callcontrol.virtualref_info + self.callinfocollection = codewriter.callcontrol.callinfocollection self.setup_jitdrivers_sd(optimizer) # # store this information for fastpath of call_assembler Modified: pypy/release/1.4.x/pypy/jit/metainterp/resume.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/metainterp/resume.py (original) +++ pypy/release/1.4.x/pypy/jit/metainterp/resume.py Mon Nov 15 20:08:31 2010 @@ -4,8 +4,7 @@ from pypy.jit.metainterp.history import INT, REF, FLOAT, HOLE from pypy.jit.metainterp.resoperation import rop from pypy.jit.metainterp import jitprof -from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec -from pypy.jit.codewriter.effectinfo import funcptr_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.rpython.lltypesystem import lltype, llmemory, rffi, rstr from pypy.rlib import rarithmetic from pypy.rlib.objectmodel import we_are_translated, specialize @@ -774,14 +773,16 @@ strbox, ConstInt(index), charbox) def concat_strings(self, str1num, str2num): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT) str1box = self.decode_box(str1num, REF) str2box = self.decode_box(str2num, REF) return self.metainterp.execute_and_record_varargs( rop.CALL, [ConstInt(func), str1box, str2box], calldescr) def slice_string(self, strnum, startnum, lengthnum): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_SLICE) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_SLICE) strbox = self.decode_box(strnum, REF) startbox = self.decode_box(startnum, INT) lengthbox = self.decode_box(lengthnum, INT) @@ -800,14 +801,16 @@ strbox, ConstInt(index), charbox) def concat_unicodes(self, str1num, str2num): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT) str1box = self.decode_box(str1num, REF) str2box = self.decode_box(str2num, REF) return self.metainterp.execute_and_record_varargs( rop.CALL, [ConstInt(func), str1box, str2box], calldescr) def slice_unicode(self, strnum, startnum, lengthnum): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE) strbox = self.decode_box(strnum, REF) startbox = self.decode_box(startnum, INT) lengthbox = self.decode_box(lengthnum, INT) @@ -903,8 +906,8 @@ def blackhole_from_resumedata(blackholeinterpbuilder, jitdriver_sd, storage, all_virtuals=None): - resumereader = ResumeDataDirectReader(blackholeinterpbuilder.cpu, storage, - all_virtuals) + resumereader = ResumeDataDirectReader(blackholeinterpbuilder.metainterp_sd, + storage, all_virtuals) vinfo = jitdriver_sd.virtualizable_info ginfo = jitdriver_sd.greenfield_info vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info @@ -939,7 +942,7 @@ return firstbh def force_from_resumedata(metainterp_sd, storage, vinfo, ginfo): - resumereader = ResumeDataDirectReader(metainterp_sd.cpu, storage) + resumereader = ResumeDataDirectReader(metainterp_sd, storage) resumereader.handling_async_forcing() vrefinfo = metainterp_sd.virtualref_info resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo) @@ -953,8 +956,9 @@ # 1: in handle_async_forcing # 2: resuming from the GUARD_NOT_FORCED - def __init__(self, cpu, storage, all_virtuals=None): - self._init(cpu, storage) + def __init__(self, metainterp_sd, storage, all_virtuals=None): + self._init(metainterp_sd.cpu, storage) + self.callinfocollection = metainterp_sd.callinfocollection if all_virtuals is None: # common case self._prepare(storage) else: @@ -1047,7 +1051,8 @@ str2 = self.decode_ref(str2num) str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str1) str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str2) - funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT) result = funcptr(str1, str2) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1056,7 +1061,8 @@ start = self.decode_int(startnum) length = self.decode_int(lengthnum) str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str) - funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_SLICE) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_SLICE) result = funcptr(str, start, start + length) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1072,7 +1078,8 @@ str2 = self.decode_ref(str2num) str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str1) str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str2) - funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT) result = funcptr(str1, str2) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1081,7 +1088,8 @@ start = self.decode_int(startnum) length = self.decode_int(lengthnum) str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str) - funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE) result = funcptr(str, start, start + length) return lltype.cast_opaque_ptr(llmemory.GCREF, result) Modified: pypy/release/1.4.x/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/release/1.4.x/pypy/jit/metainterp/test/test_optimizeopt.py Mon Nov 15 20:08:31 2010 @@ -264,6 +264,8 @@ metainterp_sd = FakeMetaInterpStaticData(self.cpu) if hasattr(self, 'vrefinfo'): metainterp_sd.virtualref_info = self.vrefinfo + if hasattr(self, 'callinfocollection'): + metainterp_sd.callinfocollection = self.callinfocollection optimize_loop_1(metainterp_sd, loop) # expected = self.parse(optops) @@ -4180,22 +4182,20 @@ # ---------- def optimize_strunicode_loop_extradescrs(self, ops, spectext, optops): from pypy.jit.metainterp.optimizeopt import string - def my_callinfo_for_oopspec(oopspecindex): - calldescrtype = type(LLtypeMixin.strequaldescr) - for value in LLtypeMixin.__dict__.values(): - if isinstance(value, calldescrtype): - if (value.get_extra_info() and - value.get_extra_info().oopspecindex == oopspecindex): - # returns 0 for 'func' in this test - return value, 0 - raise AssertionError("not found: oopspecindex=%d" % oopspecindex) + class FakeCallInfoCollection: + def callinfo_for_oopspec(self, oopspecindex): + calldescrtype = type(LLtypeMixin.strequaldescr) + for value in LLtypeMixin.__dict__.values(): + if isinstance(value, calldescrtype): + extra = value.get_extra_info() + if extra and extra.oopspecindex == oopspecindex: + # returns 0 for 'func' in this test + return value, 0 + raise AssertionError("not found: oopspecindex=%d" % + oopspecindex) # - saved = string.callinfo_for_oopspec - try: - string.callinfo_for_oopspec = my_callinfo_for_oopspec - self.optimize_strunicode_loop(ops, spectext, optops) - finally: - string.callinfo_for_oopspec = saved + self.callinfocollection = FakeCallInfoCollection() + self.optimize_strunicode_loop(ops, spectext, optops) def test_str_equal_noop1(self): ops = """ Modified: pypy/release/1.4.x/pypy/jit/metainterp/test/test_resume.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/metainterp/test/test_resume.py (original) +++ pypy/release/1.4.x/pypy/jit/metainterp/test/test_resume.py Mon Nov 15 20:08:31 2010 @@ -51,6 +51,7 @@ class MyMetaInterp: _already_allocated_resume_virtuals = None + callinfocollection = None def __init__(self, cpu=None): if cpu is None: @@ -156,12 +157,12 @@ storage.rd_numb = numb # cpu = MyCPU([42, gcref1, -66]) - reader = ResumeDataDirectReader(cpu, storage) + metainterp = MyMetaInterp(cpu) + reader = ResumeDataDirectReader(metainterp, storage) _next_section(reader, 42, 111, gcrefnull, 42, gcref1) _next_section(reader, 222, 333) _next_section(reader, 42, gcref1, -66) # - metainterp = MyMetaInterp(cpu) reader = ResumeDataBoxReader(storage, metainterp) bi, br, bf = [None]*3, [None]*2, [None]*0 info = MyBlackholeInterp([lltype.Signed, lltype.Signed, @@ -193,7 +194,7 @@ storage.rd_numb = numb # cpu = MyCPU([]) - reader = ResumeDataDirectReader(cpu, storage) + reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage) _next_section(reader, 100) @@ -211,7 +212,7 @@ class FakeMetainterp(object): _already_allocated_resume_virtuals = None cpu = None - reader = ResumeDataDirectReader(None, FakeStorage()) + reader = ResumeDataDirectReader(MyMetaInterp(None), FakeStorage()) assert reader.force_all_virtuals() == ["allocated", reader.virtual_default] # ____________________________________________________________ @@ -925,7 +926,7 @@ liveboxes = modifier.finish({}) assert storage.rd_snapshot is None cpu = MyCPU([]) - reader = ResumeDataDirectReader(cpu, storage) + reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage) _next_section(reader, sys.maxint, 2**16, -65) _next_section(reader, 2, 3) _next_section(reader, sys.maxint, 1, sys.maxint, 2**16) Modified: pypy/release/1.4.x/pypy/jit/metainterp/test/test_virtualref.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/metainterp/test/test_virtualref.py (original) +++ pypy/release/1.4.x/pypy/jit/metainterp/test/test_virtualref.py Mon Nov 15 20:08:31 2010 @@ -88,7 +88,11 @@ cpu.get_latest_value_int = lambda i:guard_op.getfailargs()[i].getint() cpu.get_latest_value_ref = lambda i:guard_op.getfailargs()[i].getref_base() cpu.clear_latest_values = lambda count: None - resumereader = ResumeDataDirectReader(cpu, guard_op.getdescr()) + class FakeMetaInterpSd: + callinfocollection = None + FakeMetaInterpSd.cpu = cpu + resumereader = ResumeDataDirectReader(FakeMetaInterpSd(), + guard_op.getdescr()) vrefinfo = self.metainterp.staticdata.virtualref_info lst = [] vrefinfo.continue_tracing = lambda vref, virtual: \ From fijal at codespeak.net Mon Nov 15 20:10:31 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 15 Nov 2010 20:10:31 +0100 (CET) Subject: [pypy-svn] r79127 - in pypy/release/1.4.x: . pypy pypy/jit/metainterp/optimizeopt pypy/module/array/benchmark pypy/module/array/test pypy/rlib pypy/rlib/test pypy/translator/goal/test2 Message-ID: <20101115191031.1DC815080F@codespeak.net> Author: fijal Date: Mon Nov 15 20:10:30 2010 New Revision: 79127 Modified: pypy/release/1.4.x/ (props changed) pypy/release/1.4.x/pypy/ (props changed) pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/Makefile (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimg.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/loop.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sum.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.py (props changed) pypy/release/1.4.x/pypy/module/array/test/test_array_old.py (props changed) pypy/release/1.4.x/pypy/rlib/rerased.py (props changed) pypy/release/1.4.x/pypy/rlib/test/test_rerased.py (props changed) pypy/release/1.4.x/pypy/translator/goal/test2/test_app_main.py Log: merge 79008 from trunk Modified: pypy/release/1.4.x/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/release/1.4.x/pypy/translator/goal/test2/test_app_main.py Mon Nov 15 20:10:30 2010 @@ -463,8 +463,10 @@ yield finally: old_cwd.chdir() - os.putenv('PYTHONPATH', old_pythonpath) - + # Can't call putenv with a None argument. + if old_pythonpath is not None: + os.putenv('PYTHONPATH', old_pythonpath) + tmpdir.join('site.py').write('print "SHOULD NOT RUN"') runme_py = tmpdir.join('runme.py') runme_py.write('print "some text"') From fijal at codespeak.net Mon Nov 15 20:11:32 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 15 Nov 2010 20:11:32 +0100 (CET) Subject: [pypy-svn] r79128 - in pypy/release/1.4.x: . pypy pypy/jit/metainterp/optimizeopt pypy/module/array/benchmark pypy/module/array/test pypy/rlib pypy/rlib/test pypy/translator/goal pypy/translator/goal/test2 Message-ID: <20101115191132.8B52F282B9C@codespeak.net> Author: fijal Date: Mon Nov 15 20:11:31 2010 New Revision: 79128 Modified: pypy/release/1.4.x/ (props changed) pypy/release/1.4.x/pypy/ (props changed) pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/Makefile (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimg.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/loop.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sum.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.py (props changed) pypy/release/1.4.x/pypy/module/array/test/test_array_old.py (props changed) pypy/release/1.4.x/pypy/rlib/rerased.py (props changed) pypy/release/1.4.x/pypy/rlib/test/test_rerased.py (props changed) pypy/release/1.4.x/pypy/translator/goal/app_main.py pypy/release/1.4.x/pypy/translator/goal/test2/test_app_main.py Log: merge 79011 from trunk Modified: pypy/release/1.4.x/pypy/translator/goal/app_main.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/goal/app_main.py (original) +++ pypy/release/1.4.x/pypy/translator/goal/app_main.py Mon Nov 15 20:11:31 2010 @@ -326,10 +326,6 @@ except: print >> sys.stderr, "'import site' failed" - # update sys.path *after* loading site.py, in case there is a - # "site.py" file in the script's directory. - sys.path.insert(0, '') - if warnoptions: sys.warnoptions.append(warnoptions) from warnings import _processoptions @@ -378,6 +374,13 @@ elif run_stdin: # handle the case where no command/filename/module is specified # on the command-line. + + # update sys.path *after* loading site.py, in case there is a + # "site.py" file in the script's directory. Only run this if we're + # executing the interactive prompt, if we're running a script we + # put it's directory on sys.path + sys.path.insert(0, '') + if go_interactive or sys.stdin.isatty(): # If stdin is a tty or if "-i" is specified, we print # a banner and run $PYTHONSTARTUP. Modified: pypy/release/1.4.x/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/release/1.4.x/pypy/translator/goal/test2/test_app_main.py Mon Nov 15 20:11:31 2010 @@ -95,6 +95,11 @@ child.expect('>>> ') child.sendline('__name__') child.expect("'__main__'") + child.expect('>>> ') + child.sendline('import sys') + child.expect('>>> ') + child.sendline("'' in sys.path") + child.expect("True") def test_run_script(self): child = self.spawn([demo_script]) @@ -487,9 +492,9 @@ with chdir_and_unset_pythonpath(tmpdir): data = self.run(cmdline2, python_flags='-S') - assert data.startswith("some new text\n") assert repr(str(tmpdir.join('otherpath'))) in data + assert "''" not in data class AppTestAppMain: From fijal at codespeak.net Mon Nov 15 20:12:58 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 15 Nov 2010 20:12:58 +0100 (CET) Subject: [pypy-svn] r79129 - in pypy/release/1.4.x: . pypy pypy/jit/metainterp/optimizeopt pypy/module/array/benchmark pypy/module/array/test pypy/rlib pypy/rlib/test pypy/rpython/memory/gc pypy/translator/c/test Message-ID: <20101115191258.928B250810@codespeak.net> Author: fijal Date: Mon Nov 15 20:12:56 2010 New Revision: 79129 Modified: pypy/release/1.4.x/ (props changed) pypy/release/1.4.x/pypy/ (props changed) pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/Makefile (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimg.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/loop.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sum.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.py (props changed) pypy/release/1.4.x/pypy/module/array/test/test_array_old.py (props changed) pypy/release/1.4.x/pypy/rlib/rerased.py (props changed) pypy/release/1.4.x/pypy/rlib/test/test_rerased.py (props changed) pypy/release/1.4.x/pypy/rpython/memory/gc/base.py pypy/release/1.4.x/pypy/rpython/memory/gc/generation.py pypy/release/1.4.x/pypy/rpython/memory/gc/inspector.py pypy/release/1.4.x/pypy/rpython/memory/gc/minimark.py pypy/release/1.4.x/pypy/rpython/memory/gc/semispace.py pypy/release/1.4.x/pypy/translator/c/test/test_newgc.py Log: merge 79029 from trunk Modified: pypy/release/1.4.x/pypy/rpython/memory/gc/base.py ============================================================================== --- pypy/release/1.4.x/pypy/rpython/memory/gc/base.py (original) +++ pypy/release/1.4.x/pypy/rpython/memory/gc/base.py Mon Nov 15 20:12:56 2010 @@ -247,6 +247,21 @@ (not self.config.taggedpointers or llmemory.cast_adr_to_int(addr) & 1 == 0)) + def enumerate_all_roots(self, callback, arg): + """For each root object, invoke callback(obj, arg). + 'callback' should not be a bound method. + Note that this method is not suitable for actually doing the + collection in a moving GC, because you cannot write back a + modified address. It is there only for inspection. + """ + # overridden in some subclasses, for GCs which have an additional + # list of last generation roots + callback2, attrname = _convert_callback_formats(callback) # :-/ + setattr(self, attrname, arg) + self.root_walker.walk_roots(callback2, callback2, callback2) + self.run_finalizers.foreach(callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' + def debug_check_consistency(self): """To use after a collection. If self.DEBUG is set, this enumerates all roots and traces all objects to check if we didn't @@ -260,8 +275,7 @@ self._debug_pending = self.AddressStack() if not we_are_translated(): self.root_walker._walk_prebuilt_gc(self._debug_record) - callback = GCBase._debug_callback - self.root_walker.walk_roots(callback, callback, callback) + self.enumerate_all_roots(GCBase._debug_callback, self) pending = self._debug_pending while pending.non_empty(): obj = pending.pop() @@ -275,9 +289,8 @@ seen.add(obj) self.debug_check_object(obj) self._debug_pending.append(obj) - def _debug_callback(self, root): - obj = root.address[0] - ll_assert(bool(obj), "NULL address from walk_roots()") + @staticmethod + def _debug_callback(obj, self): self._debug_record(obj) def _debug_callback2(self, pointer, ignored): obj = pointer.address[0] @@ -432,3 +445,17 @@ if factor != 1: return 0.0 return value + +def _convert_callback_formats(callback): + callback = getattr(callback, 'im_func', callback) + if callback not in _converted_callback_formats: + def callback2(gc, root): + obj = root.address[0] + ll_assert(bool(obj), "NULL address from walk_roots()") + callback(obj, getattr(gc, attrname)) + attrname = '_callback2_arg%d' % len(_converted_callback_formats) + _converted_callback_formats[callback] = callback2, attrname + return _converted_callback_formats[callback] + +_convert_callback_formats._annspecialcase_ = 'specialize:memo' +_converted_callback_formats = {} Modified: pypy/release/1.4.x/pypy/rpython/memory/gc/generation.py ============================================================================== --- pypy/release/1.4.x/pypy/rpython/memory/gc/generation.py (original) +++ pypy/release/1.4.x/pypy/rpython/memory/gc/generation.py Mon Nov 15 20:12:56 2010 @@ -572,16 +572,10 @@ def _compute_current_nursery_hash(self, obj): return intmask(llmemory.cast_adr_to_int(obj) + self.nursery_hash_base) - def heap_stats_walk_roots(self): - self.last_generation_root_objects.foreach( - self._track_heap_ext, None) - self.root_walker.walk_roots( - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root) - - def _track_heap_ext(self, adr, ignored): - self.trace(adr, self.track_heap_parent, adr) + def enumerate_all_roots(self, callback, arg): + self.last_generation_root_objects.foreach(callback, arg) + SemiSpaceGC.enumerate_all_roots(self, callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' def debug_check_object(self, obj): """Check the invariants about 'obj' that should be true Modified: pypy/release/1.4.x/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/release/1.4.x/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/release/1.4.x/pypy/rpython/memory/gc/inspector.py Mon Nov 15 20:12:56 2010 @@ -11,18 +11,15 @@ # ---------- implementation of pypy.rlib.rgc.get_rpy_roots() ---------- -def _counting_rpy_root(gc, root): +def _counting_rpy_root(obj, gc): gc._count_rpy += 1 def _do_count_rpy_roots(gc): gc._count_rpy = 0 - gc.root_walker.walk_roots( - _counting_rpy_root, - _counting_rpy_root, - _counting_rpy_root) + gc.enumerate_all_roots(_counting_rpy_root, gc) return gc._count_rpy -def _append_rpy_root(gc, root): +def _append_rpy_root(obj, gc): # Can use the gc list, but should not allocate! # It is essential that the list is not resizable! lst = gc._list_rpy @@ -30,15 +27,12 @@ if index >= len(lst): raise ValueError gc._count_rpy = index + 1 - lst[index] = llmemory.cast_adr_to_ptr(root.address[0], llmemory.GCREF) + lst[index] = llmemory.cast_adr_to_ptr(obj, llmemory.GCREF) def _do_append_rpy_roots(gc, lst): gc._count_rpy = 0 gc._list_rpy = lst - gc.root_walker.walk_roots( - _append_rpy_root, - _append_rpy_root, - _append_rpy_root) + gc.enumerate_all_roots(_append_rpy_root, gc) gc._list_rpy = None def get_rpy_roots(gc): @@ -172,12 +166,7 @@ self.pending.append(obj) def add_roots(self): - self.gc._heap_dumper = self - self.gc.root_walker.walk_roots( - _hd_add_root, - _hd_add_root, - _hd_add_root) - self.gc._heap_dumper = None + self.gc.enumerate_all_roots(_hd_add_root, self) pendingroots = self.pending self.pending = AddressStack() self.walk(pendingroots) @@ -188,8 +177,8 @@ while pending.non_empty(): self.writeobj(pending.pop()) -def _hd_add_root(gc, root): - gc._heap_dumper.add(root.address[0]) +def _hd_add_root(obj, heap_dumper): + heap_dumper.add(obj) def dump_rpy_heap(gc, fd): heapdumper = HeapDumper(gc, fd) Modified: pypy/release/1.4.x/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/release/1.4.x/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/release/1.4.x/pypy/rpython/memory/gc/minimark.py Mon Nov 15 20:12:56 2010 @@ -1288,6 +1288,11 @@ self.run_finalizers.foreach(self._collect_obj, self.objects_to_trace) + def enumerate_all_roots(self, callback, arg): + self.prebuilt_root_objects.foreach(callback, arg) + MovingGCBase.enumerate_all_roots(self, callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' + @staticmethod def _collect_obj(obj, objects_to_trace): objects_to_trace.append(obj) Modified: pypy/release/1.4.x/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/release/1.4.x/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/release/1.4.x/pypy/rpython/memory/gc/semispace.py Mon Nov 15 20:12:56 2010 @@ -693,15 +693,10 @@ self._ll_typeid_map[idx].size += llmemory.raw_malloc_usage(totsize) self.trace(adr, self.track_heap_parent, adr) - def _track_heap_root(self, root): - self.track_heap(root.address[0]) + @staticmethod + def _track_heap_root(obj, self): + self.track_heap(obj) - def heap_stats_walk_roots(self): - self.root_walker.walk_roots( - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root) - def heap_stats(self): self._tracked_dict = self.AddressDict() max_tid = self.root_walker.gcdata.max_type_id @@ -714,7 +709,7 @@ while i < max_tid: self._tracked_dict.add(llmemory.cast_ptr_to_adr(ll_typeid_map[i])) i += 1 - self.heap_stats_walk_roots() + self.enumerate_all_roots(SemiSpaceGC._track_heap_root, self) self._ll_typeid_map = lltype.nullptr(ARRAY_TYPEID_MAP) self._tracked_dict.delete() return ll_typeid_map Modified: pypy/release/1.4.x/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/c/test/test_newgc.py (original) +++ pypy/release/1.4.x/pypy/translator/c/test/test_newgc.py Mon Nov 15 20:12:56 2010 @@ -1173,21 +1173,22 @@ b = 0 c = 0 for i in range(len(tb)): - if tb[i].count == 10: + if tb[i].count == 10: # the type of S a += 1 nr = i for i in range(len(tb)): - if tb[i].count == 3: + if tb[i].count == 3: # the type GcArray(Ptr(S)) b += 1 c += tb[i].links[nr] - # we don't count b here since there can be more singletons, + # b can be 1 or 2 here since _heap_stats() is free to return or + # ignore the three GcStructs that point to the GcArray(Ptr(S)). # important one is c, a is for check return c * 100 + b * 10 + a return f def test_gc_heap_stats(self): res = self.run("gc_heap_stats") - assert res == 3011 + assert res == 3011 or res == 3021 def definestr_string_builder(cls): def fn(_): From fijal at codespeak.net Mon Nov 15 20:14:26 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 15 Nov 2010 20:14:26 +0100 (CET) Subject: [pypy-svn] r79130 - in pypy/release/1.4.x: . pypy pypy/jit/backend/test pypy/jit/metainterp/optimizeopt pypy/module/array/benchmark pypy/module/array/test pypy/rlib pypy/rlib/test Message-ID: <20101115191426.E9683282BD4@codespeak.net> Author: fijal Date: Mon Nov 15 20:14:25 2010 New Revision: 79130 Modified: pypy/release/1.4.x/ (props changed) pypy/release/1.4.x/pypy/ (props changed) pypy/release/1.4.x/pypy/jit/backend/test/runner_test.py pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/Makefile (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimg.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/loop.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sum.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.py (props changed) pypy/release/1.4.x/pypy/module/array/test/test_array_old.py (props changed) pypy/release/1.4.x/pypy/rlib/rerased.py (props changed) pypy/release/1.4.x/pypy/rlib/test/test_rerased.py (props changed) Log: merge 79030 from trunk Modified: pypy/release/1.4.x/pypy/jit/backend/test/runner_test.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/backend/test/runner_test.py (original) +++ pypy/release/1.4.x/pypy/jit/backend/test/runner_test.py Mon Nov 15 20:14:25 2010 @@ -2168,12 +2168,14 @@ # Tested with a function that intentionally does not cast the # result to RESTYPE, but makes sure that we return the whole # value in eax or rax. - eci = ExternalCompilationInfo(separate_module_sources=[""" + eci = ExternalCompilationInfo( + separate_module_sources=[""" long fn_test_result_of_call(long x) { return x + 1; } - """]) + """], + export_symbols=['fn_test_result_of_call']) f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed], RESTYPE, compilation_info=eci, _nowrapper=True) value = intmask(0xFFEEDDCCBBAA9988) @@ -2199,12 +2201,14 @@ # Tested with a function that intentionally does not cast the # result to RESTYPE, but makes sure that we return the whole # value in eax or rax. - eci = ExternalCompilationInfo(separate_module_sources=[""" + eci = ExternalCompilationInfo( + separate_module_sources=[""" long fn_test_result_of_call(long x) { return x + 1; } - """]) + """], + export_symbols=['fn_test_result_of_call']) f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed], RESTYPE, compilation_info=eci, _nowrapper=True) value = intmask(0xFFEEDDCCBBAA9988) From fijal at codespeak.net Mon Nov 15 20:25:18 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 15 Nov 2010 20:25:18 +0100 (CET) Subject: [pypy-svn] r79131 - in pypy/release/1.4.x: . pypy pypy/jit/backend/llsupport pypy/jit/backend/llsupport/test pypy/jit/metainterp/optimizeopt pypy/module/__pypy__ pypy/module/array/benchmark pypy/module/array/test pypy/module/gc pypy/module/gc/test pypy/objspace/std pypy/objspace/std/test pypy/rlib pypy/rlib/test pypy/rpython pypy/rpython/lltypesystem pypy/rpython/memory/gc pypy/rpython/memory/gc/test pypy/rpython/memory/gctransform pypy/tool pypy/translator/c pypy/translator/c/test Message-ID: <20101115192518.0D3DE50810@codespeak.net> Author: fijal Date: Mon Nov 15 20:25:15 2010 New Revision: 79131 Added: pypy/release/1.4.x/pypy/tool/gcdump.py - copied unchanged from r79067, pypy/trunk/pypy/tool/gcdump.py Modified: pypy/release/1.4.x/ (props changed) pypy/release/1.4.x/pypy/ (props changed) pypy/release/1.4.x/pypy/jit/backend/llsupport/descr.py pypy/release/1.4.x/pypy/jit/backend/llsupport/test/test_descr.py pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py (props changed) pypy/release/1.4.x/pypy/module/__pypy__/__init__.py pypy/release/1.4.x/pypy/module/__pypy__/interp_magic.py pypy/release/1.4.x/pypy/module/array/benchmark/Makefile (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimg.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/loop.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sum.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.py (props changed) pypy/release/1.4.x/pypy/module/array/test/test_array_old.py (props changed) pypy/release/1.4.x/pypy/module/gc/__init__.py pypy/release/1.4.x/pypy/module/gc/app_referents.py pypy/release/1.4.x/pypy/module/gc/interp_gc.py pypy/release/1.4.x/pypy/module/gc/referents.py pypy/release/1.4.x/pypy/module/gc/test/test_gc.py pypy/release/1.4.x/pypy/objspace/std/callmethod.py pypy/release/1.4.x/pypy/objspace/std/mapdict.py pypy/release/1.4.x/pypy/objspace/std/test/test_dictmultiobject.py pypy/release/1.4.x/pypy/objspace/std/test/test_mapdict.py pypy/release/1.4.x/pypy/rlib/rerased.py (props changed) pypy/release/1.4.x/pypy/rlib/rgc.py pypy/release/1.4.x/pypy/rlib/test/test_rerased.py (props changed) pypy/release/1.4.x/pypy/rpython/llinterp.py pypy/release/1.4.x/pypy/rpython/lltypesystem/lloperation.py pypy/release/1.4.x/pypy/rpython/memory/gc/inspector.py pypy/release/1.4.x/pypy/rpython/memory/gc/minimark.py pypy/release/1.4.x/pypy/rpython/memory/gc/test/test_minimark.py pypy/release/1.4.x/pypy/rpython/memory/gctransform/framework.py pypy/release/1.4.x/pypy/translator/c/node.py pypy/release/1.4.x/pypy/translator/c/test/test_lltyped.py pypy/release/1.4.x/pypy/translator/c/test/test_newgc.py Log: Merged revisions: 790{36,37,52,53,55,66,67,88} from trunk Modified: pypy/release/1.4.x/pypy/jit/backend/llsupport/descr.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/backend/llsupport/descr.py (original) +++ pypy/release/1.4.x/pypy/jit/backend/llsupport/descr.py Mon Nov 15 20:25:15 2010 @@ -130,6 +130,7 @@ # ArrayDescrs _A = lltype.GcArray(lltype.Signed) # a random gcarray +_AF = lltype.GcArray(lltype.Float) # an array of C doubles class BaseArrayDescr(AbstractDescr): @@ -171,16 +172,21 @@ _clsname = 'GcPtrArrayDescr' _is_array_of_pointers = True -_CA = rffi.CArray(lltype.Signed) +class FloatArrayDescr(BaseArrayDescr): + _clsname = 'FloatArrayDescr' + _is_array_of_floats = True + def get_base_size(self, translate_support_code): + basesize, _, _ = symbolic.get_array_token(_AF, translate_support_code) + return basesize + def get_item_size(self, translate_support_code): + return symbolic.get_size(lltype.Float, translate_support_code) class BaseArrayNoLengthDescr(BaseArrayDescr): def get_base_size(self, translate_support_code): - basesize, _, _ = symbolic.get_array_token(_CA, translate_support_code) - return basesize + return 0 def get_ofs_length(self, translate_support_code): - _, _, ofslength = symbolic.get_array_token(_CA, translate_support_code) - return ofslength + return -1 class NonGcPtrArrayNoLengthDescr(BaseArrayNoLengthDescr): _clsname = 'NonGcPtrArrayNoLengthDescr' @@ -192,6 +198,8 @@ _is_array_of_pointers = True def getArrayDescrClass(ARRAY): + if ARRAY.OF is lltype.Float: + return FloatArrayDescr return getDescrClass(ARRAY.OF, BaseArrayDescr, GcPtrArrayDescr, NonGcPtrArrayDescr, 'Array', 'get_item_size', '_is_array_of_floats', '_is_item_signed') @@ -219,7 +227,8 @@ basesize, itemsize, ofslength = symbolic.get_array_token(ARRAY, False) assert basesize == arraydescr.get_base_size(False) assert itemsize == arraydescr.get_item_size(False) - assert ofslength == arraydescr.get_ofs_length(False) + if not ARRAY._hints.get('nolength', False): + assert ofslength == arraydescr.get_ofs_length(False) if isinstance(ARRAY, lltype.GcArray): gccache.init_array_descr(ARRAY, arraydescr) cache[ARRAY] = arraydescr Modified: pypy/release/1.4.x/pypy/jit/backend/llsupport/test/test_descr.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/backend/llsupport/test/test_descr.py (original) +++ pypy/release/1.4.x/pypy/jit/backend/llsupport/test/test_descr.py Mon Nov 15 20:25:15 2010 @@ -5,6 +5,7 @@ from pypy.rpython.annlowlevel import llhelper from pypy.jit.metainterp.history import BoxInt, BoxFloat, BoxPtr from pypy.jit.metainterp import history +import struct def test_get_size_descr(): c0 = GcCache(False) @@ -130,11 +131,13 @@ assert not descr3.is_array_of_floats() assert descr4.is_array_of_floats() # - WORD = rffi.sizeof(lltype.Signed) - assert descr1.get_base_size(False) == WORD - assert descr2.get_base_size(False) == WORD - assert descr3.get_base_size(False) == WORD - assert descr4.get_base_size(False) == WORD + def get_alignment(code): + # Retrieve default alignment for the compiler/platform + return struct.calcsize('l' + code) - struct.calcsize(code) + assert descr1.get_base_size(False) == get_alignment('c') + assert descr2.get_base_size(False) == get_alignment('p') + assert descr3.get_base_size(False) == get_alignment('p') + assert descr4.get_base_size(False) == get_alignment('d') assert descr1.get_ofs_length(False) == 0 assert descr2.get_ofs_length(False) == 0 assert descr3.get_ofs_length(False) == 0 Modified: pypy/release/1.4.x/pypy/module/__pypy__/__init__.py ============================================================================== --- pypy/release/1.4.x/pypy/module/__pypy__/__init__.py (original) +++ pypy/release/1.4.x/pypy/module/__pypy__/__init__.py Mon Nov 15 20:25:15 2010 @@ -23,6 +23,9 @@ 'interp_magic.method_cache_counter') self.extra_interpdef('reset_method_cache_counter', 'interp_magic.reset_method_cache_counter') + if self.space.config.objspace.std.withmapdict: + self.extra_interpdef('mapdict_cache_counter', + 'interp_magic.mapdict_cache_counter') PYC_MAGIC = get_pyc_magic(self.space) self.extra_interpdef('PYC_MAGIC', 'space.wrap(%d)' % PYC_MAGIC) # Modified: pypy/release/1.4.x/pypy/module/__pypy__/interp_magic.py ============================================================================== --- pypy/release/1.4.x/pypy/module/__pypy__/interp_magic.py (original) +++ pypy/release/1.4.x/pypy/module/__pypy__/interp_magic.py Mon Nov 15 20:25:15 2010 @@ -2,6 +2,7 @@ from pypy.interpreter.gateway import ObjSpace from pypy.rlib.objectmodel import we_are_translated from pypy.objspace.std.typeobject import MethodCache +from pypy.objspace.std.mapdict import IndexCache def internal_repr(space, w_object): return space.wrap('%r' % (w_object,)) @@ -36,4 +37,17 @@ cache = space.fromcache(MethodCache) cache.misses = {} cache.hits = {} - + if space.config.objspace.std.withmapdict: + cache = space.fromcache(IndexCache) + cache.misses = {} + cache.hits = {} + +def mapdict_cache_counter(space, name): + """Return a tuple (index_cache_hits, index_cache_misses) for lookups + in the mapdict cache with the given attribute name.""" + assert space.config.objspace.std.withmethodcachecounter + assert space.config.objspace.std.withmapdict + cache = space.fromcache(IndexCache) + return space.newtuple([space.newint(cache.hits.get(name, 0)), + space.newint(cache.misses.get(name, 0))]) +mapdict_cache_counter.unwrap_spec = [ObjSpace, str] Modified: pypy/release/1.4.x/pypy/module/gc/__init__.py ============================================================================== --- pypy/release/1.4.x/pypy/module/gc/__init__.py (original) +++ pypy/release/1.4.x/pypy/module/gc/__init__.py Mon Nov 15 20:25:15 2010 @@ -29,6 +29,7 @@ 'get_referents': 'referents.get_referents', 'get_referrers': 'referents.get_referrers', '_dump_rpy_heap': 'referents._dump_rpy_heap', + 'get_typeids_z': 'referents.get_typeids_z', 'GcRef': 'referents.W_GcRef', }) MixedModule.__init__(self, space, w_name) Modified: pypy/release/1.4.x/pypy/module/gc/app_referents.py ============================================================================== --- pypy/release/1.4.x/pypy/module/gc/app_referents.py (original) +++ pypy/release/1.4.x/pypy/module/gc/app_referents.py Mon Nov 15 20:25:15 2010 @@ -14,11 +14,25 @@ and [addr1]..[addrn] are addresses of other objects that this object points to. The full dump is a list of such objects, with a marker [0][0][0][-1] inserted after all GC roots, before all non-roots. + + If the argument is a filename and the 'zlib' module is available, + we also write a 'typeids.txt' in the same directory, if none exists. """ if isinstance(file, str): f = open(file, 'wb') gc._dump_rpy_heap(f.fileno()) f.close() + try: + import zlib, os + except ImportError: + pass + else: + filename2 = os.path.join(os.path.dirname(file), 'typeids.txt') + if not os.path.exists(filename2): + data = zlib.decompress(gc.get_typeids_z()) + f = open(filename2, 'wb') + f.write(data) + f.close() else: if isinstance(file, int): fd = file Modified: pypy/release/1.4.x/pypy/module/gc/interp_gc.py ============================================================================== --- pypy/release/1.4.x/pypy/module/gc/interp_gc.py (original) +++ pypy/release/1.4.x/pypy/module/gc/interp_gc.py Mon Nov 15 20:25:15 2010 @@ -10,6 +10,10 @@ from pypy.objspace.std.typeobject import MethodCache cache = space.fromcache(MethodCache) cache.clear() + if space.config.objspace.std.withmapdict: + from pypy.objspace.std.mapdict import IndexCache + cache = space.fromcache(IndexCache) + cache.clear() rgc.collect() return space.wrap(0) Modified: pypy/release/1.4.x/pypy/module/gc/referents.py ============================================================================== --- pypy/release/1.4.x/pypy/module/gc/referents.py (original) +++ pypy/release/1.4.x/pypy/module/gc/referents.py Mon Nov 15 20:25:15 2010 @@ -177,3 +177,9 @@ if not ok: raise missing_operation(space) _dump_rpy_heap.unwrap_spec = [ObjSpace, int] + +def get_typeids_z(space): + a = rgc.get_typeids_z() + s = ''.join([a[i] for i in range(len(a))]) + return space.wrap(s) +get_typeids_z.unwrap_spec = [ObjSpace] Modified: pypy/release/1.4.x/pypy/module/gc/test/test_gc.py ============================================================================== --- pypy/release/1.4.x/pypy/module/gc/test/test_gc.py (original) +++ pypy/release/1.4.x/pypy/module/gc/test/test_gc.py Mon Nov 15 20:25:15 2010 @@ -117,6 +117,33 @@ pass C().f() # Fill the method cache rlist.append(weakref.ref(C)) + for i in range(10): + f() + gc.collect() # the classes C should all go away here + # the last class won't go in mapdict, as long as the code object of f + # is around + rlist.pop() + for r in rlist: + assert r() is None + +class AppTestGcMapDictIndexCache(AppTestGcMethodCache): + def setup_class(cls): + cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True, + "objspace.std.withmapdict": True}) + + + def test_clear_index_cache(self): + import gc, weakref + rlist = [] + def f(): + class C(object): + def f(self): + pass + c = C() + c.x = 1 + getattr(c, "x") # fill the index cache without using the local cache + getattr(c, "x") + rlist.append(weakref.ref(C)) for i in range(5): f() gc.collect() # the classes C should all go away here Modified: pypy/release/1.4.x/pypy/objspace/std/callmethod.py ============================================================================== --- pypy/release/1.4.x/pypy/objspace/std/callmethod.py (original) +++ pypy/release/1.4.x/pypy/objspace/std/callmethod.py Mon Nov 15 20:25:15 2010 @@ -13,6 +13,9 @@ from pypy.interpreter import function from pypy.objspace.descroperation import object_getattribute from pypy.rlib import jit, rstack # for resume points +from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict, \ + LOOKUP_METHOD_mapdict_fill_cache_method + # This module exports two extra methods for StdObjSpaceFrame implementing # the LOOKUP_METHOD and CALL_METHOD opcodes in an efficient way, as well @@ -30,6 +33,13 @@ # space = f.space w_obj = f.popvalue() + + if space.config.objspace.std.withmapdict and not jit.we_are_jitted(): + # mapdict has an extra-fast version of this function + from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict + if LOOKUP_METHOD_mapdict(f, nameindex, w_obj): + return + w_name = f.getname_w(nameindex) w_value = None @@ -50,6 +60,11 @@ # nothing in the instance f.pushvalue(w_descr) f.pushvalue(w_obj) + if (space.config.objspace.std.withmapdict and + not jit.we_are_jitted()): + # let mapdict cache stuff + LOOKUP_METHOD_mapdict_fill_cache_method( + f.getcode(), nameindex, w_obj, w_type, w_descr) return if w_value is None: w_value = space.getattr(w_obj, w_name) Modified: pypy/release/1.4.x/pypy/objspace/std/mapdict.py ============================================================================== --- pypy/release/1.4.x/pypy/objspace/std/mapdict.py (original) +++ pypy/release/1.4.x/pypy/objspace/std/mapdict.py Mon Nov 15 20:25:15 2010 @@ -1,4 +1,5 @@ from pypy.rlib import jit, objectmodel, debug +from pypy.rlib.rarithmetic import intmask, r_uint from pypy.interpreter.baseobjspace import W_Root from pypy.objspace.std.dictmultiobject import W_DictMultiObject @@ -15,24 +16,70 @@ # we want to propagate knowledge that the result cannot be negative class AbstractAttribute(object): - _immutable_fields_ = ['w_cls'] + _immutable_fields_ = ['terminator'] cache_attrs = None _size_estimate = 0 - def __init__(self, space, w_cls): + def __init__(self, space, terminator): self.space = space - self.w_cls = w_cls + assert isinstance(terminator, Terminator) + self.terminator = terminator def read(self, obj, selector): - raise NotImplementedError("abstract base class") + index = self.index(selector) + if index < 0: + return self.terminator._read_terminator(obj, selector) + return obj._mapdict_read_storage(index) def write(self, obj, selector, w_value): - raise NotImplementedError("abstract base class") + index = self.index(selector) + if index < 0: + return self.terminator._write_terminator(obj, selector, w_value) + obj._mapdict_write_storage(index, w_value) + return True def delete(self, obj, selector): return None def index(self, selector): + if (self.space.config.objspace.std.withmethodcache and + not jit.we_are_jitted()): + return self._index_cache(selector) + else: + return self._index(selector) + + @jit.dont_look_inside + def _index_cache(self, selector): + space = self.space + cache = space.fromcache(IndexCache) + SHIFT2 = r_uint.BITS - space.config.objspace.std.methodcachesizeexp + SHIFT1 = SHIFT2 - 5 + attrs_as_int = objectmodel.current_object_addr_as_int(self) + # ^^^Note: see comment in typeobject.py for + # _pure_lookup_where_with_method_cache() + hash_selector = objectmodel.compute_hash(selector) + product = intmask(attrs_as_int * hash_selector) + index_hash = (r_uint(product) ^ (r_uint(product) << SHIFT1)) >> SHIFT2 + # ^^^Note2: same comment too + cached_attr = cache.attrs[index_hash] + if cached_attr is self: + cached_selector = cache.selectors[index_hash] + if cached_selector == selector: + index = cache.indices[index_hash] + if space.config.objspace.std.withmethodcachecounter: + name = selector[0] + cache.hits[name] = cache.hits.get(name, 0) + 1 + return index + index = self._index(selector) + cache.attrs[index_hash] = self + cache.selectors[index_hash] = selector + cache.indices[index_hash] = index + if space.config.objspace.std.withmethodcachecounter: + name = selector[0] + cache.misses[name] = cache.misses.get(name, 0) + 1 + return index + + def _index(self, selector): return -1 def copy(self, obj): @@ -42,7 +89,7 @@ raise NotImplementedError("abstract base class") def get_terminator(self): - raise NotImplementedError("abstract base class") + return self.terminator def set_terminator(self, obj, terminator): raise NotImplementedError("abstract base class") @@ -95,15 +142,20 @@ raise NotImplementedError("abstract base class") def __repr__(self): - return "<%s w_cls=%s>" % (self.__class__.__name__, self.w_cls) + return "<%s>" % (self.__class__.__name__,) class Terminator(AbstractAttribute): + _immutable_fields_ = ['w_cls'] - def read(self, obj, selector): + def __init__(self, space, w_cls): + AbstractAttribute.__init__(self, space, self) + self.w_cls = w_cls + + def _read_terminator(self, obj, selector): return None - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): obj._get_mapdict_map().add_attr(obj, selector, w_value) return True @@ -116,9 +168,6 @@ def length(self): return 0 - def get_terminator(self): - return self - def set_terminator(self, obj, terminator): result = Object() result.space = self.space @@ -128,6 +177,9 @@ def remove_dict_entries(self, obj): return self.copy(obj) + def __repr__(self): + return "<%s w_cls=%s>" % (self.__class__.__name__, self.w_cls) + class DictTerminator(Terminator): _immutable_fields_ = ['devolved_dict_terminator'] def __init__(self, space, w_cls): @@ -142,27 +194,27 @@ class NoDictTerminator(Terminator): - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): if selector[1] == DICT: return False - return Terminator.write(self, obj, selector, w_value) + return Terminator._write_terminator(self, obj, selector, w_value) class DevolvedDictTerminator(Terminator): - def read(self, obj, selector): + def _read_terminator(self, obj, selector): if selector[1] == DICT: w_dict = obj.getdict() space = self.space return space.finditem_str(w_dict, selector[0]) - return Terminator.read(self, obj, selector) + return Terminator._read_terminator(self, obj, selector) - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): if selector[1] == DICT: w_dict = obj.getdict() space = self.space space.setitem_str(w_dict, selector[0], w_value) return True - return Terminator.write(self, obj, selector, w_value) + return Terminator._write_terminator(self, obj, selector, w_value) def delete(self, obj, selector): from pypy.interpreter.error import OperationError @@ -189,7 +241,7 @@ class PlainAttribute(AbstractAttribute): _immutable_fields_ = ['selector', 'position', 'back'] def __init__(self, selector, back): - AbstractAttribute.__init__(self, back.space, back.w_cls) + AbstractAttribute.__init__(self, back.space, back.terminator) self.selector = selector self.position = back.length() self.back = back @@ -199,17 +251,6 @@ w_value = self.read(obj, self.selector) new_obj._get_mapdict_map().add_attr(new_obj, self.selector, w_value) - def read(self, obj, selector): - if selector == self.selector: - return obj._mapdict_read_storage(self.position) - return self.back.read(obj, selector) - - def write(self, obj, selector, w_value): - if selector == self.selector: - obj._mapdict_write_storage(self.position, w_value) - return True - return self.back.write(obj, selector, w_value) - def delete(self, obj, selector): if selector == self.selector: # ok, attribute is deleted @@ -219,10 +260,10 @@ self._copy_attr(obj, new_obj) return new_obj - def index(self, selector): + def _index(self, selector): if selector == self.selector: return self.position - return self.back.index(selector) + return self.back._index(selector) def copy(self, obj): new_obj = self.back.copy(obj) @@ -232,9 +273,6 @@ def length(self): return self.position + 1 - def get_terminator(self): - return self.back.get_terminator() - def set_terminator(self, obj, terminator): new_obj = self.back.set_terminator(obj, terminator) self._copy_attr(obj, new_obj) @@ -268,6 +306,24 @@ # RPython reasons w_obj._set_mapdict_storage_and_map(new_obj.storage, new_obj.map) +class IndexCache(object): + def __init__(self, space): + assert space.config.objspace.std.withmethodcache + SIZE = 1 << space.config.objspace.std.methodcachesizeexp + self.attrs = [None] * SIZE + self._empty_selector = (None, INVALID) + self.selectors = [self._empty_selector] * SIZE + self.indices = [0] * SIZE + if space.config.objspace.std.withmethodcachecounter: + self.hits = {} + self.misses = {} + + def clear(self): + for i in range(len(self.attrs)): + self.attrs[i] = None + for i in range(len(self.selectors)): + self.selectors[i] = self._empty_selector + # ____________________________________________________________ # object implementation @@ -328,7 +384,7 @@ assert flag def getclass(self, space): - return self._get_mapdict_map().w_cls + return self._get_mapdict_map().terminator.w_cls def setclass(self, space, w_cls): new_obj = self._get_mapdict_map().set_terminator(self, w_cls.terminator) @@ -373,6 +429,7 @@ self.storage = make_sure_not_resized([None] * map.size_estimate()) def _mapdict_read_storage(self, index): + assert index >= 0 return self.storage[index] def _mapdict_write_storage(self, index, value): self.storage[index] = value @@ -440,6 +497,7 @@ return rerased.unerase_fixedsizelist(erased, W_Root) def _mapdict_read_storage(self, index): + assert index >= 0 for i in rangenmin1: if index == i: erased = getattr(self, "_value%s" % i) @@ -604,30 +662,54 @@ map = None version_tag = None index = 0 + w_method = None # for callmethod success_counter = 0 failure_counter = 0 + def is_valid_for_obj(self, w_obj): + map = w_obj._get_mapdict_map() + return self.is_valid_for_map(map) + + def is_valid_for_map(self, map): + if map is self.map: + version_tag = map.terminator.w_cls.version_tag() + if version_tag is self.version_tag: + # everything matches, it's incredibly fast + if map.space.config.objspace.std.withmethodcachecounter: + self.success_counter += 1 + return True + return False + INVALID_CACHE_ENTRY = CacheEntry() INVALID_CACHE_ENTRY.map = objectmodel.instantiate(AbstractAttribute) # different from any real map ^^^ -INVALID_CACHE_ENTRY.map.w_cls = None +INVALID_CACHE_ENTRY.map.terminator = None + def init_mapdict_cache(pycode): num_entries = len(pycode.co_names_w) pycode._mapdict_caches = [INVALID_CACHE_ENTRY] * num_entries +def _fill_cache(pycode, nameindex, map, version_tag, index, w_method=None): + entry = pycode._mapdict_caches[nameindex] + if entry is INVALID_CACHE_ENTRY: + entry = CacheEntry() + pycode._mapdict_caches[nameindex] = entry + entry.map = map + entry.version_tag = version_tag + entry.index = index + entry.w_method = w_method + if pycode.space.config.objspace.std.withmethodcachecounter: + entry.failure_counter += 1 + def LOAD_ATTR_caching(pycode, w_obj, nameindex): # this whole mess is to make the interpreter quite a bit faster; it's not # used if we_are_jitted(). entry = pycode._mapdict_caches[nameindex] map = w_obj._get_mapdict_map() - if map is entry.map: - version_tag = map.w_cls.version_tag() - if version_tag is entry.version_tag: - # everything matches, it's incredibly fast - if pycode.space.config.objspace.std.withmethodcachecounter: - entry.success_counter += 1 - return w_obj._mapdict_read_storage(entry.index) + if entry.is_valid_for_map(map) and entry.w_method is None: + # everything matches, it's incredibly fast + return w_obj._mapdict_read_storage(entry.index) return LOAD_ATTR_slowpath(pycode, w_obj, nameindex, map) LOAD_ATTR_caching._always_inline_ = True @@ -635,7 +717,7 @@ space = pycode.space w_name = pycode.co_names_w[nameindex] if map is not None: - w_type = map.w_cls + w_type = map.terminator.w_cls w_descr = w_type.getattribute_if_not_from_object() if w_descr is not None: return space._handle_getattribute(w_descr, w_obj, w_name) @@ -655,17 +737,30 @@ if selector[1] != INVALID: index = map.index(selector) if index >= 0: - entry = pycode._mapdict_caches[nameindex] - if entry is INVALID_CACHE_ENTRY: - entry = CacheEntry() - pycode._mapdict_caches[nameindex] = entry - entry.map = map - entry.version_tag = version_tag - entry.index = index - if space.config.objspace.std.withmethodcachecounter: - entry.failure_counter += 1 + _fill_cache(pycode, nameindex, map, version_tag, index) return w_obj._mapdict_read_storage(index) if space.config.objspace.std.withmethodcachecounter: INVALID_CACHE_ENTRY.failure_counter += 1 return space.getattr(w_obj, w_name) LOAD_ATTR_slowpath._dont_inline_ = True + +def LOOKUP_METHOD_mapdict(f, nameindex, w_obj): + space = f.space + pycode = f.getcode() + entry = pycode._mapdict_caches[nameindex] + if entry.is_valid_for_obj(w_obj): + w_method = entry.w_method + if w_method is not None: + f.pushvalue(w_method) + f.pushvalue(w_obj) + return True + return False + +def LOOKUP_METHOD_mapdict_fill_cache_method(pycode, nameindex, w_obj, w_type, w_method): + version_tag = w_type.version_tag() + if version_tag is None: + return + map = w_obj._get_mapdict_map() + if map is None: + return + _fill_cache(pycode, nameindex, map, version_tag, -1, w_method) Modified: pypy/release/1.4.x/pypy/objspace/std/test/test_dictmultiobject.py ============================================================================== --- pypy/release/1.4.x/pypy/objspace/std/test/test_dictmultiobject.py (original) +++ pypy/release/1.4.x/pypy/objspace/std/test/test_dictmultiobject.py Mon Nov 15 20:25:15 2010 @@ -616,6 +616,7 @@ withdictmeasurement = False withsmalldicts = False withcelldict = False + withmethodcache = False class opcodes: CALL_LIKELY_BUILTIN = False Modified: pypy/release/1.4.x/pypy/objspace/std/test/test_mapdict.py ============================================================================== --- pypy/release/1.4.x/pypy/objspace/std/test/test_mapdict.py (original) +++ pypy/release/1.4.x/pypy/objspace/std/test/test_mapdict.py Mon Nov 15 20:25:15 2010 @@ -24,13 +24,13 @@ hasdict = False def test_plain_attribute(): - space = " " w_cls = "class" aa = PlainAttribute(("b", DICT), PlainAttribute(("a", DICT), Terminator(space, w_cls))) assert aa.space is space - assert aa.w_cls is w_cls + assert aa.terminator.w_cls is w_cls + assert aa.get_terminator() is aa.terminator obj = Object() obj.map, obj.storage = aa, [10, 20] @@ -604,7 +604,8 @@ from pypy.interpreter import gateway cls.space = gettestobjspace( **{"objspace.std.withmapdict": True, - "objspace.std.withmethodcachecounter": True}) + "objspace.std.withmethodcachecounter": True, + "objspace.opcodes.CALL_METHOD": True}) # def check(space, w_func, name): w_code = space.getattr(w_func, space.wrap('func_code')) @@ -785,6 +786,135 @@ res = self.check(f, 'x') assert res == (0, 0, 1) + def test_call_method_uses_cache(self): + # bit sucky + global C + + class C(object): + def m(*args): + return args + C.sm = staticmethod(C.m.im_func) + C.cm = classmethod(C.m.im_func) + + exec """if 1: + + def f(): + c = C() + res = c.m(1) + assert res == (c, 1) + return 42 + + def g(): + c = C() + res = c.sm(1) + assert res == (1, ) + return 42 + + def h(): + c = C() + res = c.cm(1) + assert res == (C, 1) + return 42 + """ + res = self.check(f, 'm') + assert res == (1, 0, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + + # static methods are not cached + res = self.check(g, 'sm') + assert res == (0, 0, 0) + res = self.check(g, 'sm') + assert res == (0, 0, 0) + + # neither are class methods + res = self.check(h, 'cm') + assert res == (0, 0, 0) + res = self.check(h, 'cm') + assert res == (0, 0, 0) + + def test_mix_cache_bug(self): + # bit sucky + global C + + class C(object): + def m(*args): + return args + + exec """if 1: + + def f(): + c = C() + res = c.m(1) + assert res == (c, 1) + bm = c.m + res = bm(1) + assert res == (c, 1) + return 42 + + """ + res = self.check(f, 'm') + assert res == (1, 1, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + +class AppTestGlobalCaching(AppTestWithMapDict): + def setup_class(cls): + cls.space = gettestobjspace( + **{"objspace.std.withmethodcachecounter": True, + "objspace.std.withmapdict": True, + "objspace.opcodes.CALL_METHOD": True}) + + def test_mix_classes(self): + import __pypy__ + class A(object): + def f(self): + return 42 + class B(object): + def f(self): + return 43 + class C(object): + def f(self): + return 44 + l = [A(), B(), C()] * 10 + __pypy__.reset_method_cache_counter() + # 'exec' to make sure that a.f() is compiled with CALL_METHOD + exec """for i, a in enumerate(l): + assert a.f() == 42 + i % 3 +""" + cache_counter = __pypy__.mapdict_cache_counter("f") + assert cache_counter[0] >= 15 + assert cache_counter[1] >= 3 # should be (27, 3) + assert sum(cache_counter) == 30 + + def test_mix_classes_attribute(self): + import __pypy__ + class A(object): + def __init__(self): + self.x = 42 + class B(object): + def __init__(self): + self.x = 43 + class C(object): + def __init__(self): + self.x = 44 + l = [A(), B(), C()] * 10 + __pypy__.reset_method_cache_counter() + for i, a in enumerate(l): + assert a.x == 42 + i % 3 + cache_counter = __pypy__.mapdict_cache_counter("x") + assert cache_counter[0] >= 15 + assert cache_counter[1] >= 3 # should be (27, 3) + assert sum(cache_counter) == 30 + class TestDictSubclassShortcutBug(object): def setup_class(cls): cls.space = gettestobjspace( Modified: pypy/release/1.4.x/pypy/rlib/rgc.py ============================================================================== --- pypy/release/1.4.x/pypy/rlib/rgc.py (original) +++ pypy/release/1.4.x/pypy/rlib/rgc.py Mon Nov 15 20:25:15 2010 @@ -379,6 +379,11 @@ "NOT_RPYTHON" raise NotImplementedError +def get_typeids_z(): + "NOT_RPYTHON" + raise NotImplementedError + +ARRAY_OF_CHAR = lltype.Array(lltype.Char) NULL_GCREF = lltype.nullptr(llmemory.GCREF.TO) class _GcRef(object): @@ -530,3 +535,12 @@ vlist = hop.inputargs(lltype.Signed) hop.exception_is_here() return hop.genop('gc_dump_rpy_heap', vlist, resulttype = hop.r_result) + +class Entry(ExtRegistryEntry): + _about_ = get_typeids_z + def compute_result_annotation(self): + from pypy.annotation.model import SomePtr + return SomePtr(lltype.Ptr(ARRAY_OF_CHAR)) + def specialize_call(self, hop): + hop.exception_is_here() + return hop.genop('gc_typeids_z', [], resulttype = hop.r_result) Modified: pypy/release/1.4.x/pypy/rpython/llinterp.py ============================================================================== --- pypy/release/1.4.x/pypy/rpython/llinterp.py (original) +++ pypy/release/1.4.x/pypy/rpython/llinterp.py Mon Nov 15 20:25:15 2010 @@ -912,6 +912,9 @@ def op_gc_dump_rpy_heap(self): raise NotImplementedError("gc_dump_rpy_heap") + def op_gc_typeids_z(self): + raise NotImplementedError("gc_typeids_z") + def op_do_malloc_fixedsize_clear(self): raise NotImplementedError("do_malloc_fixedsize_clear") Modified: pypy/release/1.4.x/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/release/1.4.x/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/release/1.4.x/pypy/rpython/lltypesystem/lloperation.py Mon Nov 15 20:25:15 2010 @@ -476,6 +476,7 @@ 'gc_get_rpy_type_index': LLOp(), 'gc_is_rpy_instance' : LLOp(), 'gc_dump_rpy_heap' : LLOp(), + 'gc_typeids_z' : LLOp(), # ------- JIT & GC interaction, only for some GCs ---------- Modified: pypy/release/1.4.x/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/release/1.4.x/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/release/1.4.x/pypy/rpython/memory/gc/inspector.py Mon Nov 15 20:25:15 2010 @@ -4,7 +4,7 @@ from pypy.rpython.lltypesystem import lltype, llmemory, rffi from pypy.rlib.objectmodel import free_non_gc_object from pypy.rpython.module.ll_os import underscore_on_windows -from pypy.rlib import rposix +from pypy.rlib import rposix, rgc from pypy.rpython.memory.support import AddressDict, get_address_stack @@ -187,3 +187,7 @@ heapdumper.flush() heapdumper.delete() return True + +def get_typeids_z(gc): + srcaddress = gc.root_walker.gcdata.typeids_z + return llmemory.cast_adr_to_ptr(srcaddress, lltype.Ptr(rgc.ARRAY_OF_CHAR)) Modified: pypy/release/1.4.x/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/release/1.4.x/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/release/1.4.x/pypy/rpython/memory/gc/minimark.py Mon Nov 15 20:25:15 2010 @@ -1,3 +1,36 @@ +""" MiniMark GC. + +Environment variables can be used to fine-tune the following parameters: + + PYPY_GC_NURSERY The nursery size. Defaults to half the size of + the L2 cache. Try values like '1.2MB'. + + PYPY_GC_MAJOR_COLLECT Major collection memory factor. Default is '1.82', + which means trigger a major collection when the + memory consumed equals 1.82 times the memory + really used at the end of the previous major + collection. + + PYPY_GC_GROWTH Major collection threshold's max growth rate. + Default is '1.3'. Useful to collect more often + than normally on sudden memory growth, e.g. when + there is a temporary peak in memory usage. + + PYPY_GC_MAX The max heap size. If coming near this limit, it + will first collect more often, then raise an + RPython MemoryError, and if that is not enough, + crash the program with a fatal error. Try values + like '1.6GB'. + + PYPY_GC_MIN Don't collect while the memory size is below this + limit. Useful to avoid spending all the time in + the GC in very small programs. Defaults to 8 + times the nursery. +""" +# XXX Should find a way to bound the major collection threshold by the +# XXX total addressable size. Maybe by keeping some minimarkpage arenas +# XXX pre-reserved, enough for a few nursery collections? What about +# XXX raw-malloced memory? import sys from pypy.rpython.lltypesystem import lltype, llmemory, llarena, llgroup from pypy.rpython.lltypesystem.lloperation import llop @@ -87,13 +120,8 @@ TRANSLATION_PARAMS = { # Automatically adjust the size of the nursery and the - # 'major_collection_threshold' from the environment. For - # 'nursery_size' it will look it up in the env var - # PYPY_GC_NURSERY and fall back to half the size of - # the L2 cache. For 'major_collection_threshold' it will look - # it up in the env var PYPY_GC_MAJOR_COLLECT. It also sets - # 'max_heap_size' to PYPY_GC_MAX. Finally, PYPY_GC_MIN sets - # the minimal value of 'next_major_collection_threshold'. + # 'major_collection_threshold' from the environment. + # See docstring at the start of the file. "read_from_env": True, # The size of the nursery. Note that this is only used as a @@ -121,6 +149,13 @@ # we trigger the next major collection. "major_collection_threshold": 1.82, + # Threshold to avoid that the total heap size grows by a factor of + # major_collection_threshold at every collection: it can only + # grow at most by the following factor from one collection to the + # next. Used e.g. when there is a sudden, temporary peak in memory + # usage; this avoids that the upper bound grows too fast. + "growth_rate_max": 1.3, + # The number of array indices that are mapped to a single bit in # write_barrier_from_array(). Must be a power of two. The default # value of 128 means that card pages are 512 bytes (1024 on 64-bits) @@ -146,6 +181,7 @@ arena_size=64*WORD, small_request_threshold=5*WORD, major_collection_threshold=2.5, + growth_rate_max=2.5, # for tests card_page_indices=0, large_object=8*WORD, large_object_gcptrs=10*WORD, @@ -157,6 +193,7 @@ self.nursery_size = nursery_size self.small_request_threshold = small_request_threshold self.major_collection_threshold = major_collection_threshold + self.growth_rate_max = growth_rate_max self.num_major_collects = 0 self.min_heap_size = 0.0 self.max_heap_size = 0.0 @@ -257,9 +294,13 @@ newsize = max(newsize, minsize) # major_coll = base.read_float_from_env('PYPY_GC_MAJOR_COLLECT') - if major_coll >= 1.0: + if major_coll > 1.0: self.major_collection_threshold = major_coll # + growth = base.read_float_from_env('PYPY_GC_GROWTH') + if growth > 1.0: + self.growth_rate_max = growth + # min_heap_size = base.read_uint_from_env('PYPY_GC_MIN') if min_heap_size > 0: self.min_heap_size = float(min_heap_size) @@ -295,11 +336,19 @@ # initialize the threshold self.min_heap_size = max(self.min_heap_size, self.nursery_size * self.major_collection_threshold) + self.next_major_collection_threshold = self.min_heap_size self.set_major_threshold_from(0.0) debug_stop("gc-set-nursery-size") - def set_major_threshold_from(self, threshold): + + def set_major_threshold_from(self, threshold, reserving_size=0): # Set the next_major_collection_threshold. + threshold_max = (self.next_major_collection_threshold * + self.growth_rate_max) + if threshold > threshold_max: + threshold = threshold_max + # + threshold += reserving_size if threshold < self.min_heap_size: threshold = self.min_heap_size # @@ -1016,7 +1065,7 @@ obj = oldlist.pop() # # Add the flag GCFLAG_NO_YOUNG_PTRS. All live objects should have - # this flag after a nursery collection. + # this flag set after a nursery collection. self.header(obj).tid |= GCFLAG_NO_YOUNG_PTRS # # Trace the 'obj' to replace pointers to nursery with pointers @@ -1079,7 +1128,7 @@ # nursery are kept unchanged in this step. llmemory.raw_memcopy(obj - size_gc_header, newhdr, totalsize) # - # Set the old object's tid to -1 (containing all flags) and + # Set the old object's tid to -42 (containing all flags) and # replace the old object's content with the target address. # A bit of no-ops to convince llarena that we are changing # the layout, in non-translated versions. @@ -1198,8 +1247,8 @@ # have allocated 'major_collection_threshold' times more than # we currently have. bounded = self.set_major_threshold_from( - (self.get_total_memory_used() * self.major_collection_threshold) - + reserving_size) + self.get_total_memory_used() * self.major_collection_threshold, + reserving_size) # # Max heap size: gives an upper bound on the threshold. If we # already have at least this much allocated, raise MemoryError. @@ -1344,7 +1393,7 @@ if self.is_valid_gc_object(obj): if self.is_in_nursery(obj): # - # The object not a tagged pointer, and is it still in the + # The object is not a tagged pointer, and it is still in the # nursery. Find or allocate a "shadow" object, which is # where the object will be moved by the next minor # collection Modified: pypy/release/1.4.x/pypy/rpython/memory/gc/test/test_minimark.py ============================================================================== --- pypy/release/1.4.x/pypy/rpython/memory/gc/test/test_minimark.py (original) +++ pypy/release/1.4.x/pypy/rpython/memory/gc/test/test_minimark.py Mon Nov 15 20:25:15 2010 @@ -28,3 +28,42 @@ assert gc.card_marking_bytes_for_length(P+P+1) == 3 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P) == 8 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P+1) == 9 + +def test_set_major_threshold(): + gc = MiniMarkGC(None, major_collection_threshold=2.0, + growth_rate_max=1.5) + gc.min_heap_size = 100.0 + gc.max_heap_size = 300.0 + gc.next_major_collection_threshold = 0.0 + # first, we don't grow past min_heap_size + for i in range(5): + gc.set_major_threshold_from(100.0) + assert gc.next_major_collection_threshold == 100.0 + # then we grow a lot + b = gc.set_major_threshold_from(100 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 150.0 + b = gc.set_major_threshold_from(150 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 225.0 + b = gc.set_major_threshold_from(225 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 # max reached + b = gc.set_major_threshold_from(300 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 + # then we shrink instantly + b = gc.set_major_threshold_from(100.0) + assert b is False + assert gc.next_major_collection_threshold == 100.0 + # then we grow a bit + b = gc.set_major_threshold_from(100 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 125.0 + b = gc.set_major_threshold_from(125 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 156.25 + # check that we cannot shrink below min_heap_size + b = gc.set_major_threshold_from(42.7) + assert b is False + assert gc.next_major_collection_threshold == 100.0 Modified: pypy/release/1.4.x/pypy/rpython/memory/gctransform/framework.py ============================================================================== --- pypy/release/1.4.x/pypy/rpython/memory/gctransform/framework.py (original) +++ pypy/release/1.4.x/pypy/rpython/memory/gctransform/framework.py Mon Nov 15 20:25:15 2010 @@ -172,6 +172,7 @@ gcdata.static_root_nongcend = a_random_address # patched in finish() gcdata.static_root_end = a_random_address # patched in finish() gcdata.max_type_id = 13 # patched in finish() + gcdata.typeids_z = a_random_address # patched in finish() self.gcdata = gcdata self.malloc_fnptr_cache = {} @@ -212,6 +213,9 @@ data_classdef.generalize_attr( 'max_type_id', annmodel.SomeInteger()) + data_classdef.generalize_attr( + 'typeids_z', + annmodel.SomeAddress()) annhelper = annlowlevel.MixLevelHelperAnnotator(self.translator.rtyper) @@ -415,6 +419,11 @@ [s_gc, annmodel.SomeInteger()], annmodel.s_Bool, minimal_transform=False) + self.get_typeids_z_ptr = getfn(inspector.get_typeids_z, + [s_gc], + annmodel.SomePtr( + lltype.Ptr(rgc.ARRAY_OF_CHAR)), + minimal_transform=False) self.set_max_heap_size_ptr = getfn(GCClass.set_max_heap_size.im_func, [s_gc, @@ -572,7 +581,14 @@ newgcdependencies = [] newgcdependencies.append(ll_static_roots_inside) ll_instance.inst_max_type_id = len(group.members) - self.write_typeid_list() + typeids_z = self.write_typeid_list() + ll_typeids_z = lltype.malloc(rgc.ARRAY_OF_CHAR, + len(typeids_z), + immortal=True) + for i in range(len(typeids_z)): + ll_typeids_z[i] = typeids_z[i] + ll_instance.inst_typeids_z = llmemory.cast_ptr_to_adr(ll_typeids_z) + newgcdependencies.append(ll_typeids_z) return newgcdependencies def get_finish_tables(self): @@ -599,6 +615,11 @@ for index in range(len(self.layoutbuilder.type_info_group.members)): f.write("member%-4d %s\n" % (index, all_ids.get(index, '?'))) f.close() + try: + import zlib + return zlib.compress(udir.join("typeids.txt").read(), 9) + except ImportError: + return '' def transform_graph(self, graph): func = getattr(graph, 'func', None) @@ -988,6 +1009,13 @@ resultvar=hop.spaceop.result) self.pop_roots(hop, livevars) + def gct_gc_typeids_z(self, hop): + livevars = self.push_roots(hop) + hop.genop("direct_call", + [self.get_typeids_z_ptr, self.c_const_gc], + resultvar=hop.spaceop.result) + self.pop_roots(hop, livevars) + def gct_malloc_nonmovable_varsize(self, hop): TYPE = hop.spaceop.result.concretetype if self.gcdata.gc.can_malloc_nonmovable(): Modified: pypy/release/1.4.x/pypy/translator/c/node.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/c/node.py (original) +++ pypy/release/1.4.x/pypy/translator/c/node.py Mon Nov 15 20:25:15 2010 @@ -714,7 +714,11 @@ s = ''.join([self.obj.getitem(i) for i in range(len(self.obj.items))]) else: s = ''.join(self.obj.items) - yield '\t%s%s' % (length, c_char_array_constant(s)) + array_constant = c_char_array_constant(s) + if array_constant.startswith('{') and barebonearray(T): + assert array_constant.endswith('}') + array_constant = array_constant[1:-1].strip() + yield '\t%s%s' % (length, array_constant) yield '}' else: barebone = barebonearray(T) Modified: pypy/release/1.4.x/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/release/1.4.x/pypy/translator/c/test/test_lltyped.py Mon Nov 15 20:25:15 2010 @@ -1,6 +1,7 @@ import py from pypy.rpython.lltypesystem.lltype import * from pypy.translator.c.test import test_typed +from pypy.tool.sourcetools import func_with_new_name class TestLowLevelType(test_typed.CompilationTestCase): @@ -655,6 +656,45 @@ fn = self.getcompiled(llf) fn() + def test_prebuilt_raw_arrays(self): + from pypy.rpython.lltypesystem import rffi, ll2ctypes + # + def make_test_function(cast, haslength, length): + a = malloc(A, length, flavor='raw', immortal=True) + # two cases: a zero-terminated array if length == 6 or 1030, + # a non-zero-terminated array if length == 557 or 1031 + for i in range(length): + a[i] = cast(256 - 5 + i) + def llf(): + for i in range(length): + if a[i] != cast(256 - 5 + i): + return False + if haslength and len(a) != length: + return False + return True + return func_with_new_name(llf, repr((A, haslength, length))) + # + testfns = [] + records = [] + for OF, cast in [(Void, lambda n: None), + (Char, lambda n: chr(n & 0xFF)), + (Signed, lambda n: n)]: + for A, haslength in [(rffi.CArray(OF), False), + (Array(OF), True)]: + for length in [0, 6, 557, 1030, 1031]: + testfns.append(make_test_function(cast, haslength, length)) + records.append((A, haslength, length)) + def llf(): + i = 0 + for fn in testfns: + if not fn(): + return i # returns the index of the failing function + i += 1 + return -42 + fn = self.getcompiled(llf) + res = fn() + assert res == -42, "failing function: %r" % (records[res],) + def test_prebuilt_ll2ctypes_array(self): from pypy.rpython.lltypesystem import rffi, ll2ctypes A = rffi.CArray(Char) Modified: pypy/release/1.4.x/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/c/test/test_newgc.py (original) +++ pypy/release/1.4.x/pypy/translator/c/test/test_newgc.py Mon Nov 15 20:25:15 2010 @@ -1089,6 +1089,41 @@ assert os.path.exists(self.filename_dump) assert os.path.getsize(self.filename_dump) > 0 # minimal test + filename_dump_typeids_z = str(udir.join('test_typeids_z')) + def define_write_typeids_z(self): + U = lltype.GcForwardReference() + U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)), + ('x', lltype.Signed))) + S = lltype.GcStruct('S', ('u', lltype.Ptr(U))) + A = lltype.GcArray(lltype.Ptr(S)) + filename = self.filename_dump_typeids_z + + def fn(): + s = lltype.malloc(S) + s.u = lltype.malloc(U) + s.u.next = lltype.malloc(U) + s.u.next.next = lltype.malloc(U) + a = lltype.malloc(A, 1000) + s2 = lltype.malloc(S) + # + p = rgc.get_typeids_z() + s = ''.join([p[i] for i in range(len(p))]) + fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666) + os.write(fd, s) + os.close(fd) + return 0 + + return fn + + def test_write_typeids_z(self): + self.run("write_typeids_z") + f = open(self.filename_dump_typeids_z) + data_z = f.read() + f.close() + import zlib + data = zlib.decompress(data_z) + assert data.startswith('member0') + assert 'GcArray of * GcStruct S {' in data class TestSemiSpaceGC(TestUsingFramework, snippet.SemiSpaceGCTestDefines): gcpolicy = "semispace" From fijal at codespeak.net Mon Nov 15 20:27:05 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 15 Nov 2010 20:27:05 +0100 (CET) Subject: [pypy-svn] r79132 - pypy/trunk/pypy/doc Message-ID: <20101115192705.47E47282BD6@codespeak.net> Author: fijal Date: Mon Nov 15 20:27:02 2010 New Revision: 79132 Modified: pypy/trunk/pypy/doc/release-1.4.0.txt Log: Write down some points Modified: pypy/trunk/pypy/doc/release-1.4.0.txt ============================================================================== --- pypy/trunk/pypy/doc/release-1.4.0.txt (original) +++ pypy/trunk/pypy/doc/release-1.4.0.txt Mon Nov 15 20:27:02 2010 @@ -17,10 +17,12 @@ More highlights =============== -Virtualenv support: now PyPy is fully compatible with virtualenv_: note that -to use it, you need a recent version of virtualenv (>= 1.5). +* Virtualenv support: now PyPy is fully compatible with virtualenv_: note that + to use it, you need a recent version of virtualenv (>= 1.5). -XXX write me: better regular expressions +* Faster (and JITted) regular expressions - huge boost in speeding up + sre module. +* Faster (and JITted) calls to functions like map(). .. _virtualenv: http://pypi.python.org/pypi/virtualenv From agaynor at codespeak.net Mon Nov 15 20:30:26 2010 From: agaynor at codespeak.net (agaynor at codespeak.net) Date: Mon, 15 Nov 2010 20:30:26 +0100 (CET) Subject: [pypy-svn] r79133 - pypy/trunk/pypy/doc Message-ID: <20101115193026.1859050810@codespeak.net> Author: agaynor Date: Mon Nov 15 20:30:25 2010 New Revision: 79133 Modified: pypy/trunk/pypy/doc/release-1.4.0.txt Log: Grammar fix. Modified: pypy/trunk/pypy/doc/release-1.4.0.txt ============================================================================== --- pypy/trunk/pypy/doc/release-1.4.0.txt (original) +++ pypy/trunk/pypy/doc/release-1.4.0.txt Mon Nov 15 20:30:25 2010 @@ -10,8 +10,8 @@ own development. Among other features, this release includes numerous performance improvements -(which made fast self-hosting possible), 64-bit JIT backend, as well as serious -stabilization. As of now, we can consider 32-bit version of PyPy stable enough +(which made fast self-hosting possible), a 64-bit JIT backend, as well as serious +stabilization. As of now, we can consider the 32-bit version of PyPy stable enough to run in production. More highlights From afa at codespeak.net Mon Nov 15 22:35:00 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 22:35:00 +0100 (CET) Subject: [pypy-svn] r79134 - in pypy/trunk: lib-python/modified-2.5.2/distutils pypy/config pypy/doc/config pypy/module/imp pypy/module/imp/test Message-ID: <20101115213500.50A89282B90@codespeak.net> Author: afa Date: Mon Nov 15 22:34:55 2010 New Revision: 79134 Added: pypy/trunk/pypy/doc/config/objspace.soabi.txt (contents, props changed) Modified: pypy/trunk/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py pypy/trunk/pypy/config/pypyoption.py pypy/trunk/pypy/module/imp/importing.py pypy/trunk/pypy/module/imp/interp_imp.py pypy/trunk/pypy/module/imp/test/test_app.py pypy/trunk/pypy/module/imp/test/test_import.py Log: ABI version tagged .so files for pypy By default, 'import foo' will search extension modules named foo.pypy-14.so This is an implementation of PEP 3149, with two differences: - the default 'foo.so' is not considered - the feature is available on Windows ('foo.pypy-14.pyd') The tag name can be specified at transation time, with the option --soabi: --soabi=pypy-14 is the default. It it not recommmended to set: --soabi= this will disable the feature, allow the pypy interpreter to import 'foo.so', and probably crash when this file has been compiled for another interpreter... Modified: pypy/trunk/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py ============================================================================== --- pypy/trunk/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py (original) +++ pypy/trunk/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py Mon Nov 15 22:34:55 2010 @@ -3,6 +3,7 @@ import sys import os +import imp from distutils.errors import DistutilsPlatformError @@ -47,11 +48,17 @@ _config_vars = None +def _get_so_extension(): + for ext, mod, typ in imp.get_suffixes(): + if typ == imp.C_EXTENSION: + return ext + def _init_posix(): """Initialize the module as appropriate for POSIX systems.""" g = {} g['EXE'] = "" - g['SO'] = ".so" + g['SO'] = _get_so_extension() or ".so" + g['SOABI'] = g['SO'].rsplit('.')[0] global _config_vars _config_vars = g @@ -61,7 +68,8 @@ """Initialize the module as appropriate for NT""" g = {} g['EXE'] = ".exe" - g['SO'] = ".pyd" + g['SO'] = _get_so_extension() or ".pyd" + g['SOABI'] = g['SO'].rsplit('.')[0] global _config_vars _config_vars = g Modified: pypy/trunk/pypy/config/pypyoption.py ============================================================================== --- pypy/trunk/pypy/config/pypyoption.py (original) +++ pypy/trunk/pypy/config/pypyoption.py Mon Nov 15 22:34:55 2010 @@ -175,6 +175,11 @@ default=False, requires=[("objspace.usepycfiles", True)]), + StrOption("soabi", + "Tag to differentiate extension modules built for different Python interpreters", + cmdline="--soabi", + default=None), + BoolOption("honor__builtins__", "Honor the __builtins__ key of a module dictionary", default=False), Added: pypy/trunk/pypy/doc/config/objspace.soabi.txt ============================================================================== --- (empty file) +++ pypy/trunk/pypy/doc/config/objspace.soabi.txt Mon Nov 15 22:34:55 2010 @@ -0,0 +1,14 @@ +This option controls the tag included into extension module file names. The +default is something like `pypy-14`, which means that `import foo` will look for +a file named `foo.pypy-14.so` (or `foo.pypy-14.pyd` on Windows). + +This is an implementation of PEP3149_, with two differences: + + * the filename without tag `foo.so` is not considered. + * the feature is also available on Windows. + +When set to the empty string (with `--soabi=`), the interpreter will only look +for a file named `foo.so`, and will crash if this file was compiled for another +Python interpreter. + +.. _PEP3149: http://www.python.org/dev/peps/pep-3149/ Modified: pypy/trunk/pypy/module/imp/importing.py ============================================================================== --- pypy/trunk/pypy/module/imp/importing.py (original) +++ pypy/trunk/pypy/module/imp/importing.py Mon Nov 15 22:34:55 2010 @@ -12,7 +12,7 @@ from pypy.rlib import streamio, jit from pypy.rlib.streamio import StreamErrors from pypy.rlib.rarithmetic import intmask -from pypy.rlib.objectmodel import we_are_translated +from pypy.rlib.objectmodel import we_are_translated, specialize SEARCH_ERROR = 0 PY_SOURCE = 1 @@ -25,10 +25,22 @@ # PY_CODERESOURCE = 8 IMP_HOOK = 9 -if sys.platform.startswith('win'): - so_extension = ".pyd" +if sys.platform == 'win32': + SO = ".pyd" else: - so_extension = ".so" + SO = ".so" +DEFAULT_SOABI = 'pypy-14' + + at specialize.memo() +def get_so_extension(space): + if space.config.objspace.soabi is not None: + soabi = space.config.objspace.soabi + else: + soabi = DEFAULT_SOABI + if soabi: + return '.' + soabi + SO + else: + return SO def find_modtype(space, filepart): """Check which kind of module to import for the given filepart, @@ -53,6 +65,7 @@ return PY_COMPILED, ".pyc", "rb" if space.config.objspace.usemodules.cpyext: + so_extension = get_so_extension(space) pydfile = filepart + so_extension if os.path.exists(pydfile) and case_ok(pydfile): return C_EXTENSION, so_extension, "rb" Modified: pypy/trunk/pypy/module/imp/interp_imp.py ============================================================================== --- pypy/trunk/pypy/module/imp/interp_imp.py (original) +++ pypy/trunk/pypy/module/imp/interp_imp.py Mon Nov 15 22:34:55 2010 @@ -8,10 +8,16 @@ def get_suffixes(space): w = space.wrap - return space.newlist([ + suffixes_w = [] + if space.config.objspace.usemodules.cpyext: + suffixes_w.append( + space.newtuple([w(importing.get_so_extension(space)), + w('rb'), w(importing.C_EXTENSION)])) + suffixes_w.extend([ space.newtuple([w('.py'), w('U'), w(importing.PY_SOURCE)]), space.newtuple([w('.pyc'), w('rb'), w(importing.PY_COMPILED)]), ]) + return space.newlist(suffixes_w) def get_magic(space): x = importing.get_pyc_magic(space) Modified: pypy/trunk/pypy/module/imp/test/test_app.py ============================================================================== --- pypy/trunk/pypy/module/imp/test/test_app.py (original) +++ pypy/trunk/pypy/module/imp/test/test_app.py Mon Nov 15 22:34:55 2010 @@ -47,6 +47,9 @@ elif mode == self.imp.PY_COMPILED: assert suffix in ('.pyc', '.pyo') assert type == 'rb' + elif mode == self.imp.C_EXTENSION: + assert suffix.endswith(('.pyd', '.so')) + assert type == 'rb' def test_obscure_functions(self): Modified: pypy/trunk/pypy/module/imp/test/test_import.py ============================================================================== --- pypy/trunk/pypy/module/imp/test/test_import.py (original) +++ pypy/trunk/pypy/module/imp/test/test_import.py Mon Nov 15 22:34:55 2010 @@ -473,6 +473,17 @@ except ImportError: pass +class TestAbi: + def test_abi_tag(self): + space1 = gettestobjspace(soabi='TEST') + space2 = gettestobjspace(soabi='') + if sys.platform == 'win32': + assert importing.get_so_extension(space1) == '.TEST.pyd' + assert importing.get_so_extension(space2) == '.pyd' + else: + assert importing.get_so_extension(space1) == '.TEST.so' + assert importing.get_so_extension(space2) == '.so' + def _getlong(data): x = marshal.dumps(data) return x[-4:] From afa at codespeak.net Mon Nov 15 22:40:53 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 22:40:53 +0100 (CET) Subject: [pypy-svn] r79135 - pypy/trunk/pypy/module/cpyext/test Message-ID: <20101115214053.9BEC6282B90@codespeak.net> Author: afa Date: Mon Nov 15 22:40:51 2010 New Revision: 79135 Modified: pypy/trunk/pypy/module/cpyext/test/test_cpyext.py Log: Fix tests in cpyext module Modified: pypy/trunk/pypy/module/cpyext/test/test_cpyext.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/test/test_cpyext.py (original) +++ pypy/trunk/pypy/module/cpyext/test/test_cpyext.py Mon Nov 15 22:40:51 2010 @@ -42,7 +42,7 @@ raises(ImportError, cpyext.load_module, "missing.file", "foo") raises(ImportError, cpyext.load_module, self.libc, "invalid.function") -def compile_module(modname, **kwds): +def compile_module(space, modname, **kwds): """ Build an extension module and return the filename of the resulting native code file. @@ -65,10 +65,8 @@ [], eci, outputfilename=str(dirname/modname), standalone=False) - if sys.platform == 'win32': - pydname = soname.new(purebasename=modname, ext='.pyd') - else: - pydname = soname.new(purebasename=modname, ext='.so') + from pypy.module.imp.importing import get_so_extension + pydname = soname.new(purebasename=modname, ext=get_so_extension(space)) soname.rename(pydname) return str(pydname) @@ -153,7 +151,7 @@ kwds["link_files"] = [str(api_library + '.so')] if sys.platform == 'linux2': kwds["compile_extra"]=["-Werror=implicit-function-declaration"] - return compile_module(name, **kwds) + return compile_module(self.space, name, **kwds) def import_module(self, name, init=None, body='', load_it=True, filename=None): From afa at codespeak.net Mon Nov 15 22:48:25 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 15 Nov 2010 22:48:25 +0100 (CET) Subject: [pypy-svn] r79136 - pypy/trunk/pypy/module/cpyext Message-ID: <20101115214825.3BE5D50810@codespeak.net> Author: afa Date: Mon Nov 15 22:48:23 2010 New Revision: 79136 Modified: pypy/trunk/pypy/module/cpyext/presetup.py Log: Fix "presetup" compilation of an external module to be imported by bin/py.py Modified: pypy/trunk/pypy/module/cpyext/presetup.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/presetup.py (original) +++ pypy/trunk/pypy/module/cpyext/presetup.py Mon Nov 15 22:48:23 2010 @@ -21,6 +21,8 @@ from pypy.conftest import gettestobjspace from pypy.module.cpyext.api import build_bridge +from pypy.module.imp.importing import get_so_extension + usemodules = ['cpyext', 'thread'] if sys.platform == 'win32': usemodules.append('_winreg') # necessary in distutils @@ -35,6 +37,7 @@ def patch_distutils(): sysconfig.get_python_inc = get_python_inc + sysconfig.get_config_vars()['SO'] = get_so_extension(space) patch_distutils() From wlav at codespeak.net Tue Nov 16 00:48:09 2010 From: wlav at codespeak.net (wlav at codespeak.net) Date: Tue, 16 Nov 2010 00:48:09 +0100 (CET) Subject: [pypy-svn] r79137 - in pypy/branch/reflex-support/pypy/module/cppyy: . test Message-ID: <20101115234809.35C44282B90@codespeak.net> Author: wlav Date: Tue Nov 16 00:48:06 2010 New Revision: 79137 Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py pypy/branch/reflex-support/pypy/module/cppyy/executor.py pypy/branch/reflex-support/pypy/module/cppyy/helper.py pypy/branch/reflex-support/pypy/module/cppyy/test/test_helper.py Log: mods to be rpython compliant Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/converter.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/converter.py Tue Nov 16 00:48:06 2010 @@ -10,13 +10,16 @@ _converters = {} class TypeConverter(object): + def __init__(self, space, extra=None): + pass + def _get_fieldptr(self, space, w_obj, offset): obj = space.interpclass_w(space.findattr(w_obj, space.wrap("_cppinstance"))) return lltype.direct_ptradd(obj.rawobject, offset) def _is_abstract(self): raise NotImplementedError( - "abstract base class (actual: %s)" % type(self).__name__) + "abstract base class" ) # more detailed part is not rpython: (actual: %s)" % type(self).__name__) def convert_argument(self, space, w_obj): self._is_abstract() @@ -170,7 +173,7 @@ class ShortPtrConverter(TypeConverter): _immutable_ = True - def __init__(self, detail=None): + def __init__(self, space, detail=None): if detail is None: import sys detail = sys.maxint @@ -203,7 +206,7 @@ class LongPtrConverter(TypeConverter): _immutable_ = True - def __init__(self, detail=None): + def __init__(self, space, detail=None): if detail is None: import sys detail = sys.maxint @@ -270,18 +273,19 @@ # 1) full, exact match try: - return _converters[name]() - except KeyError: + return _converters[name](space) + except KeyError, k: pass # 2) match of decorated, unqualified type - compound, detail = helper.compound(name) + compound = helper.compound(name) clean_name = helper.clean_type(name) try: - if detail: - return _converters[clean_name+compound](detail) - return _converters[clean_name+compound]() - except KeyError: + array_index = helper.array_index(name) + if array_index: + return _converters[clean_name+compound](space, array_index) + return _converters[clean_name+compound](space) + except KeyError, k: pass cpptype = interp_cppyy.type_byname(space, clean_name) Modified: pypy/branch/reflex-support/pypy/module/cppyy/executor.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/executor.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/executor.py Tue Nov 16 00:48:06 2010 @@ -75,7 +75,7 @@ except KeyError: pass - compound, detail = helper.compound(name) + compound = helper.compound(name) cpptype = interp_cppyy.type_byname(space, helper.clean_type(name)) if compound == "*": return InstancePtrExecutor(space, cpptype) Modified: pypy/branch/reflex-support/pypy/module/cppyy/helper.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/helper.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/helper.py Tue Nov 16 00:48:06 2010 @@ -2,13 +2,20 @@ def compound(name): name = "".join(rstring.split(name, "const")) # poor man's replace + if name[-1] == "]": # array type + return "[]" i = _find_qualifier_index(name) + return "".join(name[i:].split(" ")) + +def array_index(name): + name = "".join(rstring.split(name, "const")) # poor man's replace if name[-1] == "]": # array type + i = _find_qualifier_index(name) for i in range(len(name) - 1, -1, -1): c = name[i] if c == "[": - return "[]", int(name[i+1:-1]) - return "".join(name[i:].split(" ")), None + return int(name[i+1:-1]) + return 0 def _find_qualifier_index(name): i = len(name) Modified: pypy/branch/reflex-support/pypy/module/cppyy/test/test_helper.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/test/test_helper.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/test/test_helper.py Tue Nov 16 00:48:06 2010 @@ -1,10 +1,11 @@ from pypy.module.cppyy import helper def test_compound(): - assert helper.compound("int*") == ("*", None) - assert helper.compound("int* const *&") == ("**&", None) - assert helper.compound("std::vector*") == ("*", None) - assert helper.compound("unsigned long int[5]") == ("[]", 5) + assert helper.compound("int*") == "*" + assert helper.compound("int* const *&") == "**&" + assert helper.compound("std::vector*") == "*" + assert helper.compound("unsigned long int[5]") == "[]" + assert helper.array_index("unsigned long int[5]") == 5 def test_clean_type(): From afa at codespeak.net Tue Nov 16 02:17:36 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 16 Nov 2010 02:17:36 +0100 (CET) Subject: [pypy-svn] r79138 - in pypy/trunk/pypy/module/imp: . test Message-ID: <20101116011736.EB16C282B90@codespeak.net> Author: afa Date: Tue Nov 16 02:17:34 2010 New Revision: 79138 Modified: pypy/trunk/pypy/module/imp/importing.py pypy/trunk/pypy/module/imp/test/test_import.py Log: When running an interpreter pypy (bin/py.py), add 'i' to the soabi tag. presetup.py will use it Modified: pypy/trunk/pypy/module/imp/importing.py ============================================================================== --- pypy/trunk/pypy/module/imp/importing.py (original) +++ pypy/trunk/pypy/module/imp/importing.py Tue Nov 16 02:17:34 2010 @@ -37,11 +37,15 @@ soabi = space.config.objspace.soabi else: soabi = DEFAULT_SOABI - if soabi: - return '.' + soabi + SO - else: + + if not soabi: return SO + if not space.config.translating: + soabi += 'i' + + return '.' + soabi + SO + def find_modtype(space, filepart): """Check which kind of module to import for the given filepart, which is a path without extension. Returns PY_SOURCE, PY_COMPILED or Modified: pypy/trunk/pypy/module/imp/test/test_import.py ============================================================================== --- pypy/trunk/pypy/module/imp/test/test_import.py (original) +++ pypy/trunk/pypy/module/imp/test/test_import.py Tue Nov 16 02:17:34 2010 @@ -478,10 +478,10 @@ space1 = gettestobjspace(soabi='TEST') space2 = gettestobjspace(soabi='') if sys.platform == 'win32': - assert importing.get_so_extension(space1) == '.TEST.pyd' + assert importing.get_so_extension(space1) == '.TESTi.pyd' assert importing.get_so_extension(space2) == '.pyd' else: - assert importing.get_so_extension(space1) == '.TEST.so' + assert importing.get_so_extension(space1) == '.TESTi.so' assert importing.get_so_extension(space2) == '.so' def _getlong(data): From fijal at codespeak.net Tue Nov 16 10:31:40 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 16 Nov 2010 10:31:40 +0100 (CET) Subject: [pypy-svn] r79139 - in pypy/release/1.4.x: . lib-python/modified-2.5.2/distutils pypy pypy/config pypy/doc/config pypy/jit/metainterp/optimizeopt pypy/module/array/benchmark pypy/module/array/test pypy/module/cpyext pypy/module/cpyext/test pypy/module/imp pypy/module/imp/test pypy/rlib pypy/rlib/test Message-ID: <20101116093140.37D8750810@codespeak.net> Author: fijal Date: Tue Nov 16 10:31:37 2010 New Revision: 79139 Added: pypy/release/1.4.x/pypy/doc/config/objspace.soabi.txt - copied unchanged from r79138, pypy/trunk/pypy/doc/config/objspace.soabi.txt Modified: pypy/release/1.4.x/ (props changed) pypy/release/1.4.x/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py pypy/release/1.4.x/pypy/ (props changed) pypy/release/1.4.x/pypy/config/pypyoption.py pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/Makefile (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimg.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/loop.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sum.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.py (props changed) pypy/release/1.4.x/pypy/module/array/test/test_array_old.py (props changed) pypy/release/1.4.x/pypy/module/cpyext/presetup.py pypy/release/1.4.x/pypy/module/cpyext/test/test_cpyext.py pypy/release/1.4.x/pypy/module/imp/importing.py pypy/release/1.4.x/pypy/module/imp/interp_imp.py pypy/release/1.4.x/pypy/module/imp/test/test_app.py pypy/release/1.4.x/pypy/module/imp/test/test_import.py pypy/release/1.4.x/pypy/rlib/rerased.py (props changed) pypy/release/1.4.x/pypy/rlib/test/test_rerased.py (props changed) Log: merge PEP 3149 work into release Modified: pypy/release/1.4.x/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py ============================================================================== --- pypy/release/1.4.x/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py (original) +++ pypy/release/1.4.x/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py Tue Nov 16 10:31:37 2010 @@ -3,6 +3,7 @@ import sys import os +import imp from distutils.errors import DistutilsPlatformError @@ -47,11 +48,17 @@ _config_vars = None +def _get_so_extension(): + for ext, mod, typ in imp.get_suffixes(): + if typ == imp.C_EXTENSION: + return ext + def _init_posix(): """Initialize the module as appropriate for POSIX systems.""" g = {} g['EXE'] = "" - g['SO'] = ".so" + g['SO'] = _get_so_extension() or ".so" + g['SOABI'] = g['SO'].rsplit('.')[0] global _config_vars _config_vars = g @@ -61,7 +68,8 @@ """Initialize the module as appropriate for NT""" g = {} g['EXE'] = ".exe" - g['SO'] = ".pyd" + g['SO'] = _get_so_extension() or ".pyd" + g['SOABI'] = g['SO'].rsplit('.')[0] global _config_vars _config_vars = g Modified: pypy/release/1.4.x/pypy/config/pypyoption.py ============================================================================== --- pypy/release/1.4.x/pypy/config/pypyoption.py (original) +++ pypy/release/1.4.x/pypy/config/pypyoption.py Tue Nov 16 10:31:37 2010 @@ -164,6 +164,11 @@ default=False, requires=[("objspace.usepycfiles", True)]), + StrOption("soabi", + "Tag to differentiate extension modules built for different Python interpreters", + cmdline="--soabi", + default=None), + BoolOption("honor__builtins__", "Honor the __builtins__ key of a module dictionary", default=False), Modified: pypy/release/1.4.x/pypy/module/cpyext/presetup.py ============================================================================== --- pypy/release/1.4.x/pypy/module/cpyext/presetup.py (original) +++ pypy/release/1.4.x/pypy/module/cpyext/presetup.py Tue Nov 16 10:31:37 2010 @@ -21,6 +21,8 @@ from pypy.conftest import gettestobjspace from pypy.module.cpyext.api import build_bridge +from pypy.module.imp.importing import get_so_extension + usemodules = ['cpyext', 'thread'] if sys.platform == 'win32': usemodules.append('_winreg') # necessary in distutils @@ -35,6 +37,7 @@ def patch_distutils(): sysconfig.get_python_inc = get_python_inc + sysconfig.get_config_vars()['SO'] = get_so_extension(space) patch_distutils() Modified: pypy/release/1.4.x/pypy/module/cpyext/test/test_cpyext.py ============================================================================== --- pypy/release/1.4.x/pypy/module/cpyext/test/test_cpyext.py (original) +++ pypy/release/1.4.x/pypy/module/cpyext/test/test_cpyext.py Tue Nov 16 10:31:37 2010 @@ -42,7 +42,7 @@ raises(ImportError, cpyext.load_module, "missing.file", "foo") raises(ImportError, cpyext.load_module, self.libc, "invalid.function") -def compile_module(modname, **kwds): +def compile_module(space, modname, **kwds): """ Build an extension module and return the filename of the resulting native code file. @@ -65,10 +65,8 @@ [], eci, outputfilename=str(dirname/modname), standalone=False) - if sys.platform == 'win32': - pydname = soname.new(purebasename=modname, ext='.pyd') - else: - pydname = soname.new(purebasename=modname, ext='.so') + from pypy.module.imp.importing import get_so_extension + pydname = soname.new(purebasename=modname, ext=get_so_extension(space)) soname.rename(pydname) return str(pydname) @@ -153,7 +151,7 @@ kwds["link_files"] = [str(api_library + '.so')] if sys.platform == 'linux2': kwds["compile_extra"]=["-Werror=implicit-function-declaration"] - return compile_module(name, **kwds) + return compile_module(self.space, name, **kwds) def import_module(self, name, init=None, body='', load_it=True, filename=None): Modified: pypy/release/1.4.x/pypy/module/imp/importing.py ============================================================================== --- pypy/release/1.4.x/pypy/module/imp/importing.py (original) +++ pypy/release/1.4.x/pypy/module/imp/importing.py Tue Nov 16 10:31:37 2010 @@ -12,7 +12,7 @@ from pypy.rlib import streamio, jit from pypy.rlib.streamio import StreamErrors from pypy.rlib.rarithmetic import intmask -from pypy.rlib.objectmodel import we_are_translated +from pypy.rlib.objectmodel import we_are_translated, specialize SEARCH_ERROR = 0 PY_SOURCE = 1 @@ -25,10 +25,26 @@ # PY_CODERESOURCE = 8 IMP_HOOK = 9 -if sys.platform.startswith('win'): - so_extension = ".pyd" +if sys.platform == 'win32': + SO = ".pyd" else: - so_extension = ".so" + SO = ".so" +DEFAULT_SOABI = 'pypy-14' + + at specialize.memo() +def get_so_extension(space): + if space.config.objspace.soabi is not None: + soabi = space.config.objspace.soabi + else: + soabi = DEFAULT_SOABI + + if not soabi: + return SO + + if not space.config.translating: + soabi += 'i' + + return '.' + soabi + SO def find_modtype(space, filepart): """Check which kind of module to import for the given filepart, @@ -53,6 +69,7 @@ return PY_COMPILED, ".pyc", "rb" if space.config.objspace.usemodules.cpyext: + so_extension = get_so_extension(space) pydfile = filepart + so_extension if os.path.exists(pydfile) and case_ok(pydfile): return C_EXTENSION, so_extension, "rb" Modified: pypy/release/1.4.x/pypy/module/imp/interp_imp.py ============================================================================== --- pypy/release/1.4.x/pypy/module/imp/interp_imp.py (original) +++ pypy/release/1.4.x/pypy/module/imp/interp_imp.py Tue Nov 16 10:31:37 2010 @@ -8,10 +8,16 @@ def get_suffixes(space): w = space.wrap - return space.newlist([ + suffixes_w = [] + if space.config.objspace.usemodules.cpyext: + suffixes_w.append( + space.newtuple([w(importing.get_so_extension(space)), + w('rb'), w(importing.C_EXTENSION)])) + suffixes_w.extend([ space.newtuple([w('.py'), w('U'), w(importing.PY_SOURCE)]), space.newtuple([w('.pyc'), w('rb'), w(importing.PY_COMPILED)]), ]) + return space.newlist(suffixes_w) def get_magic(space): x = importing.get_pyc_magic(space) Modified: pypy/release/1.4.x/pypy/module/imp/test/test_app.py ============================================================================== --- pypy/release/1.4.x/pypy/module/imp/test/test_app.py (original) +++ pypy/release/1.4.x/pypy/module/imp/test/test_app.py Tue Nov 16 10:31:37 2010 @@ -47,6 +47,9 @@ elif mode == self.imp.PY_COMPILED: assert suffix in ('.pyc', '.pyo') assert type == 'rb' + elif mode == self.imp.C_EXTENSION: + assert suffix.endswith(('.pyd', '.so')) + assert type == 'rb' def test_obscure_functions(self): Modified: pypy/release/1.4.x/pypy/module/imp/test/test_import.py ============================================================================== --- pypy/release/1.4.x/pypy/module/imp/test/test_import.py (original) +++ pypy/release/1.4.x/pypy/module/imp/test/test_import.py Tue Nov 16 10:31:37 2010 @@ -473,6 +473,17 @@ except ImportError: pass +class TestAbi: + def test_abi_tag(self): + space1 = gettestobjspace(soabi='TEST') + space2 = gettestobjspace(soabi='') + if sys.platform == 'win32': + assert importing.get_so_extension(space1) == '.TESTi.pyd' + assert importing.get_so_extension(space2) == '.pyd' + else: + assert importing.get_so_extension(space1) == '.TESTi.so' + assert importing.get_so_extension(space2) == '.so' + def _getlong(data): x = marshal.dumps(data) return x[-4:] From afa at codespeak.net Tue Nov 16 12:54:51 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 16 Nov 2010 12:54:51 +0100 (CET) Subject: [pypy-svn] r79141 - pypy/branch/fast-forward/pypy/module/_winreg Message-ID: <20101116115451.6E3F7282B90@codespeak.net> Author: afa Date: Tue Nov 16 12:54:49 2010 New Revision: 79141 Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py Log: Oops, remove debug prints Modified: pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py (original) +++ pypy/branch/fast-forward/pypy/module/_winreg/interp_winreg.py Tue Nov 16 12:54:49 2010 @@ -452,12 +452,10 @@ if ret == rwinreg.ERROR_MORE_DATA: # Resize and retry bufSize *= 2 - print "AFA CONTINUE", bufSize, retDataSize[0] retDataSize[0] = rffi.cast(rwin32.DWORD, bufSize) continue if ret != 0: raiseWindowsError(space, ret, 'RegQueryValueEx') - print "AFA OK", bufSize, retDataSize[0] return space.newtuple([ convert_from_regdata(space, databuf, retDataSize[0], retType[0]), From afa at codespeak.net Tue Nov 16 13:01:40 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 16 Nov 2010 13:01:40 +0100 (CET) Subject: [pypy-svn] r79142 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101116120140.650EC50810@codespeak.net> Author: afa Date: Tue Nov 16 13:01:39 2010 New Revision: 79142 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py Log: BytesIO() is readable, writable, seekable. Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py Tue Nov 16 13:01:39 2010 @@ -120,6 +120,18 @@ self.pos = 0 return space.wrap(self.pos) + @unwrap_spec('self', ObjSpace) + def readable_w(self, space): + return space.w_True + + @unwrap_spec('self', ObjSpace) + def writable_w(self, space): + return space.w_True + + @unwrap_spec('self', ObjSpace) + def seekable_w(self, space): + return space.w_True + W_BytesIO.typedef = TypeDef( 'BytesIO', W_BufferedIOBase.typedef, __new__ = generic_new_descr(W_BytesIO), @@ -130,5 +142,8 @@ getvalue = interp2app(W_BytesIO.getvalue_w), seek = interp2app(W_BytesIO.seek_w), tell = interp2app(W_BytesIO.tell_w), + readable = interp2app(W_BytesIO.readable_w), + writable = interp2app(W_BytesIO.writable_w), + seekable = interp2app(W_BytesIO.seekable_w), ) Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py Tue Nov 16 13:01:39 2010 @@ -4,6 +4,14 @@ def setup_class(cls): cls.space = gettestobjspace(usemodules=['_io']) + def test_capabilities(self): + import _io + f = _io.BytesIO() + assert f.readable() + assert f.writable() + assert f.seekable() + f.close() + def test_write(self): import _io f = _io.BytesIO() From david at codespeak.net Tue Nov 16 13:42:38 2010 From: david at codespeak.net (david at codespeak.net) Date: Tue, 16 Nov 2010 13:42:38 +0100 (CET) Subject: [pypy-svn] r79144 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101116124238.6D15A282B90@codespeak.net> Author: david Date: Tue Nov 16 13:42:36 2010 New Revision: 79144 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: (arigo, david) Fix calls from generated machine code to functions which return values smaller than a word by introducing a hack and masking the values correspondingly Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Tue Nov 16 13:42:36 2010 @@ -16,6 +16,7 @@ from pypy.jit.metainterp.history import ConstInt, BoxInt, BasicFailDescr from pypy.jit.metainterp.resoperation import rop from pypy.rlib import rgc +from pypy.rlib.objectmodel import we_are_translated from pypy.rpython.annlowlevel import llhelper from pypy.rpython.lltypesystem import lltype, rffi, rstr, llmemory @@ -255,8 +256,19 @@ regalloc.before_call(r.all_regs, save_all_regs) else: regalloc.before_call() - regalloc.force_allocate_reg(op.result, selected_reg=r.r0) self.mc.BL(adr) + #XXX Hack, Hack, Hack + if not we_are_translated(): + descr = op.getdescr() + # XXX we need descr.get_result_sign here!!!! + size = descr.get_result_size(False) + # for now just check the size of the value + if size == 1: #unsigned char + self.mc.AND_ri(r.r0.value, r.r0.value, 255) + elif size == 2: # signed short + self.mc.LSL_ri(r.r0.value, r.r0.value, 16) + self.mc.ASR_ri(r.r0.value, r.r0.value, 16) + regalloc.after_call(op.result) # readjust the sp in case we passed some args on the stack if op.numargs() > 5: From david at codespeak.net Tue Nov 16 14:14:18 2010 From: david at codespeak.net (david at codespeak.net) Date: Tue, 16 Nov 2010 14:14:18 +0100 (CET) Subject: [pypy-svn] r79145 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . helper Message-ID: <20101116131418.D152A282BD4@codespeak.net> Author: david Date: Tue Nov 16 14:14:17 2010 New Revision: 79145 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Implement same_as operation Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Tue Nov 16 14:14:17 2010 @@ -199,13 +199,12 @@ epilog_size = 3*WORD def gen_func_epilog(self,cond=c.AL): self.mc.MOV_rr(r.sp.value, r.fp.value) - self.mc.POP([r.r4.value], cond=cond) # Pop value used as forcething + self.mc.ADD_ri(r.sp.value, r.sp.value, WORD) self.mc.POP([reg.value for reg in r.callee_restored_registers], cond=cond) def gen_func_prolog(self): self.mc.PUSH([reg.value for reg in r.callee_saved_registers]) - self.mc.MOV_ri(r.r4.value, 0xCC) - self.mc.PUSH([r.r4.value]) # Push some reg to use as force thing which is restored when popping from stack + self.mc.SUB_ri(r.sp.value, r.sp.value, WORD) self.mc.MOV_rr(r.fp.value, r.sp.value) def gen_bootstrap_code(self, inputargs, regalloc, looptoken): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Tue Nov 16 14:14:17 2010 @@ -50,10 +50,8 @@ arg2 = regalloc.make_sure_var_in_reg(a1, [a0], selected_reg=r.r1, imm_fine=False) assert arg1 == r.r0 assert arg2 == r.r1 - regalloc.before_call() res = regalloc.force_allocate_reg(op.result, selected_reg=r.r0) getattr(self.mc, opname)(fcond) - regalloc.after_call(op.result) regalloc.possibly_free_vars_for_op(op) return fcond return f Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Tue Nov 16 14:14:17 2010 @@ -13,7 +13,7 @@ from pypy.jit.backend.llsupport import symbolic from pypy.jit.backend.llsupport.descr import BaseFieldDescr, BaseArrayDescr from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity, TempBox -from pypy.jit.metainterp.history import ConstInt, BoxInt, BasicFailDescr +from pypy.jit.metainterp.history import Const, ConstInt, BoxInt, BasicFailDescr from pypy.jit.metainterp.resoperation import rop from pypy.rlib import rgc from pypy.rlib.objectmodel import we_are_translated @@ -275,6 +275,17 @@ self._adjust_sp(-n, regalloc, fcond=fcond) regalloc.possibly_free_vars(locs) + def emit_op_same_as(self, op, regalloc, fcond): + resloc = regalloc.force_allocate_reg(op.result) + arg = op.getarg(0) + imm_arg = isinstance(arg, ConstInt) and (arg.getint() <= 0xFF or -1 * arg.getint() <= 0xFF) + argloc = regalloc.make_sure_var_in_reg(arg, imm_fine=imm_arg) + if argloc.is_imm(): + self.mc.MOV_ri(resloc.value, argloc.getint()) + else: + self.mc.MOV_rr(resloc.value, argloc.value) + regalloc.possibly_free_vars_for_op(op) + class FieldOpAssembler(object): _mixin_ = True From arigo at codespeak.net Tue Nov 16 14:14:47 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 16 Nov 2010 14:14:47 +0100 (CET) Subject: [pypy-svn] r79146 - in pypy/branch/jit-free/pypy/jit/metainterp: . test Message-ID: <20101116131447.9B374282BE3@codespeak.net> Author: arigo Date: Tue Nov 16 14:14:45 2010 New Revision: 79146 Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py pypy/branch/jit-free/pypy/jit/metainterp/history.py pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_basic.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Log: (antocuni, arigo) Managed to really really(?) make test_memmgr.py fail. Argh again. Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/compile.py Tue Nov 16 14:14:45 2010 @@ -42,10 +42,11 @@ name = metainterp.staticdata.stats.name_for_new_loop() return TreeLoop(name) -def make_loop_token(cpu, nb_args, jitdriver_sd): +def make_loop_token(cpu, nb_args, jitdriver_sd, greenkey): loop_token = LoopToken(cpu) loop_token.specnodes = [prebuiltNotSpecNode] * nb_args loop_token.outermost_jitdriver_sd = jitdriver_sd + loop_token.outermost_greenkey = greenkey return loop_token # ____________________________________________________________ @@ -64,8 +65,10 @@ loop.operations = [h_ops[i].clone() for i in range(start, len(h_ops))] metainterp_sd = metainterp.staticdata jitdriver_sd = metainterp.jitdriver_sd - loop_token = make_loop_token(metainterp.cpu, len(loop.inputargs), - jitdriver_sd) + loop_token = metainterp.original_loop_token + assert len(loop_token.specnodes) == len(loop.inputargs) + assert loop_token.outermost_jitdriver_sd is jitdriver_sd + assert loop_token.outermost_greenkey == greenkey loop.token = loop_token loop.operations[-1].setdescr(loop_token) # patch the target of the JUMP try: @@ -78,7 +81,9 @@ return old_loop_token send_loop_to_backend(metainterp_sd, loop, "loop") insert_loop_token(old_loop_tokens, loop_token) + # mostly for tests: make sure we don't keep a reference to the LoopToken loop.token = None + metainterp.original_loop_token = None return loop_token def insert_loop_token(old_loop_tokens, loop_token): @@ -210,8 +215,8 @@ } class ResumeDescr(AbstractFailDescr): - def __init__(self, original_greenkey): - self.original_greenkey = original_greenkey + def __init__(self, wref_original_loop_token): + self.wref_original_loop_token = wref_original_loop_token class ResumeGuardDescr(ResumeDescr): _counter = 0 # if < 0, there is one counter per value; @@ -317,7 +322,7 @@ # to the corrsponding guard_op and compile from there inputargs = metainterp.history.inputargs if not we_are_translated(): - pass #self._debug_suboperations = new_loop.operations + self._debug_suboperations = new_loop.operations send_bridge_to_backend(metainterp.staticdata, self, inputargs, new_loop.operations) @@ -331,14 +336,14 @@ res.rd_pendingfields = self.rd_pendingfields def _clone_if_mutable(self): - res = ResumeGuardDescr(self.original_greenkey) + res = ResumeGuardDescr(self.wref_original_loop_token) self.copy_all_attrbutes_into(res) return res class ResumeGuardForcedDescr(ResumeGuardDescr): - def __init__(self, metainterp_sd, original_greenkey, jitdriver_sd): - ResumeGuardDescr.__init__(self, original_greenkey) + def __init__(self, metainterp_sd, wref_original_loop_token, jitdriver_sd): + ResumeGuardDescr.__init__(self, wref_original_loop_token) self.metainterp_sd = metainterp_sd self.jitdriver_sd = jitdriver_sd @@ -406,7 +411,7 @@ def _clone_if_mutable(self): res = ResumeGuardForcedDescr(self.metainterp_sd, - self.original_greenkey, + self.wref_original_loop_token, self.jitdriver_sd) self.copy_all_attrbutes_into(res) return res @@ -473,8 +478,8 @@ class ResumeFromInterpDescr(ResumeDescr): - def __init__(self, original_greenkey, redkey): - ResumeDescr.__init__(self, original_greenkey) + def __init__(self, wref_original_loop_token, redkey): + ResumeDescr.__init__(self, wref_original_loop_token) self.redkey = redkey def compile_and_attach(self, metainterp, new_loop): @@ -485,24 +490,28 @@ metainterp_sd = metainterp.staticdata jitdriver_sd = metainterp.jitdriver_sd metainterp.history.inputargs = self.redkey - new_loop_token = make_loop_token(metainterp.cpu, len(self.redkey), - metainterp.jitdriver_sd) + new_loop_token = metainterp.original_loop_token + # assert that the wref_original_loop_token was kept alive, because + # it's just the same as metainterp.original_loop_token + assert new_loop_token is self.wref_original_loop_token() new_loop.inputargs = self.redkey new_loop.token = new_loop_token send_loop_to_backend(metainterp_sd, new_loop, "entry bridge") # send the new_loop to warmspot.py, to be called directly the next time jitdriver_sd.warmstate.attach_unoptimized_bridge_from_interp( - self.original_greenkey, new_loop_token) # store the new loop in compiled_merge_points_wref too old_loop_tokens = metainterp.get_compiled_merge_points( - self.original_greenkey) + new_loop_token.outermost_greenkey) # it always goes at the end of the list, as it is the most # general loop token old_loop_tokens.append(new_loop_token) - metainterp.set_compiled_merge_points(self.original_greenkey, + metainterp.set_compiled_merge_points(new_loop_token.outermost_greenkey, old_loop_tokens) + # mostly for tests: make sure we don't keep a reference + # to the LoopToken new_loop.token = None + metainterp.original_loop_token = None def reset_counter_from_failure(self): pass @@ -570,7 +579,7 @@ """ # 'redboxes' is only used to know the types of red arguments. inputargs = [box.clonebox() for box in redboxes] - loop_token = make_loop_token(cpu, len(inputargs), jitdriver_sd) + loop_token = make_loop_token(cpu, len(inputargs), jitdriver_sd, greenboxes) # 'nb_red_args' might be smaller than len(redboxes), # because it doesn't include the virtualizable boxes. nb_red_args = jitdriver_sd.num_red_args Modified: pypy/branch/jit-free/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/history.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/history.py Tue Nov 16 14:14:45 2010 @@ -730,13 +730,14 @@ """ terminating = False # see TerminatingLoopToken in compile.py outermost_jitdriver_sd = None + outermost_greenkey = None # specnodes = ... # and more data specified by the backend when the loop is compiled cpu = None - number = 0 + number = -1 def __init__(self, cpu=None): - assert not isinstance(cpu, int) # xxx temporary + assert 'CPU' in type(cpu).__name__ # xxx temporary self.cpu = cpu # See get_fail_descr_number() in backend/model.py: this growing # list gives the 'descr_number' of all fail descrs that belong to @@ -749,7 +750,7 @@ def __del__(self): for i in range(160): print '#', - print + print repr(self), self.cpu if self.cpu is None: return if self.generation > r_longlong(0): Modified: pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py Tue Nov 16 14:14:45 2010 @@ -1,4 +1,4 @@ -import py, os, sys +import py, os, sys, weakref from pypy.rpython.lltypesystem import lltype, llmemory, rclass from pypy.rlib.objectmodel import we_are_translated from pypy.rlib.unroll import unrolling_iterable @@ -1048,13 +1048,14 @@ else: moreargs = list(extraargs) metainterp_sd = metainterp.staticdata - original_greenkey = metainterp.resumekey.original_greenkey + resumekey = metainterp.resumekey + wref_original_loop_token = resumekey.wref_original_loop_token if opnum == rop.GUARD_NOT_FORCED: resumedescr = compile.ResumeGuardForcedDescr(metainterp_sd, - original_greenkey, + wref_original_loop_token, metainterp.jitdriver_sd) else: - resumedescr = compile.ResumeGuardDescr(original_greenkey) + resumedescr = compile.ResumeGuardDescr(wref_original_loop_token) guard_op = metainterp.history.record(opnum, moreargs, None, descr=resumedescr) virtualizable_boxes = None @@ -1642,8 +1643,15 @@ num_green_args = self.jitdriver_sd.num_green_args original_greenkey = original_boxes[:num_green_args] redkey = original_boxes[num_green_args:] - self.resumekey = compile.ResumeFromInterpDescr(original_greenkey, - redkey) + # make a new loop token and store a strong ref to it for as long as + # this MetaInterp is alive (we will give it to the MemoryManager after + # the backend has emitted assembler) + self.original_loop_token = compile.make_loop_token(self.cpu, + len(redkey), + self.jitdriver_sd, + original_greenkey) + self.resumekey = compile.ResumeFromInterpDescr( + weakref.ref(self.original_loop_token), redkey) self.seen_loop_header_for_jdindex = -1 try: self.interpret() @@ -1666,15 +1674,20 @@ debug_stop('jit-tracing') def _handle_guard_failure(self, key): - original_greenkey = key.original_greenkey - # notice that here we just put the greenkey - # use -1 to mark that we will have to give up - # because we cannot reconstruct the beginning of the proper loop - self.current_merge_points = [(original_greenkey, -1)] + # store the original_loop_token on 'self' to make sure that it stays + # alive as long as this MetaInterp self.resumekey = key - self.seen_loop_header_for_jdindex = -1 + self.original_loop_token = key.wref_original_loop_token() try: self.prepare_resume_from_failure(key.guard_opnum) + if self.original_loop_token is None: + raise SwitchToBlackhole(ABORT_BRIDGE) + # notice that here we just put the greenkey + # use -1 to mark that we will have to give up + # because we cannot reconstruct the beginning of the proper loop + original_greenkey = self.original_loop_token.outermost_greenkey + self.current_merge_points = [(original_greenkey, -1)] + self.seen_loop_header_for_jdindex = -1 self.interpret() except GenerateMergePoint, gmp: return self.designate_target_loop(gmp) Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_basic.py Tue Nov 16 14:14:45 2010 @@ -26,7 +26,7 @@ self.__compiled_merge_points = lst class FakeWarmRunnerState: - def attach_unoptimized_bridge_from_interp(self, greenkey, newloop): + def attach_unoptimized_bridge_from_interp(self, newloop): pass def jit_cell_at_key(self, greenkey): Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Tue Nov 16 14:14:45 2010 @@ -79,3 +79,7 @@ # we should see only the loop and the entry bridge self.check_tree_loop_count(2) + + + # we need another test that fails because we store + # self._debug_suboperations in compile.py Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Tue Nov 16 14:14:45 2010 @@ -255,9 +255,8 @@ debug_print("disabled inlining", loc) debug_stop("jit-disableinlining") - def attach_unoptimized_bridge_from_interp(self, greenkey, - entry_loop_token): - cell = self.jit_cell_at_key(greenkey) + def attach_unoptimized_bridge_from_interp(self, entry_loop_token): + cell = self.jit_cell_at_key(entry_loop_token.outermost_greenkey) cell.counter = -1 old_token = cell.get_entry_loop_token() cell.set_entry_loop_token(entry_loop_token) @@ -341,6 +340,7 @@ # ---------- execute assembler ---------- while True: # until interrupted by an exception fail_descr = warmrunnerdesc.execute_token(loop_token) + loop_token = None # for test_memmgr if vinfo is not None: vinfo.reset_vable_token(virtualizable) loop_token = fail_descr.handle_fail(metainterp_sd, From arigo at codespeak.net Tue Nov 16 14:22:21 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 16 Nov 2010 14:22:21 +0100 (CET) Subject: [pypy-svn] r79147 - pypy/branch/jit-free/pypy/jit/metainterp Message-ID: <20101116132221.2F28D282B9C@codespeak.net> Author: arigo Date: Tue Nov 16 14:22:19 2010 New Revision: 79147 Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py Log: (antocuni, arigo) Finally we can fix the test. Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/compile.py Tue Nov 16 14:22:19 2010 @@ -124,6 +124,8 @@ else: loop._ignore_during_counting = True metainterp_sd.log("compiled new " + type) + if metainterp_sd.warmrunnerdesc is not None: # for tests + metainterp_sd.warmrunnerdesc.memory_manager.keep_loop_alive(loop.token) def send_bridge_to_backend(metainterp_sd, faildescr, inputargs, operations): n = metainterp_sd.cpu.get_fail_descr_number(faildescr) From arigo at codespeak.net Tue Nov 16 15:49:37 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 16 Nov 2010 15:49:37 +0100 (CET) Subject: [pypy-svn] r79148 - in pypy/branch/jit-free/pypy/jit: backend backend/llgraph metainterp metainterp/test Message-ID: <20101116144937.4CBBC282B9C@codespeak.net> Author: arigo Date: Tue Nov 16 15:49:35 2010 New Revision: 79148 Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py pypy/branch/jit-free/pypy/jit/backend/model.py pypy/branch/jit-free/pypy/jit/metainterp/compile.py pypy/branch/jit-free/pypy/jit/metainterp/history.py pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Log: (antocuni, arigo) * Make even more sure that loops are freed. * Make the llgraph backend complain as soon as we are trying to run a freed loop. * Write a failing test for the memory manager. The fix is setting the _keepalive_target_looktokens. Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py Tue Nov 16 15:49:35 2010 @@ -161,6 +161,8 @@ # ____________________________________________________________ class CompiledLoop(object): + has_been_freed = False + def __init__(self): self.inputargs = [] self.operations = [] @@ -285,6 +287,11 @@ del _variables[:] return _to_opaque(CompiledLoop()) +def mark_as_free(loop): + loop = _from_opaque(loop) + assert not loop.has_been_freed + loop.has_been_freed = True + def compile_start_int_var(loop): return compile_start_ref_var(loop, lltype.Signed) @@ -429,6 +436,7 @@ verbose = True self.opindex = 0 while True: + assert not self.loop.has_been_freed op = self.loop.operations[self.opindex] args = [self.getenv(v) for v in op.args] if not op.is_final(): @@ -1609,6 +1617,7 @@ setannotation(compile_add_fail, annmodel.SomeInteger()) setannotation(compile_add_fail_arg, annmodel.s_None) setannotation(compile_redirect_fail, annmodel.s_None) +setannotation(mark_as_free, annmodel.s_None) setannotation(new_frame, s_Frame) setannotation(frame_clear, annmodel.s_None) Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py Tue Nov 16 15:49:35 2010 @@ -118,8 +118,10 @@ self._descrs[key] = descr return descr - def compile_bridge(self, faildescr, inputargs, operations, log=True): + def compile_bridge(self, faildescr, inputargs, operations, + original_loop_token, log=True): c = llimpl.compile_start() + original_loop_token._llgraph_loop_and_bridges.append(c) self._compile_loop_or_bridge(c, inputargs, operations) old, oldindex = faildescr._compiled_fail llimpl.compile_redirect_fail(old, oldindex, c) @@ -131,9 +133,15 @@ is not. """ c = llimpl.compile_start() + assert not hasattr(loopdescr, '_llgraph_loop_and_bridges') + loopdescr._llgraph_loop_and_bridges = [c] loopdescr._llgraph_compiled_version = c self._compile_loop_or_bridge(c, inputargs, operations) + def free_loop_and_bridges(self, looptoken): + for c in looptoken._llgraph_loop_and_bridges: + llimpl.mark_as_free(c) + def _compile_loop_or_bridge(self, c, inputargs, operations): var2index = {} for box in inputargs: Modified: pypy/branch/jit-free/pypy/jit/backend/model.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/model.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/model.py Tue Nov 16 15:49:35 2010 @@ -49,7 +49,8 @@ """ raise NotImplementedError - def compile_bridge(self, faildescr, inputargs, operations, log=True): + def compile_bridge(self, faildescr, inputargs, operations, + original_loop_token, log=True): """Assemble the bridge. The FailDescr is the descr of the original guard that failed. """ Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/compile.py Tue Nov 16 15:49:35 2010 @@ -82,6 +82,7 @@ send_loop_to_backend(metainterp_sd, loop, "loop") insert_loop_token(old_loop_tokens, loop_token) # mostly for tests: make sure we don't keep a reference to the LoopToken + metainterp.original_loop_token.record_loop_or_bridge(loop) loop.token = None metainterp.original_loop_token = None return loop_token @@ -127,7 +128,8 @@ if metainterp_sd.warmrunnerdesc is not None: # for tests metainterp_sd.warmrunnerdesc.memory_manager.keep_loop_alive(loop.token) -def send_bridge_to_backend(metainterp_sd, faildescr, inputargs, operations): +def send_bridge_to_backend(metainterp_sd, faildescr, inputargs, operations, + original_loop_token): n = metainterp_sd.cpu.get_fail_descr_number(faildescr) metainterp_sd.logger_ops.log_bridge(inputargs, operations, n) if not we_are_translated(): @@ -136,7 +138,8 @@ metainterp_sd.profiler.start_backend() debug_start("jit-backend") try: - metainterp_sd.cpu.compile_bridge(faildescr, inputargs, operations) + metainterp_sd.cpu.compile_bridge(faildescr, inputargs, operations, + original_loop_token) finally: debug_stop("jit-backend") metainterp_sd.profiler.end_backend() @@ -326,7 +329,8 @@ if not we_are_translated(): self._debug_suboperations = new_loop.operations send_bridge_to_backend(metainterp.staticdata, self, inputargs, - new_loop.operations) + new_loop.operations, + metainterp.original_loop_token) def copy_all_attrbutes_into(self, res): # XXX a bit ugly to have to list them all here @@ -510,10 +514,6 @@ old_loop_tokens.append(new_loop_token) metainterp.set_compiled_merge_points(new_loop_token.outermost_greenkey, old_loop_tokens) - # mostly for tests: make sure we don't keep a reference - # to the LoopToken - new_loop.token = None - metainterp.original_loop_token = None def reset_counter_from_failure(self): pass @@ -548,6 +548,11 @@ # know exactly what we must do (ResumeGuardDescr/ResumeFromInterpDescr) prepare_last_operation(new_loop, target_loop_token) resumekey.compile_and_attach(metainterp, new_loop) + # mostly for tests: make sure we don't keep a reference + # to the LoopToken + metainterp.original_loop_token.record_loop_or_bridge(new_loop) + metainterp.original_loop_token = None + new_loop.token = None return target_loop_token def prepare_last_operation(new_loop, target_loop_token): Modified: pypy/branch/jit-free/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/history.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/history.py Tue Nov 16 15:49:35 2010 @@ -744,9 +744,18 @@ # this loop or to a bridge attached to it. self.faildescr_indices = [] # For memory management of assembled loops - self.contains_jumps_to = {} # set of other LoopTokens + self._keepalive_target_looktokens = {} # set of other LoopTokens self.generation = r_longlong(0) + def record_loop_or_bridge(self, loop): + # Records that the loop starting at the LoopToken 'self' ends up + # with 'loop', which may be either the loop itself or some pseudo- + # loop representing some bridge. + other_loop_token = loop.operations[-1].getdescr() + if isinstance(other_loop_token, LoopToken): + self._keepalive_target_looktokens[other_loop_token] = None + loop.operations[-1].setdescr(None) # clear reference + def __del__(self): for i in range(160): print '#', Modified: pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py Tue Nov 16 15:49:35 2010 @@ -1,6 +1,7 @@ import math from pypy.rlib.rarithmetic import r_longlong from pypy.rlib.debug import debug_start, debug_print, debug_stop +from pypy.rlib.objectmodel import we_are_translated # # Logic to decide which loops are old and not used any more. @@ -71,3 +72,6 @@ debug_print("Loop tokens left: ", newtotal) print self.alive_loops.keys() debug_stop("jit-free-memmgr") + if not we_are_translated(): + from pypy.rlib import rgc + rgc.collect() Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Tue Nov 16 15:49:35 2010 @@ -80,6 +80,37 @@ # we should see only the loop and the entry bridge self.check_tree_loop_count(2) + def test_target_loop_kept_alive(self): + myjitdriver = JitDriver(greens=['m'], reds=['n']) + def g(m): + n = 10 + while n > 0: + myjitdriver.can_enter_jit(n=n, m=m) + myjitdriver.jit_merge_point(n=n, m=m) + n = n - 1 + return 21 + import gc + def f(): + # create the loop and the entry bridge for 'g(5)' + for i in range(8): + g(5) + # create another loop and another entry bridge for 'g(7)', + # to increase the current_generation + for i in range(20): + g(7) + # reuse the existing loop and entry bridge for 'g(5)'. + # the generation of the entry bridge for g(5) should never + # grow too old. The loop itself gets old, but is kept alive + # by the entry bridge via contains_jumps_to. + g(5) + return 42 + + res = self.meta_interp(f, [], loop_longevity=3) + assert res == 42 + + # we should see only the loop and the entry bridge for g(5) and g(7) + self.check_tree_loop_count(4) + # we need another test that fails because we store # self._debug_suboperations in compile.py From arigo at codespeak.net Tue Nov 16 17:03:31 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 16 Nov 2010 17:03:31 +0100 (CET) Subject: [pypy-svn] r79150 - in pypy/branch/jit-free/pypy/jit/metainterp: . test Message-ID: <20101116160331.428EE282B90@codespeak.net> Author: arigo Date: Tue Nov 16 17:03:29 2010 New Revision: 79150 Modified: pypy/branch/jit-free/pypy/jit/metainterp/history.py pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Log: (antocuni, arigo) Adapt a test and write a new one about throwing away old loops. That's the whole point of the branch. Modified: pypy/branch/jit-free/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/history.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/history.py Tue Nov 16 17:03:29 2010 @@ -753,7 +753,11 @@ # loop representing some bridge. other_loop_token = loop.operations[-1].getdescr() if isinstance(other_loop_token, LoopToken): - self._keepalive_target_looktokens[other_loop_token] = None + # the following test is not enough to prevent more complicated + # cases of cycles, but at least it helps in simple tests of + # test_memgr.py + if other_loop_token is not self: + self._keepalive_target_looktokens[other_loop_token] = None loop.operations[-1].setdescr(None) # clear reference def __del__(self): Modified: pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py Tue Nov 16 17:03:29 2010 @@ -63,7 +63,7 @@ print self.alive_loops.keys() debug_print("Current generation:", self.current_generation) debug_print("Loop tokens before:", oldtotal) - max_generation = self.current_generation - self.max_age + max_generation = self.current_generation - (self.max_age-1) for looptoken in self.alive_loops.keys(): if 0 <= looptoken.generation < max_generation: del self.alive_loops[looptoken] Modified: pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py Tue Nov 16 17:03:29 2010 @@ -1663,7 +1663,6 @@ def handle_guard_failure(self, key): debug_start('jit-tracing') - self.staticdata.try_to_free_some_loops() self.staticdata.profiler.start_tracing() assert isinstance(key, compile.ResumeGuardDescr) self.initialize_state_from_guard_failure(key) @@ -1678,6 +1677,7 @@ # alive as long as this MetaInterp self.resumekey = key self.original_loop_token = key.wref_original_loop_token() + self.staticdata.try_to_free_some_loops() try: self.prepare_resume_from_failure(key.guard_opnum) if self.original_loop_token is None: Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Tue Nov 16 17:03:29 2010 @@ -20,7 +20,7 @@ def test_basic(self): memmgr = MemoryManager() - memmgr.set_max_age(3, 1) + memmgr.set_max_age(4, 1) tokens = [FakeLoopToken() for i in range(10)] for token in tokens: memmgr.keep_loop_alive(token) @@ -29,7 +29,7 @@ def test_basic_2(self): memmgr = MemoryManager() - memmgr.set_max_age(3, 1) + memmgr.set_max_age(4, 1) token = FakeLoopToken() memmgr.keep_loop_alive(token) for i in range(10): @@ -41,7 +41,7 @@ def test_basic_3(self): memmgr = MemoryManager() - memmgr.set_max_age(3, 1) + memmgr.set_max_age(4, 1) tokens = [FakeLoopToken() for i in range(10)] for i in range(len(tokens)): print 'record tokens[%d]' % i @@ -74,13 +74,13 @@ g() return 42 - res = self.meta_interp(f, [], loop_longevity=1) + res = self.meta_interp(f, [], loop_longevity=2) assert res == 42 # we should see only the loop and the entry bridge self.check_tree_loop_count(2) - def test_target_loop_kept_alive(self): + def test_target_loop_kept_alive_or_not(self): myjitdriver = JitDriver(greens=['m'], reds=['n']) def g(m): n = 10 @@ -89,28 +89,58 @@ myjitdriver.jit_merge_point(n=n, m=m) n = n - 1 return 21 - import gc def f(): - # create the loop and the entry bridge for 'g(5)' + # Depending on loop_longevity, either: + # A. create the loop and the entry bridge for 'g(5)' + # B. create 8 loops (and throw them away at each iteration) for i in range(8): g(5) # create another loop and another entry bridge for 'g(7)', # to increase the current_generation for i in range(20): g(7) - # reuse the existing loop and entry bridge for 'g(5)'. - # the generation of the entry bridge for g(5) should never - # grow too old. The loop itself gets old, but is kept alive - # by the entry bridge via contains_jumps_to. + # Depending on loop_longevity, either: + # A. reuse the existing loop and entry bridge for 'g(5)'. + # The entry bridge for g(5) should never grow too old. + # The loop itself gets old, but is kept alive by the + # entry bridge via contains_jumps_to. + # B. or, create another loop (and throw away the previous one) g(5) return 42 + # case A res = self.meta_interp(f, [], loop_longevity=3) assert res == 42 - # we should see only the loop and the entry bridge for g(5) and g(7) self.check_tree_loop_count(4) + # case B, with a lower longevity + res = self.meta_interp(f, [], loop_longevity=1) + assert res == 42 + # we should see a loop for each call to g() + self.check_tree_loop_count(8 + 20*2) + + def test_throw_away_old_loops(self): + myjitdriver = JitDriver(greens=['m'], reds=['n']) + def g(m): + n = 10 + while n > 0: + myjitdriver.can_enter_jit(n=n, m=m) + myjitdriver.jit_merge_point(n=n, m=m) + n = n - 1 + return 21 + def f(): + for i in range(10): + g(1) # g(1) gets a loop and an entry bridge, stays alive + g(2) # (and an exit bridge, which does not count in + g(1) # check_tree_loop_count) + g(3) + g(1) + g(4) # g(2), g(3), g(4), g(5) are thrown away every iteration + g(1) # (no entry bridge for them) + g(5) + return 42 - # we need another test that fails because we store - # self._debug_suboperations in compile.py + res = self.meta_interp(f, [], loop_longevity=3) + assert res == 42 + self.check_tree_loop_count(2 + 10*4) # 42 :-) From agaynor at codespeak.net Tue Nov 16 17:03:52 2010 From: agaynor at codespeak.net (agaynor at codespeak.net) Date: Tue, 16 Nov 2010 17:03:52 +0100 (CET) Subject: [pypy-svn] r79151 - pypy/trunk/pypy/doc/config Message-ID: <20101116160352.58A34282B9C@codespeak.net> Author: agaynor Date: Tue Nov 16 17:03:50 2010 New Revision: 79151 Added: pypy/trunk/pypy/doc/config/objspace.translationmodules.txt Log: Added a missing config doc. Added: pypy/trunk/pypy/doc/config/objspace.translationmodules.txt ============================================================================== --- (empty file) +++ pypy/trunk/pypy/doc/config/objspace.translationmodules.txt Tue Nov 16 17:03:50 2010 @@ -0,0 +1 @@ +This option enables all modules which are needed to translate PyPy using PyPy. From afa at codespeak.net Tue Nov 16 17:31:28 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 16 Nov 2010 17:31:28 +0100 (CET) Subject: [pypy-svn] r79152 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101116163128.7B9F2282BD4@codespeak.net> Author: afa Date: Tue Nov 16 17:31:26 2010 New Revision: 79152 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Log: Start implementing _io.BufferedRWPair Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Tue Nov 16 17:31:26 2010 @@ -1,13 +1,15 @@ from __future__ import with_statement from pypy.interpreter.typedef import ( TypeDef, GetSetProperty, generic_new_descr) -from pypy.interpreter.gateway import interp2app, unwrap_spec +from pypy.interpreter.gateway import interp2app, unwrap_spec, Arguments from pypy.interpreter.baseobjspace import ObjSpace, W_Root from pypy.interpreter.error import OperationError, operationerrfmt from pypy.rpython.lltypesystem import lltype, rffi from pypy.rlib.rstring import StringBuilder from pypy.rlib.rarithmetic import r_longlong -from pypy.module._io.interp_iobase import W_IOBase, convert_size +from pypy.tool.sourcetools import func_renamer +from pypy.module._io.interp_iobase import ( + W_IOBase, convert_size, check_readable_w, check_writable_w) from pypy.module._io.interp_io import DEFAULT_BUFFER_SIZE, W_BlockingIOError from pypy.module.thread.os_lock import Lock @@ -25,6 +27,10 @@ def _check_init(self, space): raise NotImplementedError + def _deprecated_max_buffer_size(self, space): + space.warn("max_buffer_size is deprecated", + space.w_DeprecationWarning) + @unwrap_spec('self', ObjSpace, W_Root) def read_w(self, space, w_size=None): self._unsupportedoperation(space, "read") @@ -126,12 +132,8 @@ self._check_init(space) W_IOBase._check_closed(self, space, message) - def _deprecated_max_buffer_size(self, space): - space.warn("max_buffer_size is deprecated", - space.w_DeprecationWarning) - def _raw_tell(self, space): - w_pos = space.call_method(self.raw, "tell") + w_pos = space.call_method(self.w_raw, "tell") pos = space.r_longlong_w(w_pos) if pos < 0: raise OperationError(space.w_IOError, space.wrap( @@ -142,7 +144,7 @@ def closed_get_w(space, self): self._check_init(space) - return space.getattr(self.raw, space.wrap("closed")) + return space.getattr(self.w_raw, space.wrap("closed")) def _readahead(self): if self.readable and self.read_end != -1: @@ -201,7 +203,7 @@ return space.wrap(n) def _raw_seek(self, space, pos, whence): - w_pos = space.call_method(self.raw, "seek", + w_pos = space.call_method(self.w_raw, "seek", space.wrap(pos), space.wrap(whence)) pos = space.r_longlong_w(w_pos) if pos < 0: @@ -210,7 +212,7 @@ return pos def _closed(self, space): - return self.raw._closed(space) + return space.is_true(space.getattr(self.w_raw, space.wrap("closed"))) @unwrap_spec('self', ObjSpace) def close_w(self, space): @@ -220,12 +222,12 @@ return space.call_method(self, "flush") with self.lock: - space.call_method(self.raw, "close") + space.call_method(self.w_raw, "close") @unwrap_spec('self', ObjSpace) def flush_w(self, space): self._check_init(space) - return space.call_method(self.raw, "flush") + return space.call_method(self.w_raw, "flush") def _writer_flush_unlocked(self, space, restore_pos=False): if self.write_end == -1 or self.write_pos == self.write_end: @@ -268,7 +270,7 @@ def _write(self, space, data): w_data = space.wrap(data) - w_written = space.call_method(self.raw, "write", w_data) + w_written = space.call_method(self.w_raw, "write", w_data) written = space.getindex_w(w_written, space.w_IOError) if not 0 <= written <= len(data): raise OperationError(space.w_IOError, space.wrap( @@ -288,15 +290,15 @@ def detach_w(self, space): self._check_init(space) space.call_method(self, "flush") - raw = self.raw - self.raw = None + w_raw = self.w_raw + self.w_raw = None self.state = STATE_DETACHED - return space.wrap(raw) + return w_raw @unwrap_spec('self', ObjSpace) def fileno_w(self, space): self._check_init(space) - return space.call_method(self.raw, "fileno") + return space.call_method(self.w_raw, "fileno") @unwrap_spec('self', ObjSpace, W_Root) def truncate_w(self, space, w_size=None): @@ -313,16 +315,16 @@ # invalidate cached position self.abs_pos = -1 - return space.call_method(self.raw, "truncate", w_size) + return space.call_method(self.w_raw, "truncate", w_size) class W_BufferedReader(BufferedMixin, W_BufferedIOBase): @unwrap_spec('self', ObjSpace, W_Root, int) def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE): self.state = STATE_ZERO - raw = space.interp_w(W_IOBase, w_raw) - raw.check_readable_w(space) + check_readable_w(space, w_raw) + space.call_method(w_raw, "_checkReadable") - self.raw = raw + self.w_raw = w_raw self.buffer_size = buffer_size self.readable = True @@ -403,7 +405,7 @@ while True: # Read until EOF or until read() would block - w_data = space.call_method(self.raw, "read") + w_data = space.call_method(self.w_raw, "read") if space.is_w(w_data, space.w_None): break data = space.str_w(w_data) @@ -417,7 +419,7 @@ return builder.build() def _raw_read(self, space, n): - w_data = space.call_method(self.raw, "read", space.wrap(n)) + w_data = space.call_method(self.w_raw, "read", space.wrap(n)) if space.is_w(w_data, space.w_None): raise BlockingIOError() data = space.str_w(w_data) @@ -547,10 +549,9 @@ self._deprecated_max_buffer_size(space) self.state = STATE_ZERO - raw = space.interp_w(W_IOBase, w_raw) - raw.check_writable_w(space) + space.call_method(w_raw, "_checkWritable") - self.raw = raw + self.w_raw = w_raw self.buffer_size = buffer_size self.writable = True @@ -700,11 +701,81 @@ closed = GetSetProperty(W_BufferedWriter.closed_get_w), ) +def _forward_call(space, w_obj, method, __args__): + w_meth = self.getattr(w_obj, self.wrap(method)) + return self.call_args(w_meth, __args__) + +def make_forwarding_method(method, writer=False, reader=False): + @func_renamer(method + '_w') + @unwrap_spec('self', ObjSpace, Arguments) + def method_w(self, space, __args__): + if writer: + w_meth = space.getattr(self.w_writer, space.wrap(method)) + return space.call_args(w_meth, __args__) + if reader: + w_meth = space.getattr(self.w_reader, space.wrap(method)) + return space.call_args(w_meth, __args__) + return method_w + class W_BufferedRWPair(W_BufferedIOBase): - pass + w_reader = None + w_writer = None + + @unwrap_spec('self', ObjSpace, W_Root, W_Root, int, int) + def descr_init(self, space, w_reader, w_writer, + buffer_size=DEFAULT_BUFFER_SIZE, max_buffer_size=-234): + if max_buffer_size != -234: + self._deprecated_max_buffer_size(space) + + try: + self.w_reader = W_BufferedReader(space) + self.w_reader.descr_init(space, w_reader, buffer_size) + self.w_writer = W_BufferedWriter(space) + self.w_writer.descr_init(space, w_writer, buffer_size) + except Exception: + self.w_reader = None + self.w_writer = None + raise + + def __del__(self): + pass # no not close the files + + # forward to reader + for method in ['read', 'peek', 'read1', 'readinto', 'readable']: + locals()[method + '_w'] = make_forwarding_method( + method, reader=True) + + # forward to writer + for method in ['write', 'flush', 'writable']: + locals()[method + '_w'] = make_forwarding_method( + method, writer=True) + + # forward to both + for method in ['close']: + locals()[method + '_w'] = make_forwarding_method( + method, writer=True, reader=True) + + @unwrap_spec('self', ObjSpace) + def isatty_w(self, space): + if space.is_true(space.call_method(self.w_writer, "isatty")): + return space.w_True + return space.call_method(self.w_reader, "isatty") + + def closed_get_w(space, self): + return space.getattr(self.w_writer, space.wrap("closed")) + +methods = dict((method, interp2app(getattr(W_BufferedRWPair, method + '_w'))) + for method in ['read', 'peek', 'read1', 'readinto', 'readable', + 'write', 'flush', 'writable', + 'close', + 'isatty']) + W_BufferedRWPair.typedef = TypeDef( 'BufferedRWPair', W_BufferedIOBase.typedef, __new__ = generic_new_descr(W_BufferedRWPair), + __init__ = interp2app(W_BufferedRWPair.descr_init), + closed = GetSetProperty(W_BufferedRWPair.closed_get_w), + **methods ) class W_BufferedRandom(W_BufferedIOBase): Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Tue Nov 16 17:31:26 2010 @@ -11,6 +11,30 @@ else: return space.int_w(w_size) +# May be called with any object + at unwrap_spec(ObjSpace, W_Root) +def check_readable_w(space, w_obj): + if not space.is_true(space.call_method(w_obj, 'readable')): + raise OperationError( + space.w_IOError, + space.wrap("file or stream is not readable")) + +# May be called with any object + at unwrap_spec(ObjSpace, W_Root) +def check_writable_w(space, w_obj): + if not space.is_true(space.call_method(w_obj, 'writable')): + raise OperationError( + space.w_IOError, + space.wrap("file or stream is not writable")) + +# May be called with any object + at unwrap_spec(ObjSpace, W_Root) +def check_seekable_w(space, w_obj): + if not space.is_true(space.call_method(w_obj, 'seekable')): + raise OperationError( + space.w_IOError, + space.wrap("file or stream is not seekable")) + class W_IOBase(Wrappable): def __init__(self, space): # XXX: IOBase thinks it has to maintain its own internal state in @@ -114,27 +138,6 @@ def seekable_w(self, space): return space.w_False - @unwrap_spec('self', ObjSpace) - def check_readable_w(self, space): - if not space.is_true(space.call_method(self, 'readable')): - raise OperationError( - space.w_IOError, - space.wrap("file or stream is not readable")) - - @unwrap_spec('self', ObjSpace) - def check_writable_w(self, space): - if not space.is_true(space.call_method(self, 'writable')): - raise OperationError( - space.w_IOError, - space.wrap("file or stream is not writable")) - - @unwrap_spec('self', ObjSpace) - def check_seekable_w(self, space): - if not space.is_true(space.call_method(self, 'seekable')): - raise OperationError( - space.w_IOError, - space.wrap("file or stream is not seekable")) - # ______________________________________________________________ @unwrap_spec('self', ObjSpace, W_Root) @@ -234,9 +237,9 @@ readable = interp2app(W_IOBase.readable_w), writable = interp2app(W_IOBase.writable_w), seekable = interp2app(W_IOBase.seekable_w), - _checkReadable = interp2app(W_IOBase.check_readable_w), - _checkWritable = interp2app(W_IOBase.check_writable_w), - _checkSeekable = interp2app(W_IOBase.check_seekable_w), + _checkReadable = interp2app(check_readable_w), + _checkWritable = interp2app(check_writable_w), + _checkSeekable = interp2app(check_seekable_w), closed = GetSetProperty(W_IOBase.closed_get_w), readline = interp2app(W_IOBase.readline_w), Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Tue Nov 16 17:31:26 2010 @@ -195,3 +195,19 @@ assert b.seek(8) == 8 assert b.truncate() == 8 assert b.tell() == 8 + +class AppTestBufferedRWPair: + def test_pair(self): + import _io + pair = _io.BufferedRWPair(_io.BytesIO("abc"), _io.BytesIO()) + assert not pair.closed + assert pair.read() == "abc" + assert pair.write("abc") == 3 + + def test_constructor_with_not_readable(self): + import _io, io + class NotReadable(io.BytesIO): + def readable(self): + return False + + raises(IOError, _io.BufferedRWPair, NotReadable(), _io.BytesIO()) From cfbolz at codespeak.net Tue Nov 16 17:36:01 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 16 Nov 2010 17:36:01 +0100 (CET) Subject: [pypy-svn] r79153 - pypy/branch/jit-starargs/pypy/module/array Message-ID: <20101116163601.4949350810@codespeak.net> Author: cfbolz Date: Tue Nov 16 17:36:00 2010 New Revision: 79153 Modified: pypy/branch/jit-starargs/pypy/module/array/interp_array.py Log: make this at least not segfault. argument parsing here should be cleaned up, please. Modified: pypy/branch/jit-starargs/pypy/module/array/interp_array.py ============================================================================== --- pypy/branch/jit-starargs/pypy/module/array/interp_array.py (original) +++ pypy/branch/jit-starargs/pypy/module/array/interp_array.py Tue Nov 16 17:36:00 2010 @@ -27,7 +27,7 @@ typecode = typecode[0] if space.is_w(w_cls, space.gettypeobject(W_ArrayBase.typedef)): - if len(w_args.keywords_w) > 0: + if w_args.keywords: # XXX this might be forbidden fishing msg = 'array.array() does not take keyword arguments' raise OperationError(space.w_TypeError, space.wrap(msg)) From afa at codespeak.net Tue Nov 16 17:37:55 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 16 Nov 2010 17:37:55 +0100 (CET) Subject: [pypy-svn] r79154 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101116163755.3AD8B282BDC@codespeak.net> Author: afa Date: Tue Nov 16 17:37:53 2010 New Revision: 79154 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Log: Test and fix Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Tue Nov 16 17:37:53 2010 @@ -322,7 +322,6 @@ def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE): self.state = STATE_ZERO check_readable_w(space, w_raw) - space.call_method(w_raw, "_checkReadable") self.w_raw = w_raw self.buffer_size = buffer_size @@ -549,7 +548,7 @@ self._deprecated_max_buffer_size(space) self.state = STATE_ZERO - space.call_method(w_raw, "_checkWritable") + check_writable_w(space, w_raw) self.w_raw = w_raw self.buffer_size = buffer_size Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Tue Nov 16 17:37:53 2010 @@ -205,9 +205,17 @@ assert pair.write("abc") == 3 def test_constructor_with_not_readable(self): - import _io, io - class NotReadable(io.BytesIO): + import _io + class NotReadable: def readable(self): return False raises(IOError, _io.BufferedRWPair, NotReadable(), _io.BytesIO()) + + def test_constructor_with_not_writable(self): + import _io + class NotWritable: + def writable(self): + return False + + raises(IOError, _io.BufferedRWPair, _io.BytesIO(), NotWritable()) From arigo at codespeak.net Tue Nov 16 17:55:50 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 16 Nov 2010 17:55:50 +0100 (CET) Subject: [pypy-svn] r79155 - in pypy/trunk/pypy/jit/metainterp: . test Message-ID: <20101116165550.4AEE150810@codespeak.net> Author: arigo Date: Tue Nov 16 17:55:48 2010 New Revision: 79155 Modified: pypy/trunk/pypy/jit/metainterp/compile.py pypy/trunk/pypy/jit/metainterp/pyjitpl.py pypy/trunk/pypy/jit/metainterp/test/test_compile.py pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py Log: Tentative: kill ResumeDescr.original_greenkey. It was sometimes wrong, storing the greenkey that did not actually correspond to the start of the loop. It was only ever used in one corner case, which aborts compilation (ABORT_BRIDGE, showing up extremely rarely). Instead, let tracing go ahead and find (probably) that it can reuse an existing loop later. Modified: pypy/trunk/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/compile.py (original) +++ pypy/trunk/pypy/jit/metainterp/compile.py Tue Nov 16 17:55:48 2010 @@ -50,13 +50,12 @@ # ____________________________________________________________ -def compile_new_loop(metainterp, old_loop_tokens, greenkey, start): +def compile_new_loop(metainterp, old_loop_tokens, start): """Try to compile a new loop by closing the current history back to the first operation. """ history = metainterp.history loop = create_empty_loop(metainterp) - loop.greenkey = greenkey loop.inputargs = history.inputargs for box in loop.inputargs: assert isinstance(box, Box) @@ -209,8 +208,7 @@ } class ResumeDescr(AbstractFailDescr): - def __init__(self, original_greenkey): - self.original_greenkey = original_greenkey + pass class ResumeGuardDescr(ResumeDescr): _counter = 0 # if < 0, there is one counter per value; @@ -229,8 +227,7 @@ CNT_FLOAT = -0x60000000 CNT_MASK = 0x1FFFFFFF - def __init__(self, metainterp_sd, original_greenkey): - ResumeDescr.__init__(self, original_greenkey) + def __init__(self, metainterp_sd): self.metainterp_sd = metainterp_sd def store_final_boxes(self, guard_op, boxes): @@ -334,14 +331,14 @@ res.rd_pendingfields = self.rd_pendingfields def _clone_if_mutable(self): - res = ResumeGuardDescr(self.metainterp_sd, self.original_greenkey) + res = ResumeGuardDescr(self.metainterp_sd) self.copy_all_attrbutes_into(res) return res class ResumeGuardForcedDescr(ResumeGuardDescr): - def __init__(self, metainterp_sd, original_greenkey, jitdriver_sd): - ResumeGuardDescr.__init__(self, metainterp_sd, original_greenkey) + def __init__(self, metainterp_sd, jitdriver_sd): + ResumeGuardDescr.__init__(self, metainterp_sd) self.jitdriver_sd = jitdriver_sd def handle_fail(self, metainterp_sd, jitdriver_sd): @@ -408,7 +405,6 @@ def _clone_if_mutable(self): res = ResumeGuardForcedDescr(self.metainterp_sd, - self.original_greenkey, self.jitdriver_sd) self.copy_all_attrbutes_into(res) return res @@ -475,9 +471,8 @@ class ResumeFromInterpDescr(ResumeDescr): - def __init__(self, original_greenkey, redkey): - ResumeDescr.__init__(self, original_greenkey) - self.redkey = redkey + def __init__(self, original_greenkey): + self.original_greenkey = original_greenkey def compile_and_attach(self, metainterp, new_loop): # We managed to create a bridge going from the interpreter @@ -486,10 +481,8 @@ # with completely unoptimized arguments, as in the interpreter. metainterp_sd = metainterp.staticdata jitdriver_sd = metainterp.jitdriver_sd - metainterp.history.inputargs = self.redkey - new_loop_token = make_loop_token(len(self.redkey), jitdriver_sd) - new_loop.greenkey = self.original_greenkey - new_loop.inputargs = self.redkey + redargs = new_loop.inputargs + new_loop_token = make_loop_token(len(redargs), jitdriver_sd) new_loop.token = new_loop_token send_loop_to_backend(metainterp_sd, new_loop, "entry bridge") # send the new_loop to warmspot.py, to be called directly the next time Modified: pypy/trunk/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/trunk/pypy/jit/metainterp/pyjitpl.py Tue Nov 16 17:55:48 2010 @@ -14,7 +14,7 @@ from pypy.jit.metainterp.logger import Logger from pypy.jit.metainterp.jitprof import EmptyProfiler from pypy.jit.metainterp.jitprof import GUARDS, RECORDED_OPS, ABORT_ESCAPE -from pypy.jit.metainterp.jitprof import ABORT_TOO_LONG, ABORT_BRIDGE +from pypy.jit.metainterp.jitprof import ABORT_TOO_LONG from pypy.jit.metainterp.jitexc import JitException, get_llexception from pypy.rlib.rarithmetic import intmask from pypy.rlib.objectmodel import specialize @@ -1048,14 +1048,11 @@ else: moreargs = list(extraargs) metainterp_sd = metainterp.staticdata - original_greenkey = metainterp.resumekey.original_greenkey if opnum == rop.GUARD_NOT_FORCED: resumedescr = compile.ResumeGuardForcedDescr(metainterp_sd, - original_greenkey, metainterp.jitdriver_sd) else: - resumedescr = compile.ResumeGuardDescr(metainterp_sd, - original_greenkey) + resumedescr = compile.ResumeGuardDescr(metainterp_sd) guard_op = metainterp.history.record(opnum, moreargs, None, descr=resumedescr) virtualizable_boxes = None @@ -1636,9 +1633,8 @@ self.current_merge_points = [(original_boxes, 0)] num_green_args = self.jitdriver_sd.num_green_args original_greenkey = original_boxes[:num_green_args] - redkey = original_boxes[num_green_args:] - self.resumekey = compile.ResumeFromInterpDescr(original_greenkey, - redkey) + self.resumekey = compile.ResumeFromInterpDescr(original_greenkey) + self.history.inputargs = original_boxes[num_green_args:] self.seen_loop_header_for_jdindex = -1 try: self.interpret() @@ -1660,11 +1656,7 @@ debug_stop('jit-tracing') def _handle_guard_failure(self, key): - original_greenkey = key.original_greenkey - # notice that here we just put the greenkey - # use -1 to mark that we will have to give up - # because we cannot reconstruct the beginning of the proper loop - self.current_merge_points = [(original_greenkey, -1)] + self.current_merge_points = [] self.resumekey = key self.seen_loop_header_for_jdindex = -1 try: @@ -1728,7 +1720,7 @@ num_green_args = self.jitdriver_sd.num_green_args for j in range(len(self.current_merge_points)-1, -1, -1): original_boxes, start = self.current_merge_points[j] - assert len(original_boxes) == len(live_arg_boxes) or start < 0 + assert len(original_boxes) == len(live_arg_boxes) for i in range(num_green_args): box1 = original_boxes[i] box2 = live_arg_boxes[i] @@ -1737,10 +1729,6 @@ break else: # Found! Compile it as a loop. - if start < 0: - # we cannot reconstruct the beginning of the proper loop - raise SwitchToBlackhole(ABORT_BRIDGE) - # raises in case it works -- which is the common case self.compile(original_boxes, live_arg_boxes, start) # creation of the loop was cancelled! @@ -1805,8 +1793,7 @@ greenkey = original_boxes[:num_green_args] old_loop_tokens = self.get_compiled_merge_points(greenkey) self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None) - loop_token = compile.compile_new_loop(self, old_loop_tokens, - greenkey, start) + loop_token = compile.compile_new_loop(self, old_loop_tokens, start) if loop_token is not None: # raise if it *worked* correctly raise GenerateMergePoint(live_arg_boxes, loop_token) self.history.operations.pop() # remove the JUMP Modified: pypy/trunk/pypy/jit/metainterp/test/test_compile.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_compile.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_compile.py Tue Nov 16 17:55:48 2010 @@ -85,7 +85,7 @@ metainterp.history.inputargs = loop.inputargs[:] # loop_tokens = [] - loop_token = compile_new_loop(metainterp, loop_tokens, [], 0) + loop_token = compile_new_loop(metainterp, loop_tokens, 0) assert loop_tokens == [loop_token] assert loop_token.number == 1 assert staticdata.globaldata.loopnumbering == 2 @@ -101,7 +101,7 @@ metainterp.history.operations = loop.operations[:] metainterp.history.inputargs = loop.inputargs[:] # - loop_token_2 = compile_new_loop(metainterp, loop_tokens, [], 0) + loop_token_2 = compile_new_loop(metainterp, loop_tokens, 0) assert loop_token_2 is loop_token assert loop_tokens == [loop_token] assert len(cpu.seen) == 0 Modified: pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py Tue Nov 16 17:55:48 2010 @@ -41,7 +41,7 @@ b1 = BoxInt() opt = optimizeopt.Optimizer(FakeMetaInterpStaticData(LLtypeMixin.cpu), None) - fdescr = ResumeGuardDescr(None, None) + fdescr = ResumeGuardDescr(None) op = ResOperation(rop.GUARD_TRUE, ['dummy'], None, descr=fdescr) # setup rd data fi0 = resume.FrameInfo(None, "code0", 11) From cfbolz at codespeak.net Tue Nov 16 17:56:25 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 16 Nov 2010 17:56:25 +0100 (CET) Subject: [pypy-svn] r79156 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101116165625.1EF49282B9C@codespeak.net> Author: cfbolz Date: Tue Nov 16 17:56:23 2010 New Revision: 79156 Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py pypy/branch/reflex-support/pypy/module/cppyy/executor.py Log: try to fix some RPython issues Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/converter.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/converter.py Tue Nov 16 17:56:23 2010 @@ -1,3 +1,4 @@ +import sys from pypy.interpreter.error import OperationError from pypy.rpython.lltypesystem import rffi, lltype from pypy.rlib.rarithmetic import r_singlefloat @@ -10,7 +11,7 @@ _converters = {} class TypeConverter(object): - def __init__(self, space, extra=None): + def __init__(self, space, extra=-1): pass def _get_fieldptr(self, space, w_obj, offset): @@ -173,14 +174,11 @@ class ShortPtrConverter(TypeConverter): _immutable_ = True - def __init__(self, space, detail=None): - if detail is None: - import sys - detail = sys.maxint + def __init__(self, space, detail=sys.maxint): self.size = detail def convert_argument(self, space, w_obj): - assert "not yet implemented" + assert 0, "not yet implemented" def from_memory(self, space, w_obj, offset): # read access, so no copy needed @@ -206,14 +204,11 @@ class LongPtrConverter(TypeConverter): _immutable_ = True - def __init__(self, space, detail=None): - if detail is None: - import sys - detail = sys.maxint + def __init__(self, space, detail=sys.maxint): self.size = detail def convert_argument(self, space, w_obj): - assert "not yet implemented" + assert 0, "not yet implemented" def from_memory(self, space, w_obj, offset): # read access, so no copy needed Modified: pypy/branch/reflex-support/pypy/module/cppyy/executor.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/executor.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/executor.py Tue Nov 16 17:56:23 2010 @@ -7,8 +7,7 @@ class FunctionExecutor(object): def execute(self, space, func, cppthis, num_args, args): - raise NotImplementedError( - "abstract base class (actual: %s)" % type(self).__name__) + raise NotImplementedError class VoidExecutor(FunctionExecutor): From afa at codespeak.net Tue Nov 16 18:00:12 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 16 Nov 2010 18:00:12 +0100 (CET) Subject: [pypy-svn] r79157 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101116170012.202F850810@codespeak.net> Author: afa Date: Tue Nov 16 18:00:10 2010 New Revision: 79157 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Log: BufferedReader.peek() Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Tue Nov 16 18:00:10 2010 @@ -349,6 +349,34 @@ return space.wrap(res) @unwrap_spec('self', ObjSpace, int) + def peek_w(self, space, size=0): + self._check_init(space) + with self.lock: + if self.writable: + self._writer_flush_unlocked(space, restore_pos=True) + # Constraints: + # 1. we don't want to advance the file position. + # 2. we don't want to lose block alignment, so we can't shift the + # buffer to make some place. + # Therefore, we either return `have` bytes (if > 0), or a full + # buffer. + have = self._readahead() + if have > 0: + data = rffi.charpsize2str(rffi.ptradd(self.buffer, self.pos), + have) + return space.wrap(data) + + # Fill the buffer from the raw stream, and copy it to the result + self._reader_reset_buf() + try: + size = self._fill_buffer(space) + except BlockingIOError: + size = 0 + self.pos = 0 + data = rffi.charpsize2str(self.buffer, size) + return space.wrap(data) + + @unwrap_spec('self', ObjSpace, int) def read1_w(self, space, size): self._check_init(space) self._check_closed(space, "read of closed file") @@ -527,6 +555,7 @@ __init__ = interp2app(W_BufferedReader.descr_init), read = interp2app(W_BufferedReader.read_w), + peek = interp2app(W_BufferedReader.peek_w), read1 = interp2app(W_BufferedReader.read1_w), # from the mixin class Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Tue Nov 16 18:00:10 2010 @@ -32,6 +32,15 @@ assert f.read(3) == "" f.close() + def test_peek(self): + import _io + raw = _io.FileIO(self.tmpfile) + f = _io.BufferedReader(raw) + assert f.read(2) == 'a\n' + assert f.peek().startswith('b\nc') + assert f.read(3) == 'b\nc' + assert f.peek() == '' + def test_read1(self): import _io class RecordingFileIO(_io.FileIO): From afa at codespeak.net Tue Nov 16 18:09:20 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 16 Nov 2010 18:09:20 +0100 (CET) Subject: [pypy-svn] r79158 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101116170920.9D3A3282B90@codespeak.net> Author: afa Date: Tue Nov 16 18:09:19 2010 New Revision: 79158 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py Log: Test and Fix in BytesIO.read() Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py Tue Nov 16 18:09:19 2010 @@ -38,9 +38,9 @@ size = convert_size(space, w_size) # adjust invalid sizes - n = self.string_size - self.pos - if not 0 <= n <= size: - size = n + available = self.string_size - self.pos + if not 0 <= size <= available: + size = available if size < 0: size = 0 Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py Tue Nov 16 18:09:19 2010 @@ -25,6 +25,7 @@ f = _io.BytesIO("hello") assert f.read() == "hello" import gc; gc.collect() + assert f.read(8192) == "" f.close() def test_seek(self): From cfbolz at codespeak.net Tue Nov 16 18:14:23 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 16 Nov 2010 18:14:23 +0100 (CET) Subject: [pypy-svn] r79159 - pypy/branch/jit-starargs/pypy/module/pypyjit/test Message-ID: <20101116171423.F3D57282BDC@codespeak.net> Author: cfbolz Date: Tue Nov 16 18:14:22 2010 New Revision: 79159 Modified: pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py Log: some details changed, fix this test Modified: pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jit-starargs/pypy/module/pypyjit/test/test_pypy_c.py Tue Nov 16 18:14:22 2010 @@ -377,10 +377,10 @@ ([1000], 49500), ([10000], 495000), ([100000], 4950000)) - assert len(self.loops) == 2 + assert len(self.loops) == 3 op, = self.get_by_bytecode("CALL_FUNCTION_KW") # XXX a bit too many guards, but better than before - assert len(op.get_opnames("guard")) <= 10 + assert len(op.get_opnames("guard")) <= 12 def test_stararg_virtual(self): self.run_source(''' From afa at codespeak.net Tue Nov 16 18:22:40 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 16 Nov 2010 18:22:40 +0100 (CET) Subject: [pypy-svn] r79160 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101116172240.4D7A5282BE3@codespeak.net> Author: afa Date: Tue Nov 16 18:22:38 2010 New Revision: 79160 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Log: add readable() &co to BufferedReader Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Tue Nov 16 18:22:38 2010 @@ -146,6 +146,21 @@ self._check_init(space) return space.getattr(self.w_raw, space.wrap("closed")) + @unwrap_spec('self', ObjSpace) + def readable_w(self, space): + self._check_init(space) + return space.call_method(self.w_raw, "readable") + + @unwrap_spec('self', ObjSpace) + def writable_w(self, space): + self._check_init(space) + return space.call_method(self.w_raw, "writable") + + @unwrap_spec('self', ObjSpace) + def seekable_w(self, space): + self._check_init(space) + return space.call_method(self.w_raw, "seekable") + def _readahead(self): if self.readable and self.read_end != -1: return self.read_end - self.pos @@ -559,6 +574,9 @@ read1 = interp2app(W_BufferedReader.read1_w), # from the mixin class + readable = interp2app(W_BufferedReader.readable_w), + writable = interp2app(W_BufferedReader.writable_w), + seekable = interp2app(W_BufferedReader.seekable_w), seek = interp2app(W_BufferedReader.seek_w), tell = interp2app(W_BufferedReader.tell_w), close = interp2app(W_BufferedReader.close_w), @@ -720,6 +738,9 @@ flush = interp2app(W_BufferedWriter.flush_w), # from the mixin class + readable = interp2app(W_BufferedWriter.readable_w), + writable = interp2app(W_BufferedWriter.writable_w), + seekable = interp2app(W_BufferedWriter.seekable_w), seek = interp2app(W_BufferedWriter.seek_w), tell = interp2app(W_BufferedWriter.tell_w), close = interp2app(W_BufferedWriter.close_w), @@ -739,10 +760,11 @@ def method_w(self, space, __args__): if writer: w_meth = space.getattr(self.w_writer, space.wrap(method)) - return space.call_args(w_meth, __args__) + w_result = space.call_args(w_meth, __args__) if reader: w_meth = space.getattr(self.w_reader, space.wrap(method)) - return space.call_args(w_meth, __args__) + w_result = space.call_args(w_meth, __args__) + return w_result return method_w class W_BufferedRWPair(W_BufferedIOBase): Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Tue Nov 16 18:22:38 2010 @@ -210,6 +210,8 @@ import _io pair = _io.BufferedRWPair(_io.BytesIO("abc"), _io.BytesIO()) assert not pair.closed + assert pair.readable() + assert pair.writable() assert pair.read() == "abc" assert pair.write("abc") == 3 From agaynor at codespeak.net Tue Nov 16 18:43:19 2010 From: agaynor at codespeak.net (agaynor at codespeak.net) Date: Tue, 16 Nov 2010 18:43:19 +0100 (CET) Subject: [pypy-svn] r79161 - in pypy/trunk/pypy/translator/goal: . test2 Message-ID: <20101116174319.C8E86282B90@codespeak.net> Author: agaynor Date: Tue Nov 16 18:43:18 2010 New Revision: 79161 Modified: pypy/trunk/pypy/translator/goal/app_main.py pypy/trunk/pypy/translator/goal/test2/test_app_main.py Log: When running pypy with -c '' needs to be on sys.path. Modified: pypy/trunk/pypy/translator/goal/app_main.py ============================================================================== --- pypy/trunk/pypy/translator/goal/app_main.py (original) +++ pypy/trunk/pypy/translator/goal/app_main.py Tue Nov 16 18:43:18 2010 @@ -362,6 +362,9 @@ try: if run_command: # handle the "-c" command + # Put '' on sys.path + sys.path.insert(0, '') + def run_it(): exec cmd in mainmodule.__dict__ success = run_toplevel(run_it) Modified: pypy/trunk/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/trunk/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/trunk/pypy/translator/goal/test2/test_app_main.py Tue Nov 16 18:43:18 2010 @@ -496,6 +496,9 @@ assert repr(str(tmpdir.join('otherpath'))) in data assert "''" not in data + data = self.run('-c "import sys; print sys.path"') + assert data.startswith("[''") + class AppTestAppMain: From afa at codespeak.net Tue Nov 16 19:17:49 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 16 Nov 2010 19:17:49 +0100 (CET) Subject: [pypy-svn] r79162 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101116181749.F1F55282B90@codespeak.net> Author: afa Date: Tue Nov 16 19:17:48 2010 New Revision: 79162 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Log: Add __repr__, name, mode Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Tue Nov 16 19:17:48 2010 @@ -61,7 +61,7 @@ return space.wrap(len(data)) W_BufferedIOBase.typedef = TypeDef( - '_BufferedIOBase', W_IOBase.typedef, + '_io._BufferedIOBase', W_IOBase.typedef, __new__ = generic_new_descr(W_BufferedIOBase), read = interp2app(W_BufferedIOBase.read_w), read1 = interp2app(W_BufferedIOBase.read1_w), @@ -146,6 +146,12 @@ self._check_init(space) return space.getattr(self.w_raw, space.wrap("closed")) + def name_get_w(space, self): + return space.getattr(self.w_raw, space.wrap("name")) + + def mode_get_w(space, self): + return space.getattr(self.w_raw, space.wrap("mode")) + @unwrap_spec('self', ObjSpace) def readable_w(self, space): self._check_init(space) @@ -161,6 +167,21 @@ self._check_init(space) return space.call_method(self.w_raw, "seekable") + @unwrap_spec('self', ObjSpace) + def repr_w(self, space): + typename = space.type(self).getname(space, '?') + try: + w_name = space.getattr(self, space.wrap("name")) + except OperationError, e: + if not e.match(space, space.w_AttributeError): + raise + return space.wrap("<%s>" % (typename,)) + else: + w_repr = space.repr(w_name) + return space.wrap("<%s name=%s>" % (typename, space.str_w(w_repr))) + + # ______________________________________________ + def _readahead(self): if self.readable and self.read_end != -1: return self.read_end - self.pos @@ -565,7 +586,7 @@ return None W_BufferedReader.typedef = TypeDef( - 'BufferedReader', W_BufferedIOBase.typedef, + '_io.BufferedReader', W_BufferedIOBase.typedef, __new__ = generic_new_descr(W_BufferedReader), __init__ = interp2app(W_BufferedReader.descr_init), @@ -574,6 +595,7 @@ read1 = interp2app(W_BufferedReader.read1_w), # from the mixin class + __repr__ = interp2app(W_BufferedReader.repr_w), readable = interp2app(W_BufferedReader.readable_w), writable = interp2app(W_BufferedReader.writable_w), seekable = interp2app(W_BufferedReader.seekable_w), @@ -585,6 +607,8 @@ truncate = interp2app(W_BufferedReader.truncate_w), fileno = interp2app(W_BufferedReader.fileno_w), closed = GetSetProperty(W_BufferedReader.closed_get_w), + name = GetSetProperty(W_BufferedReader.name_get_w), + mode = GetSetProperty(W_BufferedReader.mode_get_w), ) class W_BufferedWriter(BufferedMixin, W_BufferedIOBase): @@ -730,7 +754,7 @@ self._reader_reset_buf() W_BufferedWriter.typedef = TypeDef( - 'BufferedWriter', W_BufferedIOBase.typedef, + '_io.BufferedWriter', W_BufferedIOBase.typedef, __new__ = generic_new_descr(W_BufferedWriter), __init__ = interp2app(W_BufferedWriter.descr_init), @@ -738,6 +762,7 @@ flush = interp2app(W_BufferedWriter.flush_w), # from the mixin class + __repr__ = interp2app(W_BufferedWriter.repr_w), readable = interp2app(W_BufferedWriter.readable_w), writable = interp2app(W_BufferedWriter.writable_w), seekable = interp2app(W_BufferedWriter.seekable_w), @@ -748,6 +773,8 @@ detach = interp2app(W_BufferedWriter.detach_w), truncate = interp2app(W_BufferedWriter.truncate_w), closed = GetSetProperty(W_BufferedWriter.closed_get_w), + name = GetSetProperty(W_BufferedReader.name_get_w), + mode = GetSetProperty(W_BufferedReader.mode_get_w), ) def _forward_call(space, w_obj, method, __args__): @@ -821,7 +848,7 @@ 'isatty']) W_BufferedRWPair.typedef = TypeDef( - 'BufferedRWPair', W_BufferedIOBase.typedef, + '_io.BufferedRWPair', W_BufferedIOBase.typedef, __new__ = generic_new_descr(W_BufferedRWPair), __init__ = interp2app(W_BufferedRWPair.descr_init), closed = GetSetProperty(W_BufferedRWPair.closed_get_w), @@ -831,7 +858,7 @@ class W_BufferedRandom(W_BufferedIOBase): pass W_BufferedRandom.typedef = TypeDef( - 'BufferedRandom', W_BufferedIOBase.typedef, + '_io.BufferedRandom', W_BufferedIOBase.typedef, __new__ = generic_new_descr(W_BufferedRandom), ) Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Tue Nov 16 19:17:48 2010 @@ -119,6 +119,12 @@ assert d1 + d2 == d3 f.close() + def test_repr(self): + import _io + raw = _io.FileIO(self.tmpfile) + f = _io.BufferedReader(raw) + assert repr(f) == '<_io.BufferedReader name=%r>' % (self.tmpfile,) + class AppTestBufferedWriter: def setup_class(cls): cls.space = gettestobjspace(usemodules=['_io']) From arigo at codespeak.net Tue Nov 16 19:25:28 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 16 Nov 2010 19:25:28 +0100 (CET) Subject: [pypy-svn] r79163 - in pypy/branch/jit-free/pypy/jit: backend backend/llgraph metainterp metainterp/test Message-ID: <20101116182528.D3A4B282B90@codespeak.net> Author: arigo Date: Tue Nov 16 19:25:26 2010 New Revision: 79163 Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py pypy/branch/jit-free/pypy/jit/backend/model.py pypy/branch/jit-free/pypy/jit/metainterp/compile.py pypy/branch/jit-free/pypy/jit/metainterp/history.py pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_basic.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_compile.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_optimizeopt.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmspot.py pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Log: Merge r79155 from trunk, and hack until tests pass. Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py Tue Nov 16 19:25:26 2010 @@ -133,7 +133,8 @@ is not. """ c = llimpl.compile_start() - assert not hasattr(loopdescr, '_llgraph_loop_and_bridges') + if not we_are_translated(): + assert not hasattr(loopdescr, '_llgraph_loop_and_bridges') loopdescr._llgraph_loop_and_bridges = [c] loopdescr._llgraph_compiled_version = c self._compile_loop_or_bridge(c, inputargs, operations) Modified: pypy/branch/jit-free/pypy/jit/backend/model.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/model.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/model.py Tue Nov 16 19:25:26 2010 @@ -25,9 +25,6 @@ n = len(lst) lst.append(descr) descr.index = n - looptoken = descr.original_loop_token - if looptoken is not None: - looptoken.faildescr_indices.append(n) return n def get_fail_descr_from_number(self, n): Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/compile.py Tue Nov 16 19:25:26 2010 @@ -1,4 +1,4 @@ - +import weakref from pypy.rpython.lltypesystem import lltype from pypy.rpython.ootypesystem import ootype from pypy.objspace.flow.model import Constant, Variable @@ -42,16 +42,43 @@ name = metainterp.staticdata.stats.name_for_new_loop() return TreeLoop(name) -def make_loop_token(cpu, nb_args, jitdriver_sd, greenkey): +def make_loop_token(cpu, nb_args, jitdriver_sd): loop_token = LoopToken(cpu) loop_token.specnodes = [prebuiltNotSpecNode] * nb_args loop_token.outermost_jitdriver_sd = jitdriver_sd - loop_token.outermost_greenkey = greenkey return loop_token +def record_loop_or_bridge(loop): + """Do post-backend recordings and cleanups on 'loop'. + """ + # get the original loop token (corresponding to 'loop', or if that is + # a bridge, to the loop that this bridge belongs to) + looptoken = loop.token + assert looptoken is not None + wref = weakref.ref(looptoken) + for op in loop.operations: + if op.is_guard(): + resumedescr = op.getdescr() + assert isinstance(resumedescr, ResumeDescr) + resumedescr.wref_original_loop_token = wref # stick it there + n = resumedescr.index + if n >= 0: # we also record the resumedescr in this list + looptoken.faildescr_indices.append(n) + # record that 'looptoken' ends up jumping to 'target_loop_token' + target_loop_token = loop.operations[-1].getdescr() + if isinstance(target_loop_token, LoopToken): + # the following test is not enough to prevent more complicated + # cases of cycles, but at least it helps in simple tests of + # test_memgr.py + if target_loop_token is not looptoken: + looptoken.record_jump_to(target_loop_token) + loop.operations[-1].setdescr(None) # clear reference + # mostly for tests: make sure we don't keep a reference to the LoopToken + loop.token = None + # ____________________________________________________________ -def compile_new_loop(metainterp, old_loop_tokens, greenkey, start): +def compile_new_loop(metainterp, old_loop_tokens, start): """Try to compile a new loop by closing the current history back to the first operation. """ @@ -64,11 +91,9 @@ h_ops = history.operations loop.operations = [h_ops[i].clone() for i in range(start, len(h_ops))] metainterp_sd = metainterp.staticdata + cpu = metainterp.cpu jitdriver_sd = metainterp.jitdriver_sd - loop_token = metainterp.original_loop_token - assert len(loop_token.specnodes) == len(loop.inputargs) - assert loop_token.outermost_jitdriver_sd is jitdriver_sd - assert loop_token.outermost_greenkey == greenkey + loop_token = make_loop_token(cpu, len(loop.inputargs), jitdriver_sd) loop.token = loop_token loop.operations[-1].setdescr(loop_token) # patch the target of the JUMP try: @@ -81,10 +106,7 @@ return old_loop_token send_loop_to_backend(metainterp_sd, loop, "loop") insert_loop_token(old_loop_tokens, loop_token) - # mostly for tests: make sure we don't keep a reference to the LoopToken - metainterp.original_loop_token.record_loop_or_bridge(loop) - loop.token = None - metainterp.original_loop_token = None + record_loop_or_bridge(loop) return loop_token def insert_loop_token(old_loop_tokens, loop_token): @@ -220,8 +242,7 @@ } class ResumeDescr(AbstractFailDescr): - def __init__(self, wref_original_loop_token): - self.wref_original_loop_token = wref_original_loop_token + pass class ResumeGuardDescr(ResumeDescr): _counter = 0 # if < 0, there is one counter per value; @@ -324,13 +345,13 @@ def compile_and_attach(self, metainterp, new_loop): # We managed to create a bridge. Attach the new operations - # to the corrsponding guard_op and compile from there + # to the corresponding guard_op and compile from there + new_loop.token = metainterp.resumekey_original_loop_token inputargs = metainterp.history.inputargs if not we_are_translated(): self._debug_suboperations = new_loop.operations send_bridge_to_backend(metainterp.staticdata, self, inputargs, - new_loop.operations, - metainterp.original_loop_token) + new_loop.operations, new_loop.token) def copy_all_attrbutes_into(self, res): # XXX a bit ugly to have to list them all here @@ -342,14 +363,13 @@ res.rd_pendingfields = self.rd_pendingfields def _clone_if_mutable(self): - res = ResumeGuardDescr(self.wref_original_loop_token) + res = ResumeGuardDescr() self.copy_all_attrbutes_into(res) return res class ResumeGuardForcedDescr(ResumeGuardDescr): - def __init__(self, metainterp_sd, wref_original_loop_token, jitdriver_sd): - ResumeGuardDescr.__init__(self, wref_original_loop_token) + def __init__(self, metainterp_sd, jitdriver_sd): self.metainterp_sd = metainterp_sd self.jitdriver_sd = jitdriver_sd @@ -417,7 +437,6 @@ def _clone_if_mutable(self): res = ResumeGuardForcedDescr(self.metainterp_sd, - self.wref_original_loop_token, self.jitdriver_sd) self.copy_all_attrbutes_into(res) return res @@ -484,9 +503,8 @@ class ResumeFromInterpDescr(ResumeDescr): - def __init__(self, wref_original_loop_token, redkey): - ResumeDescr.__init__(self, wref_original_loop_token) - self.redkey = redkey + def __init__(self, original_greenkey): + self.original_greenkey = original_greenkey def compile_and_attach(self, metainterp, new_loop): # We managed to create a bridge going from the interpreter @@ -494,25 +512,25 @@ # a loop at all but ends in a jump to the target loop. It starts # with completely unoptimized arguments, as in the interpreter. metainterp_sd = metainterp.staticdata + cpu = metainterp.cpu jitdriver_sd = metainterp.jitdriver_sd - metainterp.history.inputargs = self.redkey - new_loop_token = metainterp.original_loop_token - # assert that the wref_original_loop_token was kept alive, because - # it's just the same as metainterp.original_loop_token - assert new_loop_token is self.wref_original_loop_token() - new_loop.inputargs = self.redkey + redargs = new_loop.inputargs + # We make a new LoopToken for this entry bridge, and stick it + # to every guard in the loop. + new_loop_token = make_loop_token(cpu, len(redargs), jitdriver_sd) new_loop.token = new_loop_token send_loop_to_backend(metainterp_sd, new_loop, "entry bridge") # send the new_loop to warmspot.py, to be called directly the next time jitdriver_sd.warmstate.attach_unoptimized_bridge_from_interp( + self.original_greenkey, new_loop_token) # store the new loop in compiled_merge_points_wref too old_loop_tokens = metainterp.get_compiled_merge_points( - new_loop_token.outermost_greenkey) + self.original_greenkey) # it always goes at the end of the list, as it is the most # general loop token old_loop_tokens.append(new_loop_token) - metainterp.set_compiled_merge_points(new_loop_token.outermost_greenkey, + metainterp.set_compiled_merge_points(self.original_greenkey, old_loop_tokens) def reset_counter_from_failure(self): @@ -548,11 +566,7 @@ # know exactly what we must do (ResumeGuardDescr/ResumeFromInterpDescr) prepare_last_operation(new_loop, target_loop_token) resumekey.compile_and_attach(metainterp, new_loop) - # mostly for tests: make sure we don't keep a reference - # to the LoopToken - metainterp.original_loop_token.record_loop_or_bridge(new_loop) - metainterp.original_loop_token = None - new_loop.token = None + record_loop_or_bridge(new_loop) return target_loop_token def prepare_last_operation(new_loop, target_loop_token): @@ -586,7 +600,7 @@ """ # 'redboxes' is only used to know the types of red arguments. inputargs = [box.clonebox() for box in redboxes] - loop_token = make_loop_token(cpu, len(inputargs), jitdriver_sd, greenboxes) + loop_token = make_loop_token(cpu, len(inputargs), jitdriver_sd) # 'nb_red_args' might be smaller than len(redboxes), # because it doesn't include the virtualizable boxes. nb_red_args = jitdriver_sd.num_red_args Modified: pypy/branch/jit-free/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/history.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/history.py Tue Nov 16 19:25:26 2010 @@ -179,7 +179,6 @@ class AbstractFailDescr(AbstractDescr): index = -1 - original_loop_token = None def handle_fail(self, metainterp_sd, jitdriver_sd): raise NotImplementedError @@ -730,14 +729,13 @@ """ terminating = False # see TerminatingLoopToken in compile.py outermost_jitdriver_sd = None - outermost_greenkey = None # specnodes = ... # and more data specified by the backend when the loop is compiled cpu = None number = -1 + generation = r_longlong(0) def __init__(self, cpu=None): - assert 'CPU' in type(cpu).__name__ # xxx temporary self.cpu = cpu # See get_fail_descr_number() in backend/model.py: this growing # list gives the 'descr_number' of all fail descrs that belong to @@ -745,27 +743,11 @@ self.faildescr_indices = [] # For memory management of assembled loops self._keepalive_target_looktokens = {} # set of other LoopTokens - self.generation = r_longlong(0) - def record_loop_or_bridge(self, loop): - # Records that the loop starting at the LoopToken 'self' ends up - # with 'loop', which may be either the loop itself or some pseudo- - # loop representing some bridge. - other_loop_token = loop.operations[-1].getdescr() - if isinstance(other_loop_token, LoopToken): - # the following test is not enough to prevent more complicated - # cases of cycles, but at least it helps in simple tests of - # test_memgr.py - if other_loop_token is not self: - self._keepalive_target_looktokens[other_loop_token] = None - loop.operations[-1].setdescr(None) # clear reference + def record_jump_to(self, target_loop_token): + self._keepalive_target_looktokens[target_loop_token] = None def __del__(self): - for i in range(160): - print '#', - print repr(self), self.cpu - if self.cpu is None: - return if self.generation > r_longlong(0): # MemoryManager.keep_loop_alive() has been called on this # loop token, which means that it has been successfully Modified: pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py Tue Nov 16 19:25:26 2010 @@ -1048,14 +1048,11 @@ else: moreargs = list(extraargs) metainterp_sd = metainterp.staticdata - resumekey = metainterp.resumekey - wref_original_loop_token = resumekey.wref_original_loop_token if opnum == rop.GUARD_NOT_FORCED: resumedescr = compile.ResumeGuardForcedDescr(metainterp_sd, - wref_original_loop_token, metainterp.jitdriver_sd) else: - resumedescr = compile.ResumeGuardDescr(wref_original_loop_token) + resumedescr = compile.ResumeGuardDescr() guard_op = metainterp.history.record(opnum, moreargs, None, descr=resumedescr) virtualizable_boxes = None @@ -1642,16 +1639,8 @@ self.current_merge_points = [(original_boxes, 0)] num_green_args = self.jitdriver_sd.num_green_args original_greenkey = original_boxes[:num_green_args] - redkey = original_boxes[num_green_args:] - # make a new loop token and store a strong ref to it for as long as - # this MetaInterp is alive (we will give it to the MemoryManager after - # the backend has emitted assembler) - self.original_loop_token = compile.make_loop_token(self.cpu, - len(redkey), - self.jitdriver_sd, - original_greenkey) - self.resumekey = compile.ResumeFromInterpDescr( - weakref.ref(self.original_loop_token), redkey) + self.resumekey = compile.ResumeFromInterpDescr(original_greenkey) + self.history.inputargs = original_boxes[num_green_args:] self.seen_loop_header_for_jdindex = -1 try: self.interpret() @@ -1665,29 +1654,26 @@ debug_start('jit-tracing') self.staticdata.profiler.start_tracing() assert isinstance(key, compile.ResumeGuardDescr) + # store the resumekey.wref_original_loop_token() on 'self' to make + # sure that it stays alive as long as this MetaInterp + self.resumekey_original_loop_token = key.wref_original_loop_token() self.initialize_state_from_guard_failure(key) try: return self._handle_guard_failure(key) finally: + self.resumekey_original_loop_token = None self.staticdata.profiler.end_tracing() debug_stop('jit-tracing') def _handle_guard_failure(self, key): - # store the original_loop_token on 'self' to make sure that it stays - # alive as long as this MetaInterp + self.current_merge_points = [] self.resumekey = key - self.original_loop_token = key.wref_original_loop_token() + self.seen_loop_header_for_jdindex = -1 self.staticdata.try_to_free_some_loops() try: self.prepare_resume_from_failure(key.guard_opnum) - if self.original_loop_token is None: + if self.resumekey_original_loop_token is None: # very rare case raise SwitchToBlackhole(ABORT_BRIDGE) - # notice that here we just put the greenkey - # use -1 to mark that we will have to give up - # because we cannot reconstruct the beginning of the proper loop - original_greenkey = self.original_loop_token.outermost_greenkey - self.current_merge_points = [(original_greenkey, -1)] - self.seen_loop_header_for_jdindex = -1 self.interpret() except GenerateMergePoint, gmp: return self.designate_target_loop(gmp) @@ -1747,7 +1733,7 @@ num_green_args = self.jitdriver_sd.num_green_args for j in range(len(self.current_merge_points)-1, -1, -1): original_boxes, start = self.current_merge_points[j] - assert len(original_boxes) == len(live_arg_boxes) or start < 0 + assert len(original_boxes) == len(live_arg_boxes) for i in range(num_green_args): box1 = original_boxes[i] box2 = live_arg_boxes[i] @@ -1756,10 +1742,6 @@ break else: # Found! Compile it as a loop. - if start < 0: - # we cannot reconstruct the beginning of the proper loop - raise SwitchToBlackhole(ABORT_BRIDGE) - # raises in case it works -- which is the common case self.compile(original_boxes, live_arg_boxes, start) # creation of the loop was cancelled! @@ -1829,8 +1811,7 @@ greenkey = original_boxes[:num_green_args] old_loop_tokens = self.get_compiled_merge_points(greenkey) self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None) - loop_token = compile.compile_new_loop(self, old_loop_tokens, - greenkey, start) + loop_token = compile.compile_new_loop(self, old_loop_tokens, start) if loop_token is not None: # raise if it *worked* correctly self.set_compiled_merge_points(greenkey, old_loop_tokens) raise GenerateMergePoint(live_arg_boxes, loop_token) Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_basic.py Tue Nov 16 19:25:26 2010 @@ -26,7 +26,7 @@ self.__compiled_merge_points = lst class FakeWarmRunnerState: - def attach_unoptimized_bridge_from_interp(self, newloop): + def attach_unoptimized_bridge_from_interp(self, greenkey, newloop): pass def jit_cell_at_key(self, greenkey): Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_compile.py Tue Nov 16 19:25:26 2010 @@ -5,7 +5,6 @@ from pypy.jit.metainterp.compile import ResumeGuardDescr from pypy.jit.metainterp.compile import ResumeGuardCountersInt from pypy.jit.metainterp.compile import compile_tmp_callback -from pypy.jit.metainterp.compile import send_loop_to_backend from pypy.jit.metainterp import optimize, jitprof, typesystem, compile from pypy.jit.metainterp.test.test_optimizefindnode import LLtypeMixin from pypy.jit.tool.oparser import parse @@ -55,6 +54,7 @@ stats = Stats() profiler = jitprof.EmptyProfiler() + warmrunnerdesc = None def log(self, msg, event_kind=None): pass @@ -86,7 +86,7 @@ metainterp.history.inputargs = loop.inputargs[:] # loop_tokens = [] - loop_token = compile_new_loop(metainterp, loop_tokens, [], 0) + loop_token = compile_new_loop(metainterp, loop_tokens, 0) assert loop_tokens == [loop_token] assert loop_token.number == 1 assert staticdata.globaldata.loopnumbering == 2 @@ -102,7 +102,7 @@ metainterp.history.operations = loop.operations[:] metainterp.history.inputargs = loop.inputargs[:] # - loop_token_2 = compile_new_loop(metainterp, loop_tokens, [], 0) + loop_token_2 = compile_new_loop(metainterp, loop_tokens, 0) assert loop_token_2 is loop_token assert loop_tokens == [loop_token] assert len(cpu.seen) == 0 @@ -213,7 +213,7 @@ cpu.set_future_value_int(2, -190) fail_descr = cpu.execute_token(loop_token) try: - fail_descr.handle_fail(FakeMetaInterpSD()) + fail_descr.handle_fail(FakeMetaInterpSD(), None) except FakeMetaInterpSD.ExitFrameWithExceptionRef, e: assert lltype.cast_opaque_ptr(lltype.Ptr(EXC), e.args[1]) == llexc else: Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Tue Nov 16 19:25:26 2010 @@ -144,3 +144,5 @@ res = self.meta_interp(f, [], loop_longevity=3) assert res == 42 self.check_tree_loop_count(2 + 10*4) # 42 :-) + + #XXXcall_assembler Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_optimizeopt.py Tue Nov 16 19:25:26 2010 @@ -41,7 +41,7 @@ b1 = BoxInt() opt = optimizeopt.Optimizer(FakeMetaInterpStaticData(LLtypeMixin.cpu), None) - fdescr = ResumeGuardDescr(history.LoopToken()) + fdescr = ResumeGuardDescr() op = ResOperation(rop.GUARD_TRUE, ['dummy'], None, descr=fdescr) # setup rd data fi0 = resume.FrameInfo(None, "code0", 11) Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmspot.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmspot.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_warmspot.py Tue Nov 16 19:25:26 2010 @@ -388,7 +388,7 @@ assert isinstance(looptoken, FakeLoopToken) self.looptoken = looptoken - def handle_fail(self, metainterp_sd): + def handle_fail(self, metainterp_sd, jitdrivers_sd): no = self.looptoken.no if no == 0: raise metainterp_sd.warmrunnerdesc.DoneWithThisFrameInt(3) Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Tue Nov 16 19:25:26 2010 @@ -255,8 +255,9 @@ debug_print("disabled inlining", loc) debug_stop("jit-disableinlining") - def attach_unoptimized_bridge_from_interp(self, entry_loop_token): - cell = self.jit_cell_at_key(entry_loop_token.outermost_greenkey) + def attach_unoptimized_bridge_from_interp(self, greenkey, + entry_loop_token): + cell = self.jit_cell_at_key(greenkey) cell.counter = -1 old_token = cell.get_entry_loop_token() cell.set_entry_loop_token(entry_loop_token) From afa at codespeak.net Tue Nov 16 19:49:10 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 16 Nov 2010 19:49:10 +0100 (CET) Subject: [pypy-svn] r79164 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101116184910.BB24050810@codespeak.net> Author: afa Date: Tue Nov 16 19:49:09 2010 New Revision: 79164 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Log: Test and fix for the BlockingIOError.written attribute Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Tue Nov 16 19:49:09 2010 @@ -78,6 +78,9 @@ self.state = STATE_ZERO self.buffer = lltype.nullptr(rffi.CCHARP.TO) + + self.abs_pos = 0 # Absolute position inside the raw stream (-1 if + # unknown). self.pos = 0 # Current logical position in the buffer self.raw_pos = 0 # Position of the raw stream in the buffer. @@ -282,11 +285,13 @@ if not e.match(space, space.gettypeobject( W_BlockingIOError.typedef)): raise - self.write_pos += e.written + w_exc = e.get_w_value(space) + assert isinstance(w_exc, W_BlockingIOError) + self.write_pos += w_exc.written self.raw_pos = self.write_pos - written += e.written + written += w_exc.written # re-raise the error - e.written = written + w_exc.written = written raise self.write_pos += n self.raw_pos = self.write_pos @@ -666,6 +671,8 @@ if not e.match(space, space.gettypeobject( W_BlockingIOError.typedef)): raise + w_exc = space.get_w_value(space) + assert isinstance(w_exc, W_BlockingIOError) if self.readable: self._reader_reset_buf() # Make some place by shifting the buffer @@ -687,7 +694,7 @@ self.buffer[self.write_end + i] = data[i] self.write_end += available # Raise previous exception - e.written = available + w_exc.written = available raise # Adjust the raw stream position if it is away from the logical @@ -709,8 +716,10 @@ if not e.match(space, space.gettypeobject( W_BlockingIOError.typedef)): raise - written += e.written - remaining -= e.written + w_exc = e.get_w_value(space) + assert isinstance(w_exc, W_BlockingIOError) + written += w_exc.written + remaining -= w_exc.written if remaining > self.buffer_size: # Can't buffer everything, still buffer as much as # possible @@ -719,7 +728,7 @@ self.raw_pos = 0 self._adjust_position(self.buffer_size) self.write_end = self.buffer_size - e.written = written + self.buffer_size + w_exc.written = written + self.buffer_size raise break written += n Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Tue Nov 16 19:49:09 2010 @@ -211,6 +211,69 @@ assert b.truncate() == 8 assert b.tell() == 8 + def test_write_non_blocking(self): + import _io, io + class MockNonBlockWriterIO(io.RawIOBase): + def __init__(self): + self._write_stack = [] + self._blocker_char = None + + def writable(self): + return True + closed = False + + def pop_written(self): + s = ''.join(self._write_stack) + self._write_stack[:] = [] + return s + + def block_on(self, char): + """Block when a given char is encountered.""" + self._blocker_char = char + + def write(self, b): + try: + b = b.tobytes() + except AttributeError: + pass + n = -1 + if self._blocker_char: + try: + n = b.index(self._blocker_char) + except ValueError: + pass + else: + self._blocker_char = None + self._write_stack.append(b[:n]) + raise _io.BlockingIOError(0, "test blocking", n) + self._write_stack.append(b) + return len(b) + + raw = MockNonBlockWriterIO() + bufio = _io.BufferedWriter(raw, 8) + + assert bufio.write("abcd") == 4 + assert bufio.write("efghi") == 5 + # 1 byte will be written, the rest will be buffered + raw.block_on(b"k") + assert bufio.write("jklmn") == 5 + + # 8 bytes will be written, 8 will be buffered and the rest will be lost + raw.block_on(b"0") + try: + bufio.write(b"opqrwxyz0123456789") + except _io.BlockingIOError as e: + written = e.characters_written + else: + self.fail("BlockingIOError should have been raised") + assert written == 16 + assert raw.pop_written() == "abcdefghijklmnopqrwxyz" + + assert bufio.write("ABCDEFGHI") == 9 + s = raw.pop_written() + # Previously buffered bytes were flushed + assert s.startswith("01234567A") + class AppTestBufferedRWPair: def test_pair(self): import _io From afa at codespeak.net Tue Nov 16 20:58:29 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 16 Nov 2010 20:58:29 +0100 (CET) Subject: [pypy-svn] r79165 - pypy/branch/fast-forward/pypy/module/_io Message-ID: <20101116195829.06D25282B90@codespeak.net> Author: afa Date: Tue Nov 16 20:58:27 2010 New Revision: 79165 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Log: Translation fix Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Tue Nov 16 20:58:27 2010 @@ -671,7 +671,7 @@ if not e.match(space, space.gettypeobject( W_BlockingIOError.typedef)): raise - w_exc = space.get_w_value(space) + w_exc = e.get_w_value(space) assert isinstance(w_exc, W_BlockingIOError) if self.readable: self._reader_reset_buf() From afa at codespeak.net Wed Nov 17 00:12:17 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 17 Nov 2010 00:12:17 +0100 (CET) Subject: [pypy-svn] r79167 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101116231217.99947282B90@codespeak.net> Author: afa Date: Wed Nov 17 00:12:14 2010 New Revision: 79167 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Log: Implement BufferedRandom Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Wed Nov 17 00:12:14 2010 @@ -9,7 +9,8 @@ from pypy.rlib.rarithmetic import r_longlong from pypy.tool.sourcetools import func_renamer from pypy.module._io.interp_iobase import ( - W_IOBase, convert_size, check_readable_w, check_writable_w) + W_IOBase, convert_size, + check_readable_w, check_writable_w, check_seekable_w) from pypy.module._io.interp_io import DEFAULT_BUFFER_SIZE, W_BlockingIOError from pypy.module.thread.os_lock import Lock @@ -171,6 +172,10 @@ return space.call_method(self.w_raw, "seekable") @unwrap_spec('self', ObjSpace) + def isatty_w(self, space): + return space.call_method(self.w_raw, "isatty") + + @unwrap_spec('self', ObjSpace) def repr_w(self, space): typename = space.type(self).getname(space, '?') try: @@ -358,19 +363,8 @@ return space.call_method(self.w_raw, "truncate", w_size) -class W_BufferedReader(BufferedMixin, W_BufferedIOBase): - @unwrap_spec('self', ObjSpace, W_Root, int) - def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE): - self.state = STATE_ZERO - check_readable_w(space, w_raw) - - self.w_raw = w_raw - self.buffer_size = buffer_size - self.readable = True - - self._init(space) - self._reader_reset_buf() - self.state = STATE_OK + # ________________________________________________________________ + # Read methods @unwrap_spec('self', ObjSpace, W_Root) def read_w(self, space, w_size=None): @@ -590,49 +584,8 @@ return res return None -W_BufferedReader.typedef = TypeDef( - '_io.BufferedReader', W_BufferedIOBase.typedef, - __new__ = generic_new_descr(W_BufferedReader), - __init__ = interp2app(W_BufferedReader.descr_init), - - read = interp2app(W_BufferedReader.read_w), - peek = interp2app(W_BufferedReader.peek_w), - read1 = interp2app(W_BufferedReader.read1_w), - - # from the mixin class - __repr__ = interp2app(W_BufferedReader.repr_w), - readable = interp2app(W_BufferedReader.readable_w), - writable = interp2app(W_BufferedReader.writable_w), - seekable = interp2app(W_BufferedReader.seekable_w), - seek = interp2app(W_BufferedReader.seek_w), - tell = interp2app(W_BufferedReader.tell_w), - close = interp2app(W_BufferedReader.close_w), - flush = interp2app(W_BufferedReader.flush_w), - detach = interp2app(W_BufferedReader.detach_w), - truncate = interp2app(W_BufferedReader.truncate_w), - fileno = interp2app(W_BufferedReader.fileno_w), - closed = GetSetProperty(W_BufferedReader.closed_get_w), - name = GetSetProperty(W_BufferedReader.name_get_w), - mode = GetSetProperty(W_BufferedReader.mode_get_w), - ) - -class W_BufferedWriter(BufferedMixin, W_BufferedIOBase): - @unwrap_spec('self', ObjSpace, W_Root, int, int) - def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE, - max_buffer_size=-234): - if max_buffer_size != -234: - self._deprecated_max_buffer_size(space) - - self.state = STATE_ZERO - check_writable_w(space, w_raw) - - self.w_raw = w_raw - self.buffer_size = buffer_size - self.writable = True - - self._init(space) - self._writer_reset_buf() - self.state = STATE_OK + # ____________________________________________________ + # Write methods def _adjust_position(self, new_pos): self.pos = new_pos @@ -762,6 +715,66 @@ self._raw_seek(space, -self._raw_offset(), 1) self._reader_reset_buf() + +class W_BufferedReader(BufferedMixin, W_BufferedIOBase): + @unwrap_spec('self', ObjSpace, W_Root, int) + def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE): + self.state = STATE_ZERO + check_readable_w(space, w_raw) + + self.w_raw = w_raw + self.buffer_size = buffer_size + self.readable = True + + self._init(space) + self._reader_reset_buf() + self.state = STATE_OK + +W_BufferedReader.typedef = TypeDef( + '_io.BufferedReader', W_BufferedIOBase.typedef, + __new__ = generic_new_descr(W_BufferedReader), + __init__ = interp2app(W_BufferedReader.descr_init), + + read = interp2app(W_BufferedReader.read_w), + peek = interp2app(W_BufferedReader.peek_w), + read1 = interp2app(W_BufferedReader.read1_w), + + # from the mixin class + __repr__ = interp2app(W_BufferedReader.repr_w), + readable = interp2app(W_BufferedReader.readable_w), + writable = interp2app(W_BufferedReader.writable_w), + seekable = interp2app(W_BufferedReader.seekable_w), + seek = interp2app(W_BufferedReader.seek_w), + tell = interp2app(W_BufferedReader.tell_w), + close = interp2app(W_BufferedReader.close_w), + flush = interp2app(W_BufferedReader.flush_w), + detach = interp2app(W_BufferedReader.detach_w), + truncate = interp2app(W_BufferedReader.truncate_w), + fileno = interp2app(W_BufferedReader.fileno_w), + isatty = interp2app(W_BufferedReader.isatty_w), + closed = GetSetProperty(W_BufferedReader.closed_get_w), + name = GetSetProperty(W_BufferedReader.name_get_w), + mode = GetSetProperty(W_BufferedReader.mode_get_w), + ) + +class W_BufferedWriter(BufferedMixin, W_BufferedIOBase): + @unwrap_spec('self', ObjSpace, W_Root, int, int) + def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE, + max_buffer_size=-234): + if max_buffer_size != -234: + self._deprecated_max_buffer_size(space) + + self.state = STATE_ZERO + check_writable_w(space, w_raw) + + self.w_raw = w_raw + self.buffer_size = buffer_size + self.writable = True + + self._init(space) + self._writer_reset_buf() + self.state = STATE_OK + W_BufferedWriter.typedef = TypeDef( '_io.BufferedWriter', W_BufferedIOBase.typedef, __new__ = generic_new_descr(W_BufferedWriter), @@ -779,11 +792,12 @@ tell = interp2app(W_BufferedWriter.tell_w), close = interp2app(W_BufferedWriter.close_w), fileno = interp2app(W_BufferedWriter.fileno_w), + isatty = interp2app(W_BufferedWriter.fileno_w), detach = interp2app(W_BufferedWriter.detach_w), truncate = interp2app(W_BufferedWriter.truncate_w), closed = GetSetProperty(W_BufferedWriter.closed_get_w), - name = GetSetProperty(W_BufferedReader.name_get_w), - mode = GetSetProperty(W_BufferedReader.mode_get_w), + name = GetSetProperty(W_BufferedWriter.name_get_w), + mode = GetSetProperty(W_BufferedWriter.mode_get_w), ) def _forward_call(space, w_obj, method, __args__): @@ -864,10 +878,55 @@ **methods ) -class W_BufferedRandom(W_BufferedIOBase): - pass +class W_BufferedRandom(BufferedMixin, W_BufferedIOBase): + @unwrap_spec('self', ObjSpace, W_Root, int, int) + def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE, + max_buffer_size = -234): + if max_buffer_size != -234: + self._deprecated_buffer_size(space) + + self.state = STATE_ZERO + + check_readable_w(space, w_raw) + check_writable_w(space, w_raw) + check_seekable_w(space, w_raw) + + self.w_raw = w_raw + self.buffer_size = buffer_size + self.readable = self.writable = True + + self._init(space) + self._reader_reset_buf() + self._writer_reset_buf() + self.pos = 0 + self.state = STATE_OK + W_BufferedRandom.typedef = TypeDef( '_io.BufferedRandom', W_BufferedIOBase.typedef, __new__ = generic_new_descr(W_BufferedRandom), + __init__ = interp2app(W_BufferedRandom.descr_init), + + read = interp2app(W_BufferedRandom.read_w), + peek = interp2app(W_BufferedRandom.peek_w), + read1 = interp2app(W_BufferedRandom.read1_w), + + write = interp2app(W_BufferedRandom.write_w), + flush = interp2app(W_BufferedRandom.flush_w), + + # from the mixin class + __repr__ = interp2app(W_BufferedRandom.repr_w), + readable = interp2app(W_BufferedRandom.readable_w), + writable = interp2app(W_BufferedRandom.writable_w), + seekable = interp2app(W_BufferedRandom.seekable_w), + seek = interp2app(W_BufferedRandom.seek_w), + tell = interp2app(W_BufferedRandom.tell_w), + close = interp2app(W_BufferedRandom.close_w), + detach = interp2app(W_BufferedRandom.detach_w), + truncate = interp2app(W_BufferedRandom.truncate_w), + fileno = interp2app(W_BufferedRandom.fileno_w), + isatty = interp2app(W_BufferedRandom.isatty_w), + closed = GetSetProperty(W_BufferedRandom.closed_get_w), + name = GetSetProperty(W_BufferedRandom.name_get_w), + mode = GetSetProperty(W_BufferedRandom.mode_get_w), ) Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Wed Nov 17 00:12:14 2010 @@ -299,3 +299,19 @@ return False raises(IOError, _io.BufferedRWPair, _io.BytesIO(), NotWritable()) + +class AppTestBufferedRandom: + def setup_class(cls): + cls.space = gettestobjspace(usemodules=['_io']) + tmpfile = udir.join('tmpfile') + tmpfile.write("a\nb\nc", mode='wb') + cls.w_tmpfile = cls.space.wrap(str(tmpfile)) + + def test_simple_read(self): + import _io + raw = _io.FileIO(self.tmpfile, 'rb+') + f = _io.BufferedRandom(raw) + assert f.read(3) == 'a\nb' + f.write('xxxx') + f.seek(0) + assert f.read() == 'a\nbxxxx' From afa at codespeak.net Wed Nov 17 10:39:25 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 17 Nov 2010 10:39:25 +0100 (CET) Subject: [pypy-svn] r79171 - pypy/branch/fast-forward/pypy/module/_io Message-ID: <20101117093925.A5922282BD6@codespeak.net> Author: afa Date: Wed Nov 17 10:39:22 2010 New Revision: 79171 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Log: Translation fix Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Wed Nov 17 10:39:22 2010 @@ -883,7 +883,7 @@ def descr_init(self, space, w_raw, buffer_size=DEFAULT_BUFFER_SIZE, max_buffer_size = -234): if max_buffer_size != -234: - self._deprecated_buffer_size(space) + self._deprecated_max_buffer_size(space) self.state = STATE_ZERO From cfbolz at codespeak.net Wed Nov 17 10:44:47 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 17 Nov 2010 10:44:47 +0100 (CET) Subject: [pypy-svn] r79172 - in pypy/trunk/pypy: interpreter interpreter/test module/array module/pypyjit/test Message-ID: <20101117094447.0B37F50810@codespeak.net> Author: cfbolz Date: Wed Nov 17 10:44:46 2010 New Revision: 79172 Modified: pypy/trunk/pypy/interpreter/argument.py pypy/trunk/pypy/interpreter/test/test_argument.py pypy/trunk/pypy/module/array/interp_array.py pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Log: svn merge -r79092:HEAD http://codespeak.net/svn/pypy/branch/jit-starargs make the jit handle better calling functions with *args. Modified: pypy/trunk/pypy/interpreter/argument.py ============================================================================== --- pypy/trunk/pypy/interpreter/argument.py (original) +++ pypy/trunk/pypy/interpreter/argument.py Wed Nov 17 10:44:46 2010 @@ -64,7 +64,7 @@ return not self == other - # make it look tuply for the annotator + # make it look tuply for its use in the annotator def __len__(self): return 3 @@ -103,10 +103,11 @@ make_sure_not_resized(self.keywords_w) make_sure_not_resized(self.arguments_w) - if ((w_stararg is not None and w_stararg) or - (w_starstararg is not None and w_starstararg)): - self._combine_wrapped(w_stararg, w_starstararg) - # if we have a call where * or ** args are used at the callsite + if w_stararg is not None and space.is_true(w_stararg): + self._combine_starargs_wrapped(w_stararg) + if w_starstararg is not None and space.is_true(w_starstararg): + self._combine_starstarargs_wrapped(w_starstararg) + # if we have a call where **args are used at the callsite # we shouldn't let the JIT see the argument matching self._dont_jit = True else: @@ -142,42 +143,48 @@ def _combine_wrapped(self, w_stararg, w_starstararg): "unpack the *arg and **kwd into arguments_w and keywords_w" - # unpack the * arguments if w_stararg is not None: - self.arguments_w = (self.arguments_w + - self.space.fixedview(w_stararg)) - # unpack the ** arguments + self._combine_starargs_wrapped(w_stararg) if w_starstararg is not None: - space = self.space - if not space.is_true(space.isinstance(w_starstararg, space.w_dict)): + self._combine_starstarargs_wrapped(w_starstararg) + + def _combine_starargs_wrapped(self, w_stararg): + # unpack the * arguments + self.arguments_w = (self.arguments_w + + self.space.fixedview(w_stararg)) + + def _combine_starstarargs_wrapped(self, w_starstararg): + # unpack the ** arguments + space = self.space + if not space.is_true(space.isinstance(w_starstararg, space.w_dict)): + raise OperationError(space.w_TypeError, + space.wrap("argument after ** must be " + "a dictionary")) + keywords_w = [None] * space.int_w(space.len(w_starstararg)) + keywords = [None] * space.int_w(space.len(w_starstararg)) + i = 0 + for w_key in space.unpackiterable(w_starstararg): + try: + key = space.str_w(w_key) + except OperationError, e: + if not e.match(space, space.w_TypeError): + raise raise OperationError(space.w_TypeError, - space.wrap("argument after ** must be " - "a dictionary")) - keywords_w = [None] * space.int_w(space.len(w_starstararg)) - keywords = [None] * space.int_w(space.len(w_starstararg)) - i = 0 - for w_key in space.unpackiterable(w_starstararg): - try: - key = space.str_w(w_key) - except OperationError, e: - if not e.match(space, space.w_TypeError): - raise - raise OperationError(space.w_TypeError, - space.wrap("keywords must be strings")) - if self.keywords and key in self.keywords: - raise operationerrfmt(self.space.w_TypeError, - "got multiple values " - "for keyword argument " - "'%s'", key) - keywords[i] = key - keywords_w[i] = space.getitem(w_starstararg, w_key) - i += 1 - if self.keywords is None: - self.keywords = keywords - self.keywords_w = keywords_w - else: - self.keywords = self.keywords + keywords - self.keywords_w = self.keywords_w + keywords_w + space.wrap("keywords must be strings")) + if self.keywords and key in self.keywords: + raise operationerrfmt(self.space.w_TypeError, + "got multiple values " + "for keyword argument " + "'%s'", key) + keywords[i] = key + keywords_w[i] = space.getitem(w_starstararg, w_key) + i += 1 + if self.keywords is None: + self.keywords = keywords + self.keywords_w = keywords_w + else: + self.keywords = self.keywords + keywords + self.keywords_w = self.keywords_w + keywords_w def fixedunpack(self, argcount): """The simplest argument parsing: get the 'argcount' arguments, @@ -226,6 +233,10 @@ # argnames = list of formal parameter names # scope_w = resulting list of wrapped values # + + # some comments about the JIT: it assumes that signature is a constant, + # so all values coming from there can be assumed constant. It assumes + # that the length of the defaults_w does not vary too much. co_argcount = signature.num_argnames() # expected formal arguments, without */** has_vararg = signature.has_vararg() has_kwarg = signature.has_kwarg() @@ -245,12 +256,6 @@ args_w = self.arguments_w num_args = len(args_w) - keywords = self.keywords - keywords_w = self.keywords_w - num_kwds = 0 - if keywords is not None: - num_kwds = len(keywords) - avail = num_args + upfront if input_argcount < co_argcount: @@ -260,15 +265,24 @@ else: take = num_args + # letting the JIT unroll this loop is safe, because take is always + # smaller than co_argcount for i in range(take): scope_w[i + input_argcount] = args_w[i] input_argcount += take + keywords = self.keywords + keywords_w = self.keywords_w + num_kwds = 0 + if keywords is not None: + num_kwds = len(keywords) # the code assumes that keywords can potentially be large, but that # argnames is typically not too large num_remainingkwds = num_kwds used_keywords = None if keywords: + # letting JIT unroll the loop is *only* safe if the callsite didn't + # use **args because num_kwds can be arbitrarily large otherwise. used_keywords = [False] * num_kwds for i in range(num_kwds): name = keywords[i] @@ -276,7 +290,7 @@ if j < 0: continue elif j < input_argcount: - # check that no keyword argument conflicts with these note + # check that no keyword argument conflicts with these. note # that for this purpose we ignore the first blindargs, # which were put into place by prepend(). This way, # keywords do not conflict with the hidden extra argument Modified: pypy/trunk/pypy/interpreter/test/test_argument.py ============================================================================== --- pypy/trunk/pypy/interpreter/test/test_argument.py (original) +++ pypy/trunk/pypy/interpreter/test/test_argument.py Wed Nov 17 10:44:46 2010 @@ -52,12 +52,17 @@ assert y == "d" assert z == "e" +class dummy_wrapped_dict(dict): + def __nonzero__(self): + raise NotImplementedError class DummySpace(object): def newtuple(self, items): return tuple(items) def is_true(self, obj): + if isinstance(obj, dummy_wrapped_dict): + return bool(dict(obj)) return bool(obj) def fixedview(self, it): @@ -229,7 +234,7 @@ kwds_w = dict(kwds[:i]) keywords = kwds_w.keys() keywords_w = kwds_w.values() - w_kwds = dict(kwds[i:]) + w_kwds = dummy_wrapped_dict(kwds[i:]) if i == 2: w_kwds = None assert len(keywords) == len(keywords_w) @@ -265,7 +270,7 @@ kwds_w = dict(kwds[:i]) keywords = kwds_w.keys() keywords_w = kwds_w.values() - w_kwds = dict(kwds[i:]) + w_kwds = dummy_wrapped_dict(kwds[i:]) if i == 3: w_kwds = None args = Arguments(space, [1, 2], keywords, keywords_w, w_starstararg=w_kwds) Modified: pypy/trunk/pypy/module/array/interp_array.py ============================================================================== --- pypy/trunk/pypy/module/array/interp_array.py (original) +++ pypy/trunk/pypy/module/array/interp_array.py Wed Nov 17 10:44:46 2010 @@ -27,7 +27,7 @@ typecode = typecode[0] if space.is_w(w_cls, space.gettypeobject(W_ArrayBase.typedef)): - if len(w_args.keywords_w) > 0: + if w_args.keywords: # XXX this might be forbidden fishing msg = 'array.array() does not take keyword arguments' raise OperationError(space.w_TypeError, space.wrap(msg)) Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Wed Nov 17 10:44:46 2010 @@ -377,10 +377,75 @@ ([1000], 49500), ([10000], 495000), ([100000], 4950000)) - assert len(self.loops) == 2 + assert len(self.loops) == 3 op, = self.get_by_bytecode("CALL_FUNCTION_KW") # XXX a bit too many guards, but better than before - assert len(op.get_opnames("guard")) <= 10 + assert len(op.get_opnames("guard")) <= 12 + + def test_stararg_virtual(self): + self.run_source(''' + d = {} + + def g(*args): + return len(args) + def h(a, b, c): + return c + + def main(x): + s = 0 + for i in range(x): + l = [i, x, 2] + s += g(*l) + s += h(*l) + s += g(i, x, 2) + for i in range(x): + l = [x, 2] + s += g(i, *l) + s += h(i, *l) + return s + ''', 100000, ([100], 1300), + ([1000], 13000), + ([10000], 130000), + ([100000], 1300000)) + assert len(self.loops) == 2 + ops = self.get_by_bytecode("CALL_FUNCTION_VAR") + assert len(ops) == 4 + for op in ops: + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 + + ops = self.get_by_bytecode("CALL_FUNCTION") + for op in ops: + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 + + def test_stararg(self): + self.run_source(''' + d = {} + + def g(*args): + return args[-1] + def h(*args): + return len(args) + + def main(x): + s = 0 + l = [] + i = 0 + while i < x: + l.append(1) + s += g(*l) + i = h(*l) + return s + ''', 100000, ([100], 100), + ([1000], 1000), + ([2000], 2000), + ([4000], 4000)) + assert len(self.loops) == 1 + ops = self.get_by_bytecode("CALL_FUNCTION_VAR") + for op in ops: + assert len(op.get_opnames("new_with_vtable")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 def test_virtual_instance(self): self.run_source(''' From cfbolz at codespeak.net Wed Nov 17 10:45:11 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 17 Nov 2010 10:45:11 +0100 (CET) Subject: [pypy-svn] r79173 - pypy/branch/jit-starargs Message-ID: <20101117094511.8A444282BEF@codespeak.net> Author: cfbolz Date: Wed Nov 17 10:45:10 2010 New Revision: 79173 Removed: pypy/branch/jit-starargs/ Log: remove merged branch From arigo at codespeak.net Wed Nov 17 10:53:56 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 10:53:56 +0100 (CET) Subject: [pypy-svn] r79174 - pypy/trunk/pypy/rpython/memory/test Message-ID: <20101117095356.BAC2D50811@codespeak.net> Author: arigo Date: Wed Nov 17 10:53:55 2010 New Revision: 79174 Modified: pypy/trunk/pypy/rpython/memory/test/test_gc.py Log: Two new tests, trying to be about weakrefs to objects with a del. But they crash earlier :-( Modified: pypy/trunk/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/trunk/pypy/rpython/memory/test/test_gc.py Wed Nov 17 10:53:55 2010 @@ -278,6 +278,61 @@ res = self.interpret(f, []) assert res + def test_cycle_with_weakref_and_del(self): + import weakref, gc + class A(object): + count = 0 + a = A() + class B(object): + def __del__(self): + # when __del__ is called, the weakref should have been cleared + if self.ref() is None: + a.count += 1 # ok + else: + a.count = 666 # not ok + class C(object): + pass + def g(): + c = C() + c.b = B() + ref = weakref.ref(c) + c.b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = a.count == 1 and (ref() is None) + return result + res = self.interpret(f, []) + assert res + + def test_weakref_to_object_with_finalizer_ordering(self): + import weakref, gc + class A(object): + count = 0 + a = A() + class B(object): + def __del__(self): + # when __del__ is called, the weakref should have been cleared + if self.ref() is None: + a.count += 1 # ok + else: + a.count = 666 # not ok + def g(): + b = B() + ref = weakref.ref(b) + b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = a.count == 1 and (ref() is None) + return result + res = self.interpret(f, []) + assert res + def test_id(self): class A(object): pass From arigo at codespeak.net Wed Nov 17 11:08:40 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 11:08:40 +0100 (CET) Subject: [pypy-svn] r79175 - pypy/trunk/pypy/rpython/memory/test Message-ID: <20101117100840.9DED8282B9E@codespeak.net> Author: arigo Date: Wed Nov 17 11:08:36 2010 New Revision: 79175 Modified: pypy/trunk/pypy/rpython/memory/test/test_gc.py Log: Simplify the failure. Modified: pypy/trunk/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/trunk/pypy/rpython/memory/test/test_gc.py Wed Nov 17 11:08:36 2010 @@ -278,6 +278,25 @@ res = self.interpret(f, []) assert res + def test_bug_1(self): + import weakref + class B(object): + def __del__(self): + pass + def g(): + b = B() + ref = weakref.ref(b) + b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = (ref() is None) + return result + res = self.interpret(f, []) + assert res + def test_cycle_with_weakref_and_del(self): import weakref, gc class A(object): From arigo at codespeak.net Wed Nov 17 11:13:08 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 11:13:08 +0100 (CET) Subject: [pypy-svn] r79176 - pypy/trunk/pypy/rpython/memory/test Message-ID: <20101117101308.07890282B90@codespeak.net> Author: arigo Date: Wed Nov 17 11:13:07 2010 New Revision: 79176 Modified: pypy/trunk/pypy/rpython/memory/test/test_gc.py Log: Simplify even more. Modified: pypy/trunk/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/trunk/pypy/rpython/memory/test/test_gc.py Wed Nov 17 11:13:07 2010 @@ -281,11 +281,11 @@ def test_bug_1(self): import weakref class B(object): - def __del__(self): - pass + pass def g(): b = B() - ref = weakref.ref(b) + llop.gc__collect(lltype.Void) # force 'b' to be old + ref = weakref.ref(B()) b.ref = ref return ref def f(): From arigo at codespeak.net Wed Nov 17 11:32:03 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 11:32:03 +0100 (CET) Subject: [pypy-svn] r79177 - in pypy/trunk/pypy/rpython/memory: . gc Message-ID: <20101117103203.621B5282B90@codespeak.net> Author: arigo Date: Wed Nov 17 11:32:01 2010 New Revision: 79177 Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py pypy/trunk/pypy/rpython/memory/support.py Log: Don't use length() on AddressStack, as it returns an estimate only. Just use non_empty(). Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Wed Nov 17 11:32:01 2010 @@ -971,7 +971,7 @@ # # Now all live nursery objects should be out. Update the # young weakrefs' targets. - if self.young_objects_with_weakrefs.length() > 0: + if self.young_objects_with_weakrefs.non_empty(): self.invalidate_young_weakrefs() # # Clear this mapping. Modified: pypy/trunk/pypy/rpython/memory/support.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/support.py (original) +++ pypy/trunk/pypy/rpython/memory/support.py Wed Nov 17 11:32:01 2010 @@ -112,7 +112,7 @@ cur = next free_non_gc_object(self) - def length(self): + def _length_estimate(self): chunk = self.chunk count = self.used_in_last_chunk while chunk: @@ -135,7 +135,7 @@ foreach._annspecialcase_ = 'specialize:arg(1)' def stack2dict(self): - result = AddressDict(self.length()) + result = AddressDict(self._length_estimate()) self.foreach(_add_in_dict, result) return result From afa at codespeak.net Wed Nov 17 11:32:07 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 17 Nov 2010 11:32:07 +0100 (CET) Subject: [pypy-svn] r79178 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101117103207.98969282BEC@codespeak.net> Author: afa Date: Wed Nov 17 11:32:05 2010 New Revision: 79178 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Log: Add a __dict__ to all IO classes Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Wed Nov 17 11:32:05 2010 @@ -1,6 +1,6 @@ from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root from pypy.interpreter.typedef import ( - TypeDef, GetSetProperty, generic_new_descr) + TypeDef, GetSetProperty, generic_new_descr, descr_get_dict, descr_set_dict) from pypy.interpreter.gateway import interp2app, Arguments, unwrap_spec from pypy.interpreter.error import OperationError, operationerrfmt from pypy.rlib.rstring import StringBuilder @@ -41,8 +41,12 @@ # `__IOBase_closed` and call flush() by itself, but it is redundant # with whatever behaviour a non-trivial derived class will implement. self.space = space + self.w_dict = space.newdict() self.__IOBase_closed = False + def getdict(self): + return self.w_dict + def _closed(self, space): # This gets the derived attribute, which is *not* __IOBase_closed # in most cases! @@ -241,6 +245,7 @@ _checkWritable = interp2app(check_writable_w), _checkSeekable = interp2app(check_seekable_w), closed = GetSetProperty(W_IOBase.closed_get_w), + __dict__ = GetSetProperty(descr_get_dict, descr_set_dict, cls=W_IOBase), readline = interp2app(W_IOBase.readline_w), readlines = interp2app(W_IOBase.readlines_w), Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Wed Nov 17 11:32:05 2010 @@ -52,6 +52,24 @@ assert e.strerror == "test blocking" assert e.characters_written == 123 + def test_dict(self): + import _io + f = _io.BytesIO() + f.x = 42 + assert f.x == 42 + # + def write(data): + try: + data = data.tobytes().upper() + except AttributeError: + data = data.upper() + return _io.BytesIO.write(f, data) + f.write = write + bufio = _io.BufferedWriter(f) + bufio.write("abc") + bufio.flush() + assert f.getvalue() == "ABC" + def test_destructor(self): import io io.IOBase() From arigo at codespeak.net Wed Nov 17 11:43:21 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 11:43:21 +0100 (CET) Subject: [pypy-svn] r79179 - pypy/trunk/pypy/rpython/lltypesystem Message-ID: <20101117104321.9B3EC282B90@codespeak.net> Author: arigo Date: Wed Nov 17 11:43:20 2010 New Revision: 79179 Modified: pypy/trunk/pypy/rpython/lltypesystem/llmemory.py Log: Fix. It was about the class _gctransformed_wref, so it means that it was not an issue after translation. Modified: pypy/trunk/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/trunk/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/trunk/pypy/rpython/lltypesystem/llmemory.py Wed Nov 17 11:43:20 2010 @@ -766,8 +766,10 @@ return '<%s>' % (self,) def __str__(self): return 'gctransformed_wref(%s)' % (self._ptr,) - def _normalizedcontainer(self): - return self._ptr._obj + def _normalizedcontainer(self, check=True): + return self._ptr._getobj(check=check)._normalizedcontainer(check=check) + def _was_freed(self): + return self._ptr._was_freed() # ____________________________________________________________ From afa at codespeak.net Wed Nov 17 11:52:58 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 17 Nov 2010 11:52:58 +0100 (CET) Subject: [pypy-svn] r79180 - pypy/branch/fast-forward/pypy/module/_io Message-ID: <20101117105258.23AC9282BEF@codespeak.net> Author: afa Date: Wed Nov 17 11:52:56 2010 New Revision: 79180 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py Log: Be sure to call the base __init__ Modified: pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_stringio.py Wed Nov 17 11:52:56 2010 @@ -5,7 +5,8 @@ from pypy.interpreter.baseobjspace import ObjSpace, W_Root class W_StringIO(W_TextIOBase): - def __init__(self): + def __init__(self, space): + W_TextIOBase.__init__(self, space) self.buf = [] self.pos = 0 @@ -17,7 +18,7 @@ @unwrap_spec(ObjSpace, W_Root) def descr_new(space, w_subtype): self = space.allocate_instance(W_StringIO, w_subtype) - W_StringIO.__init__(self) + W_StringIO.__init__(self, space) return space.wrap(self) def resize_buffer(self, newlength): From afa at codespeak.net Wed Nov 17 11:54:02 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 17 Nov 2010 11:54:02 +0100 (CET) Subject: [pypy-svn] r79181 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101117105402.B3FDA282BF0@codespeak.net> Author: afa Date: Wed Nov 17 11:54:01 2010 New Revision: 79181 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Log: Fix in BufferedReader.read() when the underlying stream.read() must be called twice Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Wed Nov 17 11:54:01 2010 @@ -497,8 +497,8 @@ data = self._raw_read(space, length) size = len(data) if size > 0: - for i in range(start, start + size): - self.buffer[i] = data[i] + for i in range(size): + self.buffer[start + i] = data[i] self.read_end = self.raw_pos = start + size return size Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Wed Nov 17 11:54:01 2010 @@ -32,6 +32,20 @@ assert f.read(3) == "" f.close() + def test_slow_provider(self): + import _io + class MockIO(_io._IOBase): + def readable(self): + return True + def read(self, n=-1): # PyPy uses read() + return "abc" + def readinto(self, buf): # CPython uses readinto() + buf[:3] = "abc" + return 3 + bufio = _io.BufferedReader(MockIO()) + r = bufio.read(5) + assert r == "abcab" + def test_peek(self): import _io raw = _io.FileIO(self.tmpfile) From arigo at codespeak.net Wed Nov 17 11:56:06 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 11:56:06 +0100 (CET) Subject: [pypy-svn] r79182 - in pypy/trunk/pypy/rpython/memory: gc test Message-ID: <20101117105606.11CCA282BF3@codespeak.net> Author: arigo Date: Wed Nov 17 11:56:05 2010 New Revision: 79182 Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py pypy/trunk/pypy/rpython/memory/gc/semispace.py pypy/trunk/pypy/rpython/memory/test/test_gc.py Log: Fixing the test is as easy as swapping two phases of a major collection: we have to *first* invalidate weakrefs and *then* trace from the finalizers. Skipped in TestMarkCompactGC for now. Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Wed Nov 17 11:56:05 2010 @@ -1205,6 +1205,10 @@ self.collect_roots() self.visit_all_objects() # + # Weakref support: clear the weak pointers to dying objects + if self.old_objects_with_weakrefs.non_empty(): + self.invalidate_old_weakrefs() + # # Finalizer support: adds the flag GCFLAG_VISITED to all objects # with a finalizer and all objects reachable from there (and also # moves some objects from 'objects_with_finalizers' to @@ -1214,10 +1218,6 @@ # self.objects_to_trace.delete() # - # Weakref support: clear the weak pointers to dying objects - if self.old_objects_with_weakrefs.non_empty(): - self.invalidate_old_weakrefs() - # # Walk all rawmalloced objects and free the ones that don't # have the GCFLAG_VISITED flag. self.free_unvisited_rawmalloc_objects() Modified: pypy/trunk/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/semispace.py Wed Nov 17 11:56:05 2010 @@ -266,10 +266,10 @@ if self.run_finalizers.non_empty(): self.update_run_finalizers() scan = self.scan_copied(scan) - if self.objects_with_finalizers.non_empty(): - scan = self.deal_with_objects_with_finalizers(scan) if self.objects_with_weakrefs.non_empty(): self.invalidate_weakrefs() + if self.objects_with_finalizers.non_empty(): + scan = self.deal_with_objects_with_finalizers(scan) self.update_objects_with_id() self.finished_full_collect() self.debug_check_consistency() Modified: pypy/trunk/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/trunk/pypy/rpython/memory/test/test_gc.py Wed Nov 17 11:56:05 2010 @@ -306,7 +306,7 @@ def __del__(self): # when __del__ is called, the weakref should have been cleared if self.ref() is None: - a.count += 1 # ok + a.count += 10 # ok else: a.count = 666 # not ok class C(object): @@ -321,10 +321,10 @@ ref = g() llop.gc__collect(lltype.Void) llop.gc__collect(lltype.Void) - result = a.count == 1 and (ref() is None) + result = a.count + (ref() is None) return result res = self.interpret(f, []) - assert res + assert res == 11 def test_weakref_to_object_with_finalizer_ordering(self): import weakref, gc @@ -335,7 +335,7 @@ def __del__(self): # when __del__ is called, the weakref should have been cleared if self.ref() is None: - a.count += 1 # ok + a.count += 10 # ok else: a.count = 666 # not ok def g(): @@ -347,10 +347,10 @@ ref = g() llop.gc__collect(lltype.Void) llop.gc__collect(lltype.Void) - result = a.count == 1 and (ref() is None) + result = a.count + (ref() is None) return result res = self.interpret(f, []) - assert res + assert res == 11 def test_id(self): class A(object): @@ -731,6 +731,9 @@ def test_finalizer_order(self): py.test.skip("Not implemented yet") + def test_weakref_to_object_with_finalizer_ordering(self): + py.test.skip("Not implemented yet") + class TestHybridGC(TestGenerationalGC): from pypy.rpython.memory.gc.hybrid import HybridGC as GCClass GC_CAN_MALLOC_NONMOVABLE = True From arigo at codespeak.net Wed Nov 17 11:58:02 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 11:58:02 +0100 (CET) Subject: [pypy-svn] r79183 - in pypy/branch/jit-free/pypy/rpython: lltypesystem memory memory/gc memory/test Message-ID: <20101117105802.72DAD282BF5@codespeak.net> Author: arigo Date: Wed Nov 17 11:58:00 2010 New Revision: 79183 Modified: pypy/branch/jit-free/pypy/rpython/lltypesystem/llmemory.py pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py pypy/branch/jit-free/pypy/rpython/memory/gc/semispace.py pypy/branch/jit-free/pypy/rpython/memory/support.py pypy/branch/jit-free/pypy/rpython/memory/test/test_gc.py Log: Merge r79174-r79182 from trunk. Modified: pypy/branch/jit-free/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/branch/jit-free/pypy/rpython/lltypesystem/llmemory.py Wed Nov 17 11:58:00 2010 @@ -766,8 +766,10 @@ return '<%s>' % (self,) def __str__(self): return 'gctransformed_wref(%s)' % (self._ptr,) - def _normalizedcontainer(self): - return self._ptr._obj + def _normalizedcontainer(self, check=True): + return self._ptr._getobj(check=check)._normalizedcontainer(check=check) + def _was_freed(self): + return self._ptr._was_freed() # ____________________________________________________________ Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py Wed Nov 17 11:58:00 2010 @@ -922,7 +922,7 @@ # # Now all live nursery objects should be out. Update the # young weakrefs' targets. - if self.young_objects_with_weakrefs.length() > 0: + if self.young_objects_with_weakrefs.non_empty(): self.invalidate_young_weakrefs() # # Clear this mapping. @@ -1156,6 +1156,10 @@ self.collect_roots() self.visit_all_objects() # + # Weakref support: clear the weak pointers to dying objects + if self.old_objects_with_weakrefs.non_empty(): + self.invalidate_old_weakrefs() + # # Finalizer support: adds the flag GCFLAG_VISITED to all objects # with a finalizer and all objects reachable from there (and also # moves some objects from 'objects_with_finalizers' to @@ -1165,10 +1169,6 @@ # self.objects_to_trace.delete() # - # Weakref support: clear the weak pointers to dying objects - if self.old_objects_with_weakrefs.non_empty(): - self.invalidate_old_weakrefs() - # # Walk all rawmalloced objects and free the ones that don't # have the GCFLAG_VISITED flag. self.free_unvisited_rawmalloc_objects() Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/semispace.py Wed Nov 17 11:58:00 2010 @@ -266,10 +266,10 @@ if self.run_finalizers.non_empty(): self.update_run_finalizers() scan = self.scan_copied(scan) - if self.objects_with_finalizers.non_empty(): - scan = self.deal_with_objects_with_finalizers(scan) if self.objects_with_weakrefs.non_empty(): self.invalidate_weakrefs() + if self.objects_with_finalizers.non_empty(): + scan = self.deal_with_objects_with_finalizers(scan) self.update_objects_with_id() self.finished_full_collect() self.debug_check_consistency() Modified: pypy/branch/jit-free/pypy/rpython/memory/support.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/support.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/support.py Wed Nov 17 11:58:00 2010 @@ -112,7 +112,7 @@ cur = next free_non_gc_object(self) - def length(self): + def _length_estimate(self): chunk = self.chunk count = self.used_in_last_chunk while chunk: @@ -135,7 +135,7 @@ foreach._annspecialcase_ = 'specialize:arg(1)' def stack2dict(self): - result = AddressDict(self.length()) + result = AddressDict(self._length_estimate()) self.foreach(_add_in_dict, result) return result Modified: pypy/branch/jit-free/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/test/test_gc.py Wed Nov 17 11:58:00 2010 @@ -278,6 +278,80 @@ res = self.interpret(f, []) assert res + def test_bug_1(self): + import weakref + class B(object): + pass + def g(): + b = B() + llop.gc__collect(lltype.Void) # force 'b' to be old + ref = weakref.ref(B()) + b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = (ref() is None) + return result + res = self.interpret(f, []) + assert res + + def test_cycle_with_weakref_and_del(self): + import weakref, gc + class A(object): + count = 0 + a = A() + class B(object): + def __del__(self): + # when __del__ is called, the weakref should have been cleared + if self.ref() is None: + a.count += 10 # ok + else: + a.count = 666 # not ok + class C(object): + pass + def g(): + c = C() + c.b = B() + ref = weakref.ref(c) + c.b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = a.count + (ref() is None) + return result + res = self.interpret(f, []) + assert res == 11 + + def test_weakref_to_object_with_finalizer_ordering(self): + import weakref, gc + class A(object): + count = 0 + a = A() + class B(object): + def __del__(self): + # when __del__ is called, the weakref should have been cleared + if self.ref() is None: + a.count += 10 # ok + else: + a.count = 666 # not ok + def g(): + b = B() + ref = weakref.ref(b) + b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = a.count + (ref() is None) + return result + res = self.interpret(f, []) + assert res == 11 + def test_id(self): class A(object): pass @@ -657,6 +731,9 @@ def test_finalizer_order(self): py.test.skip("Not implemented yet") + def test_weakref_to_object_with_finalizer_ordering(self): + py.test.skip("Not implemented yet") + class TestHybridGC(TestGenerationalGC): from pypy.rpython.memory.gc.hybrid import HybridGC as GCClass GC_CAN_MALLOC_NONMOVABLE = True From afa at codespeak.net Wed Nov 17 12:05:09 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 17 Nov 2010 12:05:09 +0100 (CET) Subject: [pypy-svn] r79184 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101117110509.589B5282BF5@codespeak.net> Author: afa Date: Wed Nov 17 12:05:07 2010 New Revision: 79184 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py Log: BytesIO.truncate() Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py Wed Nov 17 12:05:07 2010 @@ -82,6 +82,25 @@ return space.wrap(length) + @unwrap_spec('self', ObjSpace, W_Root) + def truncate_w(self, space, w_size=None): + self._check_closed(space) + + if space.is_w(w_size, space.w_None): + size = self.pos + else: + size = space.r_longlong_w(w_size) + + if size < 0: + raise OperationError(space.w_ValueError, space.wrap( + "negative size value")) + + if size < self.string_size: + self.string_size = size + del self.buf[size:] + + return space.wrap(size) + @unwrap_spec('self', ObjSpace) def getvalue_w(self, space): self._check_closed(space) @@ -139,6 +158,7 @@ read = interp2app(W_BytesIO.read_w), write = interp2app(W_BytesIO.write_w), + truncate = interp2app(W_BytesIO.truncate_w), getvalue = interp2app(W_BytesIO.getvalue_w), seek = interp2app(W_BytesIO.seek_w), tell = interp2app(W_BytesIO.tell_w), Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bytesio.py Wed Nov 17 12:05:07 2010 @@ -35,3 +35,10 @@ assert f.seek(-1, 2) == 4 assert f.tell() == 4 assert f.seek(0) == 0 + + def test_truncate(self): + import _io + f = _io.BytesIO("hello") + f.seek(3) + assert f.truncate() == 3 + assert f.getvalue() == "hel" From arigo at codespeak.net Wed Nov 17 12:50:58 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 12:50:58 +0100 (CET) Subject: [pypy-svn] r79185 - pypy/branch/jit-free/pypy/translator/tool Message-ID: <20101117115058.C2426282BEA@codespeak.net> Author: arigo Date: Wed Nov 17 12:50:57 2010 New Revision: 79185 Modified: pypy/branch/jit-free/pypy/translator/tool/reftracker.py Log: Whack at reftracker.py to display more meaningful names, e.g. the name of the frames (instead of just "frame object at 0x1234") Modified: pypy/branch/jit-free/pypy/translator/tool/reftracker.py ============================================================================== --- pypy/branch/jit-free/pypy/translator/tool/reftracker.py (original) +++ pypy/branch/jit-free/pypy/translator/tool/reftracker.py Wed Nov 17 12:50:57 2010 @@ -3,7 +3,7 @@ Usage: call track(obj). """ -import autopath, sys, os +import autopath, sys, os, types import gc from pypy.translator.tool.graphpage import GraphPage, DotGen from pypy.tool.uid import uid @@ -39,7 +39,7 @@ if o2 is None: continue addedge(objectlist[i], o2) - id2typename[uid(o2)] = type(o2).__name__ + id2typename[uid(o2)] = self.shortrepr(o2) del o2 for o2 in self.get_referrers(objectlist[i]): if o2 is None: @@ -47,7 +47,7 @@ if type(o2) is list and o2 and o2[0] is MARKER: continue addedge(o2, objectlist[i]) - id2typename[uid(o2)] = type(o2).__name__ + id2typename[uid(o2)] = self.shortrepr(o2) del o2 for ids, label in edges.items(): @@ -82,13 +82,23 @@ return self.newpage(objectlist) def formatobject(self, o): + header = self.shortrepr(o, compact=False) + secondline = repr(o.__class__) + return header, secondline, repr(o) + + def shortrepr(self, o, compact=True): + t = type(o) + if t is types.FrameType: + if compact: + return 'frame %r' % (o.f_code.co_name,) + else: + return 'frame %r' % (o.f_code,) s = repr(o) if len(s) > 50: - linktext = s s = s[:20] + ' ... ' + s[-20:] - else: - linktext = '' - return type(o).__name__, s, linktext + if s.startswith('<') and s.endswith('>'): + s = s[1:-1] + return s def edgelabel(self, o1, o2): return '' From afa at codespeak.net Wed Nov 17 13:01:23 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 17 Nov 2010 13:01:23 +0100 (CET) Subject: [pypy-svn] r79186 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101117120123.B2EDB282BEC@codespeak.net> Author: afa Date: Wed Nov 17 13:01:22 2010 New Revision: 79186 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Log: All io objects are weakrefable. Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Wed Nov 17 13:01:22 2010 @@ -1,6 +1,5 @@ from pypy.interpreter.typedef import ( - TypeDef, interp_attrproperty, interp_attrproperty_w, GetSetProperty, - make_weakref_descr) + TypeDef, interp_attrproperty, interp_attrproperty_w, GetSetProperty) from pypy.interpreter.gateway import interp2app, unwrap_spec, Arguments from pypy.interpreter.baseobjspace import ObjSpace, W_Root from pypy.interpreter.error import OperationError, wrap_oserror, wrap_oserror2 @@ -436,7 +435,6 @@ __new__ = interp2app(W_FileIO.descr_new.im_func), __init__ = interp2app(W_FileIO.descr_init), __repr__ = interp2app(W_FileIO.repr_w), - __weakref__ = make_weakref_descr(W_FileIO), seek = interp2app(W_FileIO.seek_w), tell = interp2app(W_FileIO.tell_w), Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Wed Nov 17 13:01:22 2010 @@ -1,6 +1,7 @@ from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root from pypy.interpreter.typedef import ( - TypeDef, GetSetProperty, generic_new_descr, descr_get_dict, descr_set_dict) + TypeDef, GetSetProperty, generic_new_descr, descr_get_dict, descr_set_dict, + make_weakref_descr) from pypy.interpreter.gateway import interp2app, Arguments, unwrap_spec from pypy.interpreter.error import OperationError, operationerrfmt from pypy.rlib.rstring import StringBuilder @@ -246,6 +247,7 @@ _checkSeekable = interp2app(check_seekable_w), closed = GetSetProperty(W_IOBase.closed_get_w), __dict__ = GetSetProperty(descr_get_dict, descr_set_dict, cls=W_IOBase), + __weakref__ = make_weakref_descr(W_IOBase), readline = interp2app(W_IOBase.readline_w), readlines = interp2app(W_IOBase.readlines_w), Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Wed Nov 17 13:01:22 2010 @@ -96,6 +96,13 @@ return 42 assert MyIO().tell() == 42 + def test_weakref(self): + import _io + import weakref + f = _io.BytesIO() + ref = weakref.ref(f) + assert ref() is f + class AppTestOpen: def setup_class(cls): tmpfile = udir.join('tmpfile').ensure() From arigo at codespeak.net Wed Nov 17 13:19:18 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 13:19:18 +0100 (CET) Subject: [pypy-svn] r79189 - in pypy/branch/jit-free/pypy/jit: backend/llgraph metainterp metainterp/test Message-ID: <20101117121918.9D150282BD4@codespeak.net> Author: arigo Date: Wed Nov 17 13:19:16 2010 New Revision: 79189 Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py pypy/branch/jit-free/pypy/jit/metainterp/compile.py pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Log: Finish(?) the front-end work. Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py Wed Nov 17 13:19:16 2010 @@ -4,6 +4,7 @@ when executing on top of the llinterpreter. """ +import weakref from pypy.objspace.flow.model import Variable, Constant from pypy.annotation import model as annmodel from pypy.jit.metainterp.history import (ConstInt, ConstPtr, @@ -324,7 +325,7 @@ raise ValueError("CALL_ASSEMBLER not supported") loop = _from_opaque(loop) op = loop.operations[-1] - op.descr = descr + op.descr = weakref.ref(descr) def compile_add_var(loop, intvar): loop = _from_opaque(loop) @@ -847,11 +848,19 @@ finally: self._may_force = -1 - def op_call_assembler(self, loop_token, *args): + def op_call_assembler(self, wref_loop_token, *args): + if we_are_translated(): + raise ValueError("CALL_ASSEMBLER not supported") + return self._do_call_assembler(wref_loop_token, *args) + + def _do_call_assembler(self, wref_loop_token, *args): global _last_exception + loop_token = wref_loop_token() + assert loop_token, "CALL_ASSEMBLER to a target that already died" + if hasattr(loop_token, '_llgraph_redirected'): + return self._do_call_assembler(loop_token._llgraph_redirected, + *args) assert not self._forced - loop_token = self.cpu._redirected_call_assembler.get(loop_token, - loop_token) self._may_force = self.opindex try: inpargs = _from_opaque(loop_token._llgraph_compiled_version).inputargs @@ -1550,7 +1559,8 @@ OLD = _from_opaque(oldlooptoken._llgraph_compiled_version).getargtypes() NEW = _from_opaque(newlooptoken._llgraph_compiled_version).getargtypes() assert OLD == NEW - cpu._redirected_call_assembler[oldlooptoken] = newlooptoken + assert not hasattr(oldlooptoken, '_llgraph_redirected') + oldlooptoken._llgraph_redirected = weakref.ref(newlooptoken) # ____________________________________________________________ Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py Wed Nov 17 13:19:16 2010 @@ -102,7 +102,6 @@ llimpl._llinterp = LLInterpreter(self.rtyper) self._future_values = [] self._descrs = {} - self._redirected_call_assembler = {} def _freeze_(self): assert self.translate_support_code @@ -142,6 +141,7 @@ def free_loop_and_bridges(self, looptoken): for c in looptoken._llgraph_loop_and_bridges: llimpl.mark_as_free(c) + model.AbstractCPU.free_loop_and_bridges(self, looptoken) def _compile_loop_or_bridge(self, c, inputargs, operations): var2index = {} Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/compile.py Wed Nov 17 13:19:16 2010 @@ -57,22 +57,20 @@ assert looptoken is not None wref = weakref.ref(looptoken) for op in loop.operations: - if op.is_guard(): - resumedescr = op.getdescr() - assert isinstance(resumedescr, ResumeDescr) - resumedescr.wref_original_loop_token = wref # stick it there - n = resumedescr.index + descr = op.getdescr() + if isinstance(descr, ResumeDescr): + descr.wref_original_loop_token = wref # stick it there + n = descr.index if n >= 0: # we also record the resumedescr in this list looptoken.faildescr_indices.append(n) - # record that 'looptoken' ends up jumping to 'target_loop_token' - target_loop_token = loop.operations[-1].getdescr() - if isinstance(target_loop_token, LoopToken): - # the following test is not enough to prevent more complicated - # cases of cycles, but at least it helps in simple tests of - # test_memgr.py - if target_loop_token is not looptoken: - looptoken.record_jump_to(target_loop_token) - loop.operations[-1].setdescr(None) # clear reference + elif isinstance(descr, LoopToken): + # for a JUMP or a CALL_ASSEMBLER: record it as a potential jump. + # (the following test is not enough to prevent more complicated + # cases of cycles, but at least it helps in simple tests of + # test_memgr.py) + if descr is not looptoken: + looptoken.record_jump_to(descr) + op.setdescr(None) # clear reference, mostly for tests # mostly for tests: make sure we don't keep a reference to the LoopToken loop.token = None @@ -593,7 +591,8 @@ propagate_exception_descr = PropagateExceptionDescr() -def compile_tmp_callback(cpu, jitdriver_sd, greenboxes, redboxes): +def compile_tmp_callback(cpu, jitdriver_sd, greenboxes, redboxes, + memory_manager=None): """Make a LoopToken that corresponds to assembler code that just calls back the interpreter. Used temporarily: a fully compiled version of the code may end up replacing it. @@ -633,4 +632,6 @@ ] operations[1].setfailargs([]) cpu.compile_loop(inputargs, operations, loop_token, log=False) + if memory_manager is not None: # for tests + memory_manager.keep_loop_alive(loop_token) return loop_token Modified: pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py Wed Nov 17 13:19:16 2010 @@ -60,7 +60,7 @@ def _kill_old_loops_now(self): debug_start("jit-free-memmgr") oldtotal = len(self.alive_loops) - print self.alive_loops.keys() + #print self.alive_loops.keys() debug_print("Current generation:", self.current_generation) debug_print("Loop tokens before:", oldtotal) max_generation = self.current_generation - (self.max_age-1) @@ -70,8 +70,10 @@ newtotal = len(self.alive_loops) debug_print("Loop tokens freed: ", oldtotal - newtotal) debug_print("Loop tokens left: ", newtotal) - print self.alive_loops.keys() - debug_stop("jit-free-memmgr") - if not we_are_translated(): + #print self.alive_loops.keys() + if not we_are_translated() and oldtotal != newtotal: + looptoken = None from pypy.rlib import rgc - rgc.collect() + # a single one is not enough for all tests :-( + rgc.collect(); rgc.collect(); rgc.collect() + debug_stop("jit-free-memmgr") Modified: pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py Wed Nov 17 13:19:16 2010 @@ -1622,10 +1622,10 @@ # is also available as 'self.jitdriver_sd', because we need to # specialize this function and a few other ones for the '*args'. debug_start('jit-tracing') - self.staticdata.try_to_free_some_loops() self.staticdata._setup_once() self.staticdata.profiler.start_tracing() assert jitdriver_sd is self.jitdriver_sd + self.staticdata.try_to_free_some_loops() self.create_empty_history() try: original_boxes = self.initialize_original_boxes(jitdriver_sd, *args) @@ -1657,6 +1657,7 @@ # store the resumekey.wref_original_loop_token() on 'self' to make # sure that it stays alive as long as this MetaInterp self.resumekey_original_loop_token = key.wref_original_loop_token() + self.staticdata.try_to_free_some_loops() self.initialize_state_from_guard_failure(key) try: return self._handle_guard_failure(key) @@ -1669,7 +1670,6 @@ self.current_merge_points = [] self.resumekey = key self.seen_loop_header_for_jdindex = -1 - self.staticdata.try_to_free_some_loops() try: self.prepare_resume_from_failure(key.guard_opnum) if self.resumekey_original_loop_token is None: # very rare case Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Wed Nov 17 13:19:16 2010 @@ -1,6 +1,6 @@ from pypy.jit.metainterp.memmgr import MemoryManager from pypy.jit.metainterp.test.test_basic import LLJitMixin -from pypy.rlib.jit import JitDriver +from pypy.rlib.jit import JitDriver, dont_look_inside class FakeLoopToken: @@ -145,4 +145,36 @@ assert res == 42 self.check_tree_loop_count(2 + 10*4) # 42 :-) - #XXXcall_assembler + def test_call_assembler_keep_alive(self): + myjitdriver1 = JitDriver(greens=['m'], reds=['n']) + myjitdriver2 = JitDriver(greens=['m'], reds=['n', 'rec']) + def h(m, n): + while True: + myjitdriver1.can_enter_jit(n=n, m=m) + myjitdriver1.jit_merge_point(n=n, m=m) + n = n >> 1 + if n == 0: + return 21 + def g(m, rec): + n = 5 + while n > 0: + myjitdriver2.can_enter_jit(n=n, m=m, rec=rec) + myjitdriver2.jit_merge_point(n=n, m=m, rec=rec) + if rec: + h(m, rec) + n = n - 1 + return 21 + def f(u): + for i in range(8): + h(u, 32) # make a loop and an entry bridge for h(u) + g(u, 8) # make a loop for g(u) with a call_assembler + g(u, 0); g(u+1, 0) # \ + g(u, 0); g(u+2, 0) # \ make more loops for g(u+1) to g(u+4), + g(u, 0); g(u+3, 0) # / but keeps g(u) alive + g(u, 0); g(u+4, 0) # / + g(u, 8) # call g(u) again, with its call_assembler to h(u) + return 42 + + res = self.meta_interp(f, [1], loop_longevity=4, inline=True) + assert res == 42 + self.check_tree_loop_count(8) Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py Wed Nov 17 13:19:16 2010 @@ -822,5 +822,4 @@ debug_stop("jit-running") self.metainterp_sd.profiler.end_running() self.memory_manager.keep_loop_alive(loop_token) - print loop_token return fail_descr Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Wed Nov 17 13:19:16 2010 @@ -263,6 +263,10 @@ cell.set_entry_loop_token(entry_loop_token) if old_token is not None: self.cpu.redirect_call_assembler(old_token, entry_loop_token) + # entry_loop_token is also kept alive by any loop that used + # to point to old_token. Actually freeing old_token early + # is a pointless optimization (it is tiny). + old_token.record_jump_to(entry_loop_token) # ---------- @@ -541,6 +545,7 @@ if hasattr(self, 'get_location_str'): return # + warmrunnerdesc = self.warmrunnerdesc unwrap_greenkey = self.make_unwrap_greenkey() jit_getter = self.make_jitcell_getter() jd = self.jitdriver_sd @@ -565,8 +570,9 @@ entry_loop_token = cell.get_entry_loop_token() if entry_loop_token is None: from pypy.jit.metainterp.compile import compile_tmp_callback + memmgr = warmrunnerdesc.memory_manager entry_loop_token = compile_tmp_callback(cpu, jd, greenkey, - redboxes) + redboxes, memmgr) cell.set_entry_loop_token(entry_loop_token) return entry_loop_token self.get_assembler_token = get_assembler_token From arigo at codespeak.net Wed Nov 17 13:31:19 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 13:31:19 +0100 (CET) Subject: [pypy-svn] r79190 - in pypy/trunk/pypy: . annotation module/_rawffi/test module/cpyext module/cpyext/test module/rctime rlib rpython rpython/lltypesystem translator/c/test Message-ID: <20101117123119.813B4282BDC@codespeak.net> Author: arigo Date: Wed Nov 17 13:31:17 2010 New Revision: 79190 Modified: pypy/trunk/pypy/annotation/builtin.py pypy/trunk/pypy/conftest.py pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py pypy/trunk/pypy/module/_rawffi/test/test_nested.py pypy/trunk/pypy/module/cpyext/api.py pypy/trunk/pypy/module/cpyext/cdatetime.py pypy/trunk/pypy/module/cpyext/test/test_borrow.py pypy/trunk/pypy/module/cpyext/typeobject.py pypy/trunk/pypy/module/rctime/interp_time.py pypy/trunk/pypy/rlib/clibffi.py pypy/trunk/pypy/rpython/lltypesystem/llarena.py pypy/trunk/pypy/rpython/lltypesystem/lltype.py pypy/trunk/pypy/rpython/lltypesystem/rffi.py pypy/trunk/pypy/rpython/rbuiltin.py pypy/trunk/pypy/translator/c/test/test_lltyped.py Log: Merge branch/leak-finder-more, looking for leaks also in app-level tests. There are a few cpyext tests still failing. Modified: pypy/trunk/pypy/annotation/builtin.py ============================================================================== --- pypy/trunk/pypy/annotation/builtin.py (original) +++ pypy/trunk/pypy/annotation/builtin.py Wed Nov 17 13:31:17 2010 @@ -453,6 +453,9 @@ #p = lltype.malloc(T, flavor=s_flavor.const) #lltype.free(p, flavor=s_flavor.const) +def render_immortal(s_p, s_track_allocation=None): + assert s_track_allocation is None or s_track_allocation.is_constant() + def typeOf(s_val): lltype = annotation_to_lltype(s_val, info="in typeOf(): ") return immutablevalue(lltype) @@ -520,6 +523,7 @@ BUILTIN_ANALYZERS[lltype.malloc] = malloc BUILTIN_ANALYZERS[lltype.free] = free +BUILTIN_ANALYZERS[lltype.render_immortal] = render_immortal BUILTIN_ANALYZERS[lltype.typeOf] = typeOf BUILTIN_ANALYZERS[lltype.cast_primitive] = cast_primitive BUILTIN_ANALYZERS[lltype.nullptr] = nullptr Modified: pypy/trunk/pypy/conftest.py ============================================================================== --- pypy/trunk/pypy/conftest.py (original) +++ pypy/trunk/pypy/conftest.py Wed Nov 17 13:31:17 2010 @@ -327,6 +327,31 @@ class PyPyTestFunction(py.test.collect.Function): # All PyPy test items catch and display OperationErrors specially. # + def runtest(self): + self.runtest_open() + try: + self.runtest_perform() + finally: + self.runtest_close() + self.runtest_finish() + + def runtest_open(self): + leakfinder.start_tracking_allocations() + + def runtest_perform(self): + super(PyPyTestFunction, self).runtest() + + def runtest_close(self): + if leakfinder.TRACK_ALLOCATIONS: + self._pypytest_leaks = leakfinder.stop_tracking_allocations(False) + else: # stop_tracking_allocations() already called + self._pypytest_leaks = None + + def runtest_finish(self): + # check for leaks, but only if the test passed so far + if self._pypytest_leaks: + raise leakfinder.MallocMismatch(self._pypytest_leaks) + def execute_appex(self, space, target, *args): try: target(*args) @@ -353,16 +378,9 @@ def _keywords(self): return super(IntTestFunction, self)._keywords() + ['interplevel'] - def runtest(self): + def runtest_perform(self): try: - leakfinder.start_tracking_allocations() - try: - super(IntTestFunction, self).runtest() - finally: - if leakfinder.TRACK_ALLOCATIONS: - leaks = leakfinder.stop_tracking_allocations(False) - else: - leaks = None # stop_tracking_allocations() already called + super(IntTestFunction, self).runtest_perform() except OperationError, e: check_keyboard_interrupt(e) raise @@ -375,14 +393,15 @@ py.test.skip('%s: %s' % (e.__class__.__name__, e)) cls = cls.__bases__[0] raise + + def runtest_finish(self): if 'pygame' in sys.modules: global _pygame_imported if not _pygame_imported: _pygame_imported = True assert option.view, ("should not invoke Pygame " "if conftest.option.view is False") - if leaks: # check for leaks, but only if the test passed so far - raise leakfinder.MallocMismatch(leaks) + super(IntTestFunction, self).runtest_finish() class AppTestFunction(PyPyTestFunction): def _prunetraceback(self, traceback): @@ -395,7 +414,7 @@ def _keywords(self): return ['applevel'] + super(AppTestFunction, self)._keywords() - def runtest(self): + def runtest_perform(self): target = self.obj if option.runappdirect: return target() @@ -427,7 +446,7 @@ space.setattr(w_instance, space.wrap(name[2:]), getattr(instance, name)) - def runtest(self): + def runtest_perform(self): target = self.obj if option.runappdirect: return target() Modified: pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py ============================================================================== --- pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py (original) +++ pypy/trunk/pypy/module/_rawffi/test/test__rawffi.py Wed Nov 17 13:31:17 2010 @@ -296,6 +296,7 @@ assert _rawffi.charp2string(res[0]) is None arg1.free() arg2.free() + a.free() def test_raw_callable(self): import _rawffi Modified: pypy/trunk/pypy/module/_rawffi/test/test_nested.py ============================================================================== --- pypy/trunk/pypy/module/_rawffi/test/test_nested.py (original) +++ pypy/trunk/pypy/module/_rawffi/test/test_nested.py Wed Nov 17 13:31:17 2010 @@ -107,7 +107,6 @@ assert S.fieldoffset('x') == 0 assert S.fieldoffset('ar') == A5alignment s = S() - s = S() s.x = 'G' raises(TypeError, 's.ar') assert s.fieldaddress('ar') == s.buffer + S.fieldoffset('ar') Modified: pypy/trunk/pypy/module/cpyext/api.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/api.py (original) +++ pypy/trunk/pypy/module/cpyext/api.py Wed Nov 17 13:31:17 2010 @@ -908,8 +908,10 @@ from pypy.rlib import rdynload try: ll_libname = rffi.str2charp(path) - dll = rdynload.dlopen(ll_libname) - lltype.free(ll_libname, flavor='raw') + try: + dll = rdynload.dlopen(ll_libname) + finally: + lltype.free(ll_libname, flavor='raw') except rdynload.DLOpenError, e: raise operationerrfmt( space.w_ImportError, Modified: pypy/trunk/pypy/module/cpyext/cdatetime.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/cdatetime.py (original) +++ pypy/trunk/pypy/module/cpyext/cdatetime.py Wed Nov 17 13:31:17 2010 @@ -4,7 +4,7 @@ from pypy.module.cpyext.api import ( cpython_api, CANNOT_FAIL, cpython_struct, PyObjectFields) from pypy.module.cpyext.import_ import PyImport_Import -from pypy.module.cpyext.typeobject import PyTypeObjectPtr +from pypy.module.cpyext.typeobject import PyTypeObjectPtr, render_immortal from pypy.module.cpyext.state import State from pypy.interpreter.error import OperationError from pypy.tool.sourcetools import func_renamer @@ -22,25 +22,34 @@ @cpython_api([], lltype.Ptr(PyDateTime_CAPI), error=lltype.nullptr(PyDateTime_CAPI)) def _PyDateTime_Import(space): - datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw') + datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw', + track_allocation=False) if not we_are_translated(): datetimeAPI_dealloc(space) space.fromcache(State).datetimeAPI = datetimeAPI w_datetime = PyImport_Import(space, space.wrap("datetime")) + w_type = space.getattr(w_datetime, space.wrap("date")) datetimeAPI.c_DateType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DateType, w_type) + w_type = space.getattr(w_datetime, space.wrap("datetime")) datetimeAPI.c_DateTimeType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DateTimeType, w_type) + w_type = space.getattr(w_datetime, space.wrap("time")) datetimeAPI.c_TimeType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_TimeType, w_type) + w_type = space.getattr(w_datetime, space.wrap("timedelta")) datetimeAPI.c_DeltaType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DeltaType, w_type) return datetimeAPI Modified: pypy/trunk/pypy/module/cpyext/test/test_borrow.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/test/test_borrow.py (original) +++ pypy/trunk/pypy/module/cpyext/test/test_borrow.py Wed Nov 17 13:31:17 2010 @@ -31,6 +31,7 @@ g = PyTuple_GetItem(t, 0); // borrows reference again printf("Refcnt4: %i\\n", f->ob_refcnt); printf("COMPARE: %i\\n", f == g); + fflush(stdout); Py_DECREF(t); Py_RETURN_TRUE; """), Modified: pypy/trunk/pypy/module/cpyext/typeobject.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/typeobject.py (original) +++ pypy/trunk/pypy/module/cpyext/typeobject.py Wed Nov 17 13:31:17 2010 @@ -182,10 +182,10 @@ subtype = rffi.cast(PyTypeObjectPtr, make_ref(space, w_subtype)) try: - obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds) + w_obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds) finally: Py_DecRef(space, w_subtype) - return obj + return w_obj @specialize.memo() def get_new_method_def(space): @@ -193,10 +193,14 @@ if state.new_method_def: return state.new_method_def from pypy.module.cpyext.modsupport import PyMethodDef - ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True) + ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True, + immortal=True) ptr.c_ml_name = rffi.str2charp("__new__") + lltype.render_immortal(ptr.c_ml_name) rffi.setintfield(ptr, 'c_ml_flags', METH_VARARGS | METH_KEYWORDS) - ptr.c_ml_doc = rffi.str2charp("T.__new__(S, ...) -> a new object with type S, a subtype of T") + ptr.c_ml_doc = rffi.str2charp( + "T.__new__(S, ...) -> a new object with type S, a subtype of T") + lltype.render_immortal(ptr.c_ml_doc) state.new_method_def = ptr return ptr @@ -429,6 +433,12 @@ finish_type_1(space, pto) finish_type_2(space, pto, w_type) + if space.type(w_type).is_cpytype(): + # XXX Types with a C metatype are never freed, try to see why... + render_immortal(pto, w_type) + lltype.render_immortal(pto) + lltype.render_immortal(pto.c_tp_name) + pto.c_tp_basicsize = rffi.sizeof(typedescr.basestruct) if pto.c_tp_base: if pto.c_tp_base.c_tp_basicsize > pto.c_tp_basicsize: @@ -534,12 +544,25 @@ w_obj.ready() finish_type_2(space, py_type, w_obj) + render_immortal(py_type, w_obj) state = space.fromcache(RefcountState) state.non_heaptypes_w.append(w_obj) return w_obj +def render_immortal(py_type, w_obj): + lltype.render_immortal(py_type.c_tp_bases) + lltype.render_immortal(py_type.c_tp_mro) + + assert isinstance(w_obj, W_TypeObject) + if w_obj.is_cpytype(): + lltype.render_immortal(py_type.c_tp_dict) + else: + lltype.render_immortal(py_type.c_tp_name) + if not w_obj.is_cpytype() and w_obj.is_heaptype(): + lltype.render_immortal(py_type) + def finish_type_1(space, pto): """ Sets up tp_bases, necessary before creating the interpreter type. Modified: pypy/trunk/pypy/module/rctime/interp_time.py ============================================================================== --- pypy/trunk/pypy/module/rctime/interp_time.py (original) +++ pypy/trunk/pypy/module/rctime/interp_time.py Wed Nov 17 13:31:17 2010 @@ -69,7 +69,7 @@ CLOCKS_PER_SEC = cConfig.CLOCKS_PER_SEC clock_t = cConfig.clock_t tm = cConfig.tm -glob_buf = lltype.malloc(tm, flavor='raw', zero=True) +glob_buf = lltype.malloc(tm, flavor='raw', zero=True, immortal=True) if cConfig.has_gettimeofday: c_gettimeofday = external('gettimeofday', [rffi.VOIDP, rffi.VOIDP], rffi.INT) Modified: pypy/trunk/pypy/rlib/clibffi.py ============================================================================== --- pypy/trunk/pypy/rlib/clibffi.py (original) +++ pypy/trunk/pypy/rlib/clibffi.py Wed Nov 17 13:31:17 2010 @@ -432,7 +432,8 @@ flags=FUNCFLAG_CDECL): AbstractFuncPtr.__init__(self, "callback", argtypes, restype, flags) self.ll_closure = closureHeap.alloc() - self.ll_userdata = lltype.malloc(USERDATA_P.TO, flavor='raw') + self.ll_userdata = lltype.malloc(USERDATA_P.TO, flavor='raw', + track_allocation=False) self.ll_userdata.callback = rffi.llhelper(CALLBACK_TP, func) self.ll_userdata.addarg = additional_arg res = c_ffi_prep_closure(self.ll_closure, self.ll_cif, @@ -447,7 +448,7 @@ closureHeap.free(self.ll_closure) self.ll_closure = lltype.nullptr(FFI_CLOSUREP.TO) if self.ll_userdata: - lltype.free(self.ll_userdata, flavor='raw') + lltype.free(self.ll_userdata, flavor='raw', track_allocation=False) self.ll_userdata = lltype.nullptr(USERDATA_P.TO) class RawFuncPtr(AbstractFuncPtr): Modified: pypy/trunk/pypy/rpython/lltypesystem/llarena.py ============================================================================== --- pypy/trunk/pypy/rpython/lltypesystem/llarena.py (original) +++ pypy/trunk/pypy/rpython/lltypesystem/llarena.py Wed Nov 17 13:31:17 2010 @@ -440,7 +440,8 @@ [rffi.INT], rffi.INT, sandboxsafe=True, _nowrapper=True) - _dev_zero = rffi.str2charp_immortal('/dev/zero') # prebuilt + _dev_zero = rffi.str2charp('/dev/zero') # prebuilt + lltype.render_immortal(_dev_zero) def clear_large_memory_chunk(baseaddr, size): # on some Unixy platforms, reading from /dev/zero is the fastest way Modified: pypy/trunk/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/trunk/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/trunk/pypy/rpython/lltypesystem/lltype.py Wed Nov 17 13:31:17 2010 @@ -1868,6 +1868,13 @@ leakfinder.remember_free(p._obj0) p._obj0._free() +def render_immortal(p, track_allocation=True): + T = typeOf(p) + if not isinstance(T, Ptr) or p._togckind() != 'raw': + raise TypeError, "free(): only for pointers to non-gc containers" + if track_allocation: + leakfinder.remember_free(p._obj0) + def _make_scoped_allocator(T): class ScopedAlloc: def __init__(self, n=None, zero=False): Modified: pypy/trunk/pypy/rpython/lltypesystem/rffi.py ============================================================================== --- pypy/trunk/pypy/rpython/lltypesystem/rffi.py (original) +++ pypy/trunk/pypy/rpython/lltypesystem/rffi.py Wed Nov 17 13:31:17 2010 @@ -607,15 +607,6 @@ return array str2charp._annenforceargs_ = [strtype] - def str2charp_immortal(s): - "NOT_RPYTHON" - array = lltype.malloc(TYPEP.TO, len(s) + 1, flavor='raw', - immortal=True) - for i in range(len(s)): - array[i] = s[i] - array[len(s)] = lastchar - return array - def free_charp(cp): lltype.free(cp, flavor='raw') @@ -734,19 +725,19 @@ l = [cp[i] for i in range(size)] return emptystr.join(l) - return (str2charp, str2charp_immortal, free_charp, charp2str, + return (str2charp, free_charp, charp2str, get_nonmovingbuffer, free_nonmovingbuffer, alloc_buffer, str_from_buffer, keep_buffer_alive_until_here, charp2strn, charpsize2str, ) -(str2charp, str2charp_immortal, free_charp, charp2str, +(str2charp, free_charp, charp2str, get_nonmovingbuffer, free_nonmovingbuffer, alloc_buffer, str_from_buffer, keep_buffer_alive_until_here, charp2strn, charpsize2str, ) = make_string_mappings(str) -(unicode2wcharp, unicode2wcharp_immortal, free_wcharp, wcharp2unicode, +(unicode2wcharp, free_wcharp, wcharp2unicode, get_nonmoving_unicodebuffer, free_nonmoving_unicodebuffer, alloc_unicodebuffer, unicode_from_buffer, keep_unicodebuffer_alive_until_here, wcharp2unicoden, wcharpsize2unicode, Modified: pypy/trunk/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/trunk/pypy/rpython/rbuiltin.py (original) +++ pypy/trunk/pypy/rpython/rbuiltin.py Wed Nov 17 13:31:17 2010 @@ -386,6 +386,14 @@ hop.exception_cannot_occur() hop.genop('free', vlist) +def rtype_render_immortal(hop, i_track_allocation=None): + vlist = [hop.inputarg(hop.args_r[0], arg=0)] + v_track_allocation = parse_kwds(hop, + (i_track_allocation, None)) + hop.exception_cannot_occur() + if i_track_allocation is None or v_track_allocation.value: + hop.genop('track_alloc_stop', vlist) + def rtype_const_result(hop): hop.exception_cannot_occur() return hop.inputconst(hop.r_result.lowleveltype, hop.s_result.const) @@ -523,6 +531,7 @@ BUILTIN_TYPER[lltype.malloc] = rtype_malloc BUILTIN_TYPER[lltype.free] = rtype_free +BUILTIN_TYPER[lltype.render_immortal] = rtype_render_immortal BUILTIN_TYPER[lltype.cast_primitive] = rtype_cast_primitive BUILTIN_TYPER[lltype.cast_pointer] = rtype_cast_pointer BUILTIN_TYPER[lltype.cast_opaque_ptr] = rtype_cast_opaque_ptr Modified: pypy/trunk/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/trunk/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/trunk/pypy/translator/c/test/test_lltyped.py Wed Nov 17 13:31:17 2010 @@ -881,3 +881,17 @@ assert res == -98765432 res = fn(1) assert res == -9999999 + + def test_render_immortal(self): + A = FixedSizeArray(Signed, 1) + a1 = malloc(A, flavor='raw') + render_immortal(a1) + a1[0] = 42 + def llf(): + a2 = malloc(A, flavor='raw') + render_immortal(a2) + a2[0] = 3 + return a1[0] + a2[0] + fn = self.getcompiled(llf) + assert fn() == 45 + From arigo at codespeak.net Wed Nov 17 13:31:31 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 13:31:31 +0100 (CET) Subject: [pypy-svn] r79191 - pypy/branch/leak-finder-more Message-ID: <20101117123131.E29D2282BE3@codespeak.net> Author: arigo Date: Wed Nov 17 13:31:30 2010 New Revision: 79191 Removed: pypy/branch/leak-finder-more/ Log: Remove merged branch. From arigo at codespeak.net Wed Nov 17 13:45:48 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 13:45:48 +0100 (CET) Subject: [pypy-svn] r79192 - pypy/trunk/pypy/translator/c/src Message-ID: <20101117124548.DE947282BF3@codespeak.net> Author: arigo Date: Wed Nov 17 13:45:47 2010 New Revision: 79192 Modified: pypy/trunk/pypy/translator/c/src/asm_gcc_x86.h pypy/trunk/pypy/translator/c/src/g_include.h Log: Oups. Disabled also the READ_TIMESTAMP in assembler by mistake. Modified: pypy/trunk/pypy/translator/c/src/asm_gcc_x86.h ============================================================================== --- pypy/trunk/pypy/translator/c/src/asm_gcc_x86.h (original) +++ pypy/trunk/pypy/translator/c/src/asm_gcc_x86.h Wed Nov 17 13:45:47 2010 @@ -2,6 +2,8 @@ * It replaces some complex macros with native assembler instructions. */ +#if 0 /* --- disabled: does not give any speed-up --- */ + #undef OP_INT_ADD_OVF #define OP_INT_ADD_OVF(x,y,r) \ asm volatile( \ @@ -50,6 +52,13 @@ : "0"(x), "g"(y) /* inputs */ \ : "cc", "memory") /* clobber */ +extern void op_int_overflowed(void) + asm ("_op_int_overflowed") + __attribute__((used)); + +#endif /* 0 */ + + /* Pentium only! */ #define READ_TIMESTAMP(val) \ asm volatile("rdtsc" : "=A" (val)) @@ -62,19 +71,15 @@ // I don't know how important it is, comment talks about time warps -/* prototypes */ - -extern void op_int_overflowed(void) - asm ("_op_int_overflowed") - __attribute__((used)); - /* implementations */ #ifndef PYPY_NOT_MAIN_FILE +# if 0 /* disabled */ void op_int_overflowed(void) { FAIL_OVF("integer operation"); } +# endif #endif Modified: pypy/trunk/pypy/translator/c/src/g_include.h ============================================================================== --- pypy/trunk/pypy/translator/c/src/g_include.h (original) +++ pypy/trunk/pypy/translator/c/src/g_include.h Wed Nov 17 13:45:47 2010 @@ -39,10 +39,9 @@ #include "src/instrument.h" /* optional assembler bits */ -// disabled: does not give any speed-up -//#if defined(__GNUC__) && defined(__i386__) -//# include "src/asm_gcc_x86.h" -//#endif +#if defined(__GNUC__) && defined(__i386__) +# include "src/asm_gcc_x86.h" +#endif #if defined(__GNUC__) && defined(__ppc__) # include "src/asm_ppc.h" From arigo at codespeak.net Wed Nov 17 14:20:06 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 14:20:06 +0100 (CET) Subject: [pypy-svn] r79193 - pypy/branch/jit-free/pypy/translator/c Message-ID: <20101117132006.B61C2282BEF@codespeak.net> Author: arigo Date: Wed Nov 17 14:20:05 2010 New Revision: 79193 Modified: pypy/branch/jit-free/pypy/translator/c/funcgen.py Log: debug_print(long long). Modified: pypy/branch/jit-free/pypy/translator/c/funcgen.py ============================================================================== --- pypy/branch/jit-free/pypy/translator/c/funcgen.py (original) +++ pypy/branch/jit-free/pypy/translator/c/funcgen.py Wed Nov 17 14:20:05 2010 @@ -757,6 +757,10 @@ format.append('%s') argv.append('(%s) ? "True" : "False"' % self.expr(arg)) continue + elif T == SignedLongLong: + format.append('%lld') + elif T == UnsignedLongLong: + format.append('%llu') else: raise Exception("don't know how to debug_print %r" % (T,)) argv.append(self.expr(arg)) From arigo at codespeak.net Wed Nov 17 14:41:47 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 14:41:47 +0100 (CET) Subject: [pypy-svn] r79194 - pypy/branch/jit-free/pypy/jit/metainterp Message-ID: <20101117134147.6E7BD282B90@codespeak.net> Author: arigo Date: Wed Nov 17 14:41:45 2010 New Revision: 79194 Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py pypy/branch/jit-free/pypy/jit/metainterp/history.py Log: Also record how many bridges are attached to a given LoopToken, and print it when we free the loop and the attached bridges. Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/compile.py Wed Nov 17 14:41:45 2010 @@ -155,6 +155,7 @@ if not we_are_translated(): show_loop(metainterp_sd) TreeLoop.check_consistency_of(inputargs, operations) + original_loop_token.bridges_count += 1 metainterp_sd.profiler.start_backend() debug_start("jit-backend") try: Modified: pypy/branch/jit-free/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/history.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/history.py Wed Nov 17 14:41:45 2010 @@ -734,6 +734,7 @@ cpu = None number = -1 generation = r_longlong(0) + bridges_count = 0 def __init__(self, cpu=None): self.cpu = cpu @@ -753,7 +754,8 @@ # loop token, which means that it has been successfully # compiled by the backend. Free it now. debug_start("jit-free-looptoken") - debug_print("Freeing loop #", self.number) + debug_print("Freeing loop #", self.number, 'with', + self.bridges_count, 'attached bridges') self.cpu.free_loop_and_bridges(self) debug_stop("jit-free-looptoken") From antocuni at codespeak.net Wed Nov 17 15:29:36 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 17 Nov 2010 15:29:36 +0100 (CET) Subject: [pypy-svn] r79195 - pypy/trunk/pypy/jit/tool Message-ID: <20101117142936.216CF282B9D@codespeak.net> Author: antocuni Date: Wed Nov 17 15:29:34 2010 New Revision: 79195 Added: pypy/trunk/pypy/jit/tool/loopcounter.py (contents, props changed) Log: a tool to parse a logfile and produce a csv with the number of loops and bridges as a function of the timestamp (useful for building graphs) Added: pypy/trunk/pypy/jit/tool/loopcounter.py ============================================================================== --- (empty file) +++ pypy/trunk/pypy/jit/tool/loopcounter.py Wed Nov 17 15:29:34 2010 @@ -0,0 +1,44 @@ +#!/usr/bin/env python +""" +Parse and display the traces produced by pypy-c-jit when PYPYLOG is set. +""" + +import autopath +import py +import sys +import optparse + +def get_timestamp(line): + import re + match = re.match(r'\[([0-9a-f]*)\] .*', line) + return int(match.group(1), 16) + +def main(logfile, options): + log = open(logfile) + loops = 0 + bridges = 0 + time0 = None + print 'timestamp,total,loops,bridges' + for line in log: + if time0 is None and line.startswith('['): + time0 = get_timestamp(line) + if '{jit-log-opt-' in line: + time_now = get_timestamp(line) + if '{jit-log-opt-loop' in line: + loops += 1 + elif '{jit-log-opt-bridge' in line: + bridges += 1 + else: + assert False, 'unknown category %s' % line + total = loops+bridges + timestamp = time_now - time0 + print '%d,%d,%d,%d' % (timestamp, total, loops, bridges) + +if __name__ == '__main__': + parser = optparse.OptionParser(usage="%prog loopfile [options]") + options, args = parser.parse_args() + if len(args) != 1: + parser.print_help() + sys.exit(2) + + main(args[0], options) From cfbolz at codespeak.net Wed Nov 17 15:51:37 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 17 Nov 2010 15:51:37 +0100 (CET) Subject: [pypy-svn] r79196 - pypy/extradoc/talk/pepm2011 Message-ID: <20101117145137.86D25282BEC@codespeak.net> Author: cfbolz Date: Wed Nov 17 15:51:35 2010 New Revision: 79196 Modified: pypy/extradoc/talk/pepm2011/paper.tex Log: comments from the review Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Wed Nov 17 15:51:35 2010 @@ -263,7 +263,7 @@ mostly mixed-mode execution environments, they contain both an interpreter and a JIT compiler. By default the interpreter is used to execute the program, doing some light-weight profiling at the same time. This profiling is used to identify -the hot loops of the program. If a hot loop is found in that way, the +the hot loops\reva{what is that? clarify} of the program. If a hot loop is found in that way, the interpreter enters a special \emph{tracing mode}. In this tracing mode, the interpreter tries to record all operations that it is executing while running one iteration of the hot loop. This history of executed operations of one loop is @@ -296,7 +296,7 @@ on. These guards are the only mechanism to stop the execution of a trace, the loop end condition also takes the form of a guard. -If one specific guard fails often enough, the tracing JIT will generate a new +If one specific guard fails often enough\reva{when is that?}, the tracing JIT will generate a new trace that starts exactly at the position of the failing guard \cite{andreas_gal_incremental_2006}. The existing assembler is patched to jump to the new trace when the guard fails. This approach guarantees that all the @@ -373,7 +373,7 @@ dynamic language implementation. All the numbers are instances of either \lstinline{BoxedInteger} or \lstinline{BoxedFloat}, therefore they consume space on the heap. Performing many arithmetic operations produces lots of garbage quickly, -putthing pressure on the garbage collector. Using double dispatching to +putting pressure on the garbage collector. Using double dispatching to implement the numeric tower needs two method calls per arithmetic operation, which is costly due to the method dispatch. @@ -1174,7 +1174,8 @@ complex than our simple one-pass optimization. Also, stack-allocation reduces garbage-collection pressure but does not optimize away the actual accesses to the stack-allocated object. In our case, an object is not needed at all any -more. +more. \reva{Has this relation pointed out before in some paper? Is it a +novel contribution of your work?} Chang \etal describe a tracing JIT for JavaScript running on top of a JVM \cite{mason_chang_efficient_2007}. They mention in passing an approach to @@ -1212,6 +1213,10 @@ imperative language. In functional programming this idea was introduced as constructor specialisation by Mogensen \cite{mogensen_constructor_1993}. +\reva{I'd refer here to + deforestation (by Philip Wadler) and "short cut + deforestation" by Andrew Gill, John Launchbury, and Simon L + Peyton Jones} %xxx: %partially static data structures: kenichi asai's thesis? From cfbolz at codespeak.net Wed Nov 17 15:53:36 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 17 Nov 2010 15:53:36 +0100 (CET) Subject: [pypy-svn] r79197 - pypy/extradoc/talk/pepm2011 Message-ID: <20101117145336.D1F23282BF2@codespeak.net> Author: cfbolz Date: Wed Nov 17 15:53:35 2010 New Revision: 79197 Modified: pypy/extradoc/talk/pepm2011/paper.tex Log: more comments Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Wed Nov 17 15:53:35 2010 @@ -1017,7 +1017,7 @@ To evaluate our allocation removal algorithm, we look at the effectiveness when used in the generated tracing JIT of PyPy's Python interpreter. This interpreter -is a full implementation of Python 2.5 language semantics and is about 30'000 +is a full implementation of Python 2.5 language semantics and is about 30,000 lines of RPython code. The @@ -1217,6 +1217,9 @@ deforestation (by Philip Wadler) and "short cut deforestation" by Andrew Gill, John Launchbury, and Simon L Peyton Jones} +\revb{cite and relate to: boxing analysis, +soft typing, dynamic typing, etc (by various authors: Henglein, Cartright, Felleisen, +Leroy)} %xxx: %partially static data structures: kenichi asai's thesis? From cfbolz at codespeak.net Wed Nov 17 15:59:39 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 17 Nov 2010 15:59:39 +0100 (CET) Subject: [pypy-svn] r79199 - pypy/extradoc/talk/pepm2011 Message-ID: <20101117145939.A255750810@codespeak.net> Author: cfbolz Date: Wed Nov 17 15:59:38 2010 New Revision: 79199 Modified: pypy/extradoc/talk/pepm2011/math.lyx pypy/extradoc/talk/pepm2011/paper.tex Log: fix a tiny problem in the formulas: something should not be in italics Modified: pypy/extradoc/talk/pepm2011/math.lyx ============================================================================== --- pypy/extradoc/talk/pepm2011/math.lyx (original) +++ pypy/extradoc/talk/pepm2011/math.lyx Wed Nov 17 15:59:38 2010 @@ -575,7 +575,7 @@ \begin_inset Text \begin_layout Plain Layout -\begin_inset Formula ${\displaystyle \frac{\left(S\left(v^{*}\right)_{L},S\setminus\left\{ v^{*}\mapsto S\left(v^{*}\right)\right\} \right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{L},S^{\prime}\right),\,\left(S\left(v^{*}\right)_{R},S^{\prime}\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{R},S^{\prime\prime}\right)}{v^{*},S\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\mathrm{ops}_{L}::ops_{R}::\left\langle \mathtt{set}\left(v^{*},L,S\left(v^{*}\right)_{L}\right),\,\mathtt{set}\left(v^{*},R,S\left(v^{*}\right)_{R}\right)\right\rangle ,S^{\prime\prime}}}$ +\begin_inset Formula ${\displaystyle \frac{\left(S\left(v^{*}\right)_{L},S\setminus\left\{ v^{*}\mapsto S\left(v^{*}\right)\right\} \right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{L},S^{\prime}\right),\,\left(S\left(v^{*}\right)_{R},S^{\prime}\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{R},S^{\prime\prime}\right)}{v^{*},S\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\mathrm{ops}_{L}::\mathrm{ops}_{R}::\left\langle \mathtt{set}\left(v^{*},L,S\left(v^{*}\right)_{L}\right),\,\mathtt{set}\left(v^{*},R,S\left(v^{*}\right)_{R}\right)\right\rangle ,S^{\prime\prime}}}$ \end_inset Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Wed Nov 17 15:59:38 2010 @@ -832,7 +832,7 @@ & ${\displaystyle \frac{E(v)\notin\mathrm{dom}(S)\vee\mathrm{type}(S(E(v)))\neq T,\,\left(E(v),S\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops},S^{\prime}\right)}{\mathtt{guard\_class}(v,T),E,S\overset{\mathrm{opt}}{\Longrightarrow}\mathrm{ops}::\left\langle \mathtt{guard\_class}(E\left(v\right),T)\right\rangle ,E,S^{\prime}}}$\tabularnewline[3em] \emph{lifting} & ${\displaystyle \frac{v^{*}\notin\mathrm{dom}(S)}{v^{*},S\overset{\mathrm{lift}}{\Longrightarrow}\left\langle \,\right\rangle ,S}}$\tabularnewline[3em] & ${\displaystyle \frac{v^{*}\in\mathrm{dom}(S),\,\left(v^{*},S\right)\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\left(\mathrm{ops},S^{\prime}\right)}{v^{*},S\overset{\mathrm{lift}}{\Longrightarrow}\left\langle v^{*}=\mathtt{new}\left(\mathrm{type}\left(S\left(v^{*}\right)\right)\right)\right\rangle ::ops,S^{\prime}}}$\tabularnewline[3em] - & ${\displaystyle \frac{\left(S\left(v^{*}\right)_{L},S\setminus\left\{ v^{*}\mapsto S\left(v^{*}\right)\right\} \right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{L},S^{\prime}\right),\,\left(S\left(v^{*}\right)_{R},S^{\prime}\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{R},S^{\prime\prime}\right)}{v^{*},S\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\mathrm{ops}_{L}::ops_{R}::\left\langle \mathtt{set}\left(v^{*},L,S\left(v^{*}\right)_{L}\right),\,\mathtt{set}\left(v^{*},R,S\left(v^{*}\right)_{R}\right)\right\rangle ,S^{\prime\prime}}}$\tabularnewline[3em] + & ${\displaystyle \frac{\left(S\left(v^{*}\right)_{L},S\setminus\left\{ v^{*}\mapsto S\left(v^{*}\right)\right\} \right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{L},S^{\prime}\right),\,\left(S\left(v^{*}\right)_{R},S^{\prime}\right)\overset{\mathrm{lift}}{\Longrightarrow}\left(\mathrm{ops}_{R},S^{\prime\prime}\right)}{v^{*},S\overset{\mathrm{liftfields}}{=\!=\!\Longrightarrow}\mathrm{ops}_{L}::\mathrm{ops}_{R}::\left\langle \mathtt{set}\left(v^{*},L,S\left(v^{*}\right)_{L}\right),\,\mathtt{set}\left(v^{*},R,S\left(v^{*}\right)_{R}\right)\right\rangle ,S^{\prime\prime}}}$\tabularnewline[3em] \end{tabular} \end{center} From cfbolz at codespeak.net Wed Nov 17 17:49:48 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 17 Nov 2010 17:49:48 +0100 (CET) Subject: [pypy-svn] r79200 - pypy/extradoc/talk/pepm2011 Message-ID: <20101117164948.32BB9282BF0@codespeak.net> Author: cfbolz Date: Wed Nov 17 17:49:45 2010 New Revision: 79200 Modified: pypy/extradoc/talk/pepm2011/paper.tex Log: ignore things where you have to change the language Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Wed Nov 17 17:49:45 2010 @@ -1217,9 +1217,7 @@ deforestation (by Philip Wadler) and "short cut deforestation" by Andrew Gill, John Launchbury, and Simon L Peyton Jones} -\revb{cite and relate to: boxing analysis, -soft typing, dynamic typing, etc (by various authors: Henglein, Cartright, Felleisen, -Leroy)} +\revb{cite and relate to: boxing analysis} %xxx: %partially static data structures: kenichi asai's thesis? From cfbolz at codespeak.net Wed Nov 17 18:40:55 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 17 Nov 2010 18:40:55 +0100 (CET) Subject: [pypy-svn] r79204 - pypy/extradoc/talk/pepm2011 Message-ID: <20101117174055.D3E76282BDC@codespeak.net> Author: cfbolz Date: Wed Nov 17 18:40:54 2010 New Revision: 79204 Modified: pypy/extradoc/talk/pepm2011/paper.bib pypy/extradoc/talk/pepm2011/paper.tex Log: - add two deforestation references - squeeze code a bit to add some space Modified: pypy/extradoc/talk/pepm2011/paper.bib ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.bib (original) +++ pypy/extradoc/talk/pepm2011/paper.bib Wed Nov 17 18:40:54 2010 @@ -150,6 +150,33 @@ pages = {708--725} }, + at inproceedings{wadler_deforestation:_1988, + address = {Amsterdam, The Netherlands, The Netherlands}, + title = {Deforestation: transforming programs to eliminate trees}, + location = {Nancy, France}, + url = {http://portal.acm.org/citation.cfm?id=80098.80104}, + booktitle = {Proceedings of the Second European Symposium on Programming}, + publisher = {{North-Holland} Publishing Co.}, + author = {Philip Wadler}, + year = {1988}, + pages = {231{\textendash}248} +}, + + at inproceedings{gill_short_1993, + address = {New York, {NY,} {USA}}, + series = {{FPCA} '93}, + title = {A short cut to deforestation}, + isbn = {{0-89791-595-X}}, + location = {Copenhagen, Denmark}, + url = {http://doi.acm.org/10.1145/165180.165214}, + doi = {http://doi.acm.org/10.1145/165180.165214}, + booktitle = {Proceedings of the conference on Functional programming languages and computer architecture}, + publisher = {{ACM}}, + author = {Andrew Gill and John Launchbury and Simon L Peyton Jones}, + year = {1993}, + pages = {223{\textendash}232} +}, + @inproceedings{chang_tracing_2009, address = {Washington, {DC,} {USA}}, title = {Tracing for Web 3.0: Trace Compilation for the Next Generation Web Applications}, Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Wed Nov 17 18:40:54 2010 @@ -1,4 +1,4 @@ -\documentclass[preprint]{sigplanconf} +\documentclass{sigplanconf} \usepackage{ifthen} \usepackage{fancyvrb} @@ -76,7 +76,7 @@ \begin{document} \conferenceinfo{PEPM}{'11, Austin, USA} \CopyrightYear{2011} -\copyrightdata{[to be supplied]} +\copyrightdata{[to be supplied]} % XXX \title{Allocation Removal by Partial Evaluation in a Tracing JIT} @@ -319,7 +319,7 @@ Figure~\ref{fig:objmodel} (written in RPython). \begin{figure} -\begin{lstlisting}[mathescape] +\begin{lstlisting}[mathescape,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] class Base(object): pass @@ -390,7 +390,7 @@ \begin{figure} -\begin{lstlisting}[mathescape,numbers = right] +\begin{lstlisting}[mathescape,numbers = right,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] # arguments to the trace: $p_{0}$, $p_{1}$ # inside f: res.add(y) guard_class($p_{1}$, BoxedInteger) @@ -592,7 +592,7 @@ The following operations (lines 10--17) are more interesting: -\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=10] +\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=10,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] $p_{5}$ = new(BoxedInteger) |\setcounter{lstnumber}{11}| set($p_{5}$, intval, $i_{4}$) |\setcounter{lstnumber}{14}| $p_{6}$ = new(BoxedInteger) |\setcounter{lstnumber}{16}| @@ -623,7 +623,7 @@ The subsequent operations (line 20--26) in Figure~\ref{fig:unopt-trace}, which use $p_{5}$ and $p_{6}$, can then be optimized using that knowledge: -\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=20] +\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=20,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] guard_class($p_{5}$, BoxedInteger) |\setcounter{lstnumber}{21}| $i_{7}$ = get($p_{5}$, intval) guard_class($p_{6}$, BoxedInteger) |\setcounter{lstnumber}{24}| @@ -639,7 +639,7 @@ replaced by $i_{4}$ and -100. The only operation copied into the optimized trace is the addition: -\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=26] +\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=26,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] $i_{9}$ = int_add($i_{4}$, -100) \end{lstlisting} @@ -666,7 +666,7 @@ because the static objects could form an arbitrary graph structure. In our example it is simple, though: -\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=44] +\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=44,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] $p_{15}$ = new(BoxedInteger) |\setcounter{lstnumber}{45}| set($p_{15}$, intval, $i_{14}$) |\setcounter{lstnumber}{26}| $p_{10}$ = new(BoxedInteger) |\setcounter{lstnumber}{28}| @@ -692,7 +692,7 @@ original seven. \begin{figure} -\begin{lstlisting}[mathescape,numbers=right,escapechar=|, numberblanklines=false] +\begin{lstlisting}[mathescape,numbers=right,escapechar=|, numberblanklines=false,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] # arguments to the trace: $p_{0}$, $p_{1}$ |\setcounter{lstnumber}{2}| guard_class($p_1$, BoxedInteger) |\setcounter{lstnumber}{4}| $i_2$ = get($p_1$, intval) @@ -932,7 +932,7 @@ v^*), w^* \mapsto (T_2, u^*, u^*)\}$$ which contains two static objects. If $v^*$ needs to be lifted, the following residual operations are produced: -\begin{lstlisting}[mathescape,xleftmargin=20pt] +\begin{lstlisting}[mathescape,xleftmargin=20pt,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] $v^*$ = new($T_1$) $w^*$ = new($T_2$) set($w^*$, $L$, $u^*$) @@ -946,7 +946,7 @@ If we had lifted $w^*$ instead of $v^*$, then the following operations would have been produced: -\begin{lstlisting}[mathescape,xleftmargin=20pt] +\begin{lstlisting}[mathescape,xleftmargin=20pt,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] $w^*$ = new($T_2$) set($w^*$, $L$, $u^*$) set($w^*$, $R$, $u^*$) @@ -1083,7 +1083,7 @@ reasons, it would be very hard to separate the two effects). \begin{figure*} -{\small +{\footnotesize \begin{center} \begin{tabular}{|l||r|rr|rr|rr|rr|} \hline @@ -1132,7 +1132,7 @@ further optimizations. \begin{figure*} -{\small +{\footnotesize \begin{center} \begin{tabular}{|l||r|r||r|r||r|r||r|r|} \hline @@ -1213,10 +1213,8 @@ imperative language. In functional programming this idea was introduced as constructor specialisation by Mogensen \cite{mogensen_constructor_1993}. -\reva{I'd refer here to - deforestation (by Philip Wadler) and "short cut - deforestation" by Andrew Gill, John Launchbury, and Simon L - Peyton Jones} +\reva{I'd refer here to \cite{wadler_deforestation:_1988} + and \cite{gill_short_1993}} \revb{cite and relate to: boxing analysis} %xxx: From cfbolz at codespeak.net Wed Nov 17 18:57:21 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 17 Nov 2010 18:57:21 +0100 (CET) Subject: [pypy-svn] r79209 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101117175721.B2195282BDC@codespeak.net> Author: cfbolz Date: Wed Nov 17 18:57:20 2010 New Revision: 79209 Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py Log: use the official way to check whether w_value is an int Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/converter.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/converter.py Wed Nov 17 18:57:20 2010 @@ -2,7 +2,6 @@ from pypy.interpreter.error import OperationError from pypy.rpython.lltypesystem import rffi, lltype from pypy.rlib.rarithmetic import r_singlefloat -from pypy.objspace.std.intobject import W_IntObject from pypy.module._rawffi.interp_rawffi import unpack_simple_shape @@ -74,7 +73,7 @@ class CharConverter(TypeConverter): def _from_space(self, space, w_value): # allow int to pass to char and make sure that str is of length 1 - if type(w_value) == W_IntObject: + if space.isinstance_w(w_value, space.w_int): try: value = chr(space.c_int_w(w_value)) except ValueError, e: From hakanardo at codespeak.net Wed Nov 17 18:57:26 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Wed, 17 Nov 2010 18:57:26 +0100 (CET) Subject: [pypy-svn] r79210 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test Message-ID: <20101117175726.EF72F282BF2@codespeak.net> Author: hakanardo Date: Wed Nov 17 18:57:24 2010 New Revision: 79210 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py Log: Added a test with a loop that should be specilized into two separate versions. Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py Wed Nov 17 18:57:24 2010 @@ -1803,6 +1803,40 @@ assert res == 8 py.test.raises(AssertGreenFailed, self.interp_operations, f, [8, 0]) + def test_mutiple_specialied_versions(self): + myjitdriver = JitDriver(greens = [], reds = ['y', 'x', 'res']) + class Base: + def __init__(self, val): + self.val = val + class A(Base): + def add(self, other): + return A(self.val + other.val) + class B(Base): + def add(self, other): + return B(self.val * other.val) + def f(x, y): + res = x + while y > 0: + myjitdriver.can_enter_jit(y=y, x=x, res=res) + myjitdriver.jit_merge_point(y=y, x=x, res=res) + res = res.add(x) + y -= 1 + return res + def g(x, y): + a1 = f(A(x), y) + a2 = f(A(x), y) + b1 = f(B(x), y) + b2 = f(B(x), y) + assert a1.val == a2.val + assert b1.val == b2.val + return a1.val + b1.val + res = self.meta_interp(g, [6, 7]) + assert res == 6*8 + 6**8 + self.check_loop_count(5) + self.check_loops({'guard_true': 2, + 'int_add': 2, 'int_sub': 2, 'int_gt': 2, + 'jump': 2}) + class TestOOtype(BasicTests, OOJitMixin): From arigo at codespeak.net Wed Nov 17 19:01:02 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 17 Nov 2010 19:01:02 +0100 (CET) Subject: [pypy-svn] r79211 - pypy/branch/jit-free/pypy/jit/metainterp Message-ID: <20101117180102.42B95282BE3@codespeak.net> Author: arigo Date: Wed Nov 17 19:01:00 2010 New Revision: 79211 Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py Log: Add an assert, maaaaybe useful after translation. Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/compile.py Wed Nov 17 19:01:00 2010 @@ -345,6 +345,7 @@ def compile_and_attach(self, metainterp, new_loop): # We managed to create a bridge. Attach the new operations # to the corresponding guard_op and compile from there + assert metainterp.resumekey_original_loop_token is not None new_loop.token = metainterp.resumekey_original_loop_token inputargs = metainterp.history.inputargs if not we_are_translated(): From hakanardo at codespeak.net Wed Nov 17 19:07:57 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Wed, 17 Nov 2010 19:07:57 +0100 (CET) Subject: [pypy-svn] r79217 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test Message-ID: <20101117180757.D82DA50810@codespeak.net> Author: hakanardo Date: Wed Nov 17 19:07:56 2010 New Revision: 79217 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py Log: Better name Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py Wed Nov 17 19:07:56 2010 @@ -1809,17 +1809,17 @@ def __init__(self, val): self.val = val class A(Base): - def add(self, other): + def binop(self, other): return A(self.val + other.val) class B(Base): - def add(self, other): + def binop(self, other): return B(self.val * other.val) def f(x, y): res = x while y > 0: myjitdriver.can_enter_jit(y=y, x=x, res=res) myjitdriver.jit_merge_point(y=y, x=x, res=res) - res = res.add(x) + res = res.binop(x) y -= 1 return res def g(x, y): @@ -1834,8 +1834,8 @@ assert res == 6*8 + 6**8 self.check_loop_count(5) self.check_loops({'guard_true': 2, - 'int_add': 2, 'int_sub': 2, 'int_gt': 2, - 'jump': 2}) + 'int_add': 1, 'int_mul': 1, 'int_sub': 2, + 'int_gt': 2, 'jump': 2}) class TestOOtype(BasicTests, OOJitMixin): From cfbolz at codespeak.net Wed Nov 17 19:24:24 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 17 Nov 2010 19:24:24 +0100 (CET) Subject: [pypy-svn] r79219 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101117182424.AE606282BEF@codespeak.net> Author: cfbolz Date: Wed Nov 17 19:24:23 2010 New Revision: 79219 Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py Log: - factor some common code out into a helper function - also use space.interp_w to make sure that the objects in question are really W_CPPInstances. This should help translation. Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/converter.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/converter.py Wed Nov 17 19:24:23 2010 @@ -9,13 +9,20 @@ _converters = {} +def get_rawobject(space, w_obj): + from pypy.module.cppyy.interp_cppyy import W_CPPInstance + w_obj = space.findattr(w_obj, space.wrap("_cppinstance")) + obj = space.interp_w(W_CPPInstance, w_obj, can_be_None=True) + return obj.rawobject + + class TypeConverter(object): def __init__(self, space, extra=-1): pass def _get_fieldptr(self, space, w_obj, offset): - obj = space.interpclass_w(space.findattr(w_obj, space.wrap("_cppinstance"))) - return lltype.direct_ptradd(obj.rawobject, offset) + rawobject = get_rawobject(space, w_obj) + return lltype.direct_ptradd(rawobject, offset) def _is_abstract(self): raise NotImplementedError( @@ -188,8 +195,8 @@ def to_memory(self, space, w_obj, w_value, offset): # copy only the pointer value - obj = space.interpclass_w(space.findattr(w_obj, space.wrap("_cppinstance"))) - byteptr = rffi.cast(rffi.LONGP, obj.rawobject[offset]) + rawobject = get_rawobject(space, w_obj) + byteptr = rffi.cast(rffi.LONGP, rawobject[offset]) # TODO: now what ... ?? AFAICS, w_value is a pure python list, not an array? # byteptr[0] = space.unwrap(space.id(w_value.getslotvalue(2))) @@ -218,8 +225,8 @@ def to_memory(self, space, w_obj, w_value, offset): # copy only the pointer value - obj = space.interpclass_w(space.findattr(w_obj, space.wrap("_cppinstance"))) - byteptr = rffi.cast(rffi.LONGP, obj.rawobject[offset]) + rawobject = get_rawobject(space, w_obj) + byteptr = rffi.cast(rffi.LONGP, rawobject[offset]) # TODO: now what ... ?? AFAICS, w_value is a pure python list, not an array? # byteptr[0] = space.unwrap(space.id(w_value.getslotvalue(2))) From cfbolz at codespeak.net Wed Nov 17 19:48:00 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 17 Nov 2010 19:48:00 +0100 (CET) Subject: [pypy-svn] r79220 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101117184800.D41AC282BEF@codespeak.net> Author: cfbolz Date: Wed Nov 17 19:47:56 2010 New Revision: 79220 Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py Log: fix another issue Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/converter.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/converter.py Wed Nov 17 19:47:56 2010 @@ -91,7 +91,7 @@ if len(value) != 1: raise OperationError(space.w_TypeError, space.wrap("char expecter, got string of size %d" % len(value))) - return value + return value[0] # turn it into a "char" to the annotator def convert_argument(self, space, w_obj): arg = self._from_space(space, w_obj) From afa at codespeak.net Wed Nov 17 20:02:24 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 17 Nov 2010 20:02:24 +0100 (CET) Subject: [pypy-svn] r79223 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101117190224.EDA54282BF0@codespeak.net> Author: afa Date: Wed Nov 17 20:02:23 2010 New Revision: 79223 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Log: Have BufferedReader.read() used raw.readinto just like CPython. Implement a true BytesIO.readinto() Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Wed Nov 17 20:02:23 2010 @@ -4,9 +4,10 @@ from pypy.interpreter.gateway import interp2app, unwrap_spec, Arguments from pypy.interpreter.baseobjspace import ObjSpace, W_Root from pypy.interpreter.error import OperationError, operationerrfmt +from pypy.interpreter.buffer import RWBuffer from pypy.rpython.lltypesystem import lltype, rffi from pypy.rlib.rstring import StringBuilder -from pypy.rlib.rarithmetic import r_longlong +from pypy.rlib.rarithmetic import r_longlong, intmask from pypy.tool.sourcetools import func_renamer from pypy.module._io.interp_iobase import ( W_IOBase, convert_size, @@ -71,6 +72,17 @@ readinto = interp2app(W_BufferedIOBase.readinto_w), ) +class RawBuffer(RWBuffer): + def __init__(self, buf, length): + self.buf = buf + self.length = length + + def getlength(self): + return self.length + + def setitem(self, index, char): + self.buf[index] = char + class BufferedMixin: _mixin_ = True @@ -480,25 +492,24 @@ self.abs_pos += size return builder.build() - def _raw_read(self, space, n): - w_data = space.call_method(self.w_raw, "read", space.wrap(n)) - if space.is_w(w_data, space.w_None): + def _raw_read(self, space, buffer, start, length): + length = intmask(length) + w_buf = space.wrap(RawBuffer(rffi.ptradd(buffer, start), length)) + w_size = space.call_method(self.w_raw, "readinto", w_buf) + if space.is_w(w_size, space.w_None): raise BlockingIOError() - data = space.str_w(w_data) + size = space.int_w(w_size) if self.abs_pos != -1: - self.abs_pos += len(data) - return data + self.abs_pos += size + return size def _fill_buffer(self, space): start = self.read_end if start == -1: start = 0 length = self.buffer_size - start - data = self._raw_read(space, length) - size = len(data) + size = self._raw_read(space, self.buffer, start, length) if size > 0: - for i in range(size): - self.buffer[start + i] = data[i] self.read_end = self.raw_pos = start + size return size @@ -509,70 +520,64 @@ if n <= current_size: return self._read_fast(n) - builder = StringBuilder(n) - remaining = n - written = 0 - data = None - if current_size: - data = rffi.charpsize2str(rffi.ptradd(self.buffer, self.pos), - current_size) - builder.append(data) - remaining -= len(data) - written += len(data) - self._reader_reset_buf() + with rffi.scoped_alloc_buffer(n) as result_buffer: + remaining = n + written = 0 + data = None + if current_size: + for i in range(current_size): + result_buffer.raw[written + i] = self.buffer[self.pos + i] + remaining -= current_size + written += current_size + self._reader_reset_buf() - # XXX potential bug in CPython? The following is not enabled. - # We're going past the buffer's bounds, flush it - ## if self.writable: - ## self._writer_flush_unlocked(space, restore_pos=True) + # XXX potential bug in CPython? The following is not enabled. + # We're going past the buffer's bounds, flush it + ## if self.writable: + ## self._writer_flush_unlocked(space, restore_pos=True) + + # Read whole blocks, and don't buffer them + while remaining > 0: + r = self.buffer_size * (remaining // self.buffer_size) + if r == 0: + break + try: + size = self._raw_read(space, result_buffer.raw, written, r) + except BlockingIOError: + if written == 0: + return None + size = 0 + if size == 0: + return result_buffer.str(intmask(written)) + remaining -= size + written += size - # Read whole blocks, and don't buffer them - while remaining > 0: - r = self.buffer_size * (remaining // self.buffer_size) - if r == 0: - break - try: - data = self._raw_read(space, r) - except BlockingIOError: - if written == 0: - return None - data = "" - size = len(data) - if size == 0: - return builder.build() - builder.append(data) - remaining -= size - written += size + self.pos = 0 + self.raw_pos = 0 + self.read_end = 0 - self.pos = 0 - self.raw_pos = 0 - self.read_end = 0 + while remaining > 0 and self.read_end < self.buffer_size: + try: + size = self._fill_buffer(space) + except BlockingIOError: + # EOF or read() would block + if written == 0: + return None + size = 0 + if size == 0: + break - while remaining > 0 and self.read_end < self.buffer_size: - try: - size = self._fill_buffer(space) - except BlockingIOError: - # EOF or read() would block - if written == 0: - return None - size = 0 - if size == 0: - break + if remaining > 0: + if size > remaining: + size = remaining + for i in range(size): + result_buffer.raw[written + i] = self.buffer[self.pos + i] + self.pos += size - if remaining > 0: - if size > remaining: - size = remaining - # XXX inefficient - l = [] - for i in range(self.pos,self.pos + size): - l.append(self.buffer[i]) - data = ''.join(l) - builder.append(data) + written += size + remaining -= size - written += size - self.pos += size - remaining -= size - return builder.build() + return result_buffer.str(intmask(written)) def _read_fast(self, n): """Read n bytes from the buffer if it can, otherwise return None. Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py Wed Nov 17 20:02:23 2010 @@ -49,6 +49,20 @@ return space.wrap(output) @unwrap_spec('self', ObjSpace, W_Root) + def readinto_w(self, space, w_buffer): + rwbuffer = space.rwbuffer_w(w_buffer) + size = rwbuffer.getlength() + + if self.pos + size > self.string_size: + size = self.string_size - self.pos + + output = buffer2string(self.buf, self.pos, self.pos + size) + length = len(output) + rwbuffer.setslice(0, output) + self.pos += length + return space.wrap(length) + + @unwrap_spec('self', ObjSpace, W_Root) def write_w(self, space, w_data): self._check_closed(space) buf = space.buffer_w(w_data) @@ -157,6 +171,7 @@ __init__ = interp2app(W_BytesIO.descr_init), read = interp2app(W_BytesIO.read_w), + readinto = interp2app(W_BytesIO.readinto_w), write = interp2app(W_BytesIO.write_w), truncate = interp2app(W_BytesIO.truncate_w), getvalue = interp2app(W_BytesIO.getvalue_w), Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Wed Nov 17 20:02:23 2010 @@ -37,15 +37,65 @@ class MockIO(_io._IOBase): def readable(self): return True - def read(self, n=-1): # PyPy uses read() - return "abc" - def readinto(self, buf): # CPython uses readinto() + def readinto(self, buf): buf[:3] = "abc" return 3 bufio = _io.BufferedReader(MockIO()) r = bufio.read(5) assert r == "abcab" + def test_read_past_eof(self): + import _io + class MockIO(_io._IOBase): + stack = ["abc", "d", "efg"] + def readable(self): + return True + def readinto(self, buf): + if self.stack: + data = self.stack.pop(0) + buf[:len(data)] = data + return len(data) + else: + return 0 + bufio = _io.BufferedReader(MockIO()) + assert bufio.read(9000) == "abcdefg" + + def test_buffering(self): + import _io + data = b"abcdefghi" + dlen = len(data) + class MockFileIO(_io.BytesIO): + def __init__(self, data): + self.read_history = [] + _io.BytesIO.__init__(self, data) + + def read(self, n=None): + res = _io.BytesIO.read(self, n) + self.read_history.append(None if res is None else len(res)) + return res + + def readinto(self, b): + res = _io.BytesIO.readinto(self, b) + self.read_history.append(res) + return res + + + tests = [ + [ 100, [ 3, 1, 4, 8 ], [ dlen, 0 ] ], + [ 100, [ 3, 3, 3], [ dlen ] ], + [ 4, [ 1, 2, 4, 2 ], [ 4, 4, 1 ] ], + ] + + for bufsize, buf_read_sizes, raw_read_sizes in tests: + rawio = MockFileIO(data) + bufio = _io.BufferedReader(rawio, buffer_size=bufsize) + pos = 0 + for nbytes in buf_read_sizes: + assert bufio.read(nbytes) == data[pos:pos+nbytes] + pos += nbytes + # this is mildly implementation-dependent + assert rawio.read_history == raw_read_sizes + def test_peek(self): import _io raw = _io.FileIO(self.tmpfile) From hakanardo at codespeak.net Wed Nov 17 20:41:33 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Wed, 17 Nov 2010 20:41:33 +0100 (CET) Subject: [pypy-svn] r79225 - in pypy/branch/jit-unroll-loops: . lib-python/modified-2.5.2/distutils lib_pypy/ctypes_config_cache lib_pypy/ctypes_config_cache/test pypy pypy/annotation pypy/config pypy/doc pypy/doc/config pypy/interpreter pypy/interpreter/test pypy/jit/backend pypy/jit/backend/llgraph pypy/jit/backend/llsupport pypy/jit/backend/llsupport/test pypy/jit/backend/test pypy/jit/backend/x86 pypy/jit/backend/x86/test pypy/jit/backend/x86/tool pypy/jit/codewriter pypy/jit/codewriter/test pypy/jit/metainterp pypy/jit/metainterp/optimizeopt pypy/jit/metainterp/test pypy/jit/tool pypy/jit/tool/test pypy/module/__builtin__ pypy/module/__pypy__ pypy/module/__pypy__/test pypy/module/_rawffi/test pypy/module/_stackless/test pypy/module/array pypy/module/array/benchmark pypy/module/array/test pypy/module/cpyext pypy/module/cpyext/test pypy/module/gc pypy/module/gc/test pypy/module/imp pypy/module/imp/test pypy/module/posix pypy/module/pypyjit pypy/module/pypyjit/test pypy/module/rctime pypy/module/signal pypy/module/signal/test pypy/module/sys pypy/module/thread pypy/module/thread/test pypy/objspace/flow pypy/objspace/flow/test pypy/objspace/std pypy/objspace/std/test pypy/rlib pypy/rlib/test pypy/rpython pypy/rpython/lltypesystem pypy/rpython/memory pypy/rpython/memory/gc pypy/rpython/memory/gc/test pypy/rpython/memory/gctransform pypy/rpython/memory/test pypy/rpython/test pypy/tool pypy/translator/c pypy/translator/c/gcc pypy/translator/c/src pypy/translator/c/test pypy/translator/cli/src pypy/translator/goal pypy/translator/goal/test2 Message-ID: <20101117194133.6A645282BEB@codespeak.net> Author: hakanardo Date: Wed Nov 17 20:41:25 2010 New Revision: 79225 Added: pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/dumpcache.py - copied unchanged from r79224, pypy/trunk/lib_pypy/ctypes_config_cache/dumpcache.py pypy/branch/jit-unroll-loops/pypy/doc/config/objspace.soabi.txt - copied unchanged from r79224, pypy/trunk/pypy/doc/config/objspace.soabi.txt pypy/branch/jit-unroll-loops/pypy/doc/config/objspace.translationmodules.txt - copied unchanged from r79224, pypy/trunk/pypy/doc/config/objspace.translationmodules.txt pypy/branch/jit-unroll-loops/pypy/doc/release-1.4.0.txt - copied unchanged from r79224, pypy/trunk/pypy/doc/release-1.4.0.txt pypy/branch/jit-unroll-loops/pypy/jit/tool/loopcounter.py - copied unchanged from r79224, pypy/trunk/pypy/jit/tool/loopcounter.py pypy/branch/jit-unroll-loops/pypy/jit/tool/test/test_oparser.py - copied unchanged from r79224, pypy/trunk/pypy/jit/tool/test/test_oparser.py pypy/branch/jit-unroll-loops/pypy/module/_stackless/test/conftest.py - copied unchanged from r79224, pypy/trunk/pypy/module/_stackless/test/conftest.py pypy/branch/jit-unroll-loops/pypy/tool/gcdump.py - copied unchanged from r79224, pypy/trunk/pypy/tool/gcdump.py Removed: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_oparser.py Modified: pypy/branch/jit-unroll-loops/ (props changed) pypy/branch/jit-unroll-loops/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/ (props changed) pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/hashlib.ctc.py pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/locale.ctc.py pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/pyexpat.ctc.py pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/rebuild.py pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/resource.ctc.py pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/syslog.ctc.py pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/test/test_cache.py pypy/branch/jit-unroll-loops/pypy/ (props changed) pypy/branch/jit-unroll-loops/pypy/annotation/bookkeeper.py pypy/branch/jit-unroll-loops/pypy/annotation/builtin.py pypy/branch/jit-unroll-loops/pypy/config/pypyoption.py pypy/branch/jit-unroll-loops/pypy/conftest.py pypy/branch/jit-unroll-loops/pypy/interpreter/argument.py pypy/branch/jit-unroll-loops/pypy/interpreter/baseobjspace.py pypy/branch/jit-unroll-loops/pypy/interpreter/executioncontext.py pypy/branch/jit-unroll-loops/pypy/interpreter/pyopcode.py pypy/branch/jit-unroll-loops/pypy/interpreter/test/test_argument.py pypy/branch/jit-unroll-loops/pypy/interpreter/test/test_executioncontext.py pypy/branch/jit-unroll-loops/pypy/jit/backend/detect_cpu.py pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/descr.py pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/gc.py pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_descr.py pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_gc.py pypy/branch/jit-unroll-loops/pypy/jit/backend/test/runner_test.py pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/assembler.py pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_runner.py pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_zrpy_gc.py pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_ztranslation.py pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/tool/viewcode.py pypy/branch/jit-unroll-loops/pypy/jit/codewriter/assembler.py pypy/branch/jit-unroll-loops/pypy/jit/codewriter/call.py pypy/branch/jit-unroll-loops/pypy/jit/codewriter/codewriter.py pypy/branch/jit-unroll-loops/pypy/jit/codewriter/effectinfo.py pypy/branch/jit-unroll-loops/pypy/jit/codewriter/jtransform.py pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_jtransform.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/blackhole.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/logger.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (contents, props changed) pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resoperation.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_del.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_logger.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resoperation.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send_nounroll.py (props changed) pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualref.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_ztranslation.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/virtualizable.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmspot.py pypy/branch/jit-unroll-loops/pypy/jit/tool/oparser.py pypy/branch/jit-unroll-loops/pypy/jit/tool/pypytrace-mode.el pypy/branch/jit-unroll-loops/pypy/jit/tool/pypytrace.vim pypy/branch/jit-unroll-loops/pypy/jit/tool/test/test_traceviewer.py pypy/branch/jit-unroll-loops/pypy/module/__builtin__/descriptor.py pypy/branch/jit-unroll-loops/pypy/module/__pypy__/__init__.py pypy/branch/jit-unroll-loops/pypy/module/__pypy__/interp_magic.py pypy/branch/jit-unroll-loops/pypy/module/__pypy__/test/test_special.py pypy/branch/jit-unroll-loops/pypy/module/_rawffi/test/test__rawffi.py pypy/branch/jit-unroll-loops/pypy/module/_rawffi/test/test_nested.py pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/Makefile (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/intimg.c (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/loop.c (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/sum.c (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/sumtst.c (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/benchmark/sumtst.py (props changed) pypy/branch/jit-unroll-loops/pypy/module/array/interp_array.py pypy/branch/jit-unroll-loops/pypy/module/array/test/test_array_old.py (props changed) pypy/branch/jit-unroll-loops/pypy/module/cpyext/api.py pypy/branch/jit-unroll-loops/pypy/module/cpyext/cdatetime.py pypy/branch/jit-unroll-loops/pypy/module/cpyext/presetup.py pypy/branch/jit-unroll-loops/pypy/module/cpyext/pyobject.py pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_borrow.py pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_cpyext.py pypy/branch/jit-unroll-loops/pypy/module/cpyext/typeobject.py pypy/branch/jit-unroll-loops/pypy/module/gc/__init__.py pypy/branch/jit-unroll-loops/pypy/module/gc/app_referents.py pypy/branch/jit-unroll-loops/pypy/module/gc/interp_gc.py pypy/branch/jit-unroll-loops/pypy/module/gc/referents.py pypy/branch/jit-unroll-loops/pypy/module/gc/test/test_gc.py pypy/branch/jit-unroll-loops/pypy/module/imp/importing.py pypy/branch/jit-unroll-loops/pypy/module/imp/interp_imp.py pypy/branch/jit-unroll-loops/pypy/module/imp/test/test_app.py pypy/branch/jit-unroll-loops/pypy/module/imp/test/test_import.py pypy/branch/jit-unroll-loops/pypy/module/posix/interp_posix.py pypy/branch/jit-unroll-loops/pypy/module/pypyjit/interp_jit.py pypy/branch/jit-unroll-loops/pypy/module/pypyjit/policy.py pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_policy.py pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py pypy/branch/jit-unroll-loops/pypy/module/rctime/interp_time.py pypy/branch/jit-unroll-loops/pypy/module/signal/__init__.py pypy/branch/jit-unroll-loops/pypy/module/signal/interp_signal.py pypy/branch/jit-unroll-loops/pypy/module/signal/test/test_signal.py pypy/branch/jit-unroll-loops/pypy/module/sys/__init__.py pypy/branch/jit-unroll-loops/pypy/module/sys/version.py pypy/branch/jit-unroll-loops/pypy/module/sys/vm.py pypy/branch/jit-unroll-loops/pypy/module/thread/gil.py pypy/branch/jit-unroll-loops/pypy/module/thread/test/test_gil.py pypy/branch/jit-unroll-loops/pypy/objspace/flow/model.py pypy/branch/jit-unroll-loops/pypy/objspace/flow/objspace.py pypy/branch/jit-unroll-loops/pypy/objspace/flow/test/test_objspace.py pypy/branch/jit-unroll-loops/pypy/objspace/std/callmethod.py pypy/branch/jit-unroll-loops/pypy/objspace/std/mapdict.py pypy/branch/jit-unroll-loops/pypy/objspace/std/stringobject.py pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_dictmultiobject.py pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_mapdict.py pypy/branch/jit-unroll-loops/pypy/rlib/clibffi.py pypy/branch/jit-unroll-loops/pypy/rlib/jit.py pypy/branch/jit-unroll-loops/pypy/rlib/rarithmetic.py pypy/branch/jit-unroll-loops/pypy/rlib/rerased.py (props changed) pypy/branch/jit-unroll-loops/pypy/rlib/rgc.py pypy/branch/jit-unroll-loops/pypy/rlib/streamio.py pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rarithmetic.py pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rerased.py (props changed) pypy/branch/jit-unroll-loops/pypy/rpython/annlowlevel.py pypy/branch/jit-unroll-loops/pypy/rpython/llinterp.py pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llarena.py pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llmemory.py pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lloperation.py pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lltype.py pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rffi.py pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/base.py pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/generation.py pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/inspector.py pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/minimark.py pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/semispace.py pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/test/test_minimark.py pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/framework.py pypy/branch/jit-unroll-loops/pypy/rpython/memory/support.py pypy/branch/jit-unroll-loops/pypy/rpython/memory/test/test_gc.py pypy/branch/jit-unroll-loops/pypy/rpython/rbuiltin.py pypy/branch/jit-unroll-loops/pypy/rpython/rmodel.py pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rpbc.py pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/trackgcroot.py pypy/branch/jit-unroll-loops/pypy/translator/c/genc.py pypy/branch/jit-unroll-loops/pypy/translator/c/node.py pypy/branch/jit-unroll-loops/pypy/translator/c/src/asm_gcc_x86.h pypy/branch/jit-unroll-loops/pypy/translator/c/src/g_include.h pypy/branch/jit-unroll-loops/pypy/translator/c/src/signals.h pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_lltyped.py pypy/branch/jit-unroll-loops/pypy/translator/c/test/test_newgc.py pypy/branch/jit-unroll-loops/pypy/translator/cli/src/pypylib.cs pypy/branch/jit-unroll-loops/pypy/translator/goal/app_main.py pypy/branch/jit-unroll-loops/pypy/translator/goal/targetpypystandalone.py pypy/branch/jit-unroll-loops/pypy/translator/goal/test2/test_app_main.py Log: svn merge -r78744:HEAD svn+ssh://hakanardo at codespeak.net/svn/pypy/trunk Modified: pypy/branch/jit-unroll-loops/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py ============================================================================== --- pypy/branch/jit-unroll-loops/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py (original) +++ pypy/branch/jit-unroll-loops/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py Wed Nov 17 20:41:25 2010 @@ -3,6 +3,7 @@ import sys import os +import imp from distutils.errors import DistutilsPlatformError @@ -47,11 +48,17 @@ _config_vars = None +def _get_so_extension(): + for ext, mod, typ in imp.get_suffixes(): + if typ == imp.C_EXTENSION: + return ext + def _init_posix(): """Initialize the module as appropriate for POSIX systems.""" g = {} g['EXE'] = "" - g['SO'] = ".so" + g['SO'] = _get_so_extension() or ".so" + g['SOABI'] = g['SO'].rsplit('.')[0] global _config_vars _config_vars = g @@ -61,7 +68,8 @@ """Initialize the module as appropriate for NT""" g = {} g['EXE'] = ".exe" - g['SO'] = ".pyd" + g['SO'] = _get_so_extension() or ".pyd" + g['SOABI'] = g['SO'].rsplit('.')[0] global _config_vars _config_vars = g Modified: pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/hashlib.ctc.py ============================================================================== --- pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/hashlib.ctc.py (original) +++ pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/hashlib.ctc.py Wed Nov 17 20:41:25 2010 @@ -4,7 +4,8 @@ """ from ctypes import * -from ctypes_configure import configure, dumpcache +from ctypes_configure import configure +import dumpcache class CConfig: @@ -17,4 +18,4 @@ [('digest', c_void_p)]) config = configure.configure(CConfig) -dumpcache.dumpcache(__file__, '_hashlib_cache.py', config) +dumpcache.dumpcache2('hashlib', config) Modified: pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/locale.ctc.py ============================================================================== --- pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/locale.ctc.py (original) +++ pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/locale.ctc.py Wed Nov 17 20:41:25 2010 @@ -5,7 +5,7 @@ from ctypes_configure.configure import (configure, ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger, SimpleType, check_eci) -from ctypes_configure.dumpcache import dumpcache +import dumpcache # ____________________________________________________________ @@ -70,4 +70,4 @@ config['ALL_CONSTANTS'] = tuple(_CONSTANTS) config['HAS_LANGINFO'] = HAS_LANGINFO -dumpcache(__file__, '_locale_cache.py', config) +dumpcache.dumpcache2('locale', config) Modified: pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/pyexpat.ctc.py ============================================================================== --- pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/pyexpat.ctc.py (original) +++ pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/pyexpat.ctc.py Wed Nov 17 20:41:25 2010 @@ -5,7 +5,8 @@ import ctypes from ctypes import c_char_p, c_int, c_void_p, c_char -from ctypes_configure import configure, dumpcache +from ctypes_configure import configure +import dumpcache class CConfigure: @@ -41,4 +42,4 @@ config = configure.configure(CConfigure) -dumpcache.dumpcache(__file__, '_pyexpat_cache.py', config) +dumpcache.dumpcache2('pyexpat', config) Modified: pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/rebuild.py ============================================================================== --- pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/rebuild.py (original) +++ pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/rebuild.py Wed Nov 17 20:41:25 2010 @@ -31,10 +31,25 @@ sys.path[:] = path def try_rebuild(): + from pypy.jit.backend import detect_cpu + model = detect_cpu.autodetect_main_model_and_size() + # remove the files '_*_model_.py' + left = {} + for p in os.listdir(_dirpath): + if p.startswith('_') and (p.endswith('_%s_.py' % model) or + p.endswith('_%s_.pyc' % model)): + os.unlink(os.path.join(_dirpath, p)) + elif p.startswith('_') and (p.endswith('_.py') or + p.endswith('_.pyc')): + for i in range(2, len(p)-4): + left[p[:i]] = True + # remove the files '_*_cache.py' if there is no '_*_*_.py' left around for p in os.listdir(_dirpath): if p.startswith('_') and (p.endswith('_cache.py') or p.endswith('_cache.pyc')): - os.unlink(os.path.join(_dirpath, p)) + if p[:-9] not in left: + os.unlink(os.path.join(_dirpath, p)) + # for p in os.listdir(_dirpath): if p.endswith('.ctc.py'): try: Modified: pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/resource.ctc.py ============================================================================== --- pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/resource.ctc.py (original) +++ pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/resource.ctc.py Wed Nov 17 20:41:25 2010 @@ -4,7 +4,7 @@ """ from ctypes import sizeof -from ctypes_configure.dumpcache import dumpcache +import dumpcache from ctypes_configure.configure import (configure, ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger, SimpleType) @@ -58,4 +58,4 @@ del config[key] config['ALL_CONSTANTS'] = _CONSTANTS + tuple(optional_constants) -dumpcache(__file__, '_resource_cache.py', config) +dumpcache.dumpcache2('resource', config) Modified: pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/syslog.ctc.py ============================================================================== --- pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/syslog.ctc.py (original) +++ pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/syslog.ctc.py Wed Nov 17 20:41:25 2010 @@ -5,7 +5,7 @@ from ctypes_configure.configure import (configure, ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger) -from ctypes_configure.dumpcache import dumpcache +import dumpcache _CONSTANTS = ( @@ -72,4 +72,4 @@ all_constants = config.keys() all_constants.sort() config['ALL_CONSTANTS'] = tuple(all_constants) -dumpcache(__file__, '_syslog_cache.py', config) +dumpcache.dumpcache2('syslog', config) Modified: pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/test/test_cache.py ============================================================================== --- pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/test/test_cache.py (original) +++ pypy/branch/jit-unroll-loops/lib_pypy/ctypes_config_cache/test/test_cache.py Wed Nov 17 20:41:25 2010 @@ -7,19 +7,26 @@ def run(filename, outputname): filepath = dirpath.join(filename) - tmpdir = udir.ensure('testcache-' + filename, dir=True) - outputpath = tmpdir.join(outputname) - d = {'__file__': str(outputpath)} + tmpdir = udir.ensure('testcache-' + os.path.splitext(filename)[0], + dir=True) + tmpdir.join('dumpcache.py').write(dirpath.join('dumpcache.py').read()) path = sys.path[:] try: - sys.path.insert(0, str(dirpath)) - execfile(str(filepath), d) + sys.path.insert(0, str(tmpdir)) + execfile(str(filepath), {}) finally: sys.path[:] = path + sys.modules.pop('dumpcache', None) # + outputpath = tmpdir.join(outputname) assert outputpath.check(exists=1) - d = {} - execfile(str(outputpath), d) + modname = os.path.splitext(outputname)[0] + try: + sys.path.insert(0, str(tmpdir)) + d = {} + execfile(str(outputpath), d) + finally: + sys.path[:] = path return d Modified: pypy/branch/jit-unroll-loops/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/annotation/bookkeeper.py (original) +++ pypy/branch/jit-unroll-loops/pypy/annotation/bookkeeper.py Wed Nov 17 20:41:25 2010 @@ -87,13 +87,6 @@ else: return obj.knowntype.__name__ - def consider_tuple_iter(self, tup): - ctxt = "[%s]" % sys._getframe(4).f_code.co_name - if tup.is_constant(): - return ctxt, tup.const - else: - return ctxt, tuple([self.typerepr(x) for x in tup.items]) - def consider_tuple_random_getitem(self, tup): return tuple([self.typerepr(x) for x in tup.items]) Modified: pypy/branch/jit-unroll-loops/pypy/annotation/builtin.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/annotation/builtin.py (original) +++ pypy/branch/jit-unroll-loops/pypy/annotation/builtin.py Wed Nov 17 20:41:25 2010 @@ -453,6 +453,9 @@ #p = lltype.malloc(T, flavor=s_flavor.const) #lltype.free(p, flavor=s_flavor.const) +def render_immortal(s_p, s_track_allocation=None): + assert s_track_allocation is None or s_track_allocation.is_constant() + def typeOf(s_val): lltype = annotation_to_lltype(s_val, info="in typeOf(): ") return immutablevalue(lltype) @@ -520,6 +523,7 @@ BUILTIN_ANALYZERS[lltype.malloc] = malloc BUILTIN_ANALYZERS[lltype.free] = free +BUILTIN_ANALYZERS[lltype.render_immortal] = render_immortal BUILTIN_ANALYZERS[lltype.typeOf] = typeOf BUILTIN_ANALYZERS[lltype.cast_primitive] = cast_primitive BUILTIN_ANALYZERS[lltype.nullptr] = nullptr Modified: pypy/branch/jit-unroll-loops/pypy/config/pypyoption.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/config/pypyoption.py (original) +++ pypy/branch/jit-unroll-loops/pypy/config/pypyoption.py Wed Nov 17 20:41:25 2010 @@ -34,6 +34,11 @@ "_bisect"] )) +translation_modules = default_modules.copy() +translation_modules.update(dict.fromkeys( + ["fcntl", "rctime", "select", "signal", "_rawffi", "zlib", + "struct", "md5", "cStringIO", "array"])) + working_oo_modules = default_modules.copy() working_oo_modules.update(dict.fromkeys( ["md5", "sha", "cStringIO", "itertools"] @@ -149,6 +154,12 @@ cmdline="--allworkingmodules", negation=True), + BoolOption("translationmodules", + "use only those modules that are needed to run translate.py on pypy", + default=False, + cmdline="--translationmodules", + suggests=[("objspace.allworkingmodules", False)]), + BoolOption("geninterp", "specify whether geninterp should be used", cmdline=None, default=True), @@ -164,6 +175,11 @@ default=False, requires=[("objspace.usepycfiles", True)]), + StrOption("soabi", + "Tag to differentiate extension modules built for different Python interpreters", + cmdline="--soabi", + default=None), + BoolOption("honor__builtins__", "Honor the __builtins__ key of a module dictionary", default=False), @@ -369,6 +385,11 @@ modules = [name for name in modules if name not in essential_modules] config.objspace.usemodules.suggest(**dict.fromkeys(modules, True)) +def enable_translationmodules(config): + modules = translation_modules + modules = [name for name in modules if name not in essential_modules] + config.objspace.usemodules.suggest(**dict.fromkeys(modules, True)) + if __name__ == '__main__': config = get_pypy_config() Modified: pypy/branch/jit-unroll-loops/pypy/conftest.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/conftest.py (original) +++ pypy/branch/jit-unroll-loops/pypy/conftest.py Wed Nov 17 20:41:25 2010 @@ -327,6 +327,31 @@ class PyPyTestFunction(py.test.collect.Function): # All PyPy test items catch and display OperationErrors specially. # + def runtest(self): + self.runtest_open() + try: + self.runtest_perform() + finally: + self.runtest_close() + self.runtest_finish() + + def runtest_open(self): + leakfinder.start_tracking_allocations() + + def runtest_perform(self): + super(PyPyTestFunction, self).runtest() + + def runtest_close(self): + if leakfinder.TRACK_ALLOCATIONS: + self._pypytest_leaks = leakfinder.stop_tracking_allocations(False) + else: # stop_tracking_allocations() already called + self._pypytest_leaks = None + + def runtest_finish(self): + # check for leaks, but only if the test passed so far + if self._pypytest_leaks: + raise leakfinder.MallocMismatch(self._pypytest_leaks) + def execute_appex(self, space, target, *args): try: target(*args) @@ -353,16 +378,9 @@ def _keywords(self): return super(IntTestFunction, self)._keywords() + ['interplevel'] - def runtest(self): + def runtest_perform(self): try: - leakfinder.start_tracking_allocations() - try: - super(IntTestFunction, self).runtest() - finally: - if leakfinder.TRACK_ALLOCATIONS: - leaks = leakfinder.stop_tracking_allocations(False) - else: - leaks = None # stop_tracking_allocations() already called + super(IntTestFunction, self).runtest_perform() except OperationError, e: check_keyboard_interrupt(e) raise @@ -375,14 +393,15 @@ py.test.skip('%s: %s' % (e.__class__.__name__, e)) cls = cls.__bases__[0] raise + + def runtest_finish(self): if 'pygame' in sys.modules: global _pygame_imported if not _pygame_imported: _pygame_imported = True assert option.view, ("should not invoke Pygame " "if conftest.option.view is False") - if leaks: # check for leaks, but only if the test passed so far - raise leakfinder.MallocMismatch(leaks) + super(IntTestFunction, self).runtest_finish() class AppTestFunction(PyPyTestFunction): def _prunetraceback(self, traceback): @@ -395,7 +414,7 @@ def _keywords(self): return ['applevel'] + super(AppTestFunction, self)._keywords() - def runtest(self): + def runtest_perform(self): target = self.obj if option.runappdirect: return target() @@ -427,7 +446,7 @@ space.setattr(w_instance, space.wrap(name[2:]), getattr(instance, name)) - def runtest(self): + def runtest_perform(self): target = self.obj if option.runappdirect: return target() Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/interpreter/argument.py (original) +++ pypy/branch/jit-unroll-loops/pypy/interpreter/argument.py Wed Nov 17 20:41:25 2010 @@ -64,7 +64,7 @@ return not self == other - # make it look tuply for the annotator + # make it look tuply for its use in the annotator def __len__(self): return 3 @@ -103,10 +103,11 @@ make_sure_not_resized(self.keywords_w) make_sure_not_resized(self.arguments_w) - if ((w_stararg is not None and w_stararg) or - (w_starstararg is not None and w_starstararg)): - self._combine_wrapped(w_stararg, w_starstararg) - # if we have a call where * or ** args are used at the callsite + if w_stararg is not None and space.is_true(w_stararg): + self._combine_starargs_wrapped(w_stararg) + if w_starstararg is not None and space.is_true(w_starstararg): + self._combine_starstarargs_wrapped(w_starstararg) + # if we have a call where **args are used at the callsite # we shouldn't let the JIT see the argument matching self._dont_jit = True else: @@ -142,42 +143,48 @@ def _combine_wrapped(self, w_stararg, w_starstararg): "unpack the *arg and **kwd into arguments_w and keywords_w" - # unpack the * arguments if w_stararg is not None: - self.arguments_w = (self.arguments_w + - self.space.fixedview(w_stararg)) - # unpack the ** arguments + self._combine_starargs_wrapped(w_stararg) if w_starstararg is not None: - space = self.space - if not space.is_true(space.isinstance(w_starstararg, space.w_dict)): + self._combine_starstarargs_wrapped(w_starstararg) + + def _combine_starargs_wrapped(self, w_stararg): + # unpack the * arguments + self.arguments_w = (self.arguments_w + + self.space.fixedview(w_stararg)) + + def _combine_starstarargs_wrapped(self, w_starstararg): + # unpack the ** arguments + space = self.space + if not space.is_true(space.isinstance(w_starstararg, space.w_dict)): + raise OperationError(space.w_TypeError, + space.wrap("argument after ** must be " + "a dictionary")) + keywords_w = [None] * space.int_w(space.len(w_starstararg)) + keywords = [None] * space.int_w(space.len(w_starstararg)) + i = 0 + for w_key in space.unpackiterable(w_starstararg): + try: + key = space.str_w(w_key) + except OperationError, e: + if not e.match(space, space.w_TypeError): + raise raise OperationError(space.w_TypeError, - space.wrap("argument after ** must be " - "a dictionary")) - keywords_w = [None] * space.int_w(space.len(w_starstararg)) - keywords = [None] * space.int_w(space.len(w_starstararg)) - i = 0 - for w_key in space.unpackiterable(w_starstararg): - try: - key = space.str_w(w_key) - except OperationError, e: - if not e.match(space, space.w_TypeError): - raise - raise OperationError(space.w_TypeError, - space.wrap("keywords must be strings")) - if self.keywords and key in self.keywords: - raise operationerrfmt(self.space.w_TypeError, - "got multiple values " - "for keyword argument " - "'%s'", key) - keywords[i] = key - keywords_w[i] = space.getitem(w_starstararg, w_key) - i += 1 - if self.keywords is None: - self.keywords = keywords - self.keywords_w = keywords_w - else: - self.keywords = self.keywords + keywords - self.keywords_w = self.keywords_w + keywords_w + space.wrap("keywords must be strings")) + if self.keywords and key in self.keywords: + raise operationerrfmt(self.space.w_TypeError, + "got multiple values " + "for keyword argument " + "'%s'", key) + keywords[i] = key + keywords_w[i] = space.getitem(w_starstararg, w_key) + i += 1 + if self.keywords is None: + self.keywords = keywords + self.keywords_w = keywords_w + else: + self.keywords = self.keywords + keywords + self.keywords_w = self.keywords_w + keywords_w def fixedunpack(self, argcount): """The simplest argument parsing: get the 'argcount' arguments, @@ -226,6 +233,10 @@ # argnames = list of formal parameter names # scope_w = resulting list of wrapped values # + + # some comments about the JIT: it assumes that signature is a constant, + # so all values coming from there can be assumed constant. It assumes + # that the length of the defaults_w does not vary too much. co_argcount = signature.num_argnames() # expected formal arguments, without */** has_vararg = signature.has_vararg() has_kwarg = signature.has_kwarg() @@ -245,12 +256,6 @@ args_w = self.arguments_w num_args = len(args_w) - keywords = self.keywords - keywords_w = self.keywords_w - num_kwds = 0 - if keywords is not None: - num_kwds = len(keywords) - avail = num_args + upfront if input_argcount < co_argcount: @@ -260,15 +265,24 @@ else: take = num_args + # letting the JIT unroll this loop is safe, because take is always + # smaller than co_argcount for i in range(take): scope_w[i + input_argcount] = args_w[i] input_argcount += take + keywords = self.keywords + keywords_w = self.keywords_w + num_kwds = 0 + if keywords is not None: + num_kwds = len(keywords) # the code assumes that keywords can potentially be large, but that # argnames is typically not too large num_remainingkwds = num_kwds used_keywords = None if keywords: + # letting JIT unroll the loop is *only* safe if the callsite didn't + # use **args because num_kwds can be arbitrarily large otherwise. used_keywords = [False] * num_kwds for i in range(num_kwds): name = keywords[i] @@ -276,7 +290,7 @@ if j < 0: continue elif j < input_argcount: - # check that no keyword argument conflicts with these note + # check that no keyword argument conflicts with these. note # that for this purpose we ignore the first blindargs, # which were put into place by prepend(). This way, # keywords do not conflict with the hidden extra argument Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/jit-unroll-loops/pypy/interpreter/baseobjspace.py Wed Nov 17 20:41:25 2010 @@ -258,10 +258,9 @@ self.interned_strings = {} self.actionflag = ActionFlag() # changed by the signal module + self.check_signal_action = None # changed by the signal module self.user_del_action = UserDelAction(self) self.frame_trace_action = FrameTraceAction(self) - self.actionflag.register_action(self.user_del_action) - self.actionflag.register_action(self.frame_trace_action) from pypy.interpreter.pycode import cpython_magic, default_magic self.our_magic = default_magic @@ -299,8 +298,6 @@ self.timer.start("startup " + modname) mod.init(self) self.timer.stop("startup " + modname) - # Force the tick counter to have a valid value - self.actionflag.force_tick_counter() def finish(self): self.wait_for_thread_shutdown() Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/interpreter/executioncontext.py (original) +++ pypy/branch/jit-unroll-loops/pypy/interpreter/executioncontext.py Wed Nov 17 20:41:25 2010 @@ -5,6 +5,8 @@ from pypy.rlib.unroll import unrolling_iterable from pypy.rlib import jit +TICK_COUNTER_STEP = 100 + def app_profile_call(space, w_callable, frame, event, w_arg): space.call_function(w_callable, space.wrap(frame), @@ -166,24 +168,19 @@ if self.w_tracefunc is not None: self._trace(frame, 'return', w_retval) - def bytecode_trace(self, frame): + def bytecode_trace(self, frame, decr_by=TICK_COUNTER_STEP): "Trace function called before each bytecode." # this is split into a fast path and a slower path that is # not invoked every time bytecode_trace() is. actionflag = self.space.actionflag - ticker = actionflag.get() - if actionflag.has_bytecode_counter: # this "if" is constant-folded - ticker += 1 - actionflag.set(ticker) - if ticker & actionflag.interesting_bits: # fast check + if actionflag.decrement_ticker(decr_by) < 0: actionflag.action_dispatcher(self, frame) # slow path bytecode_trace._always_inline_ = True def bytecode_trace_after_exception(self, frame): "Like bytecode_trace(), but without increasing the ticker." actionflag = self.space.actionflag - ticker = actionflag.get() - if ticker & actionflag.interesting_bits: # fast check + if actionflag.get_ticker() < 0: actionflag.action_dispatcher(self, frame) # slow path bytecode_trace_after_exception._always_inline_ = True @@ -314,6 +311,13 @@ frame.last_exception = last_exception self.is_tracing -= 1 + def checksignals(self): + """Similar to PyErr_CheckSignals(). If called in the main thread, + and if signals are pending for the process, deliver them now + (i.e. call the signal handlers).""" + if self.space.check_signal_action is not None: + self.space.check_signal_action.perform(self, None) + def _freeze_(self): raise Exception("ExecutionContext instances should not be seen during" " translation. Now is a good time to inspect the" @@ -321,118 +325,95 @@ class AbstractActionFlag(object): - """This holds the global 'action flag'. It is a single bitfield - integer, with bits corresponding to AsyncAction objects that need to - be immediately triggered. The correspondance from bits to - AsyncAction instances is built at translation time. We can quickly - check if there is anything at all to do by checking if any of the - relevant bits is set. If threads are enabled, they consume the 20 - lower bits to hold a counter incremented at each bytecode, to know - when to release the GIL. + """This holds in an integer the 'ticker'. If threads are enabled, + it is decremented at each bytecode; when it reaches zero, we release + the GIL. And whether we have threads or not, it is forced to zero + whenever we fire any of the asynchronous actions. """ def __init__(self): self._periodic_actions = [] self._nonperiodic_actions = [] - self.unused_bits = self.FREE_BITS[:] self.has_bytecode_counter = False - self.interesting_bits = 0 + self.fired_actions = None + self.checkinterval_scaled = 100 * TICK_COUNTER_STEP self._rebuild_action_dispatcher() def fire(self, action): - """Request for the action to be run before the next opcode. - The action must have been registered at space initalization time.""" - ticker = self.get() - self.set(ticker | action.bitmask) - - def register_action(self, action): - "NOT_RPYTHON" - assert isinstance(action, AsyncAction) - if action.bitmask == 0: - while True: - action.bitmask = self.unused_bits.pop(0) - if not (action.bitmask & self.interesting_bits): - break - self.interesting_bits |= action.bitmask - if action.bitmask & self.BYTECODE_COUNTER_OVERFLOW_BIT: - assert action.bitmask == self.BYTECODE_COUNTER_OVERFLOW_BIT - self._periodic_actions.append(action) + """Request for the action to be run before the next opcode.""" + if not action._fired: + action._fired = True + if self.fired_actions is None: + self.fired_actions = [] + self.fired_actions.append(action) + # set the ticker to -1 in order to force action_dispatcher() + # to run at the next possible bytecode + self.reset_ticker(-1) + + def register_periodic_action(self, action, use_bytecode_counter): + """NOT_RPYTHON: + Register the PeriodicAsyncAction action to be called whenever the + tick counter becomes smaller than 0. If 'use_bytecode_counter' is + True, make sure that we decrease the tick counter at every bytecode. + This is needed for threads. Note that 'use_bytecode_counter' can be + False for signal handling, because whenever the process receives a + signal, the tick counter is set to -1 by C code in signals.h. + """ + assert isinstance(action, PeriodicAsyncAction) + self._periodic_actions.append(action) + if use_bytecode_counter: self.has_bytecode_counter = True - self.force_tick_counter() - else: - self._nonperiodic_actions.append((action, action.bitmask)) self._rebuild_action_dispatcher() - def setcheckinterval(self, space, interval): - if interval < self.CHECK_INTERVAL_MIN: - interval = self.CHECK_INTERVAL_MIN - elif interval > self.CHECK_INTERVAL_MAX: - interval = self.CHECK_INTERVAL_MAX - space.sys.checkinterval = interval - self.force_tick_counter() - - def force_tick_counter(self): - # force the tick counter to a valid value -- this actually forces - # it to reach BYTECODE_COUNTER_OVERFLOW_BIT at the next opcode. - ticker = self.get() - ticker &= ~ self.BYTECODE_COUNTER_OVERFLOW_BIT - ticker |= self.BYTECODE_COUNTER_MASK - self.set(ticker) + def getcheckinterval(self): + return self.checkinterval_scaled // TICK_COUNTER_STEP + + def setcheckinterval(self, interval): + MAX = sys.maxint // TICK_COUNTER_STEP + if interval < 1: + interval = 1 + elif interval > MAX: + interval = MAX + self.checkinterval_scaled = interval * TICK_COUNTER_STEP def _rebuild_action_dispatcher(self): periodic_actions = unrolling_iterable(self._periodic_actions) - nonperiodic_actions = unrolling_iterable(self._nonperiodic_actions) - has_bytecode_counter = self.has_bytecode_counter @jit.dont_look_inside def action_dispatcher(ec, frame): - # periodic actions - if has_bytecode_counter: - ticker = self.get() - if ticker & self.BYTECODE_COUNTER_OVERFLOW_BIT: - # We must run the periodic actions now, but first - # reset the bytecode counter (the following line - # works by assuming that we just overflowed the - # counter, i.e. BYTECODE_COUNTER_OVERFLOW_BIT is - # set but none of the BYTECODE_COUNTER_MASK bits - # are). - ticker -= ec.space.sys.checkinterval - self.set(ticker) - for action in periodic_actions: - action.perform(ec, frame) + # periodic actions (first reset the bytecode counter) + self.reset_ticker(self.checkinterval_scaled) + for action in periodic_actions: + action.perform(ec, frame) # nonperiodic actions - for action, bitmask in nonperiodic_actions: - ticker = self.get() - if ticker & bitmask: - self.set(ticker & ~ bitmask) + list = self.fired_actions + if list is not None: + self.fired_actions = None + for action in list: + action._fired = False action.perform(ec, frame) action_dispatcher._dont_inline_ = True self.action_dispatcher = action_dispatcher - # Bits reserved for the bytecode counter, if used - BYTECODE_COUNTER_MASK = (1 << 20) - 1 - BYTECODE_COUNTER_OVERFLOW_BIT = (1 << 20) - - # Free bits - FREE_BITS = [1 << _b for _b in range(21, LONG_BIT-1)] - - # The acceptable range of values for sys.checkinterval, so that - # the bytecode_counter fits in 20 bits - CHECK_INTERVAL_MIN = 1 - CHECK_INTERVAL_MAX = BYTECODE_COUNTER_OVERFLOW_BIT - class ActionFlag(AbstractActionFlag): """The normal class for space.actionflag. The signal module provides a different one.""" - _flags = 0 + _ticker = 0 + + def get_ticker(self): + return self._ticker - def get(self): - return self._flags + def reset_ticker(self, value): + self._ticker = value - def set(self, value): - self._flags = value + def decrement_ticker(self, by): + value = self._ticker + if self.has_bytecode_counter: # this 'if' is constant-folded + value -= by + self._ticker = value + return value class AsyncAction(object): @@ -440,7 +421,7 @@ asynchronously with regular bytecode execution, but that still need to occur between two opcodes, not at a completely random time. """ - bitmask = 0 # means 'please choose one bit automatically' + _fired = False def __init__(self, space): self.space = space @@ -453,10 +434,11 @@ def fire_after_thread_switch(self): """Bit of a hack: fire() the action but only the next time the GIL is released and re-acquired (i.e. after a potential thread switch). - Don't call this if threads are not enabled. + Don't call this if threads are not enabled. Currently limited to + one action (i.e. reserved for CheckSignalAction from module/signal). """ from pypy.module.thread.gil import spacestate - spacestate.set_actionflag_bit_after_thread_switch |= self.bitmask + spacestate.action_after_thread_switch = self def perform(self, executioncontext, frame): """To be overridden.""" @@ -466,7 +448,6 @@ """Abstract base class for actions that occur automatically every sys.checkinterval bytecodes. """ - bitmask = ActionFlag.BYTECODE_COUNTER_OVERFLOW_BIT class UserDelAction(AsyncAction): Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/interpreter/pyopcode.py (original) +++ pypy/branch/jit-unroll-loops/pypy/interpreter/pyopcode.py Wed Nov 17 20:41:25 2010 @@ -1140,7 +1140,7 @@ state_pack_variables = staticmethod(state_pack_variables) -class FrameBlock: +class FrameBlock(object): """Abstract base class for frame blocks from the blockstack, used by the SETUP_XXX and POP_BLOCK opcodes.""" Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/test/test_argument.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/interpreter/test/test_argument.py (original) +++ pypy/branch/jit-unroll-loops/pypy/interpreter/test/test_argument.py Wed Nov 17 20:41:25 2010 @@ -52,12 +52,17 @@ assert y == "d" assert z == "e" +class dummy_wrapped_dict(dict): + def __nonzero__(self): + raise NotImplementedError class DummySpace(object): def newtuple(self, items): return tuple(items) def is_true(self, obj): + if isinstance(obj, dummy_wrapped_dict): + return bool(dict(obj)) return bool(obj) def fixedview(self, it): @@ -229,7 +234,7 @@ kwds_w = dict(kwds[:i]) keywords = kwds_w.keys() keywords_w = kwds_w.values() - w_kwds = dict(kwds[i:]) + w_kwds = dummy_wrapped_dict(kwds[i:]) if i == 2: w_kwds = None assert len(keywords) == len(keywords_w) @@ -265,7 +270,7 @@ kwds_w = dict(kwds[:i]) keywords = kwds_w.keys() keywords_w = kwds_w.values() - w_kwds = dict(kwds[i:]) + w_kwds = dummy_wrapped_dict(kwds[i:]) if i == 3: w_kwds = None args = Arguments(space, [1, 2], keywords, keywords_w, w_starstararg=w_kwds) Modified: pypy/branch/jit-unroll-loops/pypy/interpreter/test/test_executioncontext.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/interpreter/test/test_executioncontext.py (original) +++ pypy/branch/jit-unroll-loops/pypy/interpreter/test/test_executioncontext.py Wed Nov 17 20:41:25 2010 @@ -19,7 +19,6 @@ space = self.space a1 = DemoAction(space) - space.actionflag.register_action(a1) for i in range(20): # assert does not raise: space.appexec([], """(): @@ -50,7 +49,7 @@ space = self.space a2 = DemoAction(space) - space.actionflag.register_action(a2) + space.actionflag.register_periodic_action(a2, True) try: for i in range(500): space.appexec([], """(): @@ -59,7 +58,8 @@ """) except Finished: pass - assert space.sys.checkinterval / 10 < i < space.sys.checkinterval * 3 + checkinterval = space.actionflag.getcheckinterval() + assert checkinterval / 10 < i < checkinterval * 1.1 def test_llprofile(self): l = [] Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/detect_cpu.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/detect_cpu.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/detect_cpu.py Wed Nov 17 20:41:25 2010 @@ -34,7 +34,17 @@ 'x86_64': 'x86', }[mach] except KeyError: - raise ProcessorAutodetectError, "unsupported processor '%s'" % mach + return mach + +def autodetect_main_model_and_size(): + model = autodetect_main_model() + if sys.maxint == 2**31-1: + model += '_32' + elif sys.maxint == 2**63-1: + model += '_64' + else: + raise AssertionError, "bad value for sys.maxint" + return model def autodetect(): model = autodetect_main_model() Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py Wed Nov 17 20:41:25 2010 @@ -152,7 +152,7 @@ 'unicodegetitem' : (('ref', 'int'), 'int'), 'unicodesetitem' : (('ref', 'int', 'int'), 'int'), 'cast_ptr_to_int' : (('ref',), 'int'), - 'debug_merge_point': (('ref',), None), + 'debug_merge_point': (('ref', 'int'), None), 'force_token' : ((), 'int'), 'call_may_force' : (('int', 'varargs'), 'intorptr'), 'guard_not_forced': ((), None), @@ -568,7 +568,7 @@ # return _op_default_implementation - def op_debug_merge_point(self, _, value): + def op_debug_merge_point(self, _, value, recdepth): from pypy.jit.metainterp.warmspot import get_stats loc = ConstPtr(value)._get_str() get_stats().add_merge_point_location(loc) Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/descr.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/descr.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/descr.py Wed Nov 17 20:41:25 2010 @@ -130,6 +130,7 @@ # ArrayDescrs _A = lltype.GcArray(lltype.Signed) # a random gcarray +_AF = lltype.GcArray(lltype.Float) # an array of C doubles class BaseArrayDescr(AbstractDescr): @@ -171,16 +172,21 @@ _clsname = 'GcPtrArrayDescr' _is_array_of_pointers = True -_CA = rffi.CArray(lltype.Signed) +class FloatArrayDescr(BaseArrayDescr): + _clsname = 'FloatArrayDescr' + _is_array_of_floats = True + def get_base_size(self, translate_support_code): + basesize, _, _ = symbolic.get_array_token(_AF, translate_support_code) + return basesize + def get_item_size(self, translate_support_code): + return symbolic.get_size(lltype.Float, translate_support_code) class BaseArrayNoLengthDescr(BaseArrayDescr): def get_base_size(self, translate_support_code): - basesize, _, _ = symbolic.get_array_token(_CA, translate_support_code) - return basesize + return 0 def get_ofs_length(self, translate_support_code): - _, _, ofslength = symbolic.get_array_token(_CA, translate_support_code) - return ofslength + return -1 class NonGcPtrArrayNoLengthDescr(BaseArrayNoLengthDescr): _clsname = 'NonGcPtrArrayNoLengthDescr' @@ -192,6 +198,8 @@ _is_array_of_pointers = True def getArrayDescrClass(ARRAY): + if ARRAY.OF is lltype.Float: + return FloatArrayDescr return getDescrClass(ARRAY.OF, BaseArrayDescr, GcPtrArrayDescr, NonGcPtrArrayDescr, 'Array', 'get_item_size', '_is_array_of_floats', '_is_item_signed') @@ -219,7 +227,8 @@ basesize, itemsize, ofslength = symbolic.get_array_token(ARRAY, False) assert basesize == arraydescr.get_base_size(False) assert itemsize == arraydescr.get_item_size(False) - assert ofslength == arraydescr.get_ofs_length(False) + if not ARRAY._hints.get('nolength', False): + assert ofslength == arraydescr.get_ofs_length(False) if isinstance(ARRAY, lltype.GcArray): gccache.init_array_descr(ARRAY, arraydescr) cache[ARRAY] = arraydescr Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/gc.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/gc.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/gc.py Wed Nov 17 20:41:25 2010 @@ -19,6 +19,7 @@ # ____________________________________________________________ class GcLLDescription(GcCache): + minimal_size_in_nursery = 0 def __init__(self, gcdescr, translator=None, rtyper=None): GcCache.__init__(self, translator is not None, rtyper) self.gcdescr = gcdescr @@ -386,6 +387,7 @@ (self.array_basesize, _, self.array_length_ofs) = \ symbolic.get_array_token(lltype.GcArray(lltype.Signed), True) self.max_size_of_young_obj = self.GCClass.JIT_max_size_of_young_obj() + self.minimal_size_in_nursery=self.GCClass.JIT_minimal_size_in_nursery() # make a malloc function, with three arguments def malloc_basic(size, tid): @@ -468,6 +470,7 @@ def malloc_fixedsize_slowpath(size): if self.DEBUG: random_usage_of_xmm_registers() + assert size >= self.minimal_size_in_nursery try: gcref = llop1.do_malloc_fixedsize_clear(llmemory.GCREF, 0, size, True, False, False) @@ -570,6 +573,9 @@ # GETFIELD_RAW from the array 'gcrefs.list'. # newops = [] + # we can only remember one malloc since the next malloc can possibly + # collect + last_malloc = None for op in operations: if op.getopnum() == rop.DEBUG_MERGE_POINT: continue @@ -587,22 +593,32 @@ [ConstInt(addr)], box, self.single_gcref_descr)) op.setarg(i, box) + if op.is_malloc(): + last_malloc = op.result + elif op.can_malloc(): + last_malloc = None # ---------- write barrier for SETFIELD_GC ---------- if op.getopnum() == rop.SETFIELD_GC: - v = op.getarg(1) - if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and - bool(v.value)): # store a non-NULL - self._gen_write_barrier(newops, op.getarg(0), v) - op = op.copy_and_change(rop.SETFIELD_RAW) + val = op.getarg(0) + # no need for a write barrier in the case of previous malloc + if val is not last_malloc: + v = op.getarg(1) + if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and + bool(v.value)): # store a non-NULL + self._gen_write_barrier(newops, op.getarg(0), v) + op = op.copy_and_change(rop.SETFIELD_RAW) # ---------- write barrier for SETARRAYITEM_GC ---------- if op.getopnum() == rop.SETARRAYITEM_GC: - v = op.getarg(2) - if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and - bool(v.value)): # store a non-NULL - # XXX detect when we should produce a - # write_barrier_from_array - self._gen_write_barrier(newops, op.getarg(0), v) - op = op.copy_and_change(rop.SETARRAYITEM_RAW) + val = op.getarg(0) + # no need for a write barrier in the case of previous malloc + if val is not last_malloc: + v = op.getarg(2) + if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and + bool(v.value)): # store a non-NULL + # XXX detect when we should produce a + # write_barrier_from_array + self._gen_write_barrier(newops, op.getarg(0), v) + op = op.copy_and_change(rop.SETARRAYITEM_RAW) # ---------- newops.append(op) del operations[:] Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_descr.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_descr.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_descr.py Wed Nov 17 20:41:25 2010 @@ -5,6 +5,7 @@ from pypy.rpython.annlowlevel import llhelper from pypy.jit.metainterp.history import BoxInt, BoxFloat, BoxPtr from pypy.jit.metainterp import history +import struct def test_get_size_descr(): c0 = GcCache(False) @@ -130,11 +131,13 @@ assert not descr3.is_array_of_floats() assert descr4.is_array_of_floats() # - WORD = rffi.sizeof(lltype.Signed) - assert descr1.get_base_size(False) == WORD - assert descr2.get_base_size(False) == WORD - assert descr3.get_base_size(False) == WORD - assert descr4.get_base_size(False) == WORD + def get_alignment(code): + # Retrieve default alignment for the compiler/platform + return struct.calcsize('l' + code) - struct.calcsize(code) + assert descr1.get_base_size(False) == get_alignment('c') + assert descr2.get_base_size(False) == get_alignment('p') + assert descr3.get_base_size(False) == get_alignment('p') + assert descr4.get_base_size(False) == get_alignment('d') assert descr1.get_ofs_length(False) == 0 assert descr2.get_ofs_length(False) == 0 assert descr3.get_ofs_length(False) == 0 Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_gc.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_gc.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llsupport/test/test_gc.py Wed Nov 17 20:41:25 2010 @@ -6,7 +6,9 @@ from pypy.jit.backend.llsupport.gc import * from pypy.jit.backend.llsupport import symbolic from pypy.jit.metainterp.gc import get_description - +from pypy.jit.tool.oparser import parse +from pypy.rpython.lltypesystem.rclass import OBJECT, OBJECT_VTABLE +from pypy.jit.metainterp.test.test_optimizeopt import equaloplists def test_boehm(): gc_ll_descr = GcLLDescr_boehm(None, None, None) @@ -114,7 +116,7 @@ assert gcrootmap._gcmap[i*2+1] == expected_shapeaddr[i] -class FakeLLOp: +class FakeLLOp(object): def __init__(self): self.record = [] @@ -148,19 +150,19 @@ return llhelper(FPTRTYPE, self._write_barrier_failing_case) -class TestFramework: +class TestFramework(object): gc = 'hybrid' def setup_method(self, meth): - class config_: - class translation: + class config_(object): + class translation(object): gc = self.gc gcrootfinder = 'asmgcc' gctransformer = 'framework' gcremovetypeptr = False - class FakeTranslator: + class FakeTranslator(object): config = config_ - class FakeCPU: + class FakeCPU(object): def cast_adr_to_int(self, adr): ptr = llmemory.cast_adr_to_ptr(adr, gc_ll_descr.WB_FUNCPTR) assert ptr._obj._callable == llop1._write_barrier_failing_case @@ -270,7 +272,7 @@ def test_get_rid_of_debug_merge_point(self): operations = [ - ResOperation(rop.DEBUG_MERGE_POINT, ['dummy'], None), + ResOperation(rop.DEBUG_MERGE_POINT, ['dummy', 2], None), ] gc_ll_descr = self.gc_ll_descr gc_ll_descr.rewrite_assembler(None, operations) @@ -278,11 +280,11 @@ def test_rewrite_assembler_1(self): # check rewriting of ConstPtrs - class MyFakeCPU: + class MyFakeCPU(object): def cast_adr_to_int(self, adr): assert adr == "some fake address" return 43 - class MyFakeGCRefList: + class MyFakeGCRefList(object): def get_address_of_gcref(self, s_gcref1): assert s_gcref1 == s_gcref return "some fake address" @@ -311,10 +313,10 @@ def test_rewrite_assembler_1_cannot_move(self): # check rewriting of ConstPtrs - class MyFakeCPU: + class MyFakeCPU(object): def cast_adr_to_int(self, adr): xxx # should not be called - class MyFakeGCRefList: + class MyFakeGCRefList(object): def get_address_of_gcref(self, s_gcref1): seen.append(s_gcref1) assert s_gcref1 == s_gcref @@ -394,6 +396,68 @@ assert operations[1].getarg(2) == v_value assert operations[1].getdescr() == array_descr + def test_rewrite_assembler_initialization_store(self): + S = lltype.GcStruct('S', ('parent', OBJECT), + ('x', lltype.Signed)) + s_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True) + xdescr = get_field_descr(self.gc_ll_descr, S, 'x') + ops = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + # no write barrier + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) + + def test_rewrite_assembler_initialization_store_2(self): + S = lltype.GcStruct('S', ('parent', OBJECT), + ('x', lltype.Signed)) + s_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True) + wbdescr = self.gc_ll_descr.write_barrier_descr + xdescr = get_field_descr(self.gc_ll_descr, S, 'x') + ops = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + p3 = new_with_vtable(ConstClass(s_vtable)) + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + p3 = new_with_vtable(ConstClass(s_vtable)) + cond_call_gc_wb(p0, p1, descr=wbdescr) + setfield_raw(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) + + def test_rewrite_assembler_initialization_store_3(self): + A = lltype.GcArray(lltype.Ptr(lltype.GcStruct('S'))) + arraydescr = get_array_descr(self.gc_ll_descr, A) + ops = parse(""" + [p1] + p0 = new_array(3, descr=arraydescr) + setarrayitem_gc(p0, 0, p1, descr=arraydescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_array(3, descr=arraydescr) + setarrayitem_gc(p0, 0, p1, descr=arraydescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) class TestFrameworkMiniMark(TestFramework): gc = 'minimark' Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/test/runner_test.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/test/runner_test.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/test/runner_test.py Wed Nov 17 20:41:25 2010 @@ -2168,12 +2168,14 @@ # Tested with a function that intentionally does not cast the # result to RESTYPE, but makes sure that we return the whole # value in eax or rax. - eci = ExternalCompilationInfo(separate_module_sources=[""" + eci = ExternalCompilationInfo( + separate_module_sources=[""" long fn_test_result_of_call(long x) { return x + 1; } - """]) + """], + export_symbols=['fn_test_result_of_call']) f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed], RESTYPE, compilation_info=eci, _nowrapper=True) value = intmask(0xFFEEDDCCBBAA9988) @@ -2199,12 +2201,14 @@ # Tested with a function that intentionally does not cast the # result to RESTYPE, but makes sure that we return the whole # value in eax or rax. - eci = ExternalCompilationInfo(separate_module_sources=[""" + eci = ExternalCompilationInfo( + separate_module_sources=[""" long fn_test_result_of_call(long x) { return x + 1; } - """]) + """], + export_symbols=['fn_test_result_of_call']) f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed], RESTYPE, compilation_info=eci, _nowrapper=True) value = intmask(0xFFEEDDCCBBAA9988) Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/assembler.py Wed Nov 17 20:41:25 2010 @@ -303,6 +303,7 @@ _x86_frame_depth _x86_param_depth _x86_arglocs + _x86_debug_checksum """ if not we_are_translated(): # Arguments should be unique @@ -312,7 +313,7 @@ funcname = self._find_debug_merge_point(operations) if log: self._register_counter() - operations = self._inject_debugging_code(operations) + operations = self._inject_debugging_code(looptoken, operations) regalloc = RegAlloc(self, self.cpu.translate_support_code) arglocs = regalloc.prepare_loop(inputargs, operations, looptoken) @@ -336,8 +337,9 @@ self._assemble_bootstrap_direct_call(arglocs, curadr, frame_depth+param_depth) # - debug_print("Loop #", looptoken.number, "has address", - looptoken._x86_loop_code, "to", self.mc.tell()) + debug_print("Loop #%d has address %x to %x" % (looptoken.number, + looptoken._x86_loop_code, + self.mc.tell())) self.mc.end_function() self.write_pending_failure_recoveries() @@ -350,7 +352,7 @@ funcname = self._find_debug_merge_point(operations) if log: self._register_counter() - operations = self._inject_debugging_code(operations) + operations = self._inject_debugging_code(faildescr, operations) arglocs = self.rebuild_faillocs_from_descr( faildescr._x86_failure_recovery_bytecode) @@ -358,7 +360,6 @@ assert ([loc.assembler() for loc in arglocs] == [loc.assembler() for loc in faildescr._x86_debug_faillocs]) regalloc = RegAlloc(self, self.cpu.translate_support_code) - operations = self._inject_debugging_code(operations) fail_depths = faildescr._x86_current_depths regalloc.prepare_bridge(fail_depths, inputargs, arglocs, operations) @@ -378,9 +379,8 @@ faildescr._x86_bridge_param_depth = param_depth # patch the jump from original guard self.patch_jump_for_descr(faildescr, adr_bridge) - debug_print("Bridge out of guard", - descr_number, - "has address", adr_bridge, "to", self.mc.tell()) + debug_print("Bridge out of guard %d has address %x to %x" % + (descr_number, adr_bridge, self.mc.tell())) self.mc.end_function() self.write_pending_failure_recoveries() @@ -433,9 +433,14 @@ mc.done() - def _inject_debugging_code(self, operations): + @specialize.argtype(1) + def _inject_debugging_code(self, looptoken, operations): if self._debug: # before doing anything, let's increase a counter + s = 0 + for op in operations: + s += op.getopnum() + looptoken._x86_debug_checksum = s c_adr = ConstInt(rffi.cast(lltype.Signed, self.loop_run_counters[-1][1])) box = BoxInt() @@ -1768,13 +1773,18 @@ self.implement_guard(guard_token, 'L') def genop_discard_cond_call_gc_wb(self, op, arglocs): - # use 'mc._mc' directly instead of 'mc', to avoid - # bad surprizes if the code buffer is mostly full + # Write code equivalent to write_barrier() in the GC: it checks + # a flag in the object at arglocs[0], and if set, it calls the + # function remember_young_pointer() from the GC. The two arguments + # to the call are in arglocs[:2]. The rest, arglocs[2:], contains + # registers that need to be saved and restored across the call. descr = op.getdescr() if we_are_translated(): cls = self.cpu.gc_ll_descr.has_write_barrier_class() assert cls is not None and isinstance(descr, cls) loc_base = arglocs[0] + # ensure that enough bytes are available to write the whole + # following piece of code atomically (for the JZ) self.mc.ensure_bytes_available(256) self.mc.TEST8_mi((loc_base.value, descr.jit_wb_if_flag_byteofs), descr.jit_wb_if_flag_singlebyte) @@ -1782,36 +1792,39 @@ jz_location = self.mc.get_relative_pos() # the following is supposed to be the slow path, so whenever possible # we choose the most compact encoding over the most efficient one. - # XXX improve a bit, particularly for IS_X86_64. - for i in range(len(arglocs)-1, -1, -1): + if IS_X86_32: + limit = -1 # push all arglocs on the stack + elif IS_X86_64: + limit = 1 # push only arglocs[2:] on the stack + for i in range(len(arglocs)-1, limit, -1): loc = arglocs[i] if isinstance(loc, RegLoc): self.mc.PUSH_r(loc.value) else: - if IS_X86_64: - self.mc.MOV_ri(X86_64_SCRATCH_REG.value, loc.getint()) - self.mc.PUSH_r(X86_64_SCRATCH_REG.value) - else: - self.mc.PUSH_i32(loc.getint()) - + assert not IS_X86_64 # there should only be regs in arglocs[2:] + self.mc.PUSH_i32(loc.getint()) if IS_X86_64: # We clobber these registers to pass the arguments, but that's # okay, because consider_cond_call_gc_wb makes sure that any - # caller-save registers with values in them are present in arglocs, - # so they are saved on the stack above and restored below - self.mc.MOV_rs(edi.value, 0) - self.mc.MOV_rs(esi.value, 8) + # caller-save registers with values in them are present in + # arglocs[2:] too, so they are saved on the stack above and + # restored below. + remap_frame_layout(self, arglocs[:2], [edi, esi], + X86_64_SCRATCH_REG) # misaligned stack in the call, but it's ok because the write barrier # is not going to call anything more. Also, this assumes that the - # write barrier does not touch the xmm registers. + # write barrier does not touch the xmm registers. (Slightly delicate + # assumption, given that the write barrier can end up calling the + # platform's malloc() from AddressStack.append(). XXX may need to + # be done properly) self.mc.CALL(imm(descr.get_write_barrier_fn(self.cpu))) - for i in range(len(arglocs)): + if IS_X86_32: + self.mc.ADD_ri(esp.value, 2*WORD) + for i in range(2, len(arglocs)): loc = arglocs[i] - if isinstance(loc, RegLoc): - self.mc.POP_r(loc.value) - else: - self.mc.ADD_ri(esp.value, WORD) # ignore the pushed constant + assert isinstance(loc, RegLoc) + self.mc.POP_r(loc.value) # patch the JZ above offset = self.mc.get_relative_pos() - jz_location assert 0 < offset <= 127 @@ -1848,6 +1861,7 @@ def malloc_cond_fixedsize(self, nursery_free_adr, nursery_top_adr, size, tid): + size = max(size, self.cpu.gc_ll_descr.minimal_size_in_nursery) self.mc.ensure_bytes_available(256) self.mc.MOV(eax, heap(nursery_free_adr)) self.mc.LEA_rm(edx.value, (eax.value, size)) Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_runner.py Wed Nov 17 20:41:25 2010 @@ -346,7 +346,7 @@ return self.val operations = [ - ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("hello")], None), + ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("hello"), 0], None), ResOperation(rop.INT_ADD, [i0, ConstInt(1)], i1), ResOperation(rop.INT_LE, [i1, ConstInt(9)], i2), ResOperation(rop.GUARD_TRUE, [i2], None, descr=faildescr1), @@ -365,7 +365,7 @@ bridge = [ ResOperation(rop.INT_LE, [i1b, ConstInt(19)], i3), ResOperation(rop.GUARD_TRUE, [i3], None, descr=faildescr2), - ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("bye")], None), + ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("bye"), 0], None), ResOperation(rop.JUMP, [i1b], None, descr=looptoken), ] bridge[1].setfailargs([i1b]) @@ -497,7 +497,7 @@ def test_debugger_on(self): loop = """ [i0] - debug_merge_point('xyz') + debug_merge_point('xyz', 0) i1 = int_add(i0, 1) i2 = int_ge(i1, 10) guard_false(i2) [] @@ -515,3 +515,20 @@ self.cpu.finish_once() lines = py.path.local(self.logfile + ".count").readlines() assert lines[0] == '0:10\n' # '10 xyz\n' + + def test_debugger_checksum(self): + loop = """ + [i0] + debug_merge_point('xyz', 0) + i1 = int_add(i0, 1) + i2 = int_ge(i1, 10) + guard_false(i2) [] + jump(i1) + """ + ops = parse(loop) + self.cpu.assembler.set_debug(True) + self.cpu.compile_loop(ops.inputargs, ops.operations, ops.token) + self.cpu.set_future_value_int(0, 0) + self.cpu.execute_token(ops.token) + assert ops.token._x86_debug_checksum == sum([op.getopnum() + for op in ops.operations]) Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_zrpy_gc.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_zrpy_gc.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_zrpy_gc.py Wed Nov 17 20:41:25 2010 @@ -550,3 +550,29 @@ def test_compile_framework_float(self): self.run('compile_framework_float') + + def define_compile_framework_minimal_size_in_nursery(self): + S = lltype.GcStruct('S') # no fields! + T = lltype.GcStruct('T', ('i', lltype.Signed)) + @unroll_safe + def f42(n, x, x0, x1, x2, x3, x4, x5, x6, x7, l, s): + lst1 = [] + lst2 = [] + i = 0 + while i < 42: + s1 = lltype.malloc(S) + t1 = lltype.malloc(T) + t1.i = 10000 + i + n + lst1.append(s1) + lst2.append(t1) + i += 1 + i = 0 + while i < 42: + check(lst2[i].i == 10000 + i + n) + i += 1 + n -= 1 + return n, x, x0, x1, x2, x3, x4, x5, x6, x7, l, s + return None, f42, None + + def test_compile_framework_minimal_size_in_nursery(self): + self.run('compile_framework_minimal_size_in_nursery') Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_ztranslation.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_ztranslation.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/test/test_ztranslation.py Wed Nov 17 20:41:25 2010 @@ -2,6 +2,7 @@ from pypy.tool.udir import udir from pypy.rlib.jit import JitDriver, OPTIMIZER_FULL, unroll_parameters from pypy.rlib.jit import PARAMETERS, dont_look_inside +from pypy.rlib.jit import hint from pypy.jit.metainterp.jitprof import Profiler from pypy.jit.backend.detect_cpu import getcpuclass from pypy.jit.backend.test.support import CCompiledMixin @@ -9,6 +10,7 @@ from pypy.translator.translator import TranslationContext from pypy.jit.backend.x86.arch import IS_X86_32, IS_X86_64 from pypy.config.translationoption import DEFL_GC +from pypy.rlib import rgc class TestTranslationX86(CCompiledMixin): CPUClass = getcpuclass() @@ -82,12 +84,12 @@ argchain.arg(x) res = func.call(argchain, rffi.DOUBLE) i -= 1 - return res + return int(res) # def main(i, j): return f(i, j) + libffi_stuff(i, j) - expected = f(40, -49) - res = self.meta_interp(f, [40, -49]) + expected = main(40, -49) + res = self.meta_interp(main, [40, -49]) assert res == expected def test_direct_assembler_call_translates(self): Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/tool/viewcode.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/tool/viewcode.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/x86/tool/viewcode.py Wed Nov 17 20:41:25 2010 @@ -37,7 +37,7 @@ 'x86_64': 'x86-64', 'i386': 'i386', } - objdump = ('objdump -M intel,%(backend)s -b binary -m i386 ' + objdump = ('objdump -M %(backend)s -b binary -m i386 ' '--adjust-vma=%(origin)d -D %(file)s') # f = open(tmpfile, 'wb') Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/assembler.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/assembler.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/assembler.py Wed Nov 17 20:41:25 2010 @@ -233,10 +233,9 @@ addr = llmemory.cast_ptr_to_adr(value) self.list_of_addr2name.append((addr, name)) - def finished(self): + def finished(self, callinfocollection): # Helper called at the end of assembling. Registers the extra # functions shown in _callinfo_for_oopspec. - from pypy.jit.codewriter.effectinfo import _callinfo_for_oopspec - for _, func in _callinfo_for_oopspec.values(): + for func in callinfocollection.all_function_addresses_as_int(): func = heaptracker.int2adr(func) self.see_raw_object(func.ptr) Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/call.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/call.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/call.py Wed Nov 17 20:41:25 2010 @@ -7,7 +7,7 @@ from pypy.jit.codewriter.jitcode import JitCode from pypy.jit.codewriter.effectinfo import VirtualizableAnalyzer from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze -from pypy.jit.codewriter.effectinfo import EffectInfo +from pypy.jit.codewriter.effectinfo import EffectInfo, CallInfoCollection from pypy.translator.simplify import get_funcobj, get_functype from pypy.rpython.lltypesystem import lltype, llmemory from pypy.translator.backendopt.canraise import RaiseAnalyzer @@ -23,6 +23,7 @@ self.jitdrivers_sd = jitdrivers_sd self.jitcodes = {} # map {graph: jitcode} self.unfinished_graphs = [] # list of graphs with pending jitcodes + self.callinfocollection = CallInfoCollection() if hasattr(cpu, 'rtyper'): # for tests self.rtyper = cpu.rtyper translator = self.rtyper.annotator.translator Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/codewriter.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/codewriter.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/codewriter.py Wed Nov 17 20:41:25 2010 @@ -73,7 +73,7 @@ count += 1 if not count % 500: log.info("Produced %d jitcodes" % count) - self.assembler.finished() + self.assembler.finished(self.callcontrol.callinfocollection) heaptracker.finish_registering(self.cpu) log.info("there are %d JitCode instances." % count) Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/effectinfo.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/effectinfo.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/effectinfo.py Wed Nov 17 20:41:25 2010 @@ -144,30 +144,44 @@ # ____________________________________________________________ -_callinfo_for_oopspec = {} # {oopspecindex: (calldescr, func_as_int)} - -def callinfo_for_oopspec(oopspecindex): - """A function that returns the calldescr and the function - address (as an int) of one of the OS_XYZ functions defined above. - Don't use this if there might be several implementations of the same - OS_XYZ specialized by type, e.g. OS_ARRAYCOPY.""" - try: - return _callinfo_for_oopspec[oopspecindex] - except KeyError: - return (None, 0) - - -def _funcptr_for_oopspec_memo(oopspecindex): - from pypy.jit.codewriter import heaptracker - _, func_as_int = callinfo_for_oopspec(oopspecindex) - funcadr = heaptracker.int2adr(func_as_int) - return funcadr.ptr -_funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo' - -def funcptr_for_oopspec(oopspecindex): - """A memo function that returns a pointer to the function described - by OS_XYZ (as a real low-level function pointer).""" - funcptr = _funcptr_for_oopspec_memo(oopspecindex) - assert funcptr - return funcptr -funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(0)' +class CallInfoCollection(object): + def __init__(self): + # {oopspecindex: (calldescr, func_as_int)} + self._callinfo_for_oopspec = {} + + def _freeze_(self): + return True + + def add(self, oopspecindex, calldescr, func_as_int): + self._callinfo_for_oopspec[oopspecindex] = calldescr, func_as_int + + def has_oopspec(self, oopspecindex): + return oopspecindex in self._callinfo_for_oopspec + + def all_function_addresses_as_int(self): + return [func for (_, func) in self._callinfo_for_oopspec.values()] + + def callinfo_for_oopspec(self, oopspecindex): + """A function that returns the calldescr and the function + address (as an int) of one of the OS_XYZ functions defined above. + Don't use this if there might be several implementations of the same + OS_XYZ specialized by type, e.g. OS_ARRAYCOPY.""" + try: + return self._callinfo_for_oopspec[oopspecindex] + except KeyError: + return (None, 0) + + def _funcptr_for_oopspec_memo(self, oopspecindex): + from pypy.jit.codewriter import heaptracker + _, func_as_int = self.callinfo_for_oopspec(oopspecindex) + funcadr = heaptracker.int2adr(func_as_int) + return funcadr.ptr + _funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo' + + def funcptr_for_oopspec(self, oopspecindex): + """A memo function that returns a pointer to the function described + by OS_XYZ (as a real low-level function pointer).""" + funcptr = self._funcptr_for_oopspec_memo(oopspecindex) + assert funcptr + return funcptr + funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(1)' Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/jtransform.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/jtransform.py Wed Nov 17 20:41:25 2010 @@ -6,7 +6,7 @@ from pypy.objspace.flow.model import Block, Link, c_last_exception from pypy.jit.codewriter.flatten import ListOfKind, IndirectCallTargets from pypy.jit.codewriter import support, heaptracker -from pypy.jit.codewriter.effectinfo import EffectInfo, _callinfo_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.jit.codewriter.policy import log from pypy.jit.metainterp.typesystem import deref, arrayItem from pypy.rlib import objectmodel @@ -873,6 +873,8 @@ elif oopspec_name == 'jit.assert_green': kind = getkind(args[0].concretetype) return SpaceOperation('%s_assert_green' % kind, args, None) + elif oopspec_name == 'jit.current_trace_length': + return SpaceOperation('current_trace_length', [], op.result) else: raise AssertionError("missing support for %r" % oopspec_name) @@ -1082,7 +1084,8 @@ else: func = heaptracker.adr2int( llmemory.cast_ptr_to_adr(op.args[0].value)) - _callinfo_for_oopspec[oopspecindex] = calldescr, func + self.callcontrol.callinfocollection.add(oopspecindex, + calldescr, func) op1 = self.rewrite_call(op, 'residual_call', [op.args[0], calldescr], args=args) @@ -1093,7 +1096,7 @@ def _register_extra_helper(self, oopspecindex, oopspec_name, argtypes, resulttype): # a bit hackish - if oopspecindex in _callinfo_for_oopspec: + if self.callcontrol.callinfocollection.has_oopspec(oopspecindex): return c_func, TP = support.builtin_func_for_spec(self.cpu.rtyper, oopspec_name, argtypes, @@ -1107,7 +1110,7 @@ else: func = heaptracker.adr2int( llmemory.cast_ptr_to_adr(c_func.value)) - _callinfo_for_oopspec[oopspecindex] = calldescr, func + self.callcontrol.callinfocollection.add(oopspecindex, calldescr, func) def _handle_stroruni_call(self, op, oopspec_name, args): SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE) Modified: pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_jtransform.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_jtransform.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/codewriter/test/test_jtransform.py Wed Nov 17 20:41:25 2010 @@ -74,7 +74,20 @@ def calldescr_canraise(self, calldescr): return False +class FakeCallInfoCollection: + def __init__(self): + self.seen = [] + def add(self, oopspecindex, calldescr, func): + self.seen.append((oopspecindex, calldescr, func)) + def has_oopspec(self, oopspecindex): + for i, c, f in self.seen: + if i == oopspecindex: + return True + return False + class FakeBuiltinCallControl: + def __init__(self): + self.callinfocollection = FakeCallInfoCollection() def guess_call_kind(self, op): return 'builtin' def getcalldescr(self, op, oopspecindex=None): @@ -810,7 +823,8 @@ v2 = varoftype(PSTR) v3 = varoftype(PSTR) op = SpaceOperation('direct_call', [const(func), v1, v2], v3) - tr = Transformer(FakeCPU(), FakeBuiltinCallControl()) + cc = FakeBuiltinCallControl() + tr = Transformer(FakeCPU(), cc) op1 = tr.rewrite_operation(op) assert op1.opname == 'residual_call_r_r' assert op1.args[0].value == func @@ -819,9 +833,10 @@ assert op1.result == v3 # # check the callinfo_for_oopspec - got = effectinfo.callinfo_for_oopspec(effectinfo.EffectInfo.OS_UNI_CONCAT) - assert got[0] == op1.args[1] # the calldescr - assert heaptracker.int2adr(got[1]) == llmemory.cast_ptr_to_adr(func) + got = cc.callinfocollection.seen[0] + assert got[0] == effectinfo.EffectInfo.OS_UNI_CONCAT + assert got[1] == op1.args[1] # the calldescr + assert heaptracker.int2adr(got[2]) == llmemory.cast_ptr_to_adr(func) def test_str_slice(): # test that the oopspec is present and correctly transformed @@ -893,7 +908,8 @@ v2 = varoftype(PUNICODE) v3 = varoftype(lltype.Bool) op = SpaceOperation('direct_call', [const(func), v1, v2], v3) - tr = Transformer(FakeCPU(), FakeBuiltinCallControl()) + cc = FakeBuiltinCallControl() + tr = Transformer(FakeCPU(), cc) op1 = tr.rewrite_operation(op) assert op1.opname == 'residual_call_r_i' assert op1.args[0].value == func @@ -901,9 +917,9 @@ assert op1.args[2] == ListOfKind('ref', [v1, v2]) assert op1.result == v3 # test that the OS_UNIEQ_* functions are registered - cifo = effectinfo._callinfo_for_oopspec - assert effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL in cifo - assert effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR in cifo + cic = cc.callinfocollection + assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL) + assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR) def test_list_ll_arraycopy(): from pypy.rlib.rgc import ll_arraycopy Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/blackhole.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/blackhole.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/blackhole.py Wed Nov 17 20:41:25 2010 @@ -774,6 +774,10 @@ def bhimpl_float_assert_green(x): pass + @arguments(returns="i") + def bhimpl_current_trace_length(): + return -1 + # ---------- # the main hints and recursive calls Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py Wed Nov 17 20:41:25 2010 @@ -1,4 +1,5 @@ +from pypy.rpython.lltypesystem import lltype from pypy.rpython.ootypesystem import ootype from pypy.objspace.flow.model import Constant, Variable from pypy.rlib.objectmodel import we_are_translated @@ -13,6 +14,7 @@ from pypy.jit.metainterp import history from pypy.jit.metainterp.typesystem import llhelper, oohelper from pypy.jit.metainterp.optimizeutil import InvalidLoop +from pypy.jit.metainterp.resume import NUMBERING from pypy.jit.codewriter import heaptracker def giveup(): @@ -52,7 +54,6 @@ """ history = metainterp.history loop = create_empty_loop(metainterp) - loop.greenkey = greenkey loop.inputargs = history.inputargs for box in loop.inputargs: assert isinstance(box, Box) @@ -66,7 +67,6 @@ loop.operations[-1].setdescr(loop_token) # patch the target of the JUMP loop.preamble = create_empty_loop(metainterp, 'Preamble ') - loop.preamble.greenkey = greenkey loop.preamble.inputargs = loop.inputargs loop.preamble.token = make_loop_token(len(loop.inputargs), jitdriver_sd) @@ -214,8 +214,7 @@ } class ResumeDescr(AbstractFailDescr): - def __init__(self, original_greenkey): - self.original_greenkey = original_greenkey + pass class ResumeGuardDescr(ResumeDescr): _counter = 0 # if < 0, there is one counter per value; @@ -224,7 +223,7 @@ # this class also gets the following attributes stored by resume.py code rd_snapshot = None rd_frame_info_list = None - rd_numb = None + rd_numb = lltype.nullptr(NUMBERING) rd_consts = None rd_virtuals = None rd_pendingfields = None @@ -234,8 +233,7 @@ CNT_FLOAT = -0x60000000 CNT_MASK = 0x1FFFFFFF - def __init__(self, metainterp_sd, original_greenkey): - ResumeDescr.__init__(self, original_greenkey) + def __init__(self, metainterp_sd): self.metainterp_sd = metainterp_sd def store_final_boxes(self, guard_op, boxes): @@ -339,14 +337,14 @@ res.rd_pendingfields = self.rd_pendingfields def _clone_if_mutable(self): - res = ResumeGuardDescr(self.metainterp_sd, self.original_greenkey) + res = ResumeGuardDescr(self.metainterp_sd) self.copy_all_attrbutes_into(res) return res class ResumeGuardForcedDescr(ResumeGuardDescr): - def __init__(self, metainterp_sd, original_greenkey, jitdriver_sd): - ResumeGuardDescr.__init__(self, metainterp_sd, original_greenkey) + def __init__(self, metainterp_sd, jitdriver_sd): + ResumeGuardDescr.__init__(self, metainterp_sd) self.jitdriver_sd = jitdriver_sd def handle_fail(self, metainterp_sd, jitdriver_sd): @@ -413,7 +411,6 @@ def _clone_if_mutable(self): res = ResumeGuardForcedDescr(self.metainterp_sd, - self.original_greenkey, self.jitdriver_sd) self.copy_all_attrbutes_into(res) return res @@ -480,9 +477,8 @@ class ResumeFromInterpDescr(ResumeDescr): - def __init__(self, original_greenkey, redkey): - ResumeDescr.__init__(self, original_greenkey) - self.redkey = redkey + def __init__(self, original_greenkey): + self.original_greenkey = original_greenkey def compile_and_attach(self, metainterp, new_loop): # We managed to create a bridge going from the interpreter @@ -491,10 +487,8 @@ # with completely unoptimized arguments, as in the interpreter. metainterp_sd = metainterp.staticdata jitdriver_sd = metainterp.jitdriver_sd - metainterp.history.inputargs = self.redkey - new_loop_token = make_loop_token(len(self.redkey), jitdriver_sd) - new_loop.greenkey = self.original_greenkey - new_loop.inputargs = self.redkey + redargs = new_loop.inputargs + new_loop_token = make_loop_token(len(redargs), jitdriver_sd) new_loop.token = new_loop_token send_loop_to_backend(metainterp_sd, new_loop, "entry bridge") # send the new_loop to warmspot.py, to be called directly the next time Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/logger.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/logger.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/logger.py Wed Nov 17 20:41:25 2010 @@ -81,7 +81,8 @@ op = operations[i] if op.getopnum() == rop.DEBUG_MERGE_POINT: loc = op.getarg(0)._get_str() - debug_print("debug_merge_point('%s')" % (loc,)) + reclev = op.getarg(1).getint() + debug_print("debug_merge_point('%s', %s)" % (loc, reclev)) continue args = ", ".join([self.repr_of_arg(memo, op.getarg(i)) for i in range(op.numargs())]) if op.result is not None: Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Wed Nov 17 20:41:25 2010 @@ -86,10 +86,10 @@ assert isinstance(constbox, Const) self.box = constbox self.level = LEVEL_CONSTANT - try: - val = self.box.getint() + if isinstance(constbox, ConstInt): + val = constbox.getint() self.intbound = IntBound(val, val) - except NotImplementedError: + else: self.intbound = IntUnbounded() def get_constant_class(self, cpu): Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/string.py Wed Nov 17 20:41:25 2010 @@ -9,7 +9,7 @@ from pypy.jit.metainterp.optimizeopt.optimizer import CONST_0, CONST_1 from pypy.jit.metainterp.optimizeopt.optimizer import llhelper from pypy.jit.metainterp.optimizeutil import _findall -from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.jit.codewriter import heaptracker from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.objectmodel import specialize, we_are_translated @@ -642,7 +642,8 @@ def generate_modified_call(self, oopspecindex, args, result, mode): oopspecindex += mode.OS_offset - calldescr, func = callinfo_for_oopspec(oopspecindex) + cic = self.optimizer.metainterp_sd.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(oopspecindex) op = ResOperation(rop.CALL, [ConstInt(func)] + args, result, descr=calldescr) self.optimizer.newoperations.append(op) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py Wed Nov 17 20:41:25 2010 @@ -14,7 +14,7 @@ from pypy.jit.metainterp.logger import Logger from pypy.jit.metainterp.jitprof import EmptyProfiler from pypy.jit.metainterp.jitprof import GUARDS, RECORDED_OPS, ABORT_ESCAPE -from pypy.jit.metainterp.jitprof import ABORT_TOO_LONG, ABORT_BRIDGE +from pypy.jit.metainterp.jitprof import ABORT_TOO_LONG from pypy.jit.metainterp.jitexc import JitException, get_llexception from pypy.rlib.rarithmetic import intmask from pypy.rlib.objectmodel import specialize @@ -820,7 +820,8 @@ jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex] self.verify_green_args(jitdriver_sd, greenboxes) # xxx we may disable the following line in some context later - self.debug_merge_point(jitdriver_sd, greenboxes) + self.debug_merge_point(jitdriver_sd, self.metainterp.in_recursion, + greenboxes) if self.metainterp.seen_loop_header_for_jdindex < 0: if not jitdriver_sd.no_loop_header or not any_operation: return @@ -860,13 +861,13 @@ assembler_call=True) raise ChangeFrame - def debug_merge_point(self, jitdriver_sd, greenkey): + def debug_merge_point(self, jitdriver_sd, in_recursion, greenkey): # debugging: produce a DEBUG_MERGE_POINT operation loc = jitdriver_sd.warmstate.get_location_str(greenkey) debug_print(loc) constloc = self.metainterp.cpu.ts.conststr(loc) self.metainterp.history.record(rop.DEBUG_MERGE_POINT, - [constloc], None) + [constloc, ConstInt(in_recursion)], None) @arguments("box", "label") def opimpl_goto_if_exception_mismatch(self, vtablebox, next_exc_target): @@ -948,6 +949,11 @@ opimpl_ref_assert_green = _opimpl_assert_green opimpl_float_assert_green = _opimpl_assert_green + @arguments() + def opimpl_current_trace_length(self): + trace_length = len(self.metainterp.history.operations) + return ConstInt(trace_length) + @arguments("box") def opimpl_virtual_ref(self, box): # Details on the content of metainterp.virtualref_boxes: @@ -1042,14 +1048,11 @@ else: moreargs = list(extraargs) metainterp_sd = metainterp.staticdata - original_greenkey = metainterp.resumekey.original_greenkey if opnum == rop.GUARD_NOT_FORCED: resumedescr = compile.ResumeGuardForcedDescr(metainterp_sd, - original_greenkey, metainterp.jitdriver_sd) else: - resumedescr = compile.ResumeGuardDescr(metainterp_sd, - original_greenkey) + resumedescr = compile.ResumeGuardDescr(metainterp_sd) guard_op = metainterp.history.record(opnum, moreargs, None, descr=resumedescr) virtualizable_boxes = None @@ -1255,6 +1258,7 @@ # self.jitdrivers_sd = codewriter.callcontrol.jitdrivers_sd self.virtualref_info = codewriter.callcontrol.virtualref_info + self.callinfocollection = codewriter.callcontrol.callinfocollection self.setup_jitdrivers_sd(optimizer) # # store this information for fastpath of call_assembler @@ -1618,7 +1622,7 @@ assert jitdriver_sd is self.jitdriver_sd self.create_empty_history() try: - original_boxes = self.initialize_original_boxes(jitdriver_sd,*args) + original_boxes = self.initialize_original_boxes(jitdriver_sd, *args) return self._compile_and_run_once(original_boxes) finally: self.staticdata.profiler.end_tracing() @@ -1629,9 +1633,8 @@ self.current_merge_points = [(original_boxes, 0)] num_green_args = self.jitdriver_sd.num_green_args original_greenkey = original_boxes[:num_green_args] - redkey = original_boxes[num_green_args:] - self.resumekey = compile.ResumeFromInterpDescr(original_greenkey, - redkey) + self.resumekey = compile.ResumeFromInterpDescr(original_greenkey) + self.history.inputargs = original_boxes[num_green_args:] self.seen_loop_header_for_jdindex = -1 try: self.interpret() @@ -1653,11 +1656,7 @@ debug_stop('jit-tracing') def _handle_guard_failure(self, key): - original_greenkey = key.original_greenkey - # notice that here we just put the greenkey - # use -1 to mark that we will have to give up - # because we cannot reconstruct the beginning of the proper loop - self.current_merge_points = [(original_greenkey, -1)] + self.current_merge_points = [] self.resumekey = key self.seen_loop_header_for_jdindex = -1 try: @@ -1721,7 +1720,7 @@ num_green_args = self.jitdriver_sd.num_green_args for j in range(len(self.current_merge_points)-1, -1, -1): original_boxes, start = self.current_merge_points[j] - assert len(original_boxes) == len(live_arg_boxes) or start < 0 + assert len(original_boxes) == len(live_arg_boxes) for i in range(num_green_args): box1 = original_boxes[i] box2 = live_arg_boxes[i] @@ -1730,10 +1729,6 @@ break else: # Found! Compile it as a loop. - if start < 0: - # we cannot reconstruct the beginning of the proper loop - raise SwitchToBlackhole(ABORT_BRIDGE) - # raises in case it works -- which is the common case self.compile(original_boxes, live_arg_boxes, start) # creation of the loop was cancelled! Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resoperation.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resoperation.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resoperation.py Wed Nov 17 20:41:25 2010 @@ -143,6 +143,16 @@ def can_raise(self): return rop._CANRAISE_FIRST <= self.getopnum() <= rop._CANRAISE_LAST + def is_malloc(self): + # a slightly different meaning from can_malloc + return rop._MALLOC_FIRST <= self.getopnum() <= rop._MALLOC_LAST + + def can_malloc(self): + return self.is_call() or self.is_malloc() + + def is_call(self): + return rop._CALL_FIRST <= self.getopnum() <= rop._CALL_LAST + def is_ovf(self): return rop._OVF_FIRST <= self.getopnum() <= rop._OVF_LAST @@ -441,9 +451,13 @@ 'GETARRAYITEM_RAW/2d', 'GETFIELD_GC/1d', 'GETFIELD_RAW/1d', + '_MALLOC_FIRST', 'NEW/0d', 'NEW_WITH_VTABLE/1', 'NEW_ARRAY/1d', + 'NEWSTR/1', + 'NEWUNICODE/1', + '_MALLOC_LAST', 'FORCE_TOKEN/0', 'VIRTUAL_REF/2', # removed before it's passed to the backend '_NOSIDEEFFECT_LAST', # ----- end of no_side_effect operations ----- @@ -452,19 +466,18 @@ 'SETARRAYITEM_RAW/3d', 'SETFIELD_GC/2d', 'SETFIELD_RAW/2d', - 'NEWSTR/1', 'STRSETITEM/3', 'UNICODESETITEM/3', - 'NEWUNICODE/1', #'RUNTIMENEW/1', # ootype operation 'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier) - 'DEBUG_MERGE_POINT/1', # debugging only + 'DEBUG_MERGE_POINT/2', # debugging only 'JIT_DEBUG/*', # debugging only 'VIRTUAL_REF_FINISH/2', # removed before it's passed to the backend 'COPYSTRCONTENT/5', # src, dst, srcstart, dststart, length 'COPYUNICODECONTENT/5', '_CANRAISE_FIRST', # ----- start of can_raise operations ----- + '_CALL_FIRST', 'CALL/*d', 'CALL_ASSEMBLER/*d', # call already compiled assembler 'CALL_MAY_FORCE/*d', @@ -473,6 +486,7 @@ #'OOSEND_PURE', # ootype operation 'CALL_PURE/*d', # removed before it's passed to the backend # CALL_PURE(result, func, arg_1,..,arg_n) + '_CALL_LAST', '_CANRAISE_LAST', # ----- end of can_raise operations ----- '_OVF_FIRST', # ----- start of is_ovf operations ----- Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resume.py Wed Nov 17 20:41:25 2010 @@ -4,8 +4,7 @@ from pypy.jit.metainterp.history import INT, REF, FLOAT, HOLE from pypy.jit.metainterp.resoperation import rop from pypy.jit.metainterp import jitprof -from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec -from pypy.jit.codewriter.effectinfo import funcptr_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.rpython.lltypesystem import lltype, llmemory, rffi, rstr from pypy.rlib import rarithmetic from pypy.rlib.objectmodel import we_are_translated, specialize @@ -66,12 +65,21 @@ snapshot = Snapshot(snapshot, boxes) storage.rd_snapshot = snapshot -class Numbering(object): - __slots__ = ('prev', 'nums') - - def __init__(self, prev, nums): - self.prev = prev - self.nums = nums +# +# The following is equivalent to the RPython-level declaration: +# +# class Numbering: __slots__ = ['prev', 'nums'] +# +# except that it is more compact in translated programs, because the +# array 'nums' is inlined in the single NUMBERING object. This is +# important because this is often the biggest single consumer of memory +# in a pypy-c-jit. +# +NUMBERINGP = lltype.Ptr(lltype.GcForwardReference()) +NUMBERING = lltype.GcStruct('Numbering', + ('prev', NUMBERINGP), + ('nums', lltype.Array(rffi.SHORT))) +NUMBERINGP.TO.become(NUMBERING) TAGMASK = 3 @@ -163,7 +171,7 @@ def number(self, values, snapshot): if snapshot is None: - return None, {}, 0 + return lltype.nullptr(NUMBERING), {}, 0 if snapshot in self.numberings: numb, liveboxes, v = self.numberings[snapshot] return numb, liveboxes.copy(), v @@ -172,7 +180,7 @@ n = len(liveboxes)-v boxes = snapshot.boxes length = len(boxes) - nums = [UNASSIGNED] * length + numb = lltype.malloc(NUMBERING, length) for i in range(length): box = boxes[i] value = values.get(box, None) @@ -191,9 +199,9 @@ tagged = tag(n, TAGBOX) n += 1 liveboxes[box] = tagged - nums[i] = tagged + numb.nums[i] = tagged # - numb = Numbering(numb1, nums) + numb.prev = numb1 self.numberings[snapshot] = numb, liveboxes, v return numb, liveboxes.copy(), v @@ -298,7 +306,7 @@ # compute the numbering storage = self.storage # make sure that nobody attached resume data to this guard yet - assert storage.rd_numb is None + assert not storage.rd_numb snapshot = storage.rd_snapshot assert snapshot is not None # is that true? numb, liveboxes_from_env, v = self.memo.number(values, snapshot) @@ -724,34 +732,36 @@ self.boxes_f = boxes_f self._prepare_next_section(info) - def consume_virtualizable_boxes(self, vinfo, nums): + def consume_virtualizable_boxes(self, vinfo, numb): # we have to ignore the initial part of 'nums' (containing vrefs), # find the virtualizable from nums[-1], and use it to know how many # boxes of which type we have to return. This does not write # anything into the virtualizable. - virtualizablebox = self.decode_ref(nums[-1]) + index = len(numb.nums) - 1 + virtualizablebox = self.decode_ref(numb.nums[index]) virtualizable = vinfo.unwrap_virtualizable_box(virtualizablebox) - return vinfo.load_list_of_boxes(virtualizable, self, nums) + return vinfo.load_list_of_boxes(virtualizable, self, numb) - def consume_virtualref_boxes(self, nums, end): + def consume_virtualref_boxes(self, numb, end): # Returns a list of boxes, assumed to be all BoxPtrs. # We leave up to the caller to call vrefinfo.continue_tracing(). assert (end & 1) == 0 - return [self.decode_ref(nums[i]) for i in range(end)] + return [self.decode_ref(numb.nums[i]) for i in range(end)] def consume_vref_and_vable_boxes(self, vinfo, ginfo): - nums = self.cur_numb.nums - self.cur_numb = self.cur_numb.prev + numb = self.cur_numb + self.cur_numb = numb.prev if vinfo is not None: - virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, nums) - end = len(nums) - len(virtualizable_boxes) + virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, numb) + end = len(numb.nums) - len(virtualizable_boxes) elif ginfo is not None: - virtualizable_boxes = [self.decode_ref(nums[-1])] - end = len(nums) - 1 + index = len(numb.nums) - 1 + virtualizable_boxes = [self.decode_ref(numb.nums[index])] + end = len(numb.nums) - 1 else: virtualizable_boxes = None - end = len(nums) - virtualref_boxes = self.consume_virtualref_boxes(nums, end) + end = len(numb.nums) + virtualref_boxes = self.consume_virtualref_boxes(numb, end) return virtualizable_boxes, virtualref_boxes def allocate_with_vtable(self, known_class): @@ -775,14 +785,16 @@ strbox, ConstInt(index), charbox) def concat_strings(self, str1num, str2num): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT) str1box = self.decode_box(str1num, REF) str2box = self.decode_box(str2num, REF) return self.metainterp.execute_and_record_varargs( rop.CALL, [ConstInt(func), str1box, str2box], calldescr) def slice_string(self, strnum, startnum, lengthnum): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_SLICE) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_SLICE) strbox = self.decode_box(strnum, REF) startbox = self.decode_box(startnum, INT) lengthbox = self.decode_box(lengthnum, INT) @@ -801,14 +813,16 @@ strbox, ConstInt(index), charbox) def concat_unicodes(self, str1num, str2num): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT) str1box = self.decode_box(str1num, REF) str2box = self.decode_box(str2num, REF) return self.metainterp.execute_and_record_varargs( rop.CALL, [ConstInt(func), str1box, str2box], calldescr) def slice_unicode(self, strnum, startnum, lengthnum): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE) strbox = self.decode_box(strnum, REF) startbox = self.decode_box(startnum, INT) lengthbox = self.decode_box(lengthnum, INT) @@ -904,8 +918,8 @@ def blackhole_from_resumedata(blackholeinterpbuilder, jitdriver_sd, storage, all_virtuals=None): - resumereader = ResumeDataDirectReader(blackholeinterpbuilder.cpu, storage, - all_virtuals) + resumereader = ResumeDataDirectReader(blackholeinterpbuilder.metainterp_sd, + storage, all_virtuals) vinfo = jitdriver_sd.virtualizable_info ginfo = jitdriver_sd.greenfield_info vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info @@ -940,7 +954,7 @@ return firstbh def force_from_resumedata(metainterp_sd, storage, vinfo, ginfo): - resumereader = ResumeDataDirectReader(metainterp_sd.cpu, storage) + resumereader = ResumeDataDirectReader(metainterp_sd, storage) resumereader.handling_async_forcing() vrefinfo = metainterp_sd.virtualref_info resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo) @@ -954,8 +968,9 @@ # 1: in handle_async_forcing # 2: resuming from the GUARD_NOT_FORCED - def __init__(self, cpu, storage, all_virtuals=None): - self._init(cpu, storage) + def __init__(self, metainterp_sd, storage, all_virtuals=None): + self._init(metainterp_sd.cpu, storage) + self.callinfocollection = metainterp_sd.callinfocollection if all_virtuals is None: # common case self._prepare(storage) else: @@ -974,23 +989,24 @@ info = blackholeinterp.get_current_position_info() self._prepare_next_section(info) - def consume_virtualref_info(self, vrefinfo, nums, end): + def consume_virtualref_info(self, vrefinfo, numb, end): # we have to decode a list of references containing pairs # [..., virtual, vref, ...] stopping at 'end' assert (end & 1) == 0 for i in range(0, end, 2): - virtual = self.decode_ref(nums[i]) - vref = self.decode_ref(nums[i+1]) + virtual = self.decode_ref(numb.nums[i]) + vref = self.decode_ref(numb.nums[i+1]) # For each pair, we store the virtual inside the vref. vrefinfo.continue_tracing(vref, virtual) - def consume_vable_info(self, vinfo, nums): + def consume_vable_info(self, vinfo, numb): # we have to ignore the initial part of 'nums' (containing vrefs), # find the virtualizable from nums[-1], load all other values # from the CPU stack, and copy them into the virtualizable if vinfo is None: - return len(nums) - virtualizable = self.decode_ref(nums[-1]) + return len(numb.nums) + index = len(numb.nums) - 1 + virtualizable = self.decode_ref(numb.nums[index]) virtualizable = vinfo.cast_gcref_to_vtype(virtualizable) if self.resume_after_guard_not_forced == 1: # in the middle of handle_async_forcing() @@ -1002,7 +1018,7 @@ # is and stays 0. Note the call to reset_vable_token() in # warmstate.py. assert not virtualizable.vable_token - return vinfo.write_from_resume_data_partial(virtualizable, self, nums) + return vinfo.write_from_resume_data_partial(virtualizable, self, numb) def load_value_of_type(self, TYPE, tagged): from pypy.jit.metainterp.warmstate import specialize_value @@ -1019,12 +1035,12 @@ load_value_of_type._annspecialcase_ = 'specialize:arg(1)' def consume_vref_and_vable(self, vrefinfo, vinfo, ginfo): - nums = self.cur_numb.nums - self.cur_numb = self.cur_numb.prev + numb = self.cur_numb + self.cur_numb = numb.prev if self.resume_after_guard_not_forced != 2: - end_vref = self.consume_vable_info(vinfo, nums) + end_vref = self.consume_vable_info(vinfo, numb) if ginfo is not None: end_vref -= 1 - self.consume_virtualref_info(vrefinfo, nums, end_vref) + self.consume_virtualref_info(vrefinfo, numb, end_vref) def allocate_with_vtable(self, known_class): from pypy.jit.metainterp.executor import exec_new_with_vtable @@ -1048,7 +1064,8 @@ str2 = self.decode_ref(str2num) str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str1) str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str2) - funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT) result = funcptr(str1, str2) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1057,7 +1074,8 @@ start = self.decode_int(startnum) length = self.decode_int(lengthnum) str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str) - funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_SLICE) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_SLICE) result = funcptr(str, start, start + length) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1073,7 +1091,8 @@ str2 = self.decode_ref(str2num) str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str1) str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str2) - funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT) result = funcptr(str1, str2) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1082,7 +1101,8 @@ start = self.decode_int(startnum) length = self.decode_int(lengthnum) str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str) - funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE) result = funcptr(str, start, start + length) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1173,8 +1193,9 @@ 'at', compute_unique_id(frameinfo)) frameinfo = frameinfo.prev numb = storage.rd_numb - while numb is not None: - debug_print('\tnumb', str([untag(i) for i in numb.nums]), + while numb: + debug_print('\tnumb', str([untag(numb.nums[i]) + for i in range(len(numb.nums))]), 'at', compute_unique_id(numb)) numb = numb.prev for const in storage.rd_consts: Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py Wed Nov 17 20:41:25 2010 @@ -3,6 +3,7 @@ from pypy.rlib.jit import JitDriver, we_are_jitted, hint, dont_look_inside from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE, loop_invariant from pypy.rlib.jit import jit_debug, assert_green, AssertGreenFailed +from pypy.rlib.jit import unroll_safe, current_trace_length from pypy.jit.metainterp.warmspot import ll_meta_interp, get_stats from pypy.jit.backend.llgraph import runner from pypy.jit.metainterp import pyjitpl, history @@ -1837,6 +1838,31 @@ 'int_add': 1, 'int_mul': 1, 'int_sub': 2, 'int_gt': 2, 'jump': 2}) + def test_current_trace_length(self): + myjitdriver = JitDriver(greens = ['g'], reds = ['x']) + @dont_look_inside + def residual(): + print "hi there" + @unroll_safe + def loop(g): + y = 0 + while y < g: + residual() + y += 1 + def f(x, g): + n = 0 + while x > 0: + myjitdriver.can_enter_jit(x=x, g=g) + myjitdriver.jit_merge_point(x=x, g=g) + loop(g) + x -= 1 + n = current_trace_length() + return n + res = self.meta_interp(f, [5, 8]) + assert 14 < res < 42 + res = self.meta_interp(f, [5, 2]) + assert 4 < res < 14 + class TestOOtype(BasicTests, OOJitMixin): Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py Wed Nov 17 20:41:25 2010 @@ -85,7 +85,7 @@ metainterp.history.inputargs = loop.inputargs[:] # loop_tokens = [] - loop_token = compile_new_loop(metainterp, loop_tokens, [], 0) + loop_token = compile_new_loop(metainterp, loop_tokens, 0) assert loop_tokens == [loop_token] assert loop_token.number == 1 assert staticdata.globaldata.loopnumbering == 2 @@ -101,7 +101,7 @@ metainterp.history.operations = loop.operations[:] metainterp.history.inputargs = loop.inputargs[:] # - loop_token_2 = compile_new_loop(metainterp, loop_tokens, [], 0) + loop_token_2 = compile_new_loop(metainterp, loop_tokens, 0) assert loop_token_2 is loop_token assert loop_tokens == [loop_token] assert len(cpu.seen) == 0 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_del.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_del.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_del.py Wed Nov 17 20:41:25 2010 @@ -85,6 +85,7 @@ def test_signal_action(self): from pypy.module.signal.interp_signal import SignalActionFlag action = SignalActionFlag() + action.has_bytecode_counter = True # myjitdriver = JitDriver(greens = [], reds = ['n', 'x']) class X: @@ -92,17 +93,17 @@ # def f(n): x = X() - while n > 0: + action.reset_ticker(n) + while True: myjitdriver.can_enter_jit(n=n, x=x) myjitdriver.jit_merge_point(n=n, x=x) x.foo = n n -= 1 - if action.get() != 0: + if action.decrement_ticker(1) < 0: break - action.set(0) return 42 self.meta_interp(f, [20]) - self.check_loops(getfield_raw=1, call=0, call_pure=0) + self.check_loops(getfield_raw=1, setfield_raw=1, call=0, call_pure=0) class TestOOtype(DelTests, OOJitMixin): def setup_class(cls): Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_logger.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_logger.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_logger.py Wed Nov 17 20:41:25 2010 @@ -97,7 +97,7 @@ def test_debug_merge_point(self): inp = ''' [] - debug_merge_point("info") + debug_merge_point("info", 0) ''' loop, oloop = self.reparse(inp, check_equal=False) assert loop.operations[0].getarg(0)._get_str() == 'info' Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Wed Nov 17 20:41:25 2010 @@ -26,6 +26,100 @@ self.options = Fake() self.globaldata = Fake() +def test_store_final_boxes_in_guard(): + from pypy.jit.metainterp.compile import ResumeGuardDescr + from pypy.jit.metainterp.resume import tag, TAGBOX + b0 = BoxInt() + b1 = BoxInt() + opt = optimizeopt.Optimizer(FakeMetaInterpStaticData(LLtypeMixin.cpu), + None) + fdescr = ResumeGuardDescr(None) + op = ResOperation(rop.GUARD_TRUE, ['dummy'], None, descr=fdescr) + # setup rd data + fi0 = resume.FrameInfo(None, "code0", 11) + fdescr.rd_frame_info_list = resume.FrameInfo(fi0, "code1", 33) + snapshot0 = resume.Snapshot(None, [b0]) + fdescr.rd_snapshot = resume.Snapshot(snapshot0, [b1]) + # + opt.store_final_boxes_in_guard(op) + if op.getfailargs() == [b0, b1]: + assert list(fdescr.rd_numb.nums) == [tag(1, TAGBOX)] + assert list(fdescr.rd_numb.prev.nums) == [tag(0, TAGBOX)] + else: + assert op.getfailargs() == [b1, b0] + assert list(fdescr.rd_numb.nums) == [tag(0, TAGBOX)] + assert list(fdescr.rd_numb.prev.nums) == [tag(1, TAGBOX)] + assert fdescr.rd_virtuals is None + assert fdescr.rd_consts == [] + +def test_sharing_field_lists_of_virtual(): + class FakeOptimizer(object): + class cpu(object): + pass + opt = FakeOptimizer() + virt1 = virtualize.AbstractVirtualStructValue(opt, None) + lst1 = virt1._get_field_descr_list() + assert lst1 == [] + lst2 = virt1._get_field_descr_list() + assert lst1 is lst2 + virt1.setfield(LLtypeMixin.valuedescr, optimizeopt.OptValue(None)) + lst3 = virt1._get_field_descr_list() + assert lst3 == [LLtypeMixin.valuedescr] + lst4 = virt1._get_field_descr_list() + assert lst3 is lst4 + + virt2 = virtualize.AbstractVirtualStructValue(opt, None) + lst5 = virt2._get_field_descr_list() + assert lst5 is lst1 + virt2.setfield(LLtypeMixin.valuedescr, optimizeopt.OptValue(None)) + lst6 = virt1._get_field_descr_list() + assert lst6 is lst3 + +def test_reuse_vinfo(): + class FakeVInfo(object): + def set_content(self, fieldnums): + self.fieldnums = fieldnums + def equals(self, fieldnums): + return self.fieldnums == fieldnums + class FakeVirtualValue(virtualize.AbstractVirtualValue): + def _make_virtual(self, *args): + return FakeVInfo() + v1 = FakeVirtualValue(None, None, None) + vinfo1 = v1.make_virtual_info(None, [1, 2, 4]) + vinfo2 = v1.make_virtual_info(None, [1, 2, 4]) + assert vinfo1 is vinfo2 + vinfo3 = v1.make_virtual_info(None, [1, 2, 6]) + assert vinfo3 is not vinfo2 + vinfo4 = v1.make_virtual_info(None, [1, 2, 6]) + assert vinfo3 is vinfo4 + +def test_descrlist_dict(): + from pypy.jit.metainterp import optimizeutil + h1 = optimizeutil.descrlist_hash([]) + h2 = optimizeutil.descrlist_hash([LLtypeMixin.valuedescr]) + h3 = optimizeutil.descrlist_hash( + [LLtypeMixin.valuedescr, LLtypeMixin.nextdescr]) + assert h1 != h2 + assert h2 != h3 + assert optimizeutil.descrlist_eq([], []) + assert not optimizeutil.descrlist_eq([], [LLtypeMixin.valuedescr]) + assert optimizeutil.descrlist_eq([LLtypeMixin.valuedescr], + [LLtypeMixin.valuedescr]) + assert not optimizeutil.descrlist_eq([LLtypeMixin.valuedescr], + [LLtypeMixin.nextdescr]) + assert optimizeutil.descrlist_eq([LLtypeMixin.valuedescr, LLtypeMixin.nextdescr], + [LLtypeMixin.valuedescr, LLtypeMixin.nextdescr]) + assert not optimizeutil.descrlist_eq([LLtypeMixin.nextdescr, LLtypeMixin.valuedescr], + [LLtypeMixin.valuedescr, LLtypeMixin.nextdescr]) + + # descrlist_eq should compare by identity of the descrs, not by the result + # of sort_key + class FakeDescr(object): + def sort_key(self): + return 1 + + assert not optimizeutil.descrlist_eq([FakeDescr()], [FakeDescr()]) + # ____________________________________________________________ class Storage(compile.ResumeGuardDescr): "for tests." @@ -76,6 +170,8 @@ metainterp_sd = FakeMetaInterpStaticData(self.cpu) if hasattr(self, 'vrefinfo'): metainterp_sd.virtualref_info = self.vrefinfo + if hasattr(self, 'callinfocollection'): + metainterp_sd.callinfocollection = self.callinfocollection optimize_loop_1(metainterp_sd, loop) # @@ -1430,7 +1526,7 @@ ops = """ [p1] i1 = getfield_gc(p1, descr=valuedescr) - debug_merge_point(15) + debug_merge_point(15, 0) i2 = getfield_gc(p1, descr=valuedescr) escape(i1) escape(i2) @@ -1439,7 +1535,7 @@ expected = """ [p1] i1 = getfield_gc(p1, descr=valuedescr) - debug_merge_point(15) + debug_merge_point(15, 0) escape(i1) escape(i1) jump(p1) @@ -4288,22 +4384,20 @@ # ---------- def optimize_strunicode_loop_extradescrs(self, ops, optops, preamble=None): from pypy.jit.metainterp.optimizeopt import string - def my_callinfo_for_oopspec(oopspecindex): - calldescrtype = type(LLtypeMixin.strequaldescr) - for value in LLtypeMixin.__dict__.values(): - if isinstance(value, calldescrtype): - if (value.get_extra_info() and - value.get_extra_info().oopspecindex == oopspecindex): - # returns 0 for 'func' in this test - return value, 0 - raise AssertionError("not found: oopspecindex=%d" % oopspecindex) + class FakeCallInfoCollection: + def callinfo_for_oopspec(self, oopspecindex): + calldescrtype = type(LLtypeMixin.strequaldescr) + for value in LLtypeMixin.__dict__.values(): + if isinstance(value, calldescrtype): + extra = value.get_extra_info() + if extra and extra.oopspecindex == oopspecindex: + # returns 0 for 'func' in this test + return value, 0 + raise AssertionError("not found: oopspecindex=%d" % + oopspecindex) # - saved = string.callinfo_for_oopspec - try: - string.callinfo_for_oopspec = my_callinfo_for_oopspec - self.optimize_strunicode_loop(ops, optops, preamble) - finally: - string.callinfo_for_oopspec = saved + self.callinfocollection = FakeCallInfoCollection() + self.optimize_strunicode_loop(ops, optops, preamble) def test_str_equal_noop1(self): ops = """ Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resoperation.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resoperation.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resoperation.py Wed Nov 17 20:41:25 2010 @@ -61,3 +61,10 @@ assert op.getarglist() == ['a', 'b'] assert op.result == 'c' assert op.getdescr() is mydescr + +def test_can_malloc(): + mydescr = AbstractDescr() + assert rop.ResOperation(rop.rop.NEW, [], 'b').can_malloc() + call = rop.ResOperation(rop.rop.CALL, ['a', 'b'], 'c', descr=mydescr) + assert call.can_malloc() + assert not rop.ResOperation(rop.rop.INT_ADD, ['a', 'b'], 'c').can_malloc() Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_resume.py Wed Nov 17 20:41:25 2010 @@ -52,6 +52,7 @@ class MyMetaInterp: _already_allocated_resume_virtuals = None + callinfocollection = None def __init__(self, cpu=None): if cpu is None: @@ -142,6 +143,13 @@ assert bh.written_f == expected_f +def Numbering(prev, nums): + numb = lltype.malloc(NUMBERING, len(nums)) + numb.prev = prev or lltype.nullptr(NUMBERING) + for i in range(len(nums)): + numb.nums[i] = nums[i] + return numb + def test_simple_read(): #b1, b2, b3 = [BoxInt(), BoxPtr(), BoxInt()] c1, c2, c3 = [ConstInt(111), ConstInt(222), ConstInt(333)] @@ -157,12 +165,12 @@ storage.rd_numb = numb # cpu = MyCPU([42, gcref1, -66]) - reader = ResumeDataDirectReader(cpu, storage) + metainterp = MyMetaInterp(cpu) + reader = ResumeDataDirectReader(metainterp, storage) _next_section(reader, 42, 111, gcrefnull, 42, gcref1) _next_section(reader, 222, 333) _next_section(reader, 42, gcref1, -66) # - metainterp = MyMetaInterp(cpu) reader = ResumeDataBoxReader(storage, metainterp) bi, br, bf = [None]*3, [None]*2, [None]*0 info = MyBlackholeInterp([lltype.Signed, lltype.Signed, @@ -194,7 +202,7 @@ storage.rd_numb = numb # cpu = MyCPU([]) - reader = ResumeDataDirectReader(cpu, storage) + reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage) _next_section(reader, 100) @@ -212,7 +220,7 @@ class FakeMetainterp(object): _already_allocated_resume_virtuals = None cpu = None - reader = ResumeDataDirectReader(None, FakeStorage()) + reader = ResumeDataDirectReader(MyMetaInterp(None), FakeStorage()) assert reader.force_all_virtuals() == ["allocated", reader.virtual_default] # ____________________________________________________________ @@ -391,15 +399,15 @@ assert fi1.pc == 3 def test_Numbering_create(): - l = [1, 2] + l = [rffi.r_short(1), rffi.r_short(2)] numb = Numbering(None, l) - assert numb.prev is None - assert numb.nums is l + assert not numb.prev + assert list(numb.nums) == l - l1 = ['b3'] + l1 = [rffi.r_short(3)] numb1 = Numbering(numb, l1) - assert numb1.prev is numb - assert numb1.nums is l1 + assert numb1.prev == numb + assert list(numb1.nums) == l1 def test_capture_resumedata(): b1, b2, b3 = [BoxInt(), BoxPtr(), BoxInt()] @@ -765,11 +773,12 @@ assert liveboxes == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b3: tag(2, TAGBOX)} - assert numb.nums == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), - tag(1, TAGINT)] - assert numb.prev.nums == [tag(0, TAGBOX), tag(1, TAGINT), tag(1, TAGBOX), - tag(0, TAGBOX), tag(2, TAGINT)] - assert numb.prev.prev is None + assert list(numb.nums) == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), + tag(1, TAGINT)] + assert list(numb.prev.nums) == [tag(0, TAGBOX), tag(1, TAGINT), + tag(1, TAGBOX), + tag(0, TAGBOX), tag(2, TAGINT)] + assert not numb.prev.prev numb2, liveboxes2, v = memo.number({}, snap2) assert v == 0 @@ -777,9 +786,9 @@ assert liveboxes2 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b3: tag(2, TAGBOX)} assert liveboxes2 is not liveboxes - assert numb2.nums == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb2.prev is numb.prev + assert list(numb2.nums) == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), + tag(3, TAGINT)] + assert numb2.prev == numb.prev env3 = [c3, b3, b1, c3] snap3 = Snapshot(snap, env3) @@ -800,9 +809,9 @@ assert v == 0 assert liveboxes3 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX)} - assert numb3.nums == [tag(3, TAGINT), tag(4, TAGINT), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb3.prev is numb.prev + assert list(numb3.nums) == [tag(3, TAGINT), tag(4, TAGINT), tag(0, TAGBOX), + tag(3, TAGINT)] + assert numb3.prev == numb.prev # virtual env4 = [c3, b4, b1, c3] @@ -813,9 +822,9 @@ assert liveboxes4 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b4: tag(0, TAGVIRTUAL)} - assert numb4.nums == [tag(3, TAGINT), tag(0, TAGVIRTUAL), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb4.prev is numb.prev + assert list(numb4.nums) == [tag(3, TAGINT), tag(0, TAGVIRTUAL), + tag(0, TAGBOX), tag(3, TAGINT)] + assert numb4.prev == numb.prev env5 = [b1, b4, b5] snap5 = Snapshot(snap4, env5) @@ -826,9 +835,9 @@ assert liveboxes5 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b4: tag(0, TAGVIRTUAL), b5: tag(1, TAGVIRTUAL)} - assert numb5.nums == [tag(0, TAGBOX), tag(0, TAGVIRTUAL), - tag(1, TAGVIRTUAL)] - assert numb5.prev is numb4 + assert list(numb5.nums) == [tag(0, TAGBOX), tag(0, TAGVIRTUAL), + tag(1, TAGVIRTUAL)] + assert numb5.prev == numb4 def test_ResumeDataLoopMemo_number_boxes(): memo = ResumeDataLoopMemo(FakeMetaInterpStaticData()) @@ -926,7 +935,7 @@ liveboxes = modifier.finish({}) assert storage.rd_snapshot is None cpu = MyCPU([]) - reader = ResumeDataDirectReader(cpu, storage) + reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage) _next_section(reader, sys.maxint, 2**16, -65) _next_section(reader, 2, 3) _next_section(reader, sys.maxint, 1, sys.maxint, 2**16) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualref.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualref.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtualref.py Wed Nov 17 20:41:25 2010 @@ -88,7 +88,11 @@ cpu.get_latest_value_int = lambda i:guard_op.getfailargs()[i].getint() cpu.get_latest_value_ref = lambda i:guard_op.getfailargs()[i].getref_base() cpu.clear_latest_values = lambda count: None - resumereader = ResumeDataDirectReader(cpu, guard_op.getdescr()) + class FakeMetaInterpSd: + callinfocollection = None + FakeMetaInterpSd.cpu = cpu + resumereader = ResumeDataDirectReader(FakeMetaInterpSd(), + guard_op.getdescr()) vrefinfo = self.metainterp.staticdata.virtualref_info lst = [] vrefinfo.continue_tracing = lambda vref, virtual: \ Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_ztranslation.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_ztranslation.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_ztranslation.py Wed Nov 17 20:41:25 2010 @@ -79,7 +79,7 @@ res = ll_meta_interp(main, [40, 5], CPUClass=self.CPUClass, type_system=self.type_system) assert res == main(40, 5) - res = rpython_ll_meta_interp(main, [40, 5], loops=2, + res = rpython_ll_meta_interp(main, [40, 5], CPUClass=self.CPUClass, type_system=self.type_system, optimizer=OPTIMIZER_FULL, @@ -120,7 +120,7 @@ res = ll_meta_interp(main, [40], CPUClass=self.CPUClass, type_system=self.type_system) assert res == main(40) - res = rpython_ll_meta_interp(main, [40], loops=2, CPUClass=self.CPUClass, + res = rpython_ll_meta_interp(main, [40], CPUClass=self.CPUClass, type_system=self.type_system, optimizer=OPTIMIZER_FULL, ProfilerClass=Profiler) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/virtualizable.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/virtualizable.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/virtualizable.py Wed Nov 17 20:41:25 2010 @@ -100,48 +100,48 @@ i = i + 1 assert len(boxes) == i + 1 # - def write_from_resume_data_partial(virtualizable, reader, nums): + def write_from_resume_data_partial(virtualizable, reader, numb): # Load values from the reader (see resume.py) described by # the list of numbers 'nums', and write them in their proper # place in the 'virtualizable'. This works from the end of # the list and returns the index in 'nums' of the start of # the virtualizable data found, allowing the caller to do # further processing with the start of the list. - i = len(nums) - 1 + i = len(numb.nums) - 1 assert i >= 0 for ARRAYITEMTYPE, fieldname in unroll_array_fields_rev: lst = getattr(virtualizable, fieldname) for j in range(getlength(lst)-1, -1, -1): i -= 1 assert i >= 0 - x = reader.load_value_of_type(ARRAYITEMTYPE, nums[i]) + x = reader.load_value_of_type(ARRAYITEMTYPE, numb.nums[i]) setarrayitem(lst, j, x) for FIELDTYPE, fieldname in unroll_static_fields_rev: i -= 1 assert i >= 0 - x = reader.load_value_of_type(FIELDTYPE, nums[i]) + x = reader.load_value_of_type(FIELDTYPE, numb.nums[i]) setattr(virtualizable, fieldname, x) return i # - def load_list_of_boxes(virtualizable, reader, nums): + def load_list_of_boxes(virtualizable, reader, numb): # Uses 'virtualizable' only to know the length of the arrays; # does not write anything into it. The returned list is in # the format expected of virtualizable_boxes, so it ends in # the virtualizable itself. - i = len(nums) - 1 + i = len(numb.nums) - 1 assert i >= 0 - boxes = [reader.decode_box_of_type(self.VTYPEPTR, nums[i])] + boxes = [reader.decode_box_of_type(self.VTYPEPTR, numb.nums[i])] for ARRAYITEMTYPE, fieldname in unroll_array_fields_rev: lst = getattr(virtualizable, fieldname) for j in range(getlength(lst)-1, -1, -1): i -= 1 assert i >= 0 - box = reader.decode_box_of_type(ARRAYITEMTYPE, nums[i]) + box = reader.decode_box_of_type(ARRAYITEMTYPE,numb.nums[i]) boxes.append(box) for FIELDTYPE, fieldname in unroll_static_fields_rev: i -= 1 assert i >= 0 - box = reader.decode_box_of_type(FIELDTYPE, nums[i]) + box = reader.decode_box_of_type(FIELDTYPE, numb.nums[i]) boxes.append(box) boxes.reverse() return boxes Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmspot.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/warmspot.py Wed Nov 17 20:41:25 2010 @@ -98,8 +98,7 @@ repeat -= 1 return res -def rpython_ll_meta_interp(function, args, backendopt=True, - loops='not used right now', **kwds): +def rpython_ll_meta_interp(function, args, backendopt=True, **kwds): return ll_meta_interp(function, args, backendopt=backendopt, translate_support_code=True, **kwds) Modified: pypy/branch/jit-unroll-loops/pypy/jit/tool/oparser.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/tool/oparser.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/tool/oparser.py Wed Nov 17 20:41:25 2010 @@ -192,7 +192,7 @@ descr = None if argspec.strip(): if opname == 'debug_merge_point': - allargs = [argspec] + allargs = argspec.rsplit(', ', 1) else: allargs = [arg for arg in argspec.split(",") if arg != ''] Modified: pypy/branch/jit-unroll-loops/pypy/jit/tool/pypytrace-mode.el ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/tool/pypytrace-mode.el (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/tool/pypytrace-mode.el Wed Nov 17 20:41:25 2010 @@ -26,7 +26,7 @@ ("<.*FieldDescr \\([^ ]*\\)" (1 'font-lock-variable-name-face)) ;; comment out debug_merge_point, but then highlight specific part of it ("^debug_merge_point.*" . font-lock-comment-face) - ("^\\(debug_merge_point\\).*code object\\(.*\\), file \\('.*'\\), \\(line .*\\)> \\(.*\\)')" + ("^\\(debug_merge_point\\).*code object\\(.*\\), file \\('.*'\\), \\(line .*\\)> \\(.*\\)" (1 'compilation-warning t) (2 'escape-glyph t) (3 'font-lock-string-face t) Modified: pypy/branch/jit-unroll-loops/pypy/jit/tool/pypytrace.vim ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/tool/pypytrace.vim (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/tool/pypytrace.vim Wed Nov 17 20:41:25 2010 @@ -21,10 +21,10 @@ hi def link pypyLoopStart Structure "hi def link pypyLoopArgs PreProc -hi def link pypyFailArgs String +hi def link pypyFailArgs Special "hi def link pypyOpName Statement -hi def link pypyDebugMergePoint Comment +hi def link pypyDebugMergePoint String hi def link pypyConstPtr Constant hi def link pypyNumber Number -hi def link pypyDescr String +hi def link pypyDescr PreProc hi def link pypyDescrField Label Modified: pypy/branch/jit-unroll-loops/pypy/jit/tool/test/test_traceviewer.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/tool/test/test_traceviewer.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/tool/test/test_traceviewer.py Wed Nov 17 20:41:25 2010 @@ -19,7 +19,7 @@ def test_no_of_loops(self): data = [preparse(""" # Loop 0 : loop with 39 ops - debug_merge_point('') + debug_merge_point('', 0) guard_class(p4, 141310752, descr=) [p0, p1] p60 = getfield_gc(p4, descr=) guard_nonnull(p60, descr=) [p0, p1] @@ -51,7 +51,7 @@ assert loop.right.content == 'extra' def test_postparse(self): - real_loops = [FinalBlock("debug_merge_point(' #40 POP_TOP')", None)] + real_loops = [FinalBlock("debug_merge_point(' #40 POP_TOP', 0)", None)] postprocess(real_loops, real_loops[:], {}) assert real_loops[0].header.startswith("_runCallbacks, file '/tmp/x/twisted-trunk/twisted/internet/defer.py', line 357") Modified: pypy/branch/jit-unroll-loops/pypy/module/__builtin__/descriptor.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/__builtin__/descriptor.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/__builtin__/descriptor.py Wed Nov 17 20:41:25 2010 @@ -96,6 +96,8 @@ ) class W_Property(Wrappable): + _immutable_fields_ = ["w_fget", "w_fset", "w_fdel"] + def init(self, space, w_fget=None, w_fset=None, w_fdel=None, w_doc=None): self.w_fget = w_fget self.w_fset = w_fset @@ -183,4 +185,3 @@ fget = interp_attrproperty_w('w_fget', W_Property), fset = interp_attrproperty_w('w_fset', W_Property), ) - Modified: pypy/branch/jit-unroll-loops/pypy/module/__pypy__/__init__.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/__pypy__/__init__.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/__pypy__/__init__.py Wed Nov 17 20:41:25 2010 @@ -23,5 +23,12 @@ 'interp_magic.method_cache_counter') self.extra_interpdef('reset_method_cache_counter', 'interp_magic.reset_method_cache_counter') + if self.space.config.objspace.std.withmapdict: + self.extra_interpdef('mapdict_cache_counter', + 'interp_magic.mapdict_cache_counter') PYC_MAGIC = get_pyc_magic(self.space) self.extra_interpdef('PYC_MAGIC', 'space.wrap(%d)' % PYC_MAGIC) + # + from pypy.jit.backend import detect_cpu + model = detect_cpu.autodetect_main_model_and_size() + self.extra_interpdef('cpumodel', 'space.wrap(%r)' % model) Modified: pypy/branch/jit-unroll-loops/pypy/module/__pypy__/interp_magic.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/__pypy__/interp_magic.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/__pypy__/interp_magic.py Wed Nov 17 20:41:25 2010 @@ -2,6 +2,7 @@ from pypy.interpreter.gateway import ObjSpace from pypy.rlib.objectmodel import we_are_translated from pypy.objspace.std.typeobject import MethodCache +from pypy.objspace.std.mapdict import IndexCache def internal_repr(space, w_object): return space.wrap('%r' % (w_object,)) @@ -36,4 +37,17 @@ cache = space.fromcache(MethodCache) cache.misses = {} cache.hits = {} - + if space.config.objspace.std.withmapdict: + cache = space.fromcache(IndexCache) + cache.misses = {} + cache.hits = {} + +def mapdict_cache_counter(space, name): + """Return a tuple (index_cache_hits, index_cache_misses) for lookups + in the mapdict cache with the given attribute name.""" + assert space.config.objspace.std.withmethodcachecounter + assert space.config.objspace.std.withmapdict + cache = space.fromcache(IndexCache) + return space.newtuple([space.newint(cache.hits.get(name, 0)), + space.newint(cache.misses.get(name, 0))]) +mapdict_cache_counter.unwrap_spec = [ObjSpace, str] Modified: pypy/branch/jit-unroll-loops/pypy/module/__pypy__/test/test_special.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/__pypy__/test/test_special.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/__pypy__/test/test_special.py Wed Nov 17 20:41:25 2010 @@ -17,3 +17,7 @@ from __pypy__ import isfake import select assert isfake(select) + + def test_cpumodel(self): + import __pypy__ + assert hasattr(__pypy__, 'cpumodel') Modified: pypy/branch/jit-unroll-loops/pypy/module/_rawffi/test/test__rawffi.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/_rawffi/test/test__rawffi.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/_rawffi/test/test__rawffi.py Wed Nov 17 20:41:25 2010 @@ -296,6 +296,7 @@ assert _rawffi.charp2string(res[0]) is None arg1.free() arg2.free() + a.free() def test_raw_callable(self): import _rawffi Modified: pypy/branch/jit-unroll-loops/pypy/module/_rawffi/test/test_nested.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/_rawffi/test/test_nested.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/_rawffi/test/test_nested.py Wed Nov 17 20:41:25 2010 @@ -107,7 +107,6 @@ assert S.fieldoffset('x') == 0 assert S.fieldoffset('ar') == A5alignment s = S() - s = S() s.x = 'G' raises(TypeError, 's.ar') assert s.fieldaddress('ar') == s.buffer + S.fieldoffset('ar') Modified: pypy/branch/jit-unroll-loops/pypy/module/array/interp_array.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/array/interp_array.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/array/interp_array.py Wed Nov 17 20:41:25 2010 @@ -27,7 +27,7 @@ typecode = typecode[0] if space.is_w(w_cls, space.gettypeobject(W_ArrayBase.typedef)): - if len(w_args.keywords_w) > 0: + if w_args.keywords: # XXX this might be forbidden fishing msg = 'array.array() does not take keyword arguments' raise OperationError(space.w_TypeError, space.wrap(msg)) Modified: pypy/branch/jit-unroll-loops/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/cpyext/api.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/cpyext/api.py Wed Nov 17 20:41:25 2010 @@ -226,7 +226,7 @@ def unwrapper(space, *args): from pypy.module.cpyext.pyobject import Py_DecRef from pypy.module.cpyext.pyobject import make_ref, from_ref - from pypy.module.cpyext.pyobject import BorrowPair + from pypy.module.cpyext.pyobject import Reference newargs = () to_decref = [] assert len(args) == len(api_function.argtypes) @@ -270,8 +270,8 @@ return api_function.error_value if res is None: return None - elif isinstance(res, BorrowPair): - return res.w_borrowed + elif isinstance(res, Reference): + return res.get_wrapped(space) else: return res finally: @@ -473,7 +473,7 @@ @specialize.ll() def wrapper(*args): from pypy.module.cpyext.pyobject import make_ref, from_ref - from pypy.module.cpyext.pyobject import BorrowPair + from pypy.module.cpyext.pyobject import Reference # we hope that malloc removal removes the newtuple() that is # inserted exactly here by the varargs specializer llop.gc_stack_bottom(lltype.Void) # marker for trackgcroot.py @@ -525,7 +525,7 @@ elif is_PyObject(callable.api_func.restype): if result is None: retval = make_ref(space, None) - elif isinstance(result, BorrowPair): + elif isinstance(result, Reference): retval = result.get_ref(space) elif not rffi._isllptr(result): retval = rffi.cast(callable.api_func.restype, @@ -908,8 +908,10 @@ from pypy.rlib import rdynload try: ll_libname = rffi.str2charp(path) - dll = rdynload.dlopen(ll_libname) - lltype.free(ll_libname, flavor='raw') + try: + dll = rdynload.dlopen(ll_libname) + finally: + lltype.free(ll_libname, flavor='raw') except rdynload.DLOpenError, e: raise operationerrfmt( space.w_ImportError, Modified: pypy/branch/jit-unroll-loops/pypy/module/cpyext/cdatetime.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/cpyext/cdatetime.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/cpyext/cdatetime.py Wed Nov 17 20:41:25 2010 @@ -4,7 +4,7 @@ from pypy.module.cpyext.api import ( cpython_api, CANNOT_FAIL, cpython_struct, PyObjectFields) from pypy.module.cpyext.import_ import PyImport_Import -from pypy.module.cpyext.typeobject import PyTypeObjectPtr +from pypy.module.cpyext.typeobject import PyTypeObjectPtr, render_immortal from pypy.module.cpyext.state import State from pypy.interpreter.error import OperationError from pypy.tool.sourcetools import func_renamer @@ -22,25 +22,34 @@ @cpython_api([], lltype.Ptr(PyDateTime_CAPI), error=lltype.nullptr(PyDateTime_CAPI)) def _PyDateTime_Import(space): - datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw') + datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw', + track_allocation=False) if not we_are_translated(): datetimeAPI_dealloc(space) space.fromcache(State).datetimeAPI = datetimeAPI w_datetime = PyImport_Import(space, space.wrap("datetime")) + w_type = space.getattr(w_datetime, space.wrap("date")) datetimeAPI.c_DateType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DateType, w_type) + w_type = space.getattr(w_datetime, space.wrap("datetime")) datetimeAPI.c_DateTimeType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DateTimeType, w_type) + w_type = space.getattr(w_datetime, space.wrap("time")) datetimeAPI.c_TimeType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_TimeType, w_type) + w_type = space.getattr(w_datetime, space.wrap("timedelta")) datetimeAPI.c_DeltaType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DeltaType, w_type) return datetimeAPI Modified: pypy/branch/jit-unroll-loops/pypy/module/cpyext/presetup.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/cpyext/presetup.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/cpyext/presetup.py Wed Nov 17 20:41:25 2010 @@ -21,6 +21,8 @@ from pypy.conftest import gettestobjspace from pypy.module.cpyext.api import build_bridge +from pypy.module.imp.importing import get_so_extension + usemodules = ['cpyext', 'thread'] if sys.platform == 'win32': usemodules.append('_winreg') # necessary in distutils @@ -35,6 +37,7 @@ def patch_distutils(): sysconfig.get_python_inc = get_python_inc + sysconfig.get_config_vars()['SO'] = get_so_extension(space) patch_distutils() Modified: pypy/branch/jit-unroll-loops/pypy/module/cpyext/pyobject.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/cpyext/pyobject.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/cpyext/pyobject.py Wed Nov 17 20:41:25 2010 @@ -424,7 +424,18 @@ state = space.fromcache(RefcountState) return state.make_borrowed(w_container, w_borrowed) -class BorrowPair: +class Reference: + def __init__(self, pyobj): + assert not isinstance(pyobj, W_Root) + self.pyobj = pyobj + + def get_ref(self, space): + return self.pyobj + + def get_wrapped(self, space): + return from_ref(space, self.pyobj) + +class BorrowPair(Reference): """ Delays the creation of a borrowed reference. """ @@ -435,6 +446,9 @@ def get_ref(self, space): return make_borrowed_ref(space, self.w_container, self.w_borrowed) + def get_wrapped(self, space): + return self.w_borrowed + def borrow_from(container, borrowed): return BorrowPair(container, borrowed) Modified: pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_borrow.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_borrow.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_borrow.py Wed Nov 17 20:41:25 2010 @@ -31,6 +31,7 @@ g = PyTuple_GetItem(t, 0); // borrows reference again printf("Refcnt4: %i\\n", f->ob_refcnt); printf("COMPARE: %i\\n", f == g); + fflush(stdout); Py_DECREF(t); Py_RETURN_TRUE; """), Modified: pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_cpyext.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_cpyext.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/cpyext/test/test_cpyext.py Wed Nov 17 20:41:25 2010 @@ -42,7 +42,7 @@ raises(ImportError, cpyext.load_module, "missing.file", "foo") raises(ImportError, cpyext.load_module, self.libc, "invalid.function") -def compile_module(modname, **kwds): +def compile_module(space, modname, **kwds): """ Build an extension module and return the filename of the resulting native code file. @@ -65,10 +65,8 @@ [], eci, outputfilename=str(dirname/modname), standalone=False) - if sys.platform == 'win32': - pydname = soname.new(purebasename=modname, ext='.pyd') - else: - pydname = soname.new(purebasename=modname, ext='.so') + from pypy.module.imp.importing import get_so_extension + pydname = soname.new(purebasename=modname, ext=get_so_extension(space)) soname.rename(pydname) return str(pydname) @@ -153,7 +151,7 @@ kwds["link_files"] = [str(api_library + '.so')] if sys.platform == 'linux2': kwds["compile_extra"]=["-Werror=implicit-function-declaration"] - return compile_module(name, **kwds) + return compile_module(self.space, name, **kwds) def import_module(self, name, init=None, body='', load_it=True, filename=None): Modified: pypy/branch/jit-unroll-loops/pypy/module/cpyext/typeobject.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/cpyext/typeobject.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/cpyext/typeobject.py Wed Nov 17 20:41:25 2010 @@ -182,10 +182,10 @@ subtype = rffi.cast(PyTypeObjectPtr, make_ref(space, w_subtype)) try: - obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds) + w_obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds) finally: Py_DecRef(space, w_subtype) - return obj + return w_obj @specialize.memo() def get_new_method_def(space): @@ -193,10 +193,14 @@ if state.new_method_def: return state.new_method_def from pypy.module.cpyext.modsupport import PyMethodDef - ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True) + ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True, + immortal=True) ptr.c_ml_name = rffi.str2charp("__new__") + lltype.render_immortal(ptr.c_ml_name) rffi.setintfield(ptr, 'c_ml_flags', METH_VARARGS | METH_KEYWORDS) - ptr.c_ml_doc = rffi.str2charp("T.__new__(S, ...) -> a new object with type S, a subtype of T") + ptr.c_ml_doc = rffi.str2charp( + "T.__new__(S, ...) -> a new object with type S, a subtype of T") + lltype.render_immortal(ptr.c_ml_doc) state.new_method_def = ptr return ptr @@ -429,6 +433,12 @@ finish_type_1(space, pto) finish_type_2(space, pto, w_type) + if space.type(w_type).is_cpytype(): + # XXX Types with a C metatype are never freed, try to see why... + render_immortal(pto, w_type) + lltype.render_immortal(pto) + lltype.render_immortal(pto.c_tp_name) + pto.c_tp_basicsize = rffi.sizeof(typedescr.basestruct) if pto.c_tp_base: if pto.c_tp_base.c_tp_basicsize > pto.c_tp_basicsize: @@ -534,12 +544,25 @@ w_obj.ready() finish_type_2(space, py_type, w_obj) + render_immortal(py_type, w_obj) state = space.fromcache(RefcountState) state.non_heaptypes_w.append(w_obj) return w_obj +def render_immortal(py_type, w_obj): + lltype.render_immortal(py_type.c_tp_bases) + lltype.render_immortal(py_type.c_tp_mro) + + assert isinstance(w_obj, W_TypeObject) + if w_obj.is_cpytype(): + lltype.render_immortal(py_type.c_tp_dict) + else: + lltype.render_immortal(py_type.c_tp_name) + if not w_obj.is_cpytype() and w_obj.is_heaptype(): + lltype.render_immortal(py_type) + def finish_type_1(space, pto): """ Sets up tp_bases, necessary before creating the interpreter type. Modified: pypy/branch/jit-unroll-loops/pypy/module/gc/__init__.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/gc/__init__.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/gc/__init__.py Wed Nov 17 20:41:25 2010 @@ -29,6 +29,7 @@ 'get_referents': 'referents.get_referents', 'get_referrers': 'referents.get_referrers', '_dump_rpy_heap': 'referents._dump_rpy_heap', + 'get_typeids_z': 'referents.get_typeids_z', 'GcRef': 'referents.W_GcRef', }) MixedModule.__init__(self, space, w_name) Modified: pypy/branch/jit-unroll-loops/pypy/module/gc/app_referents.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/gc/app_referents.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/gc/app_referents.py Wed Nov 17 20:41:25 2010 @@ -14,11 +14,25 @@ and [addr1]..[addrn] are addresses of other objects that this object points to. The full dump is a list of such objects, with a marker [0][0][0][-1] inserted after all GC roots, before all non-roots. + + If the argument is a filename and the 'zlib' module is available, + we also write a 'typeids.txt' in the same directory, if none exists. """ if isinstance(file, str): f = open(file, 'wb') gc._dump_rpy_heap(f.fileno()) f.close() + try: + import zlib, os + except ImportError: + pass + else: + filename2 = os.path.join(os.path.dirname(file), 'typeids.txt') + if not os.path.exists(filename2): + data = zlib.decompress(gc.get_typeids_z()) + f = open(filename2, 'wb') + f.write(data) + f.close() else: if isinstance(file, int): fd = file Modified: pypy/branch/jit-unroll-loops/pypy/module/gc/interp_gc.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/gc/interp_gc.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/gc/interp_gc.py Wed Nov 17 20:41:25 2010 @@ -10,6 +10,10 @@ from pypy.objspace.std.typeobject import MethodCache cache = space.fromcache(MethodCache) cache.clear() + if space.config.objspace.std.withmapdict: + from pypy.objspace.std.mapdict import IndexCache + cache = space.fromcache(IndexCache) + cache.clear() rgc.collect() return space.wrap(0) Modified: pypy/branch/jit-unroll-loops/pypy/module/gc/referents.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/gc/referents.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/gc/referents.py Wed Nov 17 20:41:25 2010 @@ -177,3 +177,9 @@ if not ok: raise missing_operation(space) _dump_rpy_heap.unwrap_spec = [ObjSpace, int] + +def get_typeids_z(space): + a = rgc.get_typeids_z() + s = ''.join([a[i] for i in range(len(a))]) + return space.wrap(s) +get_typeids_z.unwrap_spec = [ObjSpace] Modified: pypy/branch/jit-unroll-loops/pypy/module/gc/test/test_gc.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/gc/test/test_gc.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/gc/test/test_gc.py Wed Nov 17 20:41:25 2010 @@ -117,6 +117,33 @@ pass C().f() # Fill the method cache rlist.append(weakref.ref(C)) + for i in range(10): + f() + gc.collect() # the classes C should all go away here + # the last class won't go in mapdict, as long as the code object of f + # is around + rlist.pop() + for r in rlist: + assert r() is None + +class AppTestGcMapDictIndexCache(AppTestGcMethodCache): + def setup_class(cls): + cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True, + "objspace.std.withmapdict": True}) + + + def test_clear_index_cache(self): + import gc, weakref + rlist = [] + def f(): + class C(object): + def f(self): + pass + c = C() + c.x = 1 + getattr(c, "x") # fill the index cache without using the local cache + getattr(c, "x") + rlist.append(weakref.ref(C)) for i in range(5): f() gc.collect() # the classes C should all go away here Modified: pypy/branch/jit-unroll-loops/pypy/module/imp/importing.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/imp/importing.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/imp/importing.py Wed Nov 17 20:41:25 2010 @@ -12,7 +12,7 @@ from pypy.rlib import streamio, jit from pypy.rlib.streamio import StreamErrors from pypy.rlib.rarithmetic import intmask -from pypy.rlib.objectmodel import we_are_translated +from pypy.rlib.objectmodel import we_are_translated, specialize SEARCH_ERROR = 0 PY_SOURCE = 1 @@ -25,10 +25,26 @@ # PY_CODERESOURCE = 8 IMP_HOOK = 9 -if sys.platform.startswith('win'): - so_extension = ".pyd" +if sys.platform == 'win32': + SO = ".pyd" else: - so_extension = ".so" + SO = ".so" +DEFAULT_SOABI = 'pypy-14' + + at specialize.memo() +def get_so_extension(space): + if space.config.objspace.soabi is not None: + soabi = space.config.objspace.soabi + else: + soabi = DEFAULT_SOABI + + if not soabi: + return SO + + if not space.config.translating: + soabi += 'i' + + return '.' + soabi + SO def find_modtype(space, filepart): """Check which kind of module to import for the given filepart, @@ -53,6 +69,7 @@ return PY_COMPILED, ".pyc", "rb" if space.config.objspace.usemodules.cpyext: + so_extension = get_so_extension(space) pydfile = filepart + so_extension if os.path.exists(pydfile) and case_ok(pydfile): return C_EXTENSION, so_extension, "rb" Modified: pypy/branch/jit-unroll-loops/pypy/module/imp/interp_imp.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/imp/interp_imp.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/imp/interp_imp.py Wed Nov 17 20:41:25 2010 @@ -8,10 +8,16 @@ def get_suffixes(space): w = space.wrap - return space.newlist([ + suffixes_w = [] + if space.config.objspace.usemodules.cpyext: + suffixes_w.append( + space.newtuple([w(importing.get_so_extension(space)), + w('rb'), w(importing.C_EXTENSION)])) + suffixes_w.extend([ space.newtuple([w('.py'), w('U'), w(importing.PY_SOURCE)]), space.newtuple([w('.pyc'), w('rb'), w(importing.PY_COMPILED)]), ]) + return space.newlist(suffixes_w) def get_magic(space): x = importing.get_pyc_magic(space) Modified: pypy/branch/jit-unroll-loops/pypy/module/imp/test/test_app.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/imp/test/test_app.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/imp/test/test_app.py Wed Nov 17 20:41:25 2010 @@ -47,6 +47,9 @@ elif mode == self.imp.PY_COMPILED: assert suffix in ('.pyc', '.pyo') assert type == 'rb' + elif mode == self.imp.C_EXTENSION: + assert suffix.endswith(('.pyd', '.so')) + assert type == 'rb' def test_obscure_functions(self): Modified: pypy/branch/jit-unroll-loops/pypy/module/imp/test/test_import.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/imp/test/test_import.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/imp/test/test_import.py Wed Nov 17 20:41:25 2010 @@ -473,6 +473,17 @@ except ImportError: pass +class TestAbi: + def test_abi_tag(self): + space1 = gettestobjspace(soabi='TEST') + space2 = gettestobjspace(soabi='') + if sys.platform == 'win32': + assert importing.get_so_extension(space1) == '.TESTi.pyd' + assert importing.get_so_extension(space2) == '.pyd' + else: + assert importing.get_so_extension(space1) == '.TESTi.so' + assert importing.get_so_extension(space2) == '.so' + def _getlong(data): x = marshal.dumps(data) return x[-4:] Modified: pypy/branch/jit-unroll-loops/pypy/module/posix/interp_posix.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/posix/interp_posix.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/posix/interp_posix.py Wed Nov 17 20:41:25 2010 @@ -454,7 +454,8 @@ self.w_environ = space.newdict() if _WIN: self.cryptProviderPtr = lltype.malloc( - rffi.CArray(HCRYPTPROV), 1, zero=True, flavor='raw') + rffi.CArray(HCRYPTPROV), 1, zero=True, + flavor='raw', immortal=True) def startup(self, space): _convertenviron(space, self.w_environ) def _freeze_(self): Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/interp_jit.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/interp_jit.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/interp_jit.py Wed Nov 17 20:41:25 2010 @@ -6,6 +6,7 @@ from pypy.tool.pairtype import extendabletype from pypy.rlib.rarithmetic import r_uint, intmask from pypy.rlib.jit import JitDriver, hint, we_are_jitted, dont_look_inside +from pypy.rlib.jit import current_trace_length import pypy.interpreter.pyopcode # for side-effects from pypy.interpreter.error import OperationError, operationerrfmt from pypy.interpreter.gateway import ObjSpace, Arguments, W_Root @@ -80,9 +81,22 @@ def jump_absolute(self, jumpto, _, ec=None): if we_are_jitted(): + # Normally, the tick counter is decremented by 100 for every + # Python opcode. Here, to better support JIT compilation of + # small loops, we decrement it by a possibly smaller constant. + # We get the maximum 100 when the (unoptimized) trace length + # is at least 3200 (a bit randomly). + trace_length = r_uint(current_trace_length()) + decr_by = trace_length // 32 + if decr_by < 1: + decr_by = 1 + elif decr_by > 100: # also if current_trace_length() returned -1 + decr_by = 100 + # self.last_instr = intmask(jumpto) - ec.bytecode_trace(self) + ec.bytecode_trace(self, intmask(decr_by)) jumpto = r_uint(self.last_instr) + # pypyjitdriver.can_enter_jit(frame=self, ec=ec, next_instr=jumpto, pycode=self.getcode()) return jumpto Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/policy.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/policy.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/policy.py Wed Nov 17 20:41:25 2010 @@ -6,7 +6,8 @@ if (modname == '__builtin__.operation' or modname == '__builtin__.abstractinst' or modname == '__builtin__.interp_classobj' or - modname == '__builtin__.functional'): + modname == '__builtin__.functional' or + modname == '__builtin__.descriptor'): return True if '.' in modname: modname, _ = modname.split('.', 1) @@ -19,7 +20,7 @@ # this function should never actually return True directly # but instead call the base implementation mod = func.__module__ or '?' - + if mod.startswith('pypy.objspace.'): # gc_id operation if func.__name__ == 'id__ANY': @@ -36,5 +37,5 @@ modname = mod[len('pypy.module.'):] if not self.look_inside_pypy_module(modname): return False - + return True Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_policy.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_policy.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_policy.py Wed Nov 17 20:41:25 2010 @@ -12,7 +12,7 @@ def test_rlocale(): from pypy.rlib.rlocale import setlocale - assert not pypypolicy.look_inside_function(setlocale) + assert not pypypolicy.look_inside_function(setlocale) def test_geninterp(): d = {'_geninterp_': True} @@ -28,6 +28,10 @@ from pypy.interpreter.pyparser import parser assert not pypypolicy.look_inside_function(parser.Grammar.__init__.im_func) +def test_property(): + from pypy.module.__builtin__.descriptor import W_Property + assert pypypolicy.look_inside_function(W_Property.get.im_func) + def test_pypy_module(): from pypy.module._random.interp_random import W_Random assert not pypypolicy.look_inside_function(W_Random.random) @@ -35,6 +39,7 @@ assert pypypolicy.look_inside_pypy_module('__builtin__.operation') assert pypypolicy.look_inside_pypy_module('__builtin__.abstractinst') assert pypypolicy.look_inside_pypy_module('__builtin__.functional') + assert pypypolicy.look_inside_pypy_module('__builtin__.descriptor') assert pypypolicy.look_inside_pypy_module('exceptions.interp_exceptions') for modname in 'pypyjit', 'signal', 'micronumpy', 'math', 'imp': assert pypypolicy.look_inside_pypy_module(modname) @@ -42,4 +47,3 @@ def test_see_jit_module(): assert pypypolicy.look_inside_pypy_module('pypyjit.interp_jit') - Modified: pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/pypyjit/test/test_pypy_c.py Wed Nov 17 20:41:25 2010 @@ -422,10 +422,75 @@ ([1000], 49500), ([10000], 495000), ([100000], 4950000)) - assert len(self.loops) == 2 + assert len(self.loops) == 3 op, = self.get_by_bytecode("CALL_FUNCTION_KW") # XXX a bit too many guards, but better than before - assert len(op.get_opnames("guard")) <= 10 + assert len(op.get_opnames("guard")) <= 12 + + def test_stararg_virtual(self): + self.run_source(''' + d = {} + + def g(*args): + return len(args) + def h(a, b, c): + return c + + def main(x): + s = 0 + for i in range(x): + l = [i, x, 2] + s += g(*l) + s += h(*l) + s += g(i, x, 2) + for i in range(x): + l = [x, 2] + s += g(i, *l) + s += h(i, *l) + return s + ''', 100000, ([100], 1300), + ([1000], 13000), + ([10000], 130000), + ([100000], 1300000)) + assert len(self.loops) == 2 + ops = self.get_by_bytecode("CALL_FUNCTION_VAR") + assert len(ops) == 4 + for op in ops: + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 + + ops = self.get_by_bytecode("CALL_FUNCTION") + for op in ops: + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 + + def test_stararg(self): + self.run_source(''' + d = {} + + def g(*args): + return args[-1] + def h(*args): + return len(args) + + def main(x): + s = 0 + l = [] + i = 0 + while i < x: + l.append(1) + s += g(*l) + i = h(*l) + return s + ''', 100000, ([100], 100), + ([1000], 1000), + ([2000], 2000), + ([4000], 4000)) + assert len(self.loops) == 1 + ops = self.get_by_bytecode("CALL_FUNCTION_VAR") + for op in ops: + assert len(op.get_opnames("new_with_vtable")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 def test_virtual_instance(self): self.run_source(''' Modified: pypy/branch/jit-unroll-loops/pypy/module/rctime/interp_time.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/rctime/interp_time.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/rctime/interp_time.py Wed Nov 17 20:41:25 2010 @@ -69,7 +69,7 @@ CLOCKS_PER_SEC = cConfig.CLOCKS_PER_SEC clock_t = cConfig.clock_t tm = cConfig.tm -glob_buf = lltype.malloc(tm, flavor='raw', zero=True) +glob_buf = lltype.malloc(tm, flavor='raw', zero=True, immortal=True) if cConfig.has_gettimeofday: c_gettimeofday = external('gettimeofday', [rffi.VOIDP, rffi.VOIDP], rffi.INT) Modified: pypy/branch/jit-unroll-loops/pypy/module/signal/__init__.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/signal/__init__.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/signal/__init__.py Wed Nov 17 20:41:25 2010 @@ -34,9 +34,7 @@ MixedModule.__init__(self, space, *args) # add the signal-checking callback as an action on the space space.check_signal_action = interp_signal.CheckSignalAction(space) - space.actionflag.register_action(space.check_signal_action) - # use the C-level pypysig_occurred variable as the action flag - # (the result is that the C-level signal handler will directly - # set the flag for the CheckSignalAction) + space.actionflag.register_periodic_action(space.check_signal_action, + use_bytecode_counter=False) space.actionflag.__class__ = interp_signal.SignalActionFlag # xxx yes I know the previous line is a hack Modified: pypy/branch/jit-unroll-loops/pypy/module/signal/interp_signal.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/signal/interp_signal.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/signal/interp_signal.py Wed Nov 17 20:41:25 2010 @@ -1,6 +1,7 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.baseobjspace import W_Root, ObjSpace from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag +from pypy.interpreter.executioncontext import PeriodicAsyncAction import signal as cpy_signal from pypy.rpython.lltypesystem import lltype, rffi from pypy.translator.tool.cbuild import ExternalCompilationInfo @@ -52,19 +53,29 @@ class SignalActionFlag(AbstractActionFlag): - def get(self): + # This class uses the C-level pypysig_counter variable as the tick + # counter. The C-level signal handler will reset it to -1 whenever + # a signal is received. + + def get_ticker(self): p = pypysig_getaddr_occurred() return p.c_value - def set(self, value): + + def reset_ticker(self, value): p = pypysig_getaddr_occurred() p.c_value = value + def decrement_ticker(self, by): + p = pypysig_getaddr_occurred() + value = p.c_value + if self.has_bytecode_counter: # this 'if' is constant-folded + value -= by + p.c_value = value + return value -class CheckSignalAction(AsyncAction): - """An action that is automatically invoked when a signal is received.""" - # The C-level signal handler sets the bit 30 of pypysig_occurred: - bitmask = 1 << 30 +class CheckSignalAction(PeriodicAsyncAction): + """An action that is automatically invoked when a signal is received.""" def __init__(self, space): AsyncAction.__init__(self, space) @@ -73,7 +84,6 @@ # need a helper action in case signals arrive in a non-main thread self.pending_signals = {} self.reissue_signal_action = ReissueSignalAction(space) - space.actionflag.register_action(self.reissue_signal_action) else: self.reissue_signal_action = None Modified: pypy/branch/jit-unroll-loops/pypy/module/signal/test/test_signal.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/signal/test/test_signal.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/signal/test/test_signal.py Wed Nov 17 20:41:25 2010 @@ -1,6 +1,38 @@ import os, py +import signal as cpy_signal from pypy.conftest import gettestobjspace + +class TestCheckSignals: + + def setup_class(cls): + if not hasattr(os, 'kill') or not hasattr(os, 'getpid'): + py.test.skip("requires os.kill() and os.getpid()") + cls.space = gettestobjspace(usemodules=['signal']) + + def test_checksignals(self): + space = self.space + w_received = space.appexec([], """(): + import signal + received = [] + def myhandler(signum, frame): + received.append(signum) + signal.signal(signal.SIGUSR1, myhandler) + return received""") + # + assert not space.is_true(w_received) + # + # send the signal now + os.kill(os.getpid(), cpy_signal.SIGUSR1) + # + # myhandler() should not be immediately called + assert not space.is_true(w_received) + # + # calling ec.checksignals() should call it + space.getexecutioncontext().checksignals() + assert space.is_true(w_received) + + class AppTestSignal: def setup_class(cls): @@ -24,18 +56,12 @@ signal.signal(signal.SIGUSR1, myhandler) posix.kill(posix.getpid(), signal.SIGUSR1) - for i in range(10000): - # wait a bit for the signal to be delivered to the handler - if received: - break + # the signal should be delivered to the handler immediately assert received == [signal.SIGUSR1] del received[:] posix.kill(posix.getpid(), signal.SIGUSR1) - for i in range(10000): - # wait a bit for the signal to be delivered to the handler - if received: - break + # the signal should be delivered to the handler immediately assert received == [signal.SIGUSR1] del received[:] Modified: pypy/branch/jit-unroll-loops/pypy/module/sys/__init__.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/sys/__init__.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/sys/__init__.py Wed Nov 17 20:41:25 2010 @@ -10,7 +10,6 @@ if space.config.translating: del self.__class__.interpleveldefs['pypy_getudir'] super(Module, self).__init__(space, w_name) - self.checkinterval = 100 self.recursionlimit = 100 self.w_default_encoder = None self.defaultencoding = "ascii" Modified: pypy/branch/jit-unroll-loops/pypy/module/sys/version.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/sys/version.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/sys/version.py Wed Nov 17 20:41:25 2010 @@ -8,7 +8,7 @@ CPYTHON_VERSION = (2, 5, 2, "beta", 42) #XXX # sync patchlevel.h CPYTHON_API_VERSION = 1012 #XXX # sync with include/modsupport.h -PYPY_VERSION = (1, 3, 0, "beta", '?') #XXX # sync patchlevel.h +PYPY_VERSION = (1, 4, 0, "beta", '?') #XXX # sync patchlevel.h # the last item is replaced by the svn revision ^^^ TRIM_URL_UP_TO = 'svn/pypy/' Modified: pypy/branch/jit-unroll-loops/pypy/module/sys/vm.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/sys/vm.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/sys/vm.py Wed Nov 17 20:41:25 2010 @@ -67,7 +67,7 @@ def setcheckinterval(space, interval): """Tell the Python interpreter to check for asynchronous events every n instructions. This also affects how often thread switches occur.""" - space.actionflag.setcheckinterval(space, interval) + space.actionflag.setcheckinterval(interval) setcheckinterval.unwrap_spec = [ObjSpace, int] def getcheckinterval(space): @@ -77,7 +77,7 @@ # return 0. The idea is that according to the CPython docs, <= 0 # means "check every virtual instruction, maximizing responsiveness # as well as overhead". - result = space.sys.checkinterval + result = space.actionflag.getcheckinterval() if result <= 1: result = 0 return space.wrap(result) Modified: pypy/branch/jit-unroll-loops/pypy/module/thread/gil.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/thread/gil.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/thread/gil.py Wed Nov 17 20:41:25 2010 @@ -20,7 +20,8 @@ def initialize(self, space): # add the GIL-releasing callback as an action on the space - space.actionflag.register_action(GILReleaseAction(space)) + space.actionflag.register_periodic_action(GILReleaseAction(space), + use_bytecode_counter=True) def setup_threads(self, space): """Enable threads in the object space, if they haven't already been.""" @@ -44,7 +45,6 @@ # test_compile_lock. As a workaround, we repatch these global # fields systematically. spacestate.ll_GIL = self.ll_GIL - spacestate.actionflag = space.actionflag invoke_around_extcall(before_external_call, after_external_call) return result @@ -68,18 +68,17 @@ def _freeze_(self): self.ll_GIL = thread.null_ll_lock - self.actionflag = None - self.set_actionflag_bit_after_thread_switch = 0 + self.action_after_thread_switch = None + # ^^^ set by AsyncAction.fire_after_thread_switch() return False def after_thread_switch(self): # this is support logic for the signal module, to help it deliver # signals to the main thread. - actionflag = self.actionflag - if actionflag is not None: - flag = actionflag.get() - flag |= self.set_actionflag_bit_after_thread_switch - actionflag.set(flag) + action = self.action_after_thread_switch + if action is not None: + self.action_after_thread_switch = None + action.fire() spacestate = SpaceState() spacestate._freeze_() Modified: pypy/branch/jit-unroll-loops/pypy/module/thread/test/test_gil.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/module/thread/test/test_gil.py (original) +++ pypy/branch/jit-unroll-loops/pypy/module/thread/test/test_gil.py Wed Nov 17 20:41:25 2010 @@ -8,7 +8,7 @@ pass class FakeActionFlag(object): - def register_action(self, action): + def register_periodic_action(self, action, use_bytecode_counter): pass def get(self): return 0 Modified: pypy/branch/jit-unroll-loops/pypy/objspace/flow/model.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/objspace/flow/model.py (original) +++ pypy/branch/jit-unroll-loops/pypy/objspace/flow/model.py Wed Nov 17 20:41:25 2010 @@ -355,7 +355,7 @@ return "%r = %s(%s)" % (self.result, self.opname, ", ".join(map(repr, self.args))) -class Atom: +class Atom(object): def __init__(self, name): self.__name__ = name # make save_global happy def __repr__(self): Modified: pypy/branch/jit-unroll-loops/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/objspace/flow/objspace.py (original) +++ pypy/branch/jit-unroll-loops/pypy/objspace/flow/objspace.py Wed Nov 17 20:41:25 2010 @@ -210,6 +210,11 @@ check_class = self.unwrap(w_check_class) except UnwrapException: raise Exception, "non-constant except guard" + if check_class in (NotImplementedError, AssertionError): + # if we are in geninterp, we cannot catch these exceptions + if not self.config.translation.builtins_can_raise_exceptions: + raise error.FlowingError("Catching %s is not valid in RPython" % + check_class.__name__) if not isinstance(check_class, tuple): # the simple case return ObjSpace.exception_match(self, w_exc_type, w_check_class) Modified: pypy/branch/jit-unroll-loops/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/branch/jit-unroll-loops/pypy/objspace/flow/test/test_objspace.py Wed Nov 17 20:41:25 2010 @@ -5,7 +5,7 @@ from pypy.objspace.flow.model import flatten, mkentrymap, c_last_exception from pypy.interpreter.argument import Arguments from pypy.translator.simplify import simplify_graph -from pypy.objspace.flow.objspace import FlowObjSpace +from pypy.objspace.flow.objspace import FlowObjSpace, error from pypy.objspace.flow import objspace, flowcontext from pypy import conftest from pypy.tool.stdlib_opcode import bytecode_spec @@ -953,6 +953,22 @@ assert op.args[0] == Constant(g) + def test_cannot_catch_special_exceptions(self): + def f(): + try: + f() + except NotImplementedError: + pass + py.test.raises(error.FlowingError, "self.codetest(f)") + # + def f(): + try: + f() + except AssertionError: + pass + py.test.raises(error.FlowingError, "self.codetest(f)") + + class TestFlowObjSpaceDelay(Base): def setup_class(cls): cls.space = FlowObjSpace() @@ -1013,6 +1029,15 @@ expected.sort() assert excfound == expected + def test_can_catch_special_exceptions(self): + def f(): + try: + f() + except NotImplementedError: + pass + graph = self.codetest(f) + # assert did not crash + DATA = {'x': 5, 'y': 6} Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/callmethod.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/objspace/std/callmethod.py (original) +++ pypy/branch/jit-unroll-loops/pypy/objspace/std/callmethod.py Wed Nov 17 20:41:25 2010 @@ -13,6 +13,9 @@ from pypy.interpreter import function from pypy.objspace.descroperation import object_getattribute from pypy.rlib import jit, rstack # for resume points +from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict, \ + LOOKUP_METHOD_mapdict_fill_cache_method + # This module exports two extra methods for StdObjSpaceFrame implementing # the LOOKUP_METHOD and CALL_METHOD opcodes in an efficient way, as well @@ -30,6 +33,13 @@ # space = f.space w_obj = f.popvalue() + + if space.config.objspace.std.withmapdict and not jit.we_are_jitted(): + # mapdict has an extra-fast version of this function + from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict + if LOOKUP_METHOD_mapdict(f, nameindex, w_obj): + return + w_name = f.getname_w(nameindex) w_value = None @@ -50,6 +60,11 @@ # nothing in the instance f.pushvalue(w_descr) f.pushvalue(w_obj) + if (space.config.objspace.std.withmapdict and + not jit.we_are_jitted()): + # let mapdict cache stuff + LOOKUP_METHOD_mapdict_fill_cache_method( + f.getcode(), nameindex, w_obj, w_type, w_descr) return if w_value is None: w_value = space.getattr(w_obj, w_name) Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/mapdict.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/objspace/std/mapdict.py (original) +++ pypy/branch/jit-unroll-loops/pypy/objspace/std/mapdict.py Wed Nov 17 20:41:25 2010 @@ -1,4 +1,5 @@ from pypy.rlib import jit, objectmodel, debug +from pypy.rlib.rarithmetic import intmask, r_uint from pypy.interpreter.baseobjspace import W_Root from pypy.objspace.std.dictmultiobject import W_DictMultiObject @@ -15,24 +16,70 @@ # we want to propagate knowledge that the result cannot be negative class AbstractAttribute(object): - _immutable_fields_ = ['w_cls'] + _immutable_fields_ = ['terminator'] cache_attrs = None _size_estimate = 0 - def __init__(self, space, w_cls): + def __init__(self, space, terminator): self.space = space - self.w_cls = w_cls + assert isinstance(terminator, Terminator) + self.terminator = terminator def read(self, obj, selector): - raise NotImplementedError("abstract base class") + index = self.index(selector) + if index < 0: + return self.terminator._read_terminator(obj, selector) + return obj._mapdict_read_storage(index) def write(self, obj, selector, w_value): - raise NotImplementedError("abstract base class") + index = self.index(selector) + if index < 0: + return self.terminator._write_terminator(obj, selector, w_value) + obj._mapdict_write_storage(index, w_value) + return True def delete(self, obj, selector): return None def index(self, selector): + if (self.space.config.objspace.std.withmethodcache and + not jit.we_are_jitted()): + return self._index_cache(selector) + else: + return self._index(selector) + + @jit.dont_look_inside + def _index_cache(self, selector): + space = self.space + cache = space.fromcache(IndexCache) + SHIFT2 = r_uint.BITS - space.config.objspace.std.methodcachesizeexp + SHIFT1 = SHIFT2 - 5 + attrs_as_int = objectmodel.current_object_addr_as_int(self) + # ^^^Note: see comment in typeobject.py for + # _pure_lookup_where_with_method_cache() + hash_selector = objectmodel.compute_hash(selector) + product = intmask(attrs_as_int * hash_selector) + index_hash = (r_uint(product) ^ (r_uint(product) << SHIFT1)) >> SHIFT2 + # ^^^Note2: same comment too + cached_attr = cache.attrs[index_hash] + if cached_attr is self: + cached_selector = cache.selectors[index_hash] + if cached_selector == selector: + index = cache.indices[index_hash] + if space.config.objspace.std.withmethodcachecounter: + name = selector[0] + cache.hits[name] = cache.hits.get(name, 0) + 1 + return index + index = self._index(selector) + cache.attrs[index_hash] = self + cache.selectors[index_hash] = selector + cache.indices[index_hash] = index + if space.config.objspace.std.withmethodcachecounter: + name = selector[0] + cache.misses[name] = cache.misses.get(name, 0) + 1 + return index + + def _index(self, selector): return -1 def copy(self, obj): @@ -42,7 +89,7 @@ raise NotImplementedError("abstract base class") def get_terminator(self): - raise NotImplementedError("abstract base class") + return self.terminator def set_terminator(self, obj, terminator): raise NotImplementedError("abstract base class") @@ -95,15 +142,20 @@ raise NotImplementedError("abstract base class") def __repr__(self): - return "<%s w_cls=%s>" % (self.__class__.__name__, self.w_cls) + return "<%s>" % (self.__class__.__name__,) class Terminator(AbstractAttribute): + _immutable_fields_ = ['w_cls'] - def read(self, obj, selector): + def __init__(self, space, w_cls): + AbstractAttribute.__init__(self, space, self) + self.w_cls = w_cls + + def _read_terminator(self, obj, selector): return None - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): obj._get_mapdict_map().add_attr(obj, selector, w_value) return True @@ -116,9 +168,6 @@ def length(self): return 0 - def get_terminator(self): - return self - def set_terminator(self, obj, terminator): result = Object() result.space = self.space @@ -128,6 +177,9 @@ def remove_dict_entries(self, obj): return self.copy(obj) + def __repr__(self): + return "<%s w_cls=%s>" % (self.__class__.__name__, self.w_cls) + class DictTerminator(Terminator): _immutable_fields_ = ['devolved_dict_terminator'] def __init__(self, space, w_cls): @@ -142,27 +194,27 @@ class NoDictTerminator(Terminator): - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): if selector[1] == DICT: return False - return Terminator.write(self, obj, selector, w_value) + return Terminator._write_terminator(self, obj, selector, w_value) class DevolvedDictTerminator(Terminator): - def read(self, obj, selector): + def _read_terminator(self, obj, selector): if selector[1] == DICT: w_dict = obj.getdict() space = self.space return space.finditem_str(w_dict, selector[0]) - return Terminator.read(self, obj, selector) + return Terminator._read_terminator(self, obj, selector) - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): if selector[1] == DICT: w_dict = obj.getdict() space = self.space space.setitem_str(w_dict, selector[0], w_value) return True - return Terminator.write(self, obj, selector, w_value) + return Terminator._write_terminator(self, obj, selector, w_value) def delete(self, obj, selector): from pypy.interpreter.error import OperationError @@ -189,7 +241,7 @@ class PlainAttribute(AbstractAttribute): _immutable_fields_ = ['selector', 'position', 'back'] def __init__(self, selector, back): - AbstractAttribute.__init__(self, back.space, back.w_cls) + AbstractAttribute.__init__(self, back.space, back.terminator) self.selector = selector self.position = back.length() self.back = back @@ -199,17 +251,6 @@ w_value = self.read(obj, self.selector) new_obj._get_mapdict_map().add_attr(new_obj, self.selector, w_value) - def read(self, obj, selector): - if selector == self.selector: - return obj._mapdict_read_storage(self.position) - return self.back.read(obj, selector) - - def write(self, obj, selector, w_value): - if selector == self.selector: - obj._mapdict_write_storage(self.position, w_value) - return True - return self.back.write(obj, selector, w_value) - def delete(self, obj, selector): if selector == self.selector: # ok, attribute is deleted @@ -219,10 +260,10 @@ self._copy_attr(obj, new_obj) return new_obj - def index(self, selector): + def _index(self, selector): if selector == self.selector: return self.position - return self.back.index(selector) + return self.back._index(selector) def copy(self, obj): new_obj = self.back.copy(obj) @@ -232,9 +273,6 @@ def length(self): return self.position + 1 - def get_terminator(self): - return self.back.get_terminator() - def set_terminator(self, obj, terminator): new_obj = self.back.set_terminator(obj, terminator) self._copy_attr(obj, new_obj) @@ -268,6 +306,24 @@ # RPython reasons w_obj._set_mapdict_storage_and_map(new_obj.storage, new_obj.map) +class IndexCache(object): + def __init__(self, space): + assert space.config.objspace.std.withmethodcache + SIZE = 1 << space.config.objspace.std.methodcachesizeexp + self.attrs = [None] * SIZE + self._empty_selector = (None, INVALID) + self.selectors = [self._empty_selector] * SIZE + self.indices = [0] * SIZE + if space.config.objspace.std.withmethodcachecounter: + self.hits = {} + self.misses = {} + + def clear(self): + for i in range(len(self.attrs)): + self.attrs[i] = None + for i in range(len(self.selectors)): + self.selectors[i] = self._empty_selector + # ____________________________________________________________ # object implementation @@ -328,7 +384,7 @@ assert flag def getclass(self, space): - return self._get_mapdict_map().w_cls + return self._get_mapdict_map().terminator.w_cls def setclass(self, space, w_cls): new_obj = self._get_mapdict_map().set_terminator(self, w_cls.terminator) @@ -373,6 +429,7 @@ self.storage = make_sure_not_resized([None] * map.size_estimate()) def _mapdict_read_storage(self, index): + assert index >= 0 return self.storage[index] def _mapdict_write_storage(self, index, value): self.storage[index] = value @@ -440,6 +497,7 @@ return rerased.unerase_fixedsizelist(erased, W_Root) def _mapdict_read_storage(self, index): + assert index >= 0 for i in rangenmin1: if index == i: erased = getattr(self, "_value%s" % i) @@ -604,30 +662,54 @@ map = None version_tag = None index = 0 + w_method = None # for callmethod success_counter = 0 failure_counter = 0 + def is_valid_for_obj(self, w_obj): + map = w_obj._get_mapdict_map() + return self.is_valid_for_map(map) + + def is_valid_for_map(self, map): + if map is self.map: + version_tag = map.terminator.w_cls.version_tag() + if version_tag is self.version_tag: + # everything matches, it's incredibly fast + if map.space.config.objspace.std.withmethodcachecounter: + self.success_counter += 1 + return True + return False + INVALID_CACHE_ENTRY = CacheEntry() INVALID_CACHE_ENTRY.map = objectmodel.instantiate(AbstractAttribute) # different from any real map ^^^ -INVALID_CACHE_ENTRY.map.w_cls = None +INVALID_CACHE_ENTRY.map.terminator = None + def init_mapdict_cache(pycode): num_entries = len(pycode.co_names_w) pycode._mapdict_caches = [INVALID_CACHE_ENTRY] * num_entries +def _fill_cache(pycode, nameindex, map, version_tag, index, w_method=None): + entry = pycode._mapdict_caches[nameindex] + if entry is INVALID_CACHE_ENTRY: + entry = CacheEntry() + pycode._mapdict_caches[nameindex] = entry + entry.map = map + entry.version_tag = version_tag + entry.index = index + entry.w_method = w_method + if pycode.space.config.objspace.std.withmethodcachecounter: + entry.failure_counter += 1 + def LOAD_ATTR_caching(pycode, w_obj, nameindex): # this whole mess is to make the interpreter quite a bit faster; it's not # used if we_are_jitted(). entry = pycode._mapdict_caches[nameindex] map = w_obj._get_mapdict_map() - if map is entry.map: - version_tag = map.w_cls.version_tag() - if version_tag is entry.version_tag: - # everything matches, it's incredibly fast - if pycode.space.config.objspace.std.withmethodcachecounter: - entry.success_counter += 1 - return w_obj._mapdict_read_storage(entry.index) + if entry.is_valid_for_map(map) and entry.w_method is None: + # everything matches, it's incredibly fast + return w_obj._mapdict_read_storage(entry.index) return LOAD_ATTR_slowpath(pycode, w_obj, nameindex, map) LOAD_ATTR_caching._always_inline_ = True @@ -635,7 +717,7 @@ space = pycode.space w_name = pycode.co_names_w[nameindex] if map is not None: - w_type = map.w_cls + w_type = map.terminator.w_cls w_descr = w_type.getattribute_if_not_from_object() if w_descr is not None: return space._handle_getattribute(w_descr, w_obj, w_name) @@ -655,17 +737,30 @@ if selector[1] != INVALID: index = map.index(selector) if index >= 0: - entry = pycode._mapdict_caches[nameindex] - if entry is INVALID_CACHE_ENTRY: - entry = CacheEntry() - pycode._mapdict_caches[nameindex] = entry - entry.map = map - entry.version_tag = version_tag - entry.index = index - if space.config.objspace.std.withmethodcachecounter: - entry.failure_counter += 1 + _fill_cache(pycode, nameindex, map, version_tag, index) return w_obj._mapdict_read_storage(index) if space.config.objspace.std.withmethodcachecounter: INVALID_CACHE_ENTRY.failure_counter += 1 return space.getattr(w_obj, w_name) LOAD_ATTR_slowpath._dont_inline_ = True + +def LOOKUP_METHOD_mapdict(f, nameindex, w_obj): + space = f.space + pycode = f.getcode() + entry = pycode._mapdict_caches[nameindex] + if entry.is_valid_for_obj(w_obj): + w_method = entry.w_method + if w_method is not None: + f.pushvalue(w_method) + f.pushvalue(w_obj) + return True + return False + +def LOOKUP_METHOD_mapdict_fill_cache_method(pycode, nameindex, w_obj, w_type, w_method): + version_tag = w_type.version_tag() + if version_tag is None: + return + map = w_obj._get_mapdict_map() + if map is None: + return + _fill_cache(pycode, nameindex, map, version_tag, -1, w_method) Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/stringobject.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/objspace/std/stringobject.py (original) +++ pypy/branch/jit-unroll-loops/pypy/objspace/std/stringobject.py Wed Nov 17 20:41:25 2010 @@ -937,10 +937,17 @@ string = w_string._value chars = [] - for char in string: - w_char = W_StringObject.PREBUILT[ord(char)] - if not space.is_true(space.contains(w_deletechars, w_char)): - chars.append(table[ord(char)]) + deletechars = space.str_w(w_deletechars) + if len(deletechars) == 0: + for char in string: + chars.append(table[ord(char)]) + else: + deletion_table = [False] * 256 + for c in deletechars: + deletion_table[ord(c)] = True + for char in string: + if not deletion_table[ord(char)]: + chars.append(table[ord(char)]) return W_StringObject(''.join(chars)) def str_decode__String_ANY_ANY(space, w_string, w_encoding=None, w_errors=None): Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_dictmultiobject.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_dictmultiobject.py (original) +++ pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_dictmultiobject.py Wed Nov 17 20:41:25 2010 @@ -616,6 +616,7 @@ withdictmeasurement = False withsmalldicts = False withcelldict = False + withmethodcache = False class opcodes: CALL_LIKELY_BUILTIN = False Modified: pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_mapdict.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_mapdict.py (original) +++ pypy/branch/jit-unroll-loops/pypy/objspace/std/test/test_mapdict.py Wed Nov 17 20:41:25 2010 @@ -24,13 +24,13 @@ hasdict = False def test_plain_attribute(): - space = " " w_cls = "class" aa = PlainAttribute(("b", DICT), PlainAttribute(("a", DICT), Terminator(space, w_cls))) assert aa.space is space - assert aa.w_cls is w_cls + assert aa.terminator.w_cls is w_cls + assert aa.get_terminator() is aa.terminator obj = Object() obj.map, obj.storage = aa, [10, 20] @@ -604,7 +604,8 @@ from pypy.interpreter import gateway cls.space = gettestobjspace( **{"objspace.std.withmapdict": True, - "objspace.std.withmethodcachecounter": True}) + "objspace.std.withmethodcachecounter": True, + "objspace.opcodes.CALL_METHOD": True}) # def check(space, w_func, name): w_code = space.getattr(w_func, space.wrap('func_code')) @@ -785,6 +786,135 @@ res = self.check(f, 'x') assert res == (0, 0, 1) + def test_call_method_uses_cache(self): + # bit sucky + global C + + class C(object): + def m(*args): + return args + C.sm = staticmethod(C.m.im_func) + C.cm = classmethod(C.m.im_func) + + exec """if 1: + + def f(): + c = C() + res = c.m(1) + assert res == (c, 1) + return 42 + + def g(): + c = C() + res = c.sm(1) + assert res == (1, ) + return 42 + + def h(): + c = C() + res = c.cm(1) + assert res == (C, 1) + return 42 + """ + res = self.check(f, 'm') + assert res == (1, 0, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + + # static methods are not cached + res = self.check(g, 'sm') + assert res == (0, 0, 0) + res = self.check(g, 'sm') + assert res == (0, 0, 0) + + # neither are class methods + res = self.check(h, 'cm') + assert res == (0, 0, 0) + res = self.check(h, 'cm') + assert res == (0, 0, 0) + + def test_mix_cache_bug(self): + # bit sucky + global C + + class C(object): + def m(*args): + return args + + exec """if 1: + + def f(): + c = C() + res = c.m(1) + assert res == (c, 1) + bm = c.m + res = bm(1) + assert res == (c, 1) + return 42 + + """ + res = self.check(f, 'm') + assert res == (1, 1, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + +class AppTestGlobalCaching(AppTestWithMapDict): + def setup_class(cls): + cls.space = gettestobjspace( + **{"objspace.std.withmethodcachecounter": True, + "objspace.std.withmapdict": True, + "objspace.opcodes.CALL_METHOD": True}) + + def test_mix_classes(self): + import __pypy__ + class A(object): + def f(self): + return 42 + class B(object): + def f(self): + return 43 + class C(object): + def f(self): + return 44 + l = [A(), B(), C()] * 10 + __pypy__.reset_method_cache_counter() + # 'exec' to make sure that a.f() is compiled with CALL_METHOD + exec """for i, a in enumerate(l): + assert a.f() == 42 + i % 3 +""" + cache_counter = __pypy__.mapdict_cache_counter("f") + assert cache_counter[0] >= 15 + assert cache_counter[1] >= 3 # should be (27, 3) + assert sum(cache_counter) == 30 + + def test_mix_classes_attribute(self): + import __pypy__ + class A(object): + def __init__(self): + self.x = 42 + class B(object): + def __init__(self): + self.x = 43 + class C(object): + def __init__(self): + self.x = 44 + l = [A(), B(), C()] * 10 + __pypy__.reset_method_cache_counter() + for i, a in enumerate(l): + assert a.x == 42 + i % 3 + cache_counter = __pypy__.mapdict_cache_counter("x") + assert cache_counter[0] >= 15 + assert cache_counter[1] >= 3 # should be (27, 3) + assert sum(cache_counter) == 30 + class TestDictSubclassShortcutBug(object): def setup_class(cls): cls.space = gettestobjspace( Modified: pypy/branch/jit-unroll-loops/pypy/rlib/clibffi.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rlib/clibffi.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rlib/clibffi.py Wed Nov 17 20:41:25 2010 @@ -432,7 +432,8 @@ flags=FUNCFLAG_CDECL): AbstractFuncPtr.__init__(self, "callback", argtypes, restype, flags) self.ll_closure = closureHeap.alloc() - self.ll_userdata = lltype.malloc(USERDATA_P.TO, flavor='raw') + self.ll_userdata = lltype.malloc(USERDATA_P.TO, flavor='raw', + track_allocation=False) self.ll_userdata.callback = rffi.llhelper(CALLBACK_TP, func) self.ll_userdata.addarg = additional_arg res = c_ffi_prep_closure(self.ll_closure, self.ll_cif, @@ -447,7 +448,7 @@ closureHeap.free(self.ll_closure) self.ll_closure = lltype.nullptr(FFI_CLOSUREP.TO) if self.ll_userdata: - lltype.free(self.ll_userdata, flavor='raw') + lltype.free(self.ll_userdata, flavor='raw', track_allocation=False) self.ll_userdata = lltype.nullptr(USERDATA_P.TO) class RawFuncPtr(AbstractFuncPtr): Modified: pypy/branch/jit-unroll-loops/pypy/rlib/jit.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rlib/jit.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rlib/jit.py Wed Nov 17 20:41:25 2010 @@ -4,6 +4,7 @@ from pypy.rlib.objectmodel import CDefinedIntSymbolic from pypy.rlib.objectmodel import keepalive_until_here from pypy.rlib.unroll import unrolling_iterable +from pypy.rlib.nonconst import NonConstant def purefunction(func): """ Decorate a function as pure. Pure means precisely that: @@ -145,6 +146,14 @@ return hop.inputconst(lltype.Signed, _we_are_jitted) +def current_trace_length(): + """During JIT tracing, returns the current trace length (as a constant). + If not tracing, returns -1.""" + if NonConstant(False): + return 73 + return -1 +current_trace_length.oopspec = 'jit.current_trace_length()' + def jit_debug(string, arg1=-sys.maxint-1, arg2=-sys.maxint-1, arg3=-sys.maxint-1, arg4=-sys.maxint-1): """When JITted, cause an extra operation DEBUG_MERGE_POINT to appear in Modified: pypy/branch/jit-unroll-loops/pypy/rlib/rarithmetic.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rlib/rarithmetic.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rlib/rarithmetic.py Wed Nov 17 20:41:25 2010 @@ -501,6 +501,11 @@ def __cmp__(self, other): raise TypeError("not supported on r_singlefloat instances") + def __eq__(self, other): + return self.__class__ is other.__class__ and self._bytes == other._bytes + + def __ne__(self, other): + return not self.__eq__(other) class For_r_singlefloat_values_Entry(extregistry.ExtRegistryEntry): _type_ = r_singlefloat Modified: pypy/branch/jit-unroll-loops/pypy/rlib/rgc.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rlib/rgc.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rlib/rgc.py Wed Nov 17 20:41:25 2010 @@ -379,6 +379,11 @@ "NOT_RPYTHON" raise NotImplementedError +def get_typeids_z(): + "NOT_RPYTHON" + raise NotImplementedError + +ARRAY_OF_CHAR = lltype.Array(lltype.Char) NULL_GCREF = lltype.nullptr(llmemory.GCREF.TO) class _GcRef(object): @@ -530,3 +535,12 @@ vlist = hop.inputargs(lltype.Signed) hop.exception_is_here() return hop.genop('gc_dump_rpy_heap', vlist, resulttype = hop.r_result) + +class Entry(ExtRegistryEntry): + _about_ = get_typeids_z + def compute_result_annotation(self): + from pypy.annotation.model import SomePtr + return SomePtr(lltype.Ptr(ARRAY_OF_CHAR)) + def specialize_call(self, hop): + hop.exception_is_here() + return hop.genop('gc_typeids_z', [], resulttype = hop.r_result) Modified: pypy/branch/jit-unroll-loops/pypy/rlib/streamio.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rlib/streamio.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rlib/streamio.py Wed Nov 17 20:41:25 2010 @@ -12,7 +12,7 @@ * some other methods also have no default parameters. * close() should be called exactly once and no further operations performed; there is no __del__() closing the stream for you. - * some methods may raise NotImplementedError. + * some methods may raise MyNotImplementedError. * peek() returns some (or no) characters that have already been read ahead. * flushable() returns True/False if flushing that stream is useful/pointless. @@ -54,6 +54,12 @@ ('a', True): O_RDWR | O_CREAT, } +class MyNotImplementedError(Exception): + """ + Catching NotImplementedError is not RPython, so we use this custom class + instead of it + """ + # ____________________________________________________________ @@ -209,16 +215,16 @@ some methods.""" def read(self, n): - raise NotImplementedError + raise MyNotImplementedError def write(self, data): - raise NotImplementedError + raise MyNotImplementedError def tell(self): - raise NotImplementedError + raise MyNotImplementedError def seek(self, offset, whence): - raise NotImplementedError + raise MyNotImplementedError def readall(self): bufsize = 8192 @@ -251,7 +257,7 @@ return ''.join(result) def truncate(self, size): - raise NotImplementedError + raise MyNotImplementedError def flush_buffers(self): pass @@ -487,7 +493,7 @@ if self.lines or self.buf: try: self.do_seek(self.tell(), 0) - except NotImplementedError: + except MyNotImplementedError: pass else: self.lines = [] @@ -535,14 +541,14 @@ self.buf = "" try: self.do_seek(offset, 1) - except NotImplementedError: + except MyNotImplementedError: intoffset = offset2int(offset) self.read(intoffset) return if whence == 2: try: self.do_seek(offset, 2) - except NotImplementedError: + except MyNotImplementedError: pass else: self.lines = [] @@ -1019,7 +1025,7 @@ if self.buf: try: self.base.seek(-len(self.buf), 1) - except NotImplementedError: + except MyNotImplementedError: pass else: self.buf = "" Modified: pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rarithmetic.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rarithmetic.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rlib/test/test_rarithmetic.py Wed Nov 17 20:41:25 2010 @@ -325,6 +325,15 @@ assert float(x) != 2.1 assert abs(float(x) - 2.1) < 1E-6 +def test_r_singlefloat_eq(): + x = r_singlefloat(2.5) # exact number + y = r_singlefloat(2.5) + assert x == y + assert not x != y + assert not x == 2.5 + assert x != 2.5 + py.test.raises(TypeError, "x>y") + class BaseTestRarithmetic(BaseRtypingTest): def test_formatd(self): from pypy.rlib.rarithmetic import formatd @@ -358,7 +367,17 @@ assert res == 1.0 res = self.interpret(f, [1]) - assert res == 1e-100 + assert res == 1e-100 + + def test_compare_singlefloat_crashes(self): + from pypy.rlib.rarithmetic import r_singlefloat + from pypy.rpython.error import MissingRTypeOperation + def f(x): + a = r_singlefloat(x) + b = r_singlefloat(x+1) + return a == b + py.test.raises(MissingRTypeOperation, "self.interpret(f, [42.0])") + class TestLLtype(BaseTestRarithmetic, LLRtypeMixin): pass Modified: pypy/branch/jit-unroll-loops/pypy/rpython/annlowlevel.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/annlowlevel.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/annlowlevel.py Wed Nov 17 20:41:25 2010 @@ -136,7 +136,7 @@ return funcdesc.cachedgraph(TYPE, alt_name=valid_identifier(alt_name)) -class MixLevelHelperAnnotator: +class MixLevelHelperAnnotator(object): def __init__(self, rtyper): self.rtyper = rtyper Modified: pypy/branch/jit-unroll-loops/pypy/rpython/llinterp.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/llinterp.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/llinterp.py Wed Nov 17 20:41:25 2010 @@ -912,6 +912,9 @@ def op_gc_dump_rpy_heap(self): raise NotImplementedError("gc_dump_rpy_heap") + def op_gc_typeids_z(self): + raise NotImplementedError("gc_typeids_z") + def op_do_malloc_fixedsize_clear(self): raise NotImplementedError("do_malloc_fixedsize_clear") Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llarena.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llarena.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llarena.py Wed Nov 17 20:41:25 2010 @@ -440,7 +440,8 @@ [rffi.INT], rffi.INT, sandboxsafe=True, _nowrapper=True) - _dev_zero = rffi.str2charp_immortal('/dev/zero') # prebuilt + _dev_zero = rffi.str2charp('/dev/zero') # prebuilt + lltype.render_immortal(_dev_zero) def clear_large_memory_chunk(baseaddr, size): # on some Unixy platforms, reading from /dev/zero is the fastest way Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/llmemory.py Wed Nov 17 20:41:25 2010 @@ -766,8 +766,10 @@ return '<%s>' % (self,) def __str__(self): return 'gctransformed_wref(%s)' % (self._ptr,) - def _normalizedcontainer(self): - return self._ptr._obj + def _normalizedcontainer(self, check=True): + return self._ptr._getobj(check=check)._normalizedcontainer(check=check) + def _was_freed(self): + return self._ptr._was_freed() # ____________________________________________________________ Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lloperation.py Wed Nov 17 20:41:25 2010 @@ -476,6 +476,7 @@ 'gc_get_rpy_type_index': LLOp(), 'gc_is_rpy_instance' : LLOp(), 'gc_dump_rpy_heap' : LLOp(), + 'gc_typeids_z' : LLOp(), # ------- JIT & GC interaction, only for some GCs ---------- Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/lltype.py Wed Nov 17 20:41:25 2010 @@ -1868,6 +1868,13 @@ leakfinder.remember_free(p._obj0) p._obj0._free() +def render_immortal(p, track_allocation=True): + T = typeOf(p) + if not isinstance(T, Ptr) or p._togckind() != 'raw': + raise TypeError, "free(): only for pointers to non-gc containers" + if track_allocation: + leakfinder.remember_free(p._obj0) + def _make_scoped_allocator(T): class ScopedAlloc: def __init__(self, n=None, zero=False): Modified: pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rffi.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rffi.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/lltypesystem/rffi.py Wed Nov 17 20:41:25 2010 @@ -607,15 +607,6 @@ return array str2charp._annenforceargs_ = [strtype] - def str2charp_immortal(s): - "NOT_RPYTHON" - array = lltype.malloc(TYPEP.TO, len(s) + 1, flavor='raw', - immortal=True) - for i in range(len(s)): - array[i] = s[i] - array[len(s)] = lastchar - return array - def free_charp(cp): lltype.free(cp, flavor='raw') @@ -734,19 +725,19 @@ l = [cp[i] for i in range(size)] return emptystr.join(l) - return (str2charp, str2charp_immortal, free_charp, charp2str, + return (str2charp, free_charp, charp2str, get_nonmovingbuffer, free_nonmovingbuffer, alloc_buffer, str_from_buffer, keep_buffer_alive_until_here, charp2strn, charpsize2str, ) -(str2charp, str2charp_immortal, free_charp, charp2str, +(str2charp, free_charp, charp2str, get_nonmovingbuffer, free_nonmovingbuffer, alloc_buffer, str_from_buffer, keep_buffer_alive_until_here, charp2strn, charpsize2str, ) = make_string_mappings(str) -(unicode2wcharp, unicode2wcharp_immortal, free_wcharp, wcharp2unicode, +(unicode2wcharp, free_wcharp, wcharp2unicode, get_nonmoving_unicodebuffer, free_nonmoving_unicodebuffer, alloc_unicodebuffer, unicode_from_buffer, keep_unicodebuffer_alive_until_here, wcharp2unicoden, wcharpsize2unicode, Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/base.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/base.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/base.py Wed Nov 17 20:41:25 2010 @@ -247,6 +247,21 @@ (not self.config.taggedpointers or llmemory.cast_adr_to_int(addr) & 1 == 0)) + def enumerate_all_roots(self, callback, arg): + """For each root object, invoke callback(obj, arg). + 'callback' should not be a bound method. + Note that this method is not suitable for actually doing the + collection in a moving GC, because you cannot write back a + modified address. It is there only for inspection. + """ + # overridden in some subclasses, for GCs which have an additional + # list of last generation roots + callback2, attrname = _convert_callback_formats(callback) # :-/ + setattr(self, attrname, arg) + self.root_walker.walk_roots(callback2, callback2, callback2) + self.run_finalizers.foreach(callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' + def debug_check_consistency(self): """To use after a collection. If self.DEBUG is set, this enumerates all roots and traces all objects to check if we didn't @@ -260,8 +275,7 @@ self._debug_pending = self.AddressStack() if not we_are_translated(): self.root_walker._walk_prebuilt_gc(self._debug_record) - callback = GCBase._debug_callback - self.root_walker.walk_roots(callback, callback, callback) + self.enumerate_all_roots(GCBase._debug_callback, self) pending = self._debug_pending while pending.non_empty(): obj = pending.pop() @@ -275,9 +289,8 @@ seen.add(obj) self.debug_check_object(obj) self._debug_pending.append(obj) - def _debug_callback(self, root): - obj = root.address[0] - ll_assert(bool(obj), "NULL address from walk_roots()") + @staticmethod + def _debug_callback(obj, self): self._debug_record(obj) def _debug_callback2(self, pointer, ignored): obj = pointer.address[0] @@ -432,3 +445,17 @@ if factor != 1: return 0.0 return value + +def _convert_callback_formats(callback): + callback = getattr(callback, 'im_func', callback) + if callback not in _converted_callback_formats: + def callback2(gc, root): + obj = root.address[0] + ll_assert(bool(obj), "NULL address from walk_roots()") + callback(obj, getattr(gc, attrname)) + attrname = '_callback2_arg%d' % len(_converted_callback_formats) + _converted_callback_formats[callback] = callback2, attrname + return _converted_callback_formats[callback] + +_convert_callback_formats._annspecialcase_ = 'specialize:memo' +_converted_callback_formats = {} Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/generation.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/generation.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/generation.py Wed Nov 17 20:41:25 2010 @@ -572,16 +572,10 @@ def _compute_current_nursery_hash(self, obj): return intmask(llmemory.cast_adr_to_int(obj) + self.nursery_hash_base) - def heap_stats_walk_roots(self): - self.last_generation_root_objects.foreach( - self._track_heap_ext, None) - self.root_walker.walk_roots( - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root) - - def _track_heap_ext(self, adr, ignored): - self.trace(adr, self.track_heap_parent, adr) + def enumerate_all_roots(self, callback, arg): + self.last_generation_root_objects.foreach(callback, arg) + SemiSpaceGC.enumerate_all_roots(self, callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' def debug_check_object(self, obj): """Check the invariants about 'obj' that should be true Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/inspector.py Wed Nov 17 20:41:25 2010 @@ -4,25 +4,22 @@ from pypy.rpython.lltypesystem import lltype, llmemory, rffi from pypy.rlib.objectmodel import free_non_gc_object from pypy.rpython.module.ll_os import underscore_on_windows -from pypy.rlib import rposix +from pypy.rlib import rposix, rgc from pypy.rpython.memory.support import AddressDict, get_address_stack # ---------- implementation of pypy.rlib.rgc.get_rpy_roots() ---------- -def _counting_rpy_root(gc, root): +def _counting_rpy_root(obj, gc): gc._count_rpy += 1 def _do_count_rpy_roots(gc): gc._count_rpy = 0 - gc.root_walker.walk_roots( - _counting_rpy_root, - _counting_rpy_root, - _counting_rpy_root) + gc.enumerate_all_roots(_counting_rpy_root, gc) return gc._count_rpy -def _append_rpy_root(gc, root): +def _append_rpy_root(obj, gc): # Can use the gc list, but should not allocate! # It is essential that the list is not resizable! lst = gc._list_rpy @@ -30,15 +27,12 @@ if index >= len(lst): raise ValueError gc._count_rpy = index + 1 - lst[index] = llmemory.cast_adr_to_ptr(root.address[0], llmemory.GCREF) + lst[index] = llmemory.cast_adr_to_ptr(obj, llmemory.GCREF) def _do_append_rpy_roots(gc, lst): gc._count_rpy = 0 gc._list_rpy = lst - gc.root_walker.walk_roots( - _append_rpy_root, - _append_rpy_root, - _append_rpy_root) + gc.enumerate_all_roots(_append_rpy_root, gc) gc._list_rpy = None def get_rpy_roots(gc): @@ -172,12 +166,7 @@ self.pending.append(obj) def add_roots(self): - self.gc._heap_dumper = self - self.gc.root_walker.walk_roots( - _hd_add_root, - _hd_add_root, - _hd_add_root) - self.gc._heap_dumper = None + self.gc.enumerate_all_roots(_hd_add_root, self) pendingroots = self.pending self.pending = AddressStack() self.walk(pendingroots) @@ -188,8 +177,8 @@ while pending.non_empty(): self.writeobj(pending.pop()) -def _hd_add_root(gc, root): - gc._heap_dumper.add(root.address[0]) +def _hd_add_root(obj, heap_dumper): + heap_dumper.add(obj) def dump_rpy_heap(gc, fd): heapdumper = HeapDumper(gc, fd) @@ -198,3 +187,7 @@ heapdumper.flush() heapdumper.delete() return True + +def get_typeids_z(gc): + srcaddress = gc.root_walker.gcdata.typeids_z + return llmemory.cast_adr_to_ptr(srcaddress, lltype.Ptr(rgc.ARRAY_OF_CHAR)) Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/minimark.py Wed Nov 17 20:41:25 2010 @@ -1,3 +1,36 @@ +""" MiniMark GC. + +Environment variables can be used to fine-tune the following parameters: + + PYPY_GC_NURSERY The nursery size. Defaults to half the size of + the L2 cache. Try values like '1.2MB'. + + PYPY_GC_MAJOR_COLLECT Major collection memory factor. Default is '1.82', + which means trigger a major collection when the + memory consumed equals 1.82 times the memory + really used at the end of the previous major + collection. + + PYPY_GC_GROWTH Major collection threshold's max growth rate. + Default is '1.3'. Useful to collect more often + than normally on sudden memory growth, e.g. when + there is a temporary peak in memory usage. + + PYPY_GC_MAX The max heap size. If coming near this limit, it + will first collect more often, then raise an + RPython MemoryError, and if that is not enough, + crash the program with a fatal error. Try values + like '1.6GB'. + + PYPY_GC_MIN Don't collect while the memory size is below this + limit. Useful to avoid spending all the time in + the GC in very small programs. Defaults to 8 + times the nursery. +""" +# XXX Should find a way to bound the major collection threshold by the +# XXX total addressable size. Maybe by keeping some minimarkpage arenas +# XXX pre-reserved, enough for a few nursery collections? What about +# XXX raw-malloced memory? import sys from pypy.rpython.lltypesystem import lltype, llmemory, llarena, llgroup from pypy.rpython.lltypesystem.lloperation import llop @@ -77,7 +110,7 @@ # During a minor collection, the objects in the nursery that are # moved outside are changed in-place: their header is replaced with - # the value -1, and the following word is set to the address of + # the value -42, and the following word is set to the address of # where the object was moved. This means that all objects in the # nursery need to be at least 2 words long, but objects outside the # nursery don't need to. @@ -87,13 +120,8 @@ TRANSLATION_PARAMS = { # Automatically adjust the size of the nursery and the - # 'major_collection_threshold' from the environment. For - # 'nursery_size' it will look it up in the env var - # PYPY_GC_NURSERY and fall back to half the size of - # the L2 cache. For 'major_collection_threshold' it will look - # it up in the env var PYPY_GC_MAJOR_COLLECT. It also sets - # 'max_heap_size' to PYPY_GC_MAX. Finally, PYPY_GC_MIN sets - # the minimal value of 'next_major_collection_threshold'. + # 'major_collection_threshold' from the environment. + # See docstring at the start of the file. "read_from_env": True, # The size of the nursery. Note that this is only used as a @@ -121,6 +149,13 @@ # we trigger the next major collection. "major_collection_threshold": 1.82, + # Threshold to avoid that the total heap size grows by a factor of + # major_collection_threshold at every collection: it can only + # grow at most by the following factor from one collection to the + # next. Used e.g. when there is a sudden, temporary peak in memory + # usage; this avoids that the upper bound grows too fast. + "growth_rate_max": 1.3, + # The number of array indices that are mapped to a single bit in # write_barrier_from_array(). Must be a power of two. The default # value of 128 means that card pages are 512 bytes (1024 on 64-bits) @@ -146,6 +181,7 @@ arena_size=64*WORD, small_request_threshold=5*WORD, major_collection_threshold=2.5, + growth_rate_max=2.5, # for tests card_page_indices=0, large_object=8*WORD, large_object_gcptrs=10*WORD, @@ -157,6 +193,7 @@ self.nursery_size = nursery_size self.small_request_threshold = small_request_threshold self.major_collection_threshold = major_collection_threshold + self.growth_rate_max = growth_rate_max self.num_major_collects = 0 self.min_heap_size = 0.0 self.max_heap_size = 0.0 @@ -178,6 +215,7 @@ self.nursery = NULL self.nursery_free = NULL self.nursery_top = NULL + self.debug_always_do_minor_collect = False # # The ArenaCollection() handles the nonmovable objects allocation. if ArenaCollectionClass is None: @@ -245,6 +283,10 @@ # From there on, the GC is fully initialized and the code # below can use it newsize = base.read_from_env('PYPY_GC_NURSERY') + # PYPY_GC_NURSERY=1 forces a minor collect for every malloc. + # Useful to debug external factors, like trackgcroot or the + # handling of the write barrier. + self.debug_always_do_minor_collect = newsize == 1 if newsize <= 0: newsize = generation.estimate_best_nursery_size() if newsize <= 0: @@ -252,9 +294,13 @@ newsize = max(newsize, minsize) # major_coll = base.read_float_from_env('PYPY_GC_MAJOR_COLLECT') - if major_coll >= 1.0: + if major_coll > 1.0: self.major_collection_threshold = major_coll # + growth = base.read_float_from_env('PYPY_GC_GROWTH') + if growth > 1.0: + self.growth_rate_max = growth + # min_heap_size = base.read_uint_from_env('PYPY_GC_MIN') if min_heap_size > 0: self.min_heap_size = float(min_heap_size) @@ -290,11 +336,19 @@ # initialize the threshold self.min_heap_size = max(self.min_heap_size, self.nursery_size * self.major_collection_threshold) + self.next_major_collection_threshold = self.min_heap_size self.set_major_threshold_from(0.0) debug_stop("gc-set-nursery-size") - def set_major_threshold_from(self, threshold): + + def set_major_threshold_from(self, threshold, reserving_size=0): # Set the next_major_collection_threshold. + threshold_max = (self.next_major_collection_threshold * + self.growth_rate_max) + if threshold > threshold_max: + threshold = threshold_max + # + threshold += reserving_size if threshold < self.min_heap_size: threshold = self.min_heap_size # @@ -444,6 +498,10 @@ result = self.nursery_free self.nursery_free = result + totalsize ll_assert(self.nursery_free <= self.nursery_top, "nursery overflow") + # + if self.debug_always_do_minor_collect: + self.nursery_free = self.nursery_top + # return result collect_and_reserve._dont_inline_ = True @@ -647,10 +705,13 @@ def is_forwarded(self, obj): """Returns True if the nursery obj is marked as forwarded. Implemented a bit obscurely by checking an unrelated flag - that can never be set on a young object -- except if tid == -1. + that can never be set on a young object -- except if tid == -42. """ assert self.is_in_nursery(obj) - return self.header(obj).tid & GCFLAG_FINALIZATION_ORDERING + result = (self.header(obj).tid & GCFLAG_FINALIZATION_ORDERING != 0) + if result: + ll_assert(self.header(obj).tid == -42, "bogus header for young obj") + return result def get_forwarding_address(self, obj): return llmemory.cast_adr_to_ptr(obj, FORWARDSTUBPTR).forw @@ -734,6 +795,10 @@ def JIT_max_size_of_young_obj(cls): return cls.TRANSLATION_PARAMS['large_object'] + @classmethod + def JIT_minimal_size_in_nursery(cls): + return cls.minimal_size_in_nursery + def write_barrier(self, newvalue, addr_struct): if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS: self.remember_young_pointer(addr_struct, newvalue) @@ -906,7 +971,7 @@ # # Now all live nursery objects should be out. Update the # young weakrefs' targets. - if self.young_objects_with_weakrefs.length() > 0: + if self.young_objects_with_weakrefs.non_empty(): self.invalidate_young_weakrefs() # # Clear this mapping. @@ -1000,7 +1065,7 @@ obj = oldlist.pop() # # Add the flag GCFLAG_NO_YOUNG_PTRS. All live objects should have - # this flag after a nursery collection. + # this flag set after a nursery collection. self.header(obj).tid |= GCFLAG_NO_YOUNG_PTRS # # Trace the 'obj' to replace pointers to nursery with pointers @@ -1063,7 +1128,7 @@ # nursery are kept unchanged in this step. llmemory.raw_memcopy(obj - size_gc_header, newhdr, totalsize) # - # Set the old object's tid to -1 (containing all flags) and + # Set the old object's tid to -42 (containing all flags) and # replace the old object's content with the target address. # A bit of no-ops to convince llarena that we are changing # the layout, in non-translated versions. @@ -1071,7 +1136,7 @@ llarena.arena_reset(obj - size_gc_header, totalsize, 0) llarena.arena_reserve(obj - size_gc_header, size_gc_header + llmemory.sizeof(FORWARDSTUB)) - self.header(obj).tid = -1 + self.header(obj).tid = -42 newobj = newhdr + size_gc_header llmemory.cast_adr_to_ptr(obj, FORWARDSTUBPTR).forw = newobj # @@ -1140,6 +1205,10 @@ self.collect_roots() self.visit_all_objects() # + # Weakref support: clear the weak pointers to dying objects + if self.old_objects_with_weakrefs.non_empty(): + self.invalidate_old_weakrefs() + # # Finalizer support: adds the flag GCFLAG_VISITED to all objects # with a finalizer and all objects reachable from there (and also # moves some objects from 'objects_with_finalizers' to @@ -1149,10 +1218,6 @@ # self.objects_to_trace.delete() # - # Weakref support: clear the weak pointers to dying objects - if self.old_objects_with_weakrefs.non_empty(): - self.invalidate_old_weakrefs() - # # Walk all rawmalloced objects and free the ones that don't # have the GCFLAG_VISITED flag. self.free_unvisited_rawmalloc_objects() @@ -1182,8 +1247,8 @@ # have allocated 'major_collection_threshold' times more than # we currently have. bounded = self.set_major_threshold_from( - (self.get_total_memory_used() * self.major_collection_threshold) - + reserving_size) + self.get_total_memory_used() * self.major_collection_threshold, + reserving_size) # # Max heap size: gives an upper bound on the threshold. If we # already have at least this much allocated, raise MemoryError. @@ -1272,6 +1337,11 @@ self.run_finalizers.foreach(self._collect_obj, self.objects_to_trace) + def enumerate_all_roots(self, callback, arg): + self.prebuilt_root_objects.foreach(callback, arg) + MovingGCBase.enumerate_all_roots(self, callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' + @staticmethod def _collect_obj(obj, objects_to_trace): objects_to_trace.append(obj) @@ -1323,7 +1393,7 @@ if self.is_valid_gc_object(obj): if self.is_in_nursery(obj): # - # The object not a tagged pointer, and is it still in the + # The object is not a tagged pointer, and it is still in the # nursery. Find or allocate a "shadow" object, which is # where the object will be moved by the next minor # collection Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/semispace.py Wed Nov 17 20:41:25 2010 @@ -230,6 +230,10 @@ while self.max_space_size > size: self.max_space_size >>= 1 + @classmethod + def JIT_minimal_size_in_nursery(cls): + return cls.object_minimal_size + def collect(self, gen=0): self.debug_check_consistency() self.semispace_collect() @@ -262,10 +266,10 @@ if self.run_finalizers.non_empty(): self.update_run_finalizers() scan = self.scan_copied(scan) - if self.objects_with_finalizers.non_empty(): - scan = self.deal_with_objects_with_finalizers(scan) if self.objects_with_weakrefs.non_empty(): self.invalidate_weakrefs() + if self.objects_with_finalizers.non_empty(): + scan = self.deal_with_objects_with_finalizers(scan) self.update_objects_with_id() self.finished_full_collect() self.debug_check_consistency() @@ -689,15 +693,10 @@ self._ll_typeid_map[idx].size += llmemory.raw_malloc_usage(totsize) self.trace(adr, self.track_heap_parent, adr) - def _track_heap_root(self, root): - self.track_heap(root.address[0]) + @staticmethod + def _track_heap_root(obj, self): + self.track_heap(obj) - def heap_stats_walk_roots(self): - self.root_walker.walk_roots( - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root) - def heap_stats(self): self._tracked_dict = self.AddressDict() max_tid = self.root_walker.gcdata.max_type_id @@ -710,7 +709,7 @@ while i < max_tid: self._tracked_dict.add(llmemory.cast_ptr_to_adr(ll_typeid_map[i])) i += 1 - self.heap_stats_walk_roots() + self.enumerate_all_roots(SemiSpaceGC._track_heap_root, self) self._ll_typeid_map = lltype.nullptr(ARRAY_TYPEID_MAP) self._tracked_dict.delete() return ll_typeid_map Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/test/test_minimark.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/test/test_minimark.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gc/test/test_minimark.py Wed Nov 17 20:41:25 2010 @@ -28,3 +28,42 @@ assert gc.card_marking_bytes_for_length(P+P+1) == 3 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P) == 8 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P+1) == 9 + +def test_set_major_threshold(): + gc = MiniMarkGC(None, major_collection_threshold=2.0, + growth_rate_max=1.5) + gc.min_heap_size = 100.0 + gc.max_heap_size = 300.0 + gc.next_major_collection_threshold = 0.0 + # first, we don't grow past min_heap_size + for i in range(5): + gc.set_major_threshold_from(100.0) + assert gc.next_major_collection_threshold == 100.0 + # then we grow a lot + b = gc.set_major_threshold_from(100 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 150.0 + b = gc.set_major_threshold_from(150 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 225.0 + b = gc.set_major_threshold_from(225 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 # max reached + b = gc.set_major_threshold_from(300 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 + # then we shrink instantly + b = gc.set_major_threshold_from(100.0) + assert b is False + assert gc.next_major_collection_threshold == 100.0 + # then we grow a bit + b = gc.set_major_threshold_from(100 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 125.0 + b = gc.set_major_threshold_from(125 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 156.25 + # check that we cannot shrink below min_heap_size + b = gc.set_major_threshold_from(42.7) + assert b is False + assert gc.next_major_collection_threshold == 100.0 Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/framework.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/framework.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/gctransform/framework.py Wed Nov 17 20:41:25 2010 @@ -172,6 +172,7 @@ gcdata.static_root_nongcend = a_random_address # patched in finish() gcdata.static_root_end = a_random_address # patched in finish() gcdata.max_type_id = 13 # patched in finish() + gcdata.typeids_z = a_random_address # patched in finish() self.gcdata = gcdata self.malloc_fnptr_cache = {} @@ -212,6 +213,9 @@ data_classdef.generalize_attr( 'max_type_id', annmodel.SomeInteger()) + data_classdef.generalize_attr( + 'typeids_z', + annmodel.SomeAddress()) annhelper = annlowlevel.MixLevelHelperAnnotator(self.translator.rtyper) @@ -415,6 +419,11 @@ [s_gc, annmodel.SomeInteger()], annmodel.s_Bool, minimal_transform=False) + self.get_typeids_z_ptr = getfn(inspector.get_typeids_z, + [s_gc], + annmodel.SomePtr( + lltype.Ptr(rgc.ARRAY_OF_CHAR)), + minimal_transform=False) self.set_max_heap_size_ptr = getfn(GCClass.set_max_heap_size.im_func, [s_gc, @@ -572,7 +581,14 @@ newgcdependencies = [] newgcdependencies.append(ll_static_roots_inside) ll_instance.inst_max_type_id = len(group.members) - self.write_typeid_list() + typeids_z = self.write_typeid_list() + ll_typeids_z = lltype.malloc(rgc.ARRAY_OF_CHAR, + len(typeids_z), + immortal=True) + for i in range(len(typeids_z)): + ll_typeids_z[i] = typeids_z[i] + ll_instance.inst_typeids_z = llmemory.cast_ptr_to_adr(ll_typeids_z) + newgcdependencies.append(ll_typeids_z) return newgcdependencies def get_finish_tables(self): @@ -599,6 +615,11 @@ for index in range(len(self.layoutbuilder.type_info_group.members)): f.write("member%-4d %s\n" % (index, all_ids.get(index, '?'))) f.close() + try: + import zlib + return zlib.compress(udir.join("typeids.txt").read(), 9) + except ImportError: + return '' def transform_graph(self, graph): func = getattr(graph, 'func', None) @@ -988,6 +1009,13 @@ resultvar=hop.spaceop.result) self.pop_roots(hop, livevars) + def gct_gc_typeids_z(self, hop): + livevars = self.push_roots(hop) + hop.genop("direct_call", + [self.get_typeids_z_ptr, self.c_const_gc], + resultvar=hop.spaceop.result) + self.pop_roots(hop, livevars) + def gct_malloc_nonmovable_varsize(self, hop): TYPE = hop.spaceop.result.concretetype if self.gcdata.gc.can_malloc_nonmovable(): @@ -1210,7 +1238,7 @@ sizeofaddr = llmemory.sizeof(llmemory.Address) -class BaseRootWalker: +class BaseRootWalker(object): need_root_stack = False def __init__(self, gctransformer): Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/support.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/memory/support.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/support.py Wed Nov 17 20:41:25 2010 @@ -112,7 +112,7 @@ cur = next free_non_gc_object(self) - def length(self): + def _length_estimate(self): chunk = self.chunk count = self.used_in_last_chunk while chunk: @@ -135,7 +135,7 @@ foreach._annspecialcase_ = 'specialize:arg(1)' def stack2dict(self): - result = AddressDict(self.length()) + result = AddressDict(self._length_estimate()) self.foreach(_add_in_dict, result) return result Modified: pypy/branch/jit-unroll-loops/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/memory/test/test_gc.py Wed Nov 17 20:41:25 2010 @@ -278,6 +278,80 @@ res = self.interpret(f, []) assert res + def test_bug_1(self): + import weakref + class B(object): + pass + def g(): + b = B() + llop.gc__collect(lltype.Void) # force 'b' to be old + ref = weakref.ref(B()) + b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = (ref() is None) + return result + res = self.interpret(f, []) + assert res + + def test_cycle_with_weakref_and_del(self): + import weakref, gc + class A(object): + count = 0 + a = A() + class B(object): + def __del__(self): + # when __del__ is called, the weakref should have been cleared + if self.ref() is None: + a.count += 10 # ok + else: + a.count = 666 # not ok + class C(object): + pass + def g(): + c = C() + c.b = B() + ref = weakref.ref(c) + c.b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = a.count + (ref() is None) + return result + res = self.interpret(f, []) + assert res == 11 + + def test_weakref_to_object_with_finalizer_ordering(self): + import weakref, gc + class A(object): + count = 0 + a = A() + class B(object): + def __del__(self): + # when __del__ is called, the weakref should have been cleared + if self.ref() is None: + a.count += 10 # ok + else: + a.count = 666 # not ok + def g(): + b = B() + ref = weakref.ref(b) + b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = a.count + (ref() is None) + return result + res = self.interpret(f, []) + assert res == 11 + def test_id(self): class A(object): pass @@ -657,6 +731,9 @@ def test_finalizer_order(self): py.test.skip("Not implemented yet") + def test_weakref_to_object_with_finalizer_ordering(self): + py.test.skip("Not implemented yet") + class TestHybridGC(TestGenerationalGC): from pypy.rpython.memory.gc.hybrid import HybridGC as GCClass GC_CAN_MALLOC_NONMOVABLE = True Modified: pypy/branch/jit-unroll-loops/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/rbuiltin.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/rbuiltin.py Wed Nov 17 20:41:25 2010 @@ -386,6 +386,14 @@ hop.exception_cannot_occur() hop.genop('free', vlist) +def rtype_render_immortal(hop, i_track_allocation=None): + vlist = [hop.inputarg(hop.args_r[0], arg=0)] + v_track_allocation = parse_kwds(hop, + (i_track_allocation, None)) + hop.exception_cannot_occur() + if i_track_allocation is None or v_track_allocation.value: + hop.genop('track_alloc_stop', vlist) + def rtype_const_result(hop): hop.exception_cannot_occur() return hop.inputconst(hop.r_result.lowleveltype, hop.s_result.const) @@ -523,6 +531,7 @@ BUILTIN_TYPER[lltype.malloc] = rtype_malloc BUILTIN_TYPER[lltype.free] = rtype_free +BUILTIN_TYPER[lltype.render_immortal] = rtype_render_immortal BUILTIN_TYPER[lltype.cast_primitive] = rtype_cast_primitive BUILTIN_TYPER[lltype.cast_pointer] = rtype_cast_pointer BUILTIN_TYPER[lltype.cast_opaque_ptr] = rtype_cast_opaque_ptr Modified: pypy/branch/jit-unroll-loops/pypy/rpython/rmodel.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/rmodel.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/rmodel.py Wed Nov 17 20:41:25 2010 @@ -11,14 +11,14 @@ # initialization states for Repr instances -class setupstate: +class setupstate(object): NOTINITIALIZED = 0 INPROGRESS = 1 BROKEN = 2 FINISHED = 3 DELAYED = 4 -class Repr: +class Repr(object): """ An instance of Repr is associated with each instance of SomeXxx. It defines the chosen representation for the SomeXxx. The Repr subclasses generally follows the SomeXxx subclass hierarchy, but there are numerous Modified: pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rpbc.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rpbc.py (original) +++ pypy/branch/jit-unroll-loops/pypy/rpython/test/test_rpbc.py Wed Nov 17 20:41:25 2010 @@ -1547,7 +1547,7 @@ def test_always_raising_methods(self): class Base: def m(self): - raise NotImplementedError + raise KeyError class A(Base): def m(self): return 42 @@ -1560,11 +1560,11 @@ o = B() try: o.m() - except NotImplementedError: - pass + except KeyError: + assert 0 return B().m() - self.interpret_raises(NotImplementedError, f, [7]) + self.interpret_raises(KeyError, f, [7]) def test_possible_missing_attribute_access(self): py.test.skip("Should explode or give some warning") Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/trackgcroot.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/trackgcroot.py (original) +++ pypy/branch/jit-unroll-loops/pypy/translator/c/gcc/trackgcroot.py Wed Nov 17 20:41:25 2010 @@ -1905,7 +1905,7 @@ if __name__ == '__main__': - verbose = 1 + verbose = 0 shuffle = False output_raw_table = False if sys.platform == 'darwin': Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/genc.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/translator/c/genc.py (original) +++ pypy/branch/jit-unroll-loops/pypy/translator/c/genc.py Wed Nov 17 20:41:25 2010 @@ -553,7 +553,7 @@ ('debug', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT" $(TARGET)'), ('debug_exc', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DDO_LOG_EXC" $(TARGET)'), ('debug_mem', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DTRIVIAL_MALLOC_DEBUG" $(TARGET)'), - ('no_obmalloc', '', '$(MAKE) CFLAGS="-g -O1 -DRPY_ASSERT -DNO_OBMALLOC" $(TARGET)'), + ('no_obmalloc', '', '$(MAKE) CFLAGS="-g -O2 -DRPY_ASSERT -DNO_OBMALLOC" $(TARGET)'), ('linuxmemchk', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DLINUXMEMCHK" $(TARGET)'), ('llsafer', '', '$(MAKE) CFLAGS="-O2 -DRPY_LL_ASSERT" $(TARGET)'), ('lldebug', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DRPY_LL_ASSERT" $(TARGET)'), @@ -623,9 +623,15 @@ mk.definition('OBJECTS', '$(ASMLBLFILES) gcmaptable.s') mk.rule('%.s', '%.c', '$(CC) $(CFLAGS) $(CFLAGSEXTRA) -frandom-seed=$< -o $@ -S $< $(INCLUDEDIRS)') mk.rule('%.lbl.s %.gcmap', '%.s', - python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py -m$(PYPY_MAIN_FUNCTION) -t $< > $*.gcmap') + [python + + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py ' + '-m$(PYPY_MAIN_FUNCTION) -t $< > $*.gctmp', + 'mv $*.gctmp $*.gcmap']) mk.rule('gcmaptable.s', '$(GCMAPFILES)', - python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py $(GCMAPFILES) > $@') + [python + + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py ' + '$(GCMAPFILES) > $@.tmp', + 'mv $@.tmp $@']) mk.rule('.PRECIOUS', '%.s', "# don't remove .s files if Ctrl-C'ed") else: Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/node.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/translator/c/node.py (original) +++ pypy/branch/jit-unroll-loops/pypy/translator/c/node.py Wed Nov 17 20:41:25 2010 @@ -714,7 +714,11 @@ s = ''.join([self.obj.getitem(i) for i in range(len(self.obj.items))]) else: s = ''.join(self.obj.items) - yield '\t%s%s' % (length, c_char_array_constant(s)) + array_constant = c_char_array_constant(s) + if array_constant.startswith('{') and barebonearray(T): + assert array_constant.endswith('}') + array_constant = array_constant[1:-1].strip() + yield '\t%s%s' % (length, array_constant) yield '}' else: barebone = barebonearray(T) Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/src/asm_gcc_x86.h ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/translator/c/src/asm_gcc_x86.h (original) +++ pypy/branch/jit-unroll-loops/pypy/translator/c/src/asm_gcc_x86.h Wed Nov 17 20:41:25 2010 @@ -2,6 +2,8 @@ * It replaces some complex macros with native assembler instructions. */ +#if 0 /* --- disabled: does not give any speed-up --- */ + #undef OP_INT_ADD_OVF #define OP_INT_ADD_OVF(x,y,r) \ asm volatile( \ @@ -50,6 +52,13 @@ : "0"(x), "g"(y) /* inputs */ \ : "cc", "memory") /* clobber */ +extern void op_int_overflowed(void) + asm ("_op_int_overflowed") + __attribute__((used)); + +#endif /* 0 */ + + /* Pentium only! */ #define READ_TIMESTAMP(val) \ asm volatile("rdtsc" : "=A" (val)) @@ -62,19 +71,15 @@ // I don't know how important it is, comment talks about time warps -/* prototypes */ - -extern void op_int_overflowed(void) - asm ("_op_int_overflowed") - __attribute__((used)); - /* implementations */ #ifndef PYPY_NOT_MAIN_FILE +# if 0 /* disabled */ void op_int_overflowed(void) { FAIL_OVF("integer operation"); } +# endif #endif Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/src/g_include.h ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/translator/c/src/g_include.h (original) +++ pypy/branch/jit-unroll-loops/pypy/translator/c/src/g_include.h Wed Nov 17 20:41:25 2010 @@ -39,10 +39,9 @@ #include "src/instrument.h" /* optional assembler bits */ -// disabled: does not give any speed-up -//#if defined(__GNUC__) && defined(__i386__) -//# include "src/asm_gcc_x86.h" -//#endif +#if defined(__GNUC__) && defined(__i386__) +# include "src/asm_gcc_x86.h" +#endif #if defined(__GNUC__) && defined(__ppc__) # include "src/asm_ppc.h" Modified: pypy/branch/jit-unroll-loops/pypy/translator/c/src/signals.h ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/translator/c/src/signals.h (original) +++ pypy/branch/jit-unroll-loops/pypy/translator/c/src/signals.h Wed Nov 17 20:41:25 2010 @@ -47,16 +47,12 @@ /* utility to poll for signals that arrived */ int pypysig_poll(void); /* => signum or -1 */ -/* When a signal is received, the bit 30 of pypysig_occurred is set. - After all signals are processed by pypysig_poll(), the bit 30 is - cleared again. The variable is exposed and RPython code is free to - use the other bits in any way. */ -#define PENDING_SIGNAL_BIT (1 << 30) +/* When a signal is received, pypysig_counter is set to -1. */ /* This is a struct for the JIT. See interp_signal.py. */ struct pypysig_long_struct { long value; }; -extern struct pypysig_long_struct pypysig_occurred; +extern struct pypysig_long_struct pypysig_counter; /* some C tricks to get/set the variable as efficiently as possible: use macros when compiling as a stand-alone program, but still @@ -64,18 +60,20 @@ #undef pypysig_getaddr_occurred void *pypysig_getaddr_occurred(void); #ifndef PYPY_NOT_MAIN_FILE -void *pypysig_getaddr_occurred(void) { return (void *)(&pypysig_occurred); } +void *pypysig_getaddr_occurred(void) { return (void *)(&pypysig_counter); } #endif -#define pypysig_getaddr_occurred() ((void *)(&pypysig_occurred)) +#define pypysig_getaddr_occurred() ((void *)(&pypysig_counter)) /************************************************************/ /* Implementation */ #ifndef PYPY_NOT_MAIN_FILE -struct pypysig_long_struct pypysig_occurred; -static volatile long *pypysig_occurred_v = (volatile long *)&pypysig_occurred.value; -static volatile int pypysig_flags[NSIG]; +struct pypysig_long_struct pypysig_counter = {0}; +static char volatile pypysig_flags[NSIG] = {0}; +static int volatile pypysig_occurred = 0; +/* pypysig_occurred is only an optimization: it tells if any + pypysig_flags could be set. */ void pypysig_ignore(int signum) { @@ -108,10 +106,11 @@ static void signal_setflag_handler(int signum) { if (0 <= signum && signum < NSIG) + { pypysig_flags[signum] = 1; - /* the point of "*pypysig_occurred_v" instead of just "pypysig_occurred" - is the volatile declaration */ - *pypysig_occurred_v |= PENDING_SIGNAL_BIT; + pypysig_occurred = 1; + pypysig_counter.value = -1; + } } void pypysig_setflag(int signum) @@ -130,27 +129,21 @@ int pypysig_poll(void) { - /* the two commented out lines below are useful for performance in - normal usage of pypysig_poll(); however, pypy/module/signal/ is - not normal usage. It only calls pypysig_poll() if the - PENDING_SIGNAL_BIT is set, and it clears that bit first. */ - -/* if (pypysig_occurred & PENDING_SIGNAL_BIT) */ + if (pypysig_occurred) { - int i; -/* pypysig_occurred &= ~PENDING_SIGNAL_BIT; */ - for (i=0; i 0 # minimal test + assert os.path.getsize(self.filename_dump) > 64 + filename_dump_typeids_z = str(udir.join('test_typeids_z')) + def define_write_typeids_z(self): + U = lltype.GcForwardReference() + U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)), + ('x', lltype.Signed))) + S = lltype.GcStruct('S', ('u', lltype.Ptr(U))) + A = lltype.GcArray(lltype.Ptr(S)) + filename = self.filename_dump_typeids_z + + def fn(): + s = lltype.malloc(S) + s.u = lltype.malloc(U) + s.u.next = lltype.malloc(U) + s.u.next.next = lltype.malloc(U) + a = lltype.malloc(A, 1000) + s2 = lltype.malloc(S) + # + p = rgc.get_typeids_z() + s = ''.join([p[i] for i in range(len(p))]) + fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666) + os.write(fd, s) + os.close(fd) + return 0 + + return fn + + def test_write_typeids_z(self): + self.run("write_typeids_z") + f = open(self.filename_dump_typeids_z) + data_z = f.read() + f.close() + import zlib + data = zlib.decompress(data_z) + assert data.startswith('member0') + assert 'GcArray of * GcStruct S {' in data class TestSemiSpaceGC(TestUsingFramework, snippet.SemiSpaceGCTestDefines): gcpolicy = "semispace" @@ -1173,21 +1215,22 @@ b = 0 c = 0 for i in range(len(tb)): - if tb[i].count == 10: + if tb[i].count == 10: # the type of S a += 1 nr = i for i in range(len(tb)): - if tb[i].count == 3: + if tb[i].count == 3: # the type GcArray(Ptr(S)) b += 1 c += tb[i].links[nr] - # we don't count b here since there can be more singletons, + # b can be 1 or 2 here since _heap_stats() is free to return or + # ignore the three GcStructs that point to the GcArray(Ptr(S)). # important one is c, a is for check return c * 100 + b * 10 + a return f def test_gc_heap_stats(self): res = self.run("gc_heap_stats") - assert res == 3011 + assert res == 3011 or res == 3021 def definestr_string_builder(cls): def fn(_): Modified: pypy/branch/jit-unroll-loops/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/branch/jit-unroll-loops/pypy/translator/cli/src/pypylib.cs Wed Nov 17 20:41:25 2010 @@ -83,8 +83,12 @@ return Double.NegativeInfinity; else if (s == "nan") return Double.NaN; - else - return System.Convert.ToDouble(s); + else { + System.Globalization.NumberFormatInfo formatter; + formatter = new System.Globalization.NumberFormatInfo(); + formatter.NumberDecimalSeparator = "."; + return System.Convert.ToDouble(s, formatter); + } } } Modified: pypy/branch/jit-unroll-loops/pypy/translator/goal/app_main.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/translator/goal/app_main.py (original) +++ pypy/branch/jit-unroll-loops/pypy/translator/goal/app_main.py Wed Nov 17 20:41:25 2010 @@ -326,10 +326,6 @@ except: print >> sys.stderr, "'import site' failed" - # update sys.path *after* loading site.py, in case there is a - # "site.py" file in the script's directory. - sys.path.insert(0, '') - if warnoptions: sys.warnoptions.append(warnoptions) from warnings import _processoptions @@ -366,6 +362,9 @@ try: if run_command: # handle the "-c" command + # Put '' on sys.path + sys.path.insert(0, '') + def run_it(): exec cmd in mainmodule.__dict__ success = run_toplevel(run_it) @@ -378,6 +377,13 @@ elif run_stdin: # handle the case where no command/filename/module is specified # on the command-line. + + # update sys.path *after* loading site.py, in case there is a + # "site.py" file in the script's directory. Only run this if we're + # executing the interactive prompt, if we're running a script we + # put it's directory on sys.path + sys.path.insert(0, '') + if go_interactive or sys.stdin.isatty(): # If stdin is a tty or if "-i" is specified, we print # a banner and run $PYTHONSTARTUP. Modified: pypy/branch/jit-unroll-loops/pypy/translator/goal/targetpypystandalone.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/translator/goal/targetpypystandalone.py (original) +++ pypy/branch/jit-unroll-loops/pypy/translator/goal/targetpypystandalone.py Wed Nov 17 20:41:25 2010 @@ -150,6 +150,9 @@ if config.objspace.allworkingmodules: from pypy.config.pypyoption import enable_allworkingmodules enable_allworkingmodules(config) + if config.objspace.translationmodules: + from pypy.config.pypyoption import enable_translationmodules + enable_translationmodules(config) if config.translation.type_system == 'ootype': config.objspace.usemodules.suggest(rbench=True) Modified: pypy/branch/jit-unroll-loops/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/branch/jit-unroll-loops/pypy/translator/goal/test2/test_app_main.py Wed Nov 17 20:41:25 2010 @@ -95,6 +95,11 @@ child.expect('>>> ') child.sendline('__name__') child.expect("'__main__'") + child.expect('>>> ') + child.sendline('import sys') + child.expect('>>> ') + child.sendline("'' in sys.path") + child.expect("True") def test_run_script(self): child = self.spawn([demo_script]) @@ -463,8 +468,10 @@ yield finally: old_cwd.chdir() - os.putenv('PYTHONPATH', old_pythonpath) - + # Can't call putenv with a None argument. + if old_pythonpath is not None: + os.putenv('PYTHONPATH', old_pythonpath) + tmpdir.join('site.py').write('print "SHOULD NOT RUN"') runme_py = tmpdir.join('runme.py') runme_py.write('print "some text"') @@ -485,9 +492,12 @@ with chdir_and_unset_pythonpath(tmpdir): data = self.run(cmdline2, python_flags='-S') - assert data.startswith("some new text\n") assert repr(str(tmpdir.join('otherpath'))) in data + assert "''" not in data + + data = self.run('-c "import sys; print sys.path"') + assert data.startswith("[''") class AppTestAppMain: From wlav at codespeak.net Thu Nov 18 03:10:32 2010 From: wlav at codespeak.net (wlav at codespeak.net) Date: Thu, 18 Nov 2010 03:10:32 +0100 (CET) Subject: [pypy-svn] r79226 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101118021032.224F0282B90@codespeak.net> Author: wlav Date: Thu Nov 18 03:10:29 2010 New Revision: 79226 Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py pypy/branch/reflex-support/pypy/module/cppyy/executor.py pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Log: rpython and comment fixes Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/converter.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/converter.py Thu Nov 18 03:10:29 2010 @@ -90,7 +90,7 @@ if len(value) != 1: raise OperationError(space.w_TypeError, - space.wrap("char expecter, got string of size %d" % len(value))) + space.wrap("char expected, got string of size %d" % len(value))) return value[0] # turn it into a "char" to the annotator def convert_argument(self, space, w_obj): @@ -189,9 +189,9 @@ def from_memory(self, space, w_obj, offset): # read access, so no copy needed fieldptr = self._get_fieldptr(space, w_obj, offset) - shortptr = rffi.cast(rffi.SHORTP, fieldptr) + ptrval = rffi.cast(rffi.UINT, fieldptr) w_array = unpack_simple_shape(space, space.wrap('h')) - return w_array.fromaddress(space, shortptr, self.size) + return w_array.fromaddress(space, ptrval, self.size) def to_memory(self, space, w_obj, w_value, offset): # copy only the pointer value @@ -205,6 +205,7 @@ # copy the full array (uses byte copy for now) fieldptr = self._get_fieldptr(space, w_obj, offset) value = w_value.getslotvalue(2) + # TODO: get sizeof(short) from system for i in range(min(self.size*2, value.getlength())): fieldptr[i] = value.getitem(i) @@ -219,9 +220,9 @@ def from_memory(self, space, w_obj, offset): # read access, so no copy needed fieldptr = self._get_fieldptr(space, w_obj, offset) - longptr = rffi.cast(rffi.LONGP, fieldptr) + ptrval = rffi.cast(rffi.UINT, fieldptr) w_array = unpack_simple_shape(space, space.wrap('l')) - return w_array.fromaddress(space, longptr, self.size) + return w_array.fromaddress(space, ptrval, self.size) def to_memory(self, space, w_obj, w_value, offset): # copy only the pointer value @@ -235,6 +236,7 @@ # copy the full array (uses byte copy for now) fieldptr = self._get_fieldptr(space, w_obj, offset) value = w_value.getslotvalue(2) + # TODO: get sizeof(long) from system for i in range(min(self.size*4, value.getlength())): fieldptr[i] = value.getitem(i) Modified: pypy/branch/reflex-support/pypy/module/cppyy/executor.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/executor.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/executor.py Thu Nov 18 03:10:29 2010 @@ -63,7 +63,7 @@ from pypy.module.cppyy import interp_cppyy long_result = capi.c_call_l(func.cpptype.handle, func.method_index, cppthis, num_args, args) ptr_result = rffi.cast(rffi.VOIDP, long_result) - return interp_cppyy.W_CPPInstance(self.cpptype, ptr_result) + return interp_cppyy.W_CPPInstance(space, self.cpptype, ptr_result) def get_executor(space, name): Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Thu Nov 18 03:10:29 2010 @@ -144,6 +144,7 @@ return "CPPFunction(%s, %s, %r, %s)" % ( self.cpptype, self.method_index, self.executor, self.arg_types) + class CPPFunction(CPPMethod): def call(self, cppthis, args_w): if self.executor is None: @@ -167,7 +168,7 @@ except Exception, e: capi.c_deallocate(self.cpptype.handle, newthis) raise - return W_CPPInstance(self.cpptype, newthis) + return W_CPPInstance(self.space, self.cpptype, newthis) class W_CPPOverload(Wrappable): @@ -237,7 +238,7 @@ class W_CPPType(Wrappable): - _immutable_fields_ = ["name","handle"] + _immutable_fields_ = ["name", "handle"] def __init__(self, space, name, handle): self.space = space @@ -322,9 +323,8 @@ class W_CPPInstance(Wrappable): - _immutable_ = True - def __init__(self, cppclass, rawobject): - self.space = cppclass.space + def __init__(self, space, cppclass, rawobject): + self.space = space self.cppclass = cppclass self.rawobject = rawobject From afa at codespeak.net Thu Nov 18 09:05:21 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 18 Nov 2010 09:05:21 +0100 (CET) Subject: [pypy-svn] r79227 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101118080521.DF59D282B9D@codespeak.net> Author: afa Date: Thu Nov 18 09:05:19 2010 New Revision: 79227 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py Log: add .writelines to all io classes Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Thu Nov 18 09:05:19 2010 @@ -228,6 +228,21 @@ return space.newlist(lines_w) + @unwrap_spec('self', ObjSpace, W_Root) + def writelines_w(self, space, w_lines): + self._check_closed(space) + + w_iterator = space.iter(w_lines) + + while True: + try: + w_line = space.next(w_iterator) + except OperationError, e: + if not e.match(space, space.w_StopIteration): + raise + break # done + space.call_method(self, "write", w_line) + W_IOBase.typedef = TypeDef( '_IOBase', __new__ = generic_new_descr(W_IOBase), @@ -251,6 +266,7 @@ readline = interp2app(W_IOBase.readline_w), readlines = interp2app(W_IOBase.readlines_w), + writelines = interp2app(W_IOBase.writelines_w), ) class W_RawIOBase(W_IOBase): Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_fileio.py Thu Nov 18 09:05:19 2010 @@ -77,6 +77,16 @@ f.close() f2.close() + def test_writelines(self): + import _io + filename = self.tmpfile + '_w' + f = _io.FileIO(filename, 'wb') + f.writelines(["line1\n", "line2", "line3"]) + f2 = _io.FileIO(filename, 'rb') + assert f2.read() == "line1\nline2line3" + f.close() + f2.close() + def test_seek(self): import _io f = _io.FileIO(self.tmpfile, 'rb') From afa at codespeak.net Thu Nov 18 09:18:49 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 18 Nov 2010 09:18:49 +0100 (CET) Subject: [pypy-svn] r79228 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101118081849.E2A74282B9D@codespeak.net> Author: afa Date: Thu Nov 18 09:18:48 2010 New Revision: 79228 Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py Log: Magic in action: it's enough to import str_decode into bytearraytype.py for bytearray.decode() to work! Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/bytearraytype.py Thu Nov 18 09:18:48 2010 @@ -6,6 +6,7 @@ from pypy.objspace.std.stdtypedef import StdTypeDef, SMM from pypy.objspace.std.stringtype import ( + str_decode, str_count, str_index, str_rindex, str_find, str_rfind, str_replace, str_startswith, str_endswith, str_islower, str_isupper, str_isalpha, str_isalnum, str_isdigit, str_isspace, str_istitle, Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py Thu Nov 18 09:18:48 2010 @@ -186,3 +186,9 @@ assert b == 'abcDefghi' buf[4:6] = 'EF' assert b == 'abcDEFghi' + + def test_decode(self): + b = bytearray('abcdefghi') + u = b.decode('utf-8') + assert isinstance(u, unicode) + assert u == u'abcdefghi' From niko at codespeak.net Thu Nov 18 11:32:03 2010 From: niko at codespeak.net (niko at codespeak.net) Date: Thu, 18 Nov 2010 11:32:03 +0100 (CET) Subject: [pypy-svn] r79232 - pypy/extradoc/planning/hg-migration Message-ID: <20101118103203.093F2282BE3@codespeak.net> Author: niko Date: Thu Nov 18 11:32:02 2010 New Revision: 79232 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: update niko Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Thu Nov 18 11:32:02 2010 @@ -37,7 +37,7 @@ jlg=Jakub Gustak bea=Beatrice During hakanardo=Hakan Ardo -niko=Niko Matsakis +niko=Niko Matsakis alex=Alex Martelli jcreigh=Jason Creighton iko=Anders Hammarquist From cfbolz at codespeak.net Thu Nov 18 11:39:15 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 18 Nov 2010 11:39:15 +0100 (CET) Subject: [pypy-svn] r79233 - pypy/extradoc/talk/pepm2011 Message-ID: <20101118103915.42BE0282B9D@codespeak.net> Author: cfbolz Date: Thu Nov 18 11:39:13 2010 New Revision: 79233 Modified: pypy/extradoc/talk/pepm2011/paper.tex Log: copyright data Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Thu Nov 18 11:39:13 2010 @@ -74,9 +74,9 @@ \begin{tabular}{c}} % \begin{document} -\conferenceinfo{PEPM}{'11, Austin, USA} +\conferenceinfo{PEPM'11,} {January 24--25, 2011, Austin, Texas, USA.} \CopyrightYear{2011} -\copyrightdata{[to be supplied]} % XXX +\copyrightdata{978-1-4503-0485-6/11/01} \title{Allocation Removal by Partial Evaluation in a Tracing JIT} From david at codespeak.net Thu Nov 18 11:49:53 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 18 Nov 2010 11:49:53 +0100 (CET) Subject: [pypy-svn] r79234 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101118104953.435C4282BEB@codespeak.net> Author: david Date: Thu Nov 18 11:49:51 2010 New Revision: 79234 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Implement call_assembler operation, for now without reseting the vable token Generate a second entry point to a loop passing the arguments in registers and the stack according to the calling convention for ARM used which is used in the call_assembler operation. Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/arch.py Thu Nov 18 11:49:51 2010 @@ -3,6 +3,8 @@ FUNC_ALIGN=8 WORD=4 +# The Address in the PC points two words befind the current instruction +PC_OFFSET = 8 arm_int_div_sign = lltype.Ptr(lltype.FuncType([lltype.Signed, lltype.Signed], lltype.Signed)) def arm_int_div(a, b): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Thu Nov 18 11:49:51 2010 @@ -223,6 +223,38 @@ regs.append(reg) regalloc.possibly_free_var(reg) looptoken._arm_arglocs = regs + return regs + + direct_bootstrap_code_size=100*WORD + def gen_direct_bootstrap_code(self, arglocs, loop_head, regalloc): + self.mc.ensure_can_fit(self.direct_bootstrap_code_size) + self.gen_func_prolog() + if len(arglocs) > 4: + reg_args = 4 + else: + reg_args = len(arglocs) + + stack_locs = len(arglocs) - reg_args + + for i in range(reg_args): + loc = arglocs[i] + self.mc.MOV_rr(loc.value, i) + + for i in range(stack_locs): + loc = arglocs[reg_args + i] + stack_position = (len(r.callee_saved_registers) + 1 +i)*WORD + if loc.is_reg(): + self.mc.LDR_ri(loc.value, r.fp.value, stack_position) + elif loc.is_stack(): + self.mc.PUSH([r.r0.value]) + self.mc.LDR_ri(r.ip, r.fp.value, stack_position) + self.mov_loc_loc(r.ip, loc) + self.mc.POP([r.r0.value]) + else: + assert 0, 'invalid location' + sp_patch_location = self._prepare_sp_patch_location() + self.mc.B(loop_head) + self._patch_sp_offset(sp_patch_location, regalloc) # cpu interface def assemble_loop(self, inputargs, operations, looptoken): @@ -232,9 +264,10 @@ loop_start=self.mc.curraddr() self.gen_func_prolog() + + arglocs = self.gen_bootstrap_code(inputargs, regalloc, looptoken) sp_patch_location = self._prepare_sp_patch_location() - self.gen_bootstrap_code(inputargs, regalloc, looptoken) loop_head=self.mc.curraddr() looptoken._arm_bootstrap_code = loop_start looptoken._arm_loop_code = loop_head @@ -243,6 +276,11 @@ self._patch_sp_offset(sp_patch_location, regalloc) + self.align() + + looptoken._arm_direct_bootstrap_code = self.mc.curraddr() + self.gen_direct_bootstrap_code(arglocs, loop_head, regalloc) + if self._debug_asm: self._dump_trace('loop.asm') print 'Done assembling' Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/codebuilder.py Thu Nov 18 11:49:51 2010 @@ -39,6 +39,9 @@ def ensure_can_fit(self, n): raise NotImplentedError + def NOP(self): + self.MOV_rr(0, 0) + def PUSH(self, regs, cond=cond.AL): assert reg.sp.value not in regs instr = self._encode_reg_list(cond << 28 | 0x92D << 16, regs) @@ -55,7 +58,7 @@ def B(self, target, c=cond.AL, some_reg=None): if c == cond.AL: self.ensure_can_fit(2*WORD) - self.LDR_ri(reg.pc.value, reg.pc.value, -4) + self.LDR_ri(reg.pc.value, reg.pc.value, -arch.PC_OFFSET/2) self.write32(target) else: assert some_reg is not None @@ -66,8 +69,8 @@ def BL(self, target, c=cond.AL, some_reg=None): if c == cond.AL: self.ensure_can_fit(3*WORD) - self.ADD_ri(reg.lr.value, reg.pc.value, 4) - self.LDR_ri(reg.pc.value, reg.pc.value, imm=-4) + self.ADD_ri(reg.lr.value, reg.pc.value, arch.PC_OFFSET/2) + self.LDR_ri(reg.pc.value, reg.pc.value, imm=-arch.PC_OFFSET/2) self.write32(target) else: assert some_reg is not None @@ -108,6 +111,9 @@ def curraddr(self): return self.baseaddr() + self._pos + def currpos(self): + return self._pos + size_of_gen_load_int = 7 * WORD def gen_load_int(self, r, value, cond=cond.AL): """r is the register number, value is the value to be loaded to the Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Thu Nov 18 11:49:51 2010 @@ -3,7 +3,8 @@ from pypy.jit.backend.arm import registers as r from pypy.jit.backend.arm import shift from pypy.jit.backend.arm.arch import (WORD, FUNC_ALIGN, arm_int_div, - arm_int_div_sign, arm_int_mod_sign, arm_int_mod) + arm_int_div_sign, arm_int_mod_sign, + arm_int_mod, PC_OFFSET) from pypy.jit.backend.arm.helper.assembler import (gen_emit_op_by_helper_call, gen_emit_op_unary_cmp, @@ -13,7 +14,8 @@ from pypy.jit.backend.llsupport import symbolic from pypy.jit.backend.llsupport.descr import BaseFieldDescr, BaseArrayDescr from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity, TempBox -from pypy.jit.metainterp.history import Const, ConstInt, BoxInt, BasicFailDescr +from pypy.jit.metainterp.history import (Const, ConstInt, BoxInt, + BasicFailDescr, LoopToken, INT, REF) from pypy.jit.metainterp.resoperation import rop from pypy.rlib import rgc from pypy.rlib.objectmodel import we_are_translated @@ -232,24 +234,44 @@ return fcond def emit_op_call(self, op, regalloc, fcond, save_all_regs=False): + adr = self.cpu.cast_adr_to_int(op.getarg(0).getint()) + args = op.getarglist()[1:] + cond = self._emit_call(adr, args, regalloc, fcond, save_all_regs, op.result) + + descr = op.getdescr() + #XXX Hack, Hack, Hack + if not we_are_translated() and not isinstance(descr, LoopToken): + l = regalloc.loc(op.result) + # XXX we need descr.get_result_sign here!!!! + size = descr.get_result_size(False) + # for now just check the size of the value + if size == 1: #unsigned char + self.mc.AND_ri(l.value, l.value, 255) + elif size == 2: # signed short + self.mc.LSL_ri(l.value, l.value, 16) + self.mc.ASR_ri(l.value, l.value, 16) + return cond + + def _emit_call(self, adr, args, regalloc, fcond=c.AL, save_all_regs=False, result=None): locs = [] # all arguments past the 4th go on the stack # XXX support types other than int (one word types) - if op.numargs() > 5: - stack_args = op.numargs() - 5 + n = 0 + n_args = len(args) + if n_args > 4: + stack_args = n_args - 4 n = stack_args*WORD self._adjust_sp(n, regalloc, fcond=fcond) - for i in range(5, op.numargs()): - reg = regalloc.make_sure_var_in_reg(op.getarg(i)) - self.mc.STR_ri(reg.value, r.sp.value, (i-5)*WORD) + for i in range(4, n_args): + reg = regalloc.make_sure_var_in_reg(args[i]) + self.mc.STR_ri(reg.value, r.sp.value, (i-4)*WORD) regalloc.possibly_free_var(reg) - adr = self.cpu.cast_adr_to_int(op.getarg(0).getint()) - reg_args = min(op.numargs()-1, 4) - for i in range(1, reg_args+1): - l = regalloc.make_sure_var_in_reg(op.getarg(i), - selected_reg=r.all_regs[i-1]) + reg_args = min(n_args, 4) + for i in range(0, reg_args): + l = regalloc.make_sure_var_in_reg(args[i], + selected_reg=r.all_regs[i]) locs.append(l) # XXX use PUSH here instead of spilling every reg for itself if save_all_regs: @@ -257,23 +279,15 @@ else: regalloc.before_call() self.mc.BL(adr) - #XXX Hack, Hack, Hack - if not we_are_translated(): - descr = op.getdescr() - # XXX we need descr.get_result_sign here!!!! - size = descr.get_result_size(False) - # for now just check the size of the value - if size == 1: #unsigned char - self.mc.AND_ri(r.r0.value, r.r0.value, 255) - elif size == 2: # signed short - self.mc.LSL_ri(r.r0.value, r.r0.value, 16) - self.mc.ASR_ri(r.r0.value, r.r0.value, 16) - regalloc.after_call(op.result) + if result: + regalloc.after_call(result) # readjust the sp in case we passed some args on the stack - if op.numargs() > 5: + if n_args > 4: + assert n > 0 self._adjust_sp(-n, regalloc, fcond=fcond) regalloc.possibly_free_vars(locs) + return fcond def emit_op_same_as(self, op, regalloc, fcond): resloc = regalloc.force_allocate_reg(op.result) @@ -401,6 +415,7 @@ if scale > 0: self.mc.LSL_ri(ofs_loc.value, ofs_loc.value, scale) f(res.value, base_loc.value, ofs_loc.value, cond=fcond) + return fcond emit_op_getarrayitem_raw = emit_op_getarrayitem_gc emit_op_getarrayitem_gc_pure = emit_op_getarrayitem_gc @@ -543,25 +558,132 @@ self.mc.MOV_rr(res_loc.value, r.fp.value) return fcond - def emit_guard_call_may_force(self, op, guard_op, regalloc, fcond): + # from: ../x86/assembler.py:1668 + def emit_guard_call_assembler(self, op, guard_op, regalloc, fcond): faildescr = guard_op.getdescr() fail_index = self.cpu.get_fail_descr_number(faildescr) + self._write_fail_index(fail_index, regalloc) + + descr = op.getdescr() + assert isinstance(descr, LoopToken) + + resbox = TempBox() + self._emit_call(descr._arm_direct_bootstrap_code, op.getarglist(), regalloc, fcond, resbox) + #self.mc.ensure_bytes_available(256) + if op.result is None: + value = self.cpu.done_with_this_frame_void_v + else: + kind = op.result.type + if kind == INT: + value = self.cpu.done_with_this_frame_int_v + elif kind == REF: + value = self.cpu.done_with_this_frame_ref_v + elif kind == FLOAT: + value = self.cpu.done_with_this_frame_float_v + else: + raise AssertionError(kind) + assert value <= 0xff + + # check value + t = TempBox() + resloc = regalloc.force_allocate_reg(resbox) + loc = regalloc.force_allocate_reg(t, [r.r0]) + self.mc.gen_load_int(loc.value, value) + self.mc.CMP_rr(resloc.value, loc.value) + regalloc.possibly_free_var(resbox) + + fast_jmp_pos = self.mc.currpos() + fast_jmp_location = self.mc.curraddr() + self.mc.NOP() + + #if values are equal we take the fast pat + # Slow path, calling helper + # jump to merge point + jd = descr.outermost_jitdriver_sd + assert jd is not None + asm_helper_adr = self.cpu.cast_adr_to_int(jd.assembler_helper_adr) + self._emit_call(asm_helper_adr, [t, op.getarg(0)], regalloc, fcond, False, op.result) + regalloc.possibly_free_var(t) + + # jump to merge point + jmp_pos = self.mc.currpos() + jmp_location = self.mc.curraddr() + self.mc.NOP() + + # Fast Path using result boxes + # patch the jump to the fast path + offset = self.mc.currpos() - fast_jmp_pos + pmc = ARMv7InMemoryBuilder(fast_jmp_location, WORD) + pmc.ADD_ri(r.pc.value, r.pc.value, offset - PC_OFFSET, cond=c.EQ) + + # Reset the vable token --- XXX really too much special logic here:-( + # XXX Enable and fix this once the stange errors procuded by its + # presence are fixed + #if jd.index_of_virtualizable >= 0: + # from pypy.jit.backend.llsupport.descr import BaseFieldDescr + # size = jd.portal_calldescr.get_result_size(self.cpu.translate_support_code) + # vable_index = jd.index_of_virtualizable + # regalloc._sync_var(op.getarg(vable_index)) + # vable = regalloc.frame_manager.loc(op.getarg(vable_index)) + # fielddescr = jd.vable_token_descr + # assert isinstance(fielddescr, BaseFieldDescr) + # ofs = fielddescr.offset + # self.mc.MOV(eax, arglocs[1]) + # self.mc.MOV_mi((eax.value, ofs), 0) + # # in the line above, TOKEN_NONE = 0 + + if op.result is not None: + # load the return value from fail_boxes_xxx[0] + loc = regalloc.force_allocate_reg(t) + resloc = regalloc.force_allocate_reg(op.result, [t]) + kind = op.result.type + if kind == INT: + adr = self.fail_boxes_int.get_addr_for_num(0) + elif kind == REF: + adr = self.fail_boxes_ptr.get_addr_for_num(0) + else: + raise AssertionError(kind) + self.mc.gen_load_int(loc.value, adr) + self.mc.LDR_ri(resloc.value, loc.value) + regalloc.possibly_free_var(t) + + offset = self.mc.currpos() - jmp_pos + pmc = ARMv7InMemoryBuilder(jmp_location, WORD) + pmc.ADD_ri(r.pc.value, r.pc.value, offset - PC_OFFSET) t = TempBox() l0 = regalloc.force_allocate_reg(t) - self.mc.gen_load_int(l0.value, fail_index) - self.mc.STR_ri(l0.value, r.fp.value) + self.mc.LDR_ri(l0.value, r.fp.value) + self.mc.CMP_ri(l0.value, 0) + regalloc.possibly_free_var(t) + regalloc.possibly_free_vars_for_op(op) + + self._emit_guard(guard_op, regalloc, c.LT) + return fcond + + def emit_guard_call_may_force(self, op, guard_op, regalloc, fcond): + faildescr = guard_op.getdescr() + fail_index = self.cpu.get_fail_descr_number(faildescr) + self._write_fail_index(fail_index, regalloc) # force all reg values to be spilled when calling fcond = self.emit_op_call(op, regalloc, fcond, save_all_regs=True) + t = TempBox() + l0 = regalloc.force_allocate_reg(t) self.mc.LDR_ri(l0.value, r.fp.value) self.mc.CMP_ri(l0.value, 0) - regalloc.possibly_free_var(t) self._emit_guard(guard_op, regalloc, c.LT) return fcond + def _write_fail_index(self, fail_index, regalloc): + t = TempBox() + l0 = regalloc.force_allocate_reg(t) + self.mc.gen_load_int(l0.value, fail_index) + self.mc.STR_ri(l0.value, r.fp.value) + regalloc.possibly_free_var(t) + class ResOpAssembler(GuardOpAssembler, IntOpAsslember, OpAssembler, UnaryIntOpAssembler, FieldOpAssembler, ArrayOpAssember, From david at codespeak.net Thu Nov 18 13:09:37 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 18 Nov 2010 13:09:37 +0100 (CET) Subject: [pypy-svn] r79235 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101118120937.1EC1B282B9D@codespeak.net> Author: david Date: Thu Nov 18 13:09:32 2010 New Revision: 79235 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Fix call operation for calls with void return value Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Thu Nov 18 13:09:32 2010 @@ -240,7 +240,7 @@ descr = op.getdescr() #XXX Hack, Hack, Hack - if not we_are_translated() and not isinstance(descr, LoopToken): + if op.result and not we_are_translated() and not isinstance(descr, LoopToken): l = regalloc.loc(op.result) # XXX we need descr.get_result_sign here!!!! size = descr.get_result_size(False) @@ -559,6 +559,7 @@ return fcond # from: ../x86/assembler.py:1668 + # XXX Split into some helper methods def emit_guard_call_assembler(self, op, guard_op, regalloc, fcond): faildescr = guard_op.getdescr() fail_index = self.cpu.get_fail_descr_number(faildescr) From arigo at codespeak.net Thu Nov 18 13:23:11 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 18 Nov 2010 13:23:11 +0100 (CET) Subject: [pypy-svn] r79236 - pypy/trunk/pypy/rpython/memory/test Message-ID: <20101118122311.115AC282B9D@codespeak.net> Author: arigo Date: Thu Nov 18 13:23:09 2010 New Revision: 79236 Modified: pypy/trunk/pypy/rpython/memory/test/test_gc.py Log: A test showing why we cannot mark weakrefs as no-longer-valid if they only point to __del__ objects going away. Modified: pypy/trunk/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/trunk/pypy/rpython/memory/test/test_gc.py Thu Nov 18 13:23:09 2010 @@ -352,6 +352,29 @@ res = self.interpret(f, []) assert res == 11 + def test_weakref_bug_1(self): + import weakref + class A(object): + pass + class B(object): + def __del__(self): + self.wref().x += 1 + def g(a): + b = B() + b.wref = weakref.ref(a) + # the only way to reach this weakref is via B, which is an + # object with finalizer (but the weakref itself points to + # a, which does not go away but will move during the next + # gc.collect) + def f(): + a = A() + a.x = 10 + g(a) + llop.gc__collect(lltype.Void) + return a.x + res = self.interpret(f, []) + assert res == 11 + def test_id(self): class A(object): pass From arigo at codespeak.net Thu Nov 18 13:33:37 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 18 Nov 2010 13:33:37 +0100 (CET) Subject: [pypy-svn] r79237 - in pypy/trunk/pypy/rpython/memory: gc test Message-ID: <20101118123337.59EAA282B9D@codespeak.net> Author: arigo Date: Thu Nov 18 13:33:35 2010 New Revision: 79237 Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py pypy/trunk/pypy/rpython/memory/gc/semispace.py pypy/trunk/pypy/rpython/memory/test/test_gc.py Log: Revert r79182. It's not as easy, it even looks like a mess. What we can (for now) guarantee is that with this kind of relation: a --------> weakref object - - - -> b --------> c if c has a finalizer, then when the finalizer of c runs, we are sure that b is gone, so the weakref points to None. Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Thu Nov 18 13:33:35 2010 @@ -1205,10 +1205,6 @@ self.collect_roots() self.visit_all_objects() # - # Weakref support: clear the weak pointers to dying objects - if self.old_objects_with_weakrefs.non_empty(): - self.invalidate_old_weakrefs() - # # Finalizer support: adds the flag GCFLAG_VISITED to all objects # with a finalizer and all objects reachable from there (and also # moves some objects from 'objects_with_finalizers' to @@ -1218,6 +1214,10 @@ # self.objects_to_trace.delete() # + # Weakref support: clear the weak pointers to dying objects + if self.old_objects_with_weakrefs.non_empty(): + self.invalidate_old_weakrefs() + # # Walk all rawmalloced objects and free the ones that don't # have the GCFLAG_VISITED flag. self.free_unvisited_rawmalloc_objects() Modified: pypy/trunk/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/semispace.py Thu Nov 18 13:33:35 2010 @@ -266,10 +266,10 @@ if self.run_finalizers.non_empty(): self.update_run_finalizers() scan = self.scan_copied(scan) - if self.objects_with_weakrefs.non_empty(): - self.invalidate_weakrefs() if self.objects_with_finalizers.non_empty(): scan = self.deal_with_objects_with_finalizers(scan) + if self.objects_with_weakrefs.non_empty(): + self.invalidate_weakrefs() self.update_objects_with_id() self.finished_full_collect() self.debug_check_consistency() Modified: pypy/trunk/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/trunk/pypy/rpython/memory/test/test_gc.py Thu Nov 18 13:33:35 2010 @@ -304,7 +304,7 @@ a = A() class B(object): def __del__(self): - # when __del__ is called, the weakref should have been cleared + # when __del__ is called, the weakref to c should be dead if self.ref() is None: a.count += 10 # ok else: @@ -333,8 +333,10 @@ a = A() class B(object): def __del__(self): - # when __del__ is called, the weakref should have been cleared - if self.ref() is None: + # when __del__ is called, the weakref to myself is still valid + # in RPython (at least with most GCs; this test might be + # skipped for specific GCs) + if self.ref() is self: a.count += 10 # ok else: a.count = 666 # not ok @@ -732,6 +734,9 @@ class TestMarkSweepGC(GCTest): from pypy.rpython.memory.gc.marksweep import MarkSweepGC as GCClass + def test_weakref_to_object_with_finalizer_ordering(self): + py.test.skip("Does not work") + class TestSemiSpaceGC(GCTest, snippet.SemiSpaceGCTests): from pypy.rpython.memory.gc.semispace import SemiSpaceGC as GCClass GC_CAN_MOVE = True @@ -754,9 +759,6 @@ def test_finalizer_order(self): py.test.skip("Not implemented yet") - def test_weakref_to_object_with_finalizer_ordering(self): - py.test.skip("Not implemented yet") - class TestHybridGC(TestGenerationalGC): from pypy.rpython.memory.gc.hybrid import HybridGC as GCClass GC_CAN_MALLOC_NONMOVABLE = True From arigo at codespeak.net Thu Nov 18 13:34:45 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 18 Nov 2010 13:34:45 +0100 (CET) Subject: [pypy-svn] r79238 - in pypy/branch/jit-free/pypy/rpython/memory: gc test Message-ID: <20101118123445.3297A282BDC@codespeak.net> Author: arigo Date: Thu Nov 18 13:34:43 2010 New Revision: 79238 Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py pypy/branch/jit-free/pypy/rpython/memory/gc/semispace.py pypy/branch/jit-free/pypy/rpython/memory/test/test_gc.py Log: Merge r79183:79237 from trunk. Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py Thu Nov 18 13:34:43 2010 @@ -1156,10 +1156,6 @@ self.collect_roots() self.visit_all_objects() # - # Weakref support: clear the weak pointers to dying objects - if self.old_objects_with_weakrefs.non_empty(): - self.invalidate_old_weakrefs() - # # Finalizer support: adds the flag GCFLAG_VISITED to all objects # with a finalizer and all objects reachable from there (and also # moves some objects from 'objects_with_finalizers' to @@ -1169,6 +1165,10 @@ # self.objects_to_trace.delete() # + # Weakref support: clear the weak pointers to dying objects + if self.old_objects_with_weakrefs.non_empty(): + self.invalidate_old_weakrefs() + # # Walk all rawmalloced objects and free the ones that don't # have the GCFLAG_VISITED flag. self.free_unvisited_rawmalloc_objects() Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/semispace.py Thu Nov 18 13:34:43 2010 @@ -266,10 +266,10 @@ if self.run_finalizers.non_empty(): self.update_run_finalizers() scan = self.scan_copied(scan) - if self.objects_with_weakrefs.non_empty(): - self.invalidate_weakrefs() if self.objects_with_finalizers.non_empty(): scan = self.deal_with_objects_with_finalizers(scan) + if self.objects_with_weakrefs.non_empty(): + self.invalidate_weakrefs() self.update_objects_with_id() self.finished_full_collect() self.debug_check_consistency() Modified: pypy/branch/jit-free/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/test/test_gc.py Thu Nov 18 13:34:43 2010 @@ -304,7 +304,7 @@ a = A() class B(object): def __del__(self): - # when __del__ is called, the weakref should have been cleared + # when __del__ is called, the weakref to c should be dead if self.ref() is None: a.count += 10 # ok else: @@ -333,8 +333,10 @@ a = A() class B(object): def __del__(self): - # when __del__ is called, the weakref should have been cleared - if self.ref() is None: + # when __del__ is called, the weakref to myself is still valid + # in RPython (at least with most GCs; this test might be + # skipped for specific GCs) + if self.ref() is self: a.count += 10 # ok else: a.count = 666 # not ok @@ -352,6 +354,29 @@ res = self.interpret(f, []) assert res == 11 + def test_weakref_bug_1(self): + import weakref + class A(object): + pass + class B(object): + def __del__(self): + self.wref().x += 1 + def g(a): + b = B() + b.wref = weakref.ref(a) + # the only way to reach this weakref is via B, which is an + # object with finalizer (but the weakref itself points to + # a, which does not go away but will move during the next + # gc.collect) + def f(): + a = A() + a.x = 10 + g(a) + llop.gc__collect(lltype.Void) + return a.x + res = self.interpret(f, []) + assert res == 11 + def test_id(self): class A(object): pass @@ -709,6 +734,9 @@ class TestMarkSweepGC(GCTest): from pypy.rpython.memory.gc.marksweep import MarkSweepGC as GCClass + def test_weakref_to_object_with_finalizer_ordering(self): + py.test.skip("Does not work") + class TestSemiSpaceGC(GCTest, snippet.SemiSpaceGCTests): from pypy.rpython.memory.gc.semispace import SemiSpaceGC as GCClass GC_CAN_MOVE = True @@ -731,9 +759,6 @@ def test_finalizer_order(self): py.test.skip("Not implemented yet") - def test_weakref_to_object_with_finalizer_ordering(self): - py.test.skip("Not implemented yet") - class TestHybridGC(TestGenerationalGC): from pypy.rpython.memory.gc.hybrid import HybridGC as GCClass GC_CAN_MALLOC_NONMOVABLE = True From cfbolz at codespeak.net Thu Nov 18 13:38:52 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 18 Nov 2010 13:38:52 +0100 (CET) Subject: [pypy-svn] r79239 - pypy/extradoc/talk/pepm2011 Message-ID: <20101118123852.B2F40282B9D@codespeak.net> Author: cfbolz Date: Thu Nov 18 13:38:51 2010 New Revision: 79239 Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf pypy/extradoc/talk/pepm2011/paper.bib pypy/extradoc/talk/pepm2011/paper.tex Log: add related work. now we are at 11 pages again. Modified: pypy/extradoc/talk/pepm2011/escape-tracing.pdf ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/pepm2011/paper.bib ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.bib (original) +++ pypy/extradoc/talk/pepm2011/paper.bib Thu Nov 18 13:38:51 2010 @@ -177,6 +177,16 @@ pages = {223{\textendash}232} }, + at phdthesis{jrgensen_calculus_1996, + type = {{Ph.D. Thesis}}, + title = {A Calculus for Boxing Analysis of Polymorphically Typed Languages}, + abstract = {An important decision when implementing languages with polymorphic types, such as Standard {ML} or Haskell, is whether to represent data in boxed or unboxed form and when to transform them from one representation to the other. Using a language with explicit representation types and boxing/unboxing operations we axiomatize equationally the set of all explicitly boxed versions, called completions, of a given source program. In a two-stage process we give some of the equations a rewriting interpretation that captures eliminating boxing/unboxing operations without relying on a specific implementation or even the semantics of the underlying language. The resulting reduction systems operate on congruence classes of completions defined by the remaining equations E, which can be understood as moving boxing/unboxing operations along data flow paths in the source program. We call a completion e formally optimal if every other completion for the same program (and at the same representation type) reduces to e under this two-stage reduction. We show that every source program has formally optimal completions, which are unique modulo E. This is accomplished by first "polarizing" the equations in E and orienting them to obtain two canonical (confluent and strongly normalizing) rewriting systems.}, + school = {University of Copenhapen}, + author = {Jesper J{\o}rgensen}, + year = {1996}, + note = {TR {96/28}} +}, + @inproceedings{chang_tracing_2009, address = {Washington, {DC,} {USA}}, title = {Tracing for Web 3.0: Trace Compilation for the Next Generation Web Applications}, Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Thu Nov 18 13:38:51 2010 @@ -132,6 +132,8 @@ \section{Introduction} +% XXX insert "practical" somewhere + The objective of a just-in-time (JIT) compiler for a dynamic language is to improve the speed of the language over an implementation of the language that uses interpretation. The first goal of a JIT is therefore to remove the @@ -1213,9 +1215,12 @@ imperative language. In functional programming this idea was introduced as constructor specialisation by Mogensen \cite{mogensen_constructor_1993}. -\reva{I'd refer here to \cite{wadler_deforestation:_1988} - and \cite{gill_short_1993}} -\revb{cite and relate to: boxing analysis} + +A related optimization is also that of deforestation +\cite{wadler_deforestation:_1988,gill_short_1993} which tries to remove +intermediate lists or trees in functional languages. More explicitly geared at +optimizing boxing is boxing analysis \cite{jrgensen_calculus_1996} which tries +to fully remove pairs of calls to box/unbox in a functional language. %xxx: %partially static data structures: kenichi asai's thesis? From cfbolz at codespeak.net Thu Nov 18 13:42:32 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 18 Nov 2010 13:42:32 +0100 (CET) Subject: [pypy-svn] r79240 - pypy/extradoc/talk/pepm2011 Message-ID: <20101118124232.244B0282BE3@codespeak.net> Author: cfbolz Date: Thu Nov 18 13:42:31 2010 New Revision: 79240 Modified: pypy/extradoc/talk/pepm2011/paper.tex Log: try to use the word "hot" only after it is defined Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Thu Nov 18 13:42:31 2010 @@ -132,8 +132,6 @@ \section{Introduction} -% XXX insert "practical" somewhere - The objective of a just-in-time (JIT) compiler for a dynamic language is to improve the speed of the language over an implementation of the language that uses interpretation. The first goal of a JIT is therefore to remove the @@ -166,7 +164,7 @@ A recently popular approach to implementing just-in-time compilers for dynamic languages is that of a tracing JIT. A tracing JIT works by observing the running -program and recording its hot spots into \emph{linear execution traces}. Those +program and recording its commonly executed parts into \emph{linear execution traces}. Those traces are optimized and turned into machine code. One reason for the popularity of tracing JITs is their relative @@ -198,8 +196,8 @@ The contributions made by this paper are: \begin{enumerate} - \item A description of an efficient and effective algorithm for removing - object allocations in a tracing JIT. + \item A description of a practical, efficient and effective algorithm for + removing object allocations in a tracing JIT. \item A characterization of this algorithm as partial evaluation. \item Performance benchmarks for this algorithm. \end{enumerate} @@ -265,7 +263,7 @@ mostly mixed-mode execution environments, they contain both an interpreter and a JIT compiler. By default the interpreter is used to execute the program, doing some light-weight profiling at the same time. This profiling is used to identify -the hot loops\reva{what is that? clarify} of the program. If a hot loop is found in that way, the +the hot loops of the program. If a hot loop is found in that way, the interpreter enters a special \emph{tracing mode}. In this tracing mode, the interpreter tries to record all operations that it is executing while running one iteration of the hot loop. This history of executed operations of one loop is From cfbolz at codespeak.net Thu Nov 18 13:43:46 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 18 Nov 2010 13:43:46 +0100 (CET) Subject: [pypy-svn] r79241 - pypy/extradoc/talk/pepm2011 Message-ID: <20101118124346.118AD282BEA@codespeak.net> Author: cfbolz Date: Thu Nov 18 13:43:44 2010 New Revision: 79241 Modified: pypy/extradoc/talk/pepm2011/paper.tex Log: next review comment Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Thu Nov 18 13:43:44 2010 @@ -296,7 +296,7 @@ on. These guards are the only mechanism to stop the execution of a trace, the loop end condition also takes the form of a guard. -If one specific guard fails often enough\reva{when is that?}, the tracing JIT will generate a new +If one specific guard fails a lot (\ie more than some threshold), the tracing JIT will generate a new trace that starts exactly at the position of the failing guard \cite{andreas_gal_incremental_2006}. The existing assembler is patched to jump to the new trace when the guard fails. This approach guarantees that all the From cfbolz at codespeak.net Thu Nov 18 13:45:00 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 18 Nov 2010 13:45:00 +0100 (CET) Subject: [pypy-svn] r79242 - pypy/extradoc/talk/pepm2011 Message-ID: <20101118124500.A4673282BEF@codespeak.net> Author: cfbolz Date: Thu Nov 18 13:44:58 2010 New Revision: 79242 Modified: pypy/extradoc/talk/pepm2011/paper.tex Log: I will ignore this comment Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Thu Nov 18 13:44:58 2010 @@ -1174,8 +1174,7 @@ complex than our simple one-pass optimization. Also, stack-allocation reduces garbage-collection pressure but does not optimize away the actual accesses to the stack-allocated object. In our case, an object is not needed at all any -more. \reva{Has this relation pointed out before in some paper? Is it a -novel contribution of your work?} +more. Chang \etal describe a tracing JIT for JavaScript running on top of a JVM \cite{mason_chang_efficient_2007}. They mention in passing an approach to From cfbolz at codespeak.net Thu Nov 18 14:01:56 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 18 Nov 2010 14:01:56 +0100 (CET) Subject: [pypy-svn] r79243 - pypy/extradoc/talk/pepm2011 Message-ID: <20101118130156.32544282B9E@codespeak.net> Author: cfbolz Date: Thu Nov 18 14:01:53 2010 New Revision: 79243 Modified: pypy/extradoc/talk/pepm2011/paper.tex Log: get space back. bit non-beautiful, but well. Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Thu Nov 18 14:01:53 2010 @@ -21,7 +21,7 @@ \definecolor{gray}{rgb}{0.3,0.3,0.3} \lstset{ - basicstyle=\setstretch{1.1}\ttfamily\footnotesize, + basicstyle=\setstretch{1.05}\ttfamily\footnotesize, language=Python, keywordstyle=\bfseries, stringstyle=\color{blue}, @@ -319,7 +319,7 @@ Figure~\ref{fig:objmodel} (written in RPython). \begin{figure} -\begin{lstlisting}[mathescape,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] +\begin{lstlisting}[mathescape,basicstyle=\setstretch{1.05}\ttfamily\scriptsize] class Base(object): pass @@ -390,7 +390,7 @@ \begin{figure} -\begin{lstlisting}[mathescape,numbers = right,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] +\begin{lstlisting}[mathescape,numbers = right,basicstyle=\setstretch{1.05}\ttfamily\scriptsize] # arguments to the trace: $p_{0}$, $p_{1}$ # inside f: res.add(y) guard_class($p_{1}$, BoxedInteger) @@ -592,7 +592,7 @@ The following operations (lines 10--17) are more interesting: -\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=10,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] +\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=10,basicstyle=\setstretch{1.05}\ttfamily\scriptsize] $p_{5}$ = new(BoxedInteger) |\setcounter{lstnumber}{11}| set($p_{5}$, intval, $i_{4}$) |\setcounter{lstnumber}{14}| $p_{6}$ = new(BoxedInteger) |\setcounter{lstnumber}{16}| @@ -623,7 +623,7 @@ The subsequent operations (line 20--26) in Figure~\ref{fig:unopt-trace}, which use $p_{5}$ and $p_{6}$, can then be optimized using that knowledge: -\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=20,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] +\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=20,basicstyle=\setstretch{1.05}\ttfamily\scriptsize] guard_class($p_{5}$, BoxedInteger) |\setcounter{lstnumber}{21}| $i_{7}$ = get($p_{5}$, intval) guard_class($p_{6}$, BoxedInteger) |\setcounter{lstnumber}{24}| @@ -639,7 +639,7 @@ replaced by $i_{4}$ and -100. The only operation copied into the optimized trace is the addition: -\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=26,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] +\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=26,basicstyle=\setstretch{1.05}\ttfamily\scriptsize] $i_{9}$ = int_add($i_{4}$, -100) \end{lstlisting} @@ -666,7 +666,7 @@ because the static objects could form an arbitrary graph structure. In our example it is simple, though: -\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=44,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] +\begin{lstlisting}[mathescape,xleftmargin=20pt,numbers=right,escapechar=|, firstnumber=44,basicstyle=\setstretch{1.05}\ttfamily\scriptsize] $p_{15}$ = new(BoxedInteger) |\setcounter{lstnumber}{45}| set($p_{15}$, intval, $i_{14}$) |\setcounter{lstnumber}{26}| $p_{10}$ = new(BoxedInteger) |\setcounter{lstnumber}{28}| @@ -692,7 +692,7 @@ original seven. \begin{figure} -\begin{lstlisting}[mathescape,numbers=right,escapechar=|, numberblanklines=false,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] +\begin{lstlisting}[mathescape,numbers=right,escapechar=|, numberblanklines=false,basicstyle=\setstretch{1.05}\ttfamily\scriptsize] # arguments to the trace: $p_{0}$, $p_{1}$ |\setcounter{lstnumber}{2}| guard_class($p_1$, BoxedInteger) |\setcounter{lstnumber}{4}| $i_2$ = get($p_1$, intval) @@ -932,7 +932,7 @@ v^*), w^* \mapsto (T_2, u^*, u^*)\}$$ which contains two static objects. If $v^*$ needs to be lifted, the following residual operations are produced: -\begin{lstlisting}[mathescape,xleftmargin=20pt,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] +\begin{lstlisting}[mathescape,xleftmargin=20pt,basicstyle=\setstretch{1.05}\ttfamily\scriptsize] $v^*$ = new($T_1$) $w^*$ = new($T_2$) set($w^*$, $L$, $u^*$) @@ -946,7 +946,7 @@ If we had lifted $w^*$ instead of $v^*$, then the following operations would have been produced: -\begin{lstlisting}[mathescape,xleftmargin=20pt,basicstyle=\setstretch{1.1}\ttfamily\scriptsize] +\begin{lstlisting}[mathescape,xleftmargin=20pt,basicstyle=\setstretch{1.05}\ttfamily\scriptsize] $w^*$ = new($T_2$) set($w^*$, $L$, $u^*$) set($w^*$, $R$, $u^*$) From afa at codespeak.net Thu Nov 18 14:07:07 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 18 Nov 2010 14:07:07 +0100 (CET) Subject: [pypy-svn] r79244 - pypy/branch/fast-forward/pypy/module/_io Message-ID: <20101118130707.A2696282B9E@codespeak.net> Author: afa Date: Thu Nov 18 14:07:05 2010 New Revision: 79244 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Log: Add a sanity check (CPython has the same) and try to clear weakrefs correctly Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Thu Nov 18 14:07:05 2010 @@ -499,6 +499,10 @@ if space.is_w(w_size, space.w_None): raise BlockingIOError() size = space.int_w(w_size) + if size < 0 or size > length: + raise OperationError(space.w_IOError, space.wrap( + "raw readinto() returned invalid length %d " + "(should have been between 0 and %d)" % (size, length))) if self.abs_pos != -1: self.abs_pos += size return size @@ -843,7 +847,8 @@ raise def __del__(self): - pass # no not close the files + self.clear_all_weakrefs() + # Don't call the base __del__: do not close the files! # forward to reader for method in ['read', 'peek', 'read1', 'readinto', 'readable']: Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Thu Nov 18 14:07:05 2010 @@ -57,6 +57,7 @@ return False def __del__(self): + self.clear_all_weakrefs() space = self.space w_closed = space.findattr(self, space.wrap('closed')) try: From cfbolz at codespeak.net Thu Nov 18 14:22:11 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 18 Nov 2010 14:22:11 +0100 (CET) Subject: [pypy-svn] r79245 - pypy/extradoc/talk/pepm2011 Message-ID: <20101118132211.BC2EE282B9D@codespeak.net> Author: cfbolz Date: Thu Nov 18 14:22:08 2010 New Revision: 79245 Modified: pypy/extradoc/talk/pepm2011/paper.bib pypy/extradoc/talk/pepm2011/paper.tex Log: add dynamic typing reference Modified: pypy/extradoc/talk/pepm2011/paper.bib ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.bib (original) +++ pypy/extradoc/talk/pepm2011/paper.bib Thu Nov 18 14:22:08 2010 @@ -394,3 +394,22 @@ year = {2008}, pages = {123--139} } + + at article{Henglein:1994:DTS:190865.190867, + author = {Henglein, Fritz}, + title = {Dynamic typing: syntax and proof theory}, + journal = {Sci. Comput. Program.}, + volume = {22}, + issue = {3}, + month = {June}, + year = {1994}, + issn = {0167-6423}, + pages = {197--230}, + numpages = {34}, + url = {http://portal.acm.org/citation.cfm?id=190865.190867}, + doi = {10.1016/0167-6423(94)00004-2}, + acmid = {190867}, + publisher = {Elsevier North-Holland, Inc.}, + address = {Amsterdam, The Netherlands, The Netherlands}, +} + Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Thu Nov 18 14:22:08 2010 @@ -1214,19 +1214,16 @@ constructor specialisation by Mogensen \cite{mogensen_constructor_1993}. A related optimization is also that of deforestation -\cite{wadler_deforestation:_1988,gill_short_1993} which tries to remove -intermediate lists or trees in functional languages. More explicitly geared at -optimizing boxing is boxing analysis \cite{jrgensen_calculus_1996} which tries -to fully remove pairs of calls to box/unbox in a functional language. - -%xxx: -%partially static data structures: kenichi asai's thesis? - -%xxx: -%Other related work is compile-time garbage collection \cite{mazur_practical_2001}, which -%tries to ....; -% separation logic - %; John Hughes: type specialization +\cite{wadler_deforestation:_1988,gill_short_1993} which removes intermediate +lists or trees in functional languages. A more general approach is boxing +analysis \cite{jrgensen_calculus_1996} which optimizes pairs of calls to +box/unbox in a functional language. Similarly, "dynamic typing" +\cite{Henglein:1994:DTS:190865.190867} tries to remove dynamic type coercions +in a dynamically typed lambda-calculus. All these optimizations work by +analyzing the program before execution, which makes them unsuitable for dynamic +languages like Python, where almost nothing can be inferred purely by looking +at the source code. + \section{Conclusion and Future Work} \label{sec:conclusion} From cfbolz at codespeak.net Thu Nov 18 14:44:27 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 18 Nov 2010 14:44:27 +0100 (CET) Subject: [pypy-svn] r79247 - pypy/extradoc/talk/pepm2011 Message-ID: <20101118134427.310CF282BD6@codespeak.net> Author: cfbolz Date: Thu Nov 18 14:44:25 2010 New Revision: 79247 Modified: pypy/extradoc/talk/pepm2011/paper.tex Log: thank the reviewers Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Thu Nov 18 14:44:25 2010 @@ -1251,7 +1251,7 @@ The authors would like to thank Stefan Hallerstede, David Schneider and Thomas Stiehl for fruitful discussions and detailed feedback during the writing of the -paper. +paper. We thank the anonymous reviewers for the valuable comments. \bibliographystyle{abbrv} \bibliography{paper} From afa at codespeak.net Thu Nov 18 14:50:18 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 18 Nov 2010 14:50:18 +0100 (CET) Subject: [pypy-svn] r79248 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101118135018.1A76C282BD6@codespeak.net> Author: afa Date: Thu Nov 18 14:50:17 2010 New Revision: 79248 Added: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py (contents, props changed) Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Log: First tests for _io.TextIOWrapper Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Thu Nov 18 14:50:17 2010 @@ -1,17 +1,74 @@ from pypy.module._io.interp_iobase import W_IOBase from pypy.interpreter.typedef import ( - TypeDef, generic_new_descr) + TypeDef, GetSetProperty, interp_attrproperty_w, interp_attrproperty, + generic_new_descr) +from pypy.interpreter.gateway import interp2app, unwrap_spec +from pypy.interpreter.baseobjspace import ObjSpace, W_Root +from pypy.interpreter.error import OperationError, operationerrfmt class W_TextIOBase(W_IOBase): - pass + w_encoding = None + + def __init__(self, space): + W_IOBase.__init__(self, space) + + def _unsupportedoperation(self, space, message): + w_exc = space.getattr(space.getbuiltinmodule('_io'), + space.wrap('UnsupportedOperation')) + raise OperationError(w_exc, space.wrap(message)) + + @unwrap_spec('self', ObjSpace, W_Root) + def read_w(self, space, w_size=None): + self._unsupportedoperation(space, "read") + + @unwrap_spec('self', ObjSpace, W_Root) + def readline_w(self, space, w_limit=None): + self._unsupportedoperation(space, "readline") + W_TextIOBase.typedef = TypeDef( '_TextIOBase', W_IOBase.typedef, __new__ = generic_new_descr(W_TextIOBase), + + read = interp2app(W_TextIOBase.read_w), + encoding = interp_attrproperty_w("w_encoding", W_TextIOBase) ) class W_TextIOWrapper(W_TextIOBase): - pass + @unwrap_spec('self', ObjSpace, W_Root, W_Root, W_Root, W_Root, int) + def descr_init(self, space, w_buffer, w_encoding=None, + w_errors=None, w_newline=None, line_buffering=0): + self.w_buffer = w_buffer + self.w_encoding = w_encoding + + if space.is_w(w_newline, space.w_None): + newline = None + else: + newline = space.str_w(w_newline) + if newline and newline not in ('\n', '\r\n', '\r'): + raise OperationError(space.w_ValueError, space.wrap( + "illegal newline value: %s" % (newline,))) + + self.line_buffering = line_buffering + + @unwrap_spec('self', ObjSpace, W_Root) + def read_w(self, space, w_size=None): + # XXX w_size? + w_bytes = space.call_method(self.w_buffer, "read") + return space.call_method(w_bytes, "decode", self.w_encoding) + + @unwrap_spec('self', ObjSpace, W_Root) + def readline_w(self, space, w_limit=None): + # XXX w_limit? + w_bytes = space.call_method(self.w_buffer, "readline") + return space.call_method(w_bytes, "decode", self.w_encoding) + W_TextIOWrapper.typedef = TypeDef( 'TextIOWrapper', W_TextIOBase.typedef, __new__ = generic_new_descr(W_TextIOWrapper), + __init__ = interp2app(W_TextIOWrapper.descr_init), + + read = interp2app(W_TextIOWrapper.read_w), + readline = interp2app(W_TextIOWrapper.readline_w), + + line_buffering = interp_attrproperty("line_buffering", W_TextIOWrapper), ) Added: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py ============================================================================== --- (empty file) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Thu Nov 18 14:50:17 2010 @@ -0,0 +1,21 @@ +from pypy.conftest import gettestobjspace + +class AppTestTextIO: + def setup_class(cls): + cls.space = gettestobjspace(usemodules=['_io']) + + def test_constructor(self): + import _io + r = _io.BytesIO(b"\xc3\xa9\n\n") + b = _io.BufferedReader(r, 1000) + t = _io.TextIOWrapper(b) + t.__init__(b, encoding="latin1", newline="\r\n") + assert t.encoding == "latin1" + assert t.line_buffering == False + t.__init__(b, encoding="utf8", line_buffering=True) + assert t.encoding == "utf8" + assert t.line_buffering == True + assert t.readline() == u"\xe9\n" + raises(TypeError, t.__init__, b, newline=42) + raises(ValueError, t.__init__, b, newline='xyzzy') + From arigo at codespeak.net Thu Nov 18 14:55:51 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 18 Nov 2010 14:55:51 +0100 (CET) Subject: [pypy-svn] r79249 - in pypy/branch/jit-free/pypy/jit: backend backend/llgraph backend/test backend/x86 backend/x86/test metainterp Message-ID: <20101118135551.EAD445080C@codespeak.net> Author: arigo Date: Thu Nov 18 14:55:49 2010 New Revision: 79249 Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py pypy/branch/jit-free/pypy/jit/backend/model.py pypy/branch/jit-free/pypy/jit/backend/test/runner_test.py pypy/branch/jit-free/pypy/jit/backend/x86/assembler.py pypy/branch/jit-free/pypy/jit/backend/x86/test/test_runner.py pypy/branch/jit-free/pypy/jit/metainterp/compile.py pypy/branch/jit-free/pypy/jit/metainterp/history.py pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Log: (antocuni, arigo) Refactoring the LoopToken class to not have a __del__ any more. Now the __del__ is on a small class CompiledLoopToken that is attached by the backend when compiling a new loop. Solves the issue of weakrefs to objects that have a __del__. Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py Thu Nov 18 14:55:49 2010 @@ -857,13 +857,13 @@ global _last_exception loop_token = wref_loop_token() assert loop_token, "CALL_ASSEMBLER to a target that already died" - if hasattr(loop_token, '_llgraph_redirected'): - return self._do_call_assembler(loop_token._llgraph_redirected, - *args) + ctl = loop_token.compiled_loop_token + if hasattr(ctl, 'redirected'): + return self._do_call_assembler(ctl.redirected, *args) assert not self._forced self._may_force = self.opindex try: - inpargs = _from_opaque(loop_token._llgraph_compiled_version).inputargs + inpargs = _from_opaque(ctl.compiled_version).inputargs for i, inparg in enumerate(inpargs): TYPE = inparg.concretetype if TYPE is lltype.Signed: @@ -1556,11 +1556,13 @@ do_setfield_gc_int(vable, fielddescr.ofs, 0) def redirect_call_assembler(cpu, oldlooptoken, newlooptoken): - OLD = _from_opaque(oldlooptoken._llgraph_compiled_version).getargtypes() - NEW = _from_opaque(newlooptoken._llgraph_compiled_version).getargtypes() + oldclt = oldlooptoken.compiled_loop_token + newclt = newlooptoken.compiled_loop_token + OLD = _from_opaque(oldclt.compiled_version).getargtypes() + NEW = _from_opaque(newclt.compiled_version).getargtypes() assert OLD == NEW - assert not hasattr(oldlooptoken, '_llgraph_redirected') - oldlooptoken._llgraph_redirected = weakref.ref(newlooptoken) + assert not hasattr(oldclt, 'redirected') + oldclt.redirected = weakref.ref(newlooptoken) # ____________________________________________________________ Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py Thu Nov 18 14:55:49 2010 @@ -120,28 +120,30 @@ def compile_bridge(self, faildescr, inputargs, operations, original_loop_token, log=True): c = llimpl.compile_start() - original_loop_token._llgraph_loop_and_bridges.append(c) + clt = original_loop_token.compiled_loop_token + clt.loop_and_bridges.append(c) + clt.bridges_count += 1 self._compile_loop_or_bridge(c, inputargs, operations) old, oldindex = faildescr._compiled_fail llimpl.compile_redirect_fail(old, oldindex, c) - def compile_loop(self, inputargs, operations, loopdescr, log=True): + def compile_loop(self, inputargs, operations, looptoken, log=True): """In a real assembler backend, this should assemble the given list of operations. Here we just generate a similar CompiledLoop instance. The code here is RPython, whereas the code in llimpl is not. """ c = llimpl.compile_start() - if not we_are_translated(): - assert not hasattr(loopdescr, '_llgraph_loop_and_bridges') - loopdescr._llgraph_loop_and_bridges = [c] - loopdescr._llgraph_compiled_version = c + clt = model.CompiledLoopToken(self, looptoken.number) + clt.loop_and_bridges = [c] + clt.compiled_version = c + looptoken.compiled_loop_token = clt self._compile_loop_or_bridge(c, inputargs, operations) - def free_loop_and_bridges(self, looptoken): - for c in looptoken._llgraph_loop_and_bridges: + def free_loop_and_bridges(self, compiled_loop_token): + for c in compiled_loop_token.loop_and_bridges: llimpl.mark_as_free(c) - model.AbstractCPU.free_loop_and_bridges(self, looptoken) + model.AbstractCPU.free_loop_and_bridges(self, compiled_loop_token) def _compile_loop_or_bridge(self, c, inputargs, operations): var2index = {} @@ -216,7 +218,7 @@ if op.getopnum() == rop.JUMP: targettoken = op.getdescr() assert isinstance(targettoken, history.LoopToken) - compiled_version = targettoken._llgraph_compiled_version + compiled_version = targettoken.compiled_loop_token.compiled_version llimpl.compile_add_jump_target(c, compiled_version) elif op.getopnum() == rop.FINISH: faildescr = op.getdescr() @@ -226,7 +228,7 @@ assert False, "unknown operation" def _execute_token(self, loop_token): - compiled_version = loop_token._llgraph_compiled_version + compiled_version = loop_token.compiled_loop_token.compiled_version frame = llimpl.new_frame(self.is_oo, self) # setup the frame llimpl.frame_clear(frame, compiled_version) Modified: pypy/branch/jit-free/pypy/jit/backend/model.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/model.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/model.py Thu Nov 18 14:55:49 2010 @@ -1,3 +1,4 @@ +from pypy.rlib.debug import debug_start, debug_print, debug_stop from pypy.jit.metainterp import history, compile @@ -41,8 +42,9 @@ def compile_loop(self, inputargs, operations, looptoken, log=True): """Assemble the given loop. - Extra attributes should be put in the LoopToken to - point to the compiled loop in assembler. + Should create and attach a fresh CompiledLoopToken to + looptoken.compiled_loop_token and stick extra attributes + on it to point to the compiled loop in assembler. """ raise NotImplementedError @@ -120,7 +122,7 @@ oldlooptoken so that from now own they will call newlooptoken.""" raise NotImplementedError - def free_loop_and_bridges(self, looptoken): + def free_loop_and_bridges(self, compiled_loop_token): """This method is called to free resources (machine code, references to resume guards, etc.) allocated by the compilation of a loop and all bridges attached to it. After this call, the @@ -133,10 +135,10 @@ # resume descrs are the largest consumers of memory (about 3x # more than the assembler, in the case of the x86 backend). lst = self.fail_descr_list - for n in looptoken.faildescr_indices: + for n in compiled_loop_token.faildescr_indices: lst[n] = None - self.fail_descr_free_list.extend(looptoken.faildescr_indices) - # We expect 'looptoken' to be itself garbage-collected soon. + self.fail_descr_free_list.extend(compiled_loop_token.faildescr_indices) + # We expect 'compiled_loop_token' to be itself garbage-collected soon. @staticmethod def sizeof(S): @@ -262,3 +264,24 @@ def force(self, force_token): raise NotImplementedError + + +class CompiledLoopToken(object): + def __init__(self, cpu, number): + self.cpu = cpu + self.number = number + self.bridges_count = 0 + # This growing list gives the 'descr_number' of all fail descrs + # that belong to this loop or to a bridge attached to it. + # Filled by the frontend calling record_faildescr_index(). + self.faildescr_indices = [] + + def record_faildescr_index(self, n): + self.faildescr_indices.append(n) + + def __del__(self): + debug_start("jit-free-looptoken") + debug_print("Freeing loop #", self.number, 'with', + self.bridges_count, 'attached bridges') + self.cpu.free_loop_and_bridges(self) + debug_stop("jit-free-looptoken") Modified: pypy/branch/jit-free/pypy/jit/backend/test/runner_test.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/test/runner_test.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/test/runner_test.py Thu Nov 18 14:55:49 2010 @@ -199,7 +199,7 @@ ] bridge[1].setfailargs([i1b]) - self.cpu.compile_bridge(faildescr1, [i1b], bridge) + self.cpu.compile_bridge(faildescr1, [i1b], bridge, looptoken) self.cpu.set_future_value_int(0, 2) fail = self.cpu.execute_token(looptoken) @@ -233,7 +233,7 @@ ] bridge[1].setfailargs([i1b]) - self.cpu.compile_bridge(faildescr1, [i1b], bridge) + self.cpu.compile_bridge(faildescr1, [i1b], bridge, looptoken) self.cpu.set_future_value_int(0, 2) fail = self.cpu.execute_token(looptoken) @@ -1050,7 +1050,7 @@ ResOperation(rop.JUMP, [f3] + fboxes2[1:], None, descr=looptoken), ] - self.cpu.compile_bridge(faildescr1, fboxes2, bridge) + self.cpu.compile_bridge(faildescr1, fboxes2, bridge, looptoken) for i in range(len(fboxes)): self.cpu.set_future_value_float(i, 13.5 + 6.73 * i) Modified: pypy/branch/jit-free/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/x86/assembler.py Thu Nov 18 14:55:49 2010 @@ -7,6 +7,7 @@ from pypy.rpython.lltypesystem.lloperation import llop from pypy.rpython.annlowlevel import llhelper from pypy.tool.uid import fixid +from pypy.jit.backend.model import CompiledLoopToken from pypy.jit.backend.x86.regalloc import (RegAlloc, X86RegisterManager, X86XMMRegisterManager, get_ebp_ofs, _get_scale) @@ -305,6 +306,8 @@ _x86_arglocs _x86_debug_checksum """ + looptoken.compiled_loop_token = CompiledLoopToken(self.cpu, + looptoken.number) if not we_are_translated(): # Arguments should be unique assert len(set(inputargs)) == len(inputargs) Modified: pypy/branch/jit-free/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/x86/test/test_runner.py Thu Nov 18 14:55:49 2010 @@ -338,6 +338,7 @@ faildescr1 = BasicFailDescr(1) faildescr2 = BasicFailDescr(2) looptoken = LoopToken() + looptoken.number = 17 class FakeString(object): def __init__(self, val): self.val = val @@ -356,7 +357,7 @@ operations[3].setfailargs([i1]) self.cpu.compile_loop(inputargs, operations, looptoken) name, loopaddress, loopsize = agent.functions[0] - assert name == "Loop # 0: hello" + assert name == "Loop # 17: hello" assert loopaddress <= looptoken._x86_loop_code assert loopsize >= 40 # randomish number Modified: pypy/branch/jit-free/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/compile.py Thu Nov 18 14:55:49 2010 @@ -42,8 +42,8 @@ name = metainterp.staticdata.stats.name_for_new_loop() return TreeLoop(name) -def make_loop_token(cpu, nb_args, jitdriver_sd): - loop_token = LoopToken(cpu) +def make_loop_token(nb_args, jitdriver_sd): + loop_token = LoopToken() loop_token.specnodes = [prebuiltNotSpecNode] * nb_args loop_token.outermost_jitdriver_sd = jitdriver_sd return loop_token @@ -61,8 +61,8 @@ if isinstance(descr, ResumeDescr): descr.wref_original_loop_token = wref # stick it there n = descr.index - if n >= 0: # we also record the resumedescr in this list - looptoken.faildescr_indices.append(n) + if n >= 0: # we also record the resumedescr number + looptoken.compiled_loop_token.record_faildescr_index(n) elif isinstance(descr, LoopToken): # for a JUMP or a CALL_ASSEMBLER: record it as a potential jump. # (the following test is not enough to prevent more complicated @@ -89,9 +89,8 @@ h_ops = history.operations loop.operations = [h_ops[i].clone() for i in range(start, len(h_ops))] metainterp_sd = metainterp.staticdata - cpu = metainterp.cpu jitdriver_sd = metainterp.jitdriver_sd - loop_token = make_loop_token(cpu, len(loop.inputargs), jitdriver_sd) + loop_token = make_loop_token(len(loop.inputargs), jitdriver_sd) loop.token = loop_token loop.operations[-1].setdescr(loop_token) # patch the target of the JUMP try: @@ -155,7 +154,6 @@ if not we_are_translated(): show_loop(metainterp_sd) TreeLoop.check_consistency_of(inputargs, operations) - original_loop_token.bridges_count += 1 metainterp_sd.profiler.start_backend() debug_start("jit-backend") try: @@ -512,12 +510,11 @@ # a loop at all but ends in a jump to the target loop. It starts # with completely unoptimized arguments, as in the interpreter. metainterp_sd = metainterp.staticdata - cpu = metainterp.cpu jitdriver_sd = metainterp.jitdriver_sd redargs = new_loop.inputargs # We make a new LoopToken for this entry bridge, and stick it # to every guard in the loop. - new_loop_token = make_loop_token(cpu, len(redargs), jitdriver_sd) + new_loop_token = make_loop_token(len(redargs), jitdriver_sd) new_loop.token = new_loop_token send_loop_to_backend(metainterp_sd, new_loop, "entry bridge") # send the new_loop to warmspot.py, to be called directly the next time @@ -601,7 +598,7 @@ """ # 'redboxes' is only used to know the types of red arguments. inputargs = [box.clonebox() for box in redboxes] - loop_token = make_loop_token(cpu, len(inputargs), jitdriver_sd) + loop_token = make_loop_token(len(inputargs), jitdriver_sd) # 'nb_red_args' might be smaller than len(redboxes), # because it doesn't include the virtualizable boxes. nb_red_args = jitdriver_sd.num_red_args Modified: pypy/branch/jit-free/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/history.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/history.py Thu Nov 18 14:55:49 2010 @@ -5,7 +5,6 @@ from pypy.rlib.objectmodel import we_are_translated, r_dict, Symbolic from pypy.rlib.objectmodel import compute_hash, compute_unique_id from pypy.rlib.rarithmetic import intmask, r_longlong -from pypy.rlib.debug import debug_start, debug_print, debug_stop from pypy.tool.uid import uid from pypy.conftest import option @@ -731,40 +730,28 @@ outermost_jitdriver_sd = None # specnodes = ... # and more data specified by the backend when the loop is compiled - cpu = None number = -1 generation = r_longlong(0) - bridges_count = 0 + # one purpose of LoopToken is to keep alive the CompiledLoopToken + # returned by the backend. When the LoopToken goes away, the + # CompiledLoopToken has its __del__ called, which frees the assembler + # memory and the ResumeGuards. + compiled_loop_token = None - def __init__(self, cpu=None): - self.cpu = cpu - # See get_fail_descr_number() in backend/model.py: this growing - # list gives the 'descr_number' of all fail descrs that belong to - # this loop or to a bridge attached to it. - self.faildescr_indices = [] + def __init__(self): # For memory management of assembled loops self._keepalive_target_looktokens = {} # set of other LoopTokens def record_jump_to(self, target_loop_token): self._keepalive_target_looktokens[target_loop_token] = None - def __del__(self): - if self.generation > r_longlong(0): - # MemoryManager.keep_loop_alive() has been called on this - # loop token, which means that it has been successfully - # compiled by the backend. Free it now. - debug_start("jit-free-looptoken") - debug_print("Freeing loop #", self.number, 'with', - self.bridges_count, 'attached bridges') - self.cpu.free_loop_and_bridges(self) - debug_stop("jit-free-looptoken") - def __repr__(self): return '' % (self.number, self.generation) def repr_of_descr(self): return '' % self.number + class TreeLoop(object): inputargs = None operations = None Modified: pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py Thu Nov 18 14:55:49 2010 @@ -1,4 +1,4 @@ -import py, os, sys, weakref +import py, os, sys from pypy.rpython.lltypesystem import lltype, llmemory, rclass from pypy.rlib.objectmodel import we_are_translated from pypy.rlib.unroll import unrolling_iterable Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmspot.py Thu Nov 18 14:55:49 2010 @@ -816,10 +816,6 @@ # ____________________________________________________________ def execute_token(self, loop_token): - self.metainterp_sd.profiler.start_running() - debug_start("jit-running") fail_descr = self.cpu.execute_token(loop_token) - debug_stop("jit-running") - self.metainterp_sd.profiler.end_running() self.memory_manager.keep_loop_alive(loop_token) return fail_descr Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Thu Nov 18 14:55:49 2010 @@ -344,7 +344,11 @@ # ---------- execute assembler ---------- while True: # until interrupted by an exception + metainterp_sd.profiler.start_running() + debug_start("jit-running") fail_descr = warmrunnerdesc.execute_token(loop_token) + debug_stop("jit-running") + metainterp_sd.profiler.end_running() loop_token = None # for test_memmgr if vinfo is not None: vinfo.reset_vable_token(virtualizable) From david at codespeak.net Thu Nov 18 15:12:24 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 18 Nov 2010 15:12:24 +0100 (CET) Subject: [pypy-svn] r79250 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101118141224.AD6B936C222@codespeak.net> Author: david Date: Thu Nov 18 15:12:23 2010 New Revision: 79250 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Fix guard_value checking a register when the second argument is immediate Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Thu Nov 18 15:12:23 2010 @@ -48,7 +48,6 @@ l1 = regalloc.make_sure_var_in_reg(a1, forbidden_vars=[a0], imm_fine=False) res = regalloc.force_allocate_reg(op.result, forbidden_vars=[a0, a1]) self.mc.ADD_rr(res.value, l0.value, l1.value, s=1) - regalloc.possibly_free_vars_for_op(op) return fcond @@ -151,13 +150,13 @@ #XXX check for a better way of doing this def emit_op_int_neg(self, op, regalloc, fcond): - arg = op.getarg(0) - l0 = regalloc.make_sure_var_in_reg(arg, imm_fine=False) - l1 = regalloc.make_sure_var_in_reg(ConstInt(-1), [arg], imm_fine=False) - res = regalloc.force_allocate_reg(op.result, [arg]) - self.mc.MUL(res.value, l0.value, l1.value) - regalloc.possibly_free_vars([l0, l1, res]) - return fcond + arg = op.getarg(0) + l0 = regalloc.make_sure_var_in_reg(arg, imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(ConstInt(-1), [arg], imm_fine=False) + res = regalloc.force_allocate_reg(op.result, [arg]) + self.mc.MUL(res.value, l0.value, l1.value) + regalloc.possibly_free_vars([l0, l1, res]) + return fcond class GuardOpAssembler(object): @@ -194,7 +193,7 @@ l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) l1 = regalloc.make_sure_var_in_reg(a1) if l1.is_imm(): - self.mc.CMP_rr(l0.value, l1.getint()) + self.mc.CMP_ri(l0.value, l1.getint()) else: self.mc.CMP_rr(l0.value, l1.value) regalloc.possibly_free_vars_for_op(op) @@ -299,6 +298,7 @@ else: self.mc.MOV_rr(resloc.value, argloc.value) regalloc.possibly_free_vars_for_op(op) + return fcond class FieldOpAssembler(object): From tismer at codespeak.net Thu Nov 18 15:15:22 2010 From: tismer at codespeak.net (tismer at codespeak.net) Date: Thu, 18 Nov 2010 15:15:22 +0100 (CET) Subject: [pypy-svn] r79251 - pypy/extradoc/planning/hg-migration Message-ID: <20101118141522.04ABA5080C@codespeak.net> Author: tismer Date: Thu Nov 18 15:15:21 2010 New Revision: 79251 Modified: pypy/extradoc/planning/hg-migration/usermap.txt Log: added tismer's email Modified: pypy/extradoc/planning/hg-migration/usermap.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/usermap.txt (original) +++ pypy/extradoc/planning/hg-migration/usermap.txt Thu Nov 18 15:15:21 2010 @@ -13,7 +13,7 @@ hpk=Holger Krekel mwh=Michael Hudson afa=Amaury Forgeot d'Arc -tismer=Christian Tismer +tismer=Christian Tismer benjamin=Benjamin Peterson ericvrp=Eric van Riet Paap ac=Anders Chrigstrom From cfbolz at codespeak.net Thu Nov 18 15:17:06 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 18 Nov 2010 15:17:06 +0100 (CET) Subject: [pypy-svn] r79252 - pypy/extradoc/talk/pepm2011/figures Message-ID: <20101118141706.86A2E282BEB@codespeak.net> Author: cfbolz Date: Thu Nov 18 15:17:05 2010 New Revision: 79252 Modified: pypy/extradoc/talk/pepm2011/figures/obj-lifetime.pdf pypy/extradoc/talk/pepm2011/figures/obj-lifetime.svg Log: fix figure to not use transparency, which doesn't work in .ps files Modified: pypy/extradoc/talk/pepm2011/figures/obj-lifetime.pdf ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/pepm2011/figures/obj-lifetime.svg ============================================================================== --- pypy/extradoc/talk/pepm2011/figures/obj-lifetime.svg (original) +++ pypy/extradoc/talk/pepm2011/figures/obj-lifetime.svg Thu Nov 18 15:17:05 2010 @@ -9,12 +9,12 @@ xmlns="http://www.w3.org/2000/svg" xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" - width="744.09448819" - height="1052.3622047" + width="264.45795" + height="603.8692" id="svg2" version="1.1" - inkscape:version="0.47 r22583" - sodipodi:docname="loop.svg" + inkscape:version="0.48.0 r9654" + sodipodi:docname="obj-lifetime.pdf" inkscape:export-filename="/home/cfbolz/code/userdir/blog/figures/loop.png" inkscape:export-xdpi="90.25" inkscape:export-ydpi="90.25"> @@ -23,15 +23,16 @@ + style="overflow:visible"> + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt;marker-start:none" + transform="matrix(-0.8,0,0,-0.8,-10,0)" + inkscape:connector-curvature="0" /> + inkscape:snap-grids="true" + fit-margin-top="0" + fit-margin-left="0" + fit-margin-right="0" + fit-margin-bottom="0" /> @@ -236,149 +241,145 @@ + id="layer1" + transform="translate(-2.1213202,-3.7228303)"> - - - - new - - - - new - - - - new - - - - new - - - - - - new - - - objectescapes - - + + new + + new + + new + + new + + + + new + + + objectescapes + jump - - - - - + + new - - - - 4 - - - - new + + + 4 + + + new - - - - new + + + + new - - - - new + + + + new - - - - - - new + + + + + + new - - - new + + + objectescapes - - - 1 - - - - 1 - - - - - 2 - - - - 3 - + x="107.47532" + id="tspan3639-1" + sodipodi:role="line">objectescapes + + 1 + + 1 + + 2 + + 3 jump + sodipodi:nodetypes="cc" + inkscape:connector-curvature="0" /> From david at codespeak.net Thu Nov 18 15:27:14 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 18 Nov 2010 15:27:14 +0100 (CET) Subject: [pypy-svn] r79254 - pypy/branch/arm-backend/pypy/jit/backend/arm/helper Message-ID: <20101118142714.6F5735080E@codespeak.net> Author: david Date: Thu Nov 18 15:27:13 2010 New Revision: 79254 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Log: Fix result handling when calling helper functions for arithmetic operations Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Thu Nov 18 15:27:13 2010 @@ -4,6 +4,7 @@ def gen_emit_op_unary_cmp(true_cond, false_cond): def f(self, op, regalloc, fcond): + assert fcond is not None a0 = op.getarg(0) reg = regalloc.make_sure_var_in_reg(a0, imm_fine=False) res = regalloc.force_allocate_reg(op.result, [a0]) @@ -16,6 +17,7 @@ def gen_emit_op_ri(opname, imm_size=0xFF, commutative=True, allow_zero=True): def f(self, op, regalloc, fcond): + assert fcond is not None ri_op = getattr(self.mc, '%s_ri' % opname) rr_op = getattr(self.mc, '%s_rr' % opname) @@ -44,20 +46,23 @@ def gen_emit_op_by_helper_call(opname): def f(self, op, regalloc, fcond): + assert fcond is not None a0 = op.getarg(0) a1 = op.getarg(1) arg1 = regalloc.make_sure_var_in_reg(a0, selected_reg=r.r0, imm_fine=False) arg2 = regalloc.make_sure_var_in_reg(a1, [a0], selected_reg=r.r1, imm_fine=False) assert arg1 == r.r0 assert arg2 == r.r1 - res = regalloc.force_allocate_reg(op.result, selected_reg=r.r0) + regalloc.before_call() getattr(self.mc, opname)(fcond) + regalloc.after_call(op.result) regalloc.possibly_free_vars_for_op(op) return fcond return f def gen_emit_cmp_op(condition, inverse=False): def f(self, op, regalloc, fcond): + assert fcond is not None if not inverse: arg0 = op.getarg(0) arg1 = op.getarg(1) From david at codespeak.net Thu Nov 18 15:30:14 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 18 Nov 2010 15:30:14 +0100 (CET) Subject: [pypy-svn] r79255 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101118143014.50D5E282BEF@codespeak.net> Author: david Date: Thu Nov 18 15:30:12 2010 New Revision: 79255 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Log: Improve argument decoding when entering a bridge Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Thu Nov 18 15:30:12 2010 @@ -77,7 +77,7 @@ stack_loc = self.decode32(enc, i+1) value = self.decode32(stack, (frame_depth - stack_loc)*WORD) i += 4 - else: # an int for now + else: # an int in a reg location reg = ord(enc[i]) value = self.decode32(regs, reg*WORD) @@ -94,6 +94,31 @@ self.fail_boxes_count = fail_index return descr + def update_bridge_bindings(self, enc, inputargs, regalloc): + # first word contains frame depth + j = 4 + for i in range(len(inputargs)): + res = enc[j] + if res == '\xFF': + assert 0, 'reached end of encoded area' + # XXX decode imm and and stack locs and REFs + while res == '\xFE': + j += 1 + res = enc[j] + + assert res == '\xEF' # only int location for now + j += 1 + res = enc[j] + if res == '\xFD': # imm location + assert 0, 'fail' + elif res == '\xFC': # stack location + stack_loc = self.decode32(enc, j+1) + loc = regalloc.make_sure_var_in_reg(inputargs[i]) + self.mc.LDR_ri(loc.value, r.fp.value, -stack_loc) + j += 4 + else: # reg location + regalloc.force_allocate_reg(inputargs[i], selected_reg=r.all_regs[ord(res)]) + j += 1 def decode32(self, mem, index): highval = ord(mem[index+3]) if highval >= 128: @@ -271,7 +296,7 @@ loop_head=self.mc.curraddr() looptoken._arm_bootstrap_code = loop_start looptoken._arm_loop_code = loop_head - print inputargs, operations + print 'Loop', inputargs, operations self._walk_operations(operations, regalloc) self._patch_sp_offset(sp_patch_location, regalloc) @@ -353,11 +378,12 @@ longevity = compute_vars_longevity(inputargs, operations) regalloc = ARMRegisterManager(longevity, assembler=self, frame_manager=ARMFrameManager()) - regalloc.update_bindings(enc, inputargs) bridge_head = self.mc.curraddr() + self.update_bridge_bindings(enc, inputargs, regalloc) + print 'Bridge', inputargs, operations self._walk_operations(operations, regalloc) - self.gen_func_epilog() + print 'Done building bridges' self.patch_trace(faildescr, bridge_head) print 'Done patching trace' Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Thu Nov 18 15:30:12 2010 @@ -14,18 +14,6 @@ def __init__(self, longevity, frame_manager=None, assembler=None): RegisterManager.__init__(self, longevity, frame_manager, assembler) - def update_bindings(self, enc, inputargs): - # first word contains frame depth - j = 4 - for i in range(len(inputargs)): - # XXX decode imm and and stack locs and REFs - while enc[j] == '\xFE': - j += 1 - assert enc[j] == '\xEF' - j += 1 - self.force_allocate_reg(inputargs[i], selected_reg=r.all_regs[ord(enc[j])]) - j += 1 - def convert_to_imm(self, c): if isinstance(c, ConstInt): return locations.ImmLocation(c.value) From cfbolz at codespeak.net Thu Nov 18 15:44:08 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 18 Nov 2010 15:44:08 +0100 (CET) Subject: [pypy-svn] r79256 - pypy/extradoc/talk/pepm2011 Message-ID: <20101118144408.9F94350812@codespeak.net> Author: cfbolz Date: Thu Nov 18 15:44:07 2010 New Revision: 79256 Modified: pypy/extradoc/talk/pepm2011/paper.tex Log: don't force inclusion of pdf version Modified: pypy/extradoc/talk/pepm2011/paper.tex ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.tex (original) +++ pypy/extradoc/talk/pepm2011/paper.tex Thu Nov 18 15:44:07 2010 @@ -510,7 +510,7 @@ \begin{figure} \begin{center} -\includegraphics[scale=0.6]{figures/obj-lifetime.pdf} +\includegraphics[scale=0.6]{figures/obj-lifetime} \end{center} \caption{Object Lifetimes in a Trace} From cfbolz at codespeak.net Thu Nov 18 16:09:35 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 18 Nov 2010 16:09:35 +0100 (CET) Subject: [pypy-svn] r79257 - pypy/extradoc/talk/pepm2011 Message-ID: <20101118150935.7CB92282BEF@codespeak.net> Author: cfbolz Date: Thu Nov 18 16:09:33 2010 New Revision: 79257 Added: pypy/extradoc/talk/pepm2011/paper.pdf (contents, props changed) pypy/extradoc/talk/pepm2011/paper.ps Log: if everything goes right, I will use those as final versions Added: pypy/extradoc/talk/pepm2011/paper.pdf ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/pepm2011/paper.ps ============================================================================== --- (empty file) +++ pypy/extradoc/talk/pepm2011/paper.ps Thu Nov 18 16:09:33 2010 @@ -0,0 +1,9567 @@ +%!PS-Adobe-2.0 +%%Creator: dvips(k) 5.98 Copyright 2009 Radical Eye Software +%%Title: paper.dvi +%%CreationDate: Thu Nov 18 16:06:27 2010 +%%Pages: 10 +%%PageOrder: Ascend +%%BoundingBox: 0 0 612 792 +%%DocumentFonts: Times-Bold Times-Roman CMMI8 CMMI6 SFSS0900 +%%+ Times-BoldItalic Times-Italic CMSY7 BeraSansMono-Roman +%%+ BeraSansMono-Bold CMMI9 CMR6 CMSY8 BeraSansMono-Oblique CMMI7 CMR5 +%%+ CMR9 CMSY9 CMTT9 CMEX9 EUFM10 CMSY6 CMSY5 +%%DocumentPaperSizes: Letter +%%EndComments +%DVIPSWebPage: (www.radicaleye.com) +%DVIPSCommandLine: dvips -P pdf -t letter -o paper.ps paper.dvi +%DVIPSParameters: dpi=8000 +%DVIPSSource: TeX output 2010.11.18:1606 +%%BeginProcSet: tex.pro 0 0 +%! +/TeXDict 300 dict def TeXDict begin/N{def}def/B{bind def}N/S{exch}N/X{S +N}B/A{dup}B/TR{translate}N/isls false N/vsize 11 72 mul N/hsize 8.5 72 +mul N/landplus90{false}def/@rigin{isls{[0 landplus90{1 -1}{-1 1}ifelse 0 +0 0]concat}if 72 Resolution div 72 VResolution div neg scale isls{ +landplus90{VResolution 72 div vsize mul 0 exch}{Resolution -72 div hsize +mul 0}ifelse TR}if Resolution VResolution vsize -72 div 1 add mul TR[ +matrix currentmatrix{A A round sub abs 0.00001 lt{round}if}forall round +exch round exch]setmatrix}N/@landscape{/isls true N}B/@manualfeed{ +statusdict/manualfeed true put}B/@copies{/#copies X}B/FMat[1 0 0 -1 0 0] +N/FBB[0 0 0 0]N/nn 0 N/IEn 0 N/ctr 0 N/df-tail{/nn 8 dict N nn begin +/FontType 3 N/FontMatrix fntrx N/FontBBox FBB N string/base X array +/BitMaps X/BuildChar{CharBuilder}N/Encoding IEn N end A{/foo setfont}2 +array copy cvx N load 0 nn put/ctr 0 N[}B/sf 0 N/df{/sf 1 N/fntrx FMat N +df-tail}B/dfs{div/sf X/fntrx[sf 0 0 sf neg 0 0]N df-tail}B/E{pop nn A +definefont setfont}B/Cw{Cd A length 5 sub get}B/Ch{Cd A length 4 sub get +}B/Cx{128 Cd A length 3 sub get sub}B/Cy{Cd A length 2 sub get 127 sub} +B/Cdx{Cd A length 1 sub get}B/Ci{Cd A type/stringtype ne{ctr get/ctr ctr +1 add N}if}B/CharBuilder{save 3 1 roll S A/base get 2 index get S +/BitMaps get S get/Cd X pop/ctr 0 N Cdx 0 Cx Cy Ch sub Cx Cw add Cy +setcachedevice Cw Ch true[1 0 0 -1 -.1 Cx sub Cy .1 sub]{Ci}imagemask +restore}B/D{/cc X A type/stringtype ne{]}if nn/base get cc ctr put nn +/BitMaps get S ctr S sf 1 ne{A A length 1 sub A 2 index S get sf div put +}if put/ctr ctr 1 add N}B/I{cc 1 add D}B/bop{userdict/bop-hook known{ +bop-hook}if/SI save N @rigin 0 0 moveto/V matrix currentmatrix A 1 get A +mul exch 0 get A mul add .99 lt{/QV}{/RV}ifelse load def pop pop}N/eop{ +SI restore userdict/eop-hook known{eop-hook}if showpage}N/@start{ +userdict/start-hook known{start-hook}if pop/VResolution X/Resolution X +1000 div/DVImag X/IEn 256 array N 2 string 0 1 255{IEn S A 360 add 36 4 +index cvrs cvn put}for pop 65781.76 div/vsize X 65781.76 div/hsize X}N +/p{show}N/RMat[1 0 0 -1 0 0]N/BDot 260 string N/Rx 0 N/Ry 0 N/V{}B/RV/v{ +/Ry X/Rx X V}B statusdict begin/product where{pop false[(Display)(NeXT) +(LaserWriter 16/600)]{A length product length le{A length product exch 0 +exch getinterval eq{pop true exit}if}{pop}ifelse}forall}{false}ifelse +end{{gsave TR -.1 .1 TR 1 1 scale Rx Ry false RMat{BDot}imagemask +grestore}}{{gsave TR -.1 .1 TR Rx Ry scale 1 1 false RMat{BDot} +imagemask grestore}}ifelse B/QV{gsave newpath transform round exch round +exch itransform moveto Rx 0 rlineto 0 Ry neg rlineto Rx neg 0 rlineto +fill grestore}B/a{moveto}B/delta 0 N/tail{A/delta X 0 rmoveto}B/M{S p +delta add tail}B/b{S p tail}B/c{-4 M}B/d{-3 M}B/e{-2 M}B/f{-1 M}B/g{0 M} +B/h{1 M}B/i{2 M}B/j{3 M}B/k{4 M}B/w{0 rmoveto}B/l{p -4 w}B/m{p -3 w}B/n{ +p -2 w}B/o{p -1 w}B/q{p 1 w}B/r{p 2 w}B/s{p 3 w}B/t{p 4 w}B/x{0 S +rmoveto}B/y{3 2 roll p a}B/bos{/SS save N}B/eos{SS restore}B end + +%%EndProcSet +%%BeginProcSet: alt-rule.pro 0 0 +%! +% Patch by TVZ +% Makes dvips files draw rules with stroke rather than fill. +% Makes narrow rules more predictable at low resolutions +% after distilling to PDF. +% May have unknown consequences for very thick rules. +% Tested only with dvips 5.85(k). +TeXDict begin +/QV { + gsave newpath /ruleY X /ruleX X + Rx Ry gt + { ruleX ruleY Ry 2 div sub moveto Rx 0 rlineto Ry } + { ruleX Rx 2 div add ruleY moveto 0 Ry neg rlineto Rx } + ifelse + setlinewidth 0 setlinecap stroke grestore +} bind def +end + +%%EndProcSet +%%BeginProcSet: 8r.enc 0 0 +% File 8r.enc TeX Base 1 Encoding Revision 2.0 2002-10-30 +% +% @@psencodingfile@{ +% author = "S. Rahtz, P. MacKay, Alan Jeffrey, B. Horn, K. Berry, +% W. Schmidt, P. Lehman", +% version = "2.0", +% date = "27nov06", +% filename = "8r.enc", +% email = "tex-fonts@@tug.org", +% docstring = "This is the encoding vector for Type1 and TrueType +% fonts to be used with TeX. This file is part of the +% PSNFSS bundle, version 9" +% @} +% +% The idea is to have all the characters normally included in Type 1 fonts +% available for typesetting. This is effectively the characters in Adobe +% Standard encoding, ISO Latin 1, Windows ANSI including the euro symbol, +% MacRoman, and some extra characters from Lucida. +% +% Character code assignments were made as follows: +% +% (1) the Windows ANSI characters are almost all in their Windows ANSI +% positions, because some Windows users cannot easily reencode the +% fonts, and it makes no difference on other systems. The only Windows +% ANSI characters not available are those that make no sense for +% typesetting -- rubout (127 decimal), nobreakspace (160), softhyphen +% (173). quotesingle and grave are moved just because it's such an +% irritation not having them in TeX positions. +% +% (2) Remaining characters are assigned arbitrarily to the lower part +% of the range, avoiding 0, 10 and 13 in case we meet dumb software. +% +% (3) Y&Y Lucida Bright includes some extra text characters; in the +% hopes that other PostScript fonts, perhaps created for public +% consumption, will include them, they are included starting at 0x12. +% These are /dotlessj /ff /ffi /ffl. +% +% (4) hyphen appears twice for compatibility with both ASCII and Windows. +% +% (5) /Euro was assigned to 128, as in Windows ANSI +% +% (6) Missing characters from MacRoman encoding incorporated as follows: +% +% PostScript MacRoman TeXBase1 +% -------------- -------------- -------------- +% /notequal 173 0x16 +% /infinity 176 0x17 +% /lessequal 178 0x18 +% /greaterequal 179 0x19 +% /partialdiff 182 0x1A +% /summation 183 0x1B +% /product 184 0x1C +% /pi 185 0x1D +% /integral 186 0x81 +% /Omega 189 0x8D +% /radical 195 0x8E +% /approxequal 197 0x8F +% /Delta 198 0x9D +% /lozenge 215 0x9E +% +/TeXBase1Encoding [ +% 0x00 + /.notdef /dotaccent /fi /fl + /fraction /hungarumlaut /Lslash /lslash + /ogonek /ring /.notdef /breve + /minus /.notdef /Zcaron /zcaron +% 0x10 + /caron /dotlessi /dotlessj /ff + /ffi /ffl /notequal /infinity + /lessequal /greaterequal /partialdiff /summation + /product /pi /grave /quotesingle +% 0x20 + /space /exclam /quotedbl /numbersign + /dollar /percent /ampersand /quoteright + /parenleft /parenright /asterisk /plus + /comma /hyphen /period /slash +% 0x30 + /zero /one /two /three + /four /five /six /seven + /eight /nine /colon /semicolon + /less /equal /greater /question +% 0x40 + /at /A /B /C + /D /E /F /G + /H /I /J /K + /L /M /N /O +% 0x50 + /P /Q /R /S + /T /U /V /W + /X /Y /Z /bracketleft + /backslash /bracketright /asciicircum /underscore +% 0x60 + /quoteleft /a /b /c + /d /e /f /g + /h /i /j /k + /l /m /n /o +% 0x70 + /p /q /r /s + /t /u /v /w + /x /y /z /braceleft + /bar /braceright /asciitilde /.notdef +% 0x80 + /Euro /integral /quotesinglbase /florin + /quotedblbase /ellipsis /dagger /daggerdbl + /circumflex /perthousand /Scaron /guilsinglleft + /OE /Omega /radical /approxequal +% 0x90 + /.notdef /.notdef /.notdef /quotedblleft + /quotedblright /bullet /endash /emdash + /tilde /trademark /scaron /guilsinglright + /oe /Delta /lozenge /Ydieresis +% 0xA0 + /.notdef /exclamdown /cent /sterling + /currency /yen /brokenbar /section + /dieresis /copyright /ordfeminine /guillemotleft + /logicalnot /hyphen /registered /macron +% 0xB0 + /degree /plusminus /twosuperior /threesuperior + /acute /mu /paragraph /periodcentered + /cedilla /onesuperior /ordmasculine /guillemotright + /onequarter /onehalf /threequarters /questiondown +% 0xC0 + /Agrave /Aacute /Acircumflex /Atilde + /Adieresis /Aring /AE /Ccedilla + /Egrave /Eacute /Ecircumflex /Edieresis + /Igrave /Iacute /Icircumflex /Idieresis +% 0xD0 + /Eth /Ntilde /Ograve /Oacute + /Ocircumflex /Otilde /Odieresis /multiply + /Oslash /Ugrave /Uacute /Ucircumflex + /Udieresis /Yacute /Thorn /germandbls +% 0xE0 + /agrave /aacute /acircumflex /atilde + /adieresis /aring /ae /ccedilla + /egrave /eacute /ecircumflex /edieresis + /igrave /iacute /icircumflex /idieresis +% 0xF0 + /eth /ntilde /ograve /oacute + /ocircumflex /otilde /odieresis /divide + /oslash /ugrave /uacute /ucircumflex + /udieresis /yacute /thorn /ydieresis +] def + + +%%EndProcSet +%%BeginProcSet: cm-super-t1.enc 0 0 +% This file is generated from `T1uni.map' and `glyphlist.txt', `gl-other.txt' +% +% LIGKERN hyphen hyphen =: endash ; endash hyphen =: emdash ; +% LIGKERN quoteleft quoteleft =: quotedblleft ; +% LIGKERN quoteright quoteright =: quotedblright ; +% LIGKERN comma comma =: quotedblbase ; less less =: guillemotleft ; +% LIGKERN greater greater =: guillemotright ; +% LIGKERN f f =: ff ; f i =: fi ; f l =: fl ; ff i =: ffi ; ff l =: ffl ; +% +% LIGKERN space {} * ; * {} space ; zero {} * ; * {} zero ; +% LIGKERN one {} * ; * {} one ; two {} * ; * {} two ; +% LIGKERN three {} * ; * {} three ; four {} * ; * {} four ; +% LIGKERN five {} * ; * {} five ; six {} * ; * {} six ; +% LIGKERN seven {} * ; * {} seven ; eight {} * ; * {} eight ; +% LIGKERN nine {} * ; * {} nine ; +% +/T1Encoding [ +% 0x00 +/grave +/acute +/circumflex +/tilde +/dieresis +/hungarumlaut +/ring +/caron +/breve +/macron +/dotaccent +/cedilla +/ogonek +/quotesinglbase +/guilsinglleft +/guilsinglright +% 0x10 +/quotedblleft +/quotedblright +/quotedblbase +/guillemotleft +/guillemotright +/endash +/emdash +/afii61664 +/perthousandzero % PERTHOUSAND ZERO +/dotlessi +/dotlessj +/ff +/fi +/fl +/ffi +/ffl +% 0x20 +/uni2423 +/exclam +/quotedbl +/numbersign +/dollar +/percent +/ampersand +/quoteright +/parenleft +/parenright +/asterisk +/plus +/comma +/hyphen +/period +/slash +% 0x30 +/zero +/one +/two +/three +/four +/five +/six +/seven +/eight +/nine +/colon +/semicolon +/less +/equal +/greater +/question +% 0x40 +/at +/A +/B +/C +/D +/E +/F +/G +/H +/I +/J +/K +/L +/M +/N +/O +% 0x50 +/P +/Q +/R +/S +/T +/U +/V +/W +/X +/Y +/Z +/bracketleft +/backslash +/bracketright +/asciicircum +/underscore +% 0x60 +/quoteleft +/a +/b +/c +/d +/e +/f +/g +/h +/i +/j +/k +/l +/m +/n +/o +% 0x70 +/p +/q +/r +/s +/t +/u +/v +/w +/x +/y +/z +/braceleft +/bar +/braceright +/asciitilde +/hyphen.alt % HANGING HYPHEN +% 0x80 +/Abreve +/Aogonek +/Cacute +/Ccaron +/Dcaron +/Ecaron +/Eogonek +/Gbreve +/Lacute +/Lcaron +/Lslash +/Nacute +/Ncaron +/Eng +/Ohungarumlaut +/Racute +% 0x90 +/Rcaron +/Sacute +/Scaron +/Scedilla +/Tcaron +/Tcommaaccent +/Uhungarumlaut +/Uring +/Ydieresis +/Zacute +/Zcaron +/Zdotaccent +/IJ +/Idotaccent +/dcroat +/section +% 0xA0 +/abreve +/aogonek +/cacute +/ccaron +/dcaron +/ecaron +/eogonek +/gbreve +/lacute +/lcaron +/lslash +/nacute +/ncaron +/eng +/ohungarumlaut +/racute +% 0xB0 +/rcaron +/sacute +/scaron +/scedilla +/tcaron +/tcommaaccent +/uhungarumlaut +/uring +/ydieresis +/zacute +/zcaron +/zdotaccent +/ij +/exclamdown +/questiondown +/sterling +% 0xC0 +/Agrave +/Aacute +/Acircumflex +/Atilde +/Adieresis +/Aring +/AE +/Ccedilla +/Egrave +/Eacute +/Ecircumflex +/Edieresis +/Igrave +/Iacute +/Icircumflex +/Idieresis +% 0xD0 +/Eth +/Ntilde +/Ograve +/Oacute +/Ocircumflex +/Otilde +/Odieresis +/OE +/Oslash +/Ugrave +/Uacute +/Ucircumflex +/Udieresis +/Yacute +/Thorn +/SS % Germandbls +% 0xE0 +/agrave +/aacute +/acircumflex +/atilde +/adieresis +/aring +/ae +/ccedilla +/egrave +/eacute +/ecircumflex +/edieresis +/igrave +/iacute +/icircumflex +/idieresis +% 0xF0 +/eth +/ntilde +/ograve +/oacute +/ocircumflex +/otilde +/odieresis +/oe +/oslash +/ugrave +/uacute +/ucircumflex +/udieresis +/yacute +/thorn +/germandbls % or /germandbls.alt +] def + +%%EndProcSet +%%BeginProcSet: texps.pro 0 0 +%! +TeXDict begin/rf{findfont dup length 1 add dict begin{1 index/FID ne 2 +index/UniqueID ne and{def}{pop pop}ifelse}forall[1 index 0 6 -1 roll +exec 0 exch 5 -1 roll VResolution Resolution div mul neg 0 0]FontType 0 +ne{/Metrics exch def dict begin Encoding{exch dup type/integertype ne{ +pop pop 1 sub dup 0 le{pop}{[}ifelse}{FontMatrix 0 get div Metrics 0 get +div def}ifelse}forall Metrics/Metrics currentdict end def}{{1 index type +/nametype eq{exit}if exch pop}loop}ifelse[2 index currentdict end +definefont 3 -1 roll makefont/setfont cvx]cvx def}def/ObliqueSlant{dup +sin S cos div neg}B/SlantFont{4 index mul add}def/ExtendFont{3 -1 roll +mul exch}def/ReEncodeFont{CharStrings rcheck{/Encoding false def dup[ +exch{dup CharStrings exch known not{pop/.notdef/Encoding true def}if} +forall Encoding{]exch pop}{cleartomark}ifelse}if/Encoding exch def}def +end + +%%EndProcSet +%%BeginProcSet: special.pro 0 0 +%! +TeXDict begin/SDict 200 dict N SDict begin/@SpecialDefaults{/hs 612 N +/vs 792 N/ho 0 N/vo 0 N/hsc 1 N/vsc 1 N/ang 0 N/CLIP 0 N/rwiSeen false N +/rhiSeen false N/letter{}N/note{}N/a4{}N/legal{}N}B/@scaleunit 100 N +/@hscale{@scaleunit div/hsc X}B/@vscale{@scaleunit div/vsc X}B/@hsize{ +/hs X/CLIP 1 N}B/@vsize{/vs X/CLIP 1 N}B/@clip{/CLIP 2 N}B/@hoffset{/ho +X}B/@voffset{/vo X}B/@angle{/ang X}B/@rwi{10 div/rwi X/rwiSeen true N}B +/@rhi{10 div/rhi X/rhiSeen true N}B/@llx{/llx X}B/@lly{/lly X}B/@urx{ +/urx X}B/@ury{/ury X}B/magscale true def end/@MacSetUp{userdict/md known +{userdict/md get type/dicttype eq{userdict begin md length 10 add md +maxlength ge{/md md dup length 20 add dict copy def}if end md begin +/letter{}N/note{}N/legal{}N/od{txpose 1 0 mtx defaultmatrix dtransform S +atan/pa X newpath clippath mark{transform{itransform moveto}}{transform{ +itransform lineto}}{6 -2 roll transform 6 -2 roll transform 6 -2 roll +transform{itransform 6 2 roll itransform 6 2 roll itransform 6 2 roll +curveto}}{{closepath}}pathforall newpath counttomark array astore/gc xdf +pop ct 39 0 put 10 fz 0 fs 2 F/|______Courier fnt invertflag{PaintBlack} +if}N/txpose{pxs pys scale ppr aload pop por{noflips{pop S neg S TR pop 1 +-1 scale}if xflip yflip and{pop S neg S TR 180 rotate 1 -1 scale ppr 3 +get ppr 1 get neg sub neg ppr 2 get ppr 0 get neg sub neg TR}if xflip +yflip not and{pop S neg S TR pop 180 rotate ppr 3 get ppr 1 get neg sub +neg 0 TR}if yflip xflip not and{ppr 1 get neg ppr 0 get neg TR}if}{ +noflips{TR pop pop 270 rotate 1 -1 scale}if xflip yflip and{TR pop pop +90 rotate 1 -1 scale ppr 3 get ppr 1 get neg sub neg ppr 2 get ppr 0 get +neg sub neg TR}if xflip yflip not and{TR pop pop 90 rotate ppr 3 get ppr +1 get neg sub neg 0 TR}if yflip xflip not and{TR pop pop 270 rotate ppr +2 get ppr 0 get neg sub neg 0 S TR}if}ifelse scaleby96{ppr aload pop 4 +-1 roll add 2 div 3 1 roll add 2 div 2 copy TR .96 dup scale neg S neg S +TR}if}N/cp{pop pop showpage pm restore}N end}if}if}N/normalscale{ +Resolution 72 div VResolution 72 div neg scale magscale{DVImag dup scale +}if 0 setgray}N/psfts{S 65781.76 div N}N/startTexFig{/psf$SavedState +save N userdict maxlength dict begin/magscale true def normalscale +currentpoint TR/psf$ury psfts/psf$urx psfts/psf$lly psfts/psf$llx psfts +/psf$y psfts/psf$x psfts currentpoint/psf$cy X/psf$cx X/psf$sx psf$x +psf$urx psf$llx sub div N/psf$sy psf$y psf$ury psf$lly sub div N psf$sx +psf$sy scale psf$cx psf$sx div psf$llx sub psf$cy psf$sy div psf$ury sub +TR/showpage{}N/erasepage{}N/setpagedevice{pop}N/copypage{}N/p 3 def + at MacSetUp}N/doclip{psf$llx psf$lly psf$urx psf$ury currentpoint 6 2 roll +newpath 4 copy 4 2 roll moveto 6 -1 roll S lineto S lineto S lineto +closepath clip newpath moveto}N/endTexFig{end psf$SavedState restore}N +/@beginspecial{SDict begin/SpecialSave save N gsave normalscale +currentpoint TR @SpecialDefaults count/ocount X/dcount countdictstack N} +N/@setspecial{CLIP 1 eq{newpath 0 0 moveto hs 0 rlineto 0 vs rlineto hs +neg 0 rlineto closepath clip}if ho vo TR hsc vsc scale ang rotate +rwiSeen{rwi urx llx sub div rhiSeen{rhi ury lly sub div}{dup}ifelse +scale llx neg lly neg TR}{rhiSeen{rhi ury lly sub div dup scale llx neg +lly neg TR}if}ifelse CLIP 2 eq{newpath llx lly moveto urx lly lineto urx +ury lineto llx ury lineto closepath clip}if/showpage{}N/erasepage{}N +/setpagedevice{pop}N/copypage{}N newpath}N/@endspecial{count ocount sub{ +pop}repeat countdictstack dcount sub{end}repeat grestore SpecialSave +restore end}N/@defspecial{SDict begin}N/@fedspecial{end}B/li{lineto}B +/rl{rlineto}B/rc{rcurveto}B/np{/SaveX currentpoint/SaveY X N 1 +setlinecap newpath}N/st{stroke SaveX SaveY moveto}N/fil{fill SaveX SaveY +moveto}N/ellipse{/endangle X/startangle X/yrad X/xrad X/savematrix +matrix currentmatrix N TR xrad yrad scale 0 0 1 startangle endangle arc +savematrix setmatrix}N end + +%%EndProcSet +%%BeginProcSet: color.pro 0 0 +%! +TeXDict begin/setcmykcolor where{pop}{/setcmykcolor{dup 10 eq{pop +setrgbcolor}{1 sub 4 1 roll 3{3 index add neg dup 0 lt{pop 0}if 3 1 roll +}repeat setrgbcolor pop}ifelse}B}ifelse/TeXcolorcmyk{setcmykcolor}def +/TeXcolorrgb{setrgbcolor}def/TeXcolorgrey{setgray}def/TeXcolorgray{ +setgray}def/TeXcolorhsb{sethsbcolor}def/currentcmykcolor where{pop}{ +/currentcmykcolor{currentrgbcolor 10}B}ifelse/DC{exch dup userdict exch +known{pop pop}{X}ifelse}B/GreenYellow{0.15 0 0.69 0 setcmykcolor}DC +/Yellow{0 0 1 0 setcmykcolor}DC/Goldenrod{0 0.10 0.84 0 setcmykcolor}DC +/Dandelion{0 0.29 0.84 0 setcmykcolor}DC/Apricot{0 0.32 0.52 0 +setcmykcolor}DC/Peach{0 0.50 0.70 0 setcmykcolor}DC/Melon{0 0.46 0.50 0 +setcmykcolor}DC/YellowOrange{0 0.42 1 0 setcmykcolor}DC/Orange{0 0.61 +0.87 0 setcmykcolor}DC/BurntOrange{0 0.51 1 0 setcmykcolor}DC +/Bittersweet{0 0.75 1 0.24 setcmykcolor}DC/RedOrange{0 0.77 0.87 0 +setcmykcolor}DC/Mahogany{0 0.85 0.87 0.35 setcmykcolor}DC/Maroon{0 0.87 +0.68 0.32 setcmykcolor}DC/BrickRed{0 0.89 0.94 0.28 setcmykcolor}DC/Red{ +0 1 1 0 setcmykcolor}DC/OrangeRed{0 1 0.50 0 setcmykcolor}DC/RubineRed{ +0 1 0.13 0 setcmykcolor}DC/WildStrawberry{0 0.96 0.39 0 setcmykcolor}DC +/Salmon{0 0.53 0.38 0 setcmykcolor}DC/CarnationPink{0 0.63 0 0 +setcmykcolor}DC/Magenta{0 1 0 0 setcmykcolor}DC/VioletRed{0 0.81 0 0 +setcmykcolor}DC/Rhodamine{0 0.82 0 0 setcmykcolor}DC/Mulberry{0.34 0.90 +0 0.02 setcmykcolor}DC/RedViolet{0.07 0.90 0 0.34 setcmykcolor}DC +/Fuchsia{0.47 0.91 0 0.08 setcmykcolor}DC/Lavender{0 0.48 0 0 +setcmykcolor}DC/Thistle{0.12 0.59 0 0 setcmykcolor}DC/Orchid{0.32 0.64 0 +0 setcmykcolor}DC/DarkOrchid{0.40 0.80 0.20 0 setcmykcolor}DC/Purple{ +0.45 0.86 0 0 setcmykcolor}DC/Plum{0.50 1 0 0 setcmykcolor}DC/Violet{ +0.79 0.88 0 0 setcmykcolor}DC/RoyalPurple{0.75 0.90 0 0 setcmykcolor}DC +/BlueViolet{0.86 0.91 0 0.04 setcmykcolor}DC/Periwinkle{0.57 0.55 0 0 +setcmykcolor}DC/CadetBlue{0.62 0.57 0.23 0 setcmykcolor}DC +/CornflowerBlue{0.65 0.13 0 0 setcmykcolor}DC/MidnightBlue{0.98 0.13 0 +0.43 setcmykcolor}DC/NavyBlue{0.94 0.54 0 0 setcmykcolor}DC/RoyalBlue{1 +0.50 0 0 setcmykcolor}DC/Blue{1 1 0 0 setcmykcolor}DC/Cerulean{0.94 0.11 +0 0 setcmykcolor}DC/Cyan{1 0 0 0 setcmykcolor}DC/ProcessBlue{0.96 0 0 0 +setcmykcolor}DC/SkyBlue{0.62 0 0.12 0 setcmykcolor}DC/Turquoise{0.85 0 +0.20 0 setcmykcolor}DC/TealBlue{0.86 0 0.34 0.02 setcmykcolor}DC +/Aquamarine{0.82 0 0.30 0 setcmykcolor}DC/BlueGreen{0.85 0 0.33 0 +setcmykcolor}DC/Emerald{1 0 0.50 0 setcmykcolor}DC/JungleGreen{0.99 0 +0.52 0 setcmykcolor}DC/SeaGreen{0.69 0 0.50 0 setcmykcolor}DC/Green{1 0 +1 0 setcmykcolor}DC/ForestGreen{0.91 0 0.88 0.12 setcmykcolor}DC +/PineGreen{0.92 0 0.59 0.25 setcmykcolor}DC/LimeGreen{0.50 0 1 0 +setcmykcolor}DC/YellowGreen{0.44 0 0.74 0 setcmykcolor}DC/SpringGreen{ +0.26 0 0.76 0 setcmykcolor}DC/OliveGreen{0.64 0 0.95 0.40 setcmykcolor} +DC/RawSienna{0 0.72 1 0.45 setcmykcolor}DC/Sepia{0 0.83 1 0.70 +setcmykcolor}DC/Brown{0 0.81 1 0.60 setcmykcolor}DC/Tan{0.14 0.42 0.56 0 +setcmykcolor}DC/Gray{0 0 0 0.50 setcmykcolor}DC/Black{0 0 0 1 +setcmykcolor}DC/White{0 0 0 0 setcmykcolor}DC end + +%%EndProcSet +%%BeginFont: CMSY5 +%!PS-AdobeFont-1.0: CMSY5 003.002 +%%Title: CMSY5 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMSY5. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMSY5 known{/CMSY5 findfont dup/UniqueID known{dup +/UniqueID get 5096646 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMSY5 def +/FontBBox {21 -944 1448 791 }readonly def +/UniqueID 5096646 def +/PaintType 0 def +/FontInfo 9 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMSY5.) readonly def +/FullName (CMSY5) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle -14.04 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 3 /asteriskmath put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CD06DFE1BE899059C588357426D7A0 +7B684C079A47D271426064AD18CB9750D8A986D1D67C1B2AEEF8CE785CC19C81 +DE96489F740045C5E342F02DA1C9F9F3C167651E646F1A67CF379789E311EF91 +511D0F605B045B279357D6FC8537C233E7AEE6A4FDBE73E75A39EB206D20A6F6 +1021961B748D419EBEEB028B592124E174CA595C108E12725B9875544955CFFD +028B698EF742BC8C19F979E35B8E99CADDDDC89CC6C59733F2A24BC3AF36AD86 +1319147A4A219ECB92D0D9F6228B51A97C29547000FCC8A4D77D0DAC26A1FA54 +D2471EE9BE33D2E87853491D634A3F05B7672519E09291AC2D08D95BFB80CABC +FA89042E94FED8DC9929ECEFAF741D273687B2127F5B80ED7D16F146894841D5 +1A80D8085E27114AC35CD5E578760D8982CF4D8587FD3385FE50E21A253A37E2 +AB628DF3500FD2C21BCCBC8C0B6AD3AE21DE63AE6586DB06B3FB1966625EBF52 +90FEBA873F819AB480FF994D80C01DE1E2F28C665E74FACFF98B2972CCCAD279 +96F2EFF1113CDA922985E095249765989A4C6A92A76340DCFA07AFF34AC5E177 +09B739A72D0D1EAE1A6164F3AA0DF1651EFED335B68C8465E1A61025235D4A5A +C3661B9C0456C35C770242149B3B98A29784BF71030C7DBB2C13958E5CC263AE +FEAEB9FBA84E06D772E94A494B60E32AB8FF862E57175DB17F6162DFFED60EF2 +D46FB78237E778C7321ED79FAC137945A46B2D90B778DA46308899577BC7844E +3E09303AB2DD1F64D58756DAD62ADE22C62DE6C9BDA77CB12B1FCE3374CFA442 +E0CABC7C9AE699CD955077A7268FA0FD3FF5EEDE605990F0DF0078799F5CED44 +6ABDB2E7D9214BECB8B2CBCF328F7F5B8F346B55C9BA5C785A252DA59602B6DF +186D23D5D90556EB52D7C0B44E4A9A189C180FA213E12E9838F0B2902779E289 +3E2B7DF45A01F36D88C9CB814AE5D41064030D37393DDD7D5974C1A27EBEC064 +B0403898996140AB14E6EF1B48D555A1C4DE29AD0C5D284988B92917442D2C61 +3B15E55CD91837B0496FEBB51486CEA7DFD9787AEB484BE887111903124340C0 +EB034A75292E59CC038E75E1DDB0027407492EEF92D70E0303BD7B2703E131FE +BA91B289003B7E5BF85D2E5E130CB37949AEE51EA57FD424402B0BEEDEDEB5C3 +E23643F3100312F94C157C98939EA2312B11DE0A683D4018FFBA2864626C9449 +0D656EB0C79246A7F5ACDF442A55C9188D8F3FA64A9B6BFACD8F81D7546E0CD4 +3C5CC653CE9514306439AA2BD2FF2B965EA69A30331E8D65056D919476867463 +97DAFADD16BF9366E95C7C4276093B639541F7AFC9AF745260685D84DD3095D4 +1B925540372ABFB68CEF211D65756FE0E99733B5DB40D5A42016E8715208D4B0 +7A120EEB34147BD179FAFA94D0880121D1C6B27F94965ECC1A93D1B73AC1B3B7 +191B7B3CD4F2E0E52A3893255C4B3F1A70A0ED36DAD262538E02E8A227F6D6A1 +8A047A91ED6409F1E0D9B40698C817918AA47B95A082E3394AC42DE93BF9CB28 +68E63F3363FDCDB014E89990C50210750E8442D9555B0A796B243AB03190B2BD +57C39F233318B6474F1B3CCE31C21BEF68FF5E3F866789041910B1B11ACEB75F +146BB1A162AEFB268700C2DFBD9752BCF6F136ACAE815C0AA4147AA2CE61D282 +1BA716B693E2F713D0653B17C9381DA86E1CFCDD1B1FB9A72ABDA1AB2BCAD30E +A4EAB601BE3F8EB95B32DCF5DD114683AA64E14F9A958D16CC6DAC7DC3ED651D +603959B0265FD36F8DA41F3173A708D88C73EEA1533CE48C2AB26B4DB7CFA662 +A8A463CC178C31D342EF907B3B44751859F2BD510F1CA93D61AD39E8E5D7785D +D465C47F3EDFAC22DC2B5DD93B2F752EB5482E138A0A1E5CD62747E0F524E29C +A56E58A6BA543CB3AE051F4D90891B65712B63DB197E60275EAF5975007D09B7 +95F916615F97C50D8FDCFB67F1B677DC5139E6F46ABA079AC552F27F507CAE8E +E5C04D31ECDD6CF5E8A447846375C1748651245EB9C780D87FFF162AEBCC9E0B +155B318CBF420DF0CFBAA5669C2DBC81B55B5042514B05B7B82FCC827E21462B +BC8F8FA96E2CC4441E07E0901EA72495013400453E49B7D243FEF528E4078AB6 +49277EE77C81B9167D2EE1961AE265A8C01F7562ECDC47D9FEC391097474EF8E +54A5A7FE6C98A81ED29D42A3D54326219F638C92AF8A087607CA2A2A80CB1183 +8571CD40199FBDB9D70AD83979C739F533359FB2DF0AC4AE3A0E9DFE735ADE9D +6E29B3F6DB9A5B11418E9C860EFDBCB0E49BB900BBCC85860100F1FFA5AC07B5 +14F37C5AE5E1A35168BC48BA7012A297C7771789302AF44765A6DBCFDD418172 +62DF3A4785385CC9D055A9B4FFF373041D9F5E9DA83BC1C5F3265E49BA624B82 +F15BC17B135D8723324865A9CCDD2EF3A0F64C17914F9B238C946FBCABD92B30 +AF90191996ABF2E4972AA9D0EF7B64AD0126D07714D4059CE032905BDF877D7C +3440C374D7B5D0073E13ECDF3DEF2938EAC1908CC60D0E5EF5F52AE8A1B0BD34 +8A454CDA66E7340783A7152CCD74EAAD4112C7D70F3C867E140988BB610CCA20 +F1BF5E3DA8097509CAB419E01CA47D449C50EB8FC6DBB75AE2C5312BBCC5CA91 +86B221536517CAA6870429B65069FE3CBF3FFFB5631B57597E5C0C3E39199FF6 +4DC35B155A759CBFAA96409BAFC7EDFF03D2671F0396641446605843CA4AA36A +15070DE52DDBFF487C5398CF7604DE843F32CAB26A96F8A9DCD03F3DCAA57E47 +6F0D36CDA34B96B7F4986275D79A1BC1954D2BC0BE1509709E40FF23B84F8D65 +3DE1D1D8A2AD94AE3DE202D62993EC4BC5AF66CE512EFFE7CF39028BCEC1667D +8782111D9D503BFA45E4960E6197D8CB5697B1662BD2D991532074AF5491DDC3 +2EC52B4E5AF9B2DAA5D5E4CB644BED25DACC3F445E4C7D58A1E9737F4EC04A79 +238E3578311D88EE6D067725C580A146FF150871FAB2B366B5AFE5CE3B386EF2 +EFB540348505A1749BB34586851FB2FEDA92ECC395B618911E1D00B427251B05 +15B9E42AC1F0F239827B938EAB8CD20FF9705F4D7FB5F5F911CED9EE8FDABE8B +3C3442DC3B716C3686E1B0573EAE3905C3964ACB336FA3C38B17F229EA10F036 +ADDE7FA5CFC01FAAD33F75A125F52AD395C7DBD3AB6F2321D7B5B55F0ED237CC +61FA8278FFF9956C64B8ACBE357D4603CC4599BFA198AEC01E4F4AE047754C16 +E155C8FD09C996D9B5EECDB243FA48B0CED1F0882103729EAECDDB9523D6BC22 +0AA1545814579043F0DDA2D7F015E56C399EA3C644A15BA6B144361449609C40 +A4986E00918A7B44BB06E7090F73FFBE4EE3E051121939EAB0349EE84715DE36 +FB187C50AFD74C44C25BE4B7037466CD60F211FFD7044245159505196B0E181C +0177F14D0B486CDFB767708C171FE513AAFA2B60665CA55B1E5F9E0304B63666 +3BF44EC4F5B7DA58891081F78B37BE8453E86AF3557B97D8B4969CCDE2DFD255 +91FB4D9CD41E5F931751BDF7FC8C700C633470064C0BB00FE5745918A69CE430 +D86FE46DEC014C019C06621B2EC6C3F196A8A5E1E613AD8B28A9D0B981C11A29 +4AD11CD3A46D089F8EF7D8298971A5F9F6F139CA794AB4BF0AAB2D4D428A8E4D +B80EC134CC7CD74D3F08E7BC8AC5B1846E1E5DBFA97AE4DB885286C7D06B4CF8 +948E889ED85E8703CB97D7AE19E1201EEA205157BEEFAA3E17B8BDDFD01BDB17 +691010A662280C279942158F68BD351B235EA4EBE24D54C0055A19A3F230F353 +77202A1D7BE371010624CB741D4BA2E690B33FE582D666E9788A3E3348A0E100 +2843E0B2FDEE05DF75F4D49DFE2B69BE79E97DA1B3638289395444F1CF6CDAB5 +5E6EEBB6F907807B913290C25D8AA546B121EA90DD113588011DBF01B7F88AA9 +B9CA9731532478D3BA57786817C1D31D8C51C307D7F500F1B58C2CEC9594E5C4 +E5E7239D90C8B5B4A81F95C048272A2C05EF0613463E4094E922F7326815EF3E +6E82D5A36840A76B1BB25185CD66B05FF2D140E0F22CFF73EB3F57732E935BA6 +8C64BB6809E1B45089273527834789637621793414B6853A1C301612920E3F91 +458D043CC1B6CDA271864FD4CCE585FDF025FAD217F975FC36FAF8B79A6551DE +AC3863E25DE972E2A4AD997E77A2C05A1A14152BCFDEAF40072A0AD952A3944B +AB12D66ED3C408DD104B8638D67417D2A95A87E226A03C28181047CF5BE58409 +7D07D2529BF867BB61ACFD68F817BCA73815F787162614EB926563EE2F8629FA +6BEBC05465458A03C2A8673A3CFF7552855E082A7B5C2D4101A1B96449071485 +279C1EC56610507DF786427B4E4D949BDC9127C2802451F3B21D60D5827E5B7D +0A683F4402725547EEBE27C38839876A3B1B7FE0D7107D6E2292631D1B9EFC79 +87D05342972EEE2C413568BA971AF56A2D4D89C7C0D0120AD504D444655CFF3C +957D5EF19169401CAAA2C4C144FF87AC7DD3451A63CE796B8F6C600F913F55A5 +A74B928AD676ED097740BDA578DA299F42CB7B7334BD2281E403129866031EEA +1219E7F9BBA230D049EEF401CEB412BF7FB58FB2F9B7D1E47F0D774956B803E2 +872A7A5A38E5B7EC4A225C73522EFCE6E98E3EDA3BB402BC5BBBA209074BD2A8 +C856290D01A092160E8FB0D9EF324776AD39D7016C63A500D0839E670009F4C0 +20F033554B6B7CCAB7BF60494431E4F2755E4CD966ADC7A078D434FA59E5E49B +9903F4E722ED169CC3D0FADE874E3141CD47C69BAFA3183BEA4F37B388F4F48B +7D143D4B79CC5E701EEB350447D9600D39A7C03A25B5B8B6A5A7284A8DB4969C +CDE2C99C39EC07BD8559A45CB5AF23B849C1826713828F43DF265068950DDADF +3B584B90748657609A89008187BDE4286F95694D01760260274A23A41FA66A78 +773902ADDAD18F98F045FF7D20AD20EB685C145D44237EDA272619EA9AB71497 +DC8BBE9EEED31EA77C9410FE650C65A6CA71F0E72E7D83BBCC0A59DCEDA6399D +64A8F4865811A637838445D84D97204CA7A918DD476471A598D7536276C0F0D1 +F0A88D8BCCA56409342C1FF678CFF90FC7B287C36CE682E1B200B54B2CCB12DB +D1DC9BB4E1CDE2B9B6BD4B80DC226E10FCF20A9E7878A16EDFF37FB874E4868B +A9F1C452EAA7126E81763A2DE2B502132046322BE26140F04FEB0E7167920679 +FE8618E1AC5C03360AD3D8B5E913E1CF0C1BE9E446DFB050C3978165838278DD +23D17DFDC85DC7B168F173D0945D110976084CB27B6F8AB2D22410DB05BD345D +E4E38CC7EDCBAE36CCBAC590DA5C12D969D51B847A1460F4FAEE7D648818D26F +20FFE485D842C2088258F4F0B1FDCA46A7110645C25E3A31C4E6986C220985A5 +DECCA30BAA3A4B5CE0152B8A9D68B1FD67F21050A87BDAE6B665036B9C73953A +11B16AA3285456220D09A03DCD8402C06A2BB18717B1AEA7594F748817176E12 +87E685FE70E28C0382EAE8948A6E9FC2F46E2B70DAB208611DFD5A5D97D39CCA +8F9AD9A171DF53CAA6E3E5AE87257561BE15E3BB21779397A30AD4B22A1A1EBA +4B7235F59EEEC1FAB7294D521EC1E36723A4D7BA007DA4239C05F0E6DE546D5C +7A58CCD00F6AC96C20B7182CCCE3E5937D7A64F7CCD52A107BB8092D5F95A8A0 +589B2DC542511DC89A0EEEF23E711C7444BDF95F64CF486F158CA162138A36DF +5400A5D03AE62A945373E95840F6D6C7BCCC4FE65382584A4DB36D23FBA6562E +1F3AFCDEF27AED61245EF880426B14B3DB5D9731039FB98F2F51094CD7D5EC78 +1F8B03F07064CED40D139BD04C868B1E51A994731EDBA98E7D678A07F7AF6E56 +92586848C58ABFEF378F5C1B4C3E20BA860BC7310B9AC48F2143B5495F9B3A35 +85C9AD5AFEB23DF6376EE78BCCE054C8FAA925965E38EEA6F68CE1B5329D9DD4 +DAB3F3342C6E6D9E0C70E63989DE8DD2D3C4C671233888ABF56C58F69D29A742 +B5AF39EBF1092C2C4A906B872310DE0E83372B1C7F6E295D8DFFEFF960DC0F4D +83D7878BD8918B9BDFA17889D52AAF362B6162C07810672E23F66F8CF3BDBD6F +0E016C4FBC8CBAA796057A6B9BDE3233909694110279F81259A5E628FBDF6159 +CA0CD6997EBD34C3038A02FA4B422A98AAF4297B998685D7080EAEBC22E27900 +8B0178A75E5E52363A2E56507B3E44A6F9A9DD81886317A07B4823558D3690C9 +8C722915FEFFED7F6CFF4E94DA744562835E29C97333CFE50D49A5EDB886F4D5 +279DD9CD35C15656F9329CA06A0CE1BD5986784A910EC3DC7AB1A068C8ED4829 +DDBC83091AAF21092234F614A6CA2C4A854B5049B5622BEBBA68AFC0AF9C6686 +74555347CD055BB13B5CD7B5B6954A5794AFCFAF9AAA3682C7333564660E7C1A +8FB3DACF1ABACC3D68C26D07060C7D5B9AE9C9377A771C3BCD317F0ECE3CED11 +C0438A29708D99A1F3C02E67AA1D7BB57AA7D8C64A2028349BDDD4BF6DB3139E +C1BB6D7D906699484EE2A5D83296BEE09644B62EB10658E06E4176C1E5E7BC4F +36F3454F2DE69F0C5B51FD5874F67287294399923B19A40CBCA4B580BCDA347A +9AFB7AE64C8E9070D5586086E6F42EDB8C0E9729649CDAF7C0337BE5E670DB9B +D5C038262B65603C1CB1BD00D4ECE2BDAE5B77A8A129DDD79DC7D080AA89B7D7 +439A03568C2C826EF23C2ED7160B2F8F14C05F444E364DDE247748541CE7E265 +1414B5A15E7152C6E9056B0CD43421D4AA5EBF1EC2CC70516F920F08B0CDC38C +DF480EEAF04D7F12638C9CF8DC4F827DB07256F4432345FCDA86CFCB637D04D8 +931A81D2DC47674AB435738871A65422D3FD46DFAE9049F27DF0DE33A0E3207F +A6832153135BBE5A283CDFF46ABC74C78B3320EAB1E881E59E07C3134F3830B0 +8216AAD3995D3F8F6CC988C73B2B7A5B79E81C6E8377FFBDAB29B4CB6317B118 +84A437E14B0B213BE7ED610E9469405AB950965B2CFBD4CF0B3415BC9638771D +C2D8265D4FF0018FCD3851E645A82C179FB4995A337C0A854999E0D91EE8C98A +A9EB781A0DEAFAE1D666D2F5813BB2E79C232E5C96C686D414921DCF8EAD4C7D +06F79EF5078C77556346283940EAEE347B06F756A800FEB7656DC02597A674DB +6F8BAC8751BA046D7040B5BA0AC766522FF9657F71C7C89DC93E92000A00120F +301DE3ADB5D7D62B57E0824B9BBED388A5CF1B39B034AE2E44BE27CC0F5F0223 +2A219166697692167403DC2083384A698551FBE48BEC89629820596FD569543E +75174901415C56DCBCA012694D671655AA84A8D8307B34066F1AC149C8B4BE3F +578F57A39B93DC454580C44B12D6D1BC653CCEACE809E5F3F7406B9F4A868D68 +ED4C5C23D7B2497668D718621A94AF4938A16FDACFC6E3B216BB615324375050 +24A7466E820435B11E93E91F673D609932BE884AA1CDB9D433A40D189A668498 +A1B708DA358F56A1F2CE297C61D9AC19ABF79539914749E5DEE0FD0DC998F4D8 +1A4CD93D0A06E68CC764308BCE69D4D53FB2F0AB742D7E9618321EE87EB1DEBF +14561A916C8D58B9C6AFE80D586F5FCAC3C6489038178E77B515108E48C7AF9A +AC9C3F93598A28CD03864C6A65659E8E6C782F50ECFBE2C27678ABF0978877FF +C685386DF9D9D8F51EB5AA618D2B3F63FC6A0D6107469DA479D540447D785235 +2FBD6C4054C30D02DF3D2732031297F53AFC937F6CEEE01B2EF34D59FD0EBD21 +854C5CAD32C59F6D8F843795472ABE1F73107C0F5A5B9274CFF2346E7627355D +5E7B3B3451C89DF5572F4E9BDCAF526425B34D71875B5109EE341F55C551DA50 +31E2EC3BCB7A79CA3F737D3CC23200160C76E29B9A5740BCFA0355AFE7930076 +1F621C801268E18B83A108068B8DB4ACF7079069AD73E37ACB779490900CB770 +326BA14A6CB1E7FF537AB11F432809968CC441F418AD1A36075D948D487C55B3 +C6FC12080681BC9685764E9348B28A29E6FA025EBE0C175AAA6CD60CE6E2417D +FAE5E146264FDFFE499FF7B6F04466BF3406D2AF6DE542862EB51614000D9B0F +0C630C2955964D0D808C8C65211DA05FF14547308FADEAD9A907B1944DBCAAE4 +DAE093FAF4AE1AC91793F025C0F0C1E74DE7817154052436A369CF7F52E05841 +5778334C75C7ADDF2B7C1D996C18E65366EC4F8F085AF9DE24D54FF213C73716 +0C383C2741EC725E1FD07F83ED9BED3CEBA6BBF54C5CAC897D17E19F62A11AA9 +B3D13D0649DE56955AF9924057554C39C82F9349AE6BB9B64BF829E70FEC832C +4C485AA946B559E464DCB8AD35A51CF8A997F1E3432F41910F82B415DEC1878A +0650595C11ACBFB118910B1C5821F8B6A410286752DFED0F82CEA57C8D763F06 +BC262A0706D92AA301250011908C1B0E9104B7ED130CFD89387A5D6995A2ED01 +3AC41DEECC5EBCAF68D35AE4B7ECB982E7C507F9A825F7D10791563182B36150 +D3A57948C01B1C0A09D1B0B952A9AC8A62297A7FB7666F495FC71D628B3F5331 +17629C045F8B74E8E98438C5DDD73186EE47DF60177CDBEEE089F6A08A98003B +8634A36458D0E0CDD169CF236BCAD946844075C8AA53F10A5BC48EF7265B9217 +410473CA471112D10841A1F655A8BCDAF640F33BEBF94A9D74351855C984BE17 +83A0F24C3CF8C7E76C8457CFAAB359A368B3AA5CA11ADFF89D7FCEEC9F49B7 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: CMSY6 +%!PS-AdobeFont-1.0: CMSY6 003.002 +%%Title: CMSY6 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMSY6. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMSY6 known{/CMSY6 findfont dup/UniqueID known{dup +/UniqueID get 5096647 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMSY6 def +/FontBBox {-4 -948 1329 786 }readonly def +/UniqueID 5096647 def +/PaintType 0 def +/FontInfo 9 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMSY6.) readonly def +/FullName (CMSY6) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle -14.04 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 3 /asteriskmath put +dup 48 /prime put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CD06DFE1BE899059C588357426D7A0 +7B684C079A47D271426064AD18CB9750D8A986D1D67C1B2AEEF8CE785CC19C81 +DE96489F740045C5E342F02DA1C9F9F3C167651E646F1A67CF379789E311EF91 +511D0F605B045B279357D6FC8537C233E7AEE6A4FDBE73E75A39EB206D20A6F6 +1021961B748D419EBEEB028B592124E174CA595C108E12725B9875544955CFFD +028B698EF742BC8C19F979E35B8E99CADDDDC89CC6C59733F2A24BC3AF36AD86 +1319147A4A219ECB92D0D9F6228B51A97C29547000FCC8A4D6AE353EAB9DBA0D +4335868937E3A2D0F9C8A4FBEAE138AE56EF84665582DDEF7B546714518D224A +F91D19409EC600924626D79F58E272E3657E631C06412F9D42732B6AE0A93D81 +E2B13A81C76CC37FCA6DB35A1C6A572EFFEC7E1581C57B1D22EAE9C407771F40 +71D002A06A6AC249FCB7605C902A76EB298B05E39D543A63332A91B8A1ACE05B +F902540FEC1024680B8D5857E4931D00781747C87ADCC45430B5462EA480BE50 +D07C5C9EA0D8D6447324434CF0D637BB13037B6E1C48F191C52CF2B9511690B5 +405280D5C825BD376EF12429837CD44BEA223DC8E199D8F85856C0FD551471CA +B30481FAE99B71FAA56B639F549F449F2CA6F5450AADEAD4C09595A7A8530D12 +B23E012978225E112D23BD1D48DB097994A9D5EC707216D2BF5DF8B69E66ADBF +DBDA4737737F51EB7A38A4207BB46456A8117067748659D654B1D14849F14920 +E57462ECFF602A181700A346C1E95131C6F1C66132915FFD999FD4105AFFE7D6 +7DA46DFCC4FE53265B14D7326F5A44A3E97902CCB106AE99ED6434AA9EEE7E6F +60B8034EF80DF885FC63DFACB0AD1C00BE56F79B1AD41B065C59A35073B450F3 +7E6BC7B90C9D9337F9636943ED701398E6886947E126F1E12D74C5E4B5CF06EA +E60F9B121B9B15E6752B127CCB9C26017A84BE6DD918AFE46DBF27BC06E66AD9 +830BF2D6CA9E24A0199DD3E732405F3210C97C9BD21545B5E4DC10E17C1A2C71 +851ED2B272055831A6F95864137C218E555B054BE3616820FE671E5BC08E7A48 +D683ACDB91E05F469C0C8919D71027E179F4911E878F8D13F999C63C4EC2C21D +ADE1A11979B539C4C5746AA157A4EB4A7A153BC5D966197A7FF6A2DA3FDB161A +EF43C2D906C617868617E72163D5B0BC29C36A60867C120A02E08264231FD46C +1D3B62622A05129188D818E2AE83FB2770DF1D466CA38A49D15F9056BB7AB8BF +D7DEB2E3B3A8157B59D138787B21CE3B3A456E230CFEC1374AC66B7DBBE788FD +5182DA23264093CEF60D5BB0073AEDB68A61ECF1C69669244A1FA0CBA44DB83F +352AEE6012082B8A3FD62F4497B1BE0E4FF77FF94BF221212D63BC1E35B9758F +2D7F8FA002751DC0380AB954331E9AABA6C122F8F3D7E69CE8A6B359184F0531 +6B158ACF34C64C32F4671D430820B36C9A9397E9C877D734B17C3BBD8A88CE2B +9F96D7AC5B8D55415E942D6C05392D03DAE53449F8FBE2225C91F228704BE9A1 +ECC568FB2E90C73F2F7AFA62859C35F0FB26756DE719BEA5A3EB81A6481507F1 +813CD1680EBFCF612FD9D3AA4B90688521446FABD5C54CAFE7B879F24B54449D +2B30738B137B5E212B34BC5384BC2216A5D13F761D6E13A043DB2AABC0276705 +5C80CA394A8179D60DCA4651CE8B0A477165FF48A31BEC6B7C0852C9D72E11BE +FEC5E4B984DFDE90218C105999E070FC83F1716D13389B5EEAE3F26C82FE9A7D +3C67A98027D15DAD791348A21DD922C71145D287C7EA83B225377744BBB0082C +22FD44E1B0DF60956DB9C4790F4E2F81C477DD4CA9F4BBB4287914EBBB4F6E36 +976C9E573496A2311E5C7677FBC4BB755B8BF925F291DE0442FBB8921402A487 +ECF23EC556C3FD66158727B10B74C778FA91EC16629F6E949624D265FA6FDF1F +772E965A2C30C80AEEDC4A1867AE3D6D116BCE0F96D1A0C5138F294CC7C2AD72 +8BE152B314E26422FA2DDDFA89A054CEFC6607D932FACB035FC9FD9959C4A468 +CAC0F4DD41DFB83297EFFE36C352CA0A6B1485BF8433522BBBCCF9C73FB7334D +1CFC92348F91C4D0EAD90B22BAC6B27D21CDF5AE65C056BDE3B30424F76C8EFE +D7CDBC854B13EAEE0485CC348AC52F5345B7545B4FFC60B6E7CEF8D32354B233 +8DF503A03F5201EC012AA67157F4A47B902D269A060A5B8E10A37CA7CAAFD417 +1A13EED9533BD4A6E8845D7D236CB254D40930C273EA4372E0B13912B2CE9261 +275D51D202D461FD69AE352E95F72D61683267D4951BB602A0BACEAFF9516F3A +C30EFFF3ACC08AD374A730287B148FE8E168089D239E1084E0F7D83CB9815031 +5428874FA4640E1C216514AA16C74BD2B9A95EFD7B6B0C575A5D5B3AD1086CE7 +E8EEE33490CDCF2E3FBAEDC984FE53D033987580E75E74E455E44496E186349A +DE4B5125F69BF79936565986B3D91E6A25CE6E1C4527FE22580A39805A91BF6B +90EFC8AD31FBA71FCC90EA04875B5A7ED05BA2FFD1C04774E1ECE84B4105ABBA +798C339FB61ABF75C6FD3F24BC775A7FDBDD87BACCB9A86B4D928917CA62C0B8 +56343CAE0051A1172E7F234001A5FAA947CAF382FBF23E57F1144425713423E4 +4FC1AE1D096D10D6FECAA6A9186B0D519F51D06CC976A65A37E9219492E6218A +FE9F99700C0D0D18BC48632268DA0F0CC24BE660E30645E7F63B0AD49CCA26EA +CEDE4DED685DBD38F5C4BB6AD6CDFEBED1B14A456C27C448C91CBF6337FB3632 +7B9A565874F9849FC5E22C5A0437AF29E2B7FA405CF6B7AFEF66AE7449E89746 +66ED2D4C15A36C45C0F64E6F185B808FC788984C230B4BA63204B38C309F2DE2 +2937A92C184077F5252C9D6171EC6378273C32B61CF65E4DC124B321A4049294 +BB6CDD064D4E07605DA5FFB7B3B0A55A09D26C8624F03D8C197885BFF0A20B77 +257637447F628A4A642F6F92019F9ECA5B2DBCEDA63C05E5258C9DF131EDD1B2 +B4B04E7A5441035163BC2769B79C89ECFDF071E3517C59A45C9E97800724E1F5 +FDF504E2798FA9074A37B210B1C24DCC0B78263E2297ECA31D5C258BDD92D2DD +8C53DD3B3BC3835E5AF52311EF0AACD6DCD159C45743B1AE7EE5A0111DCC9193 +441FB81F945D219DFF26BF50D80606DD3392CE941242CBAEA45077FBE6AD78E3 +0DF4FA54CC33CAC27A337FF307F14E60D1FA48FB8368E307441D6811676CFD38 +094A558CBF0B92503014BB169E345C27C74543CDA9E8134F85AA0DCE45EAF612 +1AB1D86FFB5E13CBDE8E0F68B19E30D0D9F5F0A3EF2A84AEE4D7F1F63A045297 +3F5C3080F313B32066E0C209C2708636C33A6531F1A483F3AFDFEA62305EF41E +C55594660133146B24184070B87C22EE5B80D66C409076D733245FD100E936D4 +E844D8A15CF45C0CD6AB04F3B5FCB0BC7B1517BAF21CED4D6CCC3F19D21BABC5 +FFA9768D77A41BC3F5C4595D868F2FCA88D6E41558FA5A70FB562E4EB22651A3 +886F0ED1B46BFFF297A28ED67BAA28A1302BC866AA7A73418BADE0870872CE0A +305A5E46BE0CF6B5794539B8509F1E2DB4E40124D72201589BC4B55ABA2F5C40 +E167175C7F7378203C949B8967E8CCC40F44A2D1D555A4273733874265542C0A +3C47FF7144C4DB276320150FD5CF453A505BD3BC1E98738A1CBE175574037D47 +8EF9F80622F57DF4DAB8FB13004ED1DBBFB4664F8D7ADCCBD529B22235740EB1 +216799B2D8CEC7CC4D16D8273B6AEEA427D480AA6E3C463F46763864856713DB +05307CE674714CAC8351410627B8D1087155F3788E744158F3428A767CDC9769 +CFBE28A30D635D7C4FC44E43E7F13128A867389FD66CFAE939AE2745055BE71A +AA7A7817E24D38773771C77328F4A755C872AA977C062A3BF7EBCD6DEBC7746E +70346293A661684D3988A5D0B29860E6789FA8911E0C0749ADCB9A6B053B0E31 +3D490B9A2982CFC167001648FDF96D040BEFDB75A4EB90E2974927BF38F10E78 +17FFB9DF2A65CF56C1F63544662CEA0E88A5141F990DF6B5DA79CEB5B4744083 +C4697EB37422BF120332AB934D292CE8A3624E2C8FE6862C2211254558459BAB +D8F1D8C76C604098701163703FAED0BB298396C86A0427106C007A7CDBB0AC8E +35F39B240F8D9D43C1BCECAB622C02E721A323383973B61DB81115E3552E4260 +9F252F18E01C1EA2B0A1DED4BA50A5822D10AC7ABD664C460A4022BA910804E2 +2094A3AE0F2AE375FB22CBD35CE9C2542C240A79BC6A79F1135E58985837E1CC +1E5D6666BC5D61FE6EF590316F9D1E80C42B9A7B6106D8B7F575447AACBB2660 +2C96D7F024FFE3292611E82CFE62D95646B92CEF6F03EEFCB3F6950A22B0997E +52C38B55B77C2EE7A4D047F27ADEC6BF0BC639589FD0D1E0A47C948AB5BADA43 +9846CBB5B1CE53E96622B59133303AA0F559A481E50382A2A38C98C1A56D9AD1 +1668FDDBB1FE8068DCE68E5DA13C2D9918D1FEC7F69A6DDD3A29E31A57B7BDDA +EAC616202162DCC7B9DC52AF821A9C1D2D7F860A2F294904A55499432C7098CA +BF79B1B0A1527B99A4C57673AF54297BA822AF200FFBDEC475158AD6ECB55F87 +555012ACFBDD0EA527CB5793CA7CACE2FD956F30D7A8AC7F6A98979BE4BD744F +BB84A87668E3DE62A814B92D3139981200D5ADA86C5D733CF5107DF6CB5C999F +F6F63ABEBC9694F0F67EC632BA8C58E715D92FD6BDD056F29753BCC3E54B0A7E +63EC33F7A29740EF0D8351B5897B126FB6C33E9A33CC83DAB91691690B8B9598 +FD3F397FAD45555CF5FB9CD80879A0CAC1CD8E8D536F05134B21FA4065452DE4 +BC8F63C82A3C34A01D00F4BB254F2AD7F6171EFFF6D7BE1213C9456AB383C568 +BEC17694C545B98E8AD734D0522FEC92EB3182DDDAC54740AD352E7645D08B4D +CB4DA9B381D36C920C77869619855B4CF730E590B0405CD25884D6DB4D49F1E3 +D79EC4D0489514359CABED492BD751104091A661A510F268EC681F818780F304 +D960A15E583F354DD574767B860C57CC9CCA991E710576C626EA909C6677057A +C409AEE846B4D5C0535F849F33470FA395CCF5C05CBB9A1FB62CF656F7A2AA42 +34730F786153A9856C337112F6A555A70EFA76404E2B88DA2DB91E8A2BC5A05C +CC339DCA364A87738B60497CB790EF16875CBC506695DC7D70C6ACCDB2A00555 +3877A013482D55281C83BB630CB3EFB0657E5490176CF23420B97265B22B4CFE +1DF9E0B294F064F5160540EF3A452641C82E347A96514E4063A7FF16832736A8 +6F21F6881E269CA09FB66DF37CAE7FF46D6B9AE42B6396EBF9A7A06453A81EBD +808B047421B0F88F62B3FF217542FBDA872D03A476FBC7C0D658C03AF70ABB40 +03F1A3027E0FE186A22AD461835877ECF843C0E8B8340306092FBC5E91783A48 +E010EBD29F114BC273BDC53D2C94275F8956A74561813AA6C2D896CB3F6E13E3 +30F6C230C00917A40EC6F4689B65BEE32F5722BB1D1147585AB3B3D2D5EB8273 +CBE30D1C6CA1BCC4BEA9B87A014E7ED3CE53D13ACC8D426267CD3BFB8497E1DC +C51CEB14F9CFB4D4BC9D6EA5A49DC3871DE59C89F855F079299F45D363F78634 +493829329E3F370D117DEAA18E7A6989C515667C2229779B9A5494DF0B4109E7 +71D79D379FA7B408A4675FA3DE17ED0ED632F9A443527CDA8E960411AD4070EE +173ABBFAF9101BE92BE59707F98419503A85AC58B949E8595D66D7B13B8FD4EF +11B28FCB5955C5F147BD7FA141AE0CA2ABCB1CAF82FE22D3281852205087CF45 +549F4D5B47F52AF50E8F6FA0AEC0F95FD0B1EBDCF965BC8A924CE0F4598AC546 +C15CEE4EDE7E6DD0E5FEC3A531B944CEA406DBEE21C4CF3384A79D0D5CD51BD8 +AA5B8B1FC987AEB57E86D3C6FDFADF998A6D6043449FA5B88BB2DCBF2945A513 +AD7ECD01B39BE46A577437E1774C3FEA176287C51ACCA15909E987B5AC2CA664 +709AA5605EA89A8DCE6B4014CFA781E822E8B05F377BD0187D5EAA0585A98E2D +6726C471578295D824D1FBFE89F04A67F1A360D799CDF92C1218F29C24DE22CB +C6B625D044404C22C89C433715BD77D0BF171030BA84E292B7A81E2FD99D4E67 +BA702C2EFA11D4461D833E74E97C9E295C6D170DB6BD73D615D03E678B0296C0 +F8BF99321767684127625942B4C3241BD0608460C81D97FA658BC179917B5065 +74A4D1F729CEB1D51C82F9C90F0C712917AC482D413ED7CAAFD545508652DACC +1A03842E354F734C5FCEC8C7BEA4C69B066DBC83BFAA473D0A632DA2CEDF20B7 +6C7C83B4AE2AF7E05AA2C081BB6870AE38848F3870DD62C7D56EF1CCDD4BA3B0 +32C61ABE629BBC464BBAB06AA0202772C16D82427ABAB4830ACEA267B0109A81 +BA5D832731D1E9C9F0552BEF7A9B7AAA73760C6A09B910009BA9D93F1A56298B +A43A32D43CAA3988643DE373FF47C176CE92D7931601F0034E6CC303876498AC +026BB32D2F656A3541815697CBA2B6D527B69816161B2E13171E073D7CD85435 +3D32708391F63D592B71D98805CD19978C2BAF780CB124372EEA0A3DAB537683 +DC74F9E9EF98843F170FB086AEFD526CB81A7CC88046C52E1E56EB40F6F5C75A +FE9D5ADE49E82078858072F68BCD02487EBAF4E3F7F309628C1FDCD4B01F7D00 +E1AE0A7CD3455953FFC77EEDCC5B5ECB1FA583064EC17132A3C12C0153C4B87D +AA57FF73AD33C22C81C545F0593DC7EF438FF7A5F76A04E52848E00CF2C8CB06 +F295B1CD3BDB1D9E4EDD765D3753EA8DB76DAACAD75CEAC866FC4CA41501200F +F12998E8DA573786AA9A7232287A6D19A12536026BA4423EF0B6D81D38BDCD9D +2BEF15BCF32824F8EE802B461F52BD926A5BEB2E6CB31AF83E98E849D4339FFF +95419941B12D2F3B76CCD34D6B78B91BDFA5DB4D4879501DF3E514A5EA8F1713 +E96330C5B4F1B7BA42D42F05E229FC303AE00E954A81F9BD43016F5930A0C490 +6C753157E4355F6ED3C5D1963779EB27402522B5E9D459206B3A06E594DF198B +785F96B96E13045594EB42EF4A8BFE0DFCB4C5A2DA9594431FBE72A5AE18991A +36FEB5FD9E8B238B3A095939B6ECAF5644BCE7DFFD8EA3E723F66B305F142B71 +B26F34E0538E4B58C4B5984883C6B08F3D1F596B56578FFB0EDFFE3F95E3B6AB +63CEFA183A5A49F07F0F7559994F56D7305139D06AC8DF9466E715D90AECB1E2 +C35E3CD3F387579B3C9CA6F5BD741A7A5D7B63FC0130AF374124DC8C6AFA4FAC +E40BD5923FE889E87126FB22C1AC4FFBD351F884D70803BBE1BD0D40D3CDA0DA +F6984C373D3174358BC41C8DD4A215283ECCBC24F345FE65E316F24940198A6C +3B5483129706AEB60D6C0CCE0DF9C130593475C72A92662A323EA5B689E5AD0F +C949536875C37AA2517281A36A1C1D5650AC96E0CC00C03266E279B54232A1E6 +E6697196A4270E1DC794D8DBDF0690372AE19121B85FFD357E965228C66BBE6B +D2E55533F7A1BF30B2EABBAC98CE35C619556CF9F0A522E8EFE4BE3EC61FDFE4 +E364608963C928AA8B302B1636DE15C83711C8FA060C5BBB1BCB8E02215A4F4D +560DBC4A235F6F07EFD20ABAFEDE0795AA3932556D6AE3F6EB8FD0C1C509FAA2 +55CCC2703661C9F48233B429F020E01C1F3B8090D331C3B925DD5D1CF9D19F95 +6B192C593225B5186109D3E6636E87BA6F24EFEACD1F06A29DB1D1F4DE0D0253 +87641AA4A1113755D0DDA6EE4234CD9AF550DF63A46FC6F698E623BBF6F05278 +858904B415033C1A33B7770D99247815E2A8F1216EB42EF9742EEB4E1D90EC3E +F2C0661DFDD996CE1FD66F946B9D4FFADF86861CF8EDCD570CB400D181697747 +83F38B92B1803874C24AF99C66E5F966B0E72B725FFBC6172341596D8AE51C3C +4676A65E4D32F499FB1ED170554DA7AFD79DE5953988516A8B2B123FA207BC49 +D80645DFB16A79549BD5E41606D80424DBFD6F5BA1DF0EC271C596EBB4FD0B84 +DC5EEC45C3FF598241368442A3FB06B460C7D46678069011FFAA3072402D4DD2 +3DA6139A548FA99A68AC61134087071ED8B4A37A0A8A1A3B48A2A8CCAA351D75 +1828BA7DD5B7758A43F98F4D449ECB58B20B26C40B35AC82623961CD892B5DE3 +777908625E1A91DDE62DEBB2EE5EA3CAD7E089F421DF613D976660D547CCB02B +3C6A727CAA1107C976C63F1775AF554AC9D6E72C27E58EB91C766DE3C9B18C70 +02F76150373FA95630C23C92C3B6C925388988408791753AA23161F2CF0CD44F +934E3CD426A8A81D716EF220D70D2BA19D0ECB521503EEC0DEB24F72AA850A8D +77DBFFB11716E43EEB769687DD62EEBDD103E5BAB7D6339C498E11403B2FCEFE +8359AB475B9D30634B4B17FABC1FD9DB8FACA1B59982A8A3B1E8DF5D3353B5F7 +55C1EDDB7D596D1C402F6A84BADF53D471D5BB4355CFA0E7A929A58AF9F079F9 +41C0B50A3AA4CDCA7295FC2C7968A3428E23C95B7E7259A3C1784CD952B889EC +8795ED2D3BB7672A24748FE8D81A6C443DFBDE5732BC857EB9A69C73586C4C09 +C73A8DDE73DFBCEE9E3508D57A95C197A14F600AED8F452FEC6AE6E020B8AE15 +68D0CFC52A228D9EF5E0F6F5273613D3368B9D0EAA2246EC50B5E223CD993117 +ABA04A1EA8C70E52EA1F91304C52F682DF4F457BC004367681F4DA2B6B0E7A65 +FEFBCD76D92B7D03A3EEA4992722F1EC95517A3EDB4596BA65F37E336FD4C68E +59EF3338994987D07CFE8BD0937970CB8A1054F86235F525C2470D54FFBEBF6B +83CEDAF7A570B1D591F63DDBF4F1FB02AA5982BDBD4DE8F64D18AFC2EE210DEB +0E22A03B5277661FEC7F4EB6E80476EB4A +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: EUFM10 +%!PS-AdobeFont-1.0: EUFM10 003.002 +%%Title: EUFM10 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name EUFM10. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/EUFM10 known{/EUFM10 findfont dup/UniqueID known{dup +/UniqueID get 5031986 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /EUFM10 def +/FontBBox {-28 -257 1055 741 }readonly def +/UniqueID 5031986 def +/PaintType 0 def +/FontInfo 9 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name EUFM10.) readonly def +/FullName (EUFM10) readonly def +/FamilyName (Euler) readonly def +/Weight (Medium) readonly def +/ItalicAngle 0 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 84 /T put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CD06DFE1BE899059C588357426D7A0 +7B684C079A47D271426064AD18CB9750D8A986D1D67C1B2AEEF8CE785CC19C81 +DE96489F740045C5E342F02DA1C9F9F3C167651E646F1A67CF379789E311EF91 +511D0F605B045B279357D6FC8537C233E7AEE6A4FDBE73E75A39EB206D20A6F6 +1021961B748D419EBEEB028B592124E174CA595C108E12725B9875544955CFFD +028B698EF742BC8C19F979E35B8E99CADDDDC89CC6C59733F2A24BC3AF36AD86 +1319147A4A219ECB92D0D9F6228B51A97C2954700AEB687D409334C3AA4C4576 +E8852B6EED963A2F12E1100C2F11AD64F9834008B17D7275137C8C6D065343D3 +6248876705271ADBF66792903C82D020865C425B823891577122B2D7910FDFE2 +F4377C1205D6BCE44FDE602980F41785C74039DC7E77C42814E47564A330585C +0EAA804F53354D2BF30DD8999BC82881D04A63FD7C16EB3DE7A78BCD1062B2E2 +7F849386AD6B44361BC7B2A27EEF08BF9252C9A554DB2552DE3D80C400948542 +151B97B4A6FCFEDF0AEA4A87151F6BDADDE1A02A27BA84FDE7C186444F70AEC9 +6285D8FAFC80273B979C260AD93F76ED3FD8D834918E8CDB5CAA3EBBC58211D6 +CA7777DE276EA9EA5E4306D1C4AB235004A5A92A878D5530D0BF76B3CAB2B70B +51086A20BD83153741973134DB567F287C1A40FA0CB348695F0A2DF000FDAA99 +63E9185B925995AC9E4379E85F5E2EE247D10282D8AA57E577CB9821562DB547 +BF4CB33A47E39B73F8832EB3460CFA5739653A69BA1D4AE7B37EF1B4BA92A819 +556EE6B71B483EF1E4F09023E0F10CDD59ED9F1C141885F065F053CCE6E7C3B4 +5AA830C57A7620D3A3DA81505E731C4BA2F9CF3B222047AF9EAC6AD77C0FB923 +89159CB2F90003255BEE1310D5493723FB75D7CBECE52EF2F52BD4DB260F4A86 +6DB8E36CFC5DF31870994DFCDEFC799FB1DA860E2574A3082D1D68F3ACBA5E69 +156630E8293D1CA94A40BD813F2E4D28E7BCDA3455FD734A606A8D1B178DDE19 +07469221E549388BA5C6675D93F522E27648ABBFE3397225B48EFAD1B4BB6453 +7D170F2FC43086983541177C2767884B79C0DEE036C410235C618DC539AFB4CD +5AFCF6328F5CBECE827E1FD2D6FB3737D876D3295406983A31FCC6AA4B7E9BB4 +2E02806DFC1BC32320CAA6886427BCAC16579607E1C0E824FF4E98D4A2D3852D +68722BC9ED1EFAFBC005208814EBC75AE76A94C40C7DE72A8FA8AF1E8ECB325F +81132BE4E2A2637B1D3B115D15A6353392965EFD8640B9D380F931A9739FCCBB +099F00C9D622415E4D4017C5577F7ABE95D3F13DB230F95677ECDDB8D82CA371 +274CD508F77BE835BFF3C38EF1A27FAF4DD9A0E5FB9F2B6CC863CC52376F0E5A +3DC147E438685418FF91FB0D7F2A7CF41D9E6F8D7CFDB0B6F4ADD20FCE139F5F +63D62BA3D19D147F3CE880078429BB573F3C8508AA0CF2CC7937F484CBE7800E +E4B41ABD773B5CFBF51FF4FA18FEDC002207FD56BC49CED92A6472D9D631E45C +BB3439D9229CFDCEDC45ED8A12BC3DB78426953385AC118D60FE14EFEB0BE54B +6D9904F63F9DE0E4CB3F161F682A173B4DCD9A97AB2DA5C3D8C43149098E19FC +8AEA6686FC1C2727E4322B7C679DA359FD3BB798F62AF4AAA4FAF3A04EF40722 +A0DE5E8D78BD73DA6CD98F2AAB93B57E173C61F086B71700EAE519475E954EC3 +4FE7B41765070B8F6CDD4806917ABBE8A5107391969AD95045AB41E7E390E8D0 +4A32BD366FB9206328892625ED99C6B3492A0F9D0559DE7BF7F55B39B9B3BD10 +2D00C7358FA032CF71AF7302C3AA6ECD1F25F006ABAD0C8EA89BF98148EACCF3 +EAA24EBD4CCFF079233EF20FFA32FE10A0A4F227EF904DA9BB20D9D9480BD607 +92636C36C70ACBC70798B1A16A5EBD7C3E8E34280A89BA07E07CF813E91C6974 +487DF2F85A2085477686918B6BAB721D968DFF3B291BF9A772A22295C9A1A010 +C535D796AF658A813305878D9CDE915B014AB17F461C9395E19933956CC29805 +5D76456671D1B7ED8431F85D9C3AC9D7E51D4AC509DAE09303EB86961A4C48A7 +7509F2300BE605C819602C6D969E003674241E5EF1E87A71E78CB6BA412B7EF4 +EF933DCD09CEE6F1BDD23BD99933E675F8827B7D8EC11656AFE1AE77B2AEB592 +2F994F39006720EF3B2093B9AA8FC84847A5DD0568376505531238DBF13C8BA0 +F9FAE24261F0A7855AE3F5D477151CAEC906B8CEC6E51427BBB28D8F181417E2 +DCE55956FA3F3283857B09B3577630CD06A43E8D9019B00A0E62CB549C7453D1 +85276A431C3A22370FC1A3D6428C63D779EC73E5D93CEF8BA5E68AE9DC6BE6CF +6EE94659A3EC54EA34F8054B6B2EE1EF067EE093B40811FE85E15EDC61F71A55 +C47284266277E1C1E18901ECEA82C4EA183091006CE95E9D9441F60955EB43C0 +B74BFBD1E1FB99BF3C6AB6BB665E50CF7D42EA10F036ADDE55839581F427E156 +A72B4E8D10B4A74D459C24D3DB747755F73590712E8804324670EC25789B3AF9 +06483AB21635C075E75A5656F228E58888EDCE5EEA541A30DBE91656228398F2 +437F320600C0F037C9E95CA50067EC8F17545C7C0BCAB973F211E2EC7151A85C +6B52C522B085DFCBFB087BCD5ADFB29B84B16AAFDC42D1EFEC922409D33F08A2 +270010D2FC8C8ABA455EA2E866760604BBEBAF382ACDA623273B6915E3732BFB +8401E38AF114E6D0735A796732C0A4F2573EFF7C82C8D149003E83BB5FE9ADE6 +0AAC56947AA394E01103B515FD73BB4167DB0CB148FA20938A8098991FD78142 +FE14EC1EB3BA4B9F24AF8FCCC0FE3EFF0CC474DEE6AE233D66EAD198B48AF4C6 +815425A863D1C38054390788F9A529E53ABF08E5C0976F403E86F50EE4B83639 +9080B37B5286E62992DF0169898A60B1C0B83820AF640A157F0505DA970D6A0A +B5C36D4293E29796621E7F428D3CEB65704AEE15EB5F575EE40D1EC994DBAE08 +F4492A2451C7E3A801A03FD4F10EE46AAAD8AD5E5AF2092CBA6C392DC408C0F0 +27F2E4FD3A6A5F26352F065937FC6DAE1B838A8DEAE7B045F16EB98A0282EBC3 +0924F6F80F52F4A013D64FE8156350B079C4DD410BFF6511E062125F1452402C +D702513F68DBAE266D3AA3EB6EA903FFD7D1D80B9A61E2D4CEFA84385F4D412A +5E08A6CBA614DAEABE02DBAADDEFB3008E57177973EDF2765CEC2A7CAC2D33D2 +2ADE1F5C7AA37B2F50ECFBFF71399E8199F4C31AFE0C2DCB46C449272F3A1D8E +7AAC8B6297D4BA1D8FB2BDDF3E6C11916D8850EB9F015880948CA564A6D320F3 +DF313BA41D5EF386AD9367B66829A40DE1BC753BC8BDB26467ACFB504F627D44 +88E1CAF6EAF28E1AE7C229616B277A4972ACC39C3F5CA060D848B7C6B52F9C49 +A8FD03CD59DE1242321681BEF58A1328DA0F04C8245C3B46E21421D6236A0FB2 +4AAB7212A4884FD68867A4CF56D83882284EE9B23AD5D6CA3CD06A390C52E518 +C6555D9EC12FFCF0FA22A0A3131F65DD8887FB74551FA2DDB695E0A2D8076852 +9C1C2474C5B226D2DE288E1C045D895AD9DD77F86EB8E477B58ADA6EB39D45F0 +4C9C31EC9546088A2EAA16250C3CB2B69744C29EE17BA5DA565AB595336F6A4A +F4B935E3C3CB45E3A781E1BCD215ADE4EA41CE0E4FE7C6AFE376A91FF4B32BFB +4907875C95F01FF669C5FBD1490EB4C0494599D938564947E2EB66AFDAE42C6E +A146A87FA0EA5A4A87300D0E0779E65D65FC8150362720F69F7091B8AEEBCD4E +1CA80254BDBB46DCF63FE6C639F6BA0B53216BCF4C3DCDF3F5B6ADC750836480 +8A81E80CE7F64246632EF334893727F45F53B77283C341DEA3CA971E8F0281B6 +0A9FC2A262FDA6F6A08AEE2A25FFBFD209113E689E2CC3B6C7DF8A4D7D83918D +26C7D9E28CAB82A4E08783E1D677E3BA8B03B1B850F18B6A5C75DBAB7EE8F232 +9BF1D20D4BDE64A7D1F57EC7EF76B56375D55178FF5E6C9A2D5BB4D74B8A57DD +7251F041335F234253534648D36D008892E7D3377A3E117927879A854150631B +E0EA648CA7E9784C5FB129B77A7ECDD0E55125D37225DFB34B115A06BF80F1EC +9918C6739DFED6DECAE3AFC029620C6D2BE5458C8EF2CF28F373E5B98999F31A +CDC03658FD6B089A2E939B1ABEDBF965C895C7392EB3B8ADA4FB9DA7D5FA250B +43B32BEE49AE20D02C81D693B4C74F2FCEA305EB969074C8F64C7F92BCCDA499 +DB8C4AABA59278C9BFC68F2DFCE626A5AC8F2324E196EFEF2A4E62029BE2DCDD +DBC8B6071960E3DD4BE58300863172328C3A2F612E104A31731102B31EDB747D +8D2687A1C20E2BE3675F4AE46AD26B06C56526503716F993E5CC1259FBB589E7 +C0C3D57AB03E869704B75AE5E370C218303348F32EB3BF386AEEA427C0B40614 +2C4370E468CBE81F20EA547F40EAB8B859327F455E579AF787D6ACA83DB2BA4C +EA6C099E32E1A3FB511353A49BD652B173B5003185F2C7C86301594F540AD7E6 +3911C6799F334096CD34AFCBCE2B75118A79A6A1A93A064BF07F6AE76DCA13E1 +E6D01EB7C5B654C6DBD56B3FAB7D8A9D0622B353A216E22A314B574F03097C5C +674AF8DE867FF051359B825B049D1456808C62726456A7AE9B101FDBDB55B722 +B3AF027BB211B00DBFF5BCC7577788515F69B5E58247693F9DB6FA10F03B99DC +316A6BC6207D5B1CC8E950D9D4F5DF6DA101027CD0DAEC8B5C75074FDA080EFB +EE888C74D622664250554DF7451D9CD4CA640CA92D92EFACD1092A43EFD508D2 +62C41ADFFDABF3E7CF7A53E6177AFF2AE269AB7424E3C592DDF1033AA5D3580C +C29DF68E3E30D79941571693D57F94F0E3D87F7EE8E8247B300EBDF80FA429DD +CB6217A45EDD8894A8766FE0705615D250501BA2C1DED89160EFBB5B16753E0F +1EFE90D5F0227F7D76641515BC41FF4AB1132A9DCCEF47B58AD0387DF4DB078D +F87A7EA145DCA6AD24F98DFE04DB5F305EAE66D692F56103BD5B3A57623CCC60 +8EB47F7581CEA75387CB31D9B35BDEA0EC5F9E90D8CEB17B06CEB045392820BD +856C1FF7BC427A32F6B8C53DDF041B8D373FA22B504E60BA53277503C38C5B76 +AD80A21DF0B6FC5AAA58DBE8AD8872B81F4484AE450306013F69EDAC486EC64D +73D11BF67F575DF605F929CB8202FD8EDD1F5449C89F4D3B3891E45B3251D97A +7E9DBF742BF9C72CC02D0E742F60C12F198BB5B698A5A0AC66B243E1A8C611CB +A0B8AA2DFBBBFBAAC107389ACDCF209F33A8149BF7D24C106D0D59BB1665A43E +27F3C4E5DB68F654D3E364F599AB641DE380F29218A23D87E2ED05CA888A1E87 +34774ACFD3A870099D8D1C192E955461F36C94B00C732B7F2BAF532C71AE6D9D +DD781966F7EB77DAC4D33F4901232F4B4E7FAC2AA452DDE07B090539AE7E20A2 +D6BE09AF40D8357C7A12329034E2E9BA70CBE18B1CA05BB535025A6361CBB0E9 +7602B0AB13977D14408B756398FC0EB3724EB0B40DDEAF12D80C51066928E241 +4EEC3A3BFDCDF1D3AA1BBAB655AE05DA07499C9FF5A19096E67FEEE762BA61C7 +EFFDF956FF894EFC9FA14014BEB7DDE1AD62D4AE44B391536033CF74C86F19D4 +4368036018F93F2F79B10B1CAD5C454F19A10EEC2BF92C5FF8FD3124B6448076 +086840BF6E4079A2DA05BC8A1215E8CC8360AEABC591EC2D3A386E81C47CAE34 +F2CD9B45C6247F5561BC7CC4316C52F20A8668C79D8ACED3E353DE19A86E4812 +F028B8E693A2754A3BC77FFE2EBE00D3BAA847106E66161A1F14F6949A7B175A +6CBE5934C8CE635C96A24E5DF7B46AAA7131F12F11A5C168CA9F96675790015F +128E05EEA1C09CF4D57B99866C611A02EB9AD4F19B0D306CE98971D1EC585F35 +2EDBE736E406027E6E7016B38BEF06F005C14890DC5B02EC2114D923874D8C59 +3CB9444570B6483A78CBA2D18E57E33A5757A59984CFD0860D5B3C2CC781E877 +43B5E6825900E1E8E705D9BF121212B33E8E6C2FFB0D84D341D6C8250410517D +A088F3D175197D60BE2036ACF8413D0B91B02106EBEE28FA11873B80EA1401F3 +F99D101C6620E9802BB78C4534E80C2A683F48EEC7A62CE4831F513E8FF4F97E +56F75C44C5F421512BE0B0F6973A1CD462D61EF67CFC490057F0C07023B87D06 +908818A6FA00F3815CCBBDB018A16BA92E616AEC5F7AB105DB5CCD8977B69AC4 +75DB731D2AB714360597A0436FA72DC70380FB47650EAB4404436AEA973A9F61 +26744E1553BF842ADA45BA01438746009D87FCD25F40D7B01F08918CF0B39006 +CF85DCD960C17540DA574501B7BAD490881FCD90C244C47B912AB0D77D7298DF +3E7D91995D9CE83A4BEA802FB9F1226A17A5F9A3BEA95B7A86FAE92CEBEAA58B +A75704FCCC2C1B0D49AD63F8F9D1E128A0ED6DAC69203F4698DE624B8E423601 +C7626C05EBF39246E9AB373D861681AB6BCF48F3D49370B11F2756E79328E1C9 +651392FDD8EDEA936C696CDB60E54779DB38665141CBEF463D798F4B6C668023 +BC5566718F4B3AA207493D44FC67E686D8E3BED6D06411FED5B52598915173ED +BFF8670435AF434AAE1C33AF9A88F858FE4C12B64E0ADDD71E2C2FFF016BA124 +CC327A0404AFCF9C2FDDA96C26C4914AA24279782820ECF6CB88B6AF8282A9E0 +998718BFA6FFFF1DBD12992F3EA027274EBC2D84D3E7D41573355524DA75D159 +2661ACBE435BAE264712C975A8AA5B8E246E4698682B54AB931DC6838B699834 +7AE55BE022B807662CC5E4F4ECFDA1FBAA417262F8B6BAC14263692E0EE39765 +D278021BEA0D3C71FC98CEDB47D16EAC540E58D578437C1D8361CDCE0FC7FF45 +0AF01EE12BCD35F71ABE8C6B767DC95D956F2AD282185402C30A93EF973770DA +CB8EBEAB7A58460B8CD3F6E957D9E233478B3A06B0DC72DAF29EA5CD341F8089 +926293195A3971FFA5C9CD02F99CCAD7F19AB98BE763224B19616DEF40306EA2 +2023A1B9A595D989631244CEE6F42DC830855C80141404CC220B2492565C5D3F +24FDF03570ED0DA8D729D7E8DB8E51C06503C3D9717A92651035E58683241F48 +992CEEB2DC24047F16D9D2776C32D8AAA6B654911DA4FC34EA3D7647CE0BF768 +19C94F27B33C5B1ED265BD64F2DDA79FA28FFDF3170D3A59D9CE329D883B50AB +1F6E9256B4C198EE5DEC65434E241BBD28CFFD8E5F30DE477E384CDA277E4BF2 +CA4063540D14615268DEDE79305F6B8E033ABC52897C0D508C9906E481B1BCBE +AB87A8D3D528C5AE35CDD9ABE39D612AC5B337026D8892A0DBDF538A64E2AB3F +A834ECCCE3E09DD7D05E494B76E31C88062C6275890A9F1079D0BFC7D4F58DDE +F6CBD261ACD98E51F1116C03D3D2BAB27C2276C6F4242AD23E4B93EE63EAAB97 +5D2F5FF5723364147B5A02780D46BE9F62A701B585FDBE40A8FABBF35A757B18 +1C4F3070D85A0032D29239552E4F770AC41D49AE91276D376D57432D16258986 +1EFF7C7D9AA542FC01290B7F3A501462AE509969F2C2ED81E088FF7CF1B8A118 +E9DF4667832E097B0651FE7279D472D6975115AEF6EBAE731349D4696EF024C9 +995A75536AAC35B8CCF5B2304E2AA418435F250BCC993320E2D571D0E82C1BCC +DD91EF8466BB12B3FD52518C1CC8E1C5999850D448177049A8773556DC31024C +BCAD8B66129AB1297C0BC7AF03C63A867E7A597E8881988827D61141DBA2EDB3 +90AF992AB04296B99CF39293EFD019B78A50FB850DF0C675BCDD451292C5068A +4F5D967A7C2EBE4E2D5DF7E97969E68F5CB6DDE7945904AAACB8EC1E5A02ED87 +1FA08A4088335FA9C23EAD7DC0279F2DD9CA9749321E5A133FA22A5FFE05D92F +71F0D192378813F39779B260B6B5D3FB56F623D7C5F05796D2406154AA78DA06 +435AC0EE613CEAE3112C78BD3E20BF60D3CC4C8721FD8CBCA6B1BEA501F351AE +16CDB2AA0841222122FB923C46973FB2E190523B19C4BAF7CE0AE35B006E1ED3 +E0F73D754CF7CAB03CA3366A11FDF24772E1966FE6537F776BC5DAC5AD5B5F9C +815B4F86BED54E4A1C42CEC8030092ED99E78819CC4FEBEAAAC220439FDC8F59 +AAC8AA545C3617FABCD7605C487A00CD2F7BD6810796E92F1FFB71E7BBC0DDB6 +2F905F9EFE36171713366FD9AB35CEFAE60FC97C6B9EF843E74818A90E27D468 +5FDAECAE65DF331D3DD3CAE01EF1E27CB74C4AF7625027B562291999274391E1 +C23AFE3ED42C803A9D0E87E1EF71AE0732F89F6DCFC226CE27A83713347F7ED8 +8BF18767637093A0A313154FF042EF40ADC363F94F2CC9B0606B790D5B476C1A +FC610D1F8DFA49314F27EE8C681F79CBFF4F970C58A3622E284FD0A3624584C8 +39CE100683A0B37FB647942E839101E4FE35829CFF931EBF83B2FD73B18D239D +5BB6F9A2DC70D5CB34D4FCCDDDAE19B930F82A6D7401A83EC2992BF4BAA3DD67 +2D8102BC70E3E35309AC87625F94F964C1490A67675785A3C5B07B1F24414F8D +1DADFED229A7046AD7E03A90C8FD8A7DA0654AE5D32E1840436FB73A1DF14B79 +2B7B44B94F5A909E267621A79AFCBF380482AF87B4BFBE0479FE52AACBC5DF25 +E08EA909AB5E7025266D5F6FEDCA001117F5342EE87C072939CCCEAFE16642EF +DAC870BFF53552CFC8FB5FE4A70DBB3BDE6451319324D623DED2458F7EFF2A10 +EA60ECEE04CC749EB310D1B49B345A1F152B7B2931D03B8B63012D064113DD1D +5C175AEB61FC962B7E1DE3A55FA4AF91B5BC758C7270B6FA659EDCBF94B0DC4B +7B39E620C89875FA9924B4E5B107DE51A1BF2EF2CD369E5470EA573603C79004 +6FF9A0FE750A16DB41F5B59755581DDF6DB0F7C94B8F90B193A22873D4DA4102 +A85353 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: CMEX9 +%!PS-AdobeFont-1.0: CMEX9 003.002 +%%Title: CMEX9 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMEX9. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMEX9 known{/CMEX9 findfont dup/UniqueID known{dup +/UniqueID get 5092765 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMEX9 def +/FontBBox {-25 -2958 1495 772 }readonly def +/UniqueID 5092765 def +/PaintType 0 def +/FontInfo 9 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMEX9.) readonly def +/FullName (CMEX9) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle 0 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 0 /parenleftbig put +dup 1 /parenrightbig put +dup 2 /bracketleftbig put +dup 3 /bracketrightbig put +dup 10 /angbracketleftbig put +dup 11 /angbracketrightbig put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CE32340DC6F28AF40857E4451976E7 +5182433CF9F333A38BD841C0D4E68BF9E012EB32A8FFB76B5816306B5EDF7C99 +8B3A16D9B4BC056662E32C7CD0123DFAEB734C7532E64BBFBF5A60336E646716 +EFB852C877F440D329172C71F1E5D59CE9473C26B8AEF7AD68EF0727B6EC2E0C +02CE8D8B07183838330C0284BD419CBDAE42B141D3D4BE492473F240CEED931D +46E9F999C5CB3235E2C6DAAA2C0169E1991BEAEA0D704BF49CEA3E98E8C2361A +4B60D020D325E4C2450F3BCF59223103D20DB69434BD2F5E56C8FF46EC332DB0 +76984939803A4D2E4F7065326BC983B6DC4219499435B304C1AE83C8A14C4085 +131D1CB66E24EF3AECA881EB3CAFBD7CC6E70F905DF2FF583BF11F2E5A674AF9 +11C625E782E1503243619CCD6B1222599F709EC8E7657B1053FEE4D83E29B295 +494CE2255E9D555D71DACD865C95E1A3ACC8D19A468D316142A854F452DE76EC +B45C35C92BCCA844FD24BF5D5A43BED6C829FAC12BEC9C216414BA7783DB2747 +0B114774A6508C9D617C38A99EB5DCC26758D9D55A563F8240143878C5C49256 +AD489CE0A01657DE0051A2E845EFA4EB61043270B4A8E5E0676E90149CAB94D5 +AAA65B9198DF0EC8439A194D5CC14FB65FF3BC2D67707BAA7B23484DA100F9DA +DB0610AF3AD273DB18B69EC987D6D9A0BF699BBBEA34A0AF1AE401429DA0C85B +AE89B1AB922D16074CE2208BA2C3F88AE84F7BD87F4F23EE64789BB62814BFD3 +946734BDD1586BBD77254160BDA350FCC2C30B7F6B6AE44B928191DD564693B1 +29D840BF729843DA6A707EA375F7541D398527AA15017B3B625000ADAE91FCD5 +B749C01D15B2A8C60E3E4D97D5C454974BFB02CC706122041709F96659FA31D7 +C27D613F0B45ED42ECEA2090986FD97A3AD736C9310B7FBBB8B6FDF7052B9941 +A609E7F7EBC0B652B6FAC06337ACC5A1D1260454F16F5EC6C967D5B7402600FB +546B55667896C188F50BDDA2E195EC55C0A7B48D6A025845BCBB4B567BB19946 +69191A766E28660A8DC4A15C8022948207DDE6E7451DDDED8376E52631E4F0D6 +B947E227F99E03605688E5E28EDFF6E8033E25FBE8F6A86D5A26B2937FF48C14 +BD9AC06DEB234FD1669C8F3EE9B8BF3E17139E08C308A3BB0ED0AAC81291D8E4 +2C1134A3314BCC0D93D513E3EC0D91C45C05ACD19994AB285B896334721B8C42 +420BF74A5CFE7CBD243D3DDB163362132A0285C280277B4C60D9BFC3D40944A3 +943697BD335C27F16ED870DABE54DDDA4BB5C3868DA4CC1C0DDCB50826700ED9 +9DF963A75151C5BCDEEDCAF2576E0E21288D194E540B422BC03B62CFD137E732 +D0660236BEA09052A292CDC218E48181DEE1BEC6548C828AF0FBCCF10BDCD4F8 +B71AAD95C5CF688815B9BF856FCDBBF769FD395885A88F4A13FC7B9440BDE7D1 +90324ECFF82AD30375BF19834A5CDCE28FFEC0FEFE45ABF372FA5C6F03D972C7 +AB3BC52C7E1C1783D6F80155B3A5BD54F19E6B3FF7155A6DD083438D0EA7AE2F +F5BD3F1C0A13438F4C42B3379D2868D688C14DF6202B5006BE89F09FEBC4EE0A +CCC50B10D3F9E56CE282FBA763D6F46CD62A860CA0744267D336833173C93951 +87995BAD6E789CBABD53DBAD179F29DBFE73E9E28FB75ABC0C3014A72D9C3CE0 +A80412E418D69741397C867907C795A20D45306EDCF032DA7A4A248C8FFA0189 +2552A041D06D13826860886B2FFC1361EEADCD6BE3710CD640E881B2811E81A7 +45BE7BF836AFDBC85E5AE36274D19BB1A12083A35BC231D6C1D98F8470ECDF8D +8AFF93893D5C0557152DB776FA755663D4E1A7659DD7D7206305378297175CE0 +444B24650996FFAAEF4A23052AA4E77553F992DF1D6819C195B0B962F5571346 +7CC036EE1D527B68965DC2EED66818821AD02BDB5A27B3129C43986C3928E391 +BA170B1B161818BB2E2529024B721839DB116A174DCE89F92FAF4DC83033434F +C96D97A6202D48E7FADD753825159E50FD051D7D268C5807225D52D161569EE7 +F94A99E78DB76DF5860227F5B6ECB1FA255E162C3D2AF6D96099CF3286E96DE5 +31D324EB92FBDCE0FAFC586FECDA5D9376D40C9B1A43BF1D52D3D54BA06745AB +58E755921509C3335F72DBB2366726C4CB688000C1D6BCE706A3DE168DD0D5B7 +45BF139E205273B248F5C4E3844D4CD211E6208A2BBDBCD88AA8DABCE4FD0594 +151AAB58FAF716298DDFFA7A97C02AC060A114FDB3A0FA6EF7B8F1CD8606BE46 +8B7DAC1E72788C555BF9B5701865B263725AF314DB471239363AA8437EF51206 +2181B31E56C30103126582F77EFC23CD8F37DE389C16333F6C4F78699D187746 +EF94B8A9BF312AC2ECE4F23F9A7AA0487D09CBA71C11241155AFDA73BC914425 +4943E10468A243ECDBC5E70A06A00529EC2DF656F21EC7522BF41EF0F8BF7D35 +E0E76D6589E77900943C06AE933DBE3668099BA91C38C19E945BDDBFA4FA5C49 +ECDC4F183CB9189609F753DBE5D5B1AD7FA8BB9E28E3BF72DE2C604D88F43552 +ED0C3D02277BEDAB0DB5B7A9CCABC8932006E8D44300E748C149C08D41F86608 +F980E68813988906B7A4C5436CA1C10E54EF608945632E82B22021193A51342F +5530D708792FCE9BEEB224D03DEA481107BAFAFEA271D16C3E8B89A481933D27 +03A044FB481748F2C7CCDD0562AC53F9AC2A9423FB0C7DDDE764022EEC181FC1 +1EEED102ED0F29A61D17004FACE1A1CBDB8994E2AC2F88DA74B173C45362EBBA +597EA55D327886FCAB9221CE187D9589BB795BAA03F931CBF86DE3E8668CE8E0 +F8FD3B098CFE2ABA2099BFF8DA83E54E9996D2DC42E7D9416DC2D7FCBA3A0DDC +B8CF9B37F7EB7AAAA6AD85297B042900688BEDB470E6E38CE0929AD5176A6C7F +059174C1123F3AFB6FE77E4CBA0394C9A305FBC8FC817BFA6F97D530C77AFBEA +29CA42B9551906D7187398AFDABD2EE9B06243E95BB467EC8E0D895474C9C455 +1CEE93868A6412C4BEFF0BD0272BA95DDCC0369AF8160EC2FE1C60F5E535D25F +B0632C01BF40867E507E7BDD03BCC3BC21337FA3B974244E5D4591437FDFB185 +78F0176F5DEA5803EAE72E013AA7FFE07C40EB971DD7937DFE111BBC33F4F8D4 +5504AA54C414D94A9FA908B7B5356B913CA1AC0E71BD21E3CE550226FC5ED00A +86C05BAB4571AF93066C4F4612CC3C51BEA3F241EAE68A79284DB837F5B1904E +B642AA64A984EFB35F870912ED178F53DBAEA4A0E6E116256930BF507C01DE54 +8BB70AE4F99C5639BE4F065C51C95D3C97512E6C7E2E796E90CC540B09DF02A5 +CCD2694E35C075BA264B1C1C83CBDAFCA48807A008B1873273AA1028923AABF6 +6660CE500CF174183CB593014D8A272EBCF3A09133603CFFEF43A6FF059BCE52 +5BF232A5E067BADBAE7D90A56392F6B5922443B8A14DE64434A0018E21C1CC89 +CD5D85A03022B4D80FC1C3C8740140DD6793C65319FC3E906C7CB714DB22068C +A66714E765A17F446CBDC8DBEFFD8A8E284DBAA179C9DD9BD9FD0A1DCEE171D9 +D5E47F8246FBBBDEEA159301B189724C21E5318D1B73BADDEBD21FCB6E8DBCBA +4872FB568BEEFFC852FC84255B5014DB80589DC29AF1EDA5A57309FA19167514 +967F9F9D5DE56CA3E66A3D937C041E83C29AC39C357667715A824ABC7A39F398 +60C43B9ABB86F0CB56F9CDC50B6CADBA7938D020CDD81D1B59430611632E2368 +E7483D596A272D1D99B5CDF4BA2B66C9F50625ACFF60905652C251B724FF8C68 +D45C0174D12B87E95E7D480DADA7EF001F12A1B3528ADEFC3052E7CBAE406533 +32CE0D5F96B7A2DF24ED4B5C6FB220E751257161A0F095017D90E95F1DCB6B5B +BF0413399B722A309C1155A0A9F2903CBB7F192149DB130BC24E5C41056FC750 +1E83C4E6EE02EA45465598DCF7C4892D00F14A954A4AA6C2E616D39F09BDEA3D +27F0BABE6CC9D417A17D3EB2C138A842F1E53B30F58F05C9C77F3A0CD8B3ED6E +5F67D05F551E95D9D1133D33C9F8CF57839C998E204A1FD39F24EAA20FB6B3D4 +9FF31D4D1826745AD4D3E95EEF71DD2D4E6300C42BA9316F1560E2D9A094007E +79C04433D225BFDFDEA8F2E6D42C456B722F80EF4223ABB38991344ED1C78CBD +830400A52BD0F2253E20A30122A842971BE77794DA8CC6290EAFEC44E3273E10 +54A6A2AE4E8F8072FF4A37265CC0C1FC4F773190B439DC006D769F3089928C4B +DFD6A6012A39A0C8E997599EA3F4A73FF32C011819711BC6FBC4B6260A02DC29 +7348188F3B7EF0BDEBB8EAFF7347A3023C478F446CE904FC25AA2DDA51D976DB +584A2ACD393B9E9C601AF9DA9629DE5C003F742AFE6D414A9BF1377D823B465C +CCCD4A7D7D108A9D639C6BBC15844D9CF1AB8B8B2260AE601E7E9F7394E212B2 +693B47C8AB94612A779255A9FD9685585B0D1A7FB8B7E586197426517B122055 +CF9A5207EEF696E1B881E64754AFDF0C60838783E6BD49D470FD0553649DB0FF +7B8CC86CF80D05B0701580E75D874515C4C8A7823CE924E9E7ADF1C97085A6CE +24EE43E7969701EE5D3E5F07F59492A9A946F300DD83F9FBA4B8CDE73C12C563 +0691D30B57F253DB008E4595AE2D472376E88E30246642031DFF3132F971D172 +19F4B6C6F87303ABE5E1B4CF1AE504D1E4B5BF5C86E232B42304840A9BB34E73 +BA98A3186A01B72D98C92032180DCBD3A8715016557108B939293E23697D81C3 +FEBC37582B7F7AA541C220DD5D93E555AC54D7D277AD6DE29158CE8B89E29C67 +A305FD1F200B5ED02588CC1E0CCC93F9C8CED113066C401E1FCB4EDFB49C9224 +0FDD03519559B710C046DF2FD60DFE4029FD9505867321D98507F0523E93739A +81A9E72430D30D0585977DD323CE222D408E4E2DDA7FBFD91A1EE1F75B530C6C +A5F6ABA3BB9B6CF94C29837A4186EA933C7BA60AEA9CF1F7F632F5F561A4CB79 +CA533B32069D8A8D2DBD8089F295FB7E0631ACF087B7E777BF77EC394DEE5BE3 +B69A27B122339829F3593ED92F42FF2721840133DD96450BD9DBBCF984B3383B +DF60B5C6603EA17DD5A3D0BBA5C61E1A7737E5F7719E2CAF59EEB87293BBC72C +0BE5F5A97CF28FD8CE19177B37E8C223549F926B6A979F89C06FEBFE86178B82 +44F429C3C3C1F6506CED46DE0774ED12D7BF94756D38A8B1801B086CE71FBD37 +8C17A639A05280033FB5A7B8770A953C0CCB14B386D336C1CAE1CAFEEECCAA1C +936164553CFA7ADC9E540353BC77C57AF8F96CD0FB7C41D5265011C86FD66BA7 +993FE45C8BD006B9BF0CFCD721A80368A1C4DD30EE15EE526D6F511DD5B42A75 +9C53639E5CD73845FCA08B21A3EF803C0DD4DC61598CC3446AE68FD548B9F5E3 +94B6A56EF1D7BDDAED84F8213EB7DCC84E699BD985ED8E5CFF2C729C02A925D8 +B7906366BD4520AF1B56A6D16D5B8D1A9B9AA1EDD2CCD597BF963F42611D48E8 +D2B4F733DAF0336890511388ADFC1A8801D25944E95BB859DC761B72B9E99A53 +584DB945DEB895BE8B6CD42F39D9568CCE9F89781614437088584A57EF884C63 +12FBD3E7874C42F97E8E2DEE2130ABF60DD60A3D84224AF5F6BC06F1DE598843 +C9016FCA08B9A343FC5638A5F849C84B248C3BD487895F602268D3E6F5EE8AFD +0016BEAFE2901B570E75465A86AA357D15EF02EE39D18AA90D784DD117CC911D +47BCCD7B6221F4A0741BDCF8BF30EBA66596F2164C5C9B9BB1E013C9F92D0E26 +A8C82CE7E6EEB0870BB2A6D09BF336CC2AC4B35D740403A60C31984F11B0D4E9 +8A8B95DCF96A7BC07C1FF7510EA6B709A2FAF612CDF75B71C1E471ECF2650192 +4643D9FCCDB1FDEC4FD158AE04768C099C7CC1FA1F693F884E04522E179C9E63 +778D704419A3EA4217A85AD8E70B6BF53F87430E511C3FBDEC1909CB52912EFC +3E645F84680F4750762BC29CDEE4D355E3F2DA698F474D585E4068CE0BA9C253 +89E76F438987B389480A7CB39CBD601D9A2A5640EE3894A345548F18269D275B +D43F757712E746E7EA971055FC27159818DB2E464628E362EEA21A9C5250AD98 +58DB14F128F2C1394D38F5D12417141DF703A471BE813269CDB23E3F8AAE820C +65A72B14E8663ED16AE4BB7B979545FA7A3EA413E6435E0BAC5F16EDA64F5AC4 +5B3B4188BACE769342E1B5405A3D87A3E8D337E8C6BB74C0FBAE053BFBC86746 +3ACA563C0B70FE33B38C5F5C0D704B90D8A081F5A21ECDFE0C800CCFE29086AC +1AFD3E9F3A37C62656182F8FA82BDC9B6D818FBC9377271CB5F4F60E2AA1422B +4A52A35434D55740380F28026DFC190C9FB8F2D60A87E9D6C2FB65E3B745EBED +917DCC5CD77BEEA69D2C9698A0A2BB3FDCA2C338B1D4FD47B06FE48A1B643DCB +AEFEE774A7636F7D29FDB77B8DFC56FB7769521857ACE49C2A46A19EE83FA941 +F1F79A39384D9C0C69F3AB2E22B142FAD9F6EC6F4D9FDE36E12BD766206EDAD7 +91BBC76CC9F6EBD6668BB435501709CBE1CF23C32C5C2A877B57445EB3B879D2 +FC850D78BCB7E116C1155981F8B1B7225FB495E0927B42DE27F16AA7DF593762 +67A6AD250B378667E70198B04B040132AEB3747E89F9F2A213AC9D008558D32A +23DA61B5AB7D18DF90C68BF29FF638630D5E69872C3B7D905182BA65661CF042 +EBD958B8648F5B650425B86336F3F57ADC4C3A02CAAC08E87DB5EBF6DE8C6E40 +CEE8B7138105873B76925E5F08EB999C4A24E1F3EBEFD79B17D31BC0E2D12C4C +52708F03EA1F6859B3F91B885F76E6EE97D3A813F2635231F9C93D642EBBF666 +EDE5B814BC413C5B85E31DB99EC2A1D6861801C7ADA21BFE167441B26E1E2BDC +6BBE9A438B41B5E3A7006637708163CF0E95AA0C47C15DDA87893AC37C3130E4 +81F084FA674E69515E490C819676C2E4FB666A4125C7677FEB8C32299E6751AE +0ED2F1CB6AEDC00E0F8D8EEDC6C735A5CB813F1DCC95BD568030E63AD7DC317C +6F61B99A39C65B93D0828817A1079F40909A01A2F11A01FA256D2C57F7A28BFF +82D4E3E4CE220F270536A6424468A2A55C567AF75363C7BBE849BA0547D3867F +34A50839A7061287272464F0C5CA47C965A265E044A366CA620896AD19EE956A +8049A30C2B93FA109424AD69F8C369379C5B6DC1C695DB16E43912308DE3E3F3 +18F592F0C2559951F9B1ED4D9B66D0F691BE22D45361D74A216C4D91B72AC393 +28A28481B8377D8A20E2DCBB87FEAAA6C47861D12AB22FE1DE62CC65798D5614 +58AFBBDE546B5712A69F199C1D84CC20281C24246CFBE9B2DAF3A9DCDBA4E5D1 +BA82CB7082E3F560522D4947B51182431BBC02250648D7D2B771FFE8B272F269 +14D4A76B275B1DB785C374A227624FC944DD8E80A0316EFAC25B7CB975195EC8 +B248452D9D09C2B304FB2F5D416BB6387C2B0EB06223AD089BB9BF21B6079B1F +22DFF38A0E248B9733CF97D23214A30167C32E63803515168556062D710FE258 +C6E3AD9D3DCB888950940F7E91F5B5D25C03B5A25AD033C8443DA01914A20EA8 +F2E6CAFA15F6E10D5645DB2EEF7D627BB5DA85DA1DD2009CBFF9D2DD159FF95C +211360B517EFA263CFD0D2368C13381D3BBF877CA98DC6ACCA103B099DC2F3FA +B7A6C1A01CEA32F205716B83CCCFD20C8ED17A3D778B36C1F28C88B7151637D8 +ACA1CD904829034DE2B388CCBE8789A647604825949FAB2921C78C670DD741AE +E988968EF346E13CE3021D1A0DABB13746EE4D782630708F40346358D1E71DCE +A3889ADAED210ACA14904C4F06EF1F86542C5ABF455201647355CD7AE4106036 +F1B7EE78A25934E8D077D323F961A71FA3CCCDF5A2C468B48D08BE2763346935 +0AE028DB5DC32E84F6D20CDCC34837AE20D6FBE559C10A65DD5ACF4BD1AF9BD0 +2D5280D6763618A3803F4777CB8D89F7F96B93EC116373B47C98A95D9C332C38 +4DD9CBC706C928A414FA1FCC656C3D91023125015202411418DBAEAE0472AE86 +C7CEF889388A03B9BADC361256A312AC34F3A4D860D8913EC169F62D3F454A67 +DCD2AA7A2527B1C5D97A19D1B3BD7ABCF3AD100AFC0E9034072DDB3C5E9798A4 +9E1169F9A3F7B02F3B296B007B58B0C6D339F22A74D7AFE603133A68D0F093DB +579758C48EE8F4A0BF1105A77651F1187EA3D249A27EF54454648921301F24CF +C5AE350D7F2AC05B1116EA029A87DF78E35B96E549B9B74E7D804BF42EBD80D0 +E4AB8E4D1296F3D644CB65D974826AE82940A572E9430A66AA87A3EAC94B9AA7 +6A5242253BF167C43B1DB2E3C6600355FD8C83A461FD54E32EFD8E992D8B4184 +16295C2E5C5847E58B67AA946FC892391338DDE87062E01EF9F4914F8627CD9E +4D0EB45BB62535AEC9E73C10ACB3F88BD99CDFC163B313B7F88815AF3FF8E199 +8A2904B0467BC14F361999E7EF0FDD87F98844CF18D3BBF8C793F361828C78B0 +E9762F6F2860A846EA6A26F10F9329983100DA195D9158737E5447B421A2B46D +B4096871FC65E79F2CD4DF5DE3B18C5E140BC00099EF7EF7F6DAC95A9D54DFEC +E813323A101E725A14BDB45D923760EE2534AF6A65D51C8ACF117B992076AAE3 +F68A6CEE21532B4AFFBFAD7D994E53C6A5FEF74D396569CCB6461F1CCAFFCCB0 +5F787026FE3BC0AF27C2DA214606A83F104516310BA1C06DD22AC1DBB2242E69 +E4DE7C8A03181BBA8A0E73371CFC4D0FE2927F4254BCABE0B59E36E8E284E822 +D5090B7BF1C1C6008FBC8A0D6F8CF0B8BC39956B73DD38E2F315C4831DF89CBF +9FB1BF817A22BDB5286981B17F89D6C30C4A1F33F52EDB6CEC33A62F7567AFD8 +C3D7244931F2A26F6D2B55116AF2C348879E1071E7AEDD1AC891F501ABA5C2E6 +9CF740FB918920CADAC98C2E08AF942DA04B00F8231DA92C0790FADDD7DE5B6F +14078F485AD485F2287FA5D26571669149C0CE185C4BD0462A2B2E1E6E2C8105 +118B5360B24AB0ADAABB3B65378E86ED85823EF49FB3B748818BE4CB6EB8137A +7ABF1874BF8864CC28D7AA0A1D369BB98F0E73494A90B55983B1284DAD99003E +C2F6A69CFBB57F6850EAA9E5E2AC6D07F48118041E8222D73886A117621F788D +5EF76586759DCDDF002A3AE4B4B92D724257F55FF420132F38CC47597EB93E29 +8B4D73736D3452B24742FF3331E0FF3D232C5796AE8FBA58D80119A0B21C75BC +69ABF5CA4CE35D2A471109DF7610BFF46F50A54EF5F435E959824D9F16C95225 +3F35F1ADF728DE5A66DFFEB89122F53954864855E1FBD7AE8F90485A720F8CE9 +89B27F90 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: CMTT9 +%!PS-AdobeFont-1.0: CMTT9 003.002 +%%Title: CMTT9 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMTT9. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMTT9 known{/CMTT9 findfont dup/UniqueID known{dup +/UniqueID get 5000831 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMTT9 def +/FontBBox {-6 -233 542 698 }readonly def +/UniqueID 5000831 def +/PaintType 0 def +/FontInfo 9 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMTT9.) readonly def +/FullName (CMTT9) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle 0 def +/isFixedPitch true def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 97 /a put +dup 99 /c put +dup 100 /d put +dup 101 /e put +dup 103 /g put +dup 108 /l put +dup 110 /n put +dup 114 /r put +dup 115 /s put +dup 116 /t put +dup 117 /u put +dup 119 /w put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CE3DD325E55798292D7BD972BD75FA +0E079529AF9C82DF72F64195C9C210DCE34528F540DA1FFD7BEBB9B40787BA93 +51BBFB7CFC5F9152D1E5BB0AD8D016C6CFA4EB41B3C51D091C2D5440E67CFD71 +7C56816B03B901BF4A25A07175380E50A213F877C44778B3C5AADBCC86D6E551 +E6AF364B0BFCAAD22D8D558C5C81A7D425A1629DD5182206742D1D082A12F078 +0FD4F5F6D3129FCFFF1F4A912B0A7DEC8D33A57B5AE0328EF9D57ADDAC543273 +C01924195A181D03F5054A93B71E5065F8D92FE23794DDF2E6BABDA4215500A0 +42D1A3D0D02C0C98BB1D6ED0B7791274C38B038FC7921FF1FB8FAE7258C09259 +4B8E1BD9EDCEDE9ADAD9BD9598EEA9691589649A9A21539161E374075BEE3457 +689F308A4A7AC9F2FE4B301A6C36B0442FB92E3B002623493DC087800B5A0521 +0DB96A23175AC584DE166F59142779F26FEE9783E28DE49FC3A8D6583EE63FBA +610DA773CA18ACE6F64A4867A1A7817120ABF9DE4D17782866E6CB6B65A9F6D8 +3667C8D3E61E5356E35343FDD4C6436DF73934470916CB5F0ECEA6BFF092E735 +C7C355B56189D1DD5715EC97E50145FFC17BB1497315A9585D713A7A6DFC7933 +995468EFD0F59E3C15865B87925A3F2930E20D5A35970E2C44F1629FA16E00EE +EE21EFC50D49F5BC02300D0A7BB85E649CB4E2E828C8B1C5469463013E71D723 +2CB11BCBAC191AC751A2AF7FC228395CE9472DC1809052012AEC2CD66695DAF0 +4CA04234F0187F4116C93F59A7F1F8123DE87F111853B785A20CA8B49B3B0CEC +B11AD345E1A11578D2EFEB0536D125237086CC8CD9F34A5137AC5DDFD8746014 +D74AAE8239B81ACF65F379CF2153B06A238A2D767F294CAE0D79228F0B7D45CE +510AC9657A1776202FEF42F96D476E7DF407786AEA12DEA0013D3B4C5D0640F5 +BC5BB72C34066270399CE595827175B23B25072723BD24E07F6BCD9EF0175DEF +93714BAA53960F81103CFB731CED4A267B53727BCA3C97B0BA5004055D4EF0EC +F725658E53AC86E4061B489AD4154915C3981B3B703E1E2A8D390CCECCA99385 +45EBE35441B062D7D12DAB2B31569387187D74A4043FD71F1C6D352EAE0F6757 +4345FBFB6DB15CAE47CAC4BAE47AECAE5FF5EC19057DCEFA1B23F47364ABDF47 +088A7C6A2AE26B10459B6D41CB69182FD1472F326CE3A15B59255D1DE3B616D8 +9D1F12561038839781E657C896B8C58A32DF5AEA23732A0966D96C68C988ED7A +09B7E2C8F9F3D0D56879764781566299A4EDD3588BDF70E3D924D25074F30988 +E35BDD827AE4D0B4A06F55A9976BF0DB3C0B1D09CD08E8CB168B50617691638C +0EC1A791C228177D4FFB021EC3DF5082CA3487AD2EFC8DE9466A690ADDB4C52A +FE2A6DB4CC275CD33D9136E735279FBB2008D59E667905EBB04326EC33C98B2C +94744B7F540D86E90DED64572ECF1EAD3A58EC101642B245A9C7232DC8FB8741 +03F97883BB32FB955C22F878FA0FD114451A3B3859B0B5537AFAB73AEC7DB2BF +409E1FB41D473714F6BEA73CB085139879FA31710E01915C2938C37BAD6D7D71 +45B897E00857D3931A489EAC7B42BCE4E65F73F67FE027CE482DC47598ABCB95 +39E98DA8ECA3E23F0799D5963ABA6E2984DEACBE7B46B40ADC6213E0F4D08971 +58F68C946C748E4B4217CBA2391BE2086C9758F4E32C9B6413E48D84D33A6E85 +84747029C0A9C9B92841D217A902BA8EB333999D62FDA9F82BFC8ED11F67988A +0CAE42182E414A9766AFFF4B046A09D476F8E3F15A8C7829BEE982D8350BDF5F +F215F2BBBF68D4B567BAB798B9604C79306C475926E9FEC0F07A99F43473C6FD +B15AC29C3D07FEBAD1BAFF75AAF2FBE94F104F1DBF838044FAD94B661B06AECD +D9AEBD02B60CA4546DD6B5B5C1A3833ED07845671CEFCA8955CE0DE5DB8FC93B +3306683CBFB8E5B79A863DE78D455DE9D592043C2686F88A43140F8B9F3B553B +7047420E93E753829F8D47AC7621CFE3626F271E31F0019CC02D0B57F67BB47D +8CFB63E902EA3231C00EC66EEC0D30FE8394558BD3535C888C4CEFC6EB72E737 +712ADC6300162D5D79BEE0CA1F6E4127A0BC90656C01692F6D82C85550AFC97E +C2693E379160FDB9636FA41AE9C75B7F6643B05971C6D67CE30971D590FC07B3 +E0B36B4D1C7F25110B5DA2130D574FA292B47322975A2BADBDB39AAE69BDDBDA +A880F9AAB580117708C79204DFFDC08BF4A48919B5C22228845CE8C3109E93AC +2479E523B8A1C12A6E541118F121DC6B4EAED83491A03192D5C3A2A45D1A2467 +757E7B377C635CF5CAE11A7CB49D49F3A1BB2286090B5F0E4F89869D1771D50C +54B5C5E091E3048A2C194F0ED00DD64FB95BAC6FA9D61ECD093ED416DA3A4981 +DB07CFF17C4F55C62DF628EBFF06FAC3F3D3F91C30EBB34052BE1A08F5EDA4B9 +08977197950A282B84E21D43C64BE3AE4BCE22C70E7D392DE09D89B7F23351AD +6AD37225C12BA79EC9951F5DA1E505DB26200190ADE0E549305B7530CB86EFD2 +A896F13A97E51754F70B609CB4511CEFC38BA579C071E9510A49982389980DC5 +336D6C4A2DB100DFEC4055C7AA9C55880F94FBEA9EB280BEF66CB8E1E38A359D +E5AFB12B540CD599085ADDA7FC2C72E7C873015773FFEECA2C596B75BC39A3EB +3C43FA2E53C0D7993042F3D652BCC483E48B7F6C94C3FF6D38E276086A6AE67A +E5A571B9C72E0D7824E0BC2ADF51A393B9E334649F786EC1923C854382B89627 +1B9E701AE5A6C42E672B2C6A33C8BBCA8F69B9061E787D6B92183F20CF4C3903 +FF5417427B84798C82BE28D2C81624E3920CA61EC9EADB364B5A6E50E49A1A72 +A9A090A1FCD84814B8B2708AD787D2B5015DA1305874F58C5EB62F843685FCB6 +465FCA80176CAB2B2FE65E0A270BCE1E3DB97564BEDFAE5CA44395A8DF4505C0 +3E103CC3B914359B2870DA6CD30382EAE8949131CFE31E9E75C3E47A3834BB32 +CF183D4A8B9001710D0A11390C9DAD116196568591D38C2AF4ADD852F31494EF +573462759A35415900360882739789D6B89ACEFA251C5ED90ED704DD7C3C80CA +9F6CDED69537D201D520C99E69EEAD5D3C0EB84C166660B3C190166D93EDFE6D +15BCB6DC5CDCA825E48D33845CC2FB15291AAB823F25CF8BB0A1EAED8BEC524D +D9CA016027141FAC9D35B64FB9C224552F29EF6B32497254E319090E698FD8A5 +15491CDFE1B988C79A0E3B9D01E12FF084E9FA86CCAE02A3EE6F2917B61A2CC1 +64B8CAF309D1AB48A34227A7729DFF99CB6EC282E3FAEDD2673779AA7E4C1789 +D93FDC37FE95F087C5F88F53D30A2DA9C913BF205FC6BDD060A40184F4AAEB3C +D080D63B89CA3DEFF310D09EF0A83F3914BD5B7932980ECE139EF0313C20B4C8 +576EE0FE3F28FAF4D3CE7CD0890BC824A85B8EF4636BDF1EF1BB519F93D36540 +ED09FAF93FD71992CA2CE2E83F5355162ECEB32AD218092F45D5A61A44E67135 +EF0453589CECDC6962D0E8DA7E7567603BAF50B2C8F1CA65EA5320984E7D69AC +9A7D3D7F92565D79E8C9DD2D92CCA7DE9CD058545E9F98AA47904D70E1897099 +3C4C852B3BA131DDD348433C336BDF5FBDFB62120DDEAEB3255E3207B0C84A0A +1ECF9EC869DB9BFA3693B03FCB27C5A5D3CDD62630DEDE91B4DD5B9784BF0BDD +FC6EEC3FA7ACA9E15FAE47CDD9B7FCD2BF0EFA10716F08C0AF25FF67CB6F9598 +C607D2FCA452417D2C69DC808A9441A66492394C3450BD30632AE739EAD654BA +4343459CA36B6D5B2C12C39495952F2EF93D82C73E33236785A79609E260C4E0 +CF3A3C950DE71DDC3939D42DB1CB1CA917CEAD56979A70F8F3B207C805319FA7 +3C000AE2B21D711A6D78C7BFB901334DC06F59EAB6D94B507734C27971F8458D +D00193645AB92FB8FE163D5C51AE4F40BDB4F2C51691E76EE0636F071F37AAA9 +BA78BD12459CA499210EB0CE2F8BD317387797C33F5933AE7A6264DA06B4A6A6 +1188326147A16B205D1F965872DED7D8EDB3294FAD2FCDF0D423329E9CCF879D +4E0B966D509F45527F7609DD09694D286F6FF7535EF8971B7DFBAF608A19D442 +C133207EB1152ABBD11C455D0977F66A9B73E51381D1CA4B66E87C0C7175A63D +80C699A052F00C41DAEF42E7A40E07B1B14107AB0787E24E17C1462960E3C54C +AE73BE4924464FB177EC62F116B2822842541543EFF7ABDDEE197D6BD8F8D4E6 +59175D8C5957550B70BE775AD52FFF6E7C00DA7CDC16E1DF7446BB5D8FD82647 +3E9F87D5EA365C82A2D991321ECB14A9E3AEADC5A56665DF7072D6DAE402BCB6 +14D92B17F9E063E4E9D8D239C91F5C7C0BCD2FBD936C9D4A0B57659420343B59 +B395BBD1AB5B6003F653699D57E7581F9813CC98D4F072FB78899D6DECC42D34 +F2787EDEA64058B46C4BFAA2BB96E9BE5CACE8D91E4C080ADFC0FA0D4A29C6B8 +54FEA9E11DBCF53D9CA40A21AE5076451EDAB3593E56B6D453DC8EAB8C78B588 +34D4C4F36861B5649BC1E9F3091E704BDA7613ED45C911DFECA74EEA05165191 +825F95A947CAF382FBAF01F3B8B041ACCDF39718D7DC5BA6CA12BB20EEE96439 +BF2E2628AA3BD2C91998E6247A690FCB0CC95F286F427345CC4F1115BA3A6E54 +4743355F2CC991CBDFF5725902C1F5A6DEFDC8638A26EA456C33C27773D6214F +66536CD2E44FD253531732D5A8C44B336B1BB47B0477350EB8CF74889B93402E +2356A9CAAFCA562315D8E0B3F42F08932CB87BA2499A875AFA08D11DA73B38AF +F46D03B7F639A8D7BF88CF07FFF4E91716DCCE6E2CCAB60A64D5E40EFD8B336A +1BFCC4CB04F49DE1FBDE7AA5B2092A6EDBD913D161A3271AB6411622D0E14416 +37F81E0102F5B0F2F9A2B27819E4BACD7C50E29D6291AE5B0973C657761545A6 +741729620EF2BF1046B3913399C10982EE5F4142CF461EA31042E432CC79A1A1 +39C607D22E45A6DEC008CB4BF6007CDE9DD5802B49A62C8E02A6D448B64177CC +887AD71D171B99E7ABE2085B37D90B3BD8513995D9A57F53184DA474F6DB5E49 +B73E04CC214EA5398DF7D7541F94E623E8687B511640457A48A68E9D9D6584CD +15B57CC044D8091C771D175F2EEDD411099BC8F7B4317DC503BB5E405AEEB526 +5E6E1B1F2705275D274E012A98F66075CEB90AFC648B964DDC0E9C4AE7B24CE1 +80B051022E5781A533A21DCFB97893847D685137EAD85BA708A7E118C72FA839 +A9E460B5D17365A0AF1F53A98319FB64A5819B087F554BC056C4BE44113A5404 +BEF759F890C1CA5E7AE156F4F8106FDB4F8DFCCC640976983EADB30976344048 +2A86D7B2AF4A01CA736B98D52ACE392AD4BECE7E61C710B08B66F01857CA460B +B8376E257113E10F6DEDF14CE2A4E6A99ECBCD302C36CADB713D849EAE9EB598 +F29DC98531D793B79F83091F9B136809E006F34E423D528CC4309AFFB3EEB47B +9A9DE4D5B25CE953345C326BCBE2B4912641780637783084D3D12693F8135483 +CBB0AC4EE0B5610D7CEB7DF205830BDB9BB404DC1B28FB0824CC187B26C19A91 +DA0025EC739BF3993700101D042DED86D67F5FB87912CFC51AA7DF53F2162D62 +6314A2CE13810D0B8D81F45771391A236422CFA0F35F7A0CDF14ACB2724AA57B +7C2C28D53029B1146558610E0CFBBF72A85AB9BA308F846228F299F13F68E8F7 +D963B2EE9EF7D4C21690632B640BDDAD0556EFA4EFBF035F13377ABB5CBC280B +9E0C12AACB153C93351E5BA95A7D149010E204950A59C7FC6581D9703468C1E9 +EFAE37E7E6ACB892B3F8D1248D9A4A72F642FECC5E0B25C15EEB921EDDE84D12 +0E524FE6133C4921FF4921242392C12FBE69744D53739F7E849C1B96C4020AB2 +1FF10DEA608F111749E2FBD8DBCB17F353DCB3075B4F4B8186963EFE95A76A10 +85AA5BB6DB4095291974221829A8E436680F4860E01C3843BE5BB3101D0869C0 +EFCE08D187BC04F58C7A450A59093680A0F09E8E3F12DF5223E7EAFEFA01978F +D8354753A68022CC92C71F2CA732DADAA8A466D4AAE5999B0DC077715671F518 +E6277741F44AE798EE50DF44CCF71FCF8BC71F76374005FEBC4883C6EDA854B0 +88C0C2B476709AA809ECE41AE786DB1A32B3FBBCC14921673578D3514C8CA842 +E1FF90BE33F7B93ADF6BFB8B1AFBBD080783BEF056A6BFAEF676F7BF9F2DFCC8 +01D255A9F0391951210D60D4D4DCA93AA858B38C0D7B8FD740D5FC6F277C2A68 +54CC2DE1F40B6347201FCA2A0A91822708D820CE645C3E4E5A09FE25721AB33A +97871ED448F38FC5A349D81F402B34461D840D5768BFC6849439AB6115104F78 +B87115B1DAE12542EA898F86ACE247709817850B067F537E6137196101D46DD2 +D842EA03EF4501E34074E8458E638ACC4EB349A7430AB035BEF2DD4CE00554F9 +18F9FE32A55AC1E7E50D64AAFDA278D77A7149C59DC5B1E3064A4B281A54C9CE +A5EA94ABEAE4C6D5674C208ABC72563976487136AF2E21F835BEFD232D7F0D13 +1D19932367F51D5379934DA7F1635AC51EE5CEBFA63D4D32F018DEF13624EE62 +31DAE68A08DBE3B4FDAAFC75291C8C6CC7A657E3C7453C7D1461A36E88E633D5 +408253B673AD87A9FB2D0F56DF1305916D14D5DD62051E27BCE09CEE9A1F14AF +1D7164BA5FB6E6EC8D38750F7E28BE330909F303ECDEE692E347DE13C8C2F82E +29C8BE6EFD76546F362A12A1C2DC12389EA95ACB4DCBE95620F0C193EAD91B33 +BAAC5801AE827B9AB3FCE5D11D1D7854F8FA8A31670119CC0CA98628F801838B +AAC7EF90AC5466BE69CE3E3CD9951A5EB9AC08014285422F6DA6F6E221BB30F8 +0042A11F2E4B765BB0D142AD52F4D85785EA71B2E1CE20728B9E9306CE93268D +99B822A5AB5232EC7E26EE1160850AD3905864A01357F22722B6A54D4EBE58CE +480EAD9FBF068EE965AC4B5FD2FA8CCB91ECFC6E90B9C49268CA0B0FDAD23ADC +D5A74B41149BB08454054C451AD0DA4CCF8B60F2EBD061AA03A011D548B6B481 +FAB00AF9225BB5463F27FD67333FB51F8664536267E95CFAA0BE3BC1B8F889CB +587A3A4FA2B45864F07E11372C9507A625C0030EF7030A0B4D931BCC48F6DD51 +A4D1F63FDC4B59C1CB18E6242E9F4B4B8AD9755B870FE60D640181FB7EB8120C +C56F51DC8C47FCC6318C2145EDCBEFA7BC4253315BA67FD2B3D4AF6A9F3F229C +AB75B592EADE15B1FB5FDBA1C0F786BD21A51506B7A2E42C2D086BA6F84D1B3D +AC7531545F0B01346831FF36A52CAC1E390F99AEDC265B44B0FC9C581BBA6BE4 +48B723811EBCAEA5FEFAEA7E5B987F2C7B3E9A65D2D14A7B74F099401C57E367 +385352D0776D2A908F7A5A2E4D4160946C5591397877025C8C387CA413EFED56 +8B142E8341E349DB4DBA422A4FEE56A573972A0C66590175158E48850A9F7F38 +4B95726787B8F969FDBC97491CC81CABC976CD00A27D1DFCA7CF467A956C1C6C +839817AEF8794B6151FAE9261119DD5DB787DC9D3B420FD325ED6599FACADE0C +320D54C2E0D296537E22C1783670A9D9BECAEC63853EC2F05A990260DC189D63 +7CCC0BDDF2CF7585071ABAC14630666737041194D0777EA4292AE60BD7F7100E +DB568C90F0D899EA006CA423CFFD6EC70A5D3D8AC43C747DBAD3B02219E47D8D +DE030631F4678C357A58ECC52782B31B50CFD44EC33F41585E51B27E3997D33F +461BEF897220AEC80007F13C5A1EE3A0430CA899047DF944831F8B010A7DE74A +BFD26001472DC00CDC9F17CC435F61ADAD4E9AE062ED477FC621FDDF9242C449 +1BB3F77FDD1519A251B663A693D84B42BF0962F537757F38CE5C5D56B98AB10A +3B70C8AE8D52DCAFCEC22E7B09D3C4EFDA1841C74CA975E4F8294F7BDC796500 +0ABE197ED3737A65F7BAE601C91DB3983EAE147AE54A37CE291DB94742CFA9B2 +9789CF391A95DC4C962D0B5CE2B9132D0E36B86C9800B83A849EB440CC766B5B +CBBC9A49408580D725D5F913A4CF6E0C6109CE4ACCC19596D4CA99AA88692E6E +AD5C5D73C945970C5AB0DFC8CAA5E13E9D2FA4B7F5017C5C7A63E930C351BEC9 +88D2A964ACE7C26AECF93594A3D5119118EF7A7419053488DD1CB91B7871D998 +20DA57D42661A44A1088DD0DF19468B49FC1F87AE8E2EFEC60C91C701B50F9A5 +872CF0AE5392FC00C77340E6EDD70B29DC2449F391F258E341D77276F70FC252 +89846C3DD8492E4B3DD17B6FF26DBCBC48F78B8955BA2D5F87AE4A801B70D53A +719699842A26C694F82F1937A7BC1A912695912F4ED5212D87F97261F3502B58 +A214E37B3680B7E12601B2861AED05C9650C94C1E8A49498E1CD08CAAE2A7446 +4050F73003DF668CEE2293954358BA1C974E8970BAD3F16A89D5E38F0965AF55 +ACA3013D763DA88DC72FE6A704C8AA55B282716F23FB80E028B41CA705C29468 +B9BD5997380526A2082EBBE00E51DD8DD5B31C74D5124C32B9FFD4662CC562F9 +21F846F6034C67360BB46CEF9E14021E439F183500DB13A904C27FB6EA94CF58 +4872F7EEEAD3FC535B7380E4DB3415A4821EC58B308FFDC9F4A6D9EE558F7DD6 +02D14F972D7E215ACF63BF91BB7C04DD1004C61AFB0DD1578E390DC5681E2552 +1AAE563FB8EC492E9FBCFFF2D6B5B8B8B22B4E05EF60EB24E367288AB80CE6D7 +76A752CC52AAD69574461ADD2C4871D2BFD011686B7F24CF8C7E6757041038FF +ECBEE7F3AEDEF6419DCD219B251D3F5BD9F5075AC01752C1FE3E6092B4B89E70 +149E6F48C2241F14463B1E995482B72F81DB73914AD4F530BD0A1F01975BEBF6 +893FD1CA2CC7673CAED0352DECD34C37E58EC596C5C803A25B4096066DB49564 +F99F2ABE088155D87694609DA3259B01CBEF21290E11807F74BC5CE518A042F2 +81A3131E5893236F40D0FD696F7483AD210C35B1EC52A000C3918156E8A74A23 +211A3D09B39FABF3552EEE7B621ADFDE29C6B1131C3AD8C62AB4094EC5479B97 +FA912C3208C49D7C4A1A7E58E3DFA685389B8BDAB9FAFB9A44FB3E7FE0969BC8 +AF6CCD6790DD48B2F6E2C2D25D7DC7EA394AD779F603927A2B336ED1FFC14786 +6454F62E47648B86EFEBA563B9264907B496DF21E0DCF9BB0E22C346F4EEE677 +D8FD0DC6734397A4AC4EA7674F4E744CDB34CE23212DEDB0CFBB893947421169 +4A151998B81BF564CD912B422EEF78DE4DA322C003686F94F553A5400C769ED7 +241B4E16878716B42F395BD7F2108E754E07763EED98C9E2C82976CAC5887E5A +562DE571134D6CC9D29532D568158FAC8666D9BA4C9E1620F96E57B3FF454B2C +31CB7B6CB9F2D469BF47CFE6F33AFCEDD072156DF5410E0AA1B8BEE1C8AC7EF0 +1413F29C4C192BF74360BE6FF67DF527C90428DA9B81837744DAEF1F040AD4A1 +4B7002FFD9A512021D3643E462510D84D48D18CAEE3512E113605BFA7214E9E6 +B13C478E5C63CDDD0BD7C7435592DF907252C89B8B525F775D07CB7BB82571B9 +AE5851105BB5A0C214021CC2B59F5D4AE89C0B35BF736185A9FC506B09D938F8 +457A9A3D9918E51D6B62124A616DBA9AB989DAB794D2B39EADB6B90E8E00B4F9 +39267A2C777740A10AD7C9DE3F9AD4D8D05159829B3E155301A663A962DF61D3 +27A07A8382492AA385CD780631E360AA6DBA1EF96797EA7870D4F2CF98710C65 +E6CE09586CAFB53D8A6B71F1CBB34BA1FCE35AC823E16EFB42A0A32B1AA782D7 +F416B7CC91C64EE6B80BC0B51E4A10547B5677D293CDEA40F11221D46909B3F2 +5BB38EC3C6476CD66F9E6DAFEEACCA0AA5AFA4F83984664A66E4B54F9C265133 +A8F4B8BD5E1A248D02EA353EA83439A36539001691E9B814A300BAD3D6B8D22B +EDA0B3A7DAF716777CE2F12D328F680D6594120720BBEDAB667ABB368B0894A3 +420AEA0815BC9F6E915C0DBD888FF306C0564123AECFF7D000C13DE8418101C1 +1FD641C6A7AFBCE6B1594B737CF9B4795F79756FD7918475155DCF835C6C5DDC +E76DCF92F1DEFCFA58069AC4ABD269CBF7197FB8C667150257A59784F8B50782 +07E51B01FEC61DDA085DF5088CEFE4875AA30EC8D40E7526D9565C423CA6A430 +B694598648D332AD1F016FEF9346A89FE74E811ED16D156A67BB2CFD80D6B8CC +6C0C401E61BBC69ACD7B0F0B0EBB2837BF4A452688F64FEFD1EE0BEF213D1044 +CA5B70FE3B2F971876C60CC9542BDCFC39E9569DFF0A79F7F0DDD0A90AC1AFCE +654C09D361E25EE1B649CA8124F6EBCA3651E8D5806B7F5F1098BC20BE8D4A35 +4082281F57AD4FB0528892536BCD1C1388B4A01417B10A3C7850B3E6D88D022B +05253990E2B7982A7CE34A1E63A619A0D40E056C90F12625B0DE1AEEFC9D4E45 +DFDE2636A20B4E85FE6C617B2C5EC972230B9AC6F2C6B9957EBA6F5E56CC7C89 +EB0DF5653B3C3305690A43355E8CDDCA858239E04DF37E81506983D7598D70EF +27A7F3F8F8AF30ECEF0D86125DE05A550779CEE124990BE109393A9B16664D96 +D02F5806A0642A8F050EE116D3304715A53D9E551CFD621F7B664C9AC6DB20AA +9F63A024297F58B84430A7B5C3A43485AA740423DBABBDBEFDE227FA56B12541 +BF13D836BED9DF893AD490225C006330ED2EA20BAE55B26DEA7C9AD23C8C60FF +EAED07028B037DC18786AE59D5BC2B8AAFF4FC2B2BD7F7CA13E741954F789824 +189D597772E38EEEA7765D3564DBC60D62C8FAC3A319762B3D1C29BD496FB8CF +63DCAF776E4101B32A054CD93130CC00A462139F6CC0DDABBDF0D18176E19761 +289F43B4BA941DBE4F86B6A6F24557E78E83A652EA6D18AB49DB59E407185E1F +554C62F0D7E49E8EFAA8 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: CMSY9 +%!PS-AdobeFont-1.0: CMSY9 003.002 +%%Title: CMSY9 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMSY9. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMSY9 known{/CMSY9 findfont dup/UniqueID known{dup +/UniqueID get 5096650 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMSY9 def +/FontBBox {-29 -958 1146 777 }readonly def +/UniqueID 5096650 def +/PaintType 0 def +/FontInfo 9 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMSY9.) readonly def +/FullName (CMSY9) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle -14.04 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 2 /multiply put +dup 33 /arrowright put +dup 41 /arrowdblright put +dup 50 /element put +dup 54 /negationslash put +dup 55 /mapsto put +dup 63 /perpendicular put +dup 91 /union put +dup 95 /logicalor put +dup 102 /braceleft put +dup 103 /braceright put +dup 104 /angbracketleft put +dup 105 /angbracketright put +dup 110 /backslash put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CD06DFE1BE899059C588357426D7A0 +7B684C079A47D271426064AD18CB9750D8A986D1D67C1B2AEEF8CE785CC19C81 +DE96489F740045C5E342F02DA1C9F9F3C167651E646F1A67CF379789E311EF91 +511D0F605B045B279357D6FC8537C233E7AEE6A4FDBE73E75A39EB206D20A6F6 +1021961B748D419EBEEB028B592124E174CA595C108E12725B9875544955CFFD +028B698EF742BC8C19F979E35B8E99CADDDDC89CC6C59733F2A24BC3AF36AD86 +1319147A4A219ECB92D0D9F6228B51A97C29547000FCC8A580F0834F28818EBF +F29D3B2C7C9AFD969BA6D3B7429E222AF78F6DE00CE2C8008A5966422936DC5A +DE549B458EB41026E123FE75E703E046F665B7DC78298A145548DEF8D579E008 +39279A5208EAF898A357DF3FD9CE4450811F20B454D86B2947BEBB11EABFC62B +187B12458E022CDBAA6EDF4A89D79300F635D8CD973E35FA8D9B1240B3D30D72 +4F30444BDF0B51ABA15A181A3196A6CDF9CD1D38D64C83E745A767446641991E +23EAE067609DC2E84B44D923CC98407812813D5AEF0EA21E560B31BD77D735BD +E35EAA564A570DD3294C703E16BB9F251D54407269C8B23DEAD063018D6EB748 +4204A415AAC8384025FA922E7074FA7B8A6C31FD3761E7B2737D4D9C9992B7B6 +DBD0CFDBD87D42E1A451FC5C7EFAAEC65D10D278470DDF42DFABC836CF15D042 +6900E0DCDD36E7153BA186B8358E74F91A1D43D62EB0B821C0E97FE63F1586D0 +98F01E8F6B6F2A435343A25CDD80D3AEB42DC8E5AE02F8837B84EE5EB508B02B +6980D2349984FE1A0D3ED12355D556BA483CA078B7921FB336EDAFC22AE05C1C +5614F5025B890009272DA8C838B81913C872A54F7E932237FB1BC2CE5D64CD04 +2D320C8E2E610E9EE36EAC7C28C833F8A14048E751B61D0CEB342CBAAB34D104 +311B421B8CE4D903D2E2518BA8B734E2637E2CA55426C4A87EEBCE637F8D71F1 +54626C52083262A5A9A3B49142948BC1AC0B7698811AE81FC3D3B1C14E6C274E +CFD6C480514F3649D7725A0A14C82C9D537B1C5A9ADC816340B8B4E484AF55AA +905EA936A83FE2BE788D084469008F407447E0077417096333F163CE94CD3098 +4B9712EE23120829AE2695EA63975492DB2615214B11C87ED0ADB776ED4BA0B1 +40D09D1A617D4F57EE1A9B05C8D5F42AECB9C6AE67757DE6307B1A9C467123DA +773239976AB0EA052B23F263031ACABEC9EEC7A43ACD0DC2D7AD66664FB0D937 +6A0F463A173E8B90B2596DD323D2350435D9D31B638112D70492F44769FC0C6B +40A291FE78AA598CAFC19E5AD195BE7264F0EC29305D36C7AB8F2A733797D75D +94C8F7AAFC49C13C108F514745A394F460294C6F3F44162BBFA878F773A5115F +062E934253B786DA2B9A6E8E843E06B97075BE99F21B330D6684998C5C340B72 +09D69B655097354CB682D42120ABF5109E5522ABEC94D05E7211DACE8737A1A9 +0BD3EC16F48798E67C7C85560D79366504BBFE5F7F5EC1B05B9F68F98149A02E +E53F321C213456D802C01871E9024F965258F71215F40EA71057C15E349F06A3 +A016BF1B7E44DC4766FC92EE500EBF7C47D36B1EA34981EE6FE38FF4903466A8 +C9020B91CFC731642003AAEB145EC0F13D6281B0838BB4236C220F6F467F7DCB +C79BB9CD5B6A180DD2ED7F94DA6B5CB833E62E898BE832DF601BFBC8148F9430 +C8AC459F23EA2536E301C9DFD3FFBAF4D01E7853B60A69B5D012DCC0054C0BA2 +1293D3B4EEDF7C5C78118138544A60FB1A500F72BA39E63EF5D97D68369850E6 +9B6336DA577128DE68FEB71877360D97204B5F3C06E808BF95AA6D6013524BCC +85CBA9FD2CFEF383645E7E960E7AD471A6A3B8B8C3A0A0B8464836499309D82F +52004FCDB68997BAD9E4B5B881634FE1213283B6B19308BB67F2364F7ECD5534 +467EF07CBE71989FFCBC6224417C0710EB3EF2676B9DE846E7FFB1AA6608EC9A +76C31724A19ABB2AD08CC102F5DE8E4E513F90A53F06D440A66B000CCAF6AD12 +6F4B161B56FA5FAED65A24FA183B84F4B45407AA69624068EE0E3E47BBE973CA +C72137823F1D676131C8CD5070092CD73FC2A93E0480310E21CFB14C031206AB +411694AFE7B639ACF35C28705A115E98BC69C66F6C58E9E79902453798D76267 +29B780AFB6B895B34CBEA51530E8300569EE4D500527BA294282FED81F730A46 +6ED19AA5AD45852739882216EE937FC45D75116BBDBA6FF4B4423B398E383CCD +A7D705B8BA85076133E823300C3EFC9EB9399575F5F6BBF868265304B299AD77 +B6B4EEA04C83E163E8CE2BFACA65F9DAAE3AD2B0F8DA1F986E6A12DDB2AFACD2 +217E37B093F46FF3E20E587A9490401C2833F5844763FE2C528E151214E8CA95 +018E1D8D47C9EB5FB4F0EA2A6D064680DEFD2C5489298F83B58EA7FA8CB019E0 +671028C59C3E007DD86CD247889ECFA6AD6DA2433B02180DCE6ECF9E495D0976 +D063A61B007233DED5E3F351D790F26D4011582735A69F37E7CCBB0AF254B925 +B378522B748BD9BF152E2187DEA12BB168C42AD8D67D18567393B35DB2AD8403 +5A7093EB59FC276525B8796FE81944F7F03349BB077110E182A61754ADECECD8 +A5255DBADB523F3DC251A7D97F17DC76281CC632FAEB382A972D60A15FDDA57E +2F42AA338D889B47F6A3C1FADAFA8D11D63EF8D0C8C0FCE61FBD8AD34EB8DB3B +722BD86CE5B1268F704ACE82A39466DC2D96C9849B9422567FA5E728B17FAF61 +59A77E9B3289DB807DF54D076139E3DFE176C3131FA4540C1542AE5AD6200FCD +B92D5DAA77DD4D3022049B6EA2A037FE96E0CEBB1608C663F1373246FA74D70A +BAE7BD37F3AC23E003204FA8270B4514DE1FDC5401B69CBC98A67A5CAB5505F2 +87C242B6ECC86E88EEF44B9C79DA5F88CFADAE8B4CF437A7CBD5E0C01A9BC479 +E92EA48C808820C567E74D62ED7597FCEC53DE9A0B3EBB9A3FB50AEACAB82BF9 +2AA56D029357AAE016582592972ED01C0ADE565E87607F17BCF4983E3321F606 +ED664B9516D404B277C2C1B0F873D469D64A9744D9C1897125FD471E71BE060B +A3A617DC8D812E184664BD83EC74EEB5197611380F2D859E1A61A14404493D04 +B5537C04541E557CB5039905F37E72B762EBD2321633E12C38C409338AD5AAD3 +1A507BB2B8F5235F0614D0CEC2BE44CAD504C9B2CD3E45437A493A33B9BF8A1C +3FE66B1E673449A54E8E3C3470688E94EE4404C888583F7C96B43A592D586D25 +8AC1F206EE5FBF3AFF7EDF9BA91EEC6BABD573D455E0AD6802E7E23A662D4F12 +FDDA00EA054CD331DF86C099C4BE8AB4EAF5D8EC1BFEFCF50302BE61E6FF26D8 +1E88C544570E8D6F7D875466C70AA91D6CCD4DF4BBA34FC67191D1692D3FB76A +66F114152956A5E38345E7495BEB1A389B7D0A59624ABA784F77C1BF7728E556 +F9D97D71858D61624C22C6914A3A995BD75922A2C37EF2CA91F923A4561CD3D5 +2DCEACD9C3A65A9FAFFFA4822E79D6D89D6366DA6362DB16FF061840582DA35D +E4EE8308040C2DA5F298944EED470D6E4E1FF58DFDDA96C20C85E2DF2652E12B +2EFB8F890D301E8CE777678FA786E552A62A60DFC369AF4BDC069F5457FC86F5 +1FB1A38A2222EEE86FE1AC9A8227663AE33217C1F381115CCD04D2F381A27E97 +36D356DC64615AD63E1952B37AD4F967317D76A6DEF039CE8446AA634086BFB7 +7B0B34F595B61ACEB346F51ED10ADDB7B852B5B2BCFE416D0435A7CAA4C9D9A6 +152D9AD938F39E9756AC752F1604D884992BAD768C720C45479F5AD8E53A4832 +1B44D6109DE4460A662DA6081465E2BE02ACBEB52259E046767DA24E1416FF44 +D5AA1D303CC4249510D703CC6CD0FD7DDB078A91089FD86ACE1721F875F019B9 +82D7C5B2530567F2306F47DE90EADDCD12AA1C693B1982827DC36A74C86474B7 +0A8A2E084CA403AF654E2F8488806B943E38ABF544ECD45FECDC441DD970B180 +FF3FFF1A50D5E22216B240A527F95DA1E43770A3867F225DD2A8AF01A965E06E +A063997C4CDC0B16B52F9B9C8B0DBA94F88F26A7A01E596DCF8F33DA3DA5B02D +2ACF771F25A1372F521A80B93DB0337886FCAC0A2E8FAE0D7E1E0C31EB43AE3D +B4C2020F322A9857A9EADFA7782CA70DE7421B9A83DE1873EED8D096701370B7 +7EC33B9C104E9CC16DF8C80ED24D1F91677E355DF037621ABBB75E09ACDBEB82 +30F1AFB82FAEC4D9E2DA93303F72C1078679EDDA35ABF2AB38DFF091AEB273CC +5A7B29C0164977160CA75F62BC20474C7C1F127AE7E4DE865FBCEF852698AB80 +B61340CAB6F86017FEF570104E46B6FB3320E78D387CC58DF1C4FAD8570C0126 +B6F54AB5BCD47904A9255366AC9C88576456D3CF605822269CC65979EC3CC740 +AE3D09257BA3AC7A364A7F5EECA3E658AA20B53423553C416D98FD8100D80A30 +A1B19F029B7473070D7C60CDE51BBEEFD2B45B1C82F4FE32A5BB6BB5BC32BBB8 +F600ABE893B83B806A5435AE2B40E5CEA9068C6ABBCE73C9B3533520439F28F5 +E171DB1DD90B2EC04A1F36767B2B97D4EC0FBF217D4E80ADA5383BADCD7E066F +5F4B23738A14F19AC7D765119AAC9AB6CD0A8AD4EAF8955CB9F5BDB0156CACFF +236D24E8A12AEBA549FAA9D7053D10AB7094B5D9B86FC7375E245EA8D7FC64B9 +859A1A37C006E0449AA8DB8B89730348C71C7E014A2EF8D871AAA856407C33C9 +392720173DA60850938D0537564455C3957631AC8F8C0B3B29BF6163828796E6 +0944C3000B990715188203B32BC4E3FCDF173A4A9C78753AB7D76BB9A4C2DA4E +3EC638753D6FF6356EBB07D0A9E8D3F50EA5451FB6C836B9E5EBB8F1331C8CA0 +B0CD271AF316502CBE87F652B5ACCD43176AAA32C6530C69E77E3C85DA496F3C +06980ACA739DECB5D986622AD81F204579218660EA791774C2D11878DE23F858 +BBC4ABC7EBA01B7FEBB32BC914FEA8E5720731FD43F8A5CC53954D13455827EE +9C7BADAE8322E77EE7413D6C86E6D378FF8044F50F529E765A1CAC29E8B1D91E +CD40F70F2D4890E0FD04502933212D5AB176DB143831958605C1667B8841E770 +CB4D513DF0C3182C23CA653A7A714B2A8DE29D6B203F5A9481E0CF3FF073FC6F +476B0B205136DA43650A940035A970584DD7D06B7CDA8E960186158B75FE3917 +9B1D0FFE45EE1606C875DF635B80F5A9E5B67A6ED748F52A53A710A45AC20BCF +FC96755E685A3B0712D5E09D860986C61C631A8D2A505C57D57C1D4AE78C552C +DC5C9EE15B82A4C3020331B78FDEFDE53974C391F375E8118EB7B2D23DFA3F15 +389043EB6F33842C82B896073760197B13D3DAC172D77779DF517AE31BAC43BF +5BD6F75B59AAA59C7EB81B776FADB720282DC06BBC9C33AF5AC0BC59F6A0044E +3898A4B5868EF32BC4F980DD863E8EDF6DE3E753A7B3E39F8E8BCB3D3BF14EDE +3395E73BD07440FCE88FB28A96878E823C1E5519650F2054EC2CF733EB8AEB0D +510E5659A3179C3FA8E31EED621BE0EA648D76C370C3CBD8043BE206F28699D7 +5871DA483AA5D3580CD55902C85DB9EF5CF5D3191738FE64DB42410F5AB8ACFC +CE07E524BECA5633B082F4793002A4A7215AD1BA31159A30D72AC60EDD0B2BAF +73C9934C85C5372B47BCD3ECEE463CADC05CE8352EC1399ABE2465AA53E5965E +79400336B1F5D1401EE2DA5807F9701D1086DE505F6AD4E64DD4F3E5E90CD64D +FDD25E6FA03B679597CFB8C5C1F03641CEB7B9B40787E70BC6B063621382E3D5 +1287365426E44A2EA208C26155CDF782F404CA7B3C6D379103EF4A51CA3F1343 +22B04A0B55CE5A7633175838B9E50480E9B80446CD3B541362E7A10BB3C58A42 +88C0FF580B4A0296073F5B6505E4FCCA5D4BA2F7E8D59BB04EC9ED4E664BC2BB +8754777594DB26DA0985663D321CCD13FB261BE8A771470DDDA783493E452A87 +D0AD01AFF8EDCB7BB159F1408FBCC52F126C46234CE6BEF241B49472102598D6 +91FF2DCBA2CD0E6658A61DB3591EA02E4DF202954B655C5A3C34B14CB60CF9C9 +2A6683AB282DFD79B33FF3A6AA524582F174F11A5568159083496B242ACD2770 +0ACE8473D363AC11250E5DCDCD3CB2281B71FCEB8AAC1367D0714796AF3BF01E +80BD4B05A0A45E6CFA6C672850B9DF113F5EB15923ED885A0E4488D4914B62A2 +FAFB6A63F8AF0CF0DA498E1BA68C4150B6E59CBA80BD17026FA09920006D2075 +BC6F1010D0A10648A20D8ABF8137BFCD6A5DB3606F6B3CA51E8A4645D32E5966 +6DFF7537F65DA4268A86BCA5791C21F5FA9F4C2F1F6C1AFC4D4BADFA63609C4F +5F5158608E6FFD9C5BF00646C3AAB983DDA5E37A9C2EA81796B161A06201FB6F +8EACE87008EE7A008BFBA69AAC04D73A8C0447FADA0249D8861ED405537E23E7 +BE68C4349FD64A14E3A53D6C6484E704BE4AD5FA0269F56D3B752A070D9872FD +A5CEE543B413F13CC4CFABCE885BA4533EA0969FDADC14DE6846B330552CFD60 +A75BC8C9DDA76102E48DB6B81975A1F3FBE182CEF410841CC6164AFA0CE51FD6 +8E8E6F0971277052E21126909B5EE51508306B886E945C11DFFE1C91BD0D79E4 +554CC7D6A774C9B98610A1E49C3326D62F157CA9D89B56699F32DBB9D49C716D +CBDEC052133F8E18E8F3D7A59008398AF3E4E1192CFFE4D0B7D2ECC8DE2ABC1D +4DF7F2EA5EC0DD756CB2FF9F95FF10E506463D3037BABD221C08506FA603DE1C +DC45A5998921C4023E501DBA494C1F41C0955A761AC3D5B13A88EC4F0BF40756 +BAE8645EA76027B8AF74726521DC129B9AED4BAE67A511EA56EE9D0D8D30C7F6 +FA8BB890375A263A0BC4E9CBBBB8F8F8613A43B10E964F8CBD6BAA2B0BAA70C4 +4CFAF6EBF4882B610AE13689823D37B046FD26DA85BD96B896CBDAB6662573D0 +2CB87D45A86A016146513441BF7A748F1E9C2AC7172460EA628BFB22F9797905 +F6DF6749F13C7F234A2CE4829D41E0F53EF3C5F52165297AAC4B5AF4747A9F53 +60370B4C0110DBB42C25308F489A856B6664371FC2126BAF4710790021255653 +CACDE1B9ED6AF5C696C3D8BEA4698DD9045B3004A5C4C03EA994A5445C9D4EB9 +865266850047E28F2D82A08617E9283FE637CFAE4F7F261CC2D616A8C746EF23 +7360553A6B434201944C04B1522533C61F615A20BA3678AF8D4BAD1FB548D82C +72D41699FB96EF40ABBABDEE633F05B4445BFC4E6E661DCBC8EC473BD2BBAF7F +75F7FC86CC7B1F3A2343EDEDF6B5F7E98A0BC4443E21611F4D807FE7F597A37A +A4DAF8A8E6D04FEFB320E0E25C61E156A72A3E4A69EDA04B0E3B72B69FC19E63 +AA33A682663A8F37F4FFC1533CCA2A80DAA675DD723A7E13D9DAE3BB0F5F3E07 +99BAB9B9BCFB29B73FC76EFA02F4F632690651DAAEC2B3688B636F0FCB156047 +F8EF7239716BE5D3DB2FD1DAEEC11DC2D65CFBEDC1C950CA737A04FCCCA509C4 +7E47664E160322366F70CCC641B9B49192AB839E0AC609AB2FDD3783095293C3 +11C56740FB50D23AE9FEAF66DB13F35367B562F15161AA270EE44713881B2387 +7889FD10198AD4D750390811714DAE7A8DC6F5770224E93408F8CCD4F8712A9B +3C583C983D98CCA7FF9DDDEF8AFA292781B1D7089336CA631413EA0728353E8B +3650EDED7ABB6E5DB5B85014A3A9C4C76EBB664A5B3E0F3CB2ECFBB6294F5C4F +69EA3AC8C071B122B129A28E04FE989DA9670EB6246806A516517DDBC52DC001 +4C027D51EF1F98506F21E63D57FEF267B913F1ED4770D892239222F3A9035C35 +FFA38D1AF3ED9A8C02235D7422695D30A2F801389DADF34CDC489EEB38089B6B +026D884EE1C24FE5785C383B988FFFEB5459A1644CA44D353503AE117E87F956 +393DEF6C47D4E1C5B1A33835C9B8E1B98809DFEE43C4D6DBEF1C2B1ADE4B8F13 +3B5D224C25927433D5738B0FE019CB88B1F373E4AC2BDB5BD86887FCFC55DA94 +E610926C5F6FE954E0D7B1D35B1E1D04A56EF39F058F7945E5E28E5D55D65251 +15766372E6F0277D8600B8F007005FB62AD3C3AE43403CEB8CEDE1268161B9A0 +3DDFFA68654EC7DF5F7070232519BFC2C9C4031CF24EAF36298E46CFC0018ABB +573A78C271861565CA746D4F1667F2CC880FBCDAE2DFCF164BB944C5F227B5BB +96526768BA33610DA9AFCC96BB51D59C946E05361A8B2CBC0F420209A2ED5CD0 +1ACC7E1D0401A7FCD4D30B20DAAA10AA02B9A6E6EEEC07FD93D19F9C0A62F656 +94328781509A5C4F020EF1D81AA3D7E7FF5CF266C34948AFDF91CEA5FD433B46 +0F57BF6F0A58B1500F615B1B8F9B04215F91ACEC454312EC1DCE0B1E40B05714 +39E5AF3F9C520E811EDED8DA4E014B8A55424EF20F1E31501B20811C18584A97 +96D33103F21E6C426CC298540C1441477B022F220DC7C268E27CEF100ABB401A +60B1A27DA0BDBE5ABED44A6F0B91F9110E17AA50A7E01776A21B4278637189D4 +5A4CBFE1457827EB635BFB6D5C8C66C3D184704C86343AC38AE8AC2D37CE3EF0 +1CD10EBE04BE03C483F5F7900AAFC2D7B2C5B3501EBE5CF18D4FCFA3EB853FE4 +83CEBA812D6FD89110D4BCCC2DCF4786269F91DE2398A718BC27A0E73479B7FA +6F9210A0E46CBFEC5B350E7405B552FAF8D2BE2AEFAC2EBC976AF49864762ABC +AA44379C962C5890A9262D1F2211DBF2D5C445E730B5B6D99AFC2732CE58012F +D29F621759BAE107CC5C0D004D58EFEE683B29C746CB6D64238A36732C3ADACB +F2D0A451F6AAB01DA97B35769DE37FF34D44A9F0B0413B027D22554CF0A440D6 +78BC5BD87B6F730EAD412A6925BEB520E9375A517F0A7790E2FB1207BF53A868 +161FA2552332B7B8981734C2AA173FC90F958F5B5EAE852F6324D7B2EBC1B4C9 +1F5F49C721AC03C6E292CE88846B00411B739BCD1DD0EDE5E4C5DD682160D99E +C747C752921F75C3F15D7AC792D8F6A2E34CC7E49F05D8A595B318D3603B1329 +06D4F88BF1E50EB9E8E592731811DA34B546C920E21E8D43962BE5396FF3F898 +B8B051F1C1221CB573FA5B18A4A6911C4F16E2B2D43AC3EA39A5305FB868AFBD +13FFE4AD4810454BBF4822D9E26C4927062C0EC6909610FB8DF637AB076F78C7 +F9439B6AEC3B17280F728F4614AB9D16AC4A30C0C298C196449836965184154E +1626224C2DAC6D959EB1924BD4198F6F71535F4E584C8B1372B68E61F2CD379A +86A1F3B4ACE861CE1234263539C1856B2F09A29D0113AF2D909333C327CDEB4C +E88DBBDAB7696991ACBBCFA7897435724ED9FC4271B1722C598A4675FCA68690 +71BEB306923439D31D6C09A086112FDDB8959A297A0BBE8798BD2FD9D72AF35C +8897EB643E41D533727ACB37FAFE37E3A14FACE1810438502C193167C0767050 +88E9AE33F6FA5E91AE702CFCFEED855A8CE825DC2F68A75FF2AA16F21A69214D +1E5AC45817E15209FA3E437C79BFE5935BC77F9B13D95B702FDF0E6E3480FF46 +36DC11FD00B2A098A863ABC98C57117054746641D67E13FE823632BB954B22F7 +0725B60DDCB4F429AFF9F71BFBF3718E3BFF39042C3A959751CBB40B19F84986 +055DD3A166FA01BAEA8196F9FA5E10C072DC49719F000785B3D5AC51F81D671C +08E1E75F7769ED943AF012CBE6977EEB50FB68B48E2202241E8355CAA28443A0 +41B65F48E30D77F34E543C2501A7259A04E7FB9A7F5062526C7A3AEFE312CC0A +819C88AA319E212445C4581778FA5F9F8B0C4DA39CE66BDDC633EF0ABFAA7CE6 +5DE9C0DB3D65B4CB384864911925ED8E9C4E28034BE95F2A5CDE40719A2B6960 +9618A3AE0CE2B5012EC5991D8C9158AF05E783054C7CFCF36885BBA426520F31 +3E34BA09E2EC53E3A4DDAF6B276ECD6224F253AE762247A7BA2701BE8CFCC707 +13B1B3049768C2874D5E1FD1EE5A4EFBFF85B262B6E9B75011365C7E0873F7B4 +DBB07CB9D20B5CC120227AD5C32ACEAF25D789D912E9DCD1263D5D7C2D831B3C +228A851D34B6EECC77D1AC51E99152334D45A9FA56AC95F770A50C93EB9B68CB +3B4FEA221278B8D7866B40DC7F00774208B5C9EE411391156FDB064EBA195C92 +26E7D9CE3C26115F0BF0DBCECD71E5A4F96F42D76E0076ABBFFBE85D37385D15 +69785E29AD1CAC714B31DFC7BA8ECA05EE8215E04533E72730C16379ED70E02F +CF95FE3D3F6434FF150BB47A2E9A2599EEA08AC9E14DF4A2B28EDF82DA12B650 +97904E35A270BF6FE2EA1990E5255A464F27E567BFE6677B7605248DA8369313 +5A954455B8D0944588B00F6C855387A2B053C5FD1A6DBD383EA4FADFC803A491 +119AE01F83D01F2F63B05D6BCC6192EF41850E8DA9424E61A50C7D97C26753FD +EC0F09E6840A5AC32C6B02B388B9B5731B9222571FDD66E8CBC6308BD4C7CB6C +8354E6DA6890AEB4A9C4D5784837E9DBCF899DD8B7F73621B99220F77819AE67 +83A9A945E10483CDE64D00F3E87E3E8D43AD494F84F9BFEE719A76294DA42FDC +C2F54FECAC4D403595265F03D9A62425AA62531B2602C0A74D79ED32A317F1FE +9F6CF08FC7C78AC2BA94A130DB14391C5E71B898744823DCB925D4BD5015A987 +8F2FE35DEF91D1F58E22B3CFD7DB2306744BE50C390FD41BDFEE515AE274A756 +45B38A972414A50B02DD8DF772459663F296323AFDE4EFB791F0D70880E6C92B +3CBD80202AA47EEA07020F030843DDEC90D8BF9C09FC6EDBE6492D3D477D72B9 +8C2FBE5C85910FFDE290D8B205BE7D93B34EAEC1B8E71E502BA4456CCF845EDD +36F02B070E234D3A5C32A6EC20EDB56E6EE83F6D80570C89FEE53727316CD70B +F669630627B5FFEB21240259E17D706C7F12DE82F4608D08FDD4950D934C2575 +E9EB4C017259B4BC653F263518B30DAE04288869540670FF58D6A78AD0F16C11 +2A16CB0BCF7D246D30B9332C4EE7CD0AD7846A5092D46A308A27A7203CBB24EC +99DD975B28EE34751F7DA715E4E943A8201FE86EC2EEC283E7E770607FFE8694 +298FF8A5AFE9856B68D0BCB7CC77B44395B43961551F52A21483A42C6E7DBCFB +59BFE42C0447495495F74ABF45CD1264B964AC75B4E0FA14A96F22A220ECDF64 +06970B6C1AF5049E9FB2EACB607F7B7BB545B8D059217603F6AE68237256D11C +56ACA071E9FA64F19F43486C1D037ABC87B995E7066E7192121FB99305893E4D +004D9EE06A1E29EA630503762E073D763DA88DC72FE6A71277C83D53B78E2AAF +3AB064553E4BEF3AE532C3FC0ECB4D9F362C6BCD7FC59F789B70ED780EB8A31F +390A1940FBA84166844D2A0C68B6E470B0187A7AB9198BAE1E04F5D7C14E7D3A +E0577D68A5F09658CCF0BDCD9DFAD419B1CE3E5C3A1AD8064358479415C938D9 +B3E1F53BE34091C8D5485087CB630CB89F39C4AC8F29DA6283C2693EB79A94D4 +CE9BF54A26315EB91BE575F9898E2E07408C0FD8563E +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: CMR9 +%!PS-AdobeFont-1.0: CMR9 003.002 +%%Title: CMR9 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMR9. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMR9 known{/CMR9 findfont dup/UniqueID known{dup +/UniqueID get 5000792 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMR9 def +/FontBBox {-39 -250 1036 750 }readonly def +/UniqueID 5000792 def +/PaintType 0 def +/FontInfo 9 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMR9.) readonly def +/FullName (CMR9) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle 0 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 12 /fi put +dup 33 /exclam put +dup 40 /parenleft put +dup 41 /parenright put +dup 58 /colon put +dup 61 /equal put +dup 69 /E put +dup 72 /H put +dup 83 /S put +dup 91 /bracketleft put +dup 93 /bracketright put +dup 97 /a put +dup 98 /b put +dup 99 /c put +dup 100 /d put +dup 101 /e put +dup 102 /f put +dup 104 /h put +dup 105 /i put +dup 106 /j put +dup 108 /l put +dup 109 /m put +dup 110 /n put +dup 111 /o put +dup 112 /p put +dup 114 /r put +dup 115 /s put +dup 116 /t put +dup 117 /u put +dup 118 /v put +dup 121 /y put +dup 122 /z put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CE3DD325E55798292D7BD972BD75FA +0E079529AF9C82DF72F64195C9C210DCE34528F540DA1FFD7BEBB9B40787BA93 +51BBFB7CFC5F9152D1E5BB0AD8D016C6CFA4EB41B3C51D091C2D5440E67CFD71 +7C56816B03B901BF4A25A07175380E50A213F877C44778B3C5AADBCC86D6E551 +E6AF364B0BFCAAD22D8D558C5C81A7D425A1629DD5182206742D1D082A12F078 +0FD4F5F6D3129FCFFF1F4A912B0A7DEC8D33A57B5AE0328EF9D57ADDAC543273 +C01924195A181D03F5054A93B71E5065F8D92FE23794D2DB9AF72336CC4AD340 +15A449513D5F74BFB9A68ABC471020464E3E6E33008238B123DEDE18557D712E +ED5223722892A4DAC477120B8C9F3FE3FD334EACD3E8AABDC3C967C61FF003B4 +B10C56D6A490CE9594D57A2D431B9E5E10FE3D8832E227A7087611431ABCD029 +85F4865E17E17F8CFBD2CADC97E0A8820E3ACEC873F31464466A9545E967E53C +DBDDB8478E69063FBB891566BAF88B7660A4405B16834761F041CCF7650AF955 +F9E853AA9F5F4382E1FE7D0C5BB4023818A2383F91249D48CE021250EC9EEB1D +2835E18FB73026250B32A8849067D5E2258797C917F998F2D4121D96560C5FB5 +B5D3471216639A8671B6DFAC5E3554EC36D9A72518525A795590C74DD70DA3A7 +78BFC43E51D6F2BA52F17D4DD00D389D3983EC54912AFF73684A8A7E345537B7 +E62361C04A47859DA084BC72EA53512DC54132EB2EE671793603015652EAFDE3 +41C4B6B679BD60AEC5153EA0D2200CB1D097DAD770F5F31E6FC475A225995277 +B867B731D5401E2D02B85BA85158C80FF7E2BBCC42B98AC867E67D25DB656072 +55A0D32AB7AA483A5A9686CEA4E2B3031D90D84DB3E2DEE7706C91BA81CB8DAA +700E5F61E07D6998C9552C81B66FD10A10033D49EF3BCB0FF22ED0A3737523C9 +8F851C61C4BF8A213BF6EC70C956AE48B5BD276CC0437C72BF6515B10739919A +F00F6ADD2798CB211668842349171A5AEB0664D2C44397E55A4A9EBDF54A3EF4 +FBBCDAD9DAEF4B0CAEF7112FA828F2F8D9F633D37E5516AB5ECEA87342EF8DC4 +3A50548490F5BC9A8A1F98AC7AEAD9D913BFA10CA86D73AEB5BACC1FEEFDCC15 +B3655522CCA2C772E902FAB2A6FC153597D52763EB44AB7489FF061F7F58E8F2 +AEAAF4D17F36CBFC00D3C653F335D14240C87DB4339DA9D30A5BD1F502BC9013 +461B9DB2FBEEC01BB18990439A0E9CA6576BC9CF6B1A3DB9386C4A5D4AA6A5DC +CFA45FB75F22E10ECB72565DB441A194902C91427B4F676E531C661F7A2C3C85 +CD534D1C89B6779B2EDC8E44667B992C20C70B663BFBF680A6CF4383EB7CA26C +4D1F06B5EF4025BBE65795F1EDB5CCB97050872D6C07BC2974F905ACDB7A765F +291365D6C8152153E7F017A25FB4476C60FD9EAF9A121633DBEAC32F62850223 +D6418566AB350F90F4B35F19598478F76B63E347D4C61E203D4DB8ECB9889181 +C387F4B663A502C638761D2782BB96EAC81A0108D7BD6938F67FEBB69218D115 +D8E89CFABCE15C6ACC7FEB983332A51A6A73CF4E341574F366713D7FB29956D9 +9BF238A87483D37E526A2EA2F101EDD34E34CB92730DCA7235AA0027189BE405 +2DAB4AA021A30C28B26C50808E1E965C02F6212EC7C72F5683339425A7739380 +A422E6191ED8453AF0CAAA424AE44DFA7CC5C2F6EAA8D73A5101D8E9517DBCFB +2858D0E8ECB7DC430EF23A9E4428CB7DED8D035D6050251AC101A2D0E884721E +2F21E573F948048BB8FF888911C508CC198BD750083B339500C426AFCD5634A6 +AAAC1C7E91249667B231BBFC64B4317192FE07FE9DA0DDB5E517D097AAE46577 +9555F29D45C67CDE9812CAD03F220B20519F2FF32DCA56A554D4296FE2D1F3FB +B209B5270E0E695EA5A0EF1144957CE045881AEB8D05D72CE57F4D34617AED67 +0D3AF0472CD8D60933651626550366E300E72A9C89ACD475C2E2ED9BD44B472D +9DAFE943F8E02A6DC38E447EED964624C37C3130E48211CA279BB6A0BD59466B +42F3D89B5746F29E084E22CF58395AF0F29E55113F3A3F2F52CB3A6DF3D026D0 +C81754B8E2E4A15F6943BE9D0087D5166060734FD07C4C57D7C7D90E8C9C1F35 +623CEEE3ABAE75E1A18A1E3B50B7266BD2D8E812CFEB4A46B856885B185640D6 +B9C22179551002B94282F57FB433B7FF157D2F0D240836B72AF4A331668AE5D4 +E6B85415F4E8B9D2F9AF90FAFAA0A3866DF417CA5A31348CF9B41B8F5F4D2F97 +CCF7ADE851B5E2E2F6E319AAF5792EBB9DA2C6AA8B73D889F3CDAA42932CDA7D +07A7E59183CD89520DDFC36E5D513BFD8AD0886046585F29B4D7F42CC0C27AA7 +53915AB1167D292FE91957E94A57FEE2D49C20C9070ECD736BDEE0F046E60350 +EA539DC298156A4E0D019E7D481FDDA6861E20678516AB80ABEC1F09B126BCB9 +52E8272A06BB6DD87ACFC423B4A4FC9A3DC8DCAEBB807C5F748F1FF8B17B8B88 +F426206BF1B7B7D239D26BC3CF0776C467A98CFBBCA5FB6145D5900137ED19DC +D002F10704AA680EC753C22E29AAB15712EF22AF73D80820A1EEE953463D4EA3 +81FAF99518D4FD0F862A324FC44C4B9542A92C5B60CC983CC8F647CE5BDB4D6D +B92B380E0E5F7208A9CD91FA9A469548162C761C1BA05AC9D60B766764D821B6 +B4E17F56CE455F06EA1EE2D38FE47581746C4C5FBA63AEE2B58E877D1A8FA83A +31C972D53B64E92EEEA147426A92CFBF76FC614119C6E9C6476FD6A069C803BF +E949FBE50B5AB1F1463F9747E8D353F7BBD991C4F90F920BC9407D8E24720293 +846D052214E60390C3CB926D38C83AF697425D80C2B4FC4706615B905516B733 +46ACA325CEA68FB21B2D17CF0B68BA4DF249368625CF83441EDBF2B86C957C1E +44CD722BD2537CE84FBA07EC7AE15C840041B9F7F3040072E6084CD55B301C08 +A64A53BD4D3DC30DCAC6C152F316ABC59B8EE978793EBD568849DCC2A75A495A +BC83470D503F8E389F54B4A4A31624E83C601B43AC1E52CB811FAA7CA6B644A5 +1AE0BFD4FC774C9C9DFC2769ABFA9C83F900BE2DD4010416053A1D4874E6ECF4 +D86E44B4CAB15D53E5630C144B0C15B58DAAD785BA298B1893D1B09BA5D40344 +6678FD2D17FF6674433C976D6DAC659175CED26139967C9B2B9CFFD78FC2570A +E5142141C2888DBF2DC8503F9137CE7CB21A1EBC2D65BF33FCEFBC85C9CB736E +24E8595CE934AB032CC70BD6A3B0F3BDBFBBE185512FDB7BE3D4A6620478453E +75D044BF770B44C9741E31985E6DAF5A318D7BED12B02A4BCFE60D25EF12843D +EFC9BAE2A3F2EFAD66D7858E83EB46BB09D2FF8AE9C43844A7001C86ED97AF51 +C511E3A89A1BE349FF5215D1A57843EF51456B9838133846F19BE79AAA5C1AB0 +5F400E5E8E7B0BF96EFCA3B8F0894BE589F2C9FB6C97BD16D38F0A237CD4F034 +099C41F85C7E2C7BEC8E02C4F327306A53B4B48B26A8926670CEEF96F6DF2281 +7C2DAD99EF8B81BBB777227C2475AE7400DC393D9C0445E925DB1E955950F7AE +53E9AC4306794239346A419F7B5DF4168382EF5956B81F83BD4BB7635B3BCC84 +7D84D05AEDC02D14675D777CD19B08124001A4F4EA96990D96000C082A12F00F +7FEF793A7FA69D56D3A38D012168C5458B667190AFE80E02C816CAFF0A71953C +D80B085CD286027E2FDBB05452AA762FD7C813B2E19A79C74190E04E746C4933 +CE1E300CAF5DD53B08110509BDA404EF07FA1BC5224BF1205DE8E0C3276A13DD +866675103B960C5F36644F96B4FAC16F5D6E91F74629B318FCCC8E8CB13EB76B +B0B7B90718D913A52A04732EA3667674994A325A7973C601A7DDD50F658E0826 +ACB8E53D4914B0274AED98D7BC3B2B7F9D48A7ECC2F8ABEE05CF2C4F2B90360B +B7DF779EAF3E103D1D83EDBE32DDA873768D8C37DC10A5354A94B4153049AD64 +FF3E0BB51AB91D7C0B4134D8731CD0270DAAF19BED9EAD800A14B65B68EEE89B +40DD624111670DDC7C030DEFE0D1B96420E249332445C155BA96231C88E70643 +D526BDF3CA1E05FEE72CE2B881CFC01ED780C10E89F0828AD55FE29043BC56E8 +2750A6DD15AADD54492F6092618F4CC6A31766B17FC60766D18C307EFC9BB787 +39047DAD6B38419EFBA46B4E2C932F97451FE78AD75FA90DE409FC6DD46585D2 +1941F5ED47A8FBAEF5A917A240959E8D9F9917DEA3247D9CAE6BF7A88DB4C4A4 +F9F5A6DCE542420A032FF3392FE0F3357B51F884D6181583A554F75B1DF192E9 +253CC828FF06B0D992D5316435980B044BB191508C7C45CD90F797F88856424B +14A5707459C50EDCF3E3D8D1667AAA83015405354CE744C66D9A5728F29E0085 +6DBF740717FA0799E3BCC4ED7841588B496A5E549B953A7FD288B4A045DB611E +E3B2F35963FF18ACCB1C968BEEA2CBF52B3999AAF89A05320BB2E97F52CFE06B +9F10E3A79865A3059A957F97972D80ADF678A36E2B586C101FC6AFA4D137C13E +EE7102C9B8EF78CB057F8B7476F146E8FF5C897FD5503DD198128CFF7B5FB339 +FAD0AF0EA967F77B07B367A4AC9F668F8BED99B98E87FAC750EE045602D76C3F +289FC9D97694C96AAC0AD1BD3FA94DF2CBCEA24B40F47B9B59E54EECEE7AC4C3 +A3F5D19160E4C1EA830D57FBE10D8D46AC5CA0260F22FAA45236F0F542BEA9C5 +5A88F878F68B36114E0573900C65E305462B22A3429A17C7A567694414DDDA46 +5F30542B8FD4F00F6C295B2E8D3A986B953D96822DB2ECD48E8BB1763434E652 +152EF3717F5E7FA10FF0B01D9F64E22C5DBD7254629658887BACEC0ABDE972EE +67299FB84A05B3EFE22B6976DB4CCA384232DDAE38C31623A4E39EA2E82C1EA3 +BBB68F1A7DBF405DEC37CB7203A895C36A44BD2D63F45B3888AF91D37B510A59 +3C921BB44DA620892AD87B665F69F6FA510B071ECC403CB2BE2F54B3969C9E88 +713244BC97C1466DA8216DA7600C221E7E7EF5C789D2E12B36422023A03E11BF +2790FD6062FE6BF62F5010A92F0A104B76E255A0975E04F6F20F760881BDA7F5 +D834D1D328B6EC19AA7D5E5678A84C74C82553DBE8BB5765E84F5A8789032143 +6020940B4B8D45FC3433D356E28C25F42D0C19F911213D85951B2B00D01B77BB +A4C72E964F9D95422BEDE582A05CD52E03D28A996E6CC8FCD910CBAB728073F9 +F9FAEED5470FFA55930447C5BA816F826F983D53EC9941EC8364B3060FD74C95 +26D4F5CA753B574FD2FA4D1D333785241D8741B79E628BC852FDC35478C5ED9A +C1BE88C5EE7302816E65C12B58EA16FEDD4672EB3E24B6EDAD5DCE263BA8A970 +350B651E5A9F3C281D85BC3F44EADD0D93402E36489BA5185E7D388974B0B700 +70575188BB610CCA20F081E2CBDA13DCC6F72567962ADB342E02C1E763B673C5 +F7384E24C6E1730A3A790D690A2103AEF88E0C1D4480DC9B25E5C8C9E1919C95 +F83320179B4C7C4A26D559BFB24D7D596FB73758C9990C451E77FCDDD17763B8 +9C30A9534E3CB6680D3D419D4B70B0B0A0D160FCCDE169714E373F65B7144CC2 +DB9A44E041211E1517D3148E65A2486CBE5E74E625261CCF65392FB4F3091473 +F9E8DF327D59A58558E5C9F7190DB577D5DC658F5E36258291C708B3D224653D +064BB6079F91293FC733710893AD1C96169B30CBFE4E9D52E7EFAE4AFEE68FEF +1AFD5E7E9DFCE8DE332B0FDC0514F9B3090AC85BBFB527FD8034DD33E9576325 +A8769AE09AF1BA792447DDD932B98FC9486B39E0B04DDB3EFB7A30DA0940B33E +E27490E0E841E87B1C90E5248A91742ABEDC10F43A8AF0F9C5B4A4930B1AADAF +01874B9AC3B8D0DBECCDA6CD7E96471FAA15CB7F8A599C5746327CE392224C3C +40BD60AF97BCA6FF6FCAB2FEA114D7300B89E91C3BC92D5B3E2C83BB37992D8C +72F661EFD0AA034C738C019DFB79BF40651A1A34BC1EB9F5AAF58F8B3DA32645 +24AFF8636486F08BC21533B5FF7391B0679A78DFDCB03DAF6BB7475A1D51DAC1 +EE4BE9B986655D1FDB6936445EF99B58B303FE79F11275EEA96A9F6808EA8775 +D873D1052FAC93769789C700F20EB2ED6D15676F6E563A769CA9298E463FC311 +83281483B1C953370D196727A6A0E66D32D9480AB1B6DCA77868C1A2D5DB6483 +5F31EB6B18EEFEF1CDC31533E69B0AFC6B30FC9912DC89BAAEEADC30BE14F448 +1A6B70D36A5D9B01799BEEA686066114910842D022EB464A9A1E8F0A5628BA69 +AA9A1925CCADD44703BC67A89F3B48E4680726DC4360274185CF3C8AB747A8FC +4B928AD62B092EFE48B01E33ED756DB696171FDB775396BBA138E056F71EDAE3 +7A1E4CC272B8418114B0E81DE0BC43DB3C133167344488820A92DF10FFA26FB9 +65FCA2C87D302E956DE6B4FE145145440C83DB43A68F8B29A592B127BDF49063 +B7F11E155CD4CAE305525BEA56B7C412A6260426407BD892A3F2B444AC3421E6 +FB6E6425EB5C3053C5644666B80405530FA0012B54557327C98E0F4F064099A6 +4ACAAFC1870359C1B6FBE7606BB8A26026AE20C212210449905E628AF1B20490 +8CE908B7EF3E3DB551C85AEB0F7FEB6A8D215B97998E5DD9C7CCFB2A9402B8B6 +1770D4023777D4B45A73F471355353412C51D4CE71FAD1E0AFBD87B5F86307F3 +10D0B94F1194EFFB64AD5DA54A4200490F609CA8B912E149F8217ABB1E9EBB3B +C4470E7365CF5E1E761AA1945044B225BD53D142F6588C50E0644740F7DD55E4 +8F73201E5354A8BC78339211AFC4935F44701FBA043AAC4BA4698E9D7700029A +C79F992F62627C91EB855F64C4B251718FDA71EDAF082A0C7B00550949D617A0 +7071FB14F05620CCF2180941341D8E60FC88823438FD728A4042AFA8B853107F +852F631518B61B234565291B5D5B89DA818DEE3AE3B68A2869DFA63255CC882C +3B16BBA08FCE3632E57FF7A07F857A1F0FDCADAB39D77960BD827CCC8661A997 +648BF5BEBC0FD2286C2A112A8DEB9CCB6330A049170D5D68EEEEA011D3EF3EBD +855236B9380087CBBB6BE24191F728B7EAC5B50F7A547AA0989B7C7D3437DBCE +1669341264E290646F2C8C5A3ACAAC7CB63DC692FAAE13E9B40E8BD39FE16A0C +1660CE66872D061056C04DDDC265C024BEF8B7E3C3AEE76FE5C9702002C28BE0 +B180295EE00E567FA2E5CD1638226D24A7C732E1BD8103B476EF5702768689C7 +D4FCD47F2AB94A2B1FBAE6ABF87B09E7713C773FB65CA83F7318035B332B9F99 +24A2C8897527021321D003AAD7C273E4BFA2710B9BB26C2CFD3D9A5D7ED1096C +552D50028AE2476FCD6D12A5D0A897521313ED1A3A8456A70C16EAA50A3E6733 +6DC89FEC56AB54A579EF264377A103939D5EE00A90B4F2206D0023AF9491FBE0 +800C6540FC945199E20E945F46CEEA2E885F6800B9DF042BCEF4291A4B1A62C8 +6A7ACFF872B25FA3AE69E0093F3D0FF13A3313430C06F1AF94D500431566F659 +E8C859A5F80F5BD2E85C8E32603D3745628E8FE6FBC50FA68F9C3811A2BEFEA4 +5852CAE2AE5AAD3230ED050593BAD0A9581EB7B327C6916B8FC348F4C23E6FA2 +00FA28AAACCB3091C1D83F7BB88672A53A2EA3B8C7C24374E400C57F0F01019F +E52D5C47F389D4C9AF126F4080F9AB8D1C8F470932BBECCEC72A9796F6E965A4 +82057DDB43D68298A00880D4C2E2496F26F015FD83C5549215753459310339B7 +6B2961EEEE74DA31FEC8E2BDDA42D4080A32372AC372524BDDA580EF6634ACE3 +128C69D04D890DCA337212B109585C665AA83EFE47D5BABC2627A86EAD11BF7D +744176652C7F9497785A7A06A994ED8414BBE8B26E74D48CB83FA24AAFBDD507 +84A90195EA3D77BCE8C2BEDDD1DC52E8164DF15D65B916EBDF3A8A76849653DF +AE3CAF9561AF3B705F75B9E5DFD6758DB65A2FD54683759912E0D0035CFBCD86 +5D239CDECAAD9AE5D95377E0D3F61C2AB7EC99ED28D2B4A7B6004A566B43CECF +7E35419EBC2A47CC76393F4778DA801EA75B4C1FC38062AA77CC4EF0E6CB0C81 +9A797C01F89BB58A957D3DAE33E4256B1F5A7D66C8989C512DA066D6DC771548 +F2DA3B490834E7B4CECD9293A5DD581A3171DB1C41F348377A602A2DDB80C0CD +7B960942967BF3983FCE91203F8C465362CA8E91B3C759943797C24CC83176CE +2D044CC07FCD6CF9DD7FCE413A0B2CF777529685866C7855509CD57489F07108 +D995887AEA9D20B3CDA77BCA95B6F655020E2964AD82AD77382DFF4903C68680 +2E49EDA9528B7D8F8724EADAB1EA2DE9F4DFDF29B22F885988C306058ED73421 +F9D738DBD7B92C3D31DE5E0367B7006F757508FCB5D653DAD14A8108292723E9 +740CEBBBBE4A72103663FE29397D8ADC9634CDB5E5E32533B70D1E617E255A4A +7A43966ED1B4286A4364A8CE71574F2DEA72E061762B22C960240AB5E7A530B8 +D23B99D098F7160C4C3C275A1F76EFF27D853689A023B80251004C69E46C9C22 +0B8A6645E492931996E6DEFAD9D535B58A149AA49FE620EF6A1B044111D83197 +1C4BA33E9FD206A7B97E69443C94B33B7844AA6061EF7555A6B9B15BFD37418C +A33E352E0FBA35589BACDEAAFEC6E0F31862296F17D9EB26257FCB0D067F0588 +09C206D19C54BC03C10FA55CD7B401C131720FEE80C43A6B22A55ECD147F2055 +02DC5F70D8D7A6DF940E23FDF870E2858CDBC2A330398275848E8C0FB114A5EF +5F29051B43DAA627DC0F82C0B510FEBFB2CAFF3EEE454AB8DB86BA2DBBE3551F +D75890D82B97F4E301E708AAC708CD1A121ADA6EF5A42E25D30D7D156363DCEF +D2F3B2510B57E77BFBCBF2B25ADF3D65CB26FB9078866AC4AC771A7D3C768CD9 +E05B7DA8C435A28E7E3BB2F8A6C2DE3B0B6CF3D7FCDE4AA4DB8F69D76C3BBB42 +92A172F98074C2F7CFD63062E2B4F4AD2D211E3AF62967A9094A1E633B0FDA06 +4321595D57BE46A004F3145E84D01E1E497B33F12501B646A45B837B8C9FAD60 +8AF3CDD6A51B56107CDE42394634F060085FE5ECBFF744B0C5B6E0B88A2646F9 +EDE447BC88437393D604FB7128D9BF45A8C3F7D97DC5E948141063413F7E9867 +980D5F49C7EC00B7794DE47B279943CFABD3F1215CC6917D4D2261EF3BA790C9 +225B6413003010BA0635D177CA75816F315788F094FF48106945338971D7BCB5 +F3FBF883E0376E17B39AC86A143F4A0FCF1DBD9A2DC014E4213A88589F54603B +E5C0FCB61A03E88FACD342B9BD2706467E805FF860B776CD54CA244AE14D6E15 +7D22BC1766A6477585B01BA1282E6191D3F22E335F72B1DA634A0183613FFEFC +3377B0001957711027FD08EC9C500F73D409ED537B0A1212B1CB9033D3422E0E +4FF3655C97BAAD3EC3E30BA0E770EEB302023FF6228B4129D6AD974DFFD97C21 +A534C827D2CB7EF41643A28E2F7FB2DE624BE8508FE0DDE6957747F28BAC5B91 +9256A220DB336B5400DAE4FD098C724514541549EE2521FE653D031928B5E91F +9AA647032DBA0F9B5D7C906542E2F5D243948E0FA7EC0ABDA2FD4A7DDEC78919 +5D7563E187A2B10233E8396ED168B969C2CC93C7D769974842E187EBE6561E80 +CEC3AAC0FBDCEC19C04BD03C4D23A1BDC50F2DC6489B1563E3C7CCA8FCEBD280 +06919A4C66986C2EE9A510A4C0F9D4B4ABBE34548BA12DF227A05D06FCC2DD19 +696D2E006CFD6D455EB7D85772D6243C64A9997E7543507A36A011003A74A590 +48C969AAED856A78A70C8DA6591CE0E29DD2B10321075A8354E69DDA2B4A1261 +4CAB73C79ABA05786357665312FF395474FF7E82E9C5CE6055F987566C2B8C83 +D91079238C2C778E65A3C355E0F3ECB7224FC5E7E4377C6E1F717DD65F6A2749 +94C93FE69B9CF025A82867C81B23EE5C520C6D50720D65D687A790E891C5726B +66D27945E03A4F713FD239CA451DAC4988526401B2A0D3E6D8A351C14DAC4BEA +7FB1AD0C5C4C7A725889284D623CD8E5DF6BC4ADF8132CE70CB34EE0281A6006 +3AA71E48DB8A35E05C2BEA1A99F3B9CCE8FA93A71128B87293E690C242ECD53B +617056F28BC1E140DA4004919084419EF327912C030B41E68F43CFCE640C6F98 +9A7C87DBCEE97B97156C9B66E94E18FFE0ADFD99BDC3BE16BD5F7E47ACC670CC +96470832DB7A0BB67E9C5E7E8F336C45B8FBAC64ABA3CEC8F82382AD9C7CD52D +6EB9B230345211D8551EF12BD09E8AD3F0ADA05FF0A37587D7D7752B983848E9 +3E5D42027B3305DA8960309A86FFF4AA265E7FA415C413EA92DE53A8005D241B +9F91FB0604DD2674CE7F71616E426152A523DE9217C99059F4883366B8B5D35B +A91B098F71416AAE5F944C4F719E293473D27243228B17D8E8C5DC3AB15F1BFC +A127193017576B8E316615F0A7B29CB988B927AF3B136FD878CC6A0232FE3F00 +75FF9BD2D6594A5CF074875159E3B567E8B4228E22102068FBD2FF3E4DDE2EA7 +46637C38D7D848FBC63099A680BDA93F59BC23D0F7FA9018AF4F7DFCF8A21C07 +D84A2DB4427C94099EF2C8C73B2B8E3FC0C00CC2F7E815278E3FE25FF700A65D +E6A0AE4F0DD5FC9C9D7F314F73556762C502836AA0D9F0D8238BC37257503DFA +0FA600E5FE697E9B4BA570D1484C058502CB1B1C72565BD9207E52AB06215E0D +235013374B68132A3057856B38D6025E42E019314ADCE73A3DEA9EE8770EA5AF +5AD55F3B9DE39492B50BEDF2031090FEDD42B3C8B6EA37CD24B38DF8769AD8F2 +3E8867403BA84D2FB5377BA5B3506DC9C0E68D0A6375CEB7A15EACA1F473DAA1 +370E1C2EE4F9BAA1215D94B61764BDD12D6CAF905BE91E722DA12C47B4E9D365 +C285EFCA19702E02A2F56A0E9B2C442AD35C19111170742E9A8158CA2471F6C4 +364448EC7E78EF92DB82C4CD0EE786E7FAB3FCE65638D350B7D776D5D03EB8FC +AF8D7161DFA12CABCA9F05ABC67B9B6C8061B5D9388CEFB872D17AFBBC17A93D +713DC1B708DF2FDD45564002E28340074FA527F6D3D465FC378B49CE192E7CF6 +76FC7FC21CF874F5A27ECB56B66A5BC9BD3361398003ED4ECA1AEF7E2A435E5E +F54C08D566C22E66608D26301A19AF5821D04F95EAF2B63A1B3D760683E4EFFC +CD65430E232F203BB02B7D88E23E84CE4E5F433D2A3DACEC303A03CF0CB6FE86 +F99B7D3462CE1E93E4198FCCD007A6A901DB2D2C75850D187D5F6023D0B1AC3F +81E83F77C4D6F0068DC9C65C037CB1B8C37767969B455554E4BE7AEF25B2BD6B +2BAD4597A538E106E092510AADC4C5079A7E0342A61ED7CC7943E6E717D85DF4 +2AC9FB655B1DB4BBF383A6CE730B1856533B11C7BAE0408284BF6FEB4685175E +A3E66A3BDF01C8867CB36C2C9A866B7A333F216007AC6AF65E991A148E3057EF +D663660887BCE34B977A1FD8C8C64892B75ADC70C5EF7FBD556A39CAB745A350 +F02A19475C930AB197EBB31DB584469C733C1E59F2CF7737EAB2D0C93DDA3545 +6B1C28DEFB645B57029B5F89BBD1BA5294227BF464642464BFA7EF6227FEAA44 +0AF14B9238AE443FF085C21DB24F1D902E3CE8C12FFB854219431D3EFD9B1E46 +244000E5F54BC0D5308FFD22841A36D4B3D0076D2D4C39139471FE8921197111 +AE407626860B99388DC7E0DE0FF83384D553BE7109C3F7B639546FDDBD1C4FC6 +1E5989BA64E0D3360FE1374B9CD32374D0D3F7AA684F1530E33F47B75F789256 +011BD971BED1863701C23B80B6BF797F3B0CCF57441AFD2AA2DF9F885B818306 +A4EADA6FEF913D1EEDA368E5BAE40A44761CBB027C5E657BBDE469A5456BA87D +C44452FBC5008944B61CA8EBF32668E9A3699DC93C47ADB451445D1BABBF585B +CE99C75BDD12E337017ACED4B04368BA109E97EFCE6F83921B65E26DD77637FF +19ABEDF7FFD9A4A065DC8C72D34683AF2613D89858E815B666D263A0F8E09FFC +DDB95859EB3295E0615B162876DF08F7F9FB3E71BE89BFC39867592C662F793B +78C2C1E32FF8A93832127158721CBAB5B81AFC43E9A300F37CC1089E432FA6D2 +0615AD054F8E66905004258764BC1F03B91B6FADD57A9FE88FE9DC81E3695C00 +B28FCE08810A2BF78AEFAF4D7BCE4341A6C47211612487026D6C846BD445D030 +F63226E7DB67853CE3BEAB4D260A36E34CD82BD33C2A729DAE25F906909D7BDB +FF3A1036AA77F3230FC6220F198433B23E2EB37A67BB1552D5D9500EBD740BA0 +CD92B03B6F4790E938A89120957B6558F644537F8AD32F0FE0F122094C976BD1 +666CF278B3A3215426D5394C693CD63C317DFDAED9C7C397E32F12C4EDE29B05 +BCE324B20CB9D91B2DE89C15408A686E568B1585A8AC442895FFC3DF565C4D7F +DE146039944457952316D29C666229834DB80C22B248B7EDA597D0AB70E76F6F +AC915EE940A720606564EC03D8A6890672B17757E92EA2F51BF0D01B607C983B +C118B87768724A97106E3C392441C95CD62C30A8564AF6FC2510AD7A83EC103B +A0ADDC62535C0A6FE3D6A47CD69552295529C8FCBE42D245A45948CEAB816809 +4D253FF46E41FEED19AC706703C9D44C29F9F1859F4BD0EB01BCD81A99768D37 +BFF27DB67B4F95FC5434CB654D4184057468B307139C8398C30F91519E8EB125 +E1B397D6419A191F47970CD6B02D4194E6E69711802814A4A740EC402011CBA4 +EE54722F48C729BFEFA1496EC635150F8A8081AE70A1DA27676A376AD7A167D2 +DD1578EB5D1F29F9151532FE1F5C03B5717ED9E45A053299D4B581886954A1AC +A747A20D8A4A7FA5D4C3B22B2F13F969C7B36B02E83FFB94625BFF05A2A4A12A +039D194EC457E00D07041920CF9C864F06DCED9610698A60A1465C978866EAC4 +E7A98BFE12C58D2E7E184A1E23911469DA708AC0480119E7A53941B4D9D13C11 +F41236493146F32456303D59DD78288A7DC6466CD2E43C394C75FE41779038E7 +C6E4B118CC66F958280A091426D79A317E70C57F19BF7F92E5FAE09C43CFED66 +3479DD1E1B01549E1EED953599A6A7D6A3EE87F0F9D16CD0696BA3333F7A023C +CCFD17A3D6AAF00F5C7ED3C68D80C9B48413A11B9B41BC60D7FB332795C1A7B4 +5D4D77434E5EE9C3BDB52E217C2F395517B37A6B8569E03A72C4FB6775C90305 +B27303AB8B01CDC9A31FA31DC40A0BD7D35CC2116CF2949FC31CAD57A6620EAA +C7513322E6C8F8CFA809F52FBA1FAD1332211463DE154AE2B7604A98BC099725 +1997C31FC2F04C4DDABC8BCEDB15E175F93DC6CDF00017A11AC0346971D81C86 +F869D5BC059826108A9937DA6B99106A3669112C5C95B27B149AA6DFBC2CB85F +84E5904CEB90ECFE90BB79E5F6EF8F9752DE2F9F6F12246B7A51405145D9336B +CF040295A7924B713CA2ABA3095F5258374147F6163C36C12F7D2F3874A31EAE +2CC95712611BEFB800AD9B84953A76AA92A9FD5E987657F6AE96CFCE251F2094 +6D96298DB951678846E675901194C0F10433A5C7467E13CF3CB065302B8EBF24 +51346C50EA48C571C2DB0DB2B703B7C5191D73B796DE9F06D2110DDA89A3CA46 +98843507948C057313EDF59A6F73149182C7F18270F26EDC1C35D507864AD197 +0DDE10B925FAF1B5A27ED05C04C5E17D7B445B4500BC9B82DB45717E896CDD62 +A1DB815F3EE978C28F7B4C1FDD35EF17C002F655B0794C801EC73CEDA6C7030D +3EEEC417712B92DB21985050CC5A5FE4ACD2FE2F3C28133B56CBDDCF6DDB97B8 +20373EAEE6901143DB8A65AB5C2E6ABA52770D97EFD87088B8186FCAB3897977 +F512AC73798C96684623AD0E6774E15511DEB92F6E466AC1BCFC46CA6E8291A8 +180B356E1FF39E8166BF801CADDDF3C73923EF03365E4B04E241954B49F36F0D +D23CACD8233821F25DC9ACED14747F30F993A22C57CD936DB69A6F19AE758A3F +0B02C5B995683BD6CEC81F56C47C07F8AA830E2B1CA02A98FFF90A87C297207E +FE4EF87639F7BAD0C543918A0D8A54FA85884CA07A9E98481533853BE61D6CA4 +BD351B79561657882382EDAE32BB94BF380D1D09ACC234C3CE99457125FEE968 +89596857B1C490238659089C6E32AAA8C699CC33C811232A3B8CBFC83D64BADE +CD2208915FB4D3F3F56ADA05EE411F2B5EC72B6C1CDFE3FCCFF242987A7A0782 +F4F71E36128820A7FD567C78FFB7D9BC1300D8E4E0ACD32AC158880A85FAD9D2 +51F8C2F001C5018E82B44C580E315C8E1A7F51C691FF81B487DBF249DE72AC93 +FE00B3437642F385B3109252E0657D4D368F395261677BD1ABC261E023607E55 +93A39A3AE751C54B50AC2D01D224F546ADE62BC3C2ABB92313C7522D619033CB +01510D1DAEB064588A5331C084958EC7D67DCA83B02776435B7DDA2EA7AFFABD +D0D4DDFB73EE917C660C3249A7C7D1BFE22EF56D74B9FD8A7F05DE67F42D7098 +AC8A89D3137F8B8C5D9F3C8D4E5D3F687167DB239BB38F0ED80B1B44679B96F2 +9A33859EDD01CB613C1225FA3926BD2585E964C201CE4717C321E767FF13DBF0 +C96D2CE9431E586C05F5444BDCE8117F9D89ED495066BF303ED29D9BF99E0780 +C931C0A3860A0A24A9E9B0879428B3BE8DA0746E60608497D7A0814F58B2E26C +53A73EDBE949F489F1DBFDE324906EB5DFC796775E466FBCEEAD133FC34E11FA +E6F202A83165F167D35321D0E7322560FEACFDF07221CE56E1A852A1AE78E082 +461236FED33C42BB91B1C59B688B501798192EE9A33C5B2253FEFD187CB610B3 +B095F8AE1C822060134274C8B25723F1CAF25EBFE63C3F3BBD476B76D84E05A6 +A65A0DF0DD2D82B7349276142DB3064D175B3BBF520C0F5E55D917BFD6C851E5 +0D93798A408A63D802143CC24848AA7495BB784A9BA5ACA5FAD037F6B16F5EE1 +E55F20F176A0A7BD8071F6229BBC0F0CEE6F48F0F516AA878143318227D9D5E8 +5046CE335958034D63A6A6FE3B900D40A7787F8C9E33BB23EFA6962580D4694E +0215417BE09B7C7BCBD80ACA584011B55542568277A74A77D10C6C1DFC34363C +8079C82367A4D252E943974A335CA3BCDC0A9B914C4875CC31A8EFCBB3A1F4E3 +C124F15DDE045DB3CB7547D825977DCBD87B80EA0D1F74527F365F4A86277E33 +AA86A7EFB17F9D558ADC5BA5AD5E65D63140016CCEC4C27B179289497955CC43 +951941FE80843F3974CF33F19B23EEA525D635CF31650E098A418BDB78E70296 +A72E8B3D48335B72FFBC3569E5184CF486301A959FD3EE1B7A415A3532623FC4 +51E79AC3016D6AD0C07049D553C3A2D5C6BB6066CCA3846761F370392D54D3D6 +0826E27843BE708825C7BA991F95B6A8199D137EDE52A17C59B5EE56145822C3 +1826E4930664AB44A7A979C18E83A677195E2DEE258B1138BA599EA8F6A022DA +8DAAB5ACB62FB801EF3EB38A89DDE8CB65F36C3628492940B86B5817EF21EA4D +8745F8CC826B7B2658E223B3BACC49793E114448D58EE7E0E7E5481FA7BF63E2 +288B4401D4B2C07696881D8B0F466E3BE65C534169A729A18EFD226C8C470FE8 +B99D691E04CD822217FA4E39F2844A956CD6E4BF046A5822BB08833358DD1BFE +7C17D316384ED6AD676A16A6B5212E8FFB6C94B0A0DED5A0D9AEFD760B4A315F +432F95AC521EEE4B3B7771BDBF410826808EDE8D025BC47E377F196454221B4B +8138F3ACCFD8566296A4D95607FE93C9FECB97F2A66DC186242A002B6E79A96F +A3BA0007E67F53904CEFF0109B2D30C235C65EDA3B20EA227A7592DB1B5DEA54 +A984E832A4345A8D176C60B55823C2B3DC2E984BC0AB67CCE6FD49FE689424D0 +C24BAE19BA0243A85DB9F2F169A0DCE9518B94287F150C8B9FFD08E65228E9EA +6CA3B5F17A016AEBD0C30CA928E6A607D9DC01724F4AAD5638ACDF714BA14866 +91F64BA92FE22B8E8753936AA52D99F6EA992AED7D65FEAFAA4F10D6EA768B17 +78244CB4CB427E1B91A28F44FA5A9B24B1560B30D6CB3742D2121E0686E53448 +586D5C22C99F3F678C7C444F45EAB6909378612F18C8923C866712B6A3042B3E +DD24999DE47BC7394C0DB05E6B669AC37656DB5E86BC5715448E8511A8DA824F +1FFC67A5F4ED912F87EAB30E827A29D25F8D4B54C90CFD5238D008E839E5BF1B +AB6A1B9881C50486D0C30FFAAD6994C6FEE021468DC3E25DCDD07CE7620FB4A4 +D44F55A763A11A086256381363D4590E8D830AA66961DD57F335F6E45632AA7F +9148222D6DFFACE123322640912ED896D5EA160765BF95BBFD061A44821283CC +4C4860CBC1DC34B277C76F52BF0F8D84136F833C9FD16C39EC27A9AB73E2694E +FED5813F32E206BA360C22A2DDFBC8A47957530BE0628DCF61B36B14728F2B3E +66937261B860728F538F7B36D93AFEA1EB6AABB113DB2B7FD98B8748B3ADC81F +4D7A11551D5625E2B3422C10916BD5A708F0012D9E8BA48E66C5145E8C31FA49 +0FE868E0D4E2425278B0206B3A029306BF88873512982D04831B70FD49E19693 +99D0E11A576C70DD8DB0BE3ED10D55BB5F1D0F75D7984A4173AE4C9983CA43E5 +6B263E6A47182151CCD54128F0040C606A989FFD645265013E717BAAEC7B163A +9BD54B47288193AD143A7F5BF622C45398A28138BF235AA290B0C1BF94588B62 +A61DDA2345BB6FF8F5757544901325C3AE01FAFEE9F041F78256DB67D8401C52 +055FB969D20C8A73469A1894B084B59409F1F566534C85AA9E20EEA06E0A0674 +BCD2966BC86C5B6D9A56FF2AEA23165926BD59A320D436D7B6DBC4BBAA31DF44 +5BD0BEE4E7E6D9E413F9F77BA4ACF5A66660A19A86880993583E819F1F62D109 +E4ECBD12F45BC1F4A6FC77CE47E827A335E7A17E3392FC2E7CE79AA6EBFF1BAD +53F67C2BA8AB5DD8BA7CF3A58C95ADC71FE0B6E4B7E0DDB562D802E58E569A8E +4C7BED76A7EDCAB4876862CC986B6E15C8B18117780D0A5B220361AE8E1F64BA +82D0E907D1B7EA8061621E49BED04691C130F7F87331C9F2C7A96032FFBEB26B +559F90578A9CB4AA279FB3B188727FC66CED40B66E67A3095C154D7871718025 +AD9389475DADE834AD2DF0C3DDB4783B83025B16709F1485074157F308685B04 +D2B8295451EE4B3DD9A9A62549D9C9BA55B1E0A6E655BA7FFC60A7B96C55E4A8 +8B5E0A80A44523EA47784644D4D0FC2000C22A8CA13A5A092E40485A0EF1D4DF +C1E9264F166A8B2FE1899D9F38C4EAEC809B80D04EC004A85FBC552BAF8708F6 +ABA0B18BEA888D1615A98A637A9C0E46E27D904C4BFD460F9223018CCA028980 +45B9F455FC5A93B36BA016F9232A34E3D3C1FF44488F32411E4E9A303E417FBE +EDA7527ED1F9CFFA6F51C7DE2B9FEB90567A96E5F31F1D9084AC9D9D58B8BD6B +9F711DC80946299D432887592039DF8C2CAA836C2B40123B321893334F7DA0C0 +DA53CE7A17EBF373FA7AC102854F9822687054BD4C54A3BF3180B9B3AEB4BDA6 +110694C0FC8F805E14FE569A0BE6F928DB82 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: CMR5 +%!PS-AdobeFont-1.0: CMR5 003.002 +%%Title: CMR5 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMR5. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMR5 known{/CMR5 findfont dup/UniqueID known{dup +/UniqueID get 5000788 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMR5 def +/FontBBox {-10 -250 1304 750 }readonly def +/UniqueID 5000788 def +/PaintType 0 def +/FontInfo 9 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMR5.) readonly def +/FullName (CMR5) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle 0 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 48 /zero put +dup 49 /one put +dup 50 /two put +dup 51 /three put +dup 52 /four put +dup 53 /five put +dup 54 /six put +dup 55 /seven put +dup 56 /eight put +dup 57 /nine put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CE3DD325E55798292D7BD972BD75FA +0E079529AF9C82DF72F64195C9C210DCE34528F540DA1FFD7BEBB9B40787BA93 +51BBFB7CFC5F9152D1E5BB0AD8D016C6CFA4EB41B3C51D091C2D5440E67CFD71 +7C56816B03B901BF4A25A07175380E50A213F877C44778B3C5AADBCC86D6E551 +E6AF364B0BFCAAD22D8D558C5C81A7D425A1629DD5182206742D1D082A12F078 +0FD4F5F6D3129FCFFF1F4A912B0A7DEC8D33A57B5AE0328EF9D57ADDAC543273 +C01924195A181D03F5054A93B71E5065F8D92FE23794D2DAE26BF1F487876AD1 +16184ABCA7446A2352AB37C36E9C9BB67CEE105C612D36566FB459D6ACFB1DC3 +7A59448A06EBA21F8824266BDD0ACF9E356251A897A920C39F1AB236EE06C039 +8C4C0AE55FCED464E19AFAF548D454FCF73F6087102630AEA03E622AF2596870 +1353925801642BBDC11F09963F4ADF037671B73D48CD9C209DB4D5BE7F4CB381 +94887E8365D25C45D1F87EFFAE360104ABD3235E4D2F4BA163408448E814D49A +3E7C171B8F1275D49937C3255326069007B302844EBF2BC1990DDFF1A34C2770 +A262ACCE874DA1E0002188D6A713C79D5E9226178EED27D6F4E4DC24ED2669B2 +3873F42A48E5C1365AAFC83FAD8B849E5D7E52BFBB01B281005A9EBEB4EA2F35 +B691489305BCFE5516DEFBA092F66ABCA2D739A9277AC69F74F786045E63B0F8 +66D545B12B0581AEB579D5E1E2A412DB8F66276E05F131EC3B9F543E11F40FBA +8CC47071DB04A5D38A707C9F825527521CA1FC412FB54DBD72E912BA4EC8F747 +306A65040671237C8FE84AB6D6157D96EDE13B3A8FBF7140F1A9DA55B666953B +C7FA638353F278D07066AB8DDF0DFFC832F7025DF1AF9F894D52E9FA95EBBEF5 +9D6B97768FEC26E845B87C6ACF5395C75B894B5A1F50B248BA33943C5AD6CC2A +8D9AC65C08B3206E7F5E5913CD0D54C9FF741A4D92B628F85E43F52082397891 +66E6F922E0BFA2E45F0072217011D4B8EB40893F151E4E2E92ECE5862C467D4E +3833311B08340AE1C0C692095AFB739E9B9A4F56BFA0F0B2AB8455A7470C22C2 +0C2EC285026195CB73EE2A713ABBDE4945AFEB92630BF852DD4845FB4C9A8C8A +0C8C971F29440B4A786108D0DA1AEFDBE03568B6B2F3953B4D07A5B7D6FF564F +6CC0D8C08CA459D556154A45133BD7AF70C7D8F474DE66E64FA112D776478A83 +B282EA4E941CE2FD6D8F7B68D129CE0093AAF7C1D345C3D65BE74EE9391C9B89 +2D4D31764F1BA81CB12CE46D15A6BF941C9F0EACB4A2DECC8F2F6B85FFF490D4 +ED7BB6C7CDED10F03F68282B2ED20809F95840A904020C98299E17C46243D9EF +05B5B8173BB2E8CBAB3AA9B8207A9EFE6366C91A7674DBCD9FAF12A2EF4B5117 +5687F3DF1244F5439ECD8AB0587F492E990DD7B516C664AFAB76CE71084FDD70 +E7F5A38BD46926615659488D3548C5066C06E443016DD40275CFF3105573117E +369C8C34E48BEE832D93F0ADA7674FC6F5A64183DB2B70FF166E02E044B7063A +41D2F396A9158A26D3953F12982ACD1D7593F474DB98ECFD00EAF57FAAF17C7E +C1E4775B37FCBCD6DB1B5BC6209162E57E21A6AF427043562D9854B430A0BAED +20D31A9AC334C87875E53E0B956D33111955459DCD98D54E1BCC61BA58CDB41E +03009979C0A541FA9F69B55D3F3750515E5F268A7B55254B8A44CA732F2B9242 +18F34C029665B31EB68331B0147607645983340D15E322E3D0532E6376D40FA2 +1041F7988EB9F56D83946757786737D8BB4A996EA7933E79BE61B4EA1115AA4C +286B7AAFE80AB7C619329AD5FA772A6279FE85FA6B623138FAE1C6AE3FA45AB0 +95D8020D2EE2979E8A87D7E7C5DA2E6B701FEB4923F401F892D83A4928807152 +96CB664018AF24E829E582FA8B4F0E07387BAC53530AB3AADF74404EB2F311E6 +C6A70F960FA962E52A30CCC6E15097F9B7BF011626DAD38EDF78745B60C0818B +34663683577D897B1B6AEB240F3A2AA29F1F1800FE0A5C205B35C6A6438259FF +CCEC942AD2D7C3CFE01F956B795C0349DD5380D0140D3BB54FD99EEE5171DAF5 +4FA360A740589A1F8AB4FBC275CC06FF89A8350F4982F87E747501A74B7C81A0 +0F40FBA9D2875022DC3D417FD623D89FB6A0C0F1B930ACC2117C6CE94AAE5844 +0B5C9DAD579B7DDB73F2B26B05E1AAE30AF69937722E8360987AC0BDBE305B5B +DDBEF5A6C03BE1743C518E6B7B58A85AC961B8755BF37688E37DE0632BBE06E0 +53FAFAFBE52EAF276365A0484A1A682C9B9486CDCFBFBC4F4A1D6BF71378F014 +56DDF9B15A5AE0276827ACBD9AB04D685F52A62BD3DA33BDC2A262B756B58C26 +4F3B552841B81CA60C951F400551166C72B8D357D8122685FE97D14AC22AB6FD +2EE4CC429D580AD7B8D12A9C601FEE5D25C98BCBCF96C87FCFBF28419BEEFFA8 +7A4FC7B6E5FD377EBD5E743C3FDBB7554A1FB9663B648B15C4119CA952A8263B +278E5845AB929942E1A0FC4CC413F36134884EAF3704865C1478D6988880EB54 +C5C5C1BDD7CB0548342B9C7B6F94DD6341C5744A9A3C4A866F620BB70575EABD +AAA42E3B6E23AA029080B299294AF2E9388520367E964A3A66A84628D75F1BEB +C7A58C92EF7335F30BD0A26D072A986635318E0473CE3CD5B4F1BBFF6E781938 +EE4E3C11AC27A1BE0C3B55586A4565E4B967ED2EE4674121A46DA1C32C5D278A +44BEB43CCA246B730D337918AAA394728814C726DA4F9C40AC6EE877365976EE +A53276A818CF25E995CAA34039339C55DDDE74B3C11742786C1E9FA5C432EAC4 +E184F35560EA31D687D005843B02F9252C2A16F9E4C31C18D0D45B7D5F63D589 +3C1FE3D33C912B9CD79DBCCF9FD8223FDC18D55762F0DB2DB241625B91DB17F6 +206120D6300D317E82F3DDE6D2D0825547C08BE7E6FF8FB54D4DFEF2F4E475F7 +F3838B4ADBE49F8C41409D85085DB5A030858AAEEA5AB9CD2D1172C8D0EE31AE +507D8048E123AEAB05979C872ADBA4833D36626166BBB2A2CEF2F3E641DB429C +AB15C97D7180E8E6AC0D871E20678516ABAA4D616B51C132F41E5661858B8C2C +ECEE4A65A2B0FF310DFB531B213196D2E2AB015963246F44E22E603E741BBFF0 +16DBFC284906983A8C15278A36F5379F46C152304DADC8D90C313961E5F58180 +EEEC5A15131E5C48AD27F80B77EB0370482D528735FBA3E6C0548BB66CC222F7 +CCE87194F565B9EF7F32E656E6964501CE32462FFA6B87087EBAF45D454CFBA9 +ACA808C4397F3852158F1C0AE44772253ABEF6D278F726A03A9CBC4330EC80D0 +117F9131C5974445B81093C1D6E093C669CD285D1BFE864D53E5F140D40BE4CC +2689C42C50C4137DE564374B2130609785D8F519E0083DD24A44DE24A68E4071 +4A2122AB8AB9649660D1C96DC58F37F88623528078BFDA3BA3085D527ED4CAF6 +25BA663A61AEC4938CA78C49623FE0EED33DA2D08CC78F86A807E791BC9CFCFE +8FBFA55EC2259B64C31F6EBC45D2DF9B47EE09AA54011D40F55B6F12F49F2266 +EDDB09D44C521EC9568CA7E62283C483A436417B65B4FCBC7EA32AC6EB4E7B30 +080816F29F06D6B09E5506D3149C5D758BA85D820357D4955B0C1FFB383E10DA +4D61774D3B23C84A339C7348ECA6078080FF8B436B2261091DBBDC02583E366A +622ED9F95D2A85BDBA48DCE8BE45628A4ADFD5A48869800D9B54F530F43A8F53 +F765F5D5C44A9399FC1B9CEFA343966118DD11CB102423D46BA0F096795AF0D1 +BA33F089AD80A104DE7FDF8A6E7D935E6713963FE3C57CC673BCDD423DF3DCE0 +C57EADCA2AE4DBB30166C2CF1F7113DF96A27D4BBB23ABECC5A4795F1290F4F5 +5293B5870F892C9A32C776AAD06D78D0514F047373879CFC5AF9D624DCE5BC8F +7CEA5055064050AAB2F24A1C792B2609FFDADC6AB8D0F31A12403CF3436A1E1E +E24BA5F92587925C4D6795A3FB163245FBF054BB1B0FF01FD63DBF7C192D0F31 +33A17E7764699B669A3ACCF7C36849EEDA395E7F55174023A79904A766F21D58 +3DDB7B4C2808DE7FFB8717658084E70DBE046A13BBF1F9BB97B3AB527D68DE8E +924DE65486C3B883CD65A95901B0DE57821CB6C6AE37644E02F5CB0163617113 +C91D2C649A0BA10736F3709BD504F22DA8831A09BD0B2D6790A498F2FBC2F642 +7845796E606ECAB2356238AAF40323D71B236CBD1E3E43BD5443F304D52F3CC0 +28DB5D33A732695AA09C0449C53C656704D820217AC035DF45F59AE445946481 +E5BFBE5D4EC0A42B5662FBEB9FF6ECFFB5E8517312F56C91A01824E401E7203E +258C3BA073C1B2532B8312F5AE29E37F5EA71C52F171593B1370D4ED5D7341CD +567390ED5B75D612D5873BAC518587525A591164E8D36D2BD89F792BD9EE6ACE +99FB72B23CB2445F12B8A6A0876EE711166C9E34897D370174C17A2495D58B4D +3DA19DBB1AF2E09C72FA42AA3D8047D05AC4F6A27F3EBF32ADD2AD2FF26EAE05 +A9AB34C9287EE00E8EC29C6603036D365AE08E00D1C7C410EA2551696954EC92 +92080C8FA5C9E67698BD7B895C82ED29391E015F83E0DA145384E223319CBA74 +19494F070649E03CE9FF905619C6CB40F562DA49C1EDD2CA221DD7D3DAFA8279 +AB50DE1C059CB8D5C9FA8137DA711B9D30F236526AA389B2F79F3D5BA6ED9534 +8852BF4448DC1B72E361A6A2B94CF865BC9C3401D995529E2A36DE755EE001FE +C67B184AE8241D08AF88D626BD2D8AA12EE791AC4253D4761A21917C7F2F5AEC +795CD14B33F59DD7080C7654DEFFFA669F50FB1AC8C0F2E60201673A78D03DFD +DE0CB142F0F616C4051772BDB837CB4CE87A0C8D61E0FC7F90517EA0F5F6B1B4 +D57143CC0AE7CA9521C1536962867D6318DF1C9CE6476CEB9ADB21655D379B26 +7A0EAEC2D28430C14D38764C493BA4C06EF4BE6780131F2FEC066C4866E0B258 +51C48BEF77011A0B5763CB3D69A50BDD50C2D7EFBBF18BF42544B1AC196377FB +4826BACB3C16DB805D38C34F9E92AA9AB5F9A9B317D8BF32AF8A758017269914 +DE3B748625D319E571187E1BFED9CCBE11ABC18C0C6123D24D2C7303DE998847 +0F6F644748ED225135CC3B957EDE8567AF3EE47F5E942EA9293F7BF74260D1D5 +A6F5E56705E12C8379DED158E66DF54349F0504EF584C6D05C44CDD8B720809E +3DC36FF60BF55FDF1CA8C968E68346CF43C568E0B3175CB7D55343DE21CF9730 +C9DF918360036E4204CD024A67E764D3CC7677E5B229C1B909ECD1B9042134BB +DD8E47FDBF63F212D401A797EF60011F52EAAFB88E3E70E007BF19826E40FE4A +73B57D52FAB33A737B1341381EB2ED307E0B83B45D6911E04BE623DA82DD6BDB +2DA096827E6A498A8D503668E45926525789D43D1BD5B896B487C0CE04FC9DAA +CD9078D0C8A0639E27F0DCD82C552A7FABA42BFFEAD66CEBBD4D2E4B4A1B40A7 +B6FD119D37FA26C14C29A70E2EDB3253A1AAAC2650009CA3B97B430B1A05F2C6 +F2120C8981683D0976BC42FE98AB2D33014B983CE1E2F1E1AC4A524249C3CF13 +4B36652F96FCBD3660ACFC2F9778B729AFE2CE1BCF473DC5A846FCC3ACBE455E +2C37853F5204085ABF3BEE50C28D99312A078CE2F6F4C1D2D3017F0E5BB5B614 +102BE45CAF8108407EA970111B5A97CA8C7A64C70D52D28D2EDCD27879B1003A +6D168CD054E87C063744CDE097887CDF268D89180AF8BB4F170804AE5DA779B2 +1941775FC32FA189038D6050EC0E400BBED364E0CCD8DA571E902DAD812988E2 +4BF761F472EBEB411388FEB6F901777D090400F695D823C758882CC008D889E1 +02678FF81B252C33C89E3B04157C889D4F0F86D432F0B3DD7811BCF01C2FC041 +0D4B1DD9CB6F2433F9B34E7E8ED8BB1FA4F9F7CEB5354A8305C372C2EDEAFE5B +4AE23CEF34AC457961B15CEA084E31B3741CF2EC317DF484736EDD688737C380 +6D868BDC02D1E63A52D21C8E7C1FD1BD8717612FC247293CC91E0EEA08D5F805 +70217761CFCA0F9C13996DE0F045F28DB67378221722416236E4DC00194E9C45 +A3839F4BB53E226919C2CD13A44E4F67334454DB7E133B2D308925CC2E275293 +F81AFEC4DCF49230F22EFA19F48CAB7E09FA727CC2E11E5A5CCF57307506D5E7 +6DD1D26BFD3DF97E73C2DC019B0A75B9D6386469ACF1A00A5AAB55F99BA1DE49 +9883DC4FB0E8AD9421B105E050519976705F7BC889FB3B6C2921D2E198FEA294 +E192DB3738EBEBF6485075728A0E16ABD53A25BB0795A21B65AFB6ABD7EAA3F7 +1DAE1BA012651B4BA0ABE02F3CCCEB9E92979F17A3338A376C57A1F13E3C90C7 +9DE8ADE421329A57369CCC2BA622840B79C0DF098EA42B810C596DE85F51EB9E +C8F6FAA598C73F359FD7CB1D337E3B083833389846A99C87885A7537D35772DC +A7EE51E8EEECF4E4F38AA04C3F4C447080386A41F1461599739D73C58CD02D65 +0899177F7D707AE793B4C915C590AFD6E6810700BC0C3D14B67E03129A4215E4 +953B88C7A9FF1AF4B6DA293E1DF2EF5667D408AD3F2DFDBA46EF95A1D8E1CA29 +FBFA01A4B965D1101CA6C0C2BF4670E94776753FF52C2751AE41872EBE3DC66D +EE58BFD3BF8F543802C7614F64E7A9CC4474DAB5F4C59F8085FD523C91F0FCE2 +0CAEE1B9613064F9C9CE9556C841563A43660E0E71EE761F0448C41589AD0312 +958167F62FB166655800150F9E81A558949071F01BB11C9ACFB2ACA4F332D2AC +598AD5ABD3D5EB17AFA97456C0DE4D3D872750F548C357FF34919A3EDCFF9658 +F3DBF53BB3D0A204E84702DF6FCF5022E753CA92323ABEDC81071ED44C306A7B +C10D15F905672783BFCDF28D97EE156E293046C09595159C735A266ADAACB887 +7026C404951C27B88CC3EC1A6F15F016D55AE6B5DD739914DBD6861098977064 +762AE2867697967117DD7E3CDFC24D046F1D83856D49224191CBAE95C2E7B9DA +4DC5ACB06B9F637312C1378941EF6B9D966E84DA69E66C43EDC0564D4A9B201E +5CCD830900AC4D4EEDBBA4AD14ACBADA20179AE138CF35D6F830801D7F9D1F3A +B48B6C2EA3B77E7EE42DC0A26C7FDA6EB64DF4136FC3AE0FADAB84CD30392E44 +49DD3F7DFAB77AC2E3A8D16C9BC002AA17A9C1E9166A21648AE2AD7F9DB250DD +2F3F480844C93059796E89238B0DD470329C2AEC9F178B83D421063BCC18CC35 +2C562DAE86EBE6529D19066FAB70C8E73B260C71924B0B3DD652AE37DC2ADCD7 +A3E3E1936FDD259A3F32D669B6CD82D73436212D6CCBE436578139AE07BA8C5F +07A8D2158C8F21007FC0B96E2B3588466A6FAB4F70363E994AC4FDC2CAB3A478 +9C98DE079DF219648A6219EC7040A8FA91BE9D9451BEA89906677866F50B6FAB +E8FC00E826B115CD65FAF738492A919CCB657186BA12260C2816C44A632285CC +25CA1D797DB0D901EECA675FFA1182AD6D48A8A96402D07A288CCC02F8A7CE0A +39084DA379C5FD08743060851BD02A3C9CC298FFA0AEAC5F5E1928C46A6EBB42 +7D2AF9D32BD0117A8CD7B453C66FF39163092568B5A26F210900D466EA5CE01F +14BC1F57BB0D3186BEE9BA1A5648887CAC06E23057199A31781C4C37B1A07C32 +B8AE8A40F5C481C8D2854AE5C2C3C746C4A2B48189A373A438F512642A55F2B2 +3CCD2D5A06A1859B50C1C3352B1B5B56E652F00EE79060842A33A8CBD002E777 +3CF6C051A6D5F02E35480569E9D00493C494F9FB977E665AAE4468E6DBBCC950 +33876A21B75095CF391E09E814D7A03EAA19BA0D7A560610CC36EC3A9CA39CD1 +4A359FBC14B07975379B68949C9FC2DFAF037C56C13802A7F66C809549EF2FC5 +25EF3F517F06F90EEDFA6F8A3E5F1F3531A5415F3191B8FA7B0F28950A20A1E0 +6E354480D09729E3A2D6796DB81825DADFA4580FDC7813B7989B18020C3AB1A2 +87E948890C25563CB26A068ABABD94F3D42213218F2ACBB2F5B317311474B222 +BE1844548554B37C97284940D5B7AF9E4CBD884D4994A76CC1ABF8E6AB0676A4 +0A8130F05B0ABAC088FE4F9FB536924A55451C8772B29087B8E5E649DA3B449B +4A055214DF69D0218549440B3665E4D20C5A8660B3E0C3ABBA3426E7820E1A94 +5D273A6AC95BFBF247F695F4D7E3C2BF9FB7C5D7D1E4913953B9595C37A9C774 +3BC37749D7EDF4B3979F4DF8F5D0225B9237410F39AA49791D3E9DD2C1C33B75 +F7D79CF6A2B421E9704D68013F510E1408F0C243E58A3B01F2AB6488D1C43C1D +11CC206EE00A0189A780A11816C1490A3D43DA1891DC61E55CCD6C1BBDE82871 +27E7848799A526BDC965C7382E650485A05341E9BA6769B4DE062EDE099928E8 +B5A7A5748A7A1AA6707FB743625A9C2EF7AEE00D188F914C8032C9FD72F110EA +A18B8CF1F442FF1D2941ACF480E012A46BC423CF80B8B27F71AEEF12A3837683 +B835F569BF799016A0F0DED29BDDDF42FF4B0C6E9DD685DE6FF0865E46F55D8A +2FC8EC23D31F281092AEBB07C9002ED1D7516AC8CC5881CC0B75443DF6DF9AE7 +0C373DFB29294713B2D142119D8097A35EC6EE55B0765351A47A433634F8AA99 +AC77CDFD58E5B4949F68A3B3AE8CEBC1461957D757E82FF2EDDD15631AB33EF7 +93175D3F26FE229E828E09BA0E09AD83060426A04CC0D66C4A5F68FF07ADCDC2 +DD0D86454F8722AA4DBBF313AD9711DCC07FD0FD78829201FCB6E8F101E3FDB5 +604ED8B8468235741A6001E63A13DC1019804DCE8FB3E8F1275C19515045149D +757890701A6EB0262C1E616C9BBB2E223EF07EB47A4A767280CF6AA71F21F158 +C945B51897394DCCF42E5DB57A3FC6D68B9933A9AB17A11C83CA0EE1FB3F4549 +7A7AC43B07267C8BA09F3AB2A52C6E91D236554B0C0E25E50F6CED6BA46EE12F +935E4E19DBC6411493EE29335EF6C3FA87AB5E70C028DDA186055C308A2DE1A9 +1193B3E15B11D1E155513349471BEFFDB5C5071BA393A1EBDDBAE2B346E522B0 +1B2812B4696369361C43EF9A2FCFCD37D32EAD713C35A1B358A0BE920F435B9B +0D53D5573F96FC2EB63BFF0EF59B177319B02EF472054B64BC918D7B61C87250 +EE1DE8ED8C365BCAA589B9FB870CF3431D4898BE3A5BD9ADFDBF9CB20B285954 +FB18AFFC8D4C4C3CE1C15236386F2BCDB0A89BE18ECC88A189395448FB8FE090 +37DE2E03530D14C1C5A498DE640F91A4C092858AAE8ECABFD3CA48388486423B +EAAE8AA6AD668286443BFF680A4E5475C2AE27FC0842F10EC321DEF74B3C9383 +0C1728E7D5014E53D83416658D4D039987A654231BC50937B2C71318A9671AD2 +DB628C3C99D0EB41AE2E505CBD33BBA5FDAC7439DEE3FB3422118CA5ED5023EA +58BF1D47260AEF9EFBCC3122E42650DC69F3D8D8B575093A6F6D6F93E171C3DF +0C82FF4ED3B1E592FB326B8EFBFA5201E80F474D2F7A68287E3C109A0B8A064A +CDDB2330DC068B5DD69F8E7BB6FE8E647AD3A05938BB75DF93C0B3B03D76EB68 +E0DE1AC5F4A31BC2E23C3CF721D6CD0BBC3030DBD0DBB1BCEA5E136D1522E73F +2AE7CEA2D566F8B32CA888276FD69F5FD361308A71A4B0F4E2CBFC723277F36C +28C982FC82EF3ED04F5FEBFBF9C2D76370E9149E10626778ED2352CD8E7E0E5E +42E27B2E31C542B1F9C172C21F2BE5932C87A6393F9AA755DFC805B5C13E94A0 +64805D8CC48F76E412EADEE158450E74F60E3BDB8D95455A765702996968BDC8 +F013A657DD6D85400CAA4CDF9BBFC714E431DFB0E59DF43AF9816A0042528EDA +29C32D43009DE33DF99DF19C1569CBEE990A0E2FAB94AE647AF6173C690E6F78 +76C5B3019B0127DC37DB2EF1C85ED74F782F2825C52006905D8D6E27BF77DAB0 +4CEA0158482DAE525B9A2CB2686B664EAB54C75F41B90240E89C00BB1712A05D +62743A38E63AFCE7240FBBF3F08983EC757651E74D28BCEE36894A4B234777F6 +E2800ED2D740CA1880327EE6F1609F9CA0C617F2782DFC7C6C03F1D99A94FBBF +2D2D672026CE044164348BD7E4F73251514B6D9543CC6FA7E236F23CCE4DCEEC +2A373F4134FB9E2A51EFAD456C431F08F6A4336955BDAB85F8704AE7769A2F64 +6E6F27EA470E2726C4A660BDB14C8712D5A68AF1380CEAA495737037C393B2AF +07072B4247BB7D012B7AD60091E78D77917B881A4749B9BFEA05F16B9285D48D +C5E38F9EADC31F697F308E11D32448F5AAF124B540D3E5778774B8F09D4506FC +9C78EFEEA457048B2510C1F46F28B9EF8FCA0052ECBFFEB938830A1A5EF4703B +CD1C445DE17A044634ADC6317AAB5D81A7700E90BA2ABCAECE474A22664106C7 +5711C23E9CC8C600879797C785292D5FA15E294C42865B1FDD59A207C388BC27 +8EE14E1F453BA173016D90843A1CE334E917A3BFA7431F09665BE57C0FBE0C97 +0BC8969FDA31EED683C479B92143B88BF6A27F3B18B12E8F1CE2B4249DB7433F +1A0B6F7367BCDE34903049F49B4C1F2891DC381F1ABAA5795053A027E2B115FF +644520CBC66565ADACBBE849218A0EEB6F3D7C788D1E4970BA3120C219BCB162 +596CFCC2D590689276DA9AA8847F8195B412A1584280BAF8F8CADCD2B0AFA695 +76A3190C601B5038E6E29AA502A290938D3C410E7D78919014C1160E7ED51DD4 +AE48E760A719DE1F19F45AD907A5355FD7F82E44300B095C1FEC3DEC920A0897 +79235EA5A815CB73EF76C3920EA385CFBAD1C1D4EE7EBF0F0D4105F7D1C837C6 +E407E1 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: CMMI7 +%!PS-AdobeFont-1.0: CMMI7 003.002 +%%Title: CMMI7 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMMI7. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMMI7 known{/CMMI7 findfont dup/UniqueID known{dup +/UniqueID get 5087382 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMMI7 def +/FontBBox {-1 -250 1171 750 }readonly def +/UniqueID 5087382 def +/PaintType 0 def +/FontInfo 10 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMMI7.) readonly def +/FullName (CMMI7) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle -14.04 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +/ascent 750 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 76 /L put +dup 82 /R put +dup 84 /T put +dup 105 /i put +dup 112 /p put +dup 117 /u put +dup 118 /v put +dup 119 /w put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CE3C05EF98F858322DCEA45E0874C5 +45D25FE192539D9CDA4BAA46D9C431465E6ABF4E4271F89EDED7F37BE4B31FB4 +7934F62D1F46E8671F6290D6FFF601D4937BF71C22D60FB800A15796421E3AA7 +72C500501D8B10C0093F6467C553250F7C27B2C3D893772614A846374A85BC4E +BEC0B0A89C4C161C3956ECE25274B962C854E535F418279FE26D8F83E38C5C89 +974E9A224B3CBEF90A9277AF10E0C7CAC8DC11C41DC18B814A7682E5F0248674 +11453BC81C443407AF41AF8A831A85A700CFC65E2181BCBFBBAAB71645535A2B +6F0F22458E1429F4A67307E01F0BCF6F337E0E2AD89658D880B04C26306F8179 +C8121B958459B923AC3B05B594D8AB95F75870019130442FD29578D44F5690BC +7281357A5041C8A809A59D0DEE108E2A07D406656BC74A9F3317CB887E712318 +46B2ECAA341F8692ACC2D14ABABDFBCAC6F35858355F1D3228B0223EC73AC56F +3C987464DB829F243E304F4C59CDE3EF6EB53A4EF9BA91510CB89A3407261F58 +A2AE66880BA98FC1EF546112892494C85A2C39F9DCCAC5766725894A7AA148E9 +42360AE64BF3A4F1F9F0A0D0C1AAFDC4D50C52233AA595B7D0CE557D4A010D86 +6E6B76A7E9523E8A6633DA9348BC3F59302F72F492A30782AE7EF220516893D3 +DE836CDE311DED9262AF01C506040541EE84AAC539B404B23033EF56D4BCE6BE +B05F79CD633FE75C6728114D2749E39FD7454050F67763AB636377BA8E1867C3 +996C7D7D4A4A02BC49D1AD7FF174C1F49F1F205BC9D5AE42BCB02CF8554E8F5A +D1876C9285B6CCD7B8C165F75843B0AA11D8462B57077AFE75BAD086E9D9F91E +30ACFF91776132F3CACAD1CA5E08B17B36A0E45ACBAC52393B9AF9089BD821D9 +CD5A9CD9BECA59F7445D63DECC1B4502D299DB85B6E2EE7C69A1DAB91E22A3A5 +89B524FA20AF6005E7A586B90A2C6E5A93C9EFA4ABEF5F7E4C7B81363FE8D2B3 +0AD637FA863DE787581ADD7CBE463F7866C40F4E280260ED0E9C8453E5C7E668 +FFF058B9742DD3F131C264F8FA102CD0DA05F3114D13D34D422799181453FE23 +2FC6EFB01BE420C930B879D671F3DFB036197874725220644A5A52DFB467BB75 +8089E4F40CE9401777B9FE1D0AEE02E782A6EB2A185A454AE9394094CDFE7CFA +C03C23A78EAF242E4F811E4C83B59EF4DC5ACE4AD37B41616B46C263358710B2 +6137314545CA6CE89119B42A3518EC85C68DC07D26839C68B1FF55C4A9CD518B +A1FB32F9C475BB6110839FCCB94156E7B3648F27245A00D2966FC4DDE3996BFA +F463A663CB6935B596B1582ED0ABBC648AAA8A86068BF0038001C753C8BAFA0D +2058041DFA720B528E2D4B16196DB1CF30C779D3F4800FE662D5B60B208341F2 +A66EFCB8448C2FCD12DF0DD899911A8BD96C9B670054D328790E5D388518B146 +8CE92E368EB1DB3CAAFCA4834CC9D9D9DCC80FB1F34F39DACDE643052C977A7E +A95C5FA8DFED9B4DCE769E4E46256D6DA8FB18FD7FA4E4CED5D486803538F3B4 +6D3F5B3C03184F5C26C66DBB4C724918EBB6A89C4602E4EDDA81EEE2BD18B683 +FDB459F2CE0A9CED23DC208EAA8BEDB304B00E093DEE926A7B32FDB2EC70DD85 +94B9137856DDDABB402B2C76DBA87149051ADC6007018EBDD571BE1D092EBD95 +76D4E063AD7D5F62E6C26EDB88D38678F2806A1F4900B0ABC4ED034A818119A4 +E618F1A902315BC98F26775E59555A3DCEA1D0F8B20A9084920ECBE3F7F245AC +1182A40B518B194669D95DE968542BFF80FDC89669BC256C44CB66A2AB8CD7A9 +E42C69956CCB6BDE8C09AD22EF3196939B3B84EB23A6E071A36D702909E019FF +058F27562441EB5CAE87A4407F67C4390810BE89BBE867D636468E73677B84C8 +5A1228DD7DC8EADA221B1BAD5F43E832F20ADE7ADBFF170AB306F5B711816FD1 +39B7882556E30F002977FB88D8B28826A75DE0D20354A2D41F2DA8578376F7DD +F27B0F59D4DDDF5790E11E3957491DC74EEB7625CA49FAD90FA47AD8E0BDE824 +FF326A84846A47A21B70FA549BEE307F9C6970009F963B49A504F0115777826F +1D81203F655C242FFF15BA97E3BDDFBF435B10E74CE8543C98966223818839B3 +6BF3BC63F882B0AD0FDACA8C56A570277952E1D83F18BEDF084D2AC004E2B09D +70DE1740D7D220E92B54D2FD0DDEAF1E08C41FD321A8D474982DD105B23166A7 +AA9E0129DC88065B1E0F9382BEB4B4E1DAAE3EA5489BDCA921AD5A8175F2841F +9400478DFA99C5E5553F383882664D73FBDFA29BF32E52C28DCE80DAF4839434 +022FA515679DBC13FE98968D2894DF5DD69C49BD23D00F5D858B69D1F220F968 +F0700E13873579B3CFB658972098DC61F1DD580105BC27795DB4AF11A871CCD6 +2E1B9AF7F0DAAD4CE315379A7B42CECB983DAC5A2B9426B4E5E0A7F7978504C1 +DD7E30063AE3CBDFB24EA2BCCDC478AB82084FD30A4793F4707D9F8F9647B413 +F8A5C5AC6D5EA0E35628CE1096A434FB8286F4617CB4D0AD30A4A0B255A5A356 +25AA5A947FD3C4FA44B4AA80BAB44C48CC1E2C6D0A711365A37A58C3483D07ED +301A83D2650A2E8CBA9EE62FF5C2736EC82C1402959F64527F9B640619F112D9 +8E0F4A8A3078C72ACF3F34AD855AA4008C96E30D9E8C414607C34E06E29AC5B9 +2EE5DDB823E8C3EEE6A8DE228313D476A7F39B5DFBFBDEDDF7C45C1C88EE6D01 +7FB4F7BB2CBBD5DF7F0CBD98DC287FA6940FBFE1B3B136613A3CF16634CA7B90 +53D5FD5776515EFF5D37F8FCC62D8BEC8EE2216503D54D6F2032D3C2BF861E15 +FD1B45B71576F15852EEA65DD372E911EF4CC18283CD2FF4196A3F1A9D81137F +F1820EC604D6C61AF318C6C5AB6DA1EDF305CADEF7CC0183B86D31310A09972C +A4BC37D110C77ECCA614D1A281EE1C2040B4A5ECB31A3FC61760F608E44332D1 +D2C53C7891B505A3020E9E4915F3618588FCEC80B9ECC5E637D8D0F3C94B1F2A +C53FC46CAE0AFAA7E12266C212A73AAE60199752C042BD55A5DF1CD07FBDB830 +C83E7832D8554AD9C9CAEEC7CED1DAEE622090897641CF2E5B34A353D83264D4 +4687522DB290D3BA927BA315EA5D25B0D7B69350C6C180AB0C322B05E01F7C7D +F2F48651567F0C1B49AF3950E43C94D78F7B184BF2946B924BC4279AED28F3A0 +17A7D8B235698A516D3FB5DF0B18A422B2410C385E7E9439C6D60917EB3299AD +E31471616251FA40C9FA098109BB31A54D9C03B2F12947E4E9252A0851B81C4D +F39E7FC44752504B589C3911571B1D3EC3BD1E1807F99CED1DB20270E483A805 +CA2A016E7283550D1B1D35C226FAB63F983CED41A4D02A2F228FA9EF065027B3 +CC69D6F2E278C0A2D238D3A37154B0D22281F62C61D9182A69657B027BBDED64 +11E261E47620602F865221A534C5A32E2BF5B93A187911A146F2E96538B47DBB +7BFA7EF406FE940F4DAD17E6E4B80C4F031D71F65657C2F5C8233EEAC68DE8A7 +E1FC3055C122C1795D0C71A0284F89A9BF04837F61C9E08DB42644A490C97D34 +A5D3CEE475B8D578205005A0D68AF94AD27C0E855BB8EDB74775690A4EDD6543 +BCC10CF13283D6FA8A7CF3FE6C4F96470A11FF0B0160D3F9816B13B0BAE0D8F9 +B84C7631063FE658D13D108D6FE24A89799FABA72E6A6D1C943922CBE676C1B6 +11A4106ECB4F1A7F8A84B2783C2E6A109C58D63FC0B74D8C8A1CB62D527441AE +E656D94B1AA8581B4F07B653ED6486AAE1F8ADB30FA8D8914AF24721C74B0908 +D84F2EBB91144ED4BD7EF533F2584048DEE37E17CDE5FBC2992A6F924FEBAF07 +B626F988599DECDAB43C931CFECF99FC6EBB72F8E542765C26295902DFF60B7C +7B9ADDB4858BC9D808B7F0909690CF8DFBC59A786D48B891937C31A219842A43 +234425B4963062DB4C4E9F534C77F4243408805B5A6B8BBF428632CA4AC03A7A +E336DD181CE0CF3E742079E2919EAFABE16A63299771BF276EFA8D85C920F995 +5B9D4E8F1ADFCC5C29AA89BF90C186C5DE7679906B2FD4DB279D245D27D08837 +D3A8D541FE37415B706EC585C05804108C1D938E543B8B63E275EE85CE9DD843 +0A8B9163144B77DA1A552A25D5E77E94F29CF252BE9950F4E627D5F72536B6F3 +3278D4A45D10759F16AE42BAE8460865FEE84537F8EC9BF4813570E883B826FD +1ABF3F4E66DB6FEF8366E07BCF290EA67D39C9D81B2A7EA48E0A228FE3D5AA50 +1A56CCBF229C9AF2537A8FA70EEF41096ACED34CC7BEECA4EA1F23B39FBC39D8 +CCEA93E63F508CBE6722C11467A3D0D5C4C52031DE43C449333E4295104651CE +E13B821D7904653346067E971BE0042C571ABF40C3A1079A675FE4264B784D46 +1B8FAA4CDE9851C4EBF69ADF51A7B68CC8706C08D13A44909D4C1D78DB0E0B2D +0E0318304B229DD2FDC968027CDFF65722059C62154304D6F9C3F06DE22914EE +928B7D1BF1FC7E74B4D882998D59BC086AA2D4EAD0AE39F6B75B5A3FB9994506 +E21731E1A15F0F2D12F88724BA72898197A80FDAC00243A3038871EBD2F2BAB1 +C616278BB78490CB86F552CBE5DD0862F3793D72C68AC16AF8E38FE1A523A5FA +9B0428745B1455671CFA1F6BFBCCF9CA23C833113C2948E7A6AEFFF1A83509FF +C559BB5EE7F92BB43F7F37A371E661C826F63DD0C1B25E34A8119E71EC82FB66 +23C7B126FB6554E7560B1B69F2EDBB742F3B20D1648C151C37A8570CBD330A9E +7592A8607D2D727F3AAA0FF2057DF4E2A4C7D3B658C6CED38824A770420D89E7 +F6AD385DBCE9C9A9095CF0042052A67AB804A6675BB9373A99390CBDFB715984 +A069DE543E4C6ADD7F1EC7A15392EF834EAB4584679A43443953427DB13E6959 +0F2F5061C99C6D00FA5327FDB5330AEDE19A53DE3AE092634DC6AEEAF63A5BED +990F8A117AEB1CA0E7F7DBE02CB3D86465F1613B976D1CF6F3A1E69740A2FDC8 +062ACC45EDA6B863B60015F276860FB79C31D28F97A799568E66D0A8757B2C41 +E939337B467303041D0F4C59390B2E41E5F298F275DCC699D27C459ED4D5ADBD +02539F00095D7E1872862142B46BE06513D3EB1A406E6BAA64BE795122100F09 +C37E5D1834218EC1D11B031C7DFC9F5AB071A8F4DC08203821366959E9191D4B +289682D915AF28CE5858F83338DC51B6B0DD052A181D9133FBA50CF18F70EE65 +C33726A0450EBA9D0E0C3662AF6C2121AB7911AA9880D6BB6811D6D7515888E7 +199A0E632104059A88C9D85B19BB35EDF4AB95E1515BB2339572928BD5FE8CBD +2D4DAF55DCFE29FBC4C3D56336277BA0C9A889A129F9FA7052AD1420B8705163 +1A808EC1284C888D78CEA2B4BAB71AD76289F5F4986008FA9BF328E8537E6C91 +E11DBDD8447E1C9ACE18DB0EC3D5742C264C8EFA445C5D16C2930FB43669774F +A2CA52144D99EFA8FC427DB4128CD4C036A8C611B087335C780740FAA419D39B +5DD68EA89C95275F9254D947EB3683D0130255269B10C6CFF29EA0BE484C9949 +96188FCB747618A8044E2E37DFFD2DB8ABB621B34DC024259340677095B6937A +78EDCF508AC91D4CEFD872AD73F50582DC8807143CEB9F109C84DC5DA30B64E2 +E56DE973088A9D32583D6946DB4F3523902FB1781D993B89D5F56D79D5D98CC1 +7FEE73FC3A7D1BCCE90179AE450829E228B4DEAD3B2B4C79A400CFF899AB26F9 +048B0875EBC871AD23BA96F88CDA8B87FE5809A13889A6AC349ABB25E54ACAA9 +C213C5DE2D01BCB9CC0D7BBD384D23AE12E289FF8FDF1F611F5E14D4B20B15A3 +42D9B3B37A83A9CA39B5DB6C8316C51B70F211530A56CFE54D63E88169CF5233 +D1A7B2388025B3EBD2BEE0716C3A2D589EBC7A42B3DA602AC4E2FD9C9052C922 +711E44408DEEA1FE0C9FD50A39AD46D437F61F284A2EFD42EF158EDD71A1486D +4865D6B5E20E60F4F4FC3D646909FF1EE2D7573665E4CD8340A1B232CAC0202C +C35BA9BB3D2267C7E78518F6711633F888EBEF72DC750AC2CB362D528CFC8B2E +A1AE1C05456F50EED8CAA768DEF47FF85C4322F02D7F9D188C6F285C674EF589 +251B0B913339FD701FDB281338D96704ED7ED908BC113B4275A24D058955890B +12CCDD5572D63688426B0E1E9A40D6AAECFA5555C1CF9DBEF8C04CE1E5A63F14 +969D39B6DAE8A91F6AF4CD1E2DA89A4661DA34E272B6032C442C031F081F5DF5 +858F4620885773D8A2B2F5EB6DDA74C1408DF279900450E4A3E80BA9A9B1295E +F24EDC3F6EFD81A741EF74B0202820516C4FB720687BDD915EB2396128C3B262 +20E3075DA153D6FD36E1C05B855929DAA4DE694B6F15EF2145C63250B24B031A +4CF0AFDB225E91D99828B83BD90F1702D3906D45872587A3A116B138AD9627CE +E778A949C392202823C670FDBC56F1896FFFFBCF52C4B400F67BA36B5FCE44A5 +F18EEB8ADFC088C99DFF8E0A593E81A5ACA2E3693005F723C7D3E0AE2BDD3805 +8C6007A00542DEB2539709558A88B21003CE4B2C7817AF207ED576B25A41DEA0 +FC55A459BEB00ADB01309B35920F04F84B7B64F95AA99EBCB843A06CED900D99 +97BEFD7CCB9F4D85876F10160C8D63E2FDE82B7A8D945F37CC9933ABE0FD1D76 +268296B1A5AB06B2E814691128771694224781171DC6266BCC290FCE1AB59416 +85530368115BABD4F1DE45952918D1945D51EB713C283DAE8EDD559F437CD886 +A4B1DA6120D685C284673A3EE489FC1AE4297A3623B339B7D886B6B4B8F9F4A3 +7BF85E320A52FDC6323B51879B98A14C33C567BC069D9B44616514EE1BE36F90 +EC5FA33E1B6B0A46945D876EF0085E74935DF2560A03321861A752E59742B9FC +5C501FBC64BFB1602459885B63873DC857ED37F8BE1A9C6E9517B9BF5A6161BD +DEB6DB0381FFB34A8A96AB4AD48BEC40D4C198ABC599C3758AFF638AA75BBDA4 +8545D5F95FA426FB25587301A43E176F6CED7851E815AD907F2443E70740DD2D +4FBD5D978B9B37F59D6DCF0ADD0F90825DD23558FCB858513602C8BC82BFA383 +7AA6DCEA4009961D06DF233C5381A7F9541259926446B2F03664BC5978A1B6CD +EA6EBC9FE6100A65959513EEE32E69D47B55BAF30A893D77142F943982019C01 +715CE29923795EA01C58A798979939B507C5B29A32881877EF7EF0C5CB3DE591 +6B9A6C3F3FFA847F396A396F078860B59850BA4CA3115CA2376AEE6B30C05DC1 +6F9DB6781ED0F9D45D10E096C33B1B7CD12A9D57C6E49AD833C4B093DC82811F +16B3BD902BE764A1680831EC5A6C1CED84AE0DC0A65678EA5270BF20931E6409 +7AA44EACB22CCA11098F8A51096BE83A1ABA56C9EED4195D5CCF24FDAD92E823 +C439DAAFBFD652157D728F2754F28304710D3CB33763156D76A259D446647A11 +493FAC70DD28063A4CDDA162F72542368E1AC2826C4BFF7109208F66371910C1 +068F21779FC39DE03AECF1C9FB2F417930C22791961D801284DCC89B0833B6A8 +D63F153ACBFB7B7D547924613BBCCAED37D90BAC5B0264ED31C7B9DA5A2BC620 +9B20CA48424D0FF58905BCD6190BF4B5FC6ECCA1BCEF13426920197CAB41C4E6 +E82E8EE7BCB23C6BA6F8B58001533B225ED721D6CE3D6E89116EC33CAA6E905A +649F8C6A1AA187A48E20DB864596481976216DB78F0F57543DFAE3CDC0A6FC77 +2CAA49442527A5D94DC54BE93C875690CBE52EAA4EDD9F2A511361BC0F0807EE +96AD0D26B62D809E82EC14EDB158EF48A748A6FE0C3A7EE5D4479B35425F35AD +3EC7444F6FA75CEA5011AD571078293448A33C7647611CAEE87974B0A756DAC9 +4E1BA78DEE477FA59AD50BF5C52E068A5E044A4A4994D5B24CC5045F768A3C51 +D4F65E2A5AFD271A7666C6835E28C60751EE528C0742433165AFBE71562A3016 +F59676D56B0B5F7E4984D664BC3ADDAF24B4205752EE21D4B57057A943018466 +09C3FA5D2C5BCBFC22A643586BC9E7A965DC34C0A7D272B5B1617BAC2B0CB510 +5DD5EC6F7ED1226D19189FF547776698FD48B7A6A038131F869A9E24006A4FCB +9FDD5E4A6DA9C531E1F1D1F0131CF8BF06B78BD2C6109E3D5251ACCAA6661142 +7E0CF66D8C1998ED3DDDF69890FB2039F35BFBA2D9E6EA42F2E2E88E8C66D0C7 +6B2A404F1C72AD38C2742D4AC7AB941BB9306ADAADAECAC68EB229D3CE861D48 +CFA0EDBF55C5AE42B7888DB9FE6BFE987CD2D13CF5E416829ADE55B64E33933B +A1D88A6B89D36D0DCAEC29DD4B4C2D2021ED9831ABC1F94DBDF4952CBC514C25 +9F8C5C39B4E7A77247AA023F208B63AF2A8D436901C08079E59BE74DBF83018E +D58B882B96D1D0AF2274EC40BF80AA2D323DBF12839DDC16CD48B4F6549B8EDE +4B8D2EC13F781065B10E08698C3C1C07186089B5FFAEC9BEAE7F753DB024376C +044C96A9CC8F911B864C5D7230690AFAFB176EECB756DFE15133B7173A6C058C +62584ECAB08EF752434BCED665E7E1C67B7C49BFA4516BF9E870BCD342D18AD8 +5CE246CE72ABAA86A483CE01CC370C45A6A2CF9B4C84EBB2670C26013BE4BB6B +72A2AFADE105643B389616752FD84BAD4064ADE75FEFD614DF91A1D7A51A7BDA +C3FBA5FC6BB033A2228140109A3A96E66BDDC633EF0ABFE5B9171F8B6AC4195A +FB9B5A6E500797F3C699FD557D7EE72E6E6C488C1AA6A9145C03B93BE2E0D498 +14EDF39E75D3F62ED42EDA9BFF1401C8D9C032DAC774726E508145E0CB1AB50C +E6B684EF487C09DA232C595154190FE09E6A330A652F0EAD7678627D66340402 +F0983F7230053CEEFAB43D57D28EA5490F3FF4082036E44B4B3C6882A01E7DDF +0A525A55F4A93BDEC579F3B95D79D7EC560779CF97524CF43AF77F2E3E4B79AD +93DE87DD4A235711D29F6E322D18287533C6E87976DC48DBCFF7C32FA1A15699 +E51C19B8A8F7A282AD66A4F11E9DC71A95D893F55F2D25A558B54A536DC1226F +B369860D92AEFF3DBADC2138616A6DEC038225A1DB86360FDC351851BB3F6477 +A81BB64339EA068D4C69FBF46AA415AC050F4B384AB2490984D4735FAB087332 +026110F37FD85F4823610E40599825A725E1BBA02FDD8AAF552E41ED96DEA6E5 +395432C075355ECA2682198C660BC5430E281430C71A7D51B88399E2280C04C6 +4341A4A5181337B094CE54C1BA18386A1B7AB451BFF4A0BA6CB2EB75835EEE5D +C299E56C190DEB843BCBBC5341C46AD68F750D8E4C14C51D100E8DD1AB74832B +D3AD6250F9C70A00A5481238CF8BE3DC8BBDC653288D5839CC10434D8444F8EA +0CEBC60963F5F7EE015561A98BC9FEA1226CC9AA4803FA8FAA501DC4AF35FB16 +44DF25DEB1EA03873A92A44EF48C23C1F6E07858F3B27BF91CF5D71935ADAB39 +45AEA45169E6F0C5E5B3C540D205B5A3306002D27C43ECCB210CA44E4BDED337 +166CDA3F32AEDDC043C41E89B4D1618CE32B722730CB16CC8E80897857639073 +3DC92FDC47528DDB73C7774DB2A58BBF6EC17846EF6F22EEEB279051AC158458 +36707AB50BCB5E7D7626F6A0EA716218E25EC3625C3257550956670B88814716 +3776456D64F1CA72D9B6B8C279A4BF60536ED68F8CD75CCA956BC43A2838A969 +CACF133F498E866E58F6B387B1F63F14CE65831CEB004BD8FA08AF779A301173 +CCA0E328F2D90B90834D9EB594CDBEEEB115BA3D079EAA85C3A5440773EA1ABA +62C26A39C10574C2F16B6692E841CE259229D372728C1ABE1095A53F6D04C2ED +55306F319662DE5C896F1ECF80554B0D3F1C24B28B82573321E939013CC205BC +B11709F5D5BCC54891E48A913BFAF170EE22C36FD17FBCBCC6D66E4F624F8D34 +B78731A0A1835562766F316F0B6ACA5F3F980139DFD40B72D20A5B4D3DE08CB8 +82ACD3B7AC2DA97B7B72848D68DD380FB98C56AAD6553E4A4B020BC1143A5BE4 +3AF30AD92D2254959512883A1BE009807DD47181689E0855F4E88D0B2D097ACA +9EA378542772505E28A8DBAAC77FD83047F1A8D445D92DA8D7B08F2E65464768 +DA3556E4291AF5C8073B70EDD2B5CCAAF2D0D18EAE346237FC03F1383FDEE633 +04E3FAE364FAF8954A7C0BEEE3D3020263C67DEA2D9E92507F6989A716A25C69 +54910AECF9BE706E40D2CD6406159082C8178F0B2244B9784A9D2889D447664E +169A2BFA4BD2CF5E7BA0CF0F2AF10916D8E58958E4928D006CE176E9480A52D7 +3ED689EF3AC1A5D9B81DFC37D3685B97A8C16DE2BC636FDCBA1F22B5E9839055 +5D083B6881F29892725F058AC1DFE0CE6ED1F638143A6A09C42B0BEFF93A2AFC +DEC4785DAC33EC0651FFF17B64D69571E1308930341D0F54DFB85E61CA830B21 +9969DA945A2150A009A16FD1B4AE5B907F33CFB6F9A304F7937F11101B15374B +8E1543DB3B682D8B3BC2F3EB176F5E117F492B1C0612D70A61561599B20A849E +93F2421E95D46F139CC54D6F0961234730B08F4BD49A95C10881098721963BC8 +3AF937A35BD3ED0C97DAD0712D564A9016980DDA8F55E683130FE2E2F1444651 +31656EA7FD9BB6A8F1EE609B7C0876FFCF026968F7BF1196AA667B5A6C09BB49 +150DE63DE6E1364C1202AC119D54B38630C01AC40230639A3BF31EFBE5140F24 +0CD1DCFC121401B070ED96C087FFD34EFEA59AFDE7BFF59BAC4B29A7CCF35C2C +ABBC8F7E53B2114B5DC2577C147C1B5993B555405551B73189C8DF522A059316 +A810B83D9D85CDED8402DA6D104AF92926C6A8E65E4161EA1D9F963B8EAF53C3 +379CFBFA57EB1CE70AC1B77E5768C55949EEB0F4EFE080AA7D7E0D6E668C958E +1E9EF43BB0D52EBA8DD69ED7110CE794788B42AAB194623D738C286FBA57AEED +C98533BDC43E75DAECAA179A5E4F28E478FBDC0B4BE0516C42212124E1538E3E +A2877DCD24F83160899DC328A757477758178D8368D22CBBBCF201C07ACB3F29 +A82FBDE310F0980267A1FD15695F4910907E83D503078189E0D616FC60BE3700 +52A61071957121BAEB355B4C0423B4D06B00A891E23C777AF921DD15EAA19ECE +CDD14CF2AA130F59690D7A1F1FF0C692299553364BE4980F56A239E6E3C26BCA +7A03C73D8BC39667C8A8E77962116F8A4BDEB04E19EBF5C4B5B19ECECD640A22 +A2EA3C41CC282E12EC7A08B0946E9043419A8F35A8C0492C374D675125C7709F +9B4188DE22867A7B75CDFCADCF7F83C015397D +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: BeraSansMono-Oblique +%!PS-AdobeFont-1.0: BeraSansMono-Oblique 002.000 +%%CreationDate: Thu Jan 29 18:27:26 2004 +%%VMusage: 120000 150000 +11 dict begin +/FontInfo 14 dict dup begin +/version (002.000) readonly def +/FullName (Bera Sans Mono Oblique) readonly def +/FamilyName (Bera Sans Mono) readonly def +/ItalicAngle -11 def +/isFixedPitch true def +/UnderlinePosition -104 def +/UnderlineThickness 69 def +/Weight (Normal) readonly def +end readonly def +/FontName /BeraSansMono-Oblique def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 0 /.notdef put +readonly def +/PaintType 0 def +/FontType 1 def +/FontMatrix [ 0.00100 0 0 0.00100 0 0 ] readonly def +/FontBBox {-71 -236 691 928} readonly def +currentdict end +currentfile eexec +D9D66F633B846A989B9974B0179FC6CC445BCF7C3C3333173232E3FDBFF43949 +1DB866C39088C203DC22FDC758584860EC7BB67FDA28CC6208249060E18FAB32 +204779B5C03C0493BBBBC95CF02692CC4DEAA8D2EA90B5C2E64374E92BCB8501 +429B8FAE4A76C0C6B76D6FF7CF9A7D5EDFBCA0E959541C59BD05B7DE43D25D53 +FC3DDA6EF0C2743978A6D03E19CCED4A11F2EA4BCC3110BE8B8D9E2772361969 +C19258EFAFDC276CB1ADE9208A941A36D18A96F6D1C771F81C4B3B8CF0CBC2E8 +4B44D923DDCE84E17DAE82547EA9EA5E732D78F03AA245377BF0780A3752527B +6E63F6A41202E7A6C4E4F9330A0AABBD04387E12F2ABF531216BF498DC6B6BE6 +06DD50B385DDB864515170905E2BF225AB9AEF29A2181200040460795735C124 +59C90AE9BF9F01F42A2ACC5A5D97D4F6548FBC903ECEFE9FBC8492EFEE55ED6F +F29A7BB50432A42FF30DB3CEBFE1C2BF83D613F2C846B7F2173F00B4EE5FAA8E +12EDC39BDDF670F50F6AF072673281554416E77FF134014D5C465127C28647BE +3E810BBD9EC9F9580C21D2E9479B8F0435749734B76F4C06995E769710828768 +3141ACD67803BC92D5B405AEBD4B25D5DF255B110F1EF1C35D45D24AD0B1E0C6 +8816579DF17721764D5D2CE396276ECE1E6142DC2EEDF83FD7B65717064CF671 +051D92A9993F82D241AA1C3254DBD3DC53886264BDA18DE9FB8F584DEC9A32F9 +82A6131D7140BC7926C074E26639A7141B87ACCD64E61A25DD7E26F463FD7834 +964655DDF83538CF1DEA3D6363DE79EBBCDB429279F84F5DA8035CE823A76FDA +C2FE77036F5B9D71C0CBB063689C0C9770514E550CDBB1BE97D763729B78CFE3 +DD162726B3F3190F0003F4812E9877545507664B32A1BC0E6C933BA64EEB3563 +39606495CEDCB8EA6E63ABEC9F3558DAA0BAA350F45A6E9B145480D9513B87EC +14F7551489E0F96B698A104DCDAEF9160549D551093AB3F5F7EBDDEBCAA3ABA5 +F0CD388ED13E9611675BA045A8E99C002616ED89D1F3EF0537C6E787DEF9B5D3 +F20879DF1555A646A79400D3EB80A209937A45A6345C4E51748D3A5C523F9F93 +FCA3628D463FABD8537A948B3A3715BAAEE70345F5186A550FAC78ACD262E671 +AF8FDCC133800B82183CD17B4EA18EB6AD976F91FE3C1447EAB9EACE7D6B11E7 +4D1015AFF0AA59434AB021021583981D521C63675A6C426548A5CC8E4B0ECCC5 +695862EA52F05F456D95F64ED5216008507581A0D7F03997804D0245EEEBA286 +C0943443D6C32EF37801802526D1D201DA5D6F0831D55BBFEB21CE143AE60AD0 +801701D5D1B4C71AC60CCABB5B2DBA75DD6748D2809D68D08A0FC02EDF035E55 +80A35385306E02A4FE58AD6544DECC40A0B403C3629D71E8801EFA47F1A6F0D6 +BCEE740700ED57379DFE792953217878710DD6738243B6ADF65120192BB2BF3D +FF86ADB1A5CC6A769059AAF0E9DE78A900A9D91F499862A3CDF88D3A859BEAA4 +96773BB6D09063CB2A5EE7EA30709A4A49F8DBB3A8412B0C0BE23CD7FF4FB643 +22E214AC5983934BAE11E37BF6096B8211F53A1A7B07E6FBF58A400CF26785D4 +E9B948307E2860080DD5AB705E8B28BE1616B5F3E36124053EE1B79071D4140C +0A7E0EF7E23946D5B02F94B9BD0340EBCFB66060672D45DEF45B1AC742225B32 +9EEC2BAE9D72A67C4C0D992CB559E75086438EEB07FF70EAEABDAECC35827FFF +0C1A2D19421E4E4CBB179BA37C3A9AE772BB4DEB8A013441178B6F38C56DECE8 +E9B49D2D7C5F50730FB699A0B3ACB111453C6EA156609C0DE368F1876D0973E9 +E256659FA6241FC6C8BEC45ADA3CF1A80550E3994AD9FEC320BFEE78AF06375E +2EB52E57A652974FF16F1FB8060B98C1AC61D0AE912C51AA15E6AABE35B02A79 +B83AE270373D22100F356E079A1A947BD9BCFB9FAEAA1279937D5EFB4627B8C9 +7B37FC8DF5791469A2473D0C3B93A9FD4925C3631B38290202B7D93BBF5C62A6 +1F1212771A4082E0AA3FE903447F4BE93B20CE4E32797500BB02E7DB04F1293E +E65A01BF2B19965DC8C43D5A108377692426134249CAE52065C557EAF4CA02FD +35DE047E187B4D626DF88AB5A2B71ECF9FDCA521CDE0EC09FCAE5C0CD879059A +DDD23A446A5AE7A4C6206BAB9B28B8D5F637451B51366D4E3A319D3C65F782F2 +4E439CADA79BF02F98519A445380503C88D48B1C9110550F022F362DAEF98E40 +4E2450194D0A88652F6B1039AC6283D063D9C0C5054672F2B2D5A50038BEF503 +D7664A51240B34D0F29A0E101AA1E4321C4F8F92DCA331D2D2F2AABCB242630B +516445E7AB23FD63B22B0B9549972398C05259FAF4EF3C72629FF57D1539588D +F311F8DA3AAD81554806D43679D0879CA4EB7CBB28F7E6B7BC94448D6759457A +F638B80427446A047657A84CEB221646D487940055C602E222005DCB6CE28B9E +9978C70CCEDFABAC18FE2E3E85C25567E70C446BC483A1AF21E9B4AD67542C71 +4472461745AD6BF3DB8C9374E6CB56E6476900592A9D31C8DA957314EE4004CE +945375880785903717AD52172D3E4342961AABCDD2D2CDA30CEAF03FB7413605 +966E9A0D2E61DB4677507D13FFFBC8E9A98D856EBD09D0ECAFEF61BFE7ED218C +2BE5BF36ED031F602CA13F51AD9DE7F966659140B9990F4F8CCD00F5ABEE957C +A3A01E64C293A3C5EAB78289361AC6DE1D044B53BE464EAA5C2CC971AEC3A909 +4627030766B7F5BB4025DA23A224B8A8C6ECC7A07483BE3352E014AE7E6DF74D +91877AA9E5A3A56B8A17971DC49F8DFE2F802FD66113020BB63E67BBF46D5BD6 +4128A7871487242CDEE61AEEC8F02CC1890DC75E16978ADE06865B4229345FF8 +34144A8EBE08536B5F439F8C213C024A4AE7DB698F4E2D78E826C06B3312D367 +48E5A9D1DD651524129D12CCC40D9676D9E1796DF010A5106137E47FC6873AC1 +AFA8159C33AFB0AB002AF710E40D577CDF517C5A5403BEEF65AD6E18A725C439 +69421FD80C6F658AD29BE5C35E54F6207D068F424F791BDC369E353DE65996F8 +E955B21FE78E89706AA51ABB75DEB0DC0702CFBF27C489D5BAD77093AD5412D9 +21CEB2EB35A6854FAF4C2EF3672639E0E6A071ED31664D7E490B757550B7FBC1 +4BF8B7EAC158940CF910BC7C3010FFA33D8D0EA05A135BBD41A4E97930A45D32 +7AA10DBD6854D45750C34846FF7E009D3897BC89D47D607F0A57281BCC09FABA +6B154C112BB927C1EE4895F9AE02C4CF4F3FBA281F0256B3E29EA63CD521FF75 +2D7B7B77AA6BA85C87BABD9ACA77DFBD7243373AD423BDC155B24750FDBD5979 +BD7286AE12930BBC28F08AD604EA75996F409F073941D95FD31EACF66FF60D08 +503AFA76F10ED65D2377AFC034EE16379FE7E115DC7BFE1E23529E42A3A1519B +A113B08A47AC902CDD3F1401D328E0713E47879DEE2E03073C97F8E15BA724B2 +1B1767F45E152B99DC2DD01E6F5273D38DB9992920361141D6CC1B6A58A81E0B +E223270C8BE3D454E9BA929D63C5BDB885AD523A8432D34F9D9FFB0BA0A5A4FC +B4883E1A415AB2EE955E834A83717DA49D376A3108F0BCF6BC3828E805DDB44C +31CFC32F7FDC9374B012C32536E290A4B16EB7A60FF6ED08D9F1F74E2B0872B0 +148718A80025A2D5A8661DBBA6B0F627C9AD63B752A6BD50A1AA7AC94B8A5A8B +7DC86F13EDE3F912C4ED80C67D61B1E045851A5C61BA3C4C6DB996FFB3522819 +098585B9959752862143A781FD1EDEE8DCF492253235E2A12D4D37C9A6FCE1DB +3723890A3369F36E4E4341EB1350ABA1EDF7BD18C68D507D6240F4CADD71A81C +C3D412DD8064281C773368D312C25F0F663FDD5F175E9F4A9B0D23F3D027779F +7738E6869A887C552439E1CADF642D08D9A0B0A7D19A6EC464E85E7575945B21 +F712421B50424D2C2055E7EFEE0E51C4D7BE0E28CF6994FD5F8FA4BCDB4BD206 +D749416ED6DBC83D52E4859C7628CD9288AFEFCD622721705E375C439DABF76F +897FECE60AA138C2506D14FC2B8F7854D771426570AFECFE9831DC9C4594EB9A +D61E898B4338F4A57FD6C825694EFC76015BD5B5DF105323C991D64373FC108D +22A8A107230A5A118DE26D2B134FDA4A0FE3922334381AD94609BB18762F4137 +2AD265EBB5C19B833BA546FBF99954872CAD7038DE6E22D6D8363B260074A2A3 +3432E26D362ABEF95D3C4DABECB10DC7F4750FAEDFC6FFBF8C9DD43B9A1B227C +84C54319A3565E1947A199E706268D54FCCF862BA9C6DAD8BCB8284E7F7F9A6B +C335A32DCDB32CD2D4B704751934AA06FA86ED92C2E138CC4BD4BCCA41F8A726 +3D34E5DB51AA4A5242F317C9EEEF4A034A349773AAF54B3F146614836CA1DF48 +FD7B04F7E1375C5BF720556F44355A5FD18D34E57E20EB9C52EC8EA95BD1CE26 +6508A2B6F942D8BF80184F3F68A49EA26057632B88E322F5793779B63393098A +4F1DA2E8E8C06CE5FF959C22019EF1DFE83E26B47DBD131F5A7AAC4634AF4052 +2C1C317E5F51FCB7CDFBC50767780E178529AE9D9BCFCD011D1404EE44AD6A6F +AA66216BD3F6314B31BF48ACBC0323614B1DEB34B9FBEC02D34363210A76B3AB +274DC2D67ACF33522E163F5C102F0996FA2C78428B50A22C69D9F602F1B509A7 +94B765A498DC5ADD3AA9079434ED30E03C3B3719878EEC443BA72FDD2E8ED3DD +C2FEC45AB4A96A6379475D4486C9D61F255A871712C4D95A65B2CF33741CD07D +36F965CEDA8D2A78CE7A3C6782A916802368C95C90C2C1C0BBAB95217C22AE84 +F30FDA2E34B4A8871BE8E279C14BEC7185F491B1B34D68047E6F5C2CE60B0B48 +D1C408193AED7ABBC2970A4AF8601D3B39F2113A06A0D06CF1B601F8B5B088D8 +76DC229B91D2445538CDAD88BEC9605265F224D41BDA9E3B4D571BD233B9C7E2 +1F1408AE8476F0E65CB96694FBF9823BC7B83777041C563B81183F027F16DFA5 +78A09607026D0EA52B8C03732F12462E6E7BE50D585616A1346AE3CFF865B33A +CC1E0BD0FB84E66A3A162F4EEBF2A84768AE232E57EC0E5DA7108CC1D00AC383 +CC2477110E11FE616DB2691DEB44230B330EDB576DF31B0D427A9F189C22A9F4 +80461114DE25317CBB000E3C2E60556B94CA2A3312F490A6733D753CB146681C +057094FC3EEFAAFE189A68898A2EAA5961A851B01BF6CDCECF333E207A9965C6 +DB37F6A7CDAE6D17FD451420A55B276D48630CA997857179415F245135FBB40E +5BF4417E922937AE19819BFF8FB5D310E4679CDEE5EBB6DA9C8896545318920A +EA07BABF986CB09B2AE53B2D2AC1A0DA3BE5BD68DD5E17824D6AAA47D716C316 +8A7F205BA8CE208ABD7B062195F2B8BB8E3697A12E620EC92B212EE90ADB9D8A +AE61F29E1956DF567514925F81207324C164F3630FFF0E9799376AA619AF774A +92BC252A0D49D244FFC336F89297AB11ADB3F954122F49687AA5037D4BAB38F7 +0EAAF8C7265943040605EF5B6D04A56A166691CACFA44BA6ED9A12A85B9977B8 +6EE27A65C0CF0FFFECE92A5306D580573C998916419060511C8BD500AFF2AECB +852A03 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +%%EndFont +%%BeginFont: CMSY8 +%!PS-AdobeFont-1.0: CMSY8 003.002 +%%Title: CMSY8 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMSY8. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMSY8 known{/CMSY8 findfont dup/UniqueID known{dup +/UniqueID get 5096649 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMSY8 def +/FontBBox {-30 -955 1185 779 }readonly def +/UniqueID 5096649 def +/PaintType 0 def +/FontInfo 9 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMSY8.) readonly def +/FullName (CMSY8) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle -14.04 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 2 /multiply put +dup 6 /plusminus put +dup 15 /bullet put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CD06DFE1BE899059C588357426D7A0 +7B684C079A47D271426064AD18CB9750D8A986D1D67C1B2AEEF8CE785CC19C81 +DE96489F740045C5E342F02DA1C9F9F3C167651E646F1A67CF379789E311EF91 +511D0F605B045B279357D6FC8537C233E7AEE6A4FDBE73E75A39EB206D20A6F6 +1021961B748D419EBEEB028B592124E174CA595C108E12725B9875544955CFFD +028B698EF742BC8C19F979E35B8E99CADDDDC89CC6C59733F2A24BC3AF36AD86 +1319147A4A219ECB92D0D9F6228B51A97C29547000FCC8A4D80B73E7B6CB7548 +0E1D77FFC695988391DC44AEED8CC947B3D7E198B9620E2238DB3A2819182F03 +14498B8CDFBA48926DA721920B221FB33BC21A8456AF10891403501D0F6415F0 +7E041AFFE109F640E54FC1A365674711EFF94E752652A4C8DA62CDB1149DB899 +2C4A4BD77A06E81E93C5698C05DD02F74A0756082738BDB53003B483752AE498 +DD718AEA8F3FB5A6B7E2D2AE8F309065F3D556F9A34AB90C88833A54295E0982 +209C466A301BA3372AABEE20D862C6DC6B6FAC1463C8CBA8AD766ED1B4C9D712 +2BECB4E6ABF558D8AB5281C35726BB8D046982C0DDAE17BFFC9394125E4E84C0 +B283977D31460B8EED4346CCB09F64DA0ACB640C6DBF32F2DC30D54668C1CC12 +24C7280593088E9958C047125C323E9C842801346A9CE5F50413D69F6DE99471 +65D2E387765E92EA4C43F17B467DF6E266D92551009C0E52E7219AE9F5E2E8D2 +88852086FF3600BFB50AF3EAF8C009D8D5F084B510F792385F328F7EFA8C38AD +CDAB2EEDFBC6AA45F6DD7364C2F02DD2BE6C79C8361E83D4267CEC2407689864 +B57B5D967FC80AB3BE8CA43955FB0FD3081D438437559AD24A7ADD484C1E4A77 +B00BDCB0D1B121FEE983412E1EA5489BDCE5DCB4A7310135956B230C0DE7BBED +516369A92BC41FAC8D73490984942D930DC193BF0C774C1AEF627B969EE4B001 +11381AC57815D7581E2372A1DB740B09F4A7FB4318B765DA4E7B44E8805CEF85 +44EC3B10613FE7B397BF91B69A5CB2E9086D1E7A1FAE0A9ABF2A237A2D29ABFF +E392A18AEACBF28274E775D3FBAED4F46B982B9DA4602E24094EDCCBC9D6FC0F +67C60376879245606C0B9C9A678E8917516BE1CF097B1C75C9F0DDAD6899F08F +81FB7A198D45FE060CD2A550D9B8D58B882B969D7BC3EC018A1FE92BDB262835 +26516FF97AC387EA525FF987E5EB1EAAE4EA5866C9CC043D183763C530F7D7A1 +2070E4044507C4E5611FCC117FBE4396E0B24F672CD53D5FDFA56D561FB86D40 +90A52A7C7F29A119DFFB694A8C445367746A49CA5FA83163BE448EB19625DF26 +609A8C8672631A10411036CAA3D6C2F822A4B2505DCBE8E1CE6916AE515E78CE +E8C894B134BF811671D17C19248853131DFFAEDB24B15FF17EAC194662460642 +48D23515AD434C68DF56753806FE96DB3A29F9D4980E0B6EAED7383F9FAD2584 +8F85679D6D1933677ADF5D31FE6E43DABF137E834A1DEA632D36EA5728625159 +8F33348893C9D1C510501628E4A9A80587DF6E179FAF68B158406A3EBCD726E2 +AC17F2DF8B3607072824D2E5A6479F7BADE32E603E54E7A83BB2DFDEDB0D80CD +1F1BDD4F6C9025CC3A8A3685DF10A8183FA80CE0F0FE2BFF2500F76B1037AE41 +56D2D7FB468C3CA7549E3599A2AACF66302572F1B35BC8EF7C23F6FC4B720228 +1D673D903111CE378AAE83480B4DEB7EA84BB35F4F7BE997DD4FFF5A9B5E7650 +3C7365BA0965B242DC369D03215651184024A08EC284F1798B0726152884B4D8 +46411B3BC8F5CB53A70AD53BD7B1AF23528849B56CF2F75609FE715878CF6448 +38579380688B47AE3D374B0DB6ECA5B8284414090123D47F3F9DF389E023F431 +6B4C0DDEE4190DEEF4CA772742012754DA9A44A8F550FADA6D8FB0C512D5BA2C +BB7DF71B1DA41FB6936BA71B710CC6A80751E43436F0F5888F51AB370DD4A088 +40F402661E08732C960705C7E0D29B8DA1A8A3E119409E51EE575E9655F47568 +023977917C8EC610BEBDB9C1F4EBC192084D63EAA00EC87B53E86620BB5AE51A +579B0F8E1972CE1DA02832FC75184F45409DD9D413402C2FA27BCB0AE6DB4CE1 +5AFB114E23232DE07C4674969A1FC94D152C6D3A611F029ACF4D949A2C0755D0 +31C7DACDBB4DFCFAF7E87295CCF5D4AE28D87ABC3533D87EEF1507B9FF9063F7 +F007DF03734A20BC3A198B049000C92D2849C62DDAA719FC28EB372793E20128 +D786670E08948808BA45C4F6AA0DD8A8357CCC47228A760C5C7864853BD9C9A4 +285E446CB72C5C9D3F03FAB000810DC5674061FA116153040743E846506CFA67 +FA8770E785D7DBC7293F37389A0859CC22275B23C44B57B92A033A7BD194340A +D6BAC6E103402F8396A5394C617CB2A27D9500921D5DA06115EC81715EA8E360 +4A95DCE8689CBE268B8E6C8B806CFB0ECBF634250243C9E1AFFACAD65311FA4C +1DB0988203537E5EF12B86FB454D06CF05E4F4FC4143D62920E99304245B6ABC +82E21192BE94F5E980DB21E07FEDF0EB0CAF4ABDFC20EF08A2A44A7940E2862E +08FABA516152DBA899D6CA561EB08C7C8298DD511084E5230CEBF28AB1D84C85 +B5BAF4616C28FE05AFA4BABB3BA03F8CEEF3F3EB0D3034BD3D8D06629B8E9EBD +72484057E71C9EB4DFC7C2CE7749271AC926C2419D618FB0B60FD6F81F0F2C04 +E529CC74DCC9E4AF32D935D7CA65FBD0842A7BAF162D08F782E65DF3AD9EFE69 +198AFA9208B22BC21BC4ECABB9BFFBA84C8421B74748E1EAAACEB59E566838F2 +C439579D6C75074F94493F7D5C8942D881AAE6302C37A61003AF77684EE99AD7 +EB5EEEF050C52E75E4E842F8174470FA60CB213BB6F257EEB26CC55D1202931B +B981DE2535A87CF37BFF912220C117EE166CCAE58BBF624170FFBC0716726B36 +304D698F2CA131675F7B1199E6581B8C812C4FAF75814282477BE2B7EBCBA7C2 +FA6356B2876F89698C6C2493989E9EB252DE03BB374622A7C88B8EE9FEB4851E +02F85CF70CDCA7BED5EC94E81170FB557C3BFC6F32CD70B506E56A2CC5451592 +130C6F570C679577C44D60A811DCEFB3535D855259B1021AC168DB76DD38B459 +F1813E3DB20AD9275D58880CE94058490ABBBB0D8DD16D4A65F680B30057F15D +07FF54D779CC978E90A850A2D8D79961EE276BB26B594C16F56557435671A3D8 +ADD3DD0BC4BC73B0652B9DA8462FE614A14A11ADB2214EC4910823FB4CEC8938 +24354C5C62547EE5710FB5ECD93AA45DEA7411F6CA8D5B5FA222685E622AD73C +1BD5DFD1C8F253352D6C7D14FF2FACF07CA5EB11392C6A33E9CABFD99F743BED +55E8DEF49E07B9444171D1780C698E5B65B540F41F845AC188DC7C0C8B5BA5F3 +546BEAAA4F29F0DF30D5AC12A60862742623100074457BF2D820DCAAB340C1DF +22FF26992DC02F276B40A7C528C84B72CD189A4088C90070486C49A1014FDDA3 +5093FC12DAF309FBC063653E6A33E14EB51B081126AB5BCB8016E975060B796D +6CBFDFE6CB9E191F12E309254BDD6660E3227012F5DADE246B02B6D8127F1EE2 +248F48DC5ABBD229748444420C868450602120664984FE3A8B2373232CE5CA15 +E67172CAF7EA99F73A66AE33B1EDE5E8351DB02497CE3338930147873D4E46B5 +E918F5592A21789FA9F46232BE693DFC33240E2649D6AC1940B101D806089436 +F0CE230BF6D0363510378822891DBD3EEF3365430E74FB6C994BB40C8EEDA593 +CEDAFEBC3B87C7C04F52FBD7D557A77B44855CDBEC8A8D26F44D66446B5662AD +15B6B79D13BD262D79BF51C09F8F690A61C7F11EC2E05F6BEDA273648E9E1EBC +3D5F1D6ED486797F753064ADDA8494405556D43D395B1487EDE877CCC1EAE954 +1AEE2513DDA94897B733609E8179DFE1975590568202EF20D58A89B34874CC3A +A5579473AC65E0EE61545AEF578ADE10BE3FB3E2CDC96678F613E4DEF7BC1B38 +8A551C93438A0540E292F65F1FA73D9AB30A5F545284085E4930A570C1807995 +7108AA8194607304488616DD4AB189F8EDC5F4507AEFD7847817A8ECFFDD2DD5 +DE95E04EAF2774F8AF661827229F01E769A32734097C457EB6FF056D90C46C5E +9F936AA6BF2EE8244BBEDB8862E56423F5845BB10C8809730B011D650EA18945 +83C320A3F4ED095A5EF91929FC877ECC28A1C2CE74A53245E629BF46C6534983 +307B1C94EB35CECDEE86551F0C308F66690E591C5D9D8FE14F534CF3F5DB6D32 +39D7C799F392C4E65759B7BF61F4D8312E3B26C31466206A367A0DD7A90D25E2 +805D36ED5CE0A4BEBCAC0A348FADD2D2AAD670E28BE6E33F627B4A9F35078B1D +6688084C4C18840673EF9B3526172EE14EF64E97C8B006C27DF6EF73BA6E6459 +3608F10EB1EC4B824DD5360B42AFF2084788B165747AABFB2C9C0EBBA9C6246C +08F28143809315D6268E386F09BBAC54C34213438B56C386B34AA457D149CC39 +A3C163F56E3C2E637C63A79CF7F2DE969EF3AAFF75F3F20499A9ACA61279D545 +866FFE57D25998244F448AAB042B141952BD653007A889B1A716A4042CBBB827 +C1ED5E3C616C22D5FCB39D3B3539F9B71D50D25ECD2E5D953C7A5194C867DDB1 +75CE10B2C3A20E8CDE15AAE9F549D034838DC066CA649C31EE365D46450AF18B +893748D055DD88C72391FD92DDF0AA1A1316051BC756F87BC9A425019B2FA7BD +CA9F35E89E2229B3021DD24C9DF0DC3217F44DCF510116DDC9E92EFDA3E0A892 +491C8961CA484BA29B8CACD2FED2E246FA7414F892AF7B9F08698FDEC82D2E4D +8C6ED1FD9E0BA2645663E7E97B0F780033D025F2EF3B364574C03F04FFC4E09A +3D4182BBFCCCD8CA54A92434E2208803ED36F931D31C7498B7FCEE4B78AECF11 +B27123A10F031F9CBA36066680E4748E346D18FD0EFA395D1A9227530618ADD9 +64D96C50A0172F712CB427E8F7170F8A1691FB95B247A271499F0E63676BD64B +167438333A3BD9CD9641AC637E0ADB17F1E394DF34AF5A2BD02030DD087BBA02 +E1209338B74075744A9A9966039AB6D223F385E06FF359257913E6C01BF32F5B +A3640C7CA41A6C51F296E50DC1CD1D68491CE5D75E0ABDD4C5C4A97C8BE2BA35 +D91AF9F8F38E7AB8C1778B06A212D0416E6137087329AAA2679C51D014CC4DEF +0F1543E2FD97621E552E19B29AA1368CB5046C188BA59B1E1875B25FB269DD2D +C0678A42CD310702465694F2BF302AEBC43F714F7C657505132DDA71CD23D17D +628BF3400EB5DDFCE2850F31B40C5913394649AA4241CF646F3939916C2077AD +C11F8C3E478586AA7931E327333FCD8A2EB31DF4B47AA2F31B999556602DD998 +5848143C0CBD0B5187DF08E675550F2384D48C2954EA7D16AD2B94DF97C194D2 +302CC47BF968CB1F6C9641A87525D5C6972ACCA1D4A4B4DA8EC54BEA6E7312B3 +2737754A9FCBED3BA5F8220B751506BB0A5BB5CF706AFCC04BFD713A3BE3C842 +9FFDBAA0CA5691D0FCE76134841A169970903843163DE68FF09649E99945DB64 +491FD6D8B2DF237DBAF550D1B74263B3BF1EF6211F40D4DF2DEC46E4DE557088 +5D458A7CDC6F34B8C3F5C76582C8B7D24A333AAE0B43CA67436F8ECAD7C50B12 +35BAAD521033D8024E4DFCF01580AEC63BD5F8ADB778F822AC4FF0DA608EB7CC +1A52A51B9F38FFADC0D2F4D8394C5D598123B483AE5AC1215F24F2492EBAA134 +3C58AD4E3B493AABDF753EBF1F04058C6981C353D6788C2D3294165B667D3765 +7883A8493F5D772F789F73979B254BB9397FDE4F8B76CA6FFF312F4B012949F4 +3EDCF1AC60FFF2681549DFF430B498405E79CB407DADFCEDCAB1E2FAD3B0F4C7 +B575A33EAFF9F67D37F50BB06D06834EA683862991C5BE340D4A7F8D131E733D +289F72977870F9958102F5D835739F0F5A642E7194E8AF7FDA2937E5C0712CE4 +EE15CB475725030CE0F238C32775D3EC95CE3D0886046E3EEAE322F59D423E83 +DE88DDAB5CE0F70537487EBA5A16A9306BA7DDCA14A85A9F83B3BC3600DC9421 +BDADA2704A5A24F7174A3A486ACE68739EC5960038CB44D969A594E97AE1F42C +42C1B7E3A456B3CDFC7683537B337AA8777F18CA45B22C8BEB18877DC624496B +9E8EF9F386B7FE4D94ABD5CE3B5786E76FE04A35909C472C04654A4405932EE6 +A4A3E0F6DB088603FDD82F0EFCB85A962002A662D1AD0647A2F484B59075A1FA +9F4327D542ED7962620B998ECC3A95CE736089593461CED9B5AAAB05C2DA3820 +5AF29B32904E1EEC357C8E282189C6D8F562DB11291091E6897E24CD8F8E2A34 +67F35D261E01336152E69831B0CC6B03FF1AC7EA22B745F513A25FE10F70E74A +3FD71FE1EC9999BEDE7CFA6A97EA2DF621711BE950FF8960965685BAAEA71FA6 +36F165573A0A6A92DAEE41A9B0C97BF03419FF6BA1F44524D00FD671EACC4233 +9A857680DFD27B9F1E7A760058C8277B3761DB9969241D1824A7DADF70BBFC38 +89C6DB9091E3BFCB36851CB9662B365A4B7384BDBA1D385902D1E5DABA72A159 +63790096B927EA49299AE03E41C7F593F3B995D1155E91C62D5F68845ED3C797 +7798B56858F96C2FE2E37D812873B7801767082A6D6B60602CEB94B7F6D7A142 +72814734DC584A18D1FFEA7333D4C7E2DFE5B91E9AD92EB52533CA52FA888980 +C79FEEDF0FAFB3B20B497F9AB668BDAFA364B405916526D5C48E57DC30BC35B9 +BF684FDFAD199D9A28541327F3899AD505FBBBBB22F419E22E0149EE5FDD2B43 +AF442E2724EC16934C307FA731F90AD1C76B74569B78CDEA7C19808233C6364B +D049F3F50A68D3EC5F29F2B96BC58C44AF90FE13F244B7129F14B5DAC3F3F310 +B330E67CA4EB76F895DAC8025F11EC7630EEB4826B5721348FA38F28B2AEEB1C +5F7CA413C486C94B1DD96988F07F7A08D3635E28624878E55D4FFBEB5E183CB5 +93014D8A272EBCF3A09133603CFFEF43A6FF059BCE525706D3795884FB495748 +C8AFB50DB464459A5C36D7CD087631BC09C693A11BDF1C6011D9864FE66EE8C0 +4824A42E97D16E017C0BB73732D4537E2A5BB283B3568950E5681364DBC1D6DA +058A122862A44ED85029A7B39F2FA7C362D859D0B429E056891339626AE6091D +74F6110CA32F8F3E9507216997E6AD1A2F1FFBE8B407C9880DE230E234F6FD41 +A291851178BEAE5DB9DEB22E3443807E2BCEF12A4AE8C490AA9E3D2918329F47 +42314CD258A0760DE5FDAC4AF7209CEB530B8508B278952A3638CA8491C3B493 +516F461610DBAF32E923ACB834B15E2D169A10F2609EC0ACDF7BDBC777DD2C19 +2DB330FD557D3B06B78824D6AB237D8F2D57A7ED35F02CFA9290ED6893B41657 +A2485524A3F1642FF5808FDB77015D9467EDAC4A6BBE8A175833B1225A5554F7 +2C60207F7A2632A4023255E8EF1F38C1B8216C9B03319B60BEC9799E5378683F +46E698FA75F2DFD40F17C603F3335F1EC70DBA1449C7A33EDDD46CC3DE4A6557 +EE67BF349AA8680FDBF415DC527129288B5879500180F75DF80394EC7CF11BC8 +9EE9A4ABD64DE3076A2963F83FA333F19FF74F79ECF36A87A31D6F125FD2415B +3F6A6FC33C179F54CA164835F3C3DB62AF444BDF27AFF23098C7212477F688B8 +6F48D9AD080D544D57FEDD33D3D9AED9346CBB644C97C951D4AEEB2BF6F0CB18 +E2517BA7ADCFD00FE44E290E731D53E5C0D762FCBA08004562DCE6A5EF2FBB74 +B3053CBED3B6E896B1C3356DD9E8FCB69691AAF6BEBA0424D62B9F266D560B40 +89E20A9F35AD4A7F65A7183761ABBEE0F7EF26ED3565A25516A3A5A57143979A +1EEED8CD52A69E4E1AE4795F3EE0CA21B9F4D166783A9AF54B08B6C56703932A +310197E328C6E6A6BADAA11D66A952964F31FD690082FA02D1E2A4512A9F2742 +33B203FB770A3A1C7490ED630D0FAEB3066429D0237E18818EC368D0DA72EEA2 +557F8D2FE98B65685700673A9F840C8578B6D7B384C4A0B2A77C61096081F6EA +8AE970FB31850DAC38AE12D2E458850E6C6CF9E5CAA86DBEF5298290123A8B67 +E6AE0B5105EA2AF7D1E5EC95BAED8CAA8399708D4C043017B3ED93F64B37B48C +6DAFB92FC1F21A4EC7894E2A0D724A524696A0FAE044D3C8ED6C393BADB09536 +AE22A4FF834DFA815E1F3765392D2DB2F4AB03FD01A2656D715EBA1C3063499F +2468B192BA1CC682212B97EB81FD61322F1FB69E5C5D1EA498A138CD5AED49DA +28411430C42A3D1366F57F0124EE7C8370B7A537DE55B277110EC87721D864EF +9C47AFF828B76F90CCF6758EEA4132F47399C1D8FF6795F21C209B25AF3BDEA9 +B7239556A29427EC5F15889792AF24609C504CF2133506A5A20E99C49D3BACED +D5CB3A6304B44D1E9607E65DDBFA55BEADDF7432514D98A43377F80AC279BCB0 +8EE0859DA2AF3DC8F13CBD9D331F2084410710561508E3DBEBF67284C10100D4 +CDABA196D49CB1B0730BCBCB20E2D1F33D1B6DCDD2A7AF522CCF150066F7AD41 +321740DCC42EC89716C017789E6D18BDEF42491549B10D597A2A62B22493D71D +526613EE460EE27BB0E255E79C151917228441F009BE79BCF88B7203601EAEA0 +8D515EBC23BF1B8B130041567E5657CCD7F2CBAEFAEF99FEEB2313B56BF0D8A0 +70762D4BCB250896C7FD77FF97F9849A0B1E52A3122EA400A60633895B3C4659 +106A2CD1B19CD91830688F8E2C75D7460CE1F8554FDDD9B654176C32D7151D54 +3DB35F53A1630B3BF1B4AE4DA3A13D1B7A3BFDAD28CCFACA9C9189E13DDBDD8E +ED04970E642208CF806F25041575F8F0DAD3E6E077E0A12D2183496F1E0F0F60 +23C5B422A83F133AD7CDA3A283E762F2A1B159F6D9ADE54E88654E1284C979F3 +7E12A669C8517EDD7389C32D8598046EF6B0A977C6695FF1A58F18DA66B316CF +42A175E8C3DF78C5EA1175DFBC9386073E7084524EFA025D40A5A90DF2DFFDAC +BB68B39D57170DFE2BE2766EDADAB8675E50B667A3203BFB3B55FF94BF09BDB3 +58468B10948FC0000C9B78A3F02636D923C95B29AEF20B78B686E0648CADA2CD +FA951A90AA +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: CMR6 +%!PS-AdobeFont-1.0: CMR6 003.002 +%%Title: CMR6 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMR6. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMR6 known{/CMR6 findfont dup/UniqueID known{dup +/UniqueID get 5000789 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMR6 def +/FontBBox {-20 -250 1193 750 }readonly def +/UniqueID 5000789 def +/PaintType 0 def +/FontInfo 9 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMR6.) readonly def +/FullName (CMR6) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle 0 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 12 /fi put +dup 48 /zero put +dup 49 /one put +dup 50 /two put +dup 52 /four put +dup 53 /five put +dup 54 /six put +dup 55 /seven put +dup 56 /eight put +dup 100 /d put +dup 101 /e put +dup 102 /f put +dup 105 /i put +dup 108 /l put +dup 110 /n put +dup 111 /o put +dup 112 /p put +dup 114 /r put +dup 115 /s put +dup 116 /t put +dup 117 /u put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CE3DD325E55798292D7BD972BD75FA +0E079529AF9C82DF72F64195C9C210DCE34528F540DA1FFD7BEBB9B40787BA93 +51BBFB7CFC5F9152D1E5BB0AD8D016C6CFA4EB41B3C51D091C2D5440E67CFD71 +7C56816B03B901BF4A25A07175380E50A213F877C44778B3C5AADBCC86D6E551 +E6AF364B0BFCAAD22D8D558C5C81A7D425A1629DD5182206742D1D082A12F078 +0FD4F5F6D3129FCFFF1F4A912B0A7DEC8D33A57B5AE0328EF9D57ADDAC543273 +C01924195A181D03F5054A93B71E5065F8D92FE23794D2DAE339BA29C1C6F656 +1DEF13780383DAE38A868377CC7D396B2A05F341AEE0F8BD0A0191F51AD11A4D +2E927B848A1EF2BA15CFBE57A51E3AF07598275195C9613041F71C1AF39E61F9 +EFD5F6512FBDA76E29DE6B508F62F5CF9F73F5288DF1C7B0B82C92D3B6358BAD +EC3CA20BDE55DAA7CC58004AA86B6CBF8C410D8287E88BF20588A39309C2B703 +CED322F030AA6069869064486CA651DA054FF3F5E56534CA358B0829A6B954D8 +9103436E6B06DAD1658BD4A95AB41343B01F5866FC87C4EDFC71F1477C98F8E1 +1DCF27EF743FF90BF918AB8C4E5AC35841E2F745480E5EDE1C1DEAFAD8D0018D +2C1F1CFCAD9F6609859DEDFD1648A6CD23D8ABB80747F94899F17C8F3E6CA55A +E176F19CDFDAA0D7C920B3A72051A4745560AC81978C92459EEE5AFE85AB247A +32981139CBE352B248F4BE5F73503A084A3E91F05328EE521D9669E44E202584 +5407E7846F9FEE3D54EA18FFB144BF2D6803BF65AE402034B3CDBB40DD24217A +3CE0E95E2717CACD603A958187C42B3558AA38D6B6390EEEDD396F96E6041FCF +6F8888221AFA87EAD79F46E0E32CAED91E6C8500879AB6E580C581E8C8CE9B68 +2BB5EFE2604E8DCB2432D39D75EE556969F1B2FBDF6A4BC72D106AA7CF22C268 +464027898B311613E06E1584707F262F71D9F49D2149306A88E02BC60BBD6BDB +EF41D90F19197BA9AEF32B5E63D5B9FF41B5602F9F786E76621DA54D574981AB +87A72081EA05D6C6BA940EFEBD0904EA4E77BBCE17E20B42E1722617E0F6EF32 +F1ACDE9D758594E9C81049CCC10605A27C2A06872FBA9F159CB155609B496ADA +4886F478E44029B5E620DE8319E257697E93E1CDFD27D560E2E4D34507020E2C +D9FF06BFA14E056D81DF701FAC3ACE4BE6C098AE116E079F0044391EC1661F6E +7A93B9320BD7F91E8FD2E8EB3F5CAE997D5CDD35107A1D35302260D1499B8B65 +39625B7925F97D917B66BAFEEA992873F07220714F192839948CEA080BDB9A03 +77B9DD032273DDB5629CB28B5D8797EDEFDBC601823E038384C90C79012A7D96 +8F27784DA15BACE21501C26E3AFA5DCCE81B52B0ABAF71A35D33103EA86F2415 +A39A830D559C5C6CA7423945BD3DFA942B20A06D7A8D8671F9831DBB52907AB4 +4E54776D29C6085CD9970B6DD21DD3EA8EB09C49CBEC6CDCEEB0BBB1B8827109 +3BDE64DDA024D67F098D6C1998506DDFF7907ABAADA1C39C759C850E0C6F8E89 +A392D1C9329ACFFA92D361218D75E115F70A47C53B73B356D703E9C499AAD098 +AA9C8119EE9E9708A9EA3049E976FA19AD04210D5F6092C7903FD155113F3A3F +269B746560F70970AC9F8D09956E0E84DACE4112C4E7C7F6B3F0B63D26EFF95E +2B2E9699D16BC8AFC4AD9113AA3A974C9E82E877288CF71E9169D2DCC61AAAA6 +C536E5604EF0716F6487292BBB677518504B52C63822BED3BD5FD14EB41EE6A8 +AD4B6CF90D39F98E12A765B645CBA3E8552FB9A986390212CE119E7C3DD675AC +17BD006144BEC534DA2A860188619F17589008409C5A309CB83FBA70F6446B6E +2B56991B6A03B1DE10C621591CEE45BECA27C54BC8B4F1754A9E8F660812710E +117850E1BB6FD89BB13F8CE391C43DA89EA67E9C3E7A4697790EA26B0E4E2E80 +DCA508873A7AFFC11B8C02EF86C2316E8D8B6BCEA37F81A3A87546705F070C3B +9D4D28C366CEBC1EE485B8E2357DBE46E86C87B9939DADA60888AA9F1B92FECC +CC1C198DDB594BB70A8FE690ECAC21A414BAC89BF019F34D2A130F485EAE35B7 +2A10C67EA3A48A4D9734759CC93AD85C6A570500AE5AC9973FC76EFA06BF5DDC +26E20E28D16B50957EE01AF2653F8D860817967AA5A9BF9BF7ABCDA710E9F34F +4F0EBCC32B3C9C2971F6225D2DAA6A451366B83F32B2ACB83E746D365B2DE38D +C1AB7447FE7B37F9630E410E5D8F0ECE74DF46C538947B3A167AD9F3E4A7EB3D +60F5425AE75AC3A27D39311DA35696C3DC7282AF1532E7AECE63D13DDA0296A2 +01487185FDF1875AEF55A36C17D6A8DD329279D229259463A2F05CB7A874374B +E2320E1F6CFECB9C1CE62FF468C29751ACD9754AF1EABE8E7696C2888914416E +235B6766F20FFBEFF285277B639A51EA2F2E30D207BC891B00F0436008F980E0 +9EEE7FB375BB069B9E0BA11DA951A99D8E60B4F920A0495C247FA7DE904765AB +DB5C3B2D634757E43EDD6FAA4DB3C67F82D6853E1170F0B2D8CE496DD4E72B0D +28277BEF172F1402959F64527F9B640619F04416DDB9D05FB2ACD019CB9C119E +E544D24EA6DAC5C69785394EA50E6EC9AAA9E14B904EAF29A733C6D7942B63F4 +85729686742F26DEF78DF0DA1CA7CEFBB684F4CAD99021A3B3D1FE03B9C5A4B1 +BD04CAC89BB91B11952A2B17A61789BEE0C54B46C03FE9A1AE73D17CF94BA30A +237C29D414C3BCE8E3E2DDF83C0BD59DCB66C4D2C3DE73DA8378F3C6C8035D28 +7464399857E57651A53E9C4AA68DFCA91B2376CF98AC5290FDB9BDAD9EF1604E +9B0A70EDDA1E564B6D2456E7BC722454ACA8C4950FDD44B6EB9AD01169A9F845 +B06A0DDB7897C847A5B1F42258AECF3807AE936C8F52C3A7A0A85D68160AE442 +FE81543DA6702D76AB6E8701F80DFC1D87C961E350D0E52AB2A298B9E5908600 +7E14D2A87309043CBF13F69AEAAB1BC239DEA88EB5176624F6046664B1D2691F +FBB2071D3706F97DCCA355A6DCC4D09FD35DC078FBAAF672FFDECEC61050A120 +10B5A96629041303FD01ACCC7686165DED6AA712FF8E5E85DE33C4E7D877C49C +6C469A90410BAF60BE65ECD91CDC2EE7AC0CA8BA7B53865F26092BFCAA0BCA77 +B80DC51DAD09C93C8DD8E43502B4B68F3D5918C3492196292447732BA90F5AB4 +9F5E1D634ADE1CCAAD028DE5EBA9535F6FC5908DBD2D643E0A7E059C8C386FDC +E72659C0033F535C0D7F6B98D0335552D0BF3C6E302B672A5EAADFCEF81912E3 +8F54E6FB7EC2B325125159713D0AC50DEE3673B9B148643727E94C80971A2E73 +5E1E13237BE69C84FC039DCE02ECE2668AFD047F21A61BB94A9F498C9FE5CDEA +B274B40728B6F6CA9B6C15BAAF92F465B0D7311B46545CBA90D874839443CCB9 +3110F052EB247B24B45A3D2FA6FBC7EB2A4BEC2A5892914B3C5EA3F4F9B9DCBF +6F932D95700E045B49E4B1F2C9D2A42CF39CA2F5A2654E6E8E6E92473D28AACD +5E35C6705EA728F704F5996D286BED433F976AB7E018621A577AED7C0AC0A84E +A032FE1869F603E6F20386E3A190A30A21EA886249ECF8CDDE2C33D73BA8647A +3DCA7A8DD9E8EC8D9A415D126BA38B6771C489DFC419303EE9C1B83FBFB3A0B8 +97D64F30E4BCBEC24DF603FF3BF541E00D5804B6B6543D3D2B661CC551D497A9 +9DFFF535AF424B2F3150BB39AAE8CDB306AAD37767BA10BADB031DC2FAB16955 +EE78342CCC0E8B5976BF98F215461A8C6F63EBE6E2F1A1104662DDE53388CB51 +8B44F3534853B8095F3B746A2459C2EF800FB1EF7F235EBAA9731E3AB3BE4369 +1D3636E3ADD5BDF0C34FA80E90D8A1DDE770943FD196E0A7C5F1FAF6970B34C6 +4673AACA6B2B5C12B9608521AE736C1F4B97209B063D991300ED5AF3D7F27E76 +68E0B858FD8BFF86581E2B9548C691E3E5D9EC4D39C9715CDE86C7D22223CCEB +8A38C776A30AF14912390A7546DBECECD7A687D4F08646E57A12C80DCA022B7A +33399761A50B8E0ABEFA1163EDEC3DFB5DA3248792EEEDD894872D4E6814B4C3 +548BAFCDE0CABBCDB97EC6D1BA47F2E77CC1389BF19D73661749AC33F46A618E +A665A85776545BF9662F2179D7BFD604FA8EF4700591AF3AEC647E27B24B76F3 +133F9198DC15C1AED830E737909E43EB91C334C44BA35810007A3888E33F5DA5 +B3B2C35481C648AFE630CC3E08F77744E401B2934E407D1EC17ECE737606B076 +F8DE8EF3344F57495EF49D11580D6FB28AE0B1422521B320843B13467501CAE2 +3DB93D7BB779F73B6AA30050DA74BDBC3F8DBB30F32EAFD07734A151BB2BAED5 +C9B1F790059339B64BB4146470F30928C9A49AE88906BD6FDB7431A4B50809CE +0F67ABA01CDCC2320B0B097187B9299E3D80CDD7BB5DD5BFA7B28D924C5633C5 +45A23CCEE097C41C3759C1FA8DBA0DD95034BCA89BD23FAC18C70093F40FF2F8 +0FAC5DD4835F2DFD40540E9A9E9FD951A8AF2CB766597DE00147B163BACFB7E6 +EFDA4DED594F1C746D8B46A1145E0E4058F5917B3F21E9BEBDE745EE72CDCA64 +FB31EF7A2E55265F32559480E2B6726D3DE26FFC97EB4E3160F117E890C4B2E5 +8DF310E6A728ABA85540F571C024F8DD58E1D7827FE97CED5EB31547EBC36415 +02B8C0E10B7E37D816F01D56A364B8552CBFAAA95BC4BDDCFDE91CE0EF005B4B +7AB56FFB47A093AEDF0DE1EA48FC8103CA3CA1470864D2693E360006D05668A8 +AA422CCCED20DCEEBEA5CE0DA1EFB00FB93E922B18124FA11A88D0F6E0F719DA +57603DD5DA42E1C56C2FD9E5415AA199D4F334C151C1157E75C107FBBFCEB706 +5F4EA47A29B54ED8CAEB8DDA2F53D2A703B95487619780A52DA1270011648A28 +AA64338E04AA5B92C1EDF3D8DA34FA6D227A0325EA6F22E9B38B6338C657BB21 +CD4C582DC04010330F62923F817E4EDC6E5C0E6500F2A975A8A95BAA30C4A134 +BB31B5AC45A2E7F6E9CDFC810D41344C4F606049445F8E93D74271C1E29DF7CB +5459593BA28AECF64D903D3E4D77CF5C04B06DE44A41EE4D9FC769854503AC85 +69E4A5106E84016DE3D59865D4AB30BD6C9E45C45DCB5408421CC50CD6179C85 +34E55CC70FBD8FEFE9F1D5160664981716E3BC7F24B6F54E0323D9BC4B692971 +24419EE62D8B0BA726E2B4294A9A76F328B8101DA29E78BD5C4AC383350FE196 +4D42DB1653637D19530124858950C22F1E9CF5BC07D46B7A58CDE19CC88DCD2E +7FE4EEFD8AA6047E919823C8CAB2EF5274F45E861E6508CC11A8AA90AED2403A +B2BF1315C2157B3B50A3685205D93E40906EEE9DE5985405974BCE0B84BB37DB +080A45C5237B269B93C0A7CF294A18B45464A41F604C494CBEF829A381155CFD +71CEEA54CC39EEDB6DF58A9896246B09F95DC6BC40BA6916AAB5ED3D24F66154 +3662F8978FC63DA9280FF7ADB09EA5BA79D3B66E0C88BEC1EDD78DA93839073A +A4D7B0E627000C4ABA76C47CCFEE92E319315333A5584A951E34C55412049C4A +A5569FE65A006F77B416E0530AB6A8E7AD6C72340AD4CE25937158FABB2153EA +281E1D840206F5DA38E00815E9081F81DAB9FAA8F4DAB305867AC84735DB4F52 +A36129929BD2084A8EA37BB6889695204BF7290B68D5E722540BF8A276F8BB6D +451D582EE59D2FF03F6B97DDE05FA00C3D375D2D0AAC8FE298F85CC067B15481 +48D70B6A0354C705715B891915FE8EA45244677B9FCE81E72D66177E309F3F83 +F744B9EA9E55C3B30DEC6E5E03B3988FD526A82A5E8E1DC79127FC62B2FA7949 +B3AD3148868DE22BD4B5708E32CEAAE6ADEED1F463EAB9692411E18F8D6BF391 +126B2700B4CF3B59D02E3F8795130C96285A63FCD1E0F647ACB1D35E9C58BD01 +1DD06BABA00CA4343BEBEDBE677E053E9732B33A7495DF51782A07DA07F5646C +770C957AD915CC70BA8E08BE7A1F4E6BA5BB9C603E38F6FB0A2578471C4D02F4 +283069856D926B9076EC73AA39CEB0A061AFF1575C7093FDAC9F89C3DC06EA45 +06F3C2A3BC9FF21128B10CB758DF0F099B459A5264A8C24C098110D2BA1A8532 +8FAE146A91BA7D033F591AB1A94B8A6FE0FFB610F698D216D58B4EF6C87B1524 +8037CBB7E23D8550A620341C6625A1A2ECE7CEE2598D66277F857231A36155E3 +984F147783E9B93975AC38A29F2FBCF704C8A04AD84C3E04A12D2321FA56811A +5B6744813CCC187968C5C26BB8D3E6615A912FA5369C01CCF8C0DB790593B190 +1A90CFB5339B8771F325C5FC448D36C7312B11A15A8635BAB59CF3CAD176131E +026F6E141B2619EF7F3048750CC9291397F141591EEC8B612D6656DD34DB54D6 +DBDD303CED74BE76664E7DC86FCFEEF2001C9DBA56418FB61F589566A47AF36E +C94671C5E8939AF9F4D53C0DE7142B7B63C86AAFA65877EBBB48C64589AFB2CB +1280AC099FC48058855CBDEB6C2D2A0D092267996591DC3B5EC8252984E9B27D +2E9EDE8CD8303F0905DBFCAE497DE1B755B924452CDE11CF4F20893DD6FF7251 +427F520FE00580DAF1703FD968E0F8ECCDE618E1EA5820EE6CFED97C78864EF6 +26FAFEEE194A268F24249D44829AA360D731C34DC285501E966A959180718F72 +6330E4CC060588A2F65AE64A720DCAA818D49D4440F5D0B6C1F6C3A107E12445 +F1BED2D3FCBB87A9597F01C7332AA79143564056219BF87D4B907A04F77621AD +054935E883B2B137D3D1C4BC792E8335CA08B6D83227F35736C41312A0BB077A +60FC6488C5E02FD51A10AC113D4EF70038C649C1677B2204A77F2ECBE9B3C341 +F4126BECBCA61E3F3801F9188A3775924A62D30FB096B440286FA655EBA00A74 +9A4162904BEA07CE68EE76018346DEEE20839C9A2FF71179B58E1D4AB30856B5 +F5D97295A097174467010B15D733AAC5813CAA633746B430B1AAF9F997FDAAFD +436844D1A56B8E25A89D2CC4BA6EE7ABD167818FD4F6C747E07B262C99EE2C35 +323F0B471586CA50F54C6381B052B15B0C58C19DEA82C0CA29F00400B727419B +2379979CDCBFA966AD513FA903160C571C3BF1BA239540B11EF2371A3880837C +6D6CA2F374280CFA1586427AE975A2AEC34244874E4D441DBAC6CD1828841C91 +069AA87FAE849C5DC7C9EC1B9876E59F3CCDF8BB23D939F5348D7486934BFB02 +CC5A22541ED352616830A510DE7732E5D8F7E785BBD31C2BC9D348CE5632654D +2C1740F89D57FB2AA1FD8FA3304EA03F757BB8F498ED98E48485722E78D97B12 +A05F3A28438084D1CF90AC4C3FFCD7B3365941C45E1E02CB13CA1E99F7FA1D00 +1C9D489D5C95F019AB4CE89FA3B6604473DBD2CE8E278969E0A0FCBCE68C23F6 +9381882443D3FC16966555FC222F3FC4B1207522201AB7A15A7A6F22CDC9D392 +360BF4C95DAD35770E0AC7E5EFF015F2C74ED7391F40EC94B8D1C163B5DEE5B3 +911A20C2625AD3B24BD94D2A42405E655DA47D3F94F882CA2F479437B4E0BE71 +8AFA4482C6FB270F8D05B4599A01403DAAA90C01DF3AA7C2BC7E66AB6AA833AD +FB6E5EE13E45CC7CE7E200FBFE639F9CFFF5D08512C02764997FD28368969BFB +0876F236EF6189BE73AD827332DF1B2EADEAC0ED3B939CE5BC3CEC78975FC636 +44FCBC2CCF4396AC7343EC62E0E4F3DFFA2B880BF31D93ADFE201BE9CCEC8BA5 +0B9B919E05B851E0909968DA259EECC6AA0743F25247978CC09C28C4F878E29A +5070E4023BCE95FE0ACCCC01D0EE219FA8344E8F6D7D4347563BF8AC030B9097 +41F24D4BC9494915A82EE9FD37FBB6A46BF077B728FB569B1258CEA5F51F36BE +4F4D0F890D782E44748CA3FE8C8A515998371D9C7D2311F192B4B7E7C68FC6EE +3F7136714C282A2570FE591F247A08319CE9EF1E43274E4E57166E31A2ECA506 +85350DA31AA4C33C9687F5210BA225EA1007C444FBFA2126769767E47A967884 +9F68589E4BAA9ED32A7A466DE35554C132810C68ABDAE536D9D884352F28EA02 +8A555D2CE11F30598F44A65E2D86B43ECCBDEED9E4E5B5B7DCDA20EAA09D9FF7 +422FC91F2201431A9E8FC624FF44D26C0100183D77BC7E6B1A6CFBD3FA8BABC1 +AE4CB0FD382E26BE0A83169B46D91429DCB746A0326243E212F802AF6A56C709 +6E70C6C7CA25AD93C7F6CE71B508B6547004DF3447B141D6254767C770CEF529 +89D03360B92ABD1B74D75EA95BDB46FE8BCE14FFFB9B7016E5CBA23CCBA7C46C +A6F92EC008BC670F6B18154405B951D735F59C9AE321102DCB573698FBEFE327 +2E2CD907A58CD8232BB5FBA0CD9C6D4267D408148B6B817ABDCBC6F64ABE44D9 +4AFB32ECEA23D99D9D8CCD06E115EEBD7A468C097CACBE77A4B2A348BA3E8050 +17EB68356D7C964763E40CFE469280BAC02E3315525097338AB3E8D874321CCE +DED4119155AE0E9AF1A15674AA99C752581DF75C5EA6C8B69BF5F30C35E52BF1 +4AF39554938D4C03BCA8FD4F998629D71F7703919C7B844EFB6DE186DCE1D131 +A4A4B979F7EB7AD85F0AEC7138770E0374ABF80D0435C97ABDDB70D618884348 +E44AA72FC4BEC49903CB1843808E6772F727AE310C2C7E37993E5F7B59943368 +A9554B60AAC5C356F8B02C4C0BF30DE46AEE9922C12C9443B357A5D34E72D245 +E47209EED75E751A04D6F145F13CFC991004986BABCF578D73539C4F3E58DB4B +E428B8F2E4CEC34473E47F3E85A085CB182356BE51DF4DFCC740E92682E7ADC3 +4E7F5E78BFFC2A1EAF01E57AE64F6226902C42CAD3E729734FEF6B423A98E5E5 +E7C5E713FFC574588DEC1E18FF3EA1B45728E3B05A6487BC255DEF0A5E34D6C2 +009F64E4D5E55E28F73CED4640D10CD7192C5279D4F52EAC80CEF024CB2ED2D2 +A86A32F1666C964A496BF9443F29C71F857D30861AB4081B6A3E2ACDE8FC41E4 +C972958B3753B861B6F00B26BA3E1573987089E4F61A290CD342420B486BF7E4 +FAAFF2DFC50DF620801E76266B15CC8303BEF2B3FB794F52728FC692522EFDAD +220536396A1C1F43DBA904E7CF4A94DAA44589CFE1F7E00C5FE0A00AB978E588 +AD81C95F82F6F7E877E8B25CE3ED2E9BA44452FD9822F20B40CB444C51109E80 +E352BC7DC5982E425F1764AC856638CB4C3CA17B49465BBD2D8A39EF65E08C19 +B4FF384A8BC7AB11278AE307B11829283B2461C47D89279CCDEF00E9C2612107 +FEE9B4BB39726EC682E8E9E82C680206FAE5657C67E020ED2D3BE40ECDB07BFC +266CCB3AD952CF14D6E9EA32842202D8C19CEBCD1841F715B9EA8187D30E5D23 +76124545C0ED3928946B9BD45DFBF694C9E14A127A017245B0B2C93A284B931E +9638422C0CCA9098E2E66F24C537B3017DB3397179BE0E3EC7E5C2E31503BBC4 +A51573319D87184061BB5D7D11C75A25F0EC765134F025701D8806EEB8B9F5C0 +0CD1E96EC3DA15F8FE540DAC31A008AD232A80C6F310F5F4CB04803F2A2698D7 +E344F00FB600DC4C12AC002FE98F6BF7525DD7E26D922FF35DC6F74F9CEBECF1 +E82E2505AC630B3ED3D041B1DEFA4710DA077D597065297FC0DEDC5B3237DE95 +46D28BA91655D9C00E38CB99BD165052733FB56D9BDE5679B691D7BBACEC7493 +56BD1AD132EEEB82DEEE7595EA7054E1C7282F1515A0EE1FC2BBEC9892A98902 +E380016DA28EBE48AB728D6CACDD539CA6EAC7CE70E23DBA2A35F97C9D719895 +5718607D775887B3D05EC75FDEB881EF521213FD5E43CB4181984E0CBED2E541 +7A9E5A3D8316A2086CACEAD96560D7D46ABBB927DD7DDC7808B8F13FF8D8BB3D +16E5417147EB4AD7D4CE6406473385AF2D8CA8C43EBB58C5AB203C2698F089F0 +1F42BB30A58F682EE78AC6296B8D1D24E018961BF3087BCA7086E2E277472AE9 +C70F975A687BB1132D113195745908492E03D27D4F9EBA93F6A911950003FC49 +2D26FBA3782B0458A0DF48DA395FFBC13E67DCD4BD3F8F6386825C34678C3901 +FA324030204986461D648E9B4FD9E0E07696962EED3AEBF4AC2105C1C4E05701 +7356BDC0522BEAB66E4537E28E2B99957CDE8DACDD0EBECC5C6694FDC53D3B46 +41D69B65C9A613188BA4A973BB7D1355078992CE41312CC0E0091B09BCD1DCBF +59ABFC7DC2E2BE3CE38ACCE36F047BCC5A645C46E5102868F84759247CF93AB8 +0FB035B2155C7D682B5CC8C65BF2301FA57541FD774E9941C942A3DE394EAC63 +AD90AB1A64020CBC4C46CE2A78EFDE67A318962BE51E47BFE00A1FA1A84F901D +19DB500CF6614CAE7D67F40DD9404DC10406E0938129116369C4B916C3D7D637 +566BC0B321A96BD70C699FCB8FB3E1D6117F68A15F3DE80F72CA6C2C5BF0CEEE +FB796EEF7DFC1207200716442AF662563A659A11A07A852B8AB1BD57EC1C0497 +DC03FF79DC88513D12BA9DAFDE4DD75510A0C2E9BD178FDD91F668F50375EF9B +15FB4B8AF7A26CE3D3B52747F358ACBF341233D7A04AFD3871C84434014C5D2B +879C7D1507ED06B90F54DA4CB12AE8874BE90CA301D0428D4D957557599CB133 +7B6FDDF411076FA5B58D890E3AB03D3A8C48E0ECE94D178BB78642910642507F +2B98CB2C06DF248D103A4CB80D6DB49D7D5E0499894CCD9C27877CB1C3ADC78D +3C79C0AD24B5DC95E46A50E424811BA32003B3651016FB11DAF5796D6AB30DC1 +4C701570D9563168F0DC34EC30A615A7A111C4BDAD5803C69C849AEFC46DF37C +8DF3ED6B625E8763E90E222AD6377F3521BF31B962E04BD707EC201ADB535028 +101561664DDBAA4C5BBC343F7205B66B39D84082EBF131A638180768079C94D8 +BCDDED7143EA5428F7603145023839BB1F2AA1362353253A7F242952F544C760 +C1163AF07033F68446CF75B0410510BFD8726C3A1F6C4D328B3F1BBE17D63F1C +F8966E833F5C351DE0B7343D677C3CA252DB7C6B87646FC49AA3300ADE2DF60C +74E698FCF8CFB40A48FC90D21740B3FD70F245BFD2CDC8F0965C78F720B3AE8F +05710A7AA0CBD7034685B702F33F678DC997EA96434FD0173A9FB8AC8219E86C +E6B855615229E12F11848A61BDC3F7357C8F45F61A4667EE3AA977C3DE5719FD +E4B8E67981D0F1C67B460462940D646662492F5FD832FD290AAEE6FC6DF82784 +E71AB7CBD81CC384DA49A0A36797680C17D1E569CB5BAD8797B08E870F0DEDD9 +9B369AA4F570E48FD4CC01A3E2BA16E9907639F4FD96EC413424578A7E7C4B21 +34B193A53CB10BC10B350CA466C77F107E2828CA3755BC7BDC585573F22B20BA +C36742F0212B34F37469FB432728A0089905F8ED3B9ED20850EC2E6B3A80A3C1 +8EC4503D3F957D513089D11C3718F33C652DBBC91A52E015432C94AA8EE7ADC9 +65A421A19F8316435E270D1F101A3B4D4E8DB227EA8162E6B13C78FF048D9D64 +F57F9521025754F7DABEFD7033FA0CAE1F8E36AB61764C08944F6B71B3F01035 +4BA6DF76AED61B3CFA402CF122DBC6438DF66253A66DE11A3F2872DEAF78E088 +5884300A67DAAC3D444E1141960AE897C8A74579945A469BC2CDF7CCC854ED94 +6EEA71EC9C9480DE5B450BBFFA419445FD4FDA7C77716CCE03B0878ED7693C5E +114465B303563E8C16AC18FE76A0244CC94C599ECC8FC3CAD213C65463E350C0 +2A47685B9E9F70ECC54447D0DCAE36507226A9552CC9BDA411A86C30CB7ACBBD +647CBC11E613B132F0308174B3205E25F34401C5FD98B3E9FF2DE7111A1BAB76 +615F9A285798F627752FA3D4D810ADFD41BA9C93E47BBE32691E3868D4250A18 +470AF268003E358A599BE413BC17D44E1C96B4915DE66ACE64A3B6A82A039FA7 +6F2F4DFCF630BE5445359E37B83E6487831B2194097F63DC0B25E919FE0B41E9 +023BC85954D99B77F7AD0531E3F823EF84E69C230F4EF9F529967FD90BFE31DE +CD96406BFB1B72AA7FFC55FA0095A98C0A6A7FA0CB5776EBF236B4D6F4854ECD +9340E9A8CB5FCC3F4E1EB535E5622742C95FF74611BB29B24ECEFB0928E31FC9 +A45EBD7879ACDC6C319AC853B763D9D2FF1C5887827EAB54997E44D8C8B80395 +167508774ACEADC909E360DD2F7F26E0DC5A7B6DDC477F68DF01BF33E32BB420 +0A80BD39B589F0B0512DD27B7B6BBCEC0B708DCBA70E7E1FF5007039BCA8C43E +2AC77490F0EAFCEC45D92A4E37C6DF015AEF249E31EC15EDBB89A7621E1CB744 +6C35EDB73B60D0E5EB60AF4FDCF67B2944603F47BEFF1EBB987919B099E5CC2C +2955343C550C1F5C837F9DC48C81ED0D18381748D9DB0312C591B514BC644009 +CEDF1BFC6B912D08BF3F963ECA67392633C7DBE6381F6013E3A2E015BF00797F +82019CF3418FBF0AA6F2EB51C2C3CA8FFB2DA9C5E2069914A38FA65F3EFB9444 +FF4D9064BA555BC42B72A1EE46590194D513B143381B164970ED5E066462CA7E +2559E3AF6FB3400BD4E819DE6A877DE6AEA9C3175D69377CC72370A1FAB235D0 +C86BA6B96289314CF67501C18BFE7175700E9D718C19B7FE001F6360845749A6 +4E35B14D4A1DAF924571B59B0A7A8ECA51FAB539141AEE4FB4163E0B44BCFADD +BA775A6D055144B805E29485B76512C67466172EA0E1396B36A9CFC1907071D1 +DF1DEB14EFBC1C86B13E889E41A0A594645A6CE427D08B75EB838B19AC0F26B5 +E68164148CB4179409592E368C91BFDA3CA07C19B134AC24CF4E2212BF5BCEFE +48BF1E481E813E5712A9CA4B2A098441C915FBDB9D684B90968CA0E3D4F3E706 +14075B07217DA3A1B6EFED7C42F00EFB86CEE507E24EED1E2D7915B6945EC456 +6BC676FA735CCB092FB57A40AAEEDEF343A541989FC409EFCD0FB5CC4D0692CB +13E430A124123582CACF2D40009F4A7B911B96C587274318A372EBF959DC0545 +CAB5982AD57F8BC22A9FC9E3C34EE6F2812E7D5AF8CE035637034DC323EEE718 +22FFF8B9E91F92BB9D6833D3D5137A1461069E52BF474B44ECD2E57C339B9DD5 +2C91901EC13E87A88B88BEDC8DAAF9F35CAD20530BC50457B9972E2224A264F4 +0F42F6A7288D3A129C0BB7BD830709F3155DF98895E1C12DFC4505121FA5257E +51CAA7DA5B461294597E507A2DAD2828BB1D172297605ACDD81611B5B5029A61 +DFB97D0E073B83DE7447C7701B085D9612537F336E +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: CMMI9 +%!PS-AdobeFont-1.0: CMMI9 003.002 +%%Title: CMMI9 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMMI9. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMMI9 known{/CMMI9 findfont dup/UniqueID known{dup +/UniqueID get 5087384 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMMI9 def +/FontBBox {-29 -250 1075 750 }readonly def +/UniqueID 5087384 def +/PaintType 0 def +/FontInfo 10 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMMI9.) readonly def +/FullName (CMMI9) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle -14.04 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +/ascent 750 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 42 /arrowrighttophalf put +dup 58 /period put +dup 59 /comma put +dup 60 /less put +dup 61 /slash put +dup 62 /greater put +dup 69 /E put +dup 70 /F put +dup 72 /H put +dup 76 /L put +dup 82 /R put +dup 83 /S put +dup 84 /T put +dup 86 /V put +dup 105 /i put +dup 108 /l put +dup 111 /o put +dup 112 /p put +dup 115 /s put +dup 117 /u put +dup 118 /v put +dup 119 /w put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CE3C05EF98F858322DCEA45E0874C5 +45D25FE192539D9CDA4BAA46D9C431465E6ABF4E4271F89EDED7F37BE4B31FB4 +7934F62D1F46E8671F6290D6FFF601D4937BF71C22D60FB800A15796421E3AA7 +72C500501D8B10C0093F6467C553250F7C27B2C3D893772614A846374A85BC4E +BEC0B0A89C4C161C3956ECE25274B962C854E535F418279FE26D8F83E38C5C89 +974E9A224B3CBEF90A9277AF10E0C7CAC8DC11C41DC18B814A7682E5F0248674 +11453BC81C443407AF41AF8A831A85A700CFC65E2181BCBFBD07FC5A8862A8DB +7E2B90C16137614CDAFB584A32E50C0935109679E31306B8BDD29F1756946A67 +7A7C2D9BA6FAB9B20A424AA0E6F4BA64C2801C2FB5A1156CBEED0ACB95F697B8 +BC2A6E6AA7EB1F9FD8E3C9B1A16697EE1F0E7400421A7765AB218FC837A49365 +82DC6B2C877A7DA84A81E6126EE96DB25C17A207D3020A045DCDAA064360DFFC +E3CD50E21ED239D2A6450D04F879A26443ADEB6A20ACC504989876476C7D1A74 +91564FEA1F4CC2C8C8FDF666DB537F315AE1886C73CB5B00E67E7B398A6C018E +540EAEE98BB8136C4F044EDD63C33431D2CF9740F051DF365A4045D9D8782112 +7BB5D494D9235BA98CF2F30CB119F5A904C32AD04C960C43FC1F5FD8DA7D90D8 +93AFB59F3FF4F796481AE2A7548F948FECFC6C127C4D3F159B08F206AE8C296D +EE470DB2F879EA79475E029D22D7A8535C09A18689DB0609CC233E5199C02756 +972CC9C94D9FCE264DEE5D75C8D651E4E2D1189AD9588CB815722BB5EE3C379A +6F31C2E6AE1AE4CCEB29766190AFA20EA937114978752189F1A9F42B39483149 +796FCFA123BA9CCD1D9BE28289660BCAE16C40B5B504058D55CFCBFB4F4E3D94 +DDBF39F157E63946534DA81C018B1C01B9F10DDB55E0A5C2B3985ED1977C039B +D6755EA42CD09E27751E159C30B93F376DBE61CD3AED34BA36A768F232EB3B80 +E3E6B77C4A48D408217818E398B83D995AB6BC871F20991DF57313D6EB0C793D +0F28088EBDB7F38DAF7E01AAB3476EC24D7BB38A9889A7D3038D930FF4289B83 +F54A7BE1E2D98A3822098D2E4D067A0D400C20C0B2B4BBD74C13ED1B827490F9 +ECF48F8C3994C1C5AAC9CF783BFA4F307528F51EAB55F961808A42ED53F00C97 +72A432EAEDCFCFB622389BDA707B6ACC9433B065CF29EBFE93AD14B8ECD5F47F +F073F11822C49B8BE924CDFA6348C3A75E9BB9BF3F31C41716B34794B28CDAC9 +4DB8B087E180A9B3B17680F73D9C12C8D86A922C948093629F5D7F542ED882A1 +692F4F6696865E53E3E2DD43B2D5E8C989CFAA5CA5C4C5999045E170BDE9921C +BACD6F2863F5553EAB2BA2D4A9034729EC0C4201DE90DA89B0A27C5A5C974109 +4E37BFB3F46B3A506169FB0C68E1CAFC844419A8D261A1FD86A3BB78E33D5FB1 +CFC687A5975987CE45155E5FDFAF0CC5FD5568CB1C26212F92E88255F0549F59 +41B33125946DE43436BEC00804063FBF03EC796E3361B1C852EC3038D107F80A +9198968265D5488B26D7670B22C2D75EDFFD1B7B4AAFA36DFD94640C9D0E2D20 +5BCA18683EFB91834A3939AB8EB60E2F09655BE003582634C52770DA9668C292 +2E02929D812EE2B0CC65F020064AD5BDAC5F5693B30508F40ED8E20E87149BD5 +8DD41AFF83FD1944804017DC5A04512E593549FFFAE501131CE2FDB65EFD0B8B +33809CBAEE411B3941C241550B9C30DD28088708F1C0CC3125CBEDCD985EAD28 +03313741F67DB5744A87B381147D5BA70AE1145C27F794854628D87D6C1ECCA1 +749E3465B950175D3C3F40E344297BD92D3190041A4392033A79BEAEAABB8DBE +CC14E39612F43721CFAE6F79074429221CA588AA2501DE520A464DE157A03AFE +3C082FAE7628FC0C57FFC61D0330AE6332D20FDBB09BF36848FE05E782D6379F +64F9C82C45402481B0A35989027F9756BF5A79DA2D96E10F39167ADB4305578F +90B509B6891338FA1D67DCFD61804AA6621526B2EE4769589A2646581712AC05 +DA6E98D16494F07D612743058F54FEE516BD89A8EC3E03F9D7F905175D3412C8 +F7329077FD6EB25213F3CAC94BA0C3363B759401B6EF7548C7D709F3241D030D +4EB46A1AE81863C412BDDAEA6084C37143A4C5E41BC646315B1CD09F934186CF +49D1D8239E363A435307030BD79536B50B723A39DD763DB539F24A10DDA12BD4 +E467339D2D6DB177D6FC539FA77D2DE4118EBAC161E928749F7C753ADEF86117 +58619F1155C563DF2E11ACA8347908B98113AED58FCD0394150EEC94B7F986EE +88BF7171D208D8F1774B1DD478F0C2958AE372D257E7EDF0F6B5D6059CC4D5D3 +B00FCBD2E9CBE79235B9A5A3E943CC27AABB58728C95C7DBD4F4A1F8A4DA99AE +7377B0CC0BFBD454794398AE0D5F7281771FFE87B25A819F36E692286A42D776 +01794A43CA9BB30FB8FFDAAF014F909A369E34C2F6C75B7D4EB9DB0580E33F46 +19654443AFF8384B95600B86FF8E41FEFD032355626D60C7507C058EF832DF41 +194B48A36F11082D1DCF4723E21401E0C7447AABFAB4639B26E3D2730E348F55 +53EBFF39CDD03E06E2FA5FB379603C879EDB7E1A10F89695C9C47DEEE52BE0A3 +F446F187AB9D7E93E6F9387F21129034F36DF40605D28FD526AF82CA9D232BE4 +412567F06B38ECCD496EF40A7B243E46C9FEBA4F1BF4B1ECA029C5EC239353D6 +C0B100BF7E7DB33BD1277DE104F15AA19F37340A777741AD1AD693BC76DA48CC +C6F83CD84591ECFEE375979972B0FAC4C10B625E4BFB261B9FFFA83C31DA0108 +4FFB6377466E9739E0EB64424BD9FC7239C7DD834EC6788A0F97FE714AF92831 +E1BA36A8A9E24739F1DC82DC26CC3CE28C210AA7C569B19E1784D663A0CA4E81 +AFF43E86D6F5F63778847700072CEB77A4EB946DC1F23DBC00BCE773203F76DF +00F0B085F31420672974DDC642D885E95BA6BBE43E1CA8ABF464D9881CDECC7A +E98E31B9754C9B72A8BD5CF6D4D214DBC3BA7A0CDF6635953F5AC1E7639C4A91 +C7AECE4C75CA3389C348F656FC2CC96C84C85A926237B6504DB51937C9CFCDAC +B75C31ED570D180757884E27757783DB2D5F35ECC48C496CDA342D49AA947BF8 +2FDAD2F19DFE8CD1C76A8FA08F33681F3E12E229D7DAB45BE3A3F258B5ED4980 +F15340CF20D965252843E026803E8AEE736EC41CCA82167401977AB719AA2F50 +0B791EEAA82027B3C712D2EB9D14BF8F94FBDE2227609BCAC41EC08DE2BAC023 +28352F913F7DF08D4E1C66E83F764578B22B4EB7191E852B91ADCCB1BCFDB1F4 +E63DFD152E86FA9DE9BC8908130EFDE29CC4401339C05B5B9764CF8EFF14951A +C6C13AF979546996BF22F2B96D3D585B90CD27DADEC78914DA48432C6ACBDD42 +20EF583FD41F2F6D6D10C3DF7DD077304B5940BB0462656E306CBD91EB9B756B +7014B1884A36201EC582FC9345C386043DD2818FC301EF78791C1D7854F8FACE +5DE9801DE9F59D5B4271E003AB897B2EF49501589D681D59CFFD9B03F722EEF4 +74ABD29997515DA3591496B62666744EA76DCA45504F8075C0652D6779DBEAE4 +90430C2945FBD60AD53B51DDBEFC7ED703C418B4B244C8FFA5A3C1B7600C5A55 +3EBDB93C16AC191C3A28EB2279BD3F0D67C826BC6A73D3C0AD02262368AB4621 +98A1605F2887BC5880E1AF2780330E0FD01D7CAACBB0F008A42C427F38236066 +54799594E515B289044BAC4DADF8B3686B4372C5110201221FDA923F131E07E7 +93C44BAD406838BA4D1C277EF74098B8C0EDC41EEDD58C195D7DFF5FEDBF96FC +19CEBC6C3006DD2CBF76916B4298BB915663C2F61AFD7747E03A03BD7280197A +9DA590E3D081C6F53DBF94E8D6FDDDD910A70AB18A0F6D48A590FFAB314D6CFD +E3FB20C1F3C91063F00726A2C13A3D48323F9854839405E5A29D66A43E6E2B84 +A8B3765F1D817071D4D6FF42BC785C2D11AB2B9452F141696CE19C6AFB9777DB +107D6E22D8CC6C26440BC48248AD8805C4329D46BF433741CB519B21663392DA +5DC7FC9BF37E5BC396BFADD7263D09F6B4D69594AB386B7BDFCF3BACB97A0E08 +22013E716E642592A20136CF9CFD61D4E515D80E06A4CB4FC9D9B916C93CEA95 +B83B98C48CF36C1D02291D4F5C0419338D64E33C90C90EDD2BA3B96D70FAFE0D +403A060CFF448D3E28A9B1E3916018465E86095BAAB4706CF7ED350D7C554789 +D7F4FE5F180767DE8739259E68CF142040BE1E2E8C6152DE3417C1FAEA7584B6 +20781DC4A9796431EE713DAC4E713C839D7A4FDC8AB6BFEFFE767AFD8B67FDA6 +943AD387E5D3BCB09039ADB64ECC2BE2620C6EC269E708DD06C311F450099E33 +AF46AEC644222E7DC4DBB9371EE12CFBC4F9B27AB46AD1DA96CE006E1DF8291F +A550A93026CBFFC1087B134EC6EA76F5E109CDA58FF47338A0039A786A575F70 +B8A03A4F9C8D07A4C856C77D9BCC8E3EAA740172D0C2D0A15BA35C9E5717D7FA +2691774DDE730BB9D7C70D7AE103DB8D35F3728470C76EBA0E670634E1A0BA84 +2FA102BAD7271DF2680D86A4CA6FC353869987700E5E3FD778165456033D624F +E9B3E80EBF431ACC934AA0357E824B8AD73E222B510DE8445C55C07C8E5DE46D +E478F832BDDECAF2EBB11941DCF84CCD887043FAED9AA90D12BC8CA9A0C8D94F +8D3BF1F80B14B6CAE6BB1C6AA405AA64BB94D5A82CFEA548BA070796A02F9642 +87326D066101435AB9EB40BA9EA9E61B363F5F5E3B924369796E8B78DE3414A4 +2B79C6A13ECB2F34E6299658D07D2B3DEF3D4383CE009A927F0EF5C196652842 +D96B857AB5E905201E7E8BA21A5EBED1FC6863BA9A1A6E5390407F75055E2EEC +512FBDB3E82CEA13663F1A1944DA072C765D8CED06AB461470C5723BDC1271D4 +4D1D049D3EB131743F1EC9A6ADDAA038ACA2C41D139DC6A84EC3C61AC7F1E559 +6155CC2F49171F6E07CF56D721D9728E87FC7DCBCAC46455A3694C765FE807E9 +9CBC2D304AF37E0F28CCB22F239541B53A4D24D09C662559267467EA487BD33A +0BEFD4899B581D20582930703A868655C31BE935364CA6A95FBCB22CB714C040 +9718824DFE97929D0482430726CCB5A5307957DD2432A9B6271E849148DEB76B +FAA290FF6D0B18DC5B76407852E81C105EC6CFAB0F620C6DC9DA555A33C167B1 +430A8BC338BFC7D75B7099CC906AD923FA107C74D3FBB719D77A4E5A685FF9D8 +56424EE4AA074434B809D894ED50F6A60A035C5223EA25DD8983B9B34210DABE +718D7B2BEB293FF1B63CFB1CBDAFC69552963D90F5E3FF533A3FDBB626E9FAA3 +F3C119E5E01C7BFF832A033C3515BF049E29558B1DAD652F2888E339E67D15AE +95F9BD14E3253DFE9072B24C0E7E85025B71096AF51C86AECB2921126A43156B +EC812B32B1164BD9B2B947D503C015616DBF2024F5C8CB3236C1DCA653D661FE +6B1C19A22D272A176B7F1B7F9E67AF40DB0EFD4940E58B2A050249CA4E55CAF7 +6ACFD84FB46FEF952D18552B3972D79D808B4C263B8C7E1BB647A2D03E102867 +630D5C3F2C917F765A4F6FB8106BA6A9D0093E27A4CB6049C2371287D94B5111 +6E7020776EBD744C6C920464BBBC0AC206033E8240017F8CCB112596ECD7CAFA +89950CF43FD87ACA750C03A778A37FBCE9C82C2F5ABB135BB02DA8E8C0D24475 +3BEA9D79372D0022FF1ABD378C151417DBC69FE5C9CA38D23A3900E34BF924A2 +90777ACDC37930B67DD44A2E76DDBD9B89598D5F626BFD325A978D277265DA47 +38CFAF16E7FF1946E15F41CA73F7B4B02E5AE8FC4C37B115BC567E4EEEFEFC34 +EC8974B1465AE57759EDDA28DD38A9210871D35D331AE1BE6097C3EC21C770C9 +B25D040B2ECCC3AEB1EA1BF99E0C2C0F192C13BB9152CFCF75332E03F9CEC376 +9B8C285A35F53655BE38713E09AE34BA2DA9C06FA42A6FD2D00CBF2AFD2BADB9 +1571629C65DA38A431710CF5B01FCA68E8B8569922FBC3F9B64A5509B6F677AF +1B97E91FFFEB6308AB68AC58F9BA43DB5E764021E75B56170EB44C2C0A7DB86C +62B8982256D3621EBE3DB3994DBF5C5A14CF34B4AF3BD5697F8E3203085DE9D5 +84B0598169760B925463E93DC87CE70AF4C2DF0F4287D2F2069847BCCF7A37A2 +AD451D5ACE4DBCCB2E14D5DF38B226952E7446BF87BEC736EF3D5AE793304618 +D66D3299AB9F9CA1D13F134FAEDF36750046E27706C7CBD8E0877BB6276E5196 +BC2A355D109C0253644918E1CC11B717DE6FBDA201E769812752888CD66268F6 +4ACF4A9449378F9F9923D584BA1B51F33663BE7A306887BC14A37E3C5A4654E6 +531D6EB63DE3946BD8BA95CFB037991174F36D61D842071E6625605CAA350A24 +FE551025D10871FE0E2599A63900C8520EF4911C53A03897C8BEE152451708E2 +43FCF4E700C583A5E8DBCC03BF9CAB864DBD19E1760945DEA0EC0BA38BEA8256 +D3A8D4F70F6685A99C6BD2BA8B412A26C002D76138CFCC7DF6802931E5D97BA6 +0151F6A4C572235B4196B22B7B2D14B32886DF0D2CA8A277ABAAC53B63F64CE4 +E4C088192AAB674497E8AF81961359C389B51F4A257373D907C615030BFBEF53 +DBD99058FD06E352450B658478C10454AC8FC0232B70D5CB916981978053E358 +99D322A07294748BA427FFD1E45C909171017B52B7C742FD77A8560852D819DD +8DD53211A14D7B2FD11E42941722FD3985D627FDAF87EB57326A0D290B5077D1 +8A4230BEB40523A8565F95E0D44F036A571DB698EDD9D94FEC9512369E5E5E73 +A3CA5C142617944F4F99C0697ED088ACAC007FCE06E5A6EDE7D0E03A3399DCE5 +362271BC31533866BA79FD1FB3F608B22CCD4111FFB1BA35D920A23AD157C6B3 +C3DAE11069D5E46DEDA7158C6478D8B8C0D9DC237CDF0CC6633911673C43FB79 +E4F9B7F27495201E5ADE66255BC2CBE9D9F237DECB62A19D62CB41A1C92432D2 +07F0629E913A71B3F1AAF8B8C5AC66D3C8605A48F8913E39C859E163DB1DBC8F +0ACFEE80A40B6172032E95A76B752B873FB4DF23CF3A655AF1A1B88C8DC156C6 +190DE72973950565454C0A188A33395FD3D529A88F2B578356DE8EBBC12F04C4 +5B899F667D9E6F3A4EC6DD8DE71FD4C2E2B6D56823EE4E0526679D71FF1B868D +F261489F06F97B010CCBE640E2F57BA3DC3332B329F7958394BA9777D833AB50 +005E8E9232547104065ACE33396772B0E0BD66D2C6CC54DEDD071E444D8C95F8 +6F88B31E20FDB80F77C83151B7E25BD3736B4F9BDC52EE78C41E9475E5A6D94C +D348AB42F5E36B4F167D29EBDFBD43B03F77EB296B06A36880FF17D412E77EA9 +F2E7C25FD05E16BEC6732681EA21AC3FF6893B93FC09316A370CDDB86D9E6087 +F6042C3F9ECD742778389170F5F041329782FB9F9702F7533E51F355F71825AE +2BF4F8FE50D413AC9A20C41B42537FDBE8DDC5A5C793D3760C1EE13716068752 +F0AF10812250BEDFB4D7133FD58F4587BACD572505C84A7D3802D27443175FE0 +0D89C3398B55176D8642AFBAB5CBCDFD6220C8488564B4306D74A58CD2921AAD +73CF803C754DAC2F30A5324886E273064FA51781D5BC596BFEDDCE3982EA1AA2 +62CA7BAA1B16C6EBB99B2AAC4E6C9CEFB3D10F19987045C4918DB239E6E63D79 +5F44B9D097118D081153AFF96E5EB39CBFBB99A3BE30909F614869031358EB98 +F07A97EA78AE50375941B2474DB46AF3305F2B208D45921F93743A6CB8AC584F +6BEBE25ECAADD5A789EF60C9F54446687E7B030DA3E5243189F02BA46BFD28B7 +DC14822E136AC7E40CE20458DDBF356488045C95907363864CD6943643BF0109 +EE027A3091C11EA392EA91320EBFEA3B857370AD8EB86D73F035A476F7058222 +E8CDE78CA1AA9EA69A8AA6EBFF3E67324C567B914134DE042D6F8F18A9373107 +536E8D90189917D343F5299024239E2EC1D2D177D8211BA44AC3CE67694563D2 +841B62F8110E87700E6D1522A79EDECDA6C59DC3004C4C3CB2DFA31AF1544E5A +4FFE7802F51C8402B2991BBC188AB976B4E85B124B81A226930F5F80E1D34658 +2C2E711A080DBC1EC1D661782A37A121F9F469BE383F47BB8E249300A4E2A660 +56BB18F8C35B4B830993FB73790CF101C3BB9BFDF16CE498196B69EB18BA4DD2 +95682F6E109B03CAADF1C109B7FBAA515F68E3D13AF1E8067FE8AD1D722E59AC +028F18A90A93B0A8092E176ACF53A13E05692A45910501C4F0C4060C40434A26 +B014245EFB211DD41AAD932109F364CDE249E47B2500A344D3B45DC41704F107 +F9FD70DBCFD768F531C969C8F2C743E993A9064AC89AF45FA0EE8466189EF286 +59BA3CB358D134D3789A40B5359E84029E48F594BA5C5DF179176109A85170E0 +4A2753C3818A40D52CE36F3AB8B526FA6ACCA04314585D7D7666D5D9CB098D3C +83447D50F9B37B4E8E77CFE1871ACCBA3842889F281701F3B63A740BB7B6AD81 +95784D69DD751B8887D3934DCB1F8B0899A8DD20797D399C79D005230455BCB4 +B167331B05EFAD279519326AB7EC85B9E9864D46D47EC3EB0D540E55A9D66111 +8C88D188B4AD02208E594A6B76066CA59AD4BB2ED81BA6FC8A7EB43E5FC84AD3 +8900F3EC0BDFA6CF2B1DDC260AE09704A85C81DDDE7FD839040AA08071AE55D6 +447A7FE4282BA129A3477A5E05F6488F9595A5BC9F626F3A62999972527435DF +EA05CB55ABD7114DE0E77E16E6F52D56F2A48D53E818FDB3BBC090470FC8ABFD +342B69EE3B4B682C4EB12E99AC58399BF4187F9F0B320F8ED6B61A2F3E3E3C20 +93A88D38C55FE7CA3DBC9187B65262B4FBAFBEF2F2696F17DBD863873CC2A7DC +0BB611FFC36ABDFE503DD1CCB844E5061E62A6F44AB8719B95C828166B5C888B +AB2B50F10C514431F4AE568E079C682E6309D0599FFEAEFF6EEB9C0FDAAB55D3 +BEE35C0EAD701A6D36923FD5054D4181F60BBDC5F129B808B33A2A8CFF842374 +35A20FD6B1CE5907D5F660F065A935519C2EB970C010B1FF2EC4BE26E5AC6218 +F9A1CAD2BC6E8128C9E1C70D606379B86A74935F975911027BD8180C40ED9133 +56B06AC135D340C25B6C25AC512FD147447527F64E57995738891DECA8886434 +E8ACED7568DBE2B9B12A44BA92AB05E0740BE47BD774F06400450B5A5009B266 +5293D0420D7A2439AAC0870F1FA32D30572847B7CDD059604297BB464F298466 +1BFC8F4957788E3454E7422C1E26E979A3099DD248B0662B253C30C5F0AD6B3B +9C5CDDACB1D977E9B58B729306940938BB4D5621F926BC5CFBDAEF53EF0C6A4E +5BB8F75254CCF9FEEB521723E7F496820D4F36844B85ED153DF99748AC0F0076 +CAF7E30AFD7B54353DA8A077E3DE66299225D48CC977022D34B7F6BBBD03C0A9 +58B204CF7904A5853873F58166B7A54D0B1CF66CE960DFC0CCD1F9D108CFAF17 +C3FA78E0C490CFC125161190F5C57BEDFFAF450300D1124824BCD40CEBE5738F +2F3F9565B49F00E6DB6078EDE903575164A7587BE2BDCDC6A095DAD6E4CB9E35 +1D01CEA2E22B64B1911A136534A0AFA1E59AA68714AEFED29C33A1484C5EE5AD +F6085B5F940AD671325A601F17BB5E259F332E968D2ABA894C82D5423409640D +DDC808B69D8DEFE2C99E301D13CF41BFA6AEACCCA94047318EF847FC2E19B29E +5CE2CB226CC198F3550783512C66DD0D20B4994D06E7137230306FAD626D4722 +3F6D9A85F8982ACDDB1F8E20C172B78ACF60509DCAF1E81026BA6A3B8F0B6A44 +9E825EB6E4B00982D07F1ACCC4654F97CB159A261B759AB35A4E1FFB55472D4D +7A64DC43806ECF445FE9CA1F16450CF281F0DFF048CF4F2FF9FA2FBE7B45D44D +B0BFB768ED4A004B7C3AB005A8C725368B0A1F8A43C518BB453A7A456F4DE34D +94E8207BFE151ECBAC136187D69489EC4766E49ECA7FD215EF1B76550A341980 +93B967FC24C515AF4473389E8D415E617360C08068FAC4D3DBA51CEE54DD09C5 +F6EC7786E27D45DF6746546616C6E14501CFB2142DE4ED493D693254A02A147C +073B79EF58A4C66B986D426A8957DACA968A38D87F6A60D68E7DE45B1BCB14E9 +7DAEF3561763F709795FE0EC4E0E59D79D8348CE3C1F8BDFAED200B8CCEA0AE1 +51657FBCA7E5B1D2D533467AE4576BB10B2231A697FD04916384C0A2A25ADF20 +D93F11E098A22F7381CDB0976C7990EA8D4590027CF6868A9A6C8FDD5578246A +D1E5793DABF37AC1CE47DAA666F22F0096DE0389F3F68F7BA7C5154F050BD558 +51BA9C614611B81CC6E27F2D2A992E858B862FD5C95AC80254FF484975360B57 +D3BFF8620BF66CE81866B5E005D39C1471D5B4D6777F87F0A2993B0211EC2DB0 +C7BDD7DCE61C6F5322F5695CF165D03A67ACD6BC6A60C3461C38EF3D9B27A472 +06615DCBB1A6623549603CDAFAE402F832AC4F207F2A9DCC107F96914119F7CF +39CBBCF906CDE4433244401C6A38F33FC192EEAD6A8016EDB6F344E1A1494E8E +AE7B2862ECC61EB97391ADF3EB88784A96562EA8130CDC14EF70DCD0779BE00B +649027E1996290AF8E24D52C1F1EA711B66766064BE129E56705E69E8C2A4C14 +D17ECAC540B15BCF3C8E6E7276416FBF3728925933FA9345EDA777138B56BA0E +3B73895A45C18AB7785821B5A52917A7CB9A61C70C80EBEA096349B6934A4EF4 +759ABCBAA9182DE221E8BF21FA6D12A4F5D639596BDAE77AE40C4DD1153938A1 +30395F4AF698A02F04EE8A5826FB83263FB813DFBA349F7606799CC7D3000F15 +A1215AEF630E0BF7DB811D06700ACE7629DDD1D8B75ABD372C023D36AC5C039D +A81B7F1EA2ACFBFDA6AF4D33397D2839F6BE02B0833A42CFFF0EA7389FC7DEA6 +16865B4EB89CDEA054008264AA05AA6594028B1FBFC0E89E66FCD8FA80F45216 +F8F7DF35BA06E80AA6BEB98FF46D83D5245DAD42A33E1601EC5C98FF6907C5B9 +B0F660834E8042A65D74948E66372497B2ABD4412A8B8A46E277FB1360987391 +6BAFB23E0A02920141A19BA5CDC0F6855073FDB00BB865D94E4488595BDDE1A4 +D95F85E0C4B0A1194BC09668CD6C0D03750DBE68463D2C96BEEEE91E96E1AD28 +38FEDC3C6F0E62F7D887379BBBE98D35195BDE978B384B1EE39BF013C3117278 +E456E7CC3B6F95C3101B7FFAEEC4ABD7322A02E1EA4EA1A21DDFE3FBABCB0C97 +54436989BCC1F7B2678E2D9A0975F207AA966130F7F704B4C084BFB2FEB31663 +6ADD9F8BE47A5D8BC5813F455E88A0F590AD743635D51E04495B55FBCE9EC630 +6251E51A9C67A67A82CA49BBBF6AC2C11D27012B217D24896B717EDFC801038E +1F45C053B268AC3FBF04AF993EB03F27D1E6A5B490374097FA6B2D679207A95C +A2932D6501FA16C21E8F22057F3B7EBF1B2AA4182960BAD3014533B3221D7C9D +E408622572B9CAFDD8509728CFB547761E975F2AFB25246A2A994CD2BD80AB9E +3CEC30F5223B23A6738EEAA688C0FC840AF27D11EEB249EDBFC4F83280A8EE02 +92B4D418A22540CAE9E722D44CC63746AF439AD6C593D3B8EFA024C1DD42C3B3 +4668E43279684F5DB12D5862835029BF889AC58F998C085F72CBE0DEAAF21581 +B3B3F3401BEFF886774DD073D0DBCA182883458C2A3547C12C8BD1B99A75D00C +6F8A3DA8B2C1AF0A2D9358BE42BA481CF326AC5BB12B615132BC6293AB10093E +F96C918DB2FB022AC985067AA8F4F9363F48DA7E303D20B53BEE9C15F15681B9 +5205E08E6426B24E995DCEF54AF1708AB92AA3212E2CC29ABE5B415F1A46C97A +86A01BBDC3F6F142C7E7DC760413375AFA6EE9C56902F0DB2F5A45EB91C7FD3A +A1688B449AE81D3D77D69F09D4E528FEAFA3B17CFC59B8C5B7BDE97BA922E142 +59B08E8865B79ECF016E319EDFD4A514CC30B01F48D798E591DA6B2F5BF94C6D +BD38D42BC17CCED696C1BCF391030F0F445C13695266DE3FD85BB6E12B22203F +133CC4023B475227624A85E81A0FE60B9CBBC850E4C5DABB171F98C076F7BAE6 +C584844D910897C2FE8C94DF5019EAF0494ABA0526496BBCD9DF3CD52EED942C +9D53FD3D67AD8704AEF3868B7C341417EB3DF8A6F81C2DDBFF464E6E35AEA659 +BFF3181334F19707F7466093D6C68D10C1145DA1E90AE0A1DABAB8D5CC17C195 +64847DEEBA9DDAFE2CB46867EC8C93D0D2717696FCA1B75A897B443248BB8C06 +6E29FB2449C8BCBAA4B587438077DAD610BE12FCFA99C73F587363CE9D0FBD2A +6B56FCCFB4F16AAC93AA353CE4FE86DD081DE6ED9A1647E49FC805A055C922FE +45667EC2DA4D2653DBF3FDBD5BC56F79309FFC5BC9B7435F6C86132F000DF523 +DECDC83D84A8E50AC9806B792667E32E499F11FA9D7359AA3E3FF5B4CDC52515 +9BD7896893276E071EB36353C5478A8E6A6549185966B01620688B9B9F5E631F +F2D594A2024AEE3BD70451E594E0D0DFF5B231420C59BB201FB1671E61C8272D +188F8DBC75C009E0F5887589DAF8148D380DC452E9E0DF480295F8A2ED10663C +E4919C90C1CB1AA9830730449865BBD85BBB1465F61504B68282748247FD3982 +74E92C75C5CA4BF9ED5AFCC23895A5CCD3A45AF515688B18E7B3FB6F0A16EFAC +D9B7BC07D83321ED99081E43EEE3DB08001D21AA575BEEBE985DD9B3730819B3 +C8591E184EAF59C9C9362BAC305E9631F927ADD8CAA0F8245EFFD921F0CA89AB +BEE50AA23A1A54270D317B08A1F74A8EEFB164D0D977700C8125FDBB7C9D66E6 +1019C67766887DCA5660C100429B4F158FC4E109444123F6D3B10274F4E89B18 +5D319B6C6BC2B012DACA3BDAB3D908DF1BF32173B6C6DB645FEE3A6B704B9E3F +E35628C9B8ED60350319307B22DEE61AEEF986222915DEF61B8E20E19E028829 +08BC4C06DED8DB0063F6B5E6F76710E05EBE7D8064DA11AF75464C04F79E62C8 +06A5B7D3EB0CBDB79256EC0F533DF09D2EE4B2543FF47826AD62E37CD7BB703E +A0A3C6553EA938815E12F47778239FB8A393A27C75DC3E3BA068310C0F5D6AED +062FB3CC0DE56F9C16CEA8D5C0E6AA8DA42AA3E7DA9FDCD6593EE03EE27BD045 +76EAC4409BC66C868F43CC49A5DD2763C44F806434DA3CBEE9068B934AA724F3 +86BE17C0EE23737AE4360C1AF089E197513A8B27B11D5D368C2F41A97B650352 +08A0853852045CC4C00CB5E911E40B14E4C66D8C1D13DC3D6169FAF22C0F494F +D2E8C96A8D744FCE213294F1373BB3A95B07A0435DCA2B592C4D4FC1140809BE +DC6CA900FC8F97F3F6B14F85F8B07BB4AD43D8A8480E3AA70F99B04B07FD4315 +54205A2DCD96E764384FFFBB5AE725052FC78BE7458D3B957E002170616A30F2 +135DC5442A2038AB377B7D1A858B5D3D01E6848BFBE75A23EF495D1912D33328 +327BFA6BBC441B8E40C6219FF5E5848854823B6D64990EEB1E21B5A46B8E40A4 +DA5AA104302A743FF54BD31C2903C9F4F27C355A4BD6FAF285B3E0CC8AA481C4 +18BFC9755CD35CFA78BB93AF239425CF64841ED47C02199F35FD7A90B0526C64 +D2F5A6F86FFB06794A329EEFF23C6F14DC37EA755FE12F5C0ACE80A8677FA815 +3FFD057358C19FD86ECBE50876597E107B7B071F470266CB440F951D21CF603F +4FDD45FEEFBB97753FBF5BEFC0CA0BA77B247994372E2A61A1E9CC789CA94F94 +0259C10027F0370EDF7FB3ADA8FF543B424CD0EE4803FDB62C1ABF55A72AF985 +A4B391ACA504A6682AA77E8A5C6683445437D8BF89CAE556778302BF40FF3E83 +3E50C03F97DD7EB1C79B7001E2DD70C17E074E2DB0188924E365ABFDF5A312E1 +C0A088912BA36D2063E803178DC394FFB412537AAC2F5BB0BC3E81B44110E82B +76408EFDBBB5EC3C7B163AED6D084FF17F62842289854B9ADBC21F967BED5814 +5A6AA7ECFA7B3DA1183030D4E43C1CDE5B239BB4A3E5E4C7104BA769559B1B24 +B7D9E9AF28E10B68984EE8F94E07571AF0E04FD3F16A202A384E8EB8D279C553 +15186F705AB2CF45F6DF23BC14F7E2FEA8A72D867909F6420E66C97D8D2D81C9 +CD21B788A882A8B553705F9050D2E505AA097F8C1909C17D44025D47966BAD20 +4C4D6C17C203508185449158A3824541ECF1448B31839B9E60CFBB048B319B6A +213B46E6A6CC89C9F25606697946DC083C34379A194A514392531418E2106991 +AFDC2D8495D4E324785FDC24E75BD9E8E91F874BBE946A7240158B57D18FFC86 +3C7848A3644F8F0E460200C6DB9DAEA7405B2246E9430439A6927522950BD134 +40FFC5BCA26630732435D81E5259FBDCBB5CE36C094827202B80953F9C51A946 +6934A4E7C6830D5C33197488230044E4EDE470B0B7F5FB97BCB3E09D083D5B29 +3BA305228BD93FD1981F3D4A58179DDF9DD65AC1FF7E4B4EE3158644166D9986 +2004CACDF58B5D27F7AA71572EF01CF2DACC2BF109FB3BD0EC911DC51C13482E +EB9E64813DEA011B8EAC6A7D35A5BDB4ECD542D6829651F9F36BCAC020290009 +499AFFFBC6E1F7E840CBEEC7C3BA71A4E402BB +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: BeraSansMono-Roman +%!PS-AdobeFont-1.0: BeraSansMono-Roman 002.000 +%%CreationDate: Thu Jan 29 18:27:33 2004 +%%VMusage: 120000 150000 +11 dict begin +/FontInfo 14 dict dup begin +/version (002.000) readonly def +/Notice (Copyright (c) 2003 by Bitstream, Inc. All Rights Reserved.) readonly def +/FullName (Bera Sans Mono) readonly def +/FamilyName (Bera Sans Mono) readonly def +/ItalicAngle 0 def +/isFixedPitch true def +/UnderlinePosition -104 def +/UnderlineThickness 69 def +/Weight (Normal) readonly def +end readonly def +/FontName /BeraSansMono-Roman def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 0 /.notdef put +readonly def +/PaintType 0 def +/FontType 1 def +/FontMatrix [ 0.00100 0 0 0.00100 0 0 ] readonly def +/FontBBox {-5 -236 606 928} readonly def +currentdict end +currentfile eexec +D9D66F633B846A989B9974B0179FC6CC445BCF7C3C3333173232E3FDBFF43949 +1DB866C39088C203DC22FDC758584860EC7BB67FDA28CC6208249060E18FAB32 +204779B5C03C0493BBBBC95CF02692CC4DEAA8D2EA90B5C2E64374E92BCB8501 +429B8FAE4A76C0C6B76D6FF7CF9A7D5EDFBCA0E959541C59BD05B7DE43D25D53 +FC3DDA6EF0C2743978A6D03E19CCED4A11F2EA4BCC3110BE8B8D9E2772361969 +C19258EFAFDC276CB1ADE9208A941A36D18A96F6D1C771F81C4B3B8CF0CBC2E8 +4B44D923DDCE84E17DAE82547EA9EA5E732D78F03AA245377BF0780A3752527B +6E63F6A41202E7A6C4E4F9330A0AABBD04387E12F2ABF531216BF498DC6B6BE6 +06DD50B385DDB864515170905E2BF225AB9AEF29A2181200040460795735C124 +59C90AE9BF9F01F42A2ACC5A5D97D4F6548FBC903ECEFE9FBC8492EFEE55ED6F +F29A7BB50432A42FF30DB3CEBFE1C2BF83D613F2C846B7F2173F00B4EE5FAA8E +12EDC39BDDF670F50F6AF072673281554416E77FF134014D5C465127C28647BE +3E810BBD9EC9F9580C21D2E9479B8F0435749734B76F4C06995E769710828768 +3141ACD67803BC92D5B405AEBD4B25D5DF255B110F1EF1C35D45D24AD0B1E0C6 +8816579DF17721764D5D2CE396276ECE1E6142DC2EEDF83FD7B65717064CF671 +051D92A9993F82D241AA1C3254DBD3DC53886264BDA18DE9FB8F584DEC9A32F9 +82A6131D7140BC7926C074E26639A7141B87ACCD64E61A25DD7E26F463FD7834 +964655DDF83538CF1DEA3D6363DE79EBBCDB429279F84F5DA8035CE823A76FDA +C2FE77036F5B9D71C0CBB063689C0C9770514E550CDBB1BE97D763729B78CFE3 +DD162726B3F3190F0003F4812E9877545507664B32A1BC0E6C933BA64EEB3563 +39606495CEDCB8EA6E63ABEC9F3558DAA0BAA350F45A6E9B145480D9513B87EC +14F7551489E0F96B698A104DCDAEF9160549D551093AB3F5F7EBDDEBCAA3ABA5 +F0CD388ED13E9611675BA045A8E99C002616ED89D1F3EF0537C6E787DEF9B5D3 +F20879DF1555A646A79400D3EB80A209937A45A6345C4E51748D3A5C523F9F93 +FCA3628D463FABD8537A948B3A3715BAAEE70345F5186A550FAC78ACD262E671 +AF8FDCC133800B82183CD17B4EA18EB6AD976F91FE3C1447EAB9EACE7AD9354A +B29F3500141C4B6246062E38306143CB10B08216EE8AA2F99A23CEB18014CD30 +7F34DBCF8A3EEB672A6DFAA20EE2EF3BD76879E37619FC7E8998B432D4319A79 +BA058744039F384AC76B2DCE1B32094DABE91ABD547C8CC10CB0D450407320FB +1539B126837C367B0014260B8E928F11B8C0D60C4D4B514807BAF8AE9F898DE7 +E006D651EF8842838B37E0F991BC3376DE8004E1A86DE3A07CC82B166F156A01 +20CF7B8D3975893E2A84F65C0B219E276E6A73ECAE7EAFC563637F7137B4FA81 +12BD5D8B57792127727DBA31141A27320282B63A1ED8CDB89DDEB8B040EAE190 +03C35678B897E0C27C6473974FD43199B0344E462B2FCBD0672A1C7F84F27D90 +8B478094FA44EFD3AD5CEE17E30490E31356CFB37B99CC7B9B0A27D287D9D6A4 +13B354CFF2A0BC37703774F70B96635C2BEA2CBC5382F6C96B13136B9260D75F +0568A57EC7D74F93330F46C973B29E97D928CCB19F7A4396A55340308F6E75AB +CDB5B7A0EB395C742D5AB9BE3ECDA0F86562C1503F9BD5B0CECC2EDC1B902C9F +5FB13E9D7AE801E7D56BD54424861F306E3AD277A3A25E4D41CBF59FD2081EFA +FFCF9B742328EDC962FF21E19D92B5FF46AAC81E9FEFACEC65A8009BE7A4487D +5EE52C9836829722257123A65158C91DB90F3A3F755F8AF17E1833A81C11EBBB +14EC001C94279BB82D20B0424AD36DA30077C67A946C4A6540FEEA068AC25C25 +1B9B0267E589F43798CB4D7A992EFEAE22273621B0C31664978672BD6F186E2D +8FBAE8B55AAD67E4067BE47CCF3B2742811C21DC58B9FEBE6339FF1E82C95A76 +503802FCA297938A0D07C8E2B23987496D664C07759AB2A407CC193D2320D9BB +D53CEFFF13CA8A2F03A91D77F601CB162F7935E1D591D4820EEEDCCCBDE3F3BE +CE6969C9512F7688BA420F82878EA772E2E72CA4E6FCA36ADC37894B8B982498 +814CB39C2C2FFB8F30934C971417DA95D464E135877C6D2D13DFEA8BB3000057 +E791B7B9D507B9B6FB91E0341398366B213F335971758ECF12488BA2803EB9FB +869DEBAE5CA0B23276874E312B24BBC22F876F7234458D6AB3F15DF80F9E26D6 +010299DDFD174B94B7CDD45B83DC38B8BA48FF5B027696538039AD830786A0A0 +68F8BAE7AA56C7769B40FC65DB50F77A837EC449F4AC8773664387F6F598BD94 +0FB24DB5E38AB1656C6A8264593E7F788D739BD00FFE46807498FC405208D16B +D1426FAC901C6D0ECA771ED25AFBA64FF0198E5E263F6CEDA80549049BF64A74 +36F0A281DBE81758E0313AF43076ADF59DC60CC58574E147078BA2B767EAAABF +15B974FD33C8214AE7009F9D720FB1EAE296C0484F7FEF7816B019312FEEF336 +09E05764AC81CE82846EB254B065DE1D23934C82FCAE457B12F3E72B4B963D1F +6DF6B4480FC1F1927958AD1B2A4EA7EE10947DB0BAB5ABF50A761E3F9ACBAE80 +BC40EB28A8259BCBF0E7DEAE4A2FD0DC985FF621013D02C47C04DAF2F03C9C99 +069FCC31B45618092F025ADA3ED013C724CA96C4EBB50F503D776C1B0D3AF01E +64B28C0E6C177D9A64711BA079ABF4886477BAA78AA80578FAE814435327646B +28C5081C8F6D048593122A36359C8292C0CABC681563985B86F1166C9E0B1509 +49F2B13D7CE048140FDD16BF46BDFC5B1553AABB82002CCAD1227A82311A0490 +DC1AB194323D8B4FC98101165CAC6B4185121B57485075C84B5EE5269E843D22 +61E2F34D35D395783B46FAF92DB96B4873EA8FBFC8F932152F1EE7B11613AE98 +46BD157C5AF1464FF0FF5C02B3474FA3725BBE093198AD9626E1269B314CC1A3 +FE4986E6B24D6BAC414EA18320D8BDA5EE9B9FEA7E39A70C4E38D98F58996CB4 +E3B925870704BD0DE0D571F7477D4F0B065C00EA0207FA5883C22D562973FD4E +552D55D137F6C8BD8DA14BD7406F69F19B6A859FF846E725E88C85FD5E5E1520 +68F0A7530FD68B6A37D88E378D6881D82FA702FC1CD2F7CB7036907B79EE5186 +B8A07589BE5C83B6062D0A94012FCF8D8295860000255E8E4CF7A8B6B0475D23 +78C372F5DDE779C9A6E5EAEFE0F84AF9BD9ABD16F78B6C3937C6DF6E0459DA8F +861DAA0D574A3CB11A64EFC8F1F90B53D44636D654EE87ED42A3C22094477094 +495B21B2A6086F2D5CAA8E1FF33F47CA63E04855418E15B3D4919077AEF0D8CC +48D558F42D8DEA1CD9438E7D96A8028CFB8AC667BF45D8F99C3B57B509CFD962 +33644ED8E7FBCF25A6C855F420AF92869F400C240F0C80E25F0DCEFD3C934104 +A41C8F8DA71204DD72605A458A816556AAA1639F0FE6DFD8773E579822E2152A +F53D41B84B2A128A6A7FCCE9C9A6F3143608AFDAB4DF2446CE0F730F01F4D111 +F033EBBAC3B046E11CB08D40D8971ABE40CD22792B762A0F828AF80D85955993 +B3AE82D9428EE1D7BCC1ED9CEE8C293388A1A09664809D771D3719306FE95903 +B708B61027F581D2D439D003007B8A7E6AB82F3F53FFF7FAF3988E56F572F2A1 +6B006E007D50B3A445AD2C8E989F71A449EE5DD329FDAF69B1F13149EE56C097 +FEBC921ACDD53226AC2B1955B6A699BBB7BB702E49CD651FCC7601A63634AEF9 +60AEEA8992D3B3039176763EE7F35E7A05B74C147A0CF37E5F10F49DF3B616B1 +FBB383D33D0E32AA5DD11E73E16202D8AE7767E1AEB0E65BE565FCB984EDCEB4 +03730CC3A2C2A925DAE09DF45E0609942DF7400919FCE30630031F7079842B16 +AF232EE453C50C38C661B8FFE54A8534038BE4360038AC66458FC40E1B9B118A +3BF05107954E7092B120C22FD418DAD0CAC5754875CE574849D08F154B0B1FC9 +9E28768B297B46587C9E4FE9F8246776501F09F5DD197C012AA381F9E87BA7B2 +B53D93C4F7307C66C9D4B78C2B09D7DAEACABEE33D8582BAE55E49DD4545CC1B +F23C74DD4FEF3FAF80050EB66E63F50E992D3F247524D59D8CD760965A64A624 +7AF929208C1133E9176D1C135425B2298880E5B8AECA695C345F9D29E29BE2D3 +A9C4C4EBC25672170498B844AF40FDFFAB786917E801001874F74159C7BADD2E +E235399D8E1F19B92C7D66CEC194488B054B0621F6AE352B2CC4234DC3431DD8 +BEAF52B0E471858703260127235BB9FE3A98CCB92AC6B78EDDB431221D7C3534 +BFB0FEBF3DAA01A655801B496905E9872324397B7E5A709F8A2C60BA62A86F16 +CCA9CC966E9C36171B2895E77FE56496C239B0854FA64124E06BD321DEE27054 +E32C3747507880E278E23741011C2EB39F8D65A288E122F5D61DB48C0F17172D +8EFABC875B8FB50FB82C9DD9A7DED8691FE570205C708CAB02139A3C65FAA334 +3DED7C1E714418A4A833CCC3231347BB6D537B58D01D983C8B6D2F1F83806DA1 +9D9CB9C06003CA8018836518703CCE03930E280418397C27AD145669EB2C3FDE +96F13E0F3766BEE8BC075136284AB4C70346204680F25EC3057704A8C0BD3AD1 +BB469356A55BE4753A81D76025D49C033E876A02B9BCC6F74315496EA9850FFD +A4C4E9E3CCF7096491DBB2A3E4C543433B69BF0C95A16C7447ACC12913AD7F84 +CB3C217E8011AA02D6F51E2537029AA7BF02D8AE906115F44A085825F57F85A1 +AF9B53A576E7833EB94C166344764D626F08F35BC061CCEA2B3327239A166E3A +DA4D899B93A968983707EFF2C4BFFC1485E654614CFB0903AF4E95291C7EFE15 +A63D079038926FF42D7873285C00F7B92B3FC3EA92D31409D3542D196C45BF32 +5479F95A67AC0B793ED5A4FF895032E5FFB07E2AD226A759D778BD77E298C626 +B567F7A5FE3244CE8EB1A5515A05606371698FF7D2167E9F3E285A8472585F2D +62862D8DF390AABB63D241E4E14AAFCC33DAECD7EFB0F6E53625FEC0551F30F5 +A5DC867C7C7C62CE425AB442773993CBF758C8D9F524535E73F9F1DB342510F0 +C0E98431F5AC2211AAB4B782AC938200AC1DD98E70DF19BEA39AFD63D766FE46 +43FA084DAFD7C25E8A4B369A68B73683B9431F9324EB4F7E51EB5FCE0ADCB10E +0220FBB708FE9496CC2DD7B89C8E2DF14E7CC343E09BF2FBA8E7EDEFB7AD2B72 +F944D26C2AAA7EE1A5A8C629BC5CD86B0E395D276B525F6F3BFDF16F28D23786 +A38BFAD28CDE3A11071EF279776C985436C66ED805E8DAF43C1ABBBBDD6BE445 +588BCADE789EE1ADD98C9CBBF97453C22E8AA67869238ABD81E835CED2756FDC +FF68FEDF98F7450C747E1A2FD16D890DE7C9B87EF6ED946F9992E68358226E71 +1B675F55AB9D80136122F28479C4854CCB0CAE977001C0402675692A8CFE7C70 +3D069CA259B73AB02F8ADE6588743223D21D3F19FC13311C5F2423A8818BFE7E +9D1F6EED53BC0D4D599B614DF64311DB1F1882EA57226C068839A7DB3B3C57F7 +450D4EA2BCED6806C53E7615FB3EF62133E3C161EFEB8A73AB7A84BA21E2D7D9 +1666D19BFDDDF2FC0808763BAADEF3E8BBF1FFDB3C787CF2BD3014961457A104 +92254C9AFE0DAAB83C444BE4F2AFEA869696BB5F6810E5E9387F1856FEB2A99C +108C6B1BE257F0598CD58BF321DD1C5316782BC0E824557D081C905095EDB756 +D9F14CB7012C7044FE43D40346BC4F89BB2FD44F81862CED0DE337A8CEAA48E6 +E41AEC4AD4E70CE0EF74DA9CC406953FA50D40F6C4C628B27267DD86653A1B88 +491B47F3085175F73731DE7023EDC5879D5FDC4AC8B026E10F54B9B27F88D034 +ADB5E73EB53FA2D868AD8AE3B30FE8C50C041BDE3A78851ED7E2B13B83CA7070 +473DDC6C6838A9D58B734E77D33DAD505D598FBDCE623A86AEA7F86BB2B661EA +C5B71DE272DD4A1A909A5013025E48D48B035E403EBB02C9AA9F31A434833449 +4A2EF1892F4745432A1BFA48AF0DC68FB45A34323104ACA3C7F36CC29BD7289F +E0BCF289523A0EA955A8D7D5F340BBEF00969C8123777A922EFBA4B8BAE0B41B +8B4E6859A16F92BD815260E28FCA6BE3E640F3AECE6598BA1A37E95665F93232 +68B7624CCFCA8E4187573258F10577A2 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +%%EndFont +%%BeginFont: BeraSansMono-Bold +%!PS-AdobeFont-1.0: BeraSansMono-Bold 002.000 +%%CreationDate: Thu Jan 29 18:27:48 2004 +%%VMusage: 120000 150000 +11 dict begin +/FontInfo 14 dict dup begin +/version (002.000) readonly def +/FullName (Bera Sans Mono Bold) readonly def +/FamilyName (Bera Sans Mono) readonly def +/ItalicAngle 0 def +/isFixedPitch true def +/UnderlinePosition -108 def +/UnderlineThickness 120 def +/Weight (Bold) readonly def +end readonly def +/FontName /BeraSansMono-Bold def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 0 /.notdef put +readonly def +/PaintType 0 def +/FontType 1 def +/FontMatrix [ 0.00100 0 0 0.00100 0 0 ] readonly def +/FontBBox {-19 -236 606 928} readonly def +currentdict end +currentfile eexec +D9D66F633B846A989B9974B0179FC6CC445BCF7C3C3333173232E3FDBFF43949 +1DB866C39088C203DC22FDC758584860EC7BB67FDA28CC6208249060E18FAB32 +204779B5C03C0493BBBBC95CF02692CC4DEAA8D2EA90B5C2E64374E92BCB8501 +429B8FAE4A76C0C6B76D6FF7CF9A7D5EDFBCA0E959541C59BD05B7DE43D25D53 +FC3DDA6EF0C2743978A6D03E19CCED4A11F2EA4BCC3110BE8B8D9E2772361969 +C19258EFAFDC276CB1ADE9208A941A36D18A96F6D1C771F81C4B3B8CF0CBC2E8 +4B44D923DDCE84E17DAE82547EA9EA5E732D78F03AA245377BF0780A3752527B +6E63F6A41202E7A6C4E4F9330A0AABBD04387E12F2ABF531216BF498DC6B6BE6 +06DD50B385DDB8644394C4D3FD8051BE7AA17BBF8B3C41354A86C4334EA306A5 +8772FF918DD38C7BEB1DE9E6CC2149A988B8662E7034569A0631E02086B7C244 +DF8F585CE52A08716EE2832F7067728C244B87BF5A0E5D8F720F14859C2BA170 +7A223179BEF36D98791691B88239BF1292FF3C511739A3342419C30DE5C4F56C +70570E32B8A256F0D0CE6E58A26205B32CF8E4E325C2E6B9CB74294CA40B1B98 +83A602C90C2BC6F5484EAC2F33F86AC21E880023E3A820D761FB828169C8CEBD +AE34C85DF3925444B8E29C56FFE614B160B57D8A55F1A6D52092BE766E050A98 +8D1A856B2E24DDA3EA9962A54F0DD0996782017D4654C4F8BF617B529DF54BFA +86E1E32AA4C1A212212094D7942F412FC49C96309D495EB67CF229376A6987C5 +4E320CDC9808E2D7ED4BBAD182C2F3A3EAB70112B45FA062DD7C6492BA475EEB +19E78A80F74B6859647BFF707CBA462AB54F51359D9DED52A23AB9F714CA8226 +C0ECB0ABCB6E109CF5E72424ACF5601DACE099869520CE07F92DB9E600134C09 +4CE89547BBC53C6384DED35AA84888D4272F31F33CE7EDD5A0199C03382B0981 +A32057490A55690393812F465311858CF6EB237B6CA692048F7410F96B786135 +5B3D1870DFA4CEAD06744BAE44F72CC543229866032C9905A3CD2729C30877B4 +0378154914E27F6DC55C3E980C3A03CF137A1A1342922A435E50C802728CFE1C +820774CB474ED6B05B7FB52B489B0E9B262C69056AD9EA3021442069CECB6951 +51D5EB9C92E6F88C6F7794FD8791B0A6D07A5F7867D2626123AD13A0801A5F39 +E1C311C317F89ADAAFA6E13FB7BFC3AD1444DA05129BFADA52128313A6A84FB4 +CD05DB3FC14F08AD225EB68BBEA662062631BDD1044B3686A4840ABDE8C04923 +7DC62F7707B4EDC88F46EDD93501BC8286D9A228545D88455E2BAEC7C499FB3B +AF0161EFFE3AC361F1846807492FEEA2D84C86ABA3705641056942E206181B41 +0900F9299FB7DA4DDC952D5A17B8866177ABD93390F876FDCABC1F1A37C701E0 +BE7B5118988368AD0896054764A8F5C70C5B3472D8AB3C0BA86330EBBD604D34 +8681AA2451A65BED11429B7E94135471A57008864A787A50C4C6C4B11C0E579F +47DF8305F58BE04480665FE7F15E8F7D54EF3FE5A955D585A43D51AA48955EB7 +57AB5F7EC39FB3989C0DFED6A5596A432146C661746E7BC3E888F5FCEAE4DD0A +50898A96B6675921D4DBADD8F7C51691FDE69FA26185E4C1CE27C7227B38444A +D01FD9122C8986AD2CBB93F8A45AE61FA28E226A22DE5F8F16453DF31EC35E8C +E91689B6241203D414CC810268D0D1DC072F8FB9CB5D312E8E3AA3A449E9C9D2 +149F87E84BE9B61FB17C16991D0F0C62B83D820354BCF0D422A7195DC898B8B6 +F5A0BD98204D1826E452A6E5B78B7E976DA0310DB72264B0388DD0BD42A60878 +6E848DDB11C55BA4E6F05612A86A2A38D1BCC5F3B9B0ADEFA1243DE693A3A4AB +05EBDFD7C1625AEC61FDD632C251DEDF3D2F9F36F4D8E6DFB861243901FC7A1D +953BFCC91C4693F6D4E470BE800282CBC7DAE592F5814CBB89B1EEC84CFC483B +85225BE434D6F03801EBA9470FE34B23F62076C1300C5CB44907877B61CB19D1 +71A23503BE60B9098082F242C77C1E605EBF5D1932C59E83F6E076C1BCFC5700 +89D5B2F31C20D252C6E37D3FC660DD1BD951F9392E88102F589038D2DDE3A8F7 +BA88675CE9F026D39B8E92BA6193EA627F5E2F3AF1206C11460770D2B943958E +2E0B4D048EE15B1B70765C14A655F6968C94FB8DEE69D6D6EA667DFEB60DEC9C +312EAA0CEC500AD6DA063C641BF95401C2B667869CDED41CBD6FBD58C7CF6D68 +B58F52ED684CC901DE5CE1708447BBF90C99833370D0A7A49FBAB87810B32D6E +206106D075098728D0211169306C8D7CBD1D8AAE1E6A4C713FF3C10D68C29596 +5C38B02B8DC0CCD9E56208298F24C0F5E729A604B93710E20B830EFF70294449 +2D9C12300425238D85011BE8895C8663E1FDF4B1723B451E5BAEE59D9D4187DD +BBE2CD681D9B3FBA1DD3E8904962DB656D477875B80B513F09C2F155BF9D5D69 +F4967C050C7940EC75DF62FFDF5D7FEDC398A3F7D921724F1BC80DD17A005220 +80468A3C9A7D90FA68C9E72E2C14825A96075BE9DC8E713F7B32FA7EFDD32436 +1BA076C489854013023B0C86EB4D6CB1F4DEAB3F2BF244084D2B3A010ABD894C +C70354D605D5D7EB11497DCC6B1D757D0A63167040009EAF34A2804055F11133 +7A0810C16CF10E9AEFFE48CD7B0AB92720DFA5C0C1748892C1D64F2E1C70B0CA +53D4271AAAC822952CEE5F229F0548197DA94BEF1B2DC33DBD8C8BA858C6EE38 +CCA603F1EC33B96ED605711841406F68380BD45F849FBC3E2AE935CBFA084BE2 +13519A38817A092F1EE912EBF3CF34ACD9AFF153D48D88BC02AC923AE3361D60 +0816BB327BF1D51AD8029E6FD8FE27B94D9CF0A719675E3823D8C77399E1E24B +B74626D6F6062586A44BBABA5B580719B9AB63EF3E3457D6CD3A76AA5A6178DD +292D32B8494F88AECDDE24B24CA1AA67820CFCF7C8A15435A8FF528ACAAB7E9F +3D3B385D167816815671CEF53A4E5A7C961701D6F82B9BF9659E3FB1730AF4C6 +4C7F9EB3AB8CC8599E4C24F6407EA3A898D80EA2CF26F31EAAB2A9330079E344 +9C01268AB11D80F97CB55C7F8BB0E3373ECF9A293C1A069D4F579F3253D8B855 +CD5515E423A4494F909A36856CC7E0515F260BC607222BCF96255E5B1FE7623F +57315CB33624E7C51BFF42D9E05C54F6832D98FC467825873AC005542B68C2AA +8B94AE2A759FA4FA6BF288DA801DF1B388EE988919E297C14761D57B157B2DD2 +FE9CD793AEFC1412A0B1A8 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +%%EndFont +%%BeginFont: CMSY7 +%!PS-AdobeFont-1.0: CMSY7 003.002 +%%Title: CMSY7 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMSY7. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMSY7 known{/CMSY7 findfont dup/UniqueID known{dup +/UniqueID get 5096648 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMSY7 def +/FontBBox {-15 -951 1251 782 }readonly def +/UniqueID 5096648 def +/PaintType 0 def +/FontInfo 9 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMSY7.) readonly def +/FullName (CMSY7) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle -14.04 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 13 /circlecopyrt put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CD06DFE1BE899059C588357426D7A0 +7B684C079A47D271426064AD18CB9750D8A986D1D67C1B2AEEF8CE785CC19C81 +DE96489F740045C5E342F02DA1C9F9F3C167651E646F1A67CF379789E311EF91 +511D0F605B045B279357D6FC8537C233E7AEE6A4FDBE73E75A39EB206D20A6F6 +1021961B748D419EBEEB028B592124E174CA595C108E12725B9875544955CFFD +028B698EF742BC8C19F979E35B8E99CADDDDC89CC6C59733F2A24BC3AF36AD86 +1319147A4A219ECB92D0D9F6228B51A97C29547000FCC8A4D9DAFF1B3EA76067 +C5493B69F73B89C8B61804A34FCEC826343337CCDFFCE17BF343EA8034BF95AA +14C56862C2C052569AFB236E1F1795F05150C8F28DFEF6BF4BCBACB678D00036 +30EE84FEB44B1A8438185EB45654E6853C1159B073E54292D135F0961A64E8A5 +AAE49C4BA9C44156C123426212120F99F3E8B7425752A5FE384AAEF755A8464B +51F015F9E2967477D57B22627D75CEF8AAAF0AEBD504EB46D0289DFC8D86C972 +F042BD88A90A53613DD93D8A7A8460E63D85F6C15C000C0AAEE4BD5130B6E668 +8C9B3F3FFD804745DA1D5EC0AB85C96E1724FA67F9324C59275415182AB48D57 +9722DCF602396AD4B5C075A5A89A5D005C9FE11273E5FBDDD1800F11BBDF6AEC +6711C5633A73AC5DF038BA521AC492E138F7FFC7C5438FFD32FEAA1128C66E83 +0D3AA40665F05E62D7EF00B1B0596162C402A34B6BAE6300D43F3DFCC84860F5 +C0F0F1CE28FC60642BBFE9BC9102E80146774CDC88F9C250DE762D24A3484BCD +1D26B6D9FE981CA5AAB2A4BEDC528115043DC18D7105735D7528C2C5DD89A812 +75B5D7B2E5A586FBB0C061E708F92C1552F64A296490BD0F20243986A4707FF9 +8AB3C917B8DB92F19DCA6B9D4A1DB57515E51DD85D5C9D2CAF7A036AA3F9E9B1 +5B5E099CC05A9126AB274C17D75CB4FAF78052366D2F21EDAADF84B22A2D645A +3E65C4BC0F540B5D9609D88DD0E4CBEEF87C16447D43A5F98528FD45ADD10DE6 +41AEC411FD6929308F0E4F48A8D9C9EE386E920D41C1CC98A52073011DF5BD28 +5683F280B5CF7F27DC50930C81D344FF5A8A9258A207D2531AC21A735B14155B +C22C752DD22AA33C52D6D4D053B3E46FD4C9129068DFF52695A3A9184D04E8EC +93696A3FEC3AEB3814D9015EC14C22EC3ABD5070E8C28A3B42F5596D948212B4 +AFB9978A0A361135C9E18CBDC98E0D1E8BDC17E25DDB3D52E86127E5AAECC55D +FEE61693190E378978EF1BBD4D1AF005D511C7607CCFA4BCBD3EC427CAD82809 +B725B25AE8A03EE88F80A7732A571A2317E0B6A0D072EE8CE2EB9E033CDCC899 +B64CF4FA1C708A885442062F08D3D8DAF44C066EE278714D1486EB709D327865 +A483F62709E89D08291F044325208EBA758DD459481334F5D9AE3BB61B3020F2 +A4538CFC2C94BE84C920BE80806FDCEE394230730E049333A7E16509207514FD +695B5E0AEA9E4A9737311AA0B33B15F6769FF865D1ACB63DC6201C3F1062A3FD +1B446C1857460745917A36289DD57C94FE6240F4A40FBDFC10E91B91B79029D9 +9F1B9C74E8E5AA011A0ECBEC660230AD5929F01D0325D15FDC0040406F124021 +02AE176F4C98BAC1706F03C2B5B40F325A50CA4683B2BB4605E68E72D0CBDC2D +96B3BBCDD01201B650A7E7744D58D1E36D81FBF72E0A875FF29B4C109A1950FC +9621B18D58806392EEE9841794DFD39E3C4E20D45384FE07F9D445F143B922D1 +AB350AA6DFC51FCF767B141A392D6A8B633AACBCEC9F56A0CF40AB08020EE63E +08CC0BE01B40E86388A65F5869F2F4D022DD4B912031CB8CEDEDFC2473772569 +5B28F66AB74CD7902A0061AA3547D13C7F0C6EEEA7B0BD316694A94E4D672520 +EA044AB28D8D01076C486CE456EDA1811F7ACA75D27473080D27D3E681E35FC6 +447046120C6CC4C17674F0F051570A79DCA74848F3F300B58B19018430D99858 +CA5504084D6BB74CFDB635B6866974A9AF05DF201C69352B2663B0623E7828B9 +5EC5FFA8D8F10A7C28000F8C679B180067D5481D6315BF1C4194EB171C8F3CE2 +4CE319975B9E948D907F9F7EEAF07089844391555F329E331D52FF114668B8A4 +80704B3C6AC0CCAA2F5D043CE44E65EDA89A0CA854CFDCB11D549B7FA72EDB90 +D35353C34A771B1FAF96F83FCA5258AAB65384BAFFCE448690C1432A1F749C20 +5817205185F973FA098BA856584753E75EBEBF387FC155202885F5B67117DD7E +70D1CD887183C5573B6FB607D4F6CC9F8B94B09B3F3AEC2EF1E6A320CF6D0112 +63046321941D1FB3F2140B59370AA9387E24D579D389A166A10C989497FE9549 +34E1AC2E546CC06C5308460DBEF3E1AEEB6CBB0FFDAC458E61DE3391480CF5CD +34A647D4DE15B81131B7D1F9EED4C6837A32E89B0EAAD6A05F5F67518655E5DB +224D4833CEC60D5DBBDB8A03FB1A9730589BB4F0FF56191D17E73B9562E0C356 +B188882B36F9505F6F42EB2644FEE125C2A7D12227ABC8ADB924E88B0A9E8DC2 +79762523B0B88DBBE6AC7968A46BD9E9F0C3F03F5F64724CA07782195F01F130 +30DBE895C212E0EE20162D863F46A674D85232FA0DEE69A8DF019794AF6873AD +9CC2A5EEEF9393313CA519BF95C08ADF7A75B6F53EDCDC39851D20E58B97CA57 +A7523717AA1821DEA94C8A9F8B82346B16D92D15AEDC16F0011A45A44B09DE47 +08CBA46E8511D0C5CC83F952EEFA4ACFA7F3D7FA5E113EF6B70E5ABA6F1AD3B1 +E4D3B15AC6D5C3BC70A3946F411A7D965D6FA9D7B6C6ECE19B2C29A2FF476251 +EBF0CF3BF658A1D896323706172746F58B2DE49F8B7E431E20304A42694CCF73 +11C4E9E96260CC442E2938A1E27EE6744C7CAB01634C8210CE40488B9CBD757C +4277B5E3E43C7560291D945F9128AF1F85924003418F96458ADDC5BB8EC431D5 +AC9093D20DEA69B92454613BC1A82DAD4FBF8E56084494D9D2FFABD82A7C9847 +171FE36B265B546F3072B0923840E6C6BB12CA53E05A99F0E8FD4F5109782746 +7CAB9B35B68050230736AE624B7862D1244C7D9BE4D1CAAE21B123D1E8372377 +F1FEF269A9A2EDF02CE0CC8BF92FD7EF09556987B8A3BF6D8C0A663DB6B9742B +E9AC61A449106AF1EA7ACAD40AC6F59427CC51865E6A90CF2AEED8D6037BA70E +4ADDAF622CDE877C98C3B2006B4721FC9BA18E30F0752BD4ACE36221F5CD1497 +8FEDA5D643BE2EE007970A68E53D85975116E6CC09F0039A09EBAF0CA4B0EED6 +A485CC0B69E526033FD1C1190BC5686739CE13D1AE8EBCABC01FCFF26141867C +44ED291196E546369129B9F759FDD7DC21BAF0A528FC34BA9FA8937813953644 +C539F9DA4E55E83DB3D6DA309C562DA1330B157957B18F7618544AB738E25F16 +F0517CD13C1F11BB8EA056BDC575D77CDC526EF497639DD89C2098660C5C45B2 +D7CF715AC5E76847E0D3178360DAC1BAF6ACAEE72453B845B9F86621C166857B +029CEF5AFE29D1EDB4CA3AD7D008B7550A779E0066D7312DD6C7AFE1C0BFFF25 +7B062B0DF30032EA2A2FE3CC46C96A3A0BA1888D1D2B05424A59ABE3EE928ED8 +B67F507EFA78AE128F58B54634C7F534B3D0F4AFC23E38FB56EB39CFA425FD37 +848545EAD03EDC5A9E796CEFB345F527615C785963F536972EBD9CFC4A6A4A07 +5A31A508CA147FBB762ADD198CE36DF86730FCE2B643D1E7DF0BDE800DE7AF89 +44A36B04193E44231E08919EE91A8B559646DC4DFAFF0AD891890A0A88FFA8EF +B066BCB7AFCA409C51889E7FEB33F19A3CB1268BD0EA74AF29C1401BABD16F87 +ACF7DD65A8513DA9995C5092C36A774BC4260113360D29AF7ADF5D22B5B58E7E +A9BEFC33B9A91D2C397B27A81087376CB623318A8362C3FA9CBE3026675723C2 +E711910DDB328E0EE3FCE219F44FE528B70E58B8E6CDB4AAB48237DD933D9639 +E9D4F9EADAA8D46537D964D75C27F210B0C2473CB60D65F61BBD91ADE01576BF +77C49E31936138B0FBA066BE910DE1B1F0E4FFB5E81038E8656ABFF08DFD923E +6BA2AFEDCE6998BBF7045393C34811501586A4846E5B942C8E99D4C481D3AE60 +2796ADBB5242D59F1116EBB828014BD903EF58B223DFD18BFBAAE4D348876B06 +CAC10B7AF0DC270E6702A3F75D4DCEF872F2CDB9470AC9A1DC1ABCB55636D26F +9CB6BF27A0DAEB1F62AFEC12F55F78C9B59AC6DA9DB4B45444B0C582DB4DB8A4 +B31EF4AFB77988E92FC0B257374B4408406490D9AFCC495316D6C08BEC9A76C7 +12371E14417711EF802FB7151B3F6A2580C97527C9C3A0FAAB8D62FD992AA18F +EBEB36F7910186CD5F70A55DFE932757C299D9D2289796769A00A0C6ABD18F82 +E0D4E95D6477E67B4C012DBBD098FE20E2F15C412DD2AD5471A65EACE05B3A1A +0C9C430BAEF4887F2CCE668116B87FFD9DAB4B9B3605CD26E6B12488058AFA30 +8843791A95BB322DF5C47387F3EC72343855D6B23D72144EB5EB5157B8B238FD +6C71DDDA64C9539F66A7DD569FFF43DBE4A8F0608A3CBD354DD9BAB5E3C756DD +92C3C1B3E169D86A2230299432488BC04A87E08A80809F9968676DF9157B1C91 +27C664ABCCBA9997FAD8966F766B325086899D1FE44581FE07C97688B3E15B0C +234A22646C32BB965B9BFD2CD34854D1488AAF021E169BF9CA9665CF040E25A8 +16156C80A2F47397CD370AAEDA731E0D14FBEE1E51A17DB972D96DBCCE33F937 +5CDBF1A650BF1D3536BA4CB7A1CACFD5CB457E2368A660A62AC26E64A631B2BA +6B08EBE42E02D9B1B2E95BF9F0A6B59C96A122968FD46A4D17BA3D018CCBA0F9 +80BA3C1E6C683111AFF79303CF64F1D2CCBD7571C6E09DD9B27B8E101BE219F0 +E075880A0E367885AC94143E777DAE455B990383100EADF786300602C2CE28F2 +4F44662FDF03BD39A5181912D8F1243C36FF88882CFC4B34C1D4EBBC01D96A7D +9CE5303042D1B21042E4FEAA455F22A01333FCAD7E4AACA5D3A5386331985F6B +9B247EC6310BB07507321BEF3E4ECFC3B915AAA6E029B3999644C987640863B0 +5DCF58CE479497AFAD1208FEFD1796E74467E9F7867C313A3412E6923F4C9144 +C69EFA17965056DF043DB465BF2F1E191706D3AAB47E6AD5C9767E4A73B29F2D +E2E579D0262237568F82B360ADB6D0219B7535EFD02DD0688CDD23D84FC4F308 +5D2D0010B1A9F4F0321A00C154672D21708B66B91ADCF98BAC7A2F94848E9A4E +86CC82EDD0399BD9F13E43359E71F80086B9B0C3B6D08831D4479ED83E7892C4 +90C477BD1F06DFEBBF60F26516EECDEFE4787EEA8683754F2B257D0BAA607DBA +35EC6D1618C2FDF3881827F92D793ECF152D761F2423A96210F582DC9B90120F +26A33025414716A5E6F56D712E31BABE5047EC4855B767AC63D793995C9E074B +6E35C7E5255FBF4C3F17E7AD7B2A6C5F7459794FC94306B581536910F244BF5A +3158E821CE75F4B0565EBE985DF24DAA92F9C1D848EEC6B88E21FB6C51125872 +1752F7352291960E5BD36F78AABBCF6DAA4D07AF56E4B6058AAB13D41BCDAA14 +C0D63C6807FCD0E2B4B9CC892F224843173A75DC53A8F0FA396959C2E2CFE3F5 +9B1C8B62797F34E7A0BFCF0787C73FEF98442234A617CF161829498035D30B29 +ADFEAABD0B496E8A2E764D22DB7737F950FC5982F1C5F4FD414C1B0202F40FBA +62C81B8F0E836CD73D79366FD62388B437B81FC673442EE34BF27454F72A08F3 +389E60CE28A050601A42FB4491C60DC02EC008E6B9DD2495522BBEC7293E2923 +120584E88412DA7137397B41A28706B1CC6BB0C80709A2A4BA79822D245757A4 +3EE454198942ED2316FAEB981F7615E642167620EBDDC5B271E273216EB119C6 +4F2F0412F0BA6E3BA396217597575C6739194E1F839232FF088FDDFD3695A5CB +9A0E220389938596D8BDB183138E1F73F64512E4FAB5E1328F9B42364E3113B8 +004BE2CA0B074EE271BBE0260D31CE555D535C16EBB528747EBAFFF253E659DA +3A377CBE0B296276AACF0294CF90FDAADB4EAD5E2F600E5B2A018DEFB86FF61C +84296480A425687CCE37D671472537E897AFD4B8C6A6175E1ADDF9AD24DFC5C3 +A73E18AC2D9B28BDA2F17D51DB3521945850DAF0EE48B0FAC271544C1B4F3B2D +53BFC8DE32BA366FB1FEC0DD6C0B1FEA374CBE2B96F5B235A1D83A240DB442C7 +1460980A3E5B96AE3D5784DE2C2DFFA671E0A856DB2FF4130E5905F3D5338856 +C11A468D867D0C6EC585F1AD3E7164B8598BB59973B9A952FAE819F052A6554D +EDC342BCCB0525905D1D27ECB9EE43847B69AE116F494CB2DBBAFB2773F1A3E1 +C75FBDF8D66FA5AB4005757D631A0D9424FCDA91A1D2AC6FCE7CC7A23E84C65B +3E92BC684F23467DCF8521E0E27CF1441C487EC6E3BCA0AB54BB137E83776009 +833D772FD225E88A8BD992FD69819B3BA90BAAD1DDF16E4326190CC4BF9C30F2 +AF7CA1FB38E6387D9745FC5E176B248B1581BF7A4CA2FCA8E423DF340EAE29AA +7E07A25FF838F67378F9A6A9A0B404E01E86E64FEF71DD3D540D4711AEB1974D +E2E0D485DAFFC74BA6B8E9AFDA245BC8997BB39BB6BD52B496A09C68F7A8E900 +8DB3007643416040FCEC85B407EA0A946827771FBBEE49A3DA5542CC5173A31A +0280AB8E922C23C1BDD88D70627EF124633C318E7C9ACBC14AE216BFD41C0B6B +3A0161757913CA1F7B6626963C09936A52E73DD9B3D86DEEE73C0293A646FCF1 +21D4C33DFF1671DA7A53E77E20233EDE51571549AACB7968602CD03EE67ACACA +B231661CA9DA2BEC5795A83DFAF675E9B052C8BDD51490F7874C91EF5ED2E0A6 +BE9CBABB98A950F7E55DDA3823036437C11F614E27DA5BB8BC6D955FFE54B825 +0201275C2C49A3908BEF1DB3D87792DDFFED23DE7FD9CFC284F6255C77E54A39 +C2FDBCD28F2938E4CC135829AC1867CAA5705674062C9639FEEFBE49D6108091 +7C58585B80464F7E69966D7933C7019BF336B88B9E0E7073A85EAF297B71B303 +31EEE9121347A482D28CCE942AF53E94F88A97EF2F1860A92CE29A14495D67B7 +D37E207D42F3891E0423F5BCFFCAAC057FEC683696ED6FEEFA65C8FB6F1312C5 +24A1130192B4179F3B08DA1C951D988894E7FE7CFC28C56992A1CA82BF8BDBDA +E021F16E630FF67201BA4DF5F3F4D6AA65B8347FC1575C142C6C1868E8472BD2 +CF191137AE1B36F32FD84DCAD50644AD55EBA2694C93BDF984A5C9E7C92B73A0 +26769F00831537266FD2E711AB3F8AFC5F3FDA3C9E6439FFC48C3D1B5527FC56 +1FEDE991E66E8465C0E395EAD0A22A2FDC001E449AB9C5E0EF187A1DE9B74696 +BEB6A525DBF3A60DA2FBF1579150DEE1C5D1B6F55FF2708CE23289803CE123BD +C81E25DB96551A13AD713D5C7BFDD3F2E1D5C12463A195442B51909CC1724E50 +A1F6F4EADB3B7355908F36F88521F333C4E7C70B094209D1F883B961DFAC32BC +8C5A2CAF77CA5E6AAB714CC0AF2B42FFF6F73301FC71AFFA9B33A2153F55C2DB +C1C111874DEC37CB746BEC9A3A9A37A2DD098CE7C66B0FE38460ACD77A47D53C +1550F857FFB733B5A8D02FB56790A09190B29CCB4F4A3058B1C82F0CC5E1B2EB +2F8E06F2DE531E1EB81326A8EF0F82843A4AC59D267EEE45730895752820BA93 +A129C22A78C1AB28BCF67AD5DF372FECC9EE6719A02E499FD5CA866688E86089 +7EE8E5912087E0C4588DE38428114785E0CFEDB1E2EE24CC067D107DFDF1E2BD +B1C4F9C6B740F3DEA0BD315581004E851ED5A9F66C4F9E95DE97D355DB06F482 +A43B565F1255A85710B15A281E2F034B1C23FEE6CDF3A043780CB6AB18A016F1 +9EAFE545CA5A5B5AAE2459D69D2151E99D029FB5C1649B9DA784BFDF7D177385 +4D8B16B9922D149FFF6B4F99311D52BEC9A9FC098E7192180DBB38767DA9B9C6 +E8CFC98615219EF3AD4A8157D14C72BA3F91C8B78381383E0BCA1A5319749E08 +D67D1208C693A6640D1BD6B9285AC0D3110CBF8F7747AD12585141C3248D0BB5 +BDE892F91A4B3291F21F30294693518E4629A3ADD7C8640E424FF615602C603E +1E14DCB3B17D34C090BEAD2A523E260A599522329B698729A635CFB15BE7E79E +6A34DAC7C3ED57340821A4E7A0C7F88F64BAA69BB80DFA8B659F4756878E3044 +E1DB7EE4FE60074AF97818B3D36CF51110191B4E10655F52CC5D11F56517CFF9 +04BA2676F4147DC8192C98C555B00F6C8614D23084A0A70E5D9D0DCD118ABC5B +96912E91FA3F6A8B9A7B83BBB1BD9C2044B5335493C1B74A8C85971E454FE379 +C971CC567A316AA11D6745C6489340A939E49D3E402213EFD8AE74ED4DECD30B +3395BAD7D65DCAB84EF2B2BD6E178687EC870720A395B69F7F7EDCF23C703F76 +42509CF6CFDF972BBF78431564286903743B208817297B75D9F46F272A3F78C8 +9D302D05377EEF4C4109087264467E3E76615D94ECA5E9F4D1C6F05BF19C11DD +D76D595AB0A57E455857E8B1EB46CE0DE84F08D145CDBF6154E83A1CC3EA7E36 +D05BDE406BC36D8B66DC0E03E487 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: SFSS0900 +%!FontType1-1.0: SFSS0900 0.3 +%%CreationDate: Wed Sep 12 2001 +% Copyright (c) 2001 Vladimir Volovich . +% See the file COPYING (GNU General Public License) for license conditions. +% Converted from METAFONT EC/TC and LH fonts: +% ecss0900, tcss0900, lass0900, lbss0900, lcss0900, rxss0900. +11 dict begin +/FontInfo 6 dict dup begin +/version (0.3) def +/FullName (Computer Modern Sans Serif) def +/FamilyName (Computer Modern) def +/ItalicAngle 0 def +/isFixedPitch false def +/Weight (Medium) def +end readonly def +/FontName /SFSS0900 def +/Encoding StandardEncoding def +/PaintType 0 def +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0] def +/FontBBox{-209 -321 1528 907}readonly def +currentdict end +currentfile eexec +D9D66F633B846A97B686A97E45A3D0AA052BD0CE60552BD63101D7CDBEEF5B11 +69C468645FE4ED1AF2541AA0770C1DCF81623DE0ECDF49F2B522618F650CE6CB +CC8C21885DD61AF8A523AA677EAEDDFA51A1F9B1885EEE0456196D634E04EF89 +F17499DAD982502ACC349B9EEAAE4A71A73D1147318C60A8BAC10510DE90D8D3 +F46E47295D27129A5AFE0C65E22BAD10D06885A2EE623FF8E1D90287A083E00C +EF25195F68A2A98170E48759F267FE330B507C15D01A48CEA1B39561EFDC256C +5FB45057A54480A0B2F8EA705C884B2BAD426156445992C03D7C9EDD5CBA2322 +7445EBF42B99561B111A2FEAF0270F82F2D1888A340C6B27130B764396FAC49A +E525A0972590D8C963728C8C8DC427489CBD83371D357A87DD07EBD71627DAC1 +A13D2398C591F9D869661B8069F3BACCA9B3CF6F202A12DCB2BA563924AE5FD0 +DD40E412D8F90A8DA4CFBFADA63F48A504F75E1DBE685B708F61E3DD7C8613E8 +BC7A623D7E9A9DB0C57D538D6E5AD77165BF4E1C8E56DF1D5E809264C051ED46 +49D407DA6AB0CB7CDAD6EF63161653BA0D4B65853C526703878C907C82A23191 +601B75C3CEC0844B5433467732B64F4E9302EB585AE595266395759ACBC132E0 +BC10F0AF8D24DA5F4058D6A19233B3A89F30BBA72744F69B5F3984825E78590C +8DFB3F5A5FA2CE4B728B5FA2477E68DE373964BF7B09DC803519221A80391CFF +916BE5655C26293C30215165D731198DACFC66DD5AB418C0EC3514C48D96466E +0D48A521885FBA0391054575B8E6F829987FFC32802C8B3945F620EE51228F1F +B6F4F1C0296BD8FF18A07E3852AB1189D4CFAB335B14A7BD52FB4E16727D43E0 +51EB0055FCA6E538D916E2677C6C79BEA58EC3C20BCBDA1C5E2A8ED9A460C2ED +90619D2CEF8746D9C6838E4639F9CB21F656D85FA3FF222AB769600F2EDEDC0A +75AA798540DFB939F8F013F5B64969DE3EB35A3B6AF2286E21E07319733165F0 +7D86EC04033C26BB33F77B6748129E8EA2E57F742EC8DD05F16943E966B98CB9 +8BCAC7C6886E5A19E8D9AD2F896A111B89C2205A24B234B72ABCC2B8B3D56BB0 +58314AF80AE4C679368A13C6477E31978CEB2E24EC8EFB6F2BD05FF521C40018 +7EFB5737467AA8B28B73D53F872A5FCBF8EF6BEF8F3292950495C8AA9050F307 +7109AF1EBF0C4026FFED9AF9C3E089B07B4D9208383EEAF0BADBC51037DDD271 +CA81513269AA3DCD7B3B2A0F8130FFECDA4E041CD6C6C0B29CBE640B989E0752 +A84CB136F9633DEAE9CB386A900C1CFD1FCD4D690A01F044C379DD1EAE2BCEF2 +1BA42AE6863E2CBC2530816D0088780E87FBCAB3D34FDE93E95E824821FE060E +59E1F62677F4FEB7F5DF7EB0093FF36C008CEB5031624A248C26519058625F9B +E81DE69BA65DBE9B3680EF0FC47281BA7C76D2AAA41583525D6D9C9F3B820BA7 +435024D48916E6A19FB998F518A075AC7F0CCF44018D6F607603C23FCE4DDA34 +4F508C055E57C2FB2E1B43B981A9798BC43C88D28F580DD78AD06DC1E54CDD8A +7B08A4CB7CACB6A822FF706460832D64B6BE0F8D906FBF7421CC7D762078DD3E +5018FF052D11E5A12FFBD12A674BCC38AA87ADF9426DD6B63D259A918CEA4AD8 +DA1874A699B690FBDEF840106D97741D9FE5FDFF435DCDDB876051DB23A8F721 +B06623B4670BBFDE5A850CB4A138CD5AED4267096D6F5B59E434833A89074E8E +B7AE95EB52EF994980B921C47EF74592B6FA0C9073746B8C18D9E2F5B41A4BF1 +D716D67A246656438AE69E93B1997469D73BE829DDB7BD974001F9357A68A158 +BCE4AF3226433159BD59249552FF9A88ACE586100BBB3BF729C2B6B24C0CD147 +63A3169181822EB234408EC343D683908221C629994E37BC2DB86E5487C9BDB5 +03D34358394703D21364929D4747369E037E8198D27394AF7F5F27FB6EF7A00B +678192B99E415473B4BEB3114F7C6C1C8CA14B8BB7A286078BCEBC6D07A2A386 +20215298FAF5E9129EA14BA8FBEDDFD0712384E734D208957ADFC7DEF83F5E5F +4EDB351DC26BB050BE17C460EEC693C16B0542592ED9167A5AA41B2D29A6E2AA +2B1269FF7BFE1F8C0DB136501C69BCC4C6E8641E138B4D5522B2157B9068F1BE +B925836A24CB08929FB4A43A75A5D45FC8B9C52101ADD20399EB6F2622FD2F44 +F72C571BEF9E153A956D5B252A337AF57456E9DE9629F2CFEE7B8852C4C86D6F +D37C5F0AB3CE68642CE6C07888B890196AC9A81DE2280888EB37F59B8B90C463 +CFD7F9711B0FC094123A06D04576A763849C3EB885280CD0728B5E7C12E76973 +C9A662D38917CAF5511820BCB2E11A6248C0756184319E202B57F237F21C09F6 +1DFC9FD34C706AC27CFDE15D9E907FECF47F5ED5A61AFCC8EC3338F70DCE66DB +A7DE8C8EA3C60575219104394AB80C5A40BEC2B83AF3D62EF1C530875138D6E4 +ABAC4242E7D82B72DD8725B9138695EED5900EB8F57F1825974FBE0C17422B71 +F1FCD7A0D9281DB363A45BC38D2D4339EF9B2387C777AC73620B6602B27D88CE +83FEF6584B03E3312C109EE5E868EBEEAA1C39BDB8082C1132AEFD99A022A2EC +E49FEBEF1C7ACC0D84D67D21BAB7AD3B0F954F8729BA682F26DC1906CC507052 +4D1E60F045862F313CB6D8DE936E063A86DA6CAFD68B8E32E60C749802BF674E +3EE182ADB7EABC054CCA05B51212FC3C15A2E238F79E02DF55DD190619E0BD11 +3AF96A2BCF82346B957F7C590021AA644B7F3A9A0151A25A15344E93FE87E3B4 +D7A7A7804330004ECC5E7607EC75EC3DD9766FF4EA8F994E54FB3169DA2D3A10 +A63A19C2DA0DD7EA52678321C6A5CD8C1BF8E53ACA2E51B9B33866AB214C1081 +8B3AE853316002B4885791CA91048469BC43853B83886F115E57BCF79A312041 +1A8B15FCA918613FC076EDA0A940DA3730F390F61C28B4BA46BCEB1E9E0436DA +0F2BD80378AFA69E24DECF644DF92D8B8F52F5BED9E09AECA851AA1A7A7519E3 +43C0DA13BC3526A7961496AD20749D58D428EEFAC440EAD6EC6737949FFAFFC5 +B4DA51026F064598347DEAA7F60CFF85EB4042D652111D4EBDF7E0C87841CC99 +58BB44D6204A5144E1F34B481428336BB887A4546020FBD608AC12BF6325CF2A +1023D0E037DCF90962F96C215CE9B0DB5076103D55BCB99FED4DD2914F5AAE6D +6DBD7B725C804C8038310EAD5F4268604A45B3255A254A95E5C1D602FACEC94B +27E3F46EBB39E55DAFC1C94CF72D16E67673D0641D883DBBF22EB53BC8ECABC3 +DE478B07CEA2E78EC53B2848E0AFB5D74BFF42EF404AE00836828999710B974A +0CDD161BEDB622A841EF73E59F2CA466BA0C061E8509A1B57B3F8174447A9777 +CDC9C4D9CB1C5EFEE8DE9629F2CFEE7B8852C4C86D6FD3780010399868C99D30 +E2010D1ADADF1135C69265488099609475F605FD9690C2AFC60E8C051053517F +3348A94380C39AB210921BAE4A70AA4A2CA44A6B898ED798EE0972FFC24E1AF8 +566A58B62F7740A9F04D7B3A3B4B66839BF3A7B2BB5CAD740FEDC3E27F14268F +B10F586A5EF2F3E5A74743FE781F1756CDE5D4148A1651E46B86F75FE172D9B2 +6A3F4D1119104B8C9C295586F4932783C61473719233F3E815B4DEE832214A31 +72F3F13153527AC2FDAFE6979CEB210E41CD5334171F1E7DAD437A201A6E0DE4 +57EE415D0163CED17D9A124232E894752B7DA94D3E4B6A878410A0FFD1FF8879 +D537D25FB79FD116F8BDF57AFFC3621062398BB6CC8ED05DBA6D32B177EAC39E +3881138AD5B484A28AD1A541F29C163DD4A6FCA9D574718B9BB26D8D65DCB198 +029969E7E6683FADB4A550118AC728A1DD434F06C029F590EE85785A0FB48B6B +F3DC2DF69EE874DF3C2D253A93E9733812FAFAC971AA607A9CF032A8B094C1A1 +3C6E697602FCF0DA3B4FC19AC4852DD08477920F918509452B0FFB5095F90A64 +753C9F9A46B04078D23572F83B2966D43E3C18F9D95A5174F5856DB02D336DEF +A83F44C1A8474F38B234B38FA876EDBE856AF3E130AC157149FE33054339C0B5 +9D107D19D013BC3526A7961496AD20749D58D42058275294214481C13BB3EBB7 +CF6357CCABF806295563DDAB4619F6DD76A9C55F64CCC6870FF07329372A93F3 +5E5CABF73E2B931F2FDA98B0E7F904534CBC8C49FC0FCBC50FC7FA6F9E72C238 +A7BD0E5FD2E9D3B5C3729DC47A79E7B97B627C01ACC5658C9190CB037F84E841 +C65ABCF7345C51D692405CC9916BE64E01A4EC94702850D1453A1CC3F96D310C +BA86F8710A00232636239BFCE6955FA74ADEABE6ACA9EAD96CF2C11B51CE7465 +DA56CAE271F4DC09F71F2B4C1EC7EFCC33EF732B69E3A51D690ED747B1608857 +9101F08D1EDEA2AC078B18E50243714D42F90FEBD544059FDFA4311C72972E05 +FD480974B2EDDC5DA3F10FA419AC2F519FA7AE066372FB176C960F9B884EC295 +EB95236B862BEA4456AFB049EC9F206314F75C6300F139118D8ECC4AEFBAB4A4 +960FCE4C041E632D563A4BC5A1E52B8D3E938C8C15B97126727A0865EB589D9E +455C0FBB29DF2FB185495B3640FEE2BE932510FE68DE87995883C1D52F763A92 +3535CC35763C2B61D2B298AE0EAC194988A5D1498BDF7F4EE6D268670E4DB1F3 +583FD654B10DD6DFCE20133FC4D6A4317404A4A181A181DA9BA3BA2DC87FA83A +FBC507C47C686532F38D5CB26E31B2878B6C7D10901464A7DF71EF0634548E83 +ED56B9C31FE2F900C2CCFF1FC5A9A0E5E36BB89C0E13B5E67C0F3670ACF542CF +0F55EB2A2DB06B4827E3731D565C0EEA302636292DCBCD6EA6948863FE892A66 +C708FEEEE58CD87809872B97DBF0D29804EC44BF73DF6C67F68DACEB9709C38D +7D6EDA1AA0D91473371828DFE09AECA8F630484780A54250A47476D67971FF37 +784000E617168B1E02AFF0FB49045A57D4208E705ED9770942EB880572BE1D5B +C14661EE1E9CFAA6AED96BC89A61E010D6ED3EC8B1390CA2AC2975BAF2A510C7 +5C455193E84334B24497218073DA6929CD8D0028621BE55FE70B5D0C5BA4A4C7 +8E41A4025CC66646FE53991D52A7C53CE895E27E5EB32D90459C0B38EF6DE603 +9F26A4135FEFC62A95867D70D856D3D6F9D10E1C74C825AFE8B024229F805C52 +34B353CA64927852E3D9861A8675EFF467E3CF54506D6705A652CD74AC3450BB +0C9B27AD6231D5B85A76EF8372A74DF7FD8C17AE5C2624F8EAA22E9583921462 +6DB38F42455B9632C83A959B67CC5AFDC82F982FDF8B65ADE4859FF875BE0B0C +5367C18CCF49A685DD573AD8350FCEB9059D0A12F0E0343CF5B2D4A9CB76E39E +431F53CE092880A1F0085FD69CAC31A111D98F2099F0E773654F0532FFAD1FC4 +C4C8B8FD51449F12FE84CC0110D6B60B5331B03EFD744488D88E52D800FF9472 +A2508A3B5916C1A6B493573BE9D3EF8353076A4D3B4330A982C30AAA3D1325AD +7777A4FFCA82C515592F7FABF53BCA19B6B17D691A27D5F1613A6BB4807832F3 +684BE5F69BFCD3A034E320705C92CF2260A4EAD5B7E80EBD64092A55516EF78A +E5FB49FF61385FE0ABDEE0D89E467794927966A736E3620D02BAED840C491E4A +0060DE7BA3E21540450C55F67F0ACD4DEDB0546755821A036D677AD25547888A +A4D2C0926F8802E58D0AB34760F8D136DB14CFC2C2ED3148BE7DA99898D65C4B +163F6789906D3C60A0A3A9678C8C5DBFAE8CCFE98A1C1CDA82ED6B602214112B +AF5E8AA0939E3E3C72970547609ADFE0A7B7A8EE705A2181C934E401E94FFDD7 +3F1821CA3F18C0521232B40E85004859E12900139FFBCDD7F98BE18B1909EE55 +639B97195D570EC5A06EF45C2697532124FEFB3E825CC191839CA648E019F13C +A5BCE2167BAEF2EE57874A7345705B70BD772C3482A1AF98A63A5A303C846C38 +3A1C9ACE9FC47625EA3898BC7643EF69F883A6E0982FE932B24495CC5C643833 +57F6794E4F864387FA751B0A47BD5773F54A9C01458DC3CD821B995E240815E2 +A866A4A74D6EAE3CC8EC15F663E992BC02E9D49917F01C5AA3BA5C639013F87D +D7E4F24F9E2A82FC866ABAC2B6E2EA77DD8039527CB16E5C9371B0D637FFB03C +91ECA5B1AB8ED241AD2F9E6CAA4ED71A1A9AC78EF9EC472254ABFB157D3E106B +9AFC30B317EB57BB11BA3E39071E10C625D787C530D9546201953AE7D54E2E28 +A44A62AC673985CE804E119E30EB552A69B729187C98445FBCA660CA6CD43A26 +28E66F53E87681D0F632634F18648507B20C3D567D7177CA4358E2D206A01C13 +02C921477C62FFB88A41A772BCECC4A90327065F9A7182B34610E4CE877C4DE8 +5FAFD0906D681616AD49BA595940A15216368E1BF92230A99B1AEA30BE62E393 +5B0E15BE2901A684EE4353A0ED1EC8878EA229F71C9EF54BCA0BA817A02E1AC8 +82503F4DD682DCBC9C1ADB45C05739DC3C687CB28205C6155EACAE363507F819 +7943D903944A0D64CB893871F494B11C9F8EAAEB1C01FA9C2B9E0A6A9D15683C +63003EE063DD10F24A57A8797C6B05F631337D8FA4B433F58C44DB6835A27C53 +8A53BB54B0298942A1FA7F8109E7A6C1C4A16CB1E3B1BCAA86C28A11486C27E4 +284F718214BA6AE12968CBEDFF51C6FE32A2CED3A6A94C8FB201C9E18668AFFE +056C59C18BCCE62C372BEAC1BD60473DDF23A85AAC7C00EC290A9C2B6ED06F06 +1D629AB9A4FF7C70F67602E3934BB8DA51B8108E5C5376B21EF5CBE7EBAC9F93 +B1D49BF5BEB2330A4DB1F3583FD654B10DD6DFCE20133CB532A43863502E8A12 +727F5D50F3C73B53F8F68349217B65A2C6E9971456C23EEC3B25AD3839D7830E +DDA635611B705D61C42BC0CDE75D6FA957CDA8989F3955864E470E9E6C8145B4 +5A717F19085186251B59C66A760B813C240A07F5ABC6512A91DF1E449C5D223E +459C1CFF11FD40617DA8ED1557A25B16D755162BB65C9CD7A99A0A499FB001C2 +E6545F2ED526444E8468ED6AF03CA440AA671D883DBBF22EB53BC8ECABC3DE47 +88F8B0D8C342605941EFB5C5C8813CB052EC54A5F256A69DFAA5502AAA0FA2AF +035F56B864B065D684C3BA99E260252AA2E7CD068AEBE1456A7707C4F300D1B6 +FDFEF441259FEDBBC4D3C89BFCC2A1298C3BD0263C6864DFC6DF3A7FA56CE538 +54C664F25749580E218242AD44FD99F5E412B4135A0D606A8C21103074F7CB3A +5E4A41125B96B6B3108297F5239AA639312E6DF18600E51B9CB7BAEA8C242D45 +A0A3905AE977EF13BA0604F23CBADB6719AE00BFB9D1ECB790793CA2C932DE3B +534D1676FAF8121DF0C1C992AD4C3FAACB2D4CB6D0F823D45B6333C19C9298A9 +6350E4C62975A0395BDC386FFEB2753DA85A746A6529E8E7D648A8A8D9BF3900 +9A3012E645CB61DF2B4C160B584FBD4508C6AFF196D57E8DDDFF72E156E8D351 +0D3A2B184831B46490CE0F1C4EF1ED238762882F25B0A63B86215A2E3D71E07C +87C7C94E903DF03AC04C151A69B59E69D6C55924D2B294AB777DA4E020F5FF54 +CAD093CEBAE2A45D2CF47B0103ED2D42165F6CA7174C458C57C298A71A88052B +3E05FA9E962DC65A2F33BE99BF7D04B70E643F385D6B52E2CB49162985F31A70 +A9C60A8285F18EA65083169227690C81360BFC5D2630B7DE482DFDBF731E4AC5 +29026DBC56EEAA9623B8674D480788FD131B82CD491A14660AFD1DB39C4C324F +C86EC15CFA7AF146B60F049863F5E79D25E3EA85A3B09186258837404CBB2635 +97737DFE1DC582ABF78917CBF49006725185E3A6584356E881191BCF9C3FF4D5 +ADD351ABB6B75DC3FB4103645D5B24E1053D72E8324E1F7E77C7A7AC1B80EA85 +DD29E9D85CC0129882F8211A177D13FA1453DB422D96E608E58E291B99716E57 +47EBD84BF50C1FA080E251D1C143F5357833A0CF88ADF551EBCAD841B07FF8A2 +61D42541141BEC6B7B2D1BE56A453B9D769B508388DD936EC8C33A32A1951BCC +1CC6D80E28F9C32E5068F9F18B2799894825F3C1727C5D7E7F33EA477A7B7522 +EC6B606CF186D582DD972AFCE8CCB1DAEB243C4231F232F3C81073D07149D63D +DDB367F34D81081C111B9BC5FDEE25625CD8C4A36916BE2986C124C0DEA68AFF +32182928CD90769C4A60FB31D933BF448E0F2F424CF33C0351FAC0D79F9DB3B4 +62A403B913E0DF73571A10BF623F5E298698428663E94AA30411DB14F5613DDA +0C417C3C5A2F5AE58AFD4A3364A47680E82D0D6B58ECF01242F27DB8432FB415 +168A1FF84A5106269C920CF3D64F952CF68A50A751DD3ECFB2DC8313F060F5CC +88725F37CD5EE9E04D2145888A08D411812885076AFE12EB7C12A5D3EE9D7A0D +757CBF5CAC4771C8DDD31A7B724238D015D559E1593E1DF8C7B0EABD18AE4181 +B2DC502A421EBCF15C2C6B88F47E9AD7B0C263450AD5B57549CD23EB09A67E84 +3536D98526EFFB7971E9A5648148E49434D575CF2EB8BA6D03BF5EC2A9B1FB38 +27C105700ED744635406CF60DC00C638AFFF8F24F9FF11FF9D6B0C2B1D11EF47 +E35CD43BECBD46E339E95158DBA2BD4F9A977BF70890C6612BF7608DD8EA2571 +CCDC256C2DB61766B8A460BF18E59C5418E30FB914B9AD592071B5182ADC0AD0 +C2DB964037A2B5C7189C9C1B6FCA7003A22251C41DC352DE3EED3C9B1CF54831 +588D2E7910A5AABF7258DB1902402B30A79590B14D35D616A84454D484914361 +74C32B386702F3463D7690E38E858FA575E081F13919E1F7E84D780899439300 +327FA0DCC184E09CFED23A95110130678BB590AAAAB53E9AC55E4C2614DD8BA2 +551F6244AFE7A39E42F2CB6018E77B627CC36BC84A643A9E1BA1DED38FE782C3 +0E74D928B685ECEBF7F9D31E0C89C59688335ADD9D9068C1AF5E5447B6F07B72 +AE5997BC61E6402699218B7057B1E932124F3FF0BA53951A3A1CF480433211A1 +1DDFF5E7E234131ADFCA540B1FA77F4459A49FF8F652D8990C697E4A864D909E +E651C10847C2FFE69F763A990CB7D6A85629BC054F08B241A2D876C6553679C9 +C885F1076AB4EF535C6437B8951F4671985B359AF11686CFEE325472EE347D79 +1DA17756BEFA39EFAD212514F13D5AE12274CB1F6D36305F03FBB7A55513E9B3 +EF100AEFF7436B804818A020EC6795F2FF63961333FFC7DDB819936EF485102F +87335C3EF5F22B525669C19A6CB35131028296333131C21C47773024B3357D6F +9DD218082A55F75F9B7982C96840EB5FC4B33549D6BDBEDDB498AB4354E45FA2 +0F947E445DA0690BD915F1C750E36F4ED2C8B12DC3195345C5BA1710190834BE +40BC5F4B92B4C5FB706C57953C72BFEF25818BD35EFC5DE8B7B4D66F7699E650 +8B202485E62BC044BB8CCD9037B625F6A9358D47B65872E354ADBD35E9BD0744 +CCF1817CA388D64DCCA287EDEB4A1C38D39D2E15EBDAB5B884E88F385893B0EB +537564269CB46CECE7822E400741715A51FD6393AB5C460F66B6454906DA9E1A +9337646F59A60A84D4E219146060707142E47BAE8DE41B6C9F41CC0A17F3774A +9C8089A67664896B6E229EFC63EEF26EA5F5181DF0236DC17F0A9444461CBA92 +A60171AC8345E11C206BE90B9CFAA313154E5F08E21EDE64BE12CF0AA9479F98 +B2CE73CDFAFD0E58B959361214CFE9F3F16DAA0E8922752F74D35BFC13CFC939 +1C5B422C50587983CD0539F34F233666B70AF61F10C98D6D820F34604171E87E +D51B96F6126CE4CCF74BEC4FDCC7568F80E52CC37B18D458600090DA30522DF9 +CA931EC96C26E8579DADFF4B12782D58649C71A069D89ED406231E5FA20B2927 +3EC50307E0A2EFFDD63C56BE98BF43D2C80B96025676D33BF4EE1A9ECC5B3712 +45B82768A2128BAA4244B8EE4A81CBD681162DAEBB2E46C655A217090C5FAA68 +EE5D81264A2D9D513073C54421BD0D264AA296D7C06C205FA03D57E071F58D9A +609766BD25446E4497149459D43981012CDFA0F63A4BD12B0C42C254FDB64562 +915C823E8BA48BD198BC90CF51AF625C1B786EB6C2789A79634B8E6D47E986B6 +CAA9C393A7CF6EF9C1826D530F55C06A20D2D842834410F5847E66EF1EA21EF3 +6BB7C48855901385E3C4B561DC0983DFC5F77C03996F43848645FB8A19811DF6 +0680BD09BCA1A67C750B06D27805B52F7DE93E53F3693583F5044D3F27D51C9F +0154489B06A1516AB14A3C81B1A240E649AFDF53E6EEDDFB1EF7C8001B5F2476 +A240E2BC40C135BC120B77EAA9B7F1C19406F2955C7EAB22B8EA7551E296755E +B5061141DB739D9A0D64E53784F4948EE85C49C5C58C32362B07AFFCB63CAF08 +561B4150A473D7D8093A29FF5D2D02610CEF5B588273B29A2B8F481770FCD6F8 +B6899BD5F63F58E1B2BA722A4F6685A05432BA1F09D77C1D59DEB93B217E4E58 +4842783D065A3C9B06A5B472CC2897BAD84257D33F875096F0E1927E82BA0757 +35861C05DCE82D54D677580972094094E8ECF454EEECFF2BAEA5587DC1857030 +87BC21AC6ED65A3A0FA0D83967382C19E95737C55ABCECB7B9FDFE91AD2EF743 +CC12FBE1797EDDB89B18ACF93A0403A4EA1BFF450F8AF591BA51CC5A6B59CDCB +C74F6D2746F12FEAC224C713132AD3A718A9363BDC93B9401202434771E09D92 +D8ADAD2D388E70043A8EDB7A0DB52C528D323839ECDF8B7972576344B63351D8 +EE6C03399AD3BE2F8C1609974683EF6C4B713D850C455087461238EE99BD7D9A +4B470874AD8EDB71966A5919B9B5E11CC0E36A1CCC22C262183A25939076F329 +66E6F258197E2B6E9C22BD49F022467E7FC9353952A03DBCC9A1816DFFF2D39C +81915ABBB52B0ECCFF39A2AA1D3C3155B0DDC360ABA61F50F26D5DDB09EE43DC +BB24BE069EDE91E3B36D7A7D5E797D460156B1C42B1A5DEE6D91FAC9560A4415 +51A1B122AC2E63BF710C0FE3DA330C4BAC8AD5275B82F6C97AFD6F9A3EEA79F9 +C0B4B7F95A1805B5CD7C2551142FB75FB9B644A8C37DC2520A044158DA125891 +6EA9AB193A4AFD8D3372D1C8DA5879BFCF763FBFA59D940FEE3283F03D26BBA4 +2B1A33513F550A5B31663090ACBCB2D71044A0B1CE7ED65454EA7D304970D314 +DF053CDC9262FC9CC67E5F0C6FA4448C84ECE626D5006A3F56E0EB162F534973 +B31E489E9A33786894885ECAC6E53FC3623948CE227D78EA0CAF6829183FF329 +DE55E951D800F41D903E0E204EB68C1214D956B209BBB913A897EADD1C5C2430 +E8144EC0389FF334DB9DD443EB6225CF87C455AF47ACF62591F8F855543D130C +C0BC0DF266F9B8DB67B6F763BE811791A994FFBA6BB4BADA2647B0D460ACDFB6 +832D50178EF1680C6151A970D8980D203D27A58B208BA01E6573520DB5EE1D49 +A76A934286150697CF9DD3724C0A6068BC3BCCD8F647A52C491E78DFE316057B +77282D4DB45364CCEBBE5D59F22446F309A0AB80E34E3B50458EAB8BB2225557 +33EAEA70AF786C61AE7101F4FCFBCF542A1079F4C050AEBEC59B3972169D424C +57B7742DFD49FD49ADFB49ADDCD1B59B5EFD59FEDCB155B296943746A2BDF0E4 +C82316E49ACC2C242DE795483C17E76FB90ECFF40DF4B2CB9F778A30C8BE41A7 +D672566A5CE85CC742B2437137455BA61B68DBF1E7267B2DCA1816A401CEC9FB +8C54FFC9E6779D1B2FE33A9055157CBAABBE1652BEFF3DFFCE693926BD8A2024 +B295E855D994083CCD71D674D67B7A2D536D7B4072611B97E6037A7D6F1B8CE8 +5799CFE69174AE17798A3D712C45E042051F0F7FBAB3F7EB4A3B65BA09F81962 +EE6776E82DD5BAF11FB8E7E318052ED7C80021AF12AE01447CB191245C529A3A +C00339D6D94495D939513FF28C01A8731651F52BB8D96611B182230C22C206FA +E8E1597C5098E758A3FE64DF1810182476087D7C5B6D2FC8721A2BB47DD19F98 +904BD35309459E70D48E2F41685A6A1BC91C7353528847BBDE1F62020D05BFDD +AEB520B68592BF87FB0224E748569D7BB1D22A50C2CF2155C291BCC7C7D850B5 +4F3AD743A39379F628B8C143102CA40AEFE95C61C7D48DF9BFC98EDE0DCB0844 +CFCF43996D6E1923101187A8D3EF3E5E4BB25F76B457B311E399FE5B3E96C9CA +E35AD631264B3A97119C9808F904A01FBB37DB139B841FFC16AB536650AA58F5 +4E24E8B623C35F51A7C91834D96BC09292AD92817E7427DBF2BBAA436EBBD89A +F346FC07329E76104F980189773839DF2F5CC15FAF163B1A7BDBBFA3FADF57C5 +89ACE70E31BA99956AE3F9CA7000FB33811FA75E86AEA3855F58391DFF99349C +908274D2AF898D741160859F86CDE4DCF9D77AA24C36C4B0BB65281A77DF607F +D1D3855E855011CA16212C4A8F8121C79D1432E168D2E9D895E59E0791BAD2C2 +0F18EED0B307BD7EC7700A385ECB8ED0D148DFA0610A52D46EDCA25BED47292F +6C740E0CD915965C0F16C70336CDDA80E6DE0B78614F4E279913E41D2700E2F9 +4B1D4149A6113D99C6A282ED4523A3B8AEFBC31D865362EC1F574F0FCF426619 +CCE84F497108FF99F5415224E08D5632E82E898E2F15545B4CAF3685D6B58120 +1AFD767EB32A8523AAA2D254B78C80A4900EB8CFCB479F872483A14AA731E51C +5700E66F482DF426FE595F2261CF6EAB9104BCE0F0AD2E725604512482919276 +36B573DE7FF395337CFEA7119379AAF670498D171DA0093D61BB25405A348CD5 +247461A9285BD3EAD997467E57739CD3E7021748BB3D74E9C6E106A3625EBFBF +16FA87333BCB5AFE22FAE00191F17857BBEC7B1BC96E261D468BEFA1044BC7F3 +AD1BA455775AAB1170251FFA0A8C405D773089FBD1BD70AA54440B94992D7FB5 +CEED20BFE46EBAD3046FB29A3F309DA26B99D639BD06AF97FC51D7EE42BAF0D8 +4D65F279DA515A9B96729E84E7386854C90E1532CE26A09C118823368F48FB3A +029E4B2947C754555B8857AC563D64A81A663E82B46BC1F3B31D23B1F0B53B78 +29BE834A4EE4FFD780FA3A2FD31A6A7029118B6DF9D5E546C0CB19A1150AE0D9 +59DB40753B0571726038FD59DBA80BA11E004088C17F9EB121560D95338C7ED1 +161F500C6126C2C9E16E8942BF1B557C9F9D1B14C3540A +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +%%EndFont +%%BeginFont: CMMI6 +%!PS-AdobeFont-1.0: CMMI6 003.002 +%%Title: CMMI6 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMMI6. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMMI6 known{/CMMI6 findfont dup/UniqueID known{dup +/UniqueID get 5087381 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMMI6 def +/FontBBox {11 -250 1241 750 }readonly def +/UniqueID 5087381 def +/PaintType 0 def +/FontInfo 10 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMMI6.) readonly def +/FullName (CMMI6) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle -14.04 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +/ascent 750 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 70 /F put +dup 76 /L put +dup 82 /R put +dup 97 /a put +dup 98 /b put +dup 99 /c put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CE3C05EF98F858322DCEA45E0874C5 +45D25FE192539D9CDA4BAA46D9C431465E6ABF4E4271F89EDED7F37BE4B31FB4 +7934F62D1F46E8671F6290D6FFF601D4937BF71C22D60FB800A15796421E3AA7 +72C500501D8B10C0093F6467C553250F7C27B2C3D893772614A846374A85BC4E +BEC0B0A89C4C161C3956ECE25274B962C854E535F418279FE26D8F83E38C5C89 +974E9A224B3CBEF90A9277AF10E0C7CAC8DC11C41DC18B814A7682E5F0248674 +11453BC81C443407AF41AF8A831A85A700CFC65E2181BCBFB83E8A2A6085DA11 +61B1632328B94B21D3CBF2E7752D441A2C9A03F6681FDAB37C4B67D5857720F7 +0C4BCEE266586738012A2237A85FCD0425DA7E8E8632543F5BD5D50F9DBAAE69 +9E053AECA6027559DE42F7291EB381D866F1293107553809861D43508C6F2341 +5E4FCC431AF4A9B3660386AE63E877DCF5E513D0D5702B98D58B34897FEE163A +75CD6F5A2196F91CE5060CA5E72F9C5F79D18F242EEE58135BFAB02D5CA430B6 +D1B9A376849751A45E52577B1810C4F0C7F6C8196071D5B08FDB731C5384CCBF +03B460347E0CAC959E4A12620B6C2ED62E06A241D1637F418B5599A7FF3C1390 +7AF66A00F7C31B4EDF249C56C268EC9D546CD0489DA51B734C8292CE7B3D9E30 +71448C85ACC64FA99B2DEA91AA4CBCBBB9DE20833700AF96395DEEDA3FEB7D6E +D6E3BE4A62CBFA18BD0471C14F3A2FD025C88E8166A9830C5B5B94624A3D6482 +D1A1D83805D0EDAC1C79EA858A523D9FA3356DF42BC01886E77F6AA6B04E4E3F +AD107D861FC626F0A439BC3F125D48649E5101BF79C71507FDACACBBB4C4335B +C41A5C15ADF1DF69E1D68C3EB9BE30BDD5385F81D26EC8F2F206C1C7032B9EB7 +0392BBCA23B4693686AF8D2E1ACB2FBEE81D75667CD1DDB906BC5B675660ABA4 +E63429F16623F20B6A040E4942107CCA26CF6C1F52DEED9EC7EB15BE4F18F2E3 +9ABDEEAAEA9AC41E9F20C37C54B33F609F5835AE5A1EC9D674DC50FAFF6D3511 +2198195590F62B3B8A6FC8C3431BB8895A2F910F0361062DE32CDA9A7F41E761 +B09BD8545CAA2D7124B5688B54705AA52E4716E36C0303FEAD38E0510B3CB496 +6C30B115C37E02043D47B4ED796C671E30360800EC13F0F0FF3E0DBF5A642396 +EB0F39A9DAF0974678EA1D041F08BD73867065DA91B044EF2AB15FAA6546031E +189936C1FD302BCFBE55EAF369DBA4895DD0996BF7292762551A86C8A3604E9A +8AD29E38BF6FD40172051EBE7C694249F3378ED0139D37EEA3ADB66F3A80A1D9 +9C3655F054CACE93BA4AD42DC78A03F42455ED1BD5C122F234F21ACEB8FEFF9E +3BFE0DB2DA6D368478299ED63912A234ADB1DCCF1E519AD20772B71F8522B5DB +24E9658BDC1333D76F07F63D81F7A51D12C445ABF698092A24E5E5C36A52206C +0C41F7DCA529BB0A4D146DB6B4B341459FC17FFA357CA5F93DBCF30B61374613 +4846D619423BCAC59B7EB40C14C70ABD985754A0094C98350D01FE9FA371B0F4 +38884690DC7700C49236EF68BD6E806E8937AD8B77648F552C0C907007465A76 +1475BDD3FBE1CF69C706E703226471C249A3448F81C240ADBE4B5474216D0235 +6B0816BB6935A2704E851E20C002F85B653BB0700915B358DDC168ED3FAE3D91 +8280A5CA5C2708D622B03DC47A70E5FFCAD5A7C8D9AD563857BBEF92A3BDB588 +8B5B55CC9B960F20CA4AEB598DA8488004DEDBC9671488011E836D9D8B6251BF +256D08E6F1808E9D498449DF1F18E48B310D561D4DD3146C2BA64D960E66447F +6DA8B07194E3BFAF662C00737BB7222B55A5CC206A1C46F252ACF93D2063BABE +2BAB874C01AEC438D3F86AE37C9D7FCE4077F5B5785E12C1E0B685134CE2E8A2 +2E4DCDDAED2D53D13014E428FAAE22447763D4693E82600F850CA4A49E1ADC5D +C8DFC16E97B9F6164095CF7F31373B591F815621F45DFF77BFBF89EF60C9F50F +0C2FBBA4EA9602E197B9447FAB6AB57B19B1C801DAB66BF7D263F6572B9EEF88 +E06744C1FE197B8EAA71E1EB67716B7CAB30C859A2D1B2DB95ECED02B83FBA32 +E62F97EE4403FADEC936CC8DAA161F7B509B2723BA2B735572A80D7D087E6395 +6A9D34D5963E3C6E37F307C66D1CE551D2F055BFCC7360FBE93ACAFDE2B1A849 +37FD373F62BB23C1A2E3DF61C9452A8C995B03D44D6210A968C6293B4A3BE662 +C7B0B59E6EF6D53FD293B2C290436B0C448E7D15A63533DE321643E53C85EF8B +060C434E46A736EA617EA00D185323460E77B95C49070B035194A2E7693C3B15 +104ED7F56FCADABE49F27F9A576B354C574B5BAFFFC57FE9AF02FBD87B8874A1 +24C53826659BEB1EC50446AC26EF3A95877ED3D81D33A8BC1BF7473FB9C11ABA +0F27B38EEBDF77D3D3EA63AE6022DB0D4F1125DF274B86EF0A4A263E108D6044 +7A885F5E87362293A53F457CBA6C911D23653DA7CE4BB7D4FF4481AB97C0DD81 +C13515F0E3BF8A7E8DE2CE69B25BA9C185D9BF31A65DD290B0D4FDF40B61E2D9 +AF92BC6468B3F5AB5D5B88710C489ACC69198D697055B39AB3DE8B3733825C5E +6E763E2698574381A01352A7042A36EA96D17DD2EEEAFF04ABF7BF7D877B0FC7 +B6BA51BA994D431F472B6EC4B4C327399576470A1773BCBAC61CDBF0F95DA6A5 +0BE5239B7E9CD39E1E4D27EF6695D1FB56D4CA2C2370509547D6D01273CBE6E0 +66A696C35D2D0282CB8C85D7B1C51453A0AD0BC80E1D183D11AA78879643D9D9 +A3D0B2BD8EE3EDF29483CC5871426232A20CB058EEBDAAED85378B163425C270 +B0F7BB46B7D1B89CF84E6A06791195FB17F11F34A2548FE424DE10E376B39A2F +DD9B0BA2F2FDF3ED4AACBD0D60E67C0749D6DCBDE6E79041398DF3FC8D9B6CFC +693B870683CC425C52705902DFD324E157C9829F25B14A057AA92AFA17F156FB +0BA949698EAA0657B03D9538288D887959EF24A427EE7EDE47F304A1A6A715FA +2966273831B5B3A9C11021C153E67A73FA606FE3233C3DB217404C6AED6A4C59 +DD4C2A98B31AF5604D8DE8354448B4CEC66022E756B0056CD32E9B002ADC712C +0545C2C16D9457E271D98A4ED7DDE01BBD1856F5C3DA28794D2C7951FE2FE6D9 +18643CEEC41CA4A3464D3D3A102653CD6EAADA59CDD44918523B8811C2ECCABA +AC0D188F5E07B37AFD56006B10B32EDD179F4DA693EEBF4DF5425D996585F21C +8780A4D23D24FB0618C12DE65CAD42A0138E28A100257978EDCAD168C55C6F07 +78A067F0299890356C2A184820A554FB07448395ECE36C45211757552E288ED3 +7DFE46CB92EB2070442AFC85864A45000C4854D728096AD3D4C552EFC26DB6B9 +C340358451B50AF9F1B35C2E32D495DE85B5848D78940270DEFC07A14D960730 +878B5B55916FB40FD805116C9C2AC8C6D4645DA67A978F1C6FE705EC188DBF7B +FE7AD6769F2EEC4A9B05306D5BCD6EC3A240D6A29ED07D1D82E697DC8CC391AA +015DB6BCD1DA5C9A5A7BE3C5936681215E859E9837027D108A54F42AB24E38C2 +4A7D8CF3264AE049D4B1AF48BBC564BCC88AF19D37DD81CCC431D0C554DF29A6 +94ABDF8126EE4FE1DFB78559B9BF36FAFEC9791049C865694F9E2FFB7A308FEB +EF0195FAA6BFE7A081B65C5125623903828372F117EE17E588B2FA26DC9AE37D +9A6F0043C019624D3D87AFDB4B90C46AEA78919CEF5D7ACEE316F30EA0229742 +55E02E0E2533CAD78CE9BD6E0260A695D776ABA229D89CF9BEA92CE0397821A8 +830A557021B3B90CE0B07A4B34C24337FA3FA4A3728B6E013BA01071CA7FA667 +6FFB4BA3A28B5D5614ACD4E80E00938C2589CC6CF885A260AF7589240BF8987B +20A19FDAC73FAEEDD93EBCB3D46F735214164AFAF1A7828FE96E2BF7CE1EA46E +FC1FC4527A0ADF2B81CA51E3D96F3E786511AF00FDDD9764CD3178362CF9DEE5 +F05D100EC21A301F40FEFC3EF05AFEC140A9466AE68FCFD6CAC1A143BAF2B7BF +D0DF808F0CF87FE2399CD9D2DD424227CA7D4073ED002CD555FBE8057A797B60 +313D18ECDD779FA7289D4E7BF82D95A7535CE3F0F4164AF239AC666653736654 +C13E436B526EC5A1DC97E1D85C4EB563C7061AE1E81F0B38BC45B17C15FDB0DA +E5F6332721A925D4DD5A97BFF3E3F11D4F4F43BAE4E73A294D314527397F03CE +AFD12DDF9D07DAD8D6C8EFAFC2F52E20FBF62026810C0C31A1EF71FBD612654D +C68B7A24112F0ABDD20EFF3DADBE447AF9CE568F873ACE72B63283CB26B89BED +4B3315F26084453B1EB0877065A76E7B704B27F93DBEB365574A78E92FDB4B82 +D6ED91A1728F571FC9CAB636A6E81D0C1C4F5E3F6F4574B6EED48BB61EFDDC80 +1831E2AB9327B37881F0CA5D5265575F210BF4BCD8653B4193F74E3FF65028F9 +2968B8127C68CA2919DDF529EC882EE99901146CA629A07B4DC59E7868DBE083 +6E546C5769655EF6628FE2EA78AC891D130108192053D5056A7BD06C09592AF9 +6310B3586A2609DB94CB0079550358F5547B9FBB2B77917A327EFE3F972C7263 +54076627B4DD74638C3105FB25DA246484799D5326EE80503F18672B893289CB +1FFF8D3F662FB33E93025D86803551A8A1DE363D1534BFA3EF968871B19FB189 +DAD841C177E2AC98B6B69A78048221C5EBBBB86511939D3A9963B3BF021185AB +0B26CA7D0A779DC2F7693C7E48286BD60EE46615E31B93FB0E6992AE20C792A4 +72AD6FFAD6682D0F606807D7AE63A6280B5967FAB889DCDF0F76347CE302FAED +A59E1B1AB785D9D0ED695A0D05DD3DF0EF0B275AFCF9EC7031E1357E99A3DB85 +B4D137567875F7580E2BEC96B08C97ECB40876000B610E0BFE81E7884C9BBAE6 +6BE9C2C612475396BC10EC7FFBD1118A7971FE1971DFD0CB677ECA65DD837B5E +C7D36FE9F20A15A941F1E77C11093C902DF5B7639580B9C0E4AAD13518E1901C +1162FA095F301848770C6C618E371A0AC3463D0D94221EFCC0884B21300194B5 +97E03BCD5C4878CEBF352EDD1D2173B56C9B6A08BF3F33419DF226E1633CC5CA +A88A57AA85E21657569916E9B4B79E51B5535A57B2D6FA3EED1D0AA32DC0165B +12F8F5A681040B456D8DF175AD6053A2954F327F4EB4CD453A056FB1701242C2 +59755058640C62632C764E002CE928E6B908BB137C28630A3CDBC7C34CF2B643 +FFAFF20E345A86F31756C78A8EE2B9B1C5DAB2C863B29A44190770FD90BC6AED +0C25942BBD6A370C21650947B49AC8EB6825BE65DEFAFDDA32007754DFE0273F +B995B8986E505F95E58A3F8873545DDA6F3FFAADE992F3FF7B2DAB76D663280E +CF2BA18BC8871A7C361AA3600B192948CB918FEB0F2900AF2BCF169576828325 +449CBF0C7AEDC299FCBB061F491469B9ED1D56771A03F9F202550E17FDB15B51 +4772B2F06E6AABA4027D13E8730999532DB2475630D2F8EB98242E36F5B45A94 +2D8083014B63A806BB9A01906143482C614B5CFBE8C5354A3F2438B2160F6DC0 +81AFCEB201EC67F66DC5808C907A3C6CB664F444D24095D9A11D4978A7B57037 +87C67552FD59F5713D0FE03462F9A1A323BB3EFF0AAD61B1792CB109CA9AE4D4 +D7F7FBD3FB91258DCFFB4634669FA13E03C0AF699165311CBF37D24378022787 +87D1320ECF878F859149B2F994E700A32CB7253297AC58753E4583A08A1F83D5 +CBFF10F2D26D71CDB20E1C69A666E9C4D23BB0E6BF1097FBBB204EB736E61FF0 +C3FDE1986938F8099694B9F998B3C735770FF21F73B1A6B750498BFD21184F11 +5E6AF0F1FBB08EAD436C4AADE1E6B815F5E58988C113B4B2335A340798BFB201 +40586D0BCD6AF5A08CE9AA042910197F47D98CBB4411A6D435A46F46B3964F85 +032D7F8D880732276A8FD24DA772F642F06C9F3EB4C9090AE4A3AAE67D8768A1 +7B2A57E15B7FC626450372F40E3EE28D8D997CCC8CC89C644461D1110740407E +F6D884A0503E05AF830994AD0CEEA924D935749F6EE9F2DD65A58ECD59229918 +A303C90E19E9C411FAF24AD5E9B786178F8182692CDD508A4755123B4A594AF1 +A8E7287456C1CE2CA485CF4CF9DEE5238AA7328DA204816F98B7EF49EDB30875 +3CF8C48E0465BFFA8F2A1CFB6A8F165B63CC80F54C669D02BFD9A0EA0FDE2B40 +D69F0101AE4D2C10185AB10953019FE7D373BF40C2F6D4E36693F118866D1267 +12BA7987ED4313E8635DA02A4FC1F7815105F8EDFE8004A8801CC1AFA8D123FD +1768A6C39E838BD489C6BEFFD3BDF650DF48D5F8D99DD5A2E8AD235F2CAE8BC9 +C13AE9B214CF3FBFAC7BDFA54CB6E675273330608BD35600775FFB2A96FF648E +DF55A78B43CF54CABC0DA8D574CC5C5C62E9D658EC62035EA5028767749E48D5 +658212BD39C7E76ED0C6F3E0A1D138A0DB8BE574987AF1E43070F1F12C560009 +F48993FAC935EB56A5793BA13C28293601D268B4880F9BC33D553FF3B6A7900D +37900BD3B4B558C84BBAB09B4BC2B14D32E49596DA090E586E28D560903B9744 +7D014EF2A6AC4F711A025DBBC841F1C1C93BA1F8327DFE33C803A0993050D590 +C7AE262261DB7D04B9DD91F89927161A69E14BB2A48D29578410E312A4793101 +B6CD65B298988AB798F45B76E1F58BE24DB207C7E07014E17525B436D41FDB28 +D1072CBEB526E17C72DDA00ECDD06394940DEE8BD02CD3C81BFD12BE0568A459 +0F53010A0655942C932905FEE87183CE357887E2ED286DC1E89F2F401CDDDFA9 +B418F218D4115043892882C09F2E3749D4671E75AB1AD01C78F59FE3F54276D2 +A91512C21C68180CCC0E147E834002D7799FF393842CE88BEA714672E1952C9F +7CEF2FD0115B0E292079909A7775769341A7888CE6063537ACFF225B4F396972 +41BF46C599D278906F3BCFF81DF941BF5888DE8A53CB0A2D71610CDCDCC1760A +7844F89FCBB892AEA853CF5D967648995559EFE852C95D2EFDA2CF063500F88D +2C83B615618831267BD8F5438DC53C6433CE7347E9C55E103B450EBED6229219 +20E979C00B0B9D86345FC3A8EA2B049B7E148A7AA1C859AFE93381DA74CA578F +8FAAB28175D5B57F6A74F9869B2F530009E953913C6925D0045CB2E817A17742 +3C5B7574E9BF3BA7B769C161C2496EA2D346B4E19E434120FE92A0ED3F4A9ACC +A8BFE55F04D17DF035D01FC0B06182D8C27B6D32F36551B9AA56D61FB6A34254 +D0DF918C6716D8D05B973C8BAD5AD52C06EF3C6503D4E0F7F791511C79AE74E4 +4B707612149D583C5D96E534CB45F1E7CA5B23642B1BDBC881FD8E4F66CCEE96 +7CC0509B5CFB0B3596682A81E682DAF6AE58F605FD7A1F414F62572298AD8491 +B76D0A23B4AC731FB8E7682F506B5CC02F7029827FFD866EF4DBF56E07E7743A +D42CE65B38EBDEA193CE5B172E87DC88452C3DE192E363EB9DBEC4EC65F53002 +BE9E94180F73B53CE1E8EE9333DC86DF5FB2561FA1F9B2511962CBEFACA739CC +38A286651CEFBF451FF6568CFFBA8AA68555181147C1CDBA18C4F5C77BA8548D +2B1BA37E05EAF7F69879C3ECBD02783028CB1A8649842EB71023F89CEB36CEEF +A9B04E9ECB0577536B88ECE7C1D81B0D6D5A478F8E92DF83D194759437088D7B +E93C6AFD0CC6D568E60E2F1265A6613F97A6C4068C8EEBA70F6CC4E88A80F65D +EA7B498395390A74BA57221B86611D3EE5DC4632F139CA8A15E5B9D2DC97088D +7CC65197CD93F4803BE4246394F40EDC2CB4904BC273C597F13C8DC680245A3F +F1AF2D7BCE5D69E9CFC81E5E9F46927C5F55C719AFDB2CB1AF2E4D55B0E3ACA8 +6BBBCF253CCFF8F02B8542F296CED0EB8A06A88549DC959E22C3D6C48F310452 +22B32C44B046A61A4F7F8AED613DCD3C464DB2280D56E517CFD16DC32C83EACC +D3E936D2F3A883B265444D818393E30EE3AA59E6FA73DDC421F5A0FC8442F89D +CCE98189ECADC080F1973E14012C2B87F6A8289D7E168092C539BD54D5D3FFCD +2B5E02399614BD5085D49ECD04E14FDB8686A39D6B3F7FAD25D751CA66148781 +32296AFD919D83DF85BA4FCBD7FFAA1D831C8897E39819D72F271AB142F1C4B5 +D94BB96F64B76C0C8A2D1481910C9571078DC7A4E23C0F133241F06609699988 +7AD4D12DD860E6F00A162DAFE55AD781FD915597899D9BDF17C01C6EB24F8BD7 +65B39AF750F44198BA633D86B96CCEB5ADD5620CDB737199ED96497317D05218 +537325871A9760C80B1EB1585214C2BD09A1C184648ED57AFEB57CBD78021297 +4B79A02C6B95899D035DB8A28FCF5A40C9286D06982FB5C9EE3A2D28110E8DD2 +B41376B316BA24DA5EE34B3C481BEAB54023EDD93627A89E53D09DF79972B9E8 +E0BECC11F961948BFE9F951F0B03264853BD74C9A25B2C9B08BB8C9E0806634F +CBB6EEDAA33140F2D991AE3F0261EBCF267191FFBD2DAFA4EBE311753713CDC5 +5906337E4C1997ECEB4D4F619F445B64A989A20A4165C8BEA6BA277694B42AED +9B4F34CFF0F5DF2F31EC5E2D87B6388A842F52A16CE10B6CB71FB082B4051E8D +CE5A06BC4C44397C641A304B4BE6DC56192343640AFF273E31E3A2AFE994C64E +2B3DBC93C4F95C1DF62B6C55110BC9BE03CD41039D5837B0200D03819F0994E6 +9FCD7E222F35EDF5FCD6987C5D29897002C4C68587E193DEBB709D31C5867545 +9BB27B478FF4CD1F7C9C2A92E8D2F6E0EC624E1AEB8DB841F3DDF46A21659091 +5E71B129A86E1B45F66C7DA3D94A2F10515E835A61CF3B8AAFAB6345E0E85EC3 +9E02AEF4DD670E7F00299D5BC76BD421C43C735CC9F2D4D52184FABC761FFD56 +AB69302269C1AB5CF55B8D7D9D0E0C1025ECF61F6B5971549164613F63B967D9 +7854DCD2EEADC9AA863A8279CAF90894C457C8F4AA8CF8BE895713E9903D1378 +E73FFC9D6BD5A2E8975D1A7F5878C6EFB58D48CCF43A79AFEF4BAF4C4B8665B1 +0C1520961526ACFFDD1F30034DFF478DE5D34F8E44734489FA22EF4ED8FDA3E9 +A775116FFE85E44A42987E0DD9A5F39421581462637810DD5302D90EA9B5D499 +505F7467AD04DB2D0842CE157A8A0C02A53F1BFFAEBA76C4FA3838C9563E3969 +902D9E89D6D9E52A582966DBF383945F2D5B73C0E1FAF9B456DD6121ECFAA090 +86E9321F82B25B3C6277E50D9EDB0C913021568348B08D9AE6DF31C5FD19946A +8D74298AD1B10D137144A29CFA32E6AF047F07A8A6D2010487155A18A9A1CB39 +1D0DB8F09F75FA7B3583942A8E572580160788F938D0E8298E1950B71ACBAD4D +FF720BEDB008884E4F62AC4B6C0C5A09E561D06E11DB23D0CD775FD9A644F758 +74E27BE0CE3B9BE217BC9C376927193531EECAC8D3628D56C67FBE544248C20E +3BD3E995EF7985439BDEF7F0A6CAF3076AABC09883EE46C1F6192ACA0FEEAD8D +33E43F1499777FBDB099CEE55E3F8FA21680BF31DBDC0A8B732F331290A4A49F +BB448E74E90C5D1427DFAEA084615FE1E9F1C12C3B97B16DB7E5DE3BBD48EF61 +0B5699C41F7EC863C30EA843E62C4FC9E6369A6880ABE63980BE99E8A697BC69 +BD017C31C52D6B95D8FF1388CA15B469F5EE89EC27B16695FAD2ABD47065B3A9 +935E3C56FFD0AF2AC56F934A2CEA42E2C9E16B220CDA4569460FC0FEC662E410 +9D5E0A6DCDDC6FE3404CDEA47A880F9F52BC04EAA5E981B771B171F285A081A0 +5003825DF82AC7D2D49E652DB0E7529FECA70D97C8A407B4EE1C6B0F5134DBC1 +2414A301DE678CCB38D9117BE6A04DB7509E930E20A813BBE934222149C27CA2 +304B81A669C81472C62280CFE0764F32AAC39E021F89A2802975D55ACFC7371F +C77C359938C96540981BAFAC6B1DDAE14851CB30670C150BF3F9F2C883EAC956 +94A708FD98EA300386B3F6009881AAF19FB3FA8718864A28865332482ED2485B +AAF60ED6B6450AA2C051300D84104F3455CCC0FAD15A5F25E11A2210ECC92B54 +F110E5C73F453AA6B3DFAC42A27B0E681B30FEDEF96868FA82BAC0A5D3CFFAD5 +BC3D7F3F22B9143EA3E7FAB72B33F9F55F53BE63620ADAF29FEA15FC7FF77404 +D2452A4B44FDEC2CE3F2997E6BB7303C7DB669C6AEF09CB987057501F5A4C7A2 +E2BCC8B0EEC7A51A022D73F356FD3F3C3F8808B042E4BFE100E95A0129358F24 +795E6DABE5D19494C236DB623CBC85E4115E1FA79501EEC7DEAFC720D89F75F2 +949EA07D73B05CD6236584F087B10F33EBF428D0D86D4F831E33BB6A53190BF4 +8E1A660D7C929595D1E00EA9D3AEC16C49B8DE748678698E097052B4AE818FDF +7C5668BF7983A798D48FDC8C6C8CE9F20191CAD1C9E59A329B4534CC8F9F71DD +B33D4B046CC248982ABB2CBC665A83D5EC4C4CC679FEAB43C172BB16A8EBCCF9 +570FDD86E986CD950A27B9007D401ECC2C3DB3DF55A30BFF2FEB93DFAC9C9FD4 +2AAFB894 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +%%BeginFont: CMMI8 +%!PS-AdobeFont-1.0: CMMI8 003.002 +%%Title: CMMI8 +%Version: 003.002 +%%CreationDate: Mon Jul 13 16:17:00 2009 +%%Creator: David M. Jones +%Copyright: Copyright (c) 1997, 2009 American Mathematical Society +%Copyright: (), with Reserved Font Name CMMI8. +% This Font Software is licensed under the SIL Open Font License, Version 1.1. +% This license is in the accompanying file OFL.txt, and is also +% available with a FAQ at: http://scripts.sil.org/OFL. +%%EndComments +FontDirectory/CMMI8 known{/CMMI8 findfont dup/UniqueID known{dup +/UniqueID get 5087383 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /CMMI8 def +/FontBBox {-24 -250 1110 750 }readonly def +/UniqueID 5087383 def +/PaintType 0 def +/FontInfo 10 dict dup begin +/version (003.002) readonly def +/Notice (Copyright \050c\051 1997, 2009 American Mathematical Society \050\051, with Reserved Font Name CMMI8.) readonly def +/FullName (CMMI8) readonly def +/FamilyName (Computer Modern) readonly def +/Weight (Medium) readonly def +/ItalicAngle -14.04 def +/isFixedPitch false def +/UnderlinePosition -100 def +/UnderlineThickness 50 def +/ascent 750 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 97 /a put +dup 98 /b put +dup 99 /c put +readonly def +currentdict end +currentfile eexec +D9D66F633B846AB284BCF8B0411B772DE5CE3C05EF98F858322DCEA45E0874C5 +45D25FE192539D9CDA4BAA46D9C431465E6ABF4E4271F89EDED7F37BE4B31FB4 +7934F62D1F46E8671F6290D6FFF601D4937BF71C22D60FB800A15796421E3AA7 +72C500501D8B10C0093F6467C553250F7C27B2C3D893772614A846374A85BC4E +BEC0B0A89C4C161C3956ECE25274B962C854E535F418279FE26D8F83E38C5C89 +974E9A224B3CBEF90A9277AF10E0C7CAC8DC11C41DC18B814A7682E5F0248674 +11453BC81C443407AF41AF8A831A85A700CFC65E2181BCBFBA9B440A6DD72BF8 +97084C906B05FAD969086ED21AF0AA1471613182B26117D7494DD9F9270EF3ED +8DA4D957225F75D060237B6DAAD5A0AE3E702B3D1C437835B93B8AF1F9E7D966 +E739CF3AD5E256F90286A34069E5BB4122F94F18F3485658D0D25B938522A879 +8215A417CA2CBD20F71C5C5FCDE21EEA7BB27876D93BA667868A419287FE59BC +F538980597DBBA743DBBDBEBC61E3286DA7977833DC8BFC5E52FF5DF5EFD9A92 +D070EB769E31E760A50FDE012DC0057835E8B9B046FCC83F1A0C40326AFB4E3A +0CC3BFA35FCC64E32854F32EB7DF10A19F95830136BBB8139DE1663B7FD790CE +464EA431AC109FCA0E03F3E0D355FAE20AC8774D6B1CE233C27680C77DDA7356 +560A27C75993E8C980CD1E3B0683F7E8A05119B3AD567DAB4851B66E418687B7 +F9B21B3BEF607918D5973421B68E65DFD8B6C8DFDCF1CAFE2637D365148EBCE3 +FA4CC00052A2A522205EA3AE3461CEE02042E1A3F11467CB6C8C849B200CCE3D +0BC188EC7B934CBBC0AE2BF5DEA228181DBF0F774119F313516E7D97FF532621 +9278F856C166CA6547504F34991D588A0631A5CD06363F3FEE9FA0772C783447 +ECD0A200929CB58EBFB6B72008E4082B5D14AA560C24915B9463A92F38237886 +C35CBB2D4DD6D0CA8C1D4EC46093041C6181C2F6586EE3E7D4E647A107B6DB23 +DAD9AB5A0C2905455FE58075EFF6B48597078BFCCDD84812B98986F34987CE49 +7EFB19814F2A58B0233A59331F6F8EB66401F04EE7B1ECAD9BC90A2BCEBE213D +DDDB1F75C83609ED6A669A0CED58B2E269E76ECF73616D94F13CF827C9BF354A +E82202988DCFE856786B8AE569AFF3105B55C72C58D310FFC0E10B2ABAC8DB06 +40D5F72E54770E9DED1AF4616008595B8481E3D9AF4191CC9A5BFD9DDD01C9F1 +FE7165D21E488DB40879E863D470CB31CA06E5B5F1F8C3CCE04B697CEB0F3557 +ECAA358D2EC2B370519CE06138FA702314BA01F1F33881825EAE1230098BB3C9 +59666983275CA4E8D9DB34979F86535577E79393A72F84B0F768FE8C92692907 +15E9FE9894E98A0EBEA490CBC8C7E5A9F3E43B24C2C5A4BCD71DAAD3CC0B8B82 +AC13933543E295C163F61C9FD18371CB514493F90BF7FB460C029B8DD2E2BF05 +FD66B451DF277864DE1EE42100BF29E01A50258C2758F3EDE211BB3457B8243C +20BE72983FD6FA2581C5A953D94381E32E80D6D6095F2E93A5455C101BA71E8C +E560D4694E4C167EFA25FB1E9D214AEA745CE34CAA5468FAEF8F6BDB6C6BE8F4 +3D58836C26A2392E4C4DECE284A90DDB3858A16D6135FED655A600929DE71605 +6CA32F6851A2A6F71A9DF3D5D657593BB729CBCA2F4B059365B7263DC08AB211 +9C547096E6427F6AA53CB2EB87DF0AFE2ABCDBD15D7EF228D3396413B83C6B4A +79E41F9BA55A2688F62A10472675E5658F151F9FD6634EC94EC0682C17448024 +CC1633077C07A93E4DA8749D974FB8F4332B5DECF97D749C10DB60D4C90ACBFA +E65AE928C88BAE19234690EEABDB30BEDCEF2660D7464D5071058C30C572A2BC +7DEE5384BD7614A4BEC4C84E18CF7EC81C810256E8CE6520466C033E2A36D3D3 +5D6074B3857415011D8D9D49A474D994571CDBB89AF92BEA879BEBAF67663F5C +17ACAE809C2231EDD0A76641BA52FA7B19A2798D54A4A9B62C42F9905851229F +2CEE0191C8AA5AC12BB0CE9E5E3E862683AB57DBB4AAD6AC0FA8BA4F408D41E0 +755F72B82B7C18EC6B13995BF7AFD66AF4BA0EA7523DA8B75EE751744EBA9CA4 +4E8BC1FB37734503A5B24FB9F2C2D07A47CFC477F02413D55BD7DC180B0344E8 +50248801FA6BE26C97F397797F5F9DF762967E7CD92CCB8B2E587C92177619A4 +BF8046CBC72C6E69DC78B8CB6B7381A290080EF59F5B9F29C1167B261C932E9D +010D2D14BB425D157F22BC0305770AECC5BC80000F8CCFB9930255A68F299ED9 +D3B5B83A2CC00E3305EB281E1A7054734661B175C6CA0AF168790985F173DF03 +A8693B677BAFE23C3CF833FF6463B136FC370E4F0C29E322DBEF637F62C33CD9 +B0A8338FD67EC628E3BF2FCBF7CF0347D5CBA1DBE6DE878DD670176B85F69EF2 +3C5CCA1BD2B8A385F113EF1CE522F5A6AE053B9C1E39408C9459DE3E7FE2C4ED +77F026B0081BB80D40185458139C16333EA27F43EF1204BFBF80BC5301B2A3AD +B10F7EFBB4F5B7E04DA1167F68BB6D4049440B0F57385FF0A95E72760C6A12F8 +1335BB31CB74081FBAA319180DC00113CF50CC5A41D2E751E055DA1429CD75BB +0060C21CED634FDA106C49A12B356129D010E29F2919301AA7F80222AF3905ED +672FF85C9897A70241E8DDB9A53034B6BB44E140D9E739848E7A782F24B98AC8 +00DA09EBE4532787E5CF3ED815705F659D8E52DC2C2D4949374A3BF192BEEB99 +1D9A90A4F3250BF8A1FD40D91F5B34AF2CC561FD02FED712500B24330D87DA9E +4AA46B6E34BCB40B1F73A1DDE07E002B9478E9651D2BF85E67585B6ED812BE03 +A594874A235B1C1840C4BF4BA6D863583D8C12DB49EF7F8CC24DCBB6B21FBCA9 +378F3E2DC406291AB324571F76E0805DF8826090F0E8B50599CA58D124653D16 +16C782B01C91A6F3DA6346482E3163D28D95EA49866360147296838A3FD0CC53 +920F91D895F839CB61FFD2FBA296CA4C6304EEE579782AE5FD33D0FA652BA7E2 +CEC7C01DD0D3E0D56E6177EE5F609A396F7FC8EADABB465DBA7F384E215C4DCB +E64F807A50A461545107F9C3F7D7CC7D33E6EBD6D9228B1DCBFEF160703E6129 +0DCED8D45DD54E2A36E698A616E7906A50901E453BDB2A363EB77144E9EA6F2B +6BD927495EB0EBA5755165707CCFBF8759CE5856881117C7F3EF6D494EDDA7EF +E499BCA56C86467AC573DA9C2724FCC74BEB331E736FB093DCB67DAD42296655 +415D110F2729BD1D55E5C9CCE2E724116F45FB2E66AE0F790258851A5C808762 +68B8A110BD326F8D3EC45004E7CC08DA42F6CB80A6B6E7C286F139534A275BCD +2F812993DD9C9A1AEB5E7E4BDB4805DFF3A7030263AB060C9B74F0C25C5B9005 +965284884450CC2815DF28D5F9B0496DC7A3AA85E1E42741E1538797175C28D9 +FD904699C771FB066397FFDEE8E8DD1ABBDF67E6BFEF95BB700A7C1BA91354C5 +42EC3864F6E19B379E79A1CC3C786C0DA146C6B0B8E507ED58DBB1F12F613A98 +0E1F8967991427A22ED323901C4B83336CD343212131E8B59C2F5D232702ACC5 +7891BFD4EBA5D0FA35AEF9F3520CA82D121BF6885BBDAF15248A9E4649ADB94D +0735CC4D14E1D7275427D00C8E709579612F7F74DB6FC218C10C278CC63E2AE2 +37EC996B10C0229D687F0DB5E38A8C4DAFB3DD8A9E7ED37186FEFC97790A1EA6 +636A88FA9FB4D282234BAAD301A1F3AD33F252C5EEC49410562FC52809CEC466 +A0F6D148E9AF19D6DA2337C8283FBFF6005C37AAEB0B7F7217A8DC6F949B9984 +72DEF163E4D5ECE4288404448C96A7FF0AC76F732D50AD63A1D286C9180E80E7 +C218B1F48E3034FCABA6BF262CEECC284AC29E9F3CA1CFC1639A681ED66C1FBA +666F073D45C84A286E05FF809D4764FE819B6A330E73695CCF2F448B4D4EB4B3 +F63E94EC289807A2F9A1159CF328C002B467B19D6E9454CCE36FC19E0A214190 +B251818DD456EF658B0398E275514B72D9C1DA5F806EABCF1DD56BC025D69FC8 +A0C2FAAC1892B64D2AF79EA2F57F103CA623E440307600D50E783FAA998EBD40 +51D23A0CEFF8D8649B48B982DC38D613F882DCCAE5F51233A641B3CFD783F830 +D984F116DEA3ED8F0D3369AE629A006BAD4523F8E3C7C6B39A6C972508B67AE9 +32613F28CCFFC4BBC86CF31A0C25C786554F7A1F3DE97F5CFD1A941F775067A4 +784385E2D02EE1FF886701B1E87D966D3F500E15591A5012E645837FE2DBE3E6 +A3D375C6CA0ADBF96B33EC3FCFFFD888D7344B31D40427B8A8BED0FEC6FBE038 +1FB5F0714C4B5A0E607E215B5B7F76ACF0FEAA4C9790EB7E13C0E3933B7C63FE +5B934EA34F4B741C3667BF1735C685CECA63507E6FB9EB06AA010311F12AC1AB +4CE3FE8D1EA1EDB3C700BEBA516FC71D740B1CA1A60D4578003973CC3EE21DB1 +58FB1CF7E2EAEB2A4A6C742EBC3575EE6378531C6EFA6E6986E68B8E25CEEA67 +A59623FC1ED2ADDA9D72DBA627D179E47DC7F5551E07EA4D54ADB6CC8109D340 +7279F288E552EFD79C17DA3431E53EED66D16F24BF86468C2FE7EFF421560500 +12FB048D6CE2F370BE4E560F8B4AA12362ACFEBC839351C1D5100C625B14CFDC +747B66082D4AD5474A63EA0054E9C3E6295AF6B133348487B0471395857F4B73 +4BF8337DCE2FE2E1A4EAD7E7BEDC822BDDCE42B79B308C11897C98E3ADE253CD +09CEEEC0CB1DB66AB072E36E1E04911F40B535B0FD85982C21B8A587D65C38D2 +DBC5A07A0A26DFFF7460F10781069490AC1B611CF7312A14B4AA6005A4582C5D +336BCC30EB47749193BE8D457A43F54204B070DF5AC2057B6437E23705C7FE8F +7BB150560F7044BE3E48EFDDA539FEEFB0D2A7856CD4E405FCE0F5EB190D91AE +578E2EDEB9ECA218573BB1A8EF116043A27DD17A4047BCCC7C5F3C563A910778 +45ABCA32C7347E6180ACC86F9D665FF025DD8AF514FC3724B5C3510F3C37E0AC +5101D1667C6ED4E8F37F06CC2BDF66CB5A9FB7C52CAD26344FD1557571336A1E +1E340EBA149B4EB99016D1A411FB874914AAB2A415CE3F5FDFBBF5AFD7959B9F +CB127BDC68D2A2F3F07FF3D4FF32046C0371CD2E68A6471E46B08413FC3C7A80 +A107EEE57979DB387B2206D2810DB310B7232B2DAA385256C8A58964B512003F +A0C24ED21809E2576229627278118107B9C32345C1EE8C0CFB452CA362379369 +31320DEB5371037AFAD093B61E8AC7A6DCF7D49C7F8EC32DC0ECEAFD7E892810 +039570D2956289B15E078C2545911BF535F72F7DAC619BBDEEFA855BBAA81704 +18F7D351B0936357085A32157AD8E27438A58B2397D69264E748B0B8D01B33F4 +D04DC59326A7DED39E247A1C1A1AE49382BDBDE9478A1CB48F88BDF14A268B40 +A40B9FBFC4C87FD3DF1EB2464C3C14E36CA41E09EE0A9B75FEB0769F9ECEB1BA +EBF73B818427FACDBC33BB95B9654F31C59A766E931C698A8608F15290FCDBD3 +5C535D9036A19CB7B55BF54E96F9B2206DC71624E2E55FE632FDFDEC8757AEA3 +1D83D190ABED5E7A7AAE2F41FCEBC7C18626BF58F9E9F02FBAE0C8AA85E9DB21 +A3D8907522DCBAE4923C6A2A09FD2F08FE32215C544AB577B337D929E625E704 +E041C2381AFCFEA37F3133B6CA20093EFD457C772E428325E56C9CBCC447EF9A +05A8C3F28017DD4FFACC51B38E4896C5044266EAB4EB7C13FE855E790DCF8A17 +B61B1D30DD866BC57397EF6297C4891451FD6A5C6AD6D7446F58F56A68650908 +224D9F4C31C6906FD29BB51DC947465B808438E6260325752808963C808A4AAD +60422ADD62CAF315F6AE92FACEC55D5B682089AC0BC051CE1E2C06A3874736CF +0DB5F7C8F178479E4F11665402781D80397C75456F5CDF0A4F382A19EC6AD64F +71A9275264800E178F212269154DD8352167C57EBC0A38BE794AAD1601C8E541 +7E1AB8E969A76E1EB4092644958FEA2AD29635E70C4DFE2EB0D9B3E1644FAAD9 +B27AD5466EFAC724718962B62E7B8C32F412B69DFFEB792587D571FB5C591D95 +4CD441662CD1B07595E245FA537FA9EB5A20A97E5C9251EED22C9961B48B25ED +85BB7524F635F9CBA3714C6D60A6BF920C45A64F4C366C1F9D22F53084997C9A +EFE2D79FBE3347111F5093E271DB7E3770B35D253DAF93653F6A23FA145AD775 +AF11E188EA0428137D9A14542E3EDA6F7B2E5AA86C9F3D3649A85ED2F020C696 +01A339FE6D7E42BC548C8F92A4E3809C67A986C99418772403D16D0E8662595A +1F37563671D6DA0F36CAC99DAA8FEA215DF7D45E61314915A30A22FCA86A50D5 +2FF2EF08E240F9FAC030D92BDFBE40F1972DF413E6B452024CD11792BFDAA2D7 +C82716528AD4B3D637BB43E748336DCC86A952BE96F1EA423E31340FCACDC1EB +02EE932F58734AF3A5B2279361B63F1D824EE3BA9F4D2EC7B33A300A1CE8CA43 +24616444176DB8099D85AC68329B1F85E4B5B16F3B396FE2AE7774F3065D0203 +AA140DC128D6F935C44733EF585F89E8639A2096A225A2E5E49D447D8AF9FD44 +CF6C1BAD5C5E5262AECC5543EC8199B00B72BE32A0F110F64A1D0D5CCEF38FD1 +155D6198E9A343702F8ECF5052333272CAC2FE016681E12745CBE14E1065EFD5 +407DA3686080989F6F6D650A9F1EB3A813B070C30C0D3B4A1E9C206E2E4DFD51 +D8DCBE9AECF956640A2E79F1B4FD0EB8E0449AE1B8FFEBC43275743B4D7F6605 +0673B61EB3189E74F51F3780A91E6A5C6464C8CF7D563D9958D46F39B1A12087 +6BBD4898BA9ABA468AE1F24115891FD3CBC2195F75958E26DF8BF1B93F7B521A +C12112237AB23A8E5A7B7D0DC4C53692B35F3CD813EB463C0BD3A6486B0476C6 +3B36DA71FE512E5745D097FD4AF5D056E434DEE2AF926B2EE79F7FC4FEFD4130 +BB4B4BE01E5C720325A4884507CB51CBA4FFB615B78A4182444F0ECBE4161A58 +E86FE1DA2E39C2BECBCF1F1D7B9B776A26078FC252128FA8108CB83F673CFD37 +CCDA493234FB93E1550EF8D2DC049ED95B00A8A57834B024B277D3DF062E748C +B61F183F2D72AD075474F8165528CE75E4F40B38B0FAAE45751C1907F8D31619 +E88EAB02EEED415F3EE3BC5BECC6AF565D34E0BA2958FF337A2B06012DD1858E +C53DE52C108BD5AAB76C882198C72CDCC958D68EA8FD26F76F04EC1A08B2AC3F +A6D0E8724D2656555DBC0C8C42A3E22ACA7E1BC8E9F897D9AB692E0FB9EC32EC +59E31CCA4516A3C3BFD5411BAC3DEDCE374D48681CE7D67DEAB93F5B5C5290AC +FEB29C5EA2C98095692873D36C7DA24847B66F31E4CA4C7AE5C79D7CE4F0532B +78620582E3731A2A6533A03E7155B33E7CD142FE79F72721862EDB24959B9783 +F834CB616FFCB2A23497BA6D99AE34DC459A2F7B3E4DA2B54BED118ADCD92178 +66C40F4E60F6E1327D5DBCA645A2A7C770807E6D7E47E1265C753F8793BD2D1E +BDCD749CC24D4AF9315A93F01180A0F9A7F420DA1B87664DA5FD967131273271 +9DCC45C3D57EB9B8AF14771E8E751D88B98D2FFDC72F5011D402EC34FD010ACF +D3B0660304725191D64FEE106253FCB3470F1A16EB7B45C1489D3534BF94F740 +C2781DAFA5E8A9E7B25A85BD7935DF3ADDE08C960E283D8FC3976FDB4085DBB4 +B6B35FB239C28C785B18BE4FC98F3A5F410F562DB5FCA04E8074E4E790F4265E +F88117B3D0833AFAE6E8B8A71D7731BA6F14FD6F217EDA3F8CC687A494FC3914 +B84FDC37C8C335AB1E7E0BEC7FB6B7A595C50CF8F0080C8D461BCB8B579A5155 +F963B6587873FA31C3A6572740C63EFBE58A2EBB723B7517D2A243F6CB08A038 +54F4DF0F6692022B2EE8C6F6B73735ED3166BAC58D9216A06EA6FC7B63B20031 +D0F0F99D83D9030B413C2360DD2C553E34BD67851B743C3FDA676AD63C5BD759 +9131358C6BCDF05FCC048F4EBB9005899ACDD8E9EC9BB8C5A08E83485047D263 +0ED69B4D1869A38068FDA03524022A1D32FA2AE0BF7F09D3A6C93191E20C47C7 +832A25DFE46206D78D05B29615AF8A32B09006939A989F7EDEF37BB9B883334F +528A5AFA99164500A4FF96495E5E23083A673A976328E1F0D052AA49B80FE035 +9DDB6B5C01E4EDF10ADA000A7D6D9CAAC7970690C002203AD8FEB9231F5A2D0D +2AB4D8007503F64797942F9227920CAAD43D1B48EC23887DF500A243A7967826 +FD4095095317784AB2F2B4BC81933CFCDDCC723A92C5D81B3B8D5E7BD3C814E1 +7E6430C149498B186503DCA07F57B5F1164FAB22AA70670EAA5A4BF65B0FA7D0 +ED1C46C17E7F99CFAC861C96C8B4512A7FDEB9F5B7007D4DF17CA85FF1A474CA +7FE270F111A14865B5E118D34026FEE6A97289E11909EEB6A58D6C30785B0391 +78CDF986A2A3EBCF0BB8A6B7C7514AC6FD591EACD32469ED7F86DA5C2C34E982 +2BF4E32579246B526F518231408E2E5782E45E7DF67CAF2796640ED4EE5B2469 +8F6B41DC1B6B03ABB8F90AEF5624D0F77E55ADD076AC5B82EA647F5F2F937159 +654729F3331EC18BAB1D76E4A525FC72A191C8750176BA20130C89B5121ED346 +6E855AA32EA90468895524AA116CDD8D936EBFBA0282682052F6A406315441C2 +7F73AB704A34A941BE3AFEAC595C403C0636ED1188AB4EB0E6AEEBE843519583 +020937AB5E9442E6602ADA80B144DFA5B73D392B7DF3984A0E2A68646E958301 +DA6D965D1C2463A12C03DB01C9648B776F23218BB1E74D588DA6BBFEB662879F +2FE601869D3D4FF6C69F1B02ACA0198C25A6459F20143283CE1F9570298DDBD9 +94D7DA23DBE44D7AF6749D26CD1603CCA9F39BC30FB4A403AA51D0C1C5C00DFD +84108ED4A31242EAAD9320CAE80630EDA644F84B5EE3ED26EBAEEBF5A0C68896 +1A263A7E7CCD8DEBBC7EB93BB14DF514DBE9DCA9D6BCEAFDCF71FBA141CE8AAF +17230F6B95845DE93266E509D6A984700C7766DE7A307BF33C309D3ACD60B0F6 +EBFC52E833905868EF96EB9EC4F25E1E376BDBB37B9C3E5326E408FE41DFA3E5 +877B841126D5E710E3887767E8889BD0965AD3A274D5C6EA2AEF87A5C292F029 +C5A5713A7B32AE4FC4D599359465DC57DFAAA1A5666B0DA8C414485252EAC1CE +A8F77AF570F8B4553B75CD6633C4BA49D2C2A15D4A8FBB8914E4EB5330BD9F4F +CB9C46C13F6C3A8BDB2A4CB81B9B88C65664D694670C09560BB3CA20EAAF6713 +702720B3509C0BCF66220B3B26F28AD2AF9FC945C42AF9CD130DE81EB21422BF +6CC3246B51FFC3B2349D760714 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if +%%EndFont +TeXDict begin 40258431 52099146 1000 8000 8000 (paper.dvi) + at start /Fa 133[344 393 393 591 393 443 246 344 344 1[443 +443 443 639 246 393 246 246 443 443 246 393 443 393 443 +443 10[541 1[492 443 2[541 639 591 738 492 1[393 295 +639 639 541 541 639 591 541 541 7[443 1[443 2[443 443 +443 443 443 246 221 295 221 4[295 39[{TeXBase1Encoding ReEncodeFont}55 +885.568 /Times-Italic rf /Fb 252[407 3[{}1 553.48 /CMSY5 +rf /Fc 207[243 44[424 3[{}2 664.176 /CMSY6 rf /Fd 171[666 +84[{}1 996.264 /EUFM10 rf /Fe 244[484 484 6[427 427 469 +469{}6 996.264 /CMEX9 rf /Ff 136[523 1[523 523 523 523 +3[523 1[523 4[523 1[523 523 523 1[523 97[{}12 996.264 +/CMTT9 rf /Fg 145[512 4[398 398 512 512 6[683 3[683 27[796 +7[0 0 3[683 8[1024 7[1024 30[796 2[{}14 996.264 /CMSY9 +rf /Fh 133[455 540 2[540 569 398 404 401 1[569 512 569 +853 284 1[313 284 569 1[313 455 569 455 569 512 3[284 +1[284 7[569 10[768 2[697 7[796 2[284 16[398 398 6[284 +20[569 12[{}32 996.264 /CMR9 rf /Fi 198[377 377 377 377 +377 377 377 377 377 377 48[{}10 553.48 /CMR5 rf /Fj 136[640 +443 523 4[456 6[313 20[523 1[667 5[607 76[{}8 774.872 +/CMMI7 rf /Fk 134[466 466 1[466 466 466 466 466 1[466 +466 466 466 3[466 466 466 466 466 466 466 1[466 1[466 +21[466 6[466 7[466 8[466 466 1[466 466 466 2[466 466 +4[466 35[{TeXBase1Encoding ReEncodeFont}31 774.872 /BeraSansMono-Oblique +rf /Fl 198[277 277 277 277 277 277 277 277 277 277 48[{ +TeXBase1Encoding ReEncodeFont}10 553.48 /Times-Roman +rf /Fm 240[470 8[732 3[732 2[{}3 885.568 /CMSY8 rf /Fn +138[449 320 324 320 1[449 406 449 1[234 2[234 2[255 363 +449 43[406 406 406 406 406 1[406 406 406 35[467 12[{}21 +664.176 /CMR6 rf /Fo 136[734 497 589 1[477 2[514 496 +2[306 2[350 18[597 1[597 626 775 5[697 3[847 1[654 753 +6[796 512 796 284 284 15[1024 42[{}22 996.264 /CMMI9 +rf /Fp 134[466 466 466 466 466 466 466 466 1[466 466 +466 466 466 1[466 466 466 466 466 466 466 466 466 466 +1[466 21[466 2[466 3[466 3[466 466 2[466 8[466 466 1[466 +466 466 466 1[466 466 40[{TeXBase1Encoding ReEncodeFont}38 +774.872 /BeraSansMono-Roman rf /Fq 136[466 1[466 466 +466 466 1[466 1[466 1[466 2[466 466 1[466 466 466 466 +1[466 97[{TeXBase1Encoding ReEncodeFont}15 774.872 /BeraSansMono-Bold +rf /Fr 134[533 533 533 533 533 533 533 533 1[533 533 +533 533 533 533 533 533 533 533 533 533 533 533 533 533 +1[533 21[533 2[533 3[533 7[533 533 1[533 4[533 1[533 +533 533 533 45[{TeXBase1Encoding ReEncodeFont}36 885.568 +/BeraSansMono-Roman rf /Fs 133[442 498 498 719 1[554 +332 388 442 1[554 498 554 830 277 554 332 277 554 498 +332 442 554 442 554 498 1[498 10[665 1[719 1[609 775 +4[498 388 2[609 665 1[719 1[719 8[498 498 498 498 498 +498 498 498 2[249 332 41[554 3[{TeXBase1Encoding ReEncodeFont}46 +996.264 /Times-Bold rf /Ft 242[882 13[{}1 774.872 /CMSY7 +rf /Fu 175[431 2[689 7[473 19[387 4[194 4[258 39[{ +.167 SlantFont TeXBase1Encoding ReEncodeFont}6 774.872 +/Times-Roman rf /Fv 105[387 28[387 387 559 387 387 215 +301 258 387 387 387 387 603 215 387 1[215 387 387 258 +344 387 344 387 344 11[559 473 431 2[431 2[689 2[301 +6[517 1[559 7[387 387 387 387 387 387 387 387 387 387 +215 194 258 194 7[387 33[431 2[{TeXBase1Encoding ReEncodeFont}49 +774.872 /Times-Roman rf /Fw 7[443 96[886 443 1[393 393 +24[393 443 443 639 443 443 246 344 295 1[443 443 443 +689 246 443 246 246 443 443 295 393 443 393 443 393 1[443 +1[295 1[295 541 639 1[836 639 639 541 492 591 639 492 +639 639 787 541 639 344 295 639 639 492 541 639 591 591 +639 5[246 246 443 443 443 443 443 443 443 443 443 443 +246 221 295 221 2[295 295 295 1[738 29[246 3[492 492 +2[{TeXBase1Encoding ReEncodeFont}81 885.568 /Times-Roman +rf /Fx 134[442 442 665 442 498 277 388 388 1[498 498 +498 719 277 442 277 277 498 498 277 442 498 442 498 498 +10[609 1[554 498 609 1[609 719 2[554 1[442 332 4[719 +9[332 11[249 332 249 44[{TeXBase1Encoding ReEncodeFont}38 +996.264 /Times-Italic rf /Fy 134[442 1[665 1[554 277 +388 388 1[498 498 554 775 277 1[277 277 1[498 1[442 498 +442 498 498 12[609 554 7[665 3[719 2[719 665 67[{ +TeXBase1Encoding ReEncodeFont}25 996.264 /Times-BoldItalic +rf /Fz 198[332 332 332 332 332 332 332 332 332 332 48[{ +TeXBase1Encoding ReEncodeFont}10 664.176 /Times-Roman +rf /FA 136[879 609 677 405 474 541 1[677 609 677 1014 +338 677 405 338 677 609 405 541 677 541 677 609 9[1218 +2[812 1[879 2[947 2[812 1[609 474 2[744 812 879 879 812 +879 8[609 609 609 609 609 609 609 609 2[304 46[{ +TeXBase1Encoding ReEncodeFont}44 1217.66 /Times-Bold +rf /FB 133[445 1[472 2[529 370 392 350 1[529 512 529 +813 244 1[273 244 529 512 313 455 529 455 529 492 32[682 +17[284 341 16[549 28[{T1Encoding ReEncodeFont}25 996.264 +/SFSS0900 rf /FC 3[498 5[498 17[442 76[996 498 1[442 +442 24[442 498 498 719 498 498 277 388 332 498 498 498 +498 775 277 498 277 277 498 498 332 442 498 442 498 442 +1[498 1[332 1[332 609 2[940 719 719 609 554 665 1[554 +719 719 886 609 719 388 332 719 719 554 609 719 665 665 +719 5[277 277 498 498 498 498 498 498 498 498 498 498 +277 249 332 249 2[332 332 332 1[830 1[498 406 30[554 +554 2[{TeXBase1Encoding ReEncodeFont}83 996.264 /Times-Roman +rf /FD 156[371 364 448 14[612 5[562 5[515 70[{}6 664.176 +/CMMI6 rf /FE 156[408 403 500 97[{}3 885.568 /CMMI8 rf +/FF 133[541 2[879 1[609 338 474 405 2[609 609 947 338 +609 338 338 609 609 1[541 609 541 1[541 13[677 812 1[677 +2[1082 744 5[677 2[812 812 879 57[338 7[{TeXBase1Encoding ReEncodeFont} +29 1217.66 /Times-Roman rf /FG 134[996 2[996 1108 664 +1[885 2[996 1108 1660 554 2[554 1[996 1[885 1[885 1108 +996 12[1329 1[1439 1[1217 5[996 775 3[1329 3[1439 65[{ +TeXBase1Encoding ReEncodeFont}22 1992.53 /Times-Bold +rf end +%%EndProlog +%%BeginSetup +%%Feature: *Resolution 8000dpi +TeXDict begin +%%BeginPaperSize: Letter +/setpagedevice where +{ pop << /PageSize [612 792] >> setpagedevice } +{ /letter where { pop letter } if } +ifelse +%%EndPaperSize + end +%%EndSetup +%%Page: 1 1 +TeXDict begin 1 0 bop Black Black Black Black Black 193 +2148 a FG(Allocation)696 b(Remo)-20 b(v)g(al)698 b(by)f(P)-20 +b(artial)698 b(Ev)-20 b(aluation)697 b(in)g(a)h(T)-147 +b(racing)696 b(JIT)4611 6133 y FF(Carl)306 b(Friedrich)f(Bolz)14081 +5693 y FE(a)16712 6133 y FF(Antonio)g(Cuni)23376 5693 +y FE(a)26007 6133 y FF(Maciej)f(Fija\007k)-12 b(o)-30 +b(wski)35400 5693 y FE(b)37934 6133 y FF(Michael)305 +b(Leuschel)46626 5693 y FE(a)17280 7572 y FF(Samuele)f(Pedroni)25566 +7132 y FE(c)28104 7572 y FF(Armin)i(Rigo)33956 7132 y +FE(a)12639 8809 y FD(a)13143 9232 y FC(Heinrich-Heine-Uni)-25 +b(v)-15 b(ersit\344t)251 b(D\374sseldorf,)d(STUPS)h(Group,)h(German)-15 +b(y)17766 10020 y FD(b)18185 10443 y FC(merlinux)249 +b(GmbH,)h(Hildesheim,)g(German)-15 b(y)19728 11127 y +FD(c)20155 11550 y FC(Open)250 b(End,)f(G\366tebor)-18 +b(g,)250 b(Sweden)5170 12878 y FB(cfb)28 b(olz at gmx.de)2113 +b(anto.cuni at gmail.com)g(\034jal at merlinux.eu)g +(leuschel at cs.uni-duesseldo)-28 b(rf.de)15552 13985 y(samuele.p)28 +b(edroni at gmail.com)2114 b(a)-28 b(rigo at tunes.o)g(rg)p +Black -2000 20378 a FA(Abstract)-2000 21928 y FC(The)248 +b(performance)h(of)f(man)-15 b(y)249 b(dynamic)g(language)h +(implementations)f(suf-)-2000 23035 y(fers)203 b(from)g(high)h +(allocation)h(rates)e(and)h(runtime)g(type)g(checks.)h(This)e(mak)-10 +b(es)-2000 24142 y(dynamic)243 b(languages)f(less)f(applicable)j(to)d +(purely)i(algorithmic)f(problems,)-2000 25249 y(despite)273 +b(their)f(gro)-25 b(wing)273 b(popularity)-65 b(.)273 +b(In)f(this)g(paper)h(we)g(present)f(a)h(simple)-2000 +26356 y(compiler)232 b(optimization)h(based)f(on)g(online)h(partial)f +(e)-25 b(v)g(aluation)233 b(to)f(remo)-15 b(v)g(e)-2000 +27463 y(object)259 b(allocations)h(and)f(runtime)f(type)h(checks)h(in)e +(the)h(conte)-15 b(xt)260 b(of)e(a)h(trac-)-2000 28570 +y(ing)268 b(JIT)-74 b(.)267 b(W)-80 b(e)269 b(e)-25 b(v)g(aluate)269 +b(the)f(optimization)h(using)f(a)g(Python)g(VM)g(and)g(\002nd)-2000 +29714 y(that)249 b(it)g(gi)-25 b(v)-15 b(es)250 b(good)f(results)g(for) +f(all)i(our)f(\(real-life\))f(benchmarks.)21561 29291 +y Fz(1)-2000 31445 y Fy(Categories)390 b(and)g(Subject)h(Descriptors) +996 b FC(D.3.4)391 b([)p Fx(Pr)-45 b(o)-10 b(gr)-15 b(amming)390 +b(Lan-)-2000 32552 y(gua)-10 b(g)g(es)p FC(]:)268 b(Processors\227code) +g(generation,)h(interpreters,)f(run-time)g(en)-40 b(vi-)-2000 +33659 y(ronments)-2000 35389 y Fy(General)250 b(T)-92 +b(erms)995 b FC(Languages,)250 b(Performance,)g(Experimentation)-2000 +37120 y Fy(K)-25 b(eyw)-15 b(ords)995 b FC(T)-35 b(racing)250 +b(JIT)-74 b(,)248 b(P)-15 b(artial)250 b(Ev)-25 b(aluation,)250 +b(Optimization)-2000 39626 y FA(1.)1218 b(Intr)-22 b(oduction)-2000 +41175 y FC(The)337 b(objecti)-25 b(v)-15 b(e)338 b(of)e(a)h +(just-in-time)g(\(JIT\))e(compiler)i(for)f(a)h(dynamic)h(lan-)-2000 +42282 y(guage)238 b(is)f(to)h(impro)-15 b(v)g(e)238 b(the)g(speed)f(of) +h(the)f(language)i(o)-15 b(v)g(er)238 b(an)g(implementa-)-2000 +43389 y(tion)261 b(of)g(the)g(language)i(that)e(uses)f(interpretation.) +i(The)f(\002rst)f(goal)i(of)f(a)g(JIT)-2000 44496 y(is)229 +b(therefore)g(to)g(remo)-15 b(v)g(e)230 b(the)g(interpretation)g(o)-15 +b(v)g(erhead,)231 b(i.e.)e(the)h(o)-15 b(v)g(erhead)-2000 +45603 y(of)231 b(bytecode)h(\(or)e(AST\))g(dispatch)h(and)h(the)f(o)-15 +b(v)g(erhead)232 b(of)e(the)h(interpreter')-55 b(s)-2000 +46710 y(data)364 b(structures,)f(such)h(as)f(operand)i(stack)f(etc.)g +(The)g(second)g(important)-2000 47817 y(problem)241 b(that)f(an)-15 +b(y)241 b(JIT)e(for)h(a)h(dynamic)g(language)g(needs)g(to)f(solv)-15 +b(e)241 b(is)e(ho)-25 b(w)-2000 48924 y(to)228 b(deal)g(with)g(the)f(o) +-15 b(v)g(erhead)229 b(of)f(boxing)g(primiti)-25 b(v)-15 +b(e)228 b(types)f(and)h(of)g(type)g(dis-)-2000 50031 +y(patching.)221 b(Those)e(are)h(problems)f(that)h(are)f(usually)h(not)g +(present)f(or)g(at)h(least)-2000 51138 y(less)248 b(se)-25 +b(v)-15 b(ere)250 b(in)f(statically)h(typed)f(languages.)-672 +52245 y(Boxing)386 b(of)g(primiti)-25 b(v)-15 b(e)387 +b(types)f(is)f(necessary)i(because)g(dynamic)g(lan-)-2000 +53352 y(guages)310 b(need)h(to)f(be)g(able)h(to)f(handle)h(all)f +(objects,)g(e)-25 b(v)-15 b(en)311 b(inte)-15 b(gers,)310 +b(\003oats,)-2000 54459 y(booleans)220 b(etc.)g(in)g(the)f(same)h(w)-10 +b(ay)220 b(as)f(user)-20 b(-de\002ned)220 b(instances.)g(Thus)f(those) +-2000 55566 y(primiti)-25 b(v)-15 b(e)265 b(types)g(are)g(usually)g +Fx(boxed)p FC(,)g(i.e.,)g(a)g(small)g(heap-structure)g(is)f(al-)-2000 +56673 y(located)356 b(for)f(them)g(that)h(contains)f(the)h(actual)g(v) +-25 b(alue.)356 b(Boxing)f(primiti)-25 b(v)-15 b(e)-2000 +57780 y(types)261 b(can)h(be)g(v)-15 b(ery)262 b(costly)-65 +b(,)261 b(because)h(a)g(lot)f(of)g(common)h(operations,)g(par)-20 +b(-)-2000 58887 y(ticularly)335 b(all)g(arithmetic)h(operations,)f(ha) +-20 b(v)-15 b(e)336 b(to)f(produce)g(ne)-25 b(w)336 b(box)-15 +b(es,)335 b(in)p Black -2000 60005 13284 37 v -2000 60688 +a Fz(1)-1502 61000 y Fw(This)305 b(research)h(is)f(partially)h +(supported)e(by)g(the)h(BMBF)g(funded)f(project)g(PyJIT)-2000 +61997 y(\(nr)-49 b(.)222 b(01QE0913B;)e(Eureka)h(Eurostars\).)p +Black Black -2000 66693 a Fv(Permission)257 b(to)e(mak)-8 +b(e)255 b(digital)i(or)e(hard)h(copies)g(of)f(all)g(or)g(part)h(of)f +(this)h(w)-8 b(ork)256 b(for)f(personal)i(or)-2000 67578 +y(classroom)217 b(use)f(is)f(granted)i(without)h(fee)d(pro)-12 +b(vided)217 b(that)g(copies)f(are)f(not)h(made)f(or)h(distrib)-15 +b(uted)-2000 68464 y(for)178 b(pro\002t)h(or)e(commercial)i(adv)-19 +b(antage)179 b(and)g(that)f(copies)i(bear)e(this)h(notice)g(and)f(the)h +(full)f(citation)-2000 69349 y(on)213 b(the)g(\002rst)g(page.)g(T)-62 +b(o)213 b(cop)-8 b(y)214 b(otherwise,)h(to)e(republish,)i(to)e(post)h +(on)f(serv)-12 b(ers)214 b(or)f(to)g(redistrib)-15 b(ute)-2000 +70235 y(to)195 b(lists,)g(requires)h(prior)f(speci\002c)g(permission)h +(and/or)g(a)d(fee.)-2000 71342 y Fu(PEPM'11,)789 b Fv(January)195 +b(24\22625,)h(2011,)f(Austin,)h(T)-54 b(e)-12 b(xas,)195 +b(USA.)-2000 72227 y(Cop)-8 b(yright)1598 72204 y(c)1329 +72227 y Ft(\015)194 b Fv(2011)i(A)-31 b(CM)193 b +(978-1-4503-0485-6/11/01.)116 b(.)g(.)g($10.00)p Black +Black Black 27224 20378 a FC(addition)294 b(to)h(the)f(actual)h +(computation)g(the)-15 b(y)295 b(do.)f(Because)h(the)f(box)-15 +b(es)294 b(are)27224 21485 y(allocated)217 b(on)f(the)g(heap,)g +(producing)h(man)-15 b(y)216 b(of)g(them)g(puts)f(pressure)g(on)h(the) +27224 22592 y(g)-5 b(arbage)250 b(collector)-55 b(.)28552 +23699 y(T)-80 b(ype)330 b(dispatching)g(is)e(the)i(process)e(of)h +(\002nding)h(the)f(concrete)i(imple-)27224 24806 y(mentation)223 +b(that)f(is)f(applicable)i(to)f(the)g(objects)g(at)g(hand)h(when)g +(performing)27224 25913 y(a)262 b(generic)h(operation)f(on)g(them.)h +(An)f(e)-15 b(xample)263 b(w)-10 b(ould)263 b(be)f(the)g(addition)h(of) +27224 27020 y(tw)-10 b(o)285 b(objects:)f(F)-15 b(or)285 +b(addition)g(the)f(types)h(of)f(the)h(concrete)h(objects)e(need)i(to) +27224 28127 y(be)264 b(check)-10 b(ed)266 b(and)f(the)g(suiting)f +(implementation)i(chosen.)f(T)-80 b(ype)265 b(dispatch-)27224 +29271 y(ing)229 b(is)f(a)h(v)-15 b(ery)230 b(common)f(operation)h(in)f +(modern)44029 28848 y Fz(2)44646 29271 y FC(dynamic)h(languages)g(be-) +27224 30378 y(cause)j(no)g(types)g(are)g(kno)-25 b(wn)234 +b(at)f(compile)h(time.)f(Therefore)g(all)g(operations)27224 +31485 y(need)250 b(it.)28552 32592 y(A)230 b(recently)h(popular)f +(approach)h(to)f(implementing)h(just-in-time)e(com-)27224 +33699 y(pilers)271 b(for)h(dynamic)h(languages)f(is)g(that)g(of)g(a)g +(tracing)g(JIT)-74 b(.)271 b(A)h(tracing)h(JIT)27224 +34806 y(w)-10 b(orks)364 b(by)h(observing)f(the)h(running)g(program)g +(and)g(recording)g(its)e(com-)27224 35913 y(monly)245 +b(e)-15 b(x)g(ecuted)246 b(parts)f(into)f Fx(linear)h(e)-20 +b(xecution)246 b(tr)-15 b(aces)p FC(.)245 b(Those)f(traces)h(are)27224 +37020 y(optimized)250 b(and)f(turned)h(into)f(machine)i(code.)28552 +38127 y(One)376 b(reason)g(for)f(the)h(popularity)g(of)f(tracing)h +(JITs)e(is)g(their)i(relati)-25 b(v)-15 b(e)27224 39234 +y(simplicity)-65 b(.)489 b(The)-15 b(y)489 b(can)h(often)f(be)h(added)g +(to)f(an)g(e)-15 b(xisting)489 b(interpreter)-40 b(,)27224 +40341 y(reusing)428 b(a)g(lot)g(of)g(the)g(interpreter')-55 +b(s)428 b(infrastructure.)g(The)-15 b(y)428 b(gi)-25 +b(v)-15 b(e)429 b(some)27224 41448 y(important)251 b(optimizations)g +(lik)-10 b(e)251 b(inlining)h(and)f(constant-folding)g(for)g(free.) +27224 42555 y(A)349 b(tracing)g(JIT)e(al)-10 b(w)g(ays)350 +b(produces)f(linear)g(pieces)g(of)f(code,)i(which)f(sim-)27224 +43662 y(pli\002es)296 b(man)-15 b(y)296 b(of)g(the)h(hard)f(algorithms) +g(in)g(a)g(compiler)-40 b(,)297 b(such)f(as)g(re)-15 +b(gister)27224 44769 y(allocation.)28552 45876 y(The)308 +b(use)g(of)g(a)h(tracing)f(JIT)f(can)i(remo)-15 b(v)g(e)309 +b(the)g(o)-15 b(v)g(erhead)309 b(of)f(bytecode)27224 +46983 y(dispatch)386 b(and)f(that)h(of)f(the)h(interpreter)f(data)h +(structures.)f(In)g(this)g(paper)27224 48090 y(we)442 +b(w)-10 b(ant)442 b(to)g(present)f(a)h(ne)-25 b(w)443 +b(optimization)f(that)g(can)g(be)g(added)h(to)e(a)27224 +49197 y(tracing)228 b(JIT)f(that)i(further)f(remo)-15 +b(v)g(es)228 b(some)g(of)g(the)h(o)-15 b(v)g(erhead)229 +b(more)f(closely)27224 50304 y(associated)444 b(to)g(dynamic)h +(languages,)f(such)g(as)g(boxing)g(o)-15 b(v)g(erhead)445 +b(and)27224 51410 y(type)355 b(dispatching.)h(Our)f(e)-15 +b(xperimental)357 b(platform)e(is)g(the)g(PyPy)g(project,)27224 +52517 y(which)283 b(is)g(an)g(en)-40 b(vironment)284 +b(for)e(implementing)i(dynamic)g(programming)27224 53624 +y(languages.)472 b(PyPy)f(and)h(tracing)f(JITs)f(are)h(described)h(in)f +(more)g(detail)27224 54731 y(in)386 b(Section)i(2.)e(Section)i(3)e +(analyzes)i(the)f(problem)g(to)g(be)g(solv)-15 b(ed)386 +b(more)27224 55838 y(closely)-65 b(.)28552 56945 y(The)297 +b(core)g(of)f(our)h(trace)g(optimization)h(technique)g(can)f(be)g(vie) +-25 b(wed)298 b(as)27224 58052 y(partial)258 b(e)-25 +b(v)g(aluation:)259 b(the)g(partial)f(e)-25 b(v)g(aluation)259 +b(performs)e(a)i(form)e(of)h(escape)27224 59159 y(analysis)223 +b([4)o(])g(on)g(the)h(traces)f(and)g(mak)-10 b(es)224 +b(some)e(objects)i(that)f(are)g(allocated)27224 60266 +y(in)342 b(the)g(trace)h Fx(static,)e FC(which)i(means)f(that)h(the)-15 +b(y)342 b(do)g(not)g(occur)h(an)-15 b(y)342 b(more)27224 +61373 y(in)366 b(the)h(optimized)g(trace.)g(This)f(technique)i(is)d +(informally)i(described)g(in)27224 62480 y(Section)394 +b(4;)g(a)g(more)g(formal)g(description)g(is)g(gi)-25 +b(v)-15 b(en)394 b(in)g(Section)h(5.)f(The)27224 63587 +y(introduced)485 b(techniques)g(are)f(e)-25 b(v)g(aluated)486 +b(in)e(Section)g(6)g(using)g(PyPy')-55 b(s)27224 64694 +y(Python)249 b(interpreter)-55 b(.)28552 65801 y(The)249 +b(contrib)-20 b(utions)250 b(made)g(by)f(this)g(paper)g(are:)p +Black 27224 69407 V 27224 70090 a Fz(2)27722 70403 y +Fw(F)-13 b(or)197 b(languages)g(in)g(the)h(LISP)f(f)-9 +b(amily)-58 b(,)199 b(basic)f(arithmetic)g(operations)f(are)g +(typically)27224 71399 y(not)209 b(o)-13 b(v)g(erloaded;)208 +b(e)-22 b(v)-13 b(en)209 b(in)g(Smalltalk,)i(type)e(dispatching)h(is)g +(much)f(simpler)h(than)f(in)27224 72395 y(Python)221 +b(or)g(Ja)-18 b(v)c(aScript.)p Black Black Black eop +end +%%Page: 2 2 +TeXDict begin 2 1 bop Black Black Black -1696 886 a FC(1.)p +Black 388 w(A)211 b(description)g(of)f(a)h(practical,)h(ef)-25 +b(\002cient)211 b(and)h(ef)-25 b(fecti)g(v)-15 b(e)211 +b(algorithm)g(for)-561 1993 y(remo)-15 b(ving)250 b(object)g +(allocations)g(in)f(a)g(tracing)h(JIT)-74 b(.)p Black +-1696 3542 a(2.)p Black 388 w(A)249 b(characterization)j(of)c(this)h +(algorithm)h(as)f(partial)g(e)-25 b(v)g(aluation.)p Black +-1696 5092 a(3.)p Black 388 w(Performance)250 b(benchmarks)g(for)f +(this)f(algorithm.)-2000 7777 y FA(2.)1218 b(Backgr)-22 +b(ound)-2000 9327 y Fs(2.1)997 b(PyPy)-2000 10876 y FC(The)359 +b(w)-10 b(ork)358 b(described)i(in)e(this)g(paper)h(w)-10 +b(as)359 b(done)g(in)g(the)g(conte)-15 b(xt)359 b(of)g(the)-2000 +12021 y(PyPy)365 b(project)3235 11598 y Fz(3)3987 12021 +y FC([26].)g(PyPy)f(is)g(an)h(en)-40 b(vironment)366 +b(where)f(dynamic)h(lan-)-2000 13128 y(guages)374 b(can)f(be)h +(implemented)g(in)f(a)h(simple)f(yet)g(ef)-25 b(\002cient)374 +b(w)-10 b(ay)-65 b(.)374 b(When)-2000 14235 y(implementing)317 +b(a)e(language)i(with)f(PyPy)g(one)g(writes)f(an)h Fx(interpr)-37 +b(eter)338 b FC(for)-2000 15342 y(the)393 b(language)h(in)f +Fx(RPython)g FC([1].)f(RPython)h(\("restricted)g(Python"\))g(is)g(a) +-2000 16449 y(subset)232 b(of)h(Python)g(chosen)h(in)f(such)g(a)g(w)-10 +b(ay)234 b(that)f(type)g(inference)h(becomes)-2000 17556 +y(possible.)371 b(The)g(language)h(interpreter)g(can)f(then)h(be)f +(compiled)h(\(\223trans-)-2000 18663 y(lated\224\))269 +b(with)f(PyPy')-55 b(s)267 b(tools)h(into)g(a)g(VM)g(on)g(C)f(le)-25 +b(v)-15 b(el.)269 b(During)f(translation)-2000 19770 +y(to)217 b(C,)f(man)-15 b(y)218 b(lo)-25 b(w-le)g(v)-15 +b(el)218 b(aspects)e(of)h(the)g(\002nal)g(VM,)g(such)g(as)g(object)g +(layout,)-2000 20876 y(g)-5 b(arbage)320 b(collection)f(and)g(memory)g +(model,)g(are)g(w)-10 b(o)-15 b(v)g(en)320 b(into)f(the)g(gener)-20 +b(-)-2000 21983 y(ated)246 b(code.)h(Therefore)f(the)g(interpreter)g +(itself)f(can)i(remain)f(at)g(a)g(relati)-25 b(v)-15 +b(ely)-2000 23090 y(high)249 b(le)-25 b(v)-15 b(el)250 +b(of)f(abstraction.)-672 24197 y(A)460 b(number)f(of)g(languages)h(ha) +-20 b(v)-15 b(e)461 b(been)f(implemented)g(with)g(PyPy)-65 +b(.)-2000 25304 y(The)380 b(project)g(w)-10 b(as)380 +b(initiated)g(to)g(get)g(a)g(better)g(Python)g(implementation,)-2000 +26411 y(which)315 b(inspired)f(the)g(name)h(of)e(the)i(project)f(and)h +(is)e(still)g(the)i(main)f(focus)-2000 27518 y(of)445 +b(de)-25 b(v)-15 b(elopment.)446 b(In)f(addition)g(a)h(number)f(of)f +(other)i(languages)f(were)-2000 28625 y(implemented,)211 +b(among)f(them)g(a)g(Prolog)f(interpreter)h([7],)f(a)h(Smalltalk)g(VM) +-2000 29732 y([6])249 b(and)g(a)h(GameBo)-10 b(y)250 +b(emulator)f([8].)-672 30839 y(The)289 b(feature)f(that)h(mak)-10 +b(es)289 b(PyPy)f(more)g(than)h(a)f(compiler)h(with)g(a)f(run-)-2000 +31946 y(time)339 b(system)f(is)g(its)g(support)g(for)g(automated)i(JIT) +e(compiler)h(generation)-2000 33053 y([5].)317 b(During)h(the)g +(translation)f(to)h(C,)e(PyPy')-55 b(s)317 b(tools)g(can)i(generate)f +(a)g(trac-)-2000 34160 y(ing)400 b(just-in-time)g(compiler)h(for)f(the) +g(language)i(that)e(the)h(interpreter)g(is)-2000 35267 +y(implementing.)343 b(This)d(process)h(is)g(mostly)g(automatic;)h(it)g +(only)f(needs)h(to)-2000 36374 y(be)i(guided)g(by)g(the)g(language)h +(implementer)g(using)e(a)h(small)f(number)h(of)-2000 +37481 y(source-code)274 b(hints)f(in)g(the)g(interpreter)-55 +b(.)274 b(Mostly-automatically)f(generat-)-2000 38588 +y(ing)211 b(a)f(JIT)g(compiler)h(has)f(man)-15 b(y)211 +b(adv)-25 b(antages)212 b(o)-15 b(v)g(er)211 b(writing)g(one)g +(manually)-65 b(,)-2000 39695 y(an)267 b(error)-20 b(-prone)266 +b(and)h(tedious)f(process.)g(By)g(construction,)h(the)g(generated)-2000 +40802 y(JIT)235 b(has)g(the)i(same)f(semantics)f(as)h(the)g +(interpreter)-55 b(.)237 b(Optimizations)f(can)h(be)-2000 +41909 y(shared)249 b(between)i(dif)-25 b(ferent)249 b(languages)h +(implemented)h(with)e(PyPy)-65 b(.)-672 43016 y(Moreo)-15 +b(v)g(er)-40 b(,)300 b(thanks)g(to)f(the)h(internal)f(design)h(of)f +(the)h(JIT)e(generator)-40 b(,)300 b(it)-2000 44123 y(is)243 +b(v)-15 b(ery)244 b(easy)g(to)f(add)h(ne)-25 b(w)244 +b Fx(bac)-20 b(k)-10 b(ends)245 b FC(for)e(producing)h(the)g(actual)h +(machine)-2000 45230 y(code.)227 b(Examples)f(of)g(JIT)e(back)-10 +b(ends)227 b(that)g(are)f(implemented)h(are)f(those)g(for)-2000 +46337 y(Intel)347 b(x86)g(and)g(x86-64)g(and)g(an)g(e)-15 +b(xperimental)348 b(one)f(for)f(the)h(CLI)e(.NET)-2000 +47444 y(V)-60 b(irtual)250 b(Machine)g([12].)-2000 49529 +y Fs(2.2)997 b(T)-74 b(racing)249 b(JIT)f(Compilers)-2000 +51078 y FC(T)-35 b(racing)306 b(JITs)f(are)h(a)h(recently)g(popular)f +(approach)h(to)f(write)h(just-in-time)-2000 52185 y(compilers)274 +b(for)g(dynamic)h(languages.)g(Their)f(origins)g(lie)g(in)g(the)h +(Dynamo)-2000 53292 y(project,)324 b(which)h(used)e(a)h(tracing)g +(approach)h(to)f(optimize)g(machine)h(code)-2000 54399 +y(using)249 b(e)-15 b(x)g(ecution)250 b(traces)f([2].)f(T)-35 +b(racing)249 b(JITs)f(ha)-20 b(v)-15 b(e)249 b(then)h(be)f(adapted)h +(to)f(be)-2000 55506 y(used)299 b(for)f(a)h(v)-15 b(ery)300 +b(light-weight)f(Ja)-20 b(v)-25 b(a)299 b(VM)g([15])g(and)g(afterw)-10 +b(ards)299 b(used)g(in)-2000 56613 y(se)-25 b(v)-15 b(eral)270 +b(implementations)h(of)f(dynamic)g(languages,)h(such)f(as)f(Ja)-20 +b(v)-25 b(aScript)-2000 57758 y([13],)249 b(Lua)1707 +57334 y Fz(4)2344 57758 y FC(and)g(no)-25 b(w)250 b(Python)f(\(and)h +(other)f(languages\))h(via)g(PyPy)-65 b(.)-672 58865 +y(The)270 b(core)g(idea)g(of)f(tracing)h(JITs)e(is)h(to)h(focus)f(the)h +(optimization)g(ef)-25 b(fort)-2000 59972 y(of)338 b(the)h(JIT)f +(compiler)h(on)g(the)g(commonly)g(e)-15 b(x)g(ecuted,)341 +b(i.e.,)e Fx(hot)357 b FC(paths)338 b(of)-2000 61079 +y(the)402 b(core)g(loops)g(of)g(the)g(program)g(and)g(to)g(just)f(use)h +(an)g(interpreter)g(for)-2000 62185 y(the)303 b(less)f(commonly)i(e)-15 +b(x)g(ecuted)304 b(parts.)e(VMs)g(that)h(use)g(a)g(tracing)g(JIT)f(are) +-2000 63292 y(mostly)287 b(mix)-15 b(ed-mode)289 b(e)-15 +b(x)g(ecution)288 b(en)-40 b(vironments,)289 b(the)-15 +b(y)288 b(contain)g(both)g(an)-2000 64399 y(interpreter)370 +b(and)g(a)g(JIT)f(compiler)-55 b(.)370 b(By)g(def)-10 +b(ault)370 b(the)g(interpreter)g(is)f(used)-2000 65506 +y(to)h(e)-15 b(x)g(ecute)371 b(the)f(program,)g(doing)g(some)f +(light-weight)h(pro\002ling)g(at)g(the)-2000 66613 y(same)353 +b(time.)g(This)g(pro\002ling)f(is)h(used)g(to)g(identify)g(the)g(hot)g +(loops)g(of)g(the)-2000 67720 y(program.)271 b(If)e(a)i(hot)f(loop)g +(is)g(found)g(in)h(that)f(w)-10 b(ay)-65 b(,)271 b(the)g(interpreter)g +(enters)f(a)-2000 68827 y(special)318 b Fx(tr)-15 b(acing)318 +b(mode)p FC(.)g(In)f(this)g(tracing)h(mode,)g(the)g(interpreter)g +(tries)f(to)p Black -2000 70104 13284 37 v -2000 70788 +a Fz(3)-1502 71100 y Fr(http://pypy.org)-2000 72083 y +Fz(4)-1502 72395 y Fr(http://luajit.org/)p Black Black +Black 27224 886 a FC(record)225 b(all)g(operations)g(that)g(it)g(is)f +(e)-15 b(x)g(ecuting)226 b(while)g(running)f(one)g(iteration)27224 +1993 y(of)291 b(the)g(hot)g(loop.)h(This)e(history)g(of)h(e)-15 +b(x)g(ecuted)293 b(operations)f(of)e(one)i(loop)f(is)27224 +3099 y(called)307 b(a)g Fx(tr)-15 b(ace)p FC(.)307 b(Because)g(the)g +(trace)g(corresponds)f(to)h(one)g(iteration)g(of)f(a)27224 +4206 y(loop,)337 b(it)h(al)-10 b(w)g(ays)337 b(ends)h(with)f(a)h(jump)f +(to)h(its)e(o)-25 b(wn)338 b(be)-15 b(ginning.)339 b(The)e(trace)27224 +5313 y(also)406 b(contains)h(all)g(operations)f(that)h(are)g(performed) +g(in)f(functions)g(that)27224 6420 y(were)303 b(called)g(in)f(the)h +(loop,)f(thus)g(a)h(tracing)f(JIT)g(automatically)h(performs)27224 +7527 y(inlining.)329 b(This)g(trace)h(of)f(operations)h(subsequently)g +(forms)e(the)i(basis)f(of)27224 8634 y(the)372 b(generated)g(code.)h +(The)e(trace)h(is)f(\002rst)g(optimized,)h(and)g(then)g(turned)27224 +9741 y(into)205 b(machine)g(code.)h(Both)e(optimization)i(and)f +(machine)h(code)g(generation)27224 10848 y(are)225 b(simple,)f(because) +i(the)f(traces)g(are)g(linear)-55 b(.)225 b(This)f(linearity)h(mak)-10 +b(es)225 b(man)-15 b(y)27224 11955 y(optimizations)379 +b(a)g(lot)f(more)h(tractable,)g(and)h(the)e(inlining)h(that)g(happens) +27224 13062 y(gi)-25 b(v)-15 b(es)249 b(the)h(optimizations)f +(automatically)i(more)e(conte)-15 b(xt)250 b(to)f(w)-10 +b(ork)250 b(with.)28552 14169 y(Since)234 b(the)g(trace)g(corresponds)f +(to)h(one)g(concrete)g(e)-15 b(x)g(ecution)235 b(of)f(a)f(loop,)27224 +15276 y(the)358 b(code)g(generated)h(from)e(it)h(is)f(only)h(one)g +(possible)f(path)h(through)g(the)27224 16383 y(loop.)303 +b(T)-80 b(o)303 b(mak)-10 b(e)304 b(sure)e(that)i(the)f(trace)h +(maintains)f(the)g(correct)h(semantics,)27224 17490 y(it)374 +b(contains)h(a)g Fx(guar)-37 b(d)402 b FC(at)375 b(all)g(places)g +(where)g(the)g(e)-15 b(x)g(ecution)376 b(could)g(ha)-20 +b(v)-15 b(e)27224 18597 y(di)-25 b(v)-15 b(er)d(ged)245 +b(from)e(the)h(path.)h(Those)e(guards)h(check)h(the)f(assumptions)f +(under)27224 19704 y(which)369 b(e)-15 b(x)g(ecution)370 +b(can)g(stay)e(on)h(the)g(trace.)h(As)e(an)h(e)-15 b(xample,)370 +b(if)e(a)h(loop)27224 20811 y(contains)419 b(an)h(if-statement,)f(the)g +(trace)h(will)f(contain)h(the)f(e)-15 b(x)g(ecution)421 +b(of)27224 21918 y(one)307 b(of)g(the)g(paths)g(only)-65 +b(,)308 b(which)g(is)e(the)h(path)h(that)f(w)-10 b(as)307 +b(tak)-10 b(en)308 b(during)f(the)27224 23025 y(production)386 +b(of)f(the)h(trace.)g(The)g(trace)g(will)g(also)f(contain)i(a)e(guard)h +(that)27224 24132 y(checks)293 b(that)f(the)h(condition)g(of)f(the)h +(if-statement)f(is)f(the)i(same)g(as)f(during)27224 25239 +y(tracing,)249 b(because)i(if)d(it)h(isn')-18 b(t,)249 +b(the)g(rest)g(of)g(the)g(trace)h(w)-10 b(ould)250 b(not)f(be)h(v)-25 +b(alid.)28552 26346 y(When)391 b(generating)f(machine)h(code,)g(e)-25 +b(v)-15 b(ery)390 b(guard)g(is)f(turned)h(into)f(a)27224 +27453 y(quick)291 b(check)i(to)d(see)i(whether)f(the)h(assumption)f +(still)f(holds.)h(When)h(such)27224 28560 y(a)377 b(guard)g(is)f(hit)g +(during)h(the)g(e)-15 b(x)g(ecution)378 b(of)e(the)h(machine)h(code)f +(and)g(the)27224 29667 y(assumption)388 b(does)g(not)g(hold,)g(the)h(e) +-15 b(x)g(ecution)389 b(of)f(the)g(machine)h(code)g(is)27224 +30773 y(stopped,)261 b(and)g(interpreter)g(continues)g(to)f(run)h(from) +f(that)h(point)g(on.)g(These)27224 31880 y(guards)249 +b(are)i(the)f(only)g(mechanism)g(to)g(stop)g(the)g(e)-15 +b(x)g(ecution)251 b(of)f(a)g(trace,)g(the)27224 32987 +y(loop)f(end)h(condition)g(also)f(tak)-10 b(es)249 b(the)h(form)e(of)h +(a)h(guard.)28552 34094 y(If)242 b(one)i(speci\002c)f(guard)h(f)-10 +b(ails)242 b(a)h(lot)g(\(i.e.,)g(more)h(than)f(some)g(threshold\),) +27224 35201 y(the)327 b(tracing)g(JIT)f(will)h(generate)h(a)f(ne)-25 +b(w)328 b(trace)f(that)g(starts)f(e)-15 b(xactly)328 +b(at)f(the)27224 36308 y(position)210 b(of)g(the)h(f)-10 +b(ailing)211 b(guard)g([14].)f(The)h(e)-15 b(xisting)210 +b(assembler)h(is)e(patched)27224 37415 y(to)469 b(jump)g(to)g(the)h(ne) +-25 b(w)470 b(trace)f(when)h(the)g(guard)f(f)-10 b(ails.)469 +b(This)g(approach)27224 38522 y(guarantees)297 b(that)f(all)g(the)g +(hot)g(paths)g(in)g(the)g(program)g(will)g(e)-25 b(v)-15 +b(entually)297 b(be)27224 39629 y(traced)250 b(and)f(compiled)h(into)g +(ef)-25 b(\002cient)250 b(code.)27224 43171 y Fs(2.3)996 +b(Running)249 b(Example)27224 44721 y FC(F)-15 b(or)250 +b(the)h(purpose)g(of)f(this)h(paper)-40 b(,)251 b(we)g(are)g(going)g +(to)g(use)g(a)g(tin)-15 b(y)250 b(interpreter)27224 45828 +y(for)290 b(a)i(dynamic)g(language)g(with)g(a)f(v)-15 +b(ery)291 b(simple)g(object)h(model,)g(that)f(just)27224 +46935 y(supports)296 b(an)h(inte)-15 b(ger)298 b(and)f(a)g(\003oat)h +(type.)f(The)g(objects)g(support)g(only)g(tw)-10 b(o)27224 +48042 y(operations,)322 b Fr(add)p Black Black FC(,)g(which)g(adds)g +(tw)-10 b(o)322 b(objects)g(\(promoting)g(ints)f(to)h(\003oats)27224 +49149 y(in)233 b(a)g(mix)-15 b(ed)234 b(addition\))f(and)h +Fr(is)38184 48940 y(_)38717 49149 y(positive)p Black +Black 1 w FC(,)f(which)h(returns)e(whether)i(the)27224 +50256 y(number)259 b(is)f(greater)h(than)g(zero.)g(The)g +(implementation)h(of)e Fr(add)p Black Black 259 w FC(uses)g(clas-)27224 +51363 y(sical)c(Smalltalk-lik)-10 b(e)255 b(double-dispatching.)h(The)e +(classes)g(can)h(be)g(seen)g(in)27224 52470 y(Figure)249 +b(1)g(\(written)g(in)h(RPython\).)28552 53577 y(Using)405 +b(these)g(classes)f(to)h(implement)g(arithmetic)h(sho)-25 +b(ws)404 b(the)h(basic)27224 54684 y(problem)315 b(of)h(a)f(dynamic)h +(language)h(implementation.)g(All)e(the)h(numbers)27224 +55791 y(are)e(instances)g(of)g(either)g Fr(BoxedInteger)p +Black Black 315 w FC(or)g Fr(BoxedFloat)p Black Black +1 w FC(,)g(therefore)27224 56898 y(the)-15 b(y)296 b(consume)h(space)g +(on)f(the)g(heap.)h(Performing)f(man)-15 b(y)297 b(arithmetic)g(op-) +27224 58005 y(erations)319 b(produces)g(lots)f(of)h(g)-5 +b(arbage)320 b(quickly)-65 b(,)320 b(putting)f(pressure)f(on)h(the) +27224 59112 y(g)-5 b(arbage)310 b(collector)-55 b(.)310 +b(Using)f(double)g(dispatching)h(to)f(implement)h(the)f(nu-)27224 +60219 y(meric)192 b(to)-25 b(wer)193 b(needs)f(tw)-10 +b(o)193 b(method)g(calls)f(per)g(arithmetic)h(operation,)g(which)27224 +61326 y(is)248 b(costly)h(due)h(to)f(the)h(method)g(dispatch.)28552 +62433 y(Let)276 b(us)g(no)-25 b(w)277 b(consider)g(a)f(simple)g +(\223interpreter\224)i(function)e Fr(f)p Black Black +277 w FC(that)g(uses)27224 63540 y(the)213 b(object)g(model)g(\(see)g +(the)g(bottom)g(of)f(Figure)h(1\).)g(The)f(loop)h(in)g +Fr(f)p Black Black 213 w FC(iterates)27224 64646 y Fr(y)p +Black Black 334 w FC(times,)335 b(and)g(computes)g(something)f(in)h +(the)g(process.)f(Simply)g(running)27224 65753 y(this)277 +b(function)h(is)f(slo)-25 b(w)-65 b(,)278 b(because)h(there)f(are)h +(lots)e(of)g(virtual)h(method)h(calls)27224 66860 y(inside)264 +b(the)h(loop,)g(one)g(for)f(each)i Fr(is)39920 66651 +y(_)40453 66860 y(positive)p Black Black 265 w FC(and)f(e)-25 +b(v)-15 b(en)266 b(tw)-10 b(o)265 b(for)f(each)27224 +67967 y(call)446 b(to)g Fr(add)p Black Black 1 w FC(.)g(These)g(method) +g(calls)g(need)h(to)f(check)i(the)e(type)h(of)e(the)27224 +69074 y(in)-40 b(v)-20 b(olv)-15 b(ed)349 b(objects)f(repeatedly)i(and) +e(redundantly)-65 b(.)350 b(In)e(addition,)h(a)f(lot)g(of)27224 +70181 y(objects)224 b(are)g(created)h(when)g(e)-15 b(x)g(ecuting)225 +b(that)f(loop,)g(man)-15 b(y)225 b(of)e(these)h(objects)27224 +71288 y(are)341 b(short-li)-25 b(v)-15 b(ed.)342 b(The)f(actual)i +(computation)f(that)g(is)f(performed)g(by)h Fr(f)p Black +Black 341 w FC(is)27224 72395 y(simply)249 b(a)g(sequence)h(of)f +(\003oat)h(or)f(inte)-15 b(ger)249 b(additions.)p Black +Black eop end +%%Page: 3 3 +TeXDict begin 3 2 bop Black Black Black -2001 1315 a +Fq(c)o(l)o(a)o(s)o(s)463 b Fp(B)o(a)o(s)o(e\()n(o)o(b)o(j)o(e)o(c)o(t)o +(\))o(:)-606 2245 y Fq(p)o(a)o(s)o(s)-2001 4105 y(c)o(l)o(a)o(s)o(s)g +Fp(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)o(\()n(B)o(a)o(s)o(e)o +(\))o(:)-606 5034 y Fq(d)o(e)o(f)1253 4851 y Fp(_)o(_)2183 +5034 y(i)o(n)o(i)o(t)4043 4851 y(_)o(_)4973 5034 y(\()o(s)o(e)o(l)o(f)o +(,)g(i)o(n)o(t)o(v)o(a)o(l)o(\))o(:)788 5964 y(s)o(e)o(l)o(f.)n(i)o(n)o +(t)o(v)o(a)o(l)h(=)f(i)o(n)o(t)o(v)o(a)o(l)-606 7824 +y Fq(d)o(e)o(f)g Fp(a)o(d)o(d\()n(s)o(e)o(l)o(f)o(,)g(o)o(t)o(h)o(e)o +(r)o(\))o(:)788 8754 y Fq(r)o(e)o(t)o(u)o(r)o(n)h Fp(o)o(t)o(h)o(e)o(r) +o(.)n(a)o(d)o(d)8227 8571 y(_)o(_)9157 8754 y(i)o(n)o(t)o(\()o(s)o(e)o +(l)o(f)o(.)n(i)o(n)o(t)o(v)o(a)o(l)o(\))-606 10614 y +Fq(d)o(e)o(f)f Fp(a)o(d)o(d)2648 10431 y(_)o(_)3578 10614 +y(i)o(n)o(t)o(\()o(s)o(e)o(l)o(f)o(,)g(i)o(n)o(t)o(o)o(t)o(h)o(e)o(r)o +(\))o(:)788 11543 y Fq(r)o(e)o(t)o(u)o(r)o(n)h Fp(B)o(o)o(x)o(e)o(d)o +(I)o(n)o(t)o(e)o(g)o(e)o(r)n(\()o(i)o(n)o(t)o(o)o(t)o(h)o(e)o(r)g(+)f +(s)o(e)o(l)o(f)o(.)o(i)o(n)o(t)o(v)o(a)o(l)o(\))-606 +13403 y Fq(d)o(e)o(f)g Fp(a)o(d)o(d)2648 13220 y(_)o(_)3578 +13403 y(f)o(l)o(o)o(a)o(t)o(\()o(s)o(e)o(l)o(f)o(,)g(f)o(l)o(o)o(a)o(t) +o(o)o(t)o(h)o(e)o(r)o(\))o(:)788 14333 y(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l) +o(u)o(e)h(=)f(f)o(l)o(o)o(a)o(t)o(o)o(t)o(h)o(e)o(r)h(+)f(f)o(l)o(o)o +(a)o(t\()n(s)o(e)o(l)o(f)o(.)o(i)o(n)o(t)o(v)o(a)o(l)o(\))788 +15263 y Fq(r)o(e)o(t)o(u)o(r)o(n)h Fp(B)o(o)o(x)o(e)o(d)o(F)o(l)o(o)o +(a)o(t)o(\()n(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l)o(u)o(e)o(\))-606 +17123 y Fq(d)o(e)o(f)f Fp(i)o(s)2183 16940 y(_)2648 17123 +y(p)o(o)o(s)o(i)o(t)o(i)o(v)o(e)o(\()n(s)o(e)o(l)o(f)o(\))o(:)788 +18052 y Fq(r)o(e)o(t)o(u)o(r)o(n)h Fp(s)o(e)o(l)o(f)o(.)n(i)o(n)o(t)o +(v)o(a)o(l)g(>)g(0)-2001 19912 y Fq(c)o(l)o(a)o(s)o(s)f +Fp(B)o(o)o(x)o(e)o(d)o(F)o(l)o(o)o(a)o(t)o(\()o(B)o(a)o(s)o(e)n(\))o(:) +-606 20842 y Fq(d)o(e)o(f)1253 20659 y Fp(_)o(_)2183 +20842 y(i)o(n)o(i)o(t)4043 20659 y(_)o(_)4973 20842 y(\()o(s)o(e)o(l)o +(f)o(,)g(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l)o(\))o(:)788 21772 +y(s)o(e)o(l)o(f.)n(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l)h(=)f(f)o(l)o(o)o(a)o +(t)o(v)o(a)o(l)-606 23631 y Fq(d)o(e)o(f)g Fp(a)o(d)o(d\()n(s)o(e)o(l)o +(f)o(,)g(o)o(t)o(h)o(e)o(r)o(\))o(:)788 24561 y Fq(r)o(e)o(t)o(u)o(r)o +(n)h Fp(o)o(t)o(h)o(e)o(r)o(.)n(a)o(d)o(d)8227 24378 +y(_)o(_)9157 24561 y(f)o(l)o(o)o(a)o(t)o(\()o(s)o(e)o(l)o(f)o(.)n(f)o +(l)o(o)o(a)o(t)o(v)o(a)o(l)o(\))-606 26421 y Fq(d)o(e)o(f)f +Fp(a)o(d)o(d)2648 26238 y(_)o(_)3578 26421 y(i)o(n)o(t)o(\()o(s)o(e)o +(l)o(f)o(,)g(i)o(n)o(t)o(o)o(t)o(h)o(e)o(r)o(\))o(:)788 +27351 y(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l)o(u)o(e)h(=)f(f)o(l)o(o)o(a)o(t)o +(\()o(i)o(n)o(t)o(o)o(t)o(h)o(e)o(r)o(\))h(+)f(s)o(e)o(l)o(f)o(.)o(f)o +(l)o(o)o(a)o(t)o(v)o(a)o(l)788 28281 y Fq(r)o(e)o(t)o(u)o(r)o(n)h +Fp(B)o(o)o(x)o(e)o(d)o(F)o(l)o(o)o(a)o(t)o(\()n(f)o(l)o(o)o(a)o(t)o(v)o +(a)o(l)o(u)o(e)o(\))-606 30140 y Fq(d)o(e)o(f)f Fp(a)o(d)o(d)2648 +29957 y(_)o(_)3578 30140 y(f)o(l)o(o)o(a)o(t)o(\()o(s)o(e)o(l)o(f)o(,)g +(f)o(l)o(o)o(a)o(t)o(o)o(t)o(h)o(e)o(r)o(\))o(:)788 31070 +y Fq(r)o(e)o(t)o(u)o(r)o(n)h Fp(B)o(o)o(x)o(e)o(d)o(F)o(l)o(o)o(a)o(t)o +(\()n(f)o(l)o(o)o(a)o(t)o(o)o(t)o(h)o(e)o(r)g(+)f(s)o(e)o(l)o(f)o(.)o +(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l)o(\))-606 32930 y Fq(d)o(e)o(f)g +Fp(i)o(s)2183 32747 y(_)2648 32930 y(p)o(o)o(s)o(i)o(t)o(i)o(v)o(e)o +(\()n(s)o(e)o(l)o(f)o(\))o(:)788 33860 y Fq(r)o(e)o(t)o(u)o(r)o(n)h +Fp(s)o(e)o(l)o(f)o(.)n(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l)g(>)g(0)o(.)o(0) +-2001 36649 y Fq(d)o(e)o(f)g Fp(f)o(\()o(y)n(\))o(:)-606 +37579 y(r)o(e)o(s)g(=)f(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)n +(\()o(0)o(\))-607 38509 y Fq(w)o(h)o(i)o(l)o(e)i Fp(y)o(.)n(i)o(s)4043 +38326 y(_)4508 38509 y(p)o(o)o(s)o(i)o(t)o(i)o(v)o(e)n(\()o(\))o(:)788 +39439 y(r)o(e)o(s)g(=)e(r)o(e)o(s)o(.)o(a)o(d)o(d)o(\()o(y)n(\))o(.)o +(a)o(d)o(d)o(\()o(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)n(\()n +(-)o(1)o(0)o(0)o(\))o(\))789 40369 y(y)h(=)g(y)n(.)o(a)o(d)o(d)o(\()n +(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)o(\()n(-)o(1)o(\))o(\)) +-607 41299 y Fq(r)o(e)o(t)o(u)o(r)o(n)g Fp(r)o(e)o(s)p +Black Black -2000 42279 26568 37 v -2000 43407 a Fs(Figur)-18 +b(e)269 b(1.)499 b FC(An)269 b(\223Interpreter\224)h(for)e(a)h(T)-35 +b(in)-15 b(y)269 b(Dynamic)h(Language)h(Written)-2000 +44514 y(in)249 b(RPython)p Black -672 48119 a(If)336 +b(the)h(function)f(is)g(e)-15 b(x)g(ecuted)338 b(using)e(the)g(tracing) +h(JIT)-74 b(,)336 b(with)g Fr(y)p Black Black 336 w FC(being)-2000 +49226 y(a)383 b Fr(BoxedInteger)p Black Black 1 w FC(,)g(the)h +(produced)f(trace)h(looks)f(lik)-10 b(e)383 b(the)g(one)h(of)e(Fig-) +-2000 50333 y(ure)268 b(2)f(\(lines)g(starting)h(with)g(a)f(hash)h +(\223#\224)g(are)g(comments\).)g(The)g(trace)g(cor)-20 +b(-)-2000 51440 y(responds)249 b(to)g(one)h(iteration)f(of)g(the)h +(while-loop)f(in)h Fr(f)p Black Black FC(.)-672 52547 +y(The)364 b(operations)g(in)f(the)h(trace)h(are)e(indented)i +(corresponding)f(to)g(the)-2000 53654 y(stack)323 b(le)-25 +b(v)-15 b(el)324 b(of)e(the)i(function)f(that)g(contains)g(the)h +(traced)f(operation.)h(The)-2000 54761 y(trace)358 b(is)e(in)h +(single-assignment)g(form,)f(meaning)i(that)g(each)g(v)-25 +b(ariable)358 b(is)-2000 55868 y(assigned)223 b(a)h(v)-25 +b(alue)224 b(e)-15 b(xactly)225 b(once.)f(The)f(ar)-18 +b(guments)224 b Fo(p)16757 55979 y Fn(0)17442 55868 y +FC(and)f Fo(p)19617 55979 y Fn(1)20302 55868 y FC(of)g(the)h(loop)-2000 +56975 y(correspond)267 b(to)f(the)g(li)-25 b(v)-15 b(e)267 +b(v)-25 b(ariables)267 b Fr(y)p Black Black 266 w FC(and)g +Fr(res)p Black Black 267 w FC(in)f(the)h(while-loop)g(of)f(the)-2000 +58081 y(original)249 b(function.)-672 59188 y(The)305 +b(operations)h(in)f(the)g(trace)h(correspond)f(to)g(the)g(operations)h +(in)f(the)-2000 60295 y(RPython)249 b(program)g(in)g(Figure)h(1:)p +Black -1419 62335 a Fm(\017)p Black -561 62433 a Fr(new)p +Black Black 249 w FC(creates)g(a)f(ne)-25 b(w)250 b(object.)p +Black -1419 63884 a Fm(\017)p Black -561 63982 a Fr(get)p +Black Black 249 w FC(reads)g(an)f(attrib)-20 b(ute)250 +b(of)f(an)g(object.)p Black -1419 65434 a Fm(\017)p Black +-561 65532 a Fr(set)p Black Black 249 w FC(writes)g(to)g(an)h(attrib) +-20 b(ute)250 b(of)f(an)g(object.)p Black -1419 66984 +a Fm(\017)p Black -561 67082 a Fr(guard)2104 66873 y(_)2637 +67082 y(class)p Black Black 224 w FC(is)222 b(a)g(precise)h(type)g +(check)h(and)f(precedes)g(an)g(\(inlined\))-561 68189 +y(method)250 b(call)g(and)f(is)g(follo)-25 b(wed)250 +b(by)f(the)g(trace)h(of)f(the)h(called)g(method.)p Black +-1419 69640 a Fm(\017)p Black -561 69738 a Fr(int)1038 +69529 y(_)1571 69738 y(add)p Black Black 492 w FC(and)492 +b Fr(int)7191 69529 y(_)7724 69738 y(gt)p Black Black +492 w FC(are)f(inte)-15 b(ger)492 b(addition)g(and)g(comparison)-561 +70845 y(\(\223greater)250 b(than\224\),)g(respecti)-25 +b(v)-15 b(ely)-65 b(.)p Black -1419 72297 a Fm(\017)p +Black -561 72395 a Fr(guard)2104 72186 y(_)2637 72395 +y(true)p Black Black 250 w FC(checks)250 b(that)g(a)f(boolean)h(is)f +(true.)p Black Black Black 51577 1315 a Fl(1)p 0.3 0.3 0.3 +TeXcolorrgb -24631 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 464 w(a)o(r)o(g)o(u)o(m)o(e)o(n)o(t)o(s)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +463 w(t)o(o)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 464 w(t)o(h)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 463 w(t)o(r)o(a)o(c)o(e)p +Black 0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 465 w Fj(p)39768 1426 y Fi(0)p 0.3 0.3 0.3 +TeXcolorrgb 40199 1315 a Fk(,)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 465 w Fj(p)41586 1426 y Fi(1)51577 2245 y Fl(2)p +0.3 0.3 0.3 TeXcolorrgb -24631 w Fk(#)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +463 w(f)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black +0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +464 w(r)o(e)o(s)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(.)p +Black 0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)p Black +0.3 0.3 0.3 TeXcolorrgb -2 w(\()p Black 0.3 0.3 0.3 TeXcolorrgb +-1 w(y)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(\))p Black +51577 3175 a Fl(3)-24631 b Fp(g)o(u)o(a)o(r)o(d)29548 +2992 y(_)30013 3175 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 +3286 y Fi(1)33690 3175 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)n(\))51577 4105 y Fl(4)p 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb -22772 w Fk(#)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p +Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black 0.3 0.3 0.3 +TeXcolorrgb -1 w(a)o(d)o(d)p Black 51577 5034 a Fl(5)-22771 +b Fj(i)29396 5145 y Fi(2)30293 5034 y Fp(=)463 b(g)o(e)o(t)o(\()p +Fj(p)33539 5145 y Fi(1)33970 5034 y Fp(,)h(i)o(n)o(t)o(v)o(a)o(l)o(\)) +51577 5964 y Fl(6)-22772 b Fp(g)o(u)o(a)o(r)o(d)31407 +5781 y(_)31872 5964 y(c)o(l)o(a)o(s)o(s)o(\()p Fj(p)35119 +6075 y Fi(0)35550 5964 y Fp(,)463 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)o(\))51577 6894 y Fl(7)p 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb -20912 w Fk(#)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p +Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black 0.3 0.3 0.3 +TeXcolorrgb -1 w(a)o(d)o(d)42566 6711 y(_)o(_)43496 6894 +y(i)o(n)o(t)p Black 51577 7824 a Fl(8)-20911 b Fj(i)31256 +7935 y Fi(3)32153 7824 y Fp(=)463 b(g)o(e)o(t)o(\()p +Fj(p)35399 7935 y Fi(0)35830 7824 y Fp(,)h(i)o(n)o(t)o(v)o(a)o(l)o(\)) +51577 8754 y Fl(9)-20911 b Fj(i)31256 8865 y Fi(4)32153 +8754 y Fp(=)463 b(i)o(n)o(t)34477 8571 y(_)34942 8754 +y(a)o(d)o(d)o(\()p Fj(i)37116 8865 y Fi(2)37547 8754 +y Fp(,)i Fj(i)38791 8865 y Fi(3)39222 8754 y Fp(\))51577 +9684 y Fl(10)-21188 b Fj(p)31399 9795 y Fi(5)32296 9684 +y Fp(=)463 b(n)o(e)o(w)o(\()o(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o +(e)o(r)n(\))51577 10614 y Fl(11)p 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb -19329 w Fk(#)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +463 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p +Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black 0.3 0.3 0.3 +TeXcolorrgb 43030 10431 a(_)o(_)43960 10614 y(i)o(n)o(i)o(t)45820 +10431 y(_)o(_)p Black 51577 11543 a Fl(12)-19329 b Fp(s)o(e)o(t)o(\()p +Fj(p)35119 11654 y Fi(5)35550 11543 y Fp(,)464 b(i)o(n)o(t)o(v)o(a)o(l) +o(,)g Fj(i)40513 11654 y Fi(4)40945 11543 y Fp(\))51577 +12473 y Fl(13)51577 13403 y(14)p 0.3 0.3 0.3 TeXcolorrgb +-24908 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 463 w(f)p Black +0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(\()n(-)o(1)o(0)o(0)o +(\))p Black 51577 14333 a Fl(15)-24907 b Fj(p)27680 14444 +y Fi(6)28576 14333 y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o(o)o(x)o(e)o(d)o(I) +o(n)o(t)o(e)o(g)o(e)o(r)o(\))51577 15263 y Fl(16)p 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -23049 w Fk(#)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black +0.3 0.3 0.3 TeXcolorrgb 39311 15080 a(_)o(_)40241 15263 +y(i)o(n)o(i)o(t)42101 15080 y(_)o(_)p Black 51577 16193 +a Fl(17)-23049 b Fp(s)o(e)o(t)o(\()p Fj(p)31399 16304 +y Fi(6)31830 16193 y Fp(,)464 b(i)o(n)o(t)o(v)o(a)o(l)o(,)f(-)o(1)o(0)o +(0)o(\))51577 17123 y Fl(18)51577 18052 y(19)p 0.3 0.3 0.3 +TeXcolorrgb -24908 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +463 w(f)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black +0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +464 w(.)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)p +Black 0.3 0.3 0.3 TeXcolorrgb -1 w(\()p Black 0.3 0.3 0.3 +TeXcolorrgb -1 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p +Black 0.3 0.3 0.3 TeXcolorrgb -2 w(\()n(-)o(1)o(0)o(0)o(\))o(\))p +Black 51577 18982 a Fl(20)-24908 b Fp(g)o(u)o(a)o(r)o(d)29548 +18799 y(_)30013 18982 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 +19093 y Fi(5)33690 18982 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o +(e)o(g)o(e)o(r)n(\))51577 19912 y Fl(21)p 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -23049 w Fk(#)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black +0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)p Black 51577 +20842 a Fl(22)-23048 b Fj(i)29396 20953 y Fi(7)30293 +20842 y Fp(=)463 b(g)o(e)o(t)o(\()p Fj(p)33539 20953 +y Fi(5)33970 20842 y Fp(,)h(i)o(n)o(t)o(v)o(a)o(l)o(\))51577 +21772 y Fl(23)-23049 b Fp(g)o(u)o(a)o(r)o(d)31407 21589 +y(_)31872 21772 y(c)o(l)o(a)o(s)o(s)o(\()p Fj(p)35119 +21883 y Fi(6)35550 21772 y Fp(,)463 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o +(e)o(g)o(e)o(r)o(\))51577 22702 y Fl(24)p 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -21189 w Fk(#)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black +0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)42566 22519 y(_)o(_)43496 +22702 y(i)o(n)o(t)p Black 51577 23631 a Fl(25)-21188 +b Fj(i)31256 23742 y Fi(8)32153 23631 y Fp(=)463 b(g)o(e)o(t)o(\()p +Fj(p)35399 23742 y Fi(6)35830 23631 y Fp(,)h(i)o(n)o(t)o(v)o(a)o(l)o +(\))51577 24561 y Fl(26)-21188 b Fj(i)31256 24672 y Fi(9)32153 +24561 y Fp(=)463 b(i)o(n)o(t)34477 24378 y(_)34942 24561 +y(a)o(d)o(d)o(\()p Fj(i)37116 24672 y Fi(7)37547 24561 +y Fp(,)i Fj(i)38791 24672 y Fi(8)39222 24561 y Fp(\))51577 +25491 y Fl(27)-21188 b Fj(p)31399 25602 y Fi(10)32672 +25491 y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)o(\))51577 26421 y Fl(28)p 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb -19329 w Fk(#)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +463 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p +Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black 0.3 0.3 0.3 +TeXcolorrgb 43030 26238 a(_)o(_)43960 26421 y(i)o(n)o(i)o(t)45820 +26238 y(_)o(_)p Black 51577 27351 a Fl(29)-19329 b Fp(s)o(e)o(t)o(\()p +Fj(p)35119 27462 y Fi(10)35927 27351 y Fp(,)463 b(i)o(n)o(t)o(v)o(a)o +(l)o(,)i Fj(i)40890 27462 y Fi(9)41321 27351 y Fp(\))51577 +28281 y Fl(30)51577 29211 y(31)p 0.3 0.3 0.3 TeXcolorrgb +-24908 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 463 w(f)p Black +0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(\()n(-)o(1)o(\))p +Black 51577 30140 a Fl(32)-24907 b Fj(p)27680 30251 y +Fi(11)28953 30140 y Fp(=)463 b(n)o(e)o(w\()n(B)o(o)o(x)o(e)o(d)o(I)o(n) +o(t)o(e)o(g)o(e)o(r)o(\))51577 31070 y Fl(33)p 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -23049 w Fk(#)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black +0.3 0.3 0.3 TeXcolorrgb 39311 30887 a(_)o(_)40241 31070 +y(i)o(n)o(i)o(t)42101 30887 y(_)o(_)p Black 51577 32000 +a Fl(34)-23049 b Fp(s)o(e)o(t)o(\()p Fj(p)31399 32111 +y Fi(11)32207 32000 y Fp(,)464 b(i)o(n)o(t)o(v)o(a)o(l)o(,)f(-)o(1)o +(\))51577 32930 y Fl(35)51577 33860 y(36)p 0.3 0.3 0.3 +TeXcolorrgb -24908 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +463 w(f)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black +0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +464 w(y)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(.)p Black +0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)p Black 0.3 0.3 0.3 +TeXcolorrgb -1 w(\()p Black 0.3 0.3 0.3 TeXcolorrgb -2 +w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p Black +0.3 0.3 0.3 TeXcolorrgb -1 w(\()n(-)o(1)o(\))o(\))p Black +51577 34790 a Fl(37)-24908 b Fp(g)o(u)o(a)o(r)o(d)29548 +34607 y(_)30013 34790 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 +34901 y Fi(0)33690 34790 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o +(e)o(g)o(e)o(r)n(\))51577 35719 y Fl(38)p 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -23049 w Fk(#)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black +0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)p Black 51577 +36649 a Fl(39)-23048 b Fj(i)29396 36760 y Fi(12)30670 +36649 y Fp(=)463 b(g)o(e)o(t)o(\()p Fj(p)33916 36760 +y Fi(0)34347 36649 y Fp(,)h(i)o(n)o(t)o(v)o(a)o(l)o(\))51577 +37579 y Fl(40)-23049 b Fp(g)o(u)o(a)o(r)o(d)31407 37396 +y(_)31872 37579 y(c)o(l)o(a)o(s)o(s)o(\()p Fj(p)35119 +37690 y Fi(11)35927 37579 y Fp(,)463 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o +(e)o(g)o(e)o(r)o(\))51577 38509 y Fl(41)p 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -21189 w Fk(#)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black +0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)42566 38326 y(_)o(_)43496 +38509 y(i)o(n)o(t)p Black 51577 39439 a Fl(42)-21188 +b Fj(i)31256 39550 y Fi(13)32529 39439 y Fp(=)464 b(g)o(e)o(t)o(\()p +Fj(p)35776 39550 y Fi(11)36584 39439 y Fp(,)f(i)o(n)o(t)o(v)o(a)o(l)o +(\))51577 40369 y Fl(43)-21188 b Fj(i)31256 40480 y Fi(14)32529 +40369 y Fp(=)463 b(i)o(n)o(t)34853 40186 y(_)35318 40369 +y(a)o(d)o(d\()o Fj(i)37492 40480 y Fi(12)38300 40369 +y Fp(,)i Fj(i)39544 40480 y Fi(13)40352 40369 y Fp(\))51577 +41299 y Fl(44)-21188 b Fj(p)31399 41410 y Fi(15)32672 +41299 y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)o(\))51577 42228 y Fl(45)p 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb -19329 w Fk(#)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +463 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p +Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black 0.3 0.3 0.3 +TeXcolorrgb 43030 42045 a(_)o(_)43960 42228 y(i)o(n)o(i)o(t)45820 +42045 y(_)o(_)p Black 51577 43158 a Fl(46)-19329 b Fp(s)o(e)o(t)o(\()p +Fj(p)35119 43269 y Fi(15)35927 43158 y Fp(,)463 b(i)o(n)o(t)o(v)o(a)o +(l)o(,)i Fj(i)40890 43269 y Fi(14)41698 43158 y Fp(\))51577 +44088 y Fl(47)51577 45018 y(48)p 0.3 0.3 0.3 TeXcolorrgb +-24908 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 463 w(f)p Black +0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 464 w(y)p Black 0.3 0.3 0.3 +TeXcolorrgb -1 w(.)p Black 0.3 0.3 0.3 TeXcolorrgb -1 +w(i)o(s)34662 44835 y(_)35127 45018 y(p)o(o)o(s)o(i)o(t)o(i)o(v)o(e)p +Black 0.3 0.3 0.3 TeXcolorrgb -2 w(\()o(\))p Black 51577 +45948 a Fl(49)-24908 b Fp(g)o(u)o(a)o(r)o(d)29548 45765 +y(_)30013 45948 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 +46059 y Fi(15)34067 45948 y Fp(,)463 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o +(e)o(g)o(e)o(r)o(\))51577 46878 y Fl(50)p 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -23049 w Fk(#)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black +0.3 0.3 0.3 TeXcolorrgb -1 w(i)o(s)40241 46695 y(_)40706 +46878 y(p)o(o)o(s)o(i)o(t)o(i)o(v)o(e)p Black 51577 47808 +a Fl(51)-23048 b Fj(i)29396 47919 y Fi(16)30670 47808 +y Fp(=)463 b(g)o(e)o(t)o(\()p Fj(p)33916 47919 y Fi(15)34724 +47808 y Fp(,)g(i)o(n)o(t)o(v)o(a)o(l\))51577 48737 y +Fl(52)-23048 b Fj(i)29396 48848 y Fi(17)30670 48737 y +Fp(=)463 b(i)o(n)o(t)32994 48554 y(_)33459 48737 y(g)o(t)o(\()p +Fj(i)35168 48848 y Fi(16)35976 48737 y Fp(,)g(0)o(\))51577 +49667 y Fl(53)p 0.3 0.3 0.3 TeXcolorrgb -24908 w Fk(#)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 463 w(f)p Black 51577 50597 +a Fl(54)-24908 b Fp(g)o(u)o(a)o(r)o(d)29548 50414 y(_)30013 +50597 y(t)o(r)o(u)o(e)n(\()p Fj(i)32651 50708 y Fi(17)33459 +50597 y Fp(\))51577 51527 y Fl(55)g Fp(j)o(u)o(m)o(p)o(\()o +Fj(p)30004 51638 y Fi(15)30812 51527 y Fp(,)465 b Fj(p)32199 +51638 y Fi(10)33008 51527 y Fp(\))p Black Black 27224 +52507 V 28388 53636 a Fs(Figur)-18 b(e)249 b(2.)499 b +FC(An)249 b(Unoptimized)i(T)-35 b(race)249 b(of)g(the)h(Example)g +(Interpreter)p Black 28552 56913 a(Method)475 b(calls)g(in)f(the)h +(trace)g(are)g(preceded)h(by)f(a)g Fr(guard)50592 56704 +y(_)51125 56913 y(class)p Black Black 27224 58020 a FC(operation,)279 +b(to)f(check)i(that)e(the)h(class)f(of)g(the)g(recei)-25 +b(v)-15 b(er)280 b(is)d(the)i(same)f(as)g(the)27224 59165 +y(one)218 b(that)f(w)-10 b(as)218 b(observ)-15 b(ed)218 +b(during)f(tracing.)41980 58741 y Fz(5)42586 59165 y +FC(These)g(guards)g(mak)-10 b(e)218 b(the)g(trace)27224 +60272 y(speci\002c)273 b(to)h(the)f(situation)g(where)h +Fr(y)p Black Black 274 w FC(is)e(really)i(a)f Fr(BoxedInteger)p +Black Black 2 w FC(.)g(When)27224 61379 y(the)473 b(trace)g(is)f +(turned)h(into)g(machine)h(code)g(and)f(afterw)-10 b(ards)473 +b(e)-15 b(x)g(ecuted)27224 62485 y(with)252 b Fr(BoxedFloat)p +Black Black 1 w FC(,)g(the)g(\002rst)f Fr(guard)41015 +62276 y(_)41548 62485 y(class)p Black Black 253 w FC(instruction)h +(will)g(f)-10 b(ail)252 b(and)27224 63592 y(e)-15 b(x)g(ecution)250 +b(will)g(continue)g(using)f(the)g(interpreter)-55 b(.)28552 +64699 y(The)328 b(trace)g(sho)-25 b(ws)327 b(the)g(inef)-25 +b(\002ciencies)329 b(of)e Fr(f)p Black Black 327 w FC(clearly)-65 +b(,)328 b(if)f(one)h(looks)f(at)27224 65806 y(the)k(number)g(of)f +Fr(new)p Black Black 1 w FC(,)g Fr(set)q(/get)p Black +Black 331 w FC(and)h Fr(guard)43982 65597 y(_)44515 65806 +y(class)p Black Black 332 w FC(operations.)g(The)27224 +66913 y(number)219 b(of)f Fr(guard)34199 66704 y(_)34732 +66913 y(class)p Black Black 219 w FC(operation)h(is)f(particularly)h +(problematic,)h(not)27224 68020 y(only)270 b(because)h(of)f(the)g(time) +g(it)g(tak)-10 b(es)270 b(to)g(run)g(them.)g(All)g(guards)g(also)g(ha) +-20 b(v)-15 b(e)27224 69127 y(additional)310 b(information)f(attached)h +(that)f(mak)-10 b(es)310 b(it)e(possible)h(to)g(return)g(to)p +Black 27224 70403 13284 37 v 27224 71086 a Fz(5)27722 +71399 y Fr(guard)30387 71190 y(_)30920 71399 y(class)p +Black Black 358 w Fw(performs)357 b(a)g(precise)h(class)g(check,)g(not) +e(checking)h(for)g(sub-)27224 72395 y(classes.)p Black +Black Black eop end +%%Page: 4 4 +TeXDict begin 4 3 bop Black Black Black Black Black 4217 +32334 a @beginspecial 0 @llx -1 @lly 212 @urx 484 @ury +1272 @rwi @setspecial +%%BeginDocument: figures/obj-lifetime.eps +%!PS-Adobe-3.0 EPSF-3.0 +%%Creator: cairo 1.10.0 (http://cairographics.org) +%%CreationDate: Thu Nov 18 15:15:16 2010 +%%Pages: 1 +%%BoundingBox: 0 -1 212 484 +%%DocumentData: Clean7Bit +%%LanguageLevel: 2 +%%EndComments +%%BeginProlog +/cairo_eps_state save def +/dict_count countdictstack def +/op_count count 1 sub def +userdict begin +/q { gsave } bind def +/Q { grestore } bind def +/cm { 6 array astore concat } bind def +/w { setlinewidth } bind def +/J { setlinecap } bind def +/j { setlinejoin } bind def +/M { setmiterlimit } bind def +/d { setdash } bind def +/m { moveto } bind def +/l { lineto } bind def +/c { curveto } bind def +/h { closepath } bind def +/re { exch dup neg 3 1 roll 5 3 roll moveto 0 rlineto + 0 exch rlineto 0 rlineto closepath } bind def +/S { stroke } bind def +/f { fill } bind def +/f* { eofill } bind def +/n { newpath } bind def +/W { clip } bind def +/W* { eoclip } bind def +/BT { } bind def +/ET { } bind def +/pdfmark where { pop globaldict /?pdfmark /exec load put } + { globaldict begin /?pdfmark /pop load def /pdfmark + /cleartomark load def end } ifelse +/BDC { mark 3 1 roll /BDC pdfmark } bind def +/EMC { mark /EMC pdfmark } bind def +/cairo_store_point { /cairo_point_y exch def /cairo_point_x exch def } def +/Tj { show currentpoint cairo_store_point } bind def +/TJ { + { + dup + type /stringtype eq + { show } { -0.001 mul 0 cairo_font_matrix dtransform rmoveto } ifelse + } forall + currentpoint cairo_store_point +} bind def +/cairo_selectfont { cairo_font_matrix aload pop pop pop 0 0 6 array astore + cairo_font exch selectfont cairo_point_x cairo_point_y moveto } bind def +/Tf { pop /cairo_font exch def /cairo_font_matrix where + { pop cairo_selectfont } if } bind def +/Td { matrix translate cairo_font_matrix matrix concatmatrix dup + /cairo_font_matrix exch def dup 4 get exch 5 get cairo_store_point + /cairo_font where { pop cairo_selectfont } if } bind def +/Tm { 2 copy 8 2 roll 6 array astore /cairo_font_matrix exch def + cairo_store_point /cairo_font where { pop cairo_selectfont } if } bind def +/g { setgray } bind def +/rg { setrgbcolor } bind def +/d1 { setcachedevice } bind def +%%EndProlog +%!PS-AdobeFont-1.0: NimbusSanL-Regu 1.06 +%%Title: NimbusSanL-Regu +%Version: 1.06 +%%CreationDate: Thu Aug 2 14:35:58 2007 +%%Creator: frob +%Copyright: Copyright (URW)++,Copyright 1999 by (URW)++ Design & +%Copyright: Development; Cyrillic glyphs added by Valek Filippov (C) +%Copyright: 2001-2005 +% Generated by FontForge 20070723 (http://fontforge.sf.net/) +%%EndComments + +FontDirectory/NimbusSanL-Regu known{/NimbusSanL-Regu findfont dup/UniqueID known pop false {dup +/UniqueID get 5020902 eq exch/FontType get 1 eq and}{pop false}ifelse +{save true}{false}ifelse}{false}ifelse +11 dict begin +/FontType 1 def +/FontMatrix [0.001 0 0 0.001 0 0 ]readonly def +/FontName /f-0-0 def +/FontBBox {-174 -285 1022 953 }readonly def + +/PaintType 0 def +/FontInfo 9 dict dup begin + /version (1.06) readonly def + /Notice (Copyright \050URW\051++,Copyright 1999 by \050URW\051++ Design & Development; Cyrillic glyphs added by Valek Filippov \050C\051 2001-2005) readonly def + /FullName (Nimbus Sans L Regular) readonly def + /FamilyName (Nimbus Sans L) readonly def + /Weight (Regular) readonly def + /ItalicAngle 0 def + /isFixedPitch false def + /UnderlinePosition -151 def + /UnderlineThickness 50 def +end readonly def +/Encoding 256 array +0 1 255 {1 index exch /.notdef put} for +dup 19 /space put +dup 15 /one put +dup 16 /two put +dup 17 /three put +dup 14 /four put +dup 10 /a put +dup 5 /b put +dup 7 /c put +dup 2 /e put +dup 6 /j put +dup 18 /l put +dup 13 /m put +dup 1 /n put +dup 4 /o put +dup 11 /p put +dup 20 /r put +dup 9 /s put +dup 8 /t put +dup 12 /u put +dup 3 /w put +readonly def +currentdict end +currentfile eexec +f983ef0097ece61cf3a79690d73bfb4b0027b850f3158905fdac1bc024d7276e0a12b7ddcede59 +e3601ab4509dfe0977ed5bf624ebc1f818c45f1350d41b052a72743accb053eb06ed043568d319 +6a30bed220227e2a15bacef508449221cf338a8666e92410a9aa91d5a31900a93c01ec21742cd1 +4dc46bffa111ce10b78ae01abaeba7f36cdf79a4733245c63f6d36234d6b0961f1ac295d617793 +1b9ed554bb5fc6741a63c493daabf03d753c7d2b8e8c01e3e280898f810da5985212c8c0bbdee4 +e8ab9b22bea83671c0460443ede9be044168f8ab50be5874d46660f1f8241cb261280a68ae2cd6 +0e1648cff45c0ba9b15cb42f86217172a5b855265c214d4b954937d11b94b7b98738393ce09ce4 +0802e512bea7714fe6f163d1b27c8ec87419fa91767418abc44c94a3a22f97f856b0a4729be697 +3455a7f7ae72c671542e9e74258c2b8b2ad440a1b69bc7de2e54ed6a96d0bfde08b35f6fbf739a +e18676c03800ff3e63c6fc20927b3158e8d2cb6644eaf7831c6bf0e9b39aa14145e721dc46e792 +da9be78696808fadf24136f9da0c8003cb34ecb8af2b7de9e033545fbca53d881eaea40c638396 +c8d3b08b6e2b8f7d6188d055d6924bafb0cf7ef8bf2719495988b0efe373dd1aea97b0f486d055 +5e3a62cc1b36024ec3ada21bfe1629acf431928facb4ead9c41e3d4cb50ed2f651d2cb6ca8d0b1 +ef49d9e214e06cb029bc1a40b2052938f6bcd05f013d9ed1edc90bdea459c4483ed0d1470e30ee +0a77ecfb585268a94a8bc6be7d28e5746525107c0a3f05fb5e34e0a2157214be2d95a46fdec2e3 +6afd180688fc710c92b27d69b2ff8b1fc7f73cd5526270e5bb585d7674a351d9bda5dcbc17b40d +465923103f1d02e7446d949002657763c82c8fe0372afe94104d148679b4e5be01def615b295d7 +c25e24660b2a10dfb0b5b96fc02bdfdadc10afddd821fff6caf17f11d6a7cf157d3d7f1aa25a00 +cea41207c8e160663d23b16d2934ffe6a6c178d456facef20fd46c4b33f6d1ebf103b7cde86d7a +ff74174d2d3f5820186c696b9245f97554b7f62411429f1dabebf1ce4abebcdd2a4ccc1c06d6ff +8dd9d7f935ba5218765a81eb3c727a40dae317df89cb3234b1a4f1a0b16f88ed1684d3106c2a6c +c18aedd3b052df1a2ea7ccb8c6c6111abed60277aaa2b91f1da4a83c0b2d4fae8cfbe2873b7f9e +c2e9ad05045abd1133b95f70185dea97f1081565ac65139ff102004b00e1e77023782a68a75208 +8f8d8cd2393eb3f649553f0be59b83687f4425290ff1e334794e59ab4dc73b26f1beac4c5d6d4d +507dd7d13b8721144d28b82eba624965e3d8cf9334083a6a461277026b05583f96e489be17dcfa +ad73cc5b9c6021527879f2c194a5bb18699cbef3ea4e14950e1aa56abbd38bf509629e8a4f0edf +0946a47aea100a088151a9e296d1b5d7688aa0b3b121fb93bd52789d22fc32d645230419d1df1d +106a9a90948fd443eb64a212b3595a133a4a30434b5074ff14f69368de4e0452515257aad6332e +d29556152c71240df8a6746486d28c6a1315bc5c5ed08a5192c2fd7ba4ddcd7496139ae2e6e47a +c4971edc88a81e2056ff81081b72cec34df2e6bf1720bc5c6753ae32e31ce0db86b3aba65f2633 +5a57f2f12827f7495183427d3e861b6ba27f7157003a92a9405c8755f71dca78e6cf0730fe5950 +4cbe903beec22765da1f35eabc756bf32e9980eed40c0d1a201e12a71b63171df7ca971e36c03e +8137e0213fe675b392ad8b3232752dd3864c82f9ff85f814becb6fc7510a1e9917bcf12a90c4ee +29e036db54278f2413731df21833d1f8149344b46e4c3fc2fa9647e76b3cfd81b4fe88643e427e +7f6d9b01f99659e531df0c43b4f44ed236f117bb00830dc10a5b1896cbee06fde9b4fd9b0b1384 +05a0bb83d78ca5546a2b41490f5322405f0ea6276d9a13a63b98c59cc9a0428472759009cda518 +c4dfce2449e27df9caac365a6ee3a8ac451167c806889760c404164f941d035b6a1df55c44ebdd +a9484f4da9b07c059ae59617c0e86ffa71b9a70322acb3b6007112879946f950e04e6b634439b0 +6e05a9781a9ca6b9cbfaf508a835e2daaec957d95fa0031f9ae42c17cfd5adfb81766af25ac523 +7818909e284ae4a439b8b2a5050ddebe0dbde491d135155e98ab579904d045797ea516dbaa4529 +675a381372fd437e53f9372dae38f0eeece282771fe7c3d1e68061cc242e4640131ebfcf340b45 +3e4868ce4825067dfcc88c7d84aee3f06eb202a89979aa520e501288e93f742fc86b6f43e3bba4 +430be40b099cb0535504f966cf50fd9ba9fd358fa184a39eccfd4d8e5c4ec35b448bad9cf5bdf1 +5b729a4137a9618a5a44db7d9740a15e0da9c802b428d40ed5401ed76095c73512cb626630608c +74143afaa7be85562aac8d8148a5e603e3b69b89706b0c1cf5c85bdeb32611221c13325835238f +ee492e4a08bce9741094c4cdc3edd691e82d224c7866590510e2c7cccc07e11484f62d79303fd5 +59e4e511eb67bab756f30acecc560f5def3902bfb52abfb7aa48e72f2732b7eb73c0123f6dfe97 +cb69897c1db3d57ff05466c3cce4730e04e921a5e56e8add31933536393bc00b3ab3435ab95096 +5a752b913bfdff22a8e7df7bc441e5bb3a3192bea8a17829faa529e56ac69e5a25a6797615d545 +5e4b430627f9c219a364bf0c55669cb872e403ed1cdffc63cf9223ab581bb9d7967892dfef80da +723f18e199e0b57f12ef0210326aae328882d982392dfcd40e029ad3f53dcd820b4a61b580bc3f +1b492453f5c752e05c705bf8bc05485c5499f804a6ed2a28bb63bc7e80759003cc7ef0a1d9d51d +ab1f785d14eb86bd9a4f4221951da58fe7263e23e58a2414dcedb6ce1943f62a65699080bf568c +c0c67134d8728963466de6713327129138d203ab8db0079f63349ac8ea689ac7615d866b7f600b +2e91333b61d73c98401b1cb0432e90fa7bd87d9f4fdfecbc2dd5e5eee5daa7d7bd959e41817e17 +525a472ac035c391b918d549c38bf2b2426798fc27af235a725ddff5dc79afb59198c94528a36b +8c7cd5683f8af50107b45e02fda4c787b4f14ea4e663bbd615be5b30d6979456f6191b5051f458 +bf19c91ac99a9a8505a03abedc243ba2872c3305b81288a79f5ed8452e2a2caf3fa51208946c72 +b97e6d59ef74bd8b96ced9de45d7cf76a6891c7c0691298855a366737da9be57d98c25c7415122 +c898a1564a052036a3aa550c2bc83478c4e15933de6e4aad64b2e97c640a2e8a14933abaf85907 +f09588cb05432f716f05efe4a669d16c881dadc0faf5a704dc91f5006e512a61372579eeea5f60 +76cb8048d71a1e4a3097bc27433bc3613ebd8291e427bd9cf83c0969562d5363662f74f2ecd570 +71c4d331a26559bf3e307d0e2d1c9a90a3fc92254d2656555dbc0c8c3502985a00e705213e4522 +d5eecccc5f94af0f196501250a4200b04b491f64388f7c3aa37970a8a5bf78a072bcbac61cdbf0 +f95dbd13b8defacae1a53277fea4f19f829fced3b4cc6e209bc2616b9bd7312c87d78cda5c9f6b +ca02595086adb6dac9b7bc6c9fca577169940fb58c818a37575de491222128989965657ea8b5db +d96a697ef71778109a1781bd7ddb9b2132da71b3bac95f6f69fc615b7733d709402bc7fa0812d1 +8aa22fab627f91baf4a346d8f103362c0161a3b57d83856c63cc7150fa2ec1ba537a4c608bc28c +85d388bcf4105158f0c9d40a573bab28148104e2db167ff009d34134008cd1ab4cea4091c04dd6 +eb2353686278cb190506778602b0a642d3997caa982a2b3c84f8bc97e83a177fbe6c69512014cb +f7ab75d5a35a0521a2d1204d7877f75a8b0816da0bc058c0ccf411b939931bcbbcd1d3d92b0fee +9e58012677b1db645c14ae63b7c4c3e4b18b890802aaa72c0d8cb771ee4e46ddb6d714894aed78 +16d7164799b70d7b64e0a50b979c3616590b871cbae60b9449d02fab57655abd48843da96359d6 +087ce297a26e192491d1cb8420d7cbf1dc66b68ab4299ed1c500ebb26de35f956348d497c3c085 +8e812fd3d43cd76a41e24f19752bbc8a4a69510c9381cdb8a23031f8c0a914db40a659a5b51e19 +fc9024471cce4e65cbab05a3562a292939887967cbe49331cb58e936a2b6873d3dedfbb3784149 +14a2b78db4daa180a4b40a972fec68f931b75f2109fcf824cc7439ea693c8c080abed27376461d +d5f86617d1a931cf1caf206e872adf0d8b6e35499fe3df6d3833dca2daee3805fd76f7cc5ef2ac +6f05134e8da55b15204896002803ad497e67c2ea83b2646bc13a72f73d642ebe4b1d5b00a44474 +4160f3341a84335643e58ee6c64acdfcec21bf506a6a30b70b1fce06b97b23c767e7e429892085 +fbff2996aa68e4776b76de0f66edfd89b015cc3a3e4d70cdf3bcad9d48b4e17b97f568b0f3b155 +7d003cd6f087aab912a15e88e51f6e33c8c8b2ab69fa8e81f09a75e0059723f1de598a20c567e6 +81b60e954bd6d80c7515870bd6158eb93e8af8ae71542d681895e2b32489694f090ebc5b560515 +048626f803b996baa342815b5adfc437dace01d549a6319eff57096c537c9aea7c3f8dd978d426 +24304a68ef2427dbc0acd85a26960f10e348a30cdc46e46dc16700f62108fe6c35059759fe69c7 +29edd6c91e83cd49535e16746b6dff037927de39b77eeb86fadd9ac5c89268c3c529fa1747650c +f9e16c7626aec25817ea46c1ab2b0a7a9c7ba33087db199adf7594c297a938ddd7ad595c694255 +2ae76711f444b5b833a92cdfea0462b6766685359cd4c2afe4114d884b230fdc86d680a9b10a43 +33d68db6ebc595f86806392857b55452e44288a42f924754c22778c6d7bfc3d4337bb3a8f68055 +d0d6f5053700b784c480509caf63336c687c5bd339757008be54c289a10e62d5c83eee133f3216 +8a68aa259e8247d6afcb25243df6ca8c238ea64082265560188e73a47db7603a1802e7ed3693a9 +b7e5ed7b0332171196792cd5272b640c8a5363978a87f58b9d1022d5e7af38d1e45dbabd89b095 +eb9e581ce8e9c314a8152465a4663fd4a7ad1996c08afe96d3975d67fde3aef4fdbad38c739d0f +1bb4584f919b4b5b8a06aa94bc3a4162aaaf01cf607fd24e90dd4889795aa0fd42e4cce9b0c432 +3963e288dab83c1d1187c1251d77da1c6602311cf8a73e16699d6646bf62cf68b3da3855648508 +edb97cf3614ed1141d158ce8f86e055e9034fcc9c77fe4f592251dca7e61952859567c6ff4fa83 +15b554ff50b48bec4d99ad1221b7caa1d47a8f430a07d49f561296a74277adb9c21c8b4ca1d3bf +3b671506f71d8f88d37ec1b03432c2bcaa337659c35747b0eb1721073e4e60e93646c719bc4e27 +c5b5fed52f456f43158ad6ba09e9db6d0c17fa75bd7a80746258f7725116c37770e261c9bcf7f5 +78f3f12ec7037c125fd86e4d38ef23886e18f4e206397006c8c4ba0105e10fcc254a718e3c8ba6 +dd71218c4768fe0d8efe2e01454dd52ef33080a1313911ce3e0afa57f5b74d7caef2440ff148f3 +c8bbe0cb2418a0367dddb217ffb47fd5766ab6db331bd9d2779ceac6e8e3e6562b79d947e687ce +271f8f024c5cfe61e3453cc7ca45efc6b682d2da8acedda708fc65b668e285e6f542b9fdc07108 +2076c4ec2312de40214f8616ac615a2669bd490e89d5c125d6c0d6378113f4d37f1d6a06c225c7 +b0e879746f13607c0a4b98b7d77ca29572495f4edd58f292ca28417808c8e60c3dd8b1a9d185cc +805784e9a1ec1f290197637cb2f1ef65d1b114bc0e99a9d3070be7199f3d30db85cb3b65eb8cdd +60b44567730017f4f851d4b15a15bf4312f79c05d5ea03b1da801149fef5ad43eded70f470e622 +81fda1df9ef72f180630b3b1509eb5fd9c73e7f02fe23f554e5b74007d6384f6255ea4bdd2a6eb +a0310eda07327884d26c59d53d38f02b11815e8adde0804c621d6cbda143ff30856063abcd0059 +183828625bc486f27c475059855c9dcdb5831ce58532e3e6984b63b690a27373d7d5e613be666e +d52d9f743fd3c373984fbfb8ac8040dcc222ddff4a2e1f6d93f64aabad9a286e3e07679adc8538 +85c8658e37b0efdb68fe51c09ec2b6b69735b84d2aa481793f0e5982e006d985c0c2b8d394b709 +3a210168e58688ab1c07b99b857e90523f90ea3f4f4202f7751dc55dd7cc187f4afacced589cce +0189545f2a26c9d283b14ab2aae2e5393d416e014ad99e27f8814c0b1bccd6f798865e243bdc94 +08dff7e6fca3b515e5edd3af9ac11e4bf0a7b7a736959a4f48d4e519d069263b0af143dbf431ce +df2ab7d4df2cacb62627ea46e77e20233ede4166f291a3a19f62271ea68be133a61b6342782ab5 +8310c4e2b08782800f48d311a7c15d1a6d3f7c6b5bf2d91cf23b931793adab17d9f085a9aecb9d +4df2c4c50c5329503c4a8ad83fdf446b306c56a2ca97bd50f359280415f8cd4bf01817da0f01a0 +e48bfa4c7a1e85f052a03369b0ad7eb0122585bd4445dd53912b2d1ee8cbd9b7e2716166790c32 +f4e068134102aeedc6ade4d5db39ac29f424b0d5e327d081990adcfad18f6ab64c7f199cddebf0 +490b10847d158b2151a8e79b707af40ba4a38c8c02fc49dcffa7e2813ad7741ab7bb29c93e146d +81be2a2494791582618f49aa9f904f8faa1ec63adb809cd473ea6743509617fdbf260d4024b410 +1d962f66f361d7257804e354e050004155cf050ffc6adfc28c4f742a3231c5fa37c7c826985c50 +99698eda86f0a2baefff000a4e74597e5bda20f8c36f22ddd94ff45e16ac91fd250eb9d8d0971b +c4bfda5b6c0c14d850822031fcc8f231023a579c09493ff2ac9f72e81b4c0eaa64fc3401b7e128 +b2642aa708ef990878091956a777d908fa0c4e1f2da3363e098a28b6f141ef68258e2f6ae3ac08 +9b3b89de4379a0da96cfaa81a7fdfae2fe9dbdaa3b21435793e464baaa346a722c385e71398949 +d32b76c6cdef0b05c752130d43ebc45a1738f06281097150423f885470928cc1153cb01b2295df +e17cb19d3b4d576209425a806b98d83e80158b67120d6ea0a65d94582e248d3dd8ad826bbbc00b +9b9dd0f4d606a65b1a0ba8eea83cff745e222e6f4645055a9cbdf1d99160e0e4854d989f07cc3f +393f72af97c3575d6f1d8d4312848bc72bef988cc436c23f4c6a2a5569a79d04a0667611a3b0f2 +e2b8ca0a6ce0536b0e459e21dd3d74d8a94be16cdb3d65a6534be608688944c0d68c62e3989910 +4ad6e587ec0f3fc097ce54d3e25c9f081dec3be6c71b714b9f92c15517dfd763b9bf0496d601fe +4690a9a815e803221e007a7ca74743f679daf6a626af5a6525b394731febbe7cadb5de910c2163 +e640acb041736fe4e508dcb5c7d87d38e12ff80fc903c26c480b567f057b193f5d4ba352697b94 +81d780d94b815a7d96ba6dcc34e39f43caf780d101bd73488a88d213063432f8e6bea7e7248d85 +ab48b576ee702af9364c81f7d5c74ddad8315ed4345444e0cd92ea23b4394073178807892d2925 +a6a43a75c68881385a78d9d5582d3f3f8bae87ae177ea46dcba9af17217906204f751a358cd8a4 +d8c95ca76651271963b6d2ffdf282b653c85ede30f9c8bb5f113528e221aa1a6507364f74ecae9 +22bc09e6c3f42166b2118a819638a85560c6e1eb7bf2893357a81571201a2bc31a91ab89489fa2 +050d353039a7085aacb83609825ae7392a84b6325bbf17af46a9a3eb63ad14bb387987f6d756ff +c59a2a08937538a0efa89ca08f9785952d2e8bbba0e6023ebb8fceb5abcc3c0c3c1b980255861d +991fbf27e70686fa238e92ca9bffbad61bfd48c5b8e8aa8fde96fd9acb2d946b69a51137c90595 +5ceafeafea0acb8a0c561808ec2e0d4fbacdb7ef9f9abfafee72f759e8208559c232ffe5bf6cd3 +e7cbe88ea579e795455f7c25f434345506d98be333bffa7f7cf1f83bc00e38d2503ceb483519e3 +b1e5cb7751c00bd2a13c385e2f88ee5df92086b57fadc621a1a539ff102d4fc4c075ab1d800eb1 +efc6fabdc04f81065f0ef3aa5b947d904410962f6533c1aa49b86178ad45e35d44b866c938876f +2bdf1dfb28d793bb5d049df0cc3e21b8ae8f9b75ac06f00759d48bf39c61125641612bc6032f5b +6046d85ae9c7102b99b93f7674e80f96a1d74c1368d2fdfca6e8535a9d80fbaae20d681e1918b4 +af227270f40ee97fbaad4a4216908272cd9767fce64130744d0ac38825384bd51bfe19a150b55d +d390020ae35929d4026d4d8798cf2fcabc5ffeb88f20129ff22baa1bf3fc412b4042345545075f +c332a91a87c3ccaaba8c201ac784fab723c1371991d5a4c5d865727dc7170d876a83dd01b10c29 +4e651e39a2d3a7c7de70817b0508f612856ddff87bf404f4c750437a96608c130bbdb8dc140510 +61957ec0976ac3b58e117b85cc17f7be4dd59a34a81ecfea08529704d8cb73176ea03c1d873f0a +7acb41d533c8d68dbb3d9b879448ed2e52a98957c41682fc30a002015af634229c4d5d6b900586 +a0a0b06b1fedaf188610cc3bfd97eb4261460ce68154b3a5a19a218bbf3035fbc6477889c2be39 +7cad1e44c4fbbf3260bf85eaea01ca97d02b4c366a44fd08be6f197df8243ca699c2b75b54756a +b361115fcc5828dec91dff364db969960569e59a1c3fd2b76b4f88fd9309f8c9962d02ac778dca +69f07609b0b9961ffac6d942943cbc2e3682879841df4f1d05350302cefc029100f934704dcec7 +e3f153871533258ad2b5b5ad89d153f53be6390ca66461b84fb405b4722d3e3a205ce3bf339018 +b339f240c4956edd92e16ed988b46813e5e39533804efbc6e426304523b82073d9f296b3f8283f +12dbb66157440222a888acde1eef4dc861f9e8d0e20c5876f74e99c1dbb0b5fe81eae326545e84 +285864f90353f9c1c21c166deafc24ffcbe1a19522ac8cbabf3bb73c36bda0238a19da380a6ba2 +63cb28f99d3643ca4f4b21ebdb9e19f1d53947c8f43a22320f8eb83f152580d83dd6cf230e30b7 +4d2f0bf26d6b621865797b989714ddc45136974dccdc37baa1a52d021544aab74521e9e2ef0607 +d951b1e4cbe230a9a1095df65774af0c2a0c0f1770dad9e054dc06c81d30801a927e525afce8cb +8d753bfb1fc2b5d967b9e25c7cfa39bf6fb86dd482da0d871d5c8725cac7b44dcd5c3059f39a4c +b216177100c0f99572ddadf8d7d53733db55286717ce9ee2a12357b901b675c566909a0b4c71bc +bc7bbb7d36aa72a7018895b617ed93853602a5fd52daa01da6d6e3b713b7fd9d6f75e05f218572 +11d6ed29349ff0e628c26df07d1c079e556ffc60585ec8e91f38c9b4ef0a708559b4cc4155ff42 +c48c18731bd231b2dcb32399943e3fd64fb66d3011cc85e9fa20e28c741303d5b1f3b7178753cf +97364716e6bea337a89ee47eaf9b93f938d9d10d82d4c0e3927af3c34927fdd5fab439d303d997 +637e2d21725bca942bad146e4df93b37e5a38e4da57cb1d6e6e278b17a82644c800e6104eb2518 +c03ea9b8e3f9b7611066ef9afc58b893e3fadc53835440a47d93c7052ca4b617c1d958c693ceb6 +0f9a2814cfc1b4a24b32783b389678bdd14fc3ea2124f124afc5ed977097af7a1d3a4d825c55f0 +842b2ac566e43458495b6f51c011d7e78818f6d934203a74a961618625a380923926544b7c13c1 +4b2180f3576cdcd4f2140bc90b2ec84f1aca97f147781866d711448c1dc8df1fcc1a1769ece2ed +5cea053795c0cbfb7204dc61065c87be00d9c0527d5df881708fdbfb7fbe01a933775548eefb73 +6d014a560cdb4313678581a3b1aa73d546869301f309f189c2df29f3d4bebb3b5a72046354c2ed +d273c9794c3659bc72fe7fb7442e75b04fc197a49efc2ea39ae0b5d1dce9bb509b5085445a66b6 +a59979c26412f1d6f5ebae84175ff08a90c9be4406797389c439693e2fd2f2c26d2e9dea41c505 +c32477ca50fc01dcd53d388e27cdfd4d73c2ac2fa03b9eb934479f0419ef50036b682c9ad8bfda +a3adf80a03de9f29b7021cf0441deec0f002fab57da0f5be89f5da4597a2523a2e7886589002e5 +cbbde0356b391148b5502eaa3b3689871a1cebe9b8214992f5349ede7d799a57b85d1e2ae706f8 +567efcf24081165f53810a2dc9568a263f69c40eb55cf31782da10dd970dc79f6593f1cdd9ce55 +2de6c7c9fd3ab21169a2e487d5fa959d659afddd071a0d756b5c7debd602cf9ad3e6b3dc3f29d9 +54cad22ff28cf333ac67b8793f01c6f13c4ae2b6732d60633b59c2d9296ec77b6ee630906adf73 +ef86033317cf59b6ffcf42fd7a68133f0673a5c91b8e0ec7b88872e9ac8d0aa904af6000799d57 +e1dccb74496e345dfea868ad7498f3ec7b85558a1fefb6f03c7d5365cef02b044b220c162c530b +8cc899887f8846084c7b4fa1dcc631ba3a3773fa22443097b47852b119b42df9d58ef3d55ca10f +38b04f378562b3bf03140e25544a77e66cb9589e6f9dc0a11b6c2a35fac32ce9bac552dc54270b +fc025baf639b8aa5e3dfda4908b869969bd633a9b171ed6a04a2ee6e3744d7a348cdf09dacda62 +5269181d8d0fb84b468d2eea50fdf69324bf64ab1db3ad0fb2fb9583937f0be8b41f14a751d711 +5adf2d329db753cd14534cc282f2b4a4bf497a4c7b69cd25a1fb33c5e91796b885176284c77a11 +9b50b077001b566328fd23170e4ae611e3426fc26443587e3982cea97c79fd63a4dfa95efee48d +d81f90d67e40f283c27947d0f0823d2d2d26210e5243d2029b2ff059336b57b9b1334b0b615bf3 +57c6a732c8d5165f2c9a6f310e48a2e4d7dc965887b45c14c9b744bd98de8f33071f0dea693ad6 +4a55d03b2891660d09775030eb5c0e6e868ab1b90028c373d3448851e225dc9eb442aebb5912ab +0f3b1107a7d96f9d23983d8a6cfedd589e59fd355ea0aec2a448fd9d254d39331cd80dbfd653ff +fdb2702cb848144c33c28d207adc51097c28f754521e715be8b7b14c4f86aff04fddb0bd52247d +814192a884e5af3a8f675cdf0c9dcdee0668773c3a3673af8f51a71bdfea19d0bde8a2a2345316 +4edbf01c932db93128b686cc6fd9a1dbd506b6c52664c5aaa01901c5ac0a8275e0ff4ec66ae332 +6dbc1b3b60c4488b6af86fbf30ddc1e3e3a8d63d7682c222e74605c95af3327e694f3b3c160008 +3f9cc241f97c5013c9a0588c715eb4dd00944e824e43a7a1c1fd144d1098628139034731eb1cbf +633184cb857a17855f277bbc18751799ae28f48130cbc1ed559ab1a5abfa2576d59fa35772557a +6fb3d15b4ade8bd38b763be0f8d8166613cc37dbcec1d65aa06c7b9f171028fddf8b55093a2fa5 +09ce882054682fcfe32d510012ef5ddcf39f0ae5ade25af279829f8d47b65b89a40a71ab70c2cd +961aefb7004d3910c190bf32137e4d9733b971db4616ba789c741efc39c26540d3228884b4bb49 +0bea1ab957682d250cf95e1d41c116623925e7778755c11c654d42dfff1db2a49c5b57c3df98d9 +42373519cb02f367598997028e11c513a99b4abf115c61e2daedebb709068ce06692303cd0be48 +228918a12dc126af0e4f81766acf9a84526add4d2fa145cb5452c09547de307b352e141233cd8a +ea9f965070a2e6321a795f896c0cebc01ce3a15a81f14c14d74aaca4ab090125fecc310410da79 +bab5dbe07e9eeb2189788f491c457a82abf7f79f427932c8ad38466431e28659056c3d5a4be02c +8d0d054bcd4f9170aa7b7b0d2e9a4564af66eb7372fe981bf441b68cf8247d11fcf0bfeb6453f9 +d2210dc880b9b9af2bca1dad558c1be18eab687f453d4aac976fb9466a6602ffca1dc852a56c50 +556d777258a7bc26489789fb4fbe2f70034eef6174244ebdcf514c5ff72391d097750ac8a9fcfe +6c94af17b3a5f2667ca2a02e7f32424ad5a7d902751f7c41c9a7fa5fae2d8d7396f8e586be8ba8 +f960ff3fbcbe783e5ece80def85cdf792e33459d996a28775979b335a8448523c0a187dd8f3c08 +31231231f2a158a7b7642593932c9e0dce8892ae708b9458993d580656e1614f2d076f256a680c +cd3d370e37cde46dbaf38f096b032170759ec0619cfa0389b17568017f8d2811727965c75eacb1 +bf1b179b69567b0ed8bdce2ce95e98ec7a9a573a79ccd5b4990e46d517324a06ba418b34110fee +abfb71c1e43cfc1650179fc1e5f65fef017498b83a1eb5fb5e340a1b19ef7f10fd0422f6e7f640 +db9a46762fbbaf33e2eed637ee9d93cad018e33c4fc821212bb0f38f2251827a46f38c38c128df +a891f0b8ed14054c651b735837a0f862967c3d39912e288dc3939816e015e8be9ba798f68c5e89 +a4184aa18392352bdb53119115b1a100e0941b80fb8017dd198c7d0629e72f27da2d8ebb59de72 +ee46daa03e57b12513988f2023484d9f0480126a378ac134a75c2bfdc4df5b8d8fa77ba3fe2da9 +a01ec9427a00487b3a81b50a118c528b77c22b57265e5f915a063fa3e8bac0ff7cbea9ddb19ab8 +390775df6049cb85921457cfae8a868bc6584a60223fa19a363f98876ea3d1e8a69337a58ffd75 +c9b296ea5485e343c04c72a53945e21fb479495d4686302f39188c01b036253ba0a18fc87aae33 +5fa522a435094c3608cf6e2e37fdf5fa3f59048fdbdbe6c58dabafe941a7af5aa4ee7b44d64df3 +74780e9e2d691b4639e2aaef8c52de72b51f2bb6506e5e98f8ae236d183c9cf250dee46d102d96 +4aefb7b82ef32d19a03aa88ef2ab975f04a6cb5208f570af43f73d804843404eb835e5efe3e348 +87a37816ed3cb02b13e91be5bda75a112d1d27e92e462ec516100ab117b2882993af81dfbb58eb +b2f5bc5d5eb98e48f73e88144889677b31d3c660839854a5c611a52bb493b644553e6948d71f94 +f04eea57f7d458d7d78bb278d83065bdd9097f252d39a9ecfb408f2229ad5edaf4afd518c80421 +4dd2458b085d7bd382c7a1e697b5e0c1a47c25e2cf084d63bf186635783b56e6b36e7da64e3f9e +3e7135d74381d957c0c62a6d2c4dda1c21a8e49979dc4946c3bb84680e4386750d4b19c4011759 +d84f90cf31b268f0af056d8566e84939388ff4cda569118ed71a2cd82d28b44e17c84389795a43 +57f70f976891292e15448c74ce895be53e4212079fba23793f3c247827b44f0e39066964e75671 +9c628e6bf4a5f080de8402d5d1c154785564e019f13ca5b45b525b325e29f193557770f88ef82f +8d7704631969d4ad624206665ee1d38d7ef7a7979aa05d39528beb6b7764d88da381eb63f0cca2 +f797dcac11011445da21da35e051c1db3da576e698451a0675944f5749e465faadfa4bfb09eba5 +177ce8cecbe67fa06b74d75278d7f28c236a596a17fea5467785f345197a43ec5f2e1b7e7e2151 +f287ad9181ac39c9771a1713f45b5ddbb402bf674e3eef369647bbc37b3afb327576831021a1a3 +8fc4840cc4716da8026d16fae600d543a3c4dc4d7cd5baf19f1db2a24a0757ef6c3bd8047288c1 +436eda2bd3f9289e6f8b003dead7b0cf49f3aa8a56adcc26ae300976e1c84bfa04d9a1887682d1 +9ef68f52b76fdd0e03f64cc92dbb3ebeafbcc56f8a2a99af870bf7f0153dde4202ca45c530a056 +2dfae59234f4e3287b93bf427fcaa8d236606aa43aff3862063c0bd73dde6bdd3ab4a8c4d6ad9c +24da288dacffd02aca90dfdc72444c9bcbd331755876f649e4b665d69921097c364eb8b0562090 +e10f4d719b2b5afc9312ed39c77e91ece761c6c940133f940a74e5ce3c05d9703556d28a583972 +7c54800ad0e496885aa4052f65813ae552e8dce8d6f73949d0fcf5059e85ca8a1c137d751c6b8a +87a52b198254d01258074227501dbe50de2b82bd2510e5788c46ac997c65bf44dd6928ca9f7a90 +2bb957e277e7380123cdd93cf75dd6c7e5a56f93f114b6e46e0407d3ed544d781ef6fbfc704a03 +5b7b737eb332457977f223d87ba6e3dbe827b4fae9e687f6975731d42a8977cfa67f94e90df98c +a790e89713dcbc07cf27ea8c4584cc933fc342e5c5e740ee920ac3403d4deee426d35e1062a2cc +7d5e23f568b49495c8b4556a8bbb0480ade618566f537b440ce52ff7d2cd89fc806fc388e9d7fc +8bf15882a38ae0e11915ab03c272c8a312094938b9b4282ef3a321e3bd82701a50f8a4380fb325 +33cf4a100b0f96475a0f12fe9ce7e6a41784fe7c83f05869d42b70febe4f60bf749429bfc4721a +cc4e17a961bd7da411471edb7a32f4479f2f079239c9bedcceaf35119982d686545cbb1eb0e9f4 +29e9dbac1b743d28d12c6ffaf8f98a3bbd60518803d997a56bb515602dd3a75f7ace8adf16a6ee +79233fee1d21bf40914f29521638aef27eb796b06b5fe274444e0833ab9a79bddcb334435b815d +0af72964e8cc297d246c6bd6d78e0e8394d8359084963b01416db54a3e3a6d9784320814c4fa99 +aab6c4934b76a26c5e014d3fd202935f2f9399dda83b4d652ab89936578ac2c70daec483f69422 +b25472458417b87f2861ff90aa37e45c07ac19fe09a4c2d37c917941f01682e650fce1785d114a +bd5949ae733bdaaad98a802437f465bb7d2d1e0d4118d6ddc9aee8e08e606d9703c319001725da +095dc1e3a421f0c5315e28765e9571982039d5313efe20de55e7b614184c4a1e4eaf4a89c62331 +c6f2511d706c2c4248c626e6474c450db1838f1d8b264e406d6f85dc63e78936a3f7d84baaf49b +bd3ba15c58fc4734fdeb13d1f1557f29ee53fdbcca5d008c60f868ab756608ce889de583309a30 +3cebdeed91e7c4922526f12558d2e751ec11b3c0bea88065ab0241f9d1e4c6144150f92f917b62 +9295da2e8433a66b4a6e6c8571b684a675af7e77cfa6723f82dc6ac13a8a568ed06a519ea0a272 +5cdc99096ac0ce941b93207ed3f5991241f709ef7f9e8b2acb77100cf293452a1917d32c2f2a79 +8af506a827be2d3f3ce87e3d920a0d8cf8e98a4bd3bd99e12f4374e0f6ad25af32793c5f85b77e +213c7805156d53eba08d9a84ba43abbba8202d43091436113d930a138dfdd15774397f496aea91 +72ff0302555658d3b4f6926fc115bf3de066f2d3bdac9738ecb824713de70361c91374fbac965e +d71ee2aaf3a27a9cc5564dd26a4bf30dad76ec5f955915b1363ddc6bfb706b559fbee46f6b6338 +dd6a07ccdb23b33b0237c8f0a428fcbf6db62fe6906e4a593b4dfaff151b2cd167951a70f53539 +4d84b7ad727bca8403cfd661abf9a72ca0634822f0ebf66ed148fd7b29513eb151c8d76ea7b749 +02d76ef7a88d5c571873bd219a9f7e60b5d6b77bbc1afe19165f88bbcf45cffa773fa9379259c6 +1a930395ce89109c5e3a14066dcf9dc8387209c84273ecac6a6cb033d54cbe4451036b0b5335bc +4b3a2c6f6a4395fdecf9c1b88d3b8840e0a94b0516c5821cdb3b15aa06ef3fe3a2c4729e258156 +fb7ade7c1b8f673dc87a8c87d6229df03d88fdb8d7e122cbdb3c8e9eed4ddd03ef86bf2b6fb140 +187441e7d3ad4bff4bb61e57dacbb792e9f4263770c169933f99eaa94f686169ac46803bf6324b +8498c154ca78482224ad28253607ce99680a755c573b8cc5bd9840b2c8a2518cfeec51a396ec8e +3b6c3f980a3896feedde1a0605ab8d7891736e2c9502b5e2b3de8d76da62afee99a77e1b079bf6 +f92363e8a5b6e605245952f9cb351206b65eddff62ab5730b81d85311489f9b4f4eb4690844b47 +a8a2cc0881180d3b5bb8f6bd73afcef9300002c1680503cab52a1c6c10b038c312a0c5e59c6007 +2d4405937a8776bffb1c07d0a8761bb4aab03dedb634ce0192bb51b07dfc815073beb441be59d2 +a5b29a64e559a0d0f33022188ca9e9ee43530036e15604ac28bb7349b1f6b57ffea79982da0f9f +ef3b0447ce31ab3bbfc0c03b7a421a836b8bf793fccbe963059de30c4ec0ea18bd7f3d00ceadc9 +e611bf4c1376a64087008b5941734d404646b9f698dcae3c906f2208158f670d807d7ac8509a38 +a95d16679d11cc23115fd80301e84f4b200ee160fe96e54346be77e24998e679a0e1c0579135af +c37c68c5163028b2c026f5d22035539795135e468f192814b49c4967e4873799ae4fccd31cdf23 +f754ec82068c2448ee44a452a0b4bf877a7c420006ce3e9e032d830e2bcb814e447b01473cf593 +3f6d82cb74e6d1d5d6211484ff1ab6add9a46397b42fe3c7b7cf6357ccabffeda61f6aeab2ef6a +84ba261f837fc8bffdbfb3ca6901c2b4b29834e08d4d45a2dd4e8a51c229d30915ec1bcb1c5486 +a57eb7490c70c0dbfa01fd1c53c99832c7e26e50265f54b8197a0316d30a5646f6e4a71d475e03 +460d81e8b658327db8a1b1437334d3a907947b2c899e4a53ffe9e8227b2d5420ad4109b2b136ad +0292b5ccaa83556e85b2018e629113c3e0d4d484bff995a484643f661975737cb0eb80f2b1ab31 +9b1ec6251c49ebe855e2d6ea533398f9b3148316210b46ed555ef68a949b73f31a4457f4558865 +4bf21d554b39eb23850e3a10c2ae0e7369c685641f6d8f43936c9250768af2829cc655bc027fa6 +40aa14b18a70aa3a7552491145e6a7df0d81e0b7e1f3587ff086712368c48351d661add7e96a25 +b33f9f1f777cbd3ad42b6786c8b0ae23f5e61421a0b660b7381809f13664aea3067bad45ad8451 +c40e07f38f9b6185867caa2a9ddc8c37647c6b6a77d281185db7a56c39cdab0ace10890f5c0d78 +07fc420a2d0948c3880dd22e0b6aada2580f9a635d12a305a89415a24ee12b834a6d01e9f1a756 +b492f4aab442368cc18c700e1016014b12e2ca27f141ac3084ce37b42a4c41d41a3ae37aa2e400 +28aa8f3dd3f0b80df124f33e60d97ee975dc98338c7d2290544d8da601fdef3da00db855237a17 +4cd3dda0a9a404c749c849f5592390320053c4de8004c650117289b8f0c44cbef09e2fdae18a59 +3f2c7ae524cee7e3560e3116bba293c02937b0e95fe22687ee328feb89d6738ebd1604031d248c +eb6a11f9ffa7b2625d3ad21050df6f25d1bb4f3d5d269bbaf8d56d25ac40905ba0adab22ccfc77 +33dcdf3680fa85a94f65445951e960e817427a9126a9c767f263f324dfd34841554bedd86841df +ba7e43986958d688da35786f0497a6d04fcbb6eef0d1e8d9575af1fdbeece6cfe66c574e615834 +6774b185fb5c4e3bc71b45b5202f79210579b4bc35b21f60843b811df3515cc0d81d665e7eb018 +619a243e72c29636b8d9366166797292ac7023cccacc9401999bedc5920fe7f51458d13eb772d6 +500f471416f079bee7c9bfc25a0858b10dc043cc9a219ab9a71ad62d68ba22432e1dfa7d7930f8 +73a9af0ceca0cc4a4667d1102a8a685adfdbc5a1b4cea2e9a94d1766ccc8f9e74213876f3d7bfb +e0950006b11f656a37641f9cfbd0b23c19dcac6d9c2b5cf0e38d8a58ed42c7d37d88bb9a96ad28 +64fcbc6033bd464ca519c9e8b24e1033ae844ce27b866f2fe38b09d4a262867f4782f15a5a0e76 +2f76e1dae8062ba108e759c6d8f00ec4da4b4bff0e3b22b22fdc733b708e0c8ed1acc316d8677e +488adff3fd5d912fbee47a55e583d814d6a77044767539c2a1b964f5a2522b482186ec9f957684 +f090862afac4707e5b48722a97df0d272c713ba3b611aacdbaa0ed665ce873979c55cd6a9dac4f +b8aeca0b0cb0f6fe85e7c97088ac6f4ed2bce9a0d16527d001507702138709bcabdb694a61da95 +692e67999aa628b73837f53ca6892788ac702cd6eb52f8c79b92c866213a6126049797a48454b5 +3318cfdebcbe1c50fd139abe01c30feec51f0cb7e9d9a863250021dfe7c27786e88b6baedd48d9 +49dc481684e3faf67ca81ee76cfbd63b547d7a43f3ec9ad09b1bc9e31462a43245f40ee9f0cbe0 +70805a1abf940f2ed8300be39fecb86120ec4b0e680d890903458cb93153fd2ed736b87307aec4 +beb473ec4935bfb0b8c01ad84dac20d13b997a884930e3c6abdaabb083f6f66c5b936ec4624fd1 +d1ca39011688ef21faff4f69b93f7fe73a7ae036c3d15f6f5da6838536f3fe0f7543f3f9460b19 +7a7892d6424ba822cdcb26e2b3dd7ca75ad2710b9ab49c6d5810f68d1a0426987f1e462034ddb4 +aea10e8e04be42ea10cd8552670f9251994dd8e844245e97a5902b65d08fd41013de3f3cf0cd6d +fec75852ec385747f789cb0d846d0d8f4f0c823fe8a6202101f2cdb1811cac9e719f54bffb04d6 +46ebf824a17db7d78137f4e962f3fe6a19fec02697503d1f5c1a467799d2c347e8f85395eb9bd6 +3f4e6a52fa05d4a3e25700b4de22c41d97c719cd5a7576a3dc797c96bf68c5ce7922a8fc928d6a +a344fb67d4115cafa38e82f75f621c9153f7f2f9e5ab5817e000d5eba9a789eae7831e1a75a382 +c6181a130b0ae6aac0e5cec25c567f1c3eb72caeba74fcd4ab5da3b73412e76399a56533ca6044 +f2c2df96153ea47213cdc057e3cdfc8923b80543e5c5b226628dc0bece91e30fafeb8d4a860c90 +021c1a996d178c6c85e9be45f28edd3fba26d6ef2e486c477670a17494c453ff09074c083aa8b7 +4de9493f207b555c56e72fdb8867ccac10f060675e0393d1e5c6b2f44f08458dfa45cd4bdddc3c +e3aeb5c7afaa73086cc7399ec8874f529e4af63bcb4df147edda137e552ca22f45ff9fa03ac41f +724b870d8656fd9bb93690eeb0e7df81e6957e602a0159ebf6692fadbc46c9ded8ec229d9bb5cb +617518127e557671723dd8768b1ee594ec29c4a096ec4fe6192ec5038eea033d0894c70de3920d +1deac65b4fb001ed0c851779ed7154ef5a848257c973c7f02aa72c1067dd09b13575a33d313796 +9d683ebd38e8b61cf62fd3f4e3ce98c266e088aef330d9acb450ebbf96bf63600e83f13152eb03 +53801d2fca3e18eef4f1fd438afe54026028dfd703cfce0bca12b0d2fdba236918d27517393dc2 +ebd744f78e3dec0ce74805418f1a93252abef1ad71acc1809460a8dc42380086a593b8167212ad +7ee18288d203244dd3a2025b8449c47021ff7620fb89d3ed41f8547bd5328e7ac7cf5f0ebfc314 +bd4afa8323d0cba57e98058af91ff4002edb452461e95eb2bff94e464bed868dc3014915a91070 +4c5740a1ffe4eb3caffe27c199f573a2b8310ef7dca5dd1baaee4eb5add21887d9b65246d93e36 +a72766e664b83245aea45114ca3c11b50d51d218eddd1fdade2aeabd385ce371734446745552c5 +322e029c892e227dcb3fd626cccdbb1731d4cd23057686833fc20906fb3640b2481fdadf09ee20 +e487556f5dafb429d648d74f57c8f6aa04f2ef121df2fd850667df1668f97d7216472070848408 +61f2627f31e8858bfe5851c35fdbb4a81e41645ceac4873fd2447ff94ebe8b83636578eaed2be0 +3bd8f283fb6832c10a917d9ad056ac7ce149186b5664cc52cab3939ef44f35b43bf33cf06416fd +ff848179580a7c39732e7eeffce44f7b24cfb288534f47163a273f2a5d9f30ae802af8ca1dd90b +b7149e3c3a3082fa471257bff78019380b1ac8238b735666c2df052ea0b436789bee7ad5dd407e +9c20774d5ff3e0af9187e24065dc9a119cb8842c5f37421d005f27d787a611a8f52ce76aa34ee0 +ac7236905c28d6d9841415387be7eca5c533ef618397c89f3df4cb86ea2e38ec497e380efd1274 +3f6bc297629e86fd88cc269eaadd0dd63984e253caa270ac816df54077f7981cf1f1d5a77aa46b +7538794d9d414db7f7eda8831fd123f843b4b6a963648b2f31696932926626e040cda63af93f25 +d4a6e304d3b05c171c7186f5ba239f9e2156a0e9d30fd016aafff969eecff0111120c07706f095 +05359f63199c3e19e8142bc84b7efccaa61b1f13828755bf18cab9eb638f17c3f20342f60c1e18 +50332c16c8d21c2640dbb93652c7cc6469486584f2e036160ed2334c5cbf3b343c9c20fb460fe0 +492ac14315bbe89139796ad76f219f2f78aa252aba529db54863fba815857b4369dbf510ecf4db +2255912af8fb0a1d7c1a2d176fbfbbff0182b969c8b02780868d2267b49d78b449702c46d12673 +19d79a101cabf6afef50a912b2fcb8482539e49e919f3b1270599e8ccd1becd4fc4b973bf1d412 +764dcc3013c707039fc5db85b8c4759f7fb9be1fd2fc5d972e117af55ef2586b4441a440f35084 +0210186a5ed1e4019761f7b460cebf431c6d564322cc22362d93433f843166133fdcd079db0f03 +591ae83885345c227b918e762e71db2b5e86653898d20f80c18c7f811797cd7f4a0b3c6c5d3289 +c5fe808a7068f3d19e8c4ae145fb822d59d2344bd47dd56fc4f2c7237119a44710846bae73c9d1 +3db5f08114ae8fbf644831e81dec6206d933c007f15c0c0ca65091a451a3f4268529df3d047f8c +04bfbd7d7bc02b76fd3aaf76991f6f0b2b4b6a4fb87dfb7281a260920f79aa6c0cd16bd15de94e +8681fc684280948d6279ca994a078a261860c868d0a029c70f80903265db27ed97912c5a80e648 +aa078a321e5cb431c0bb2a9d21e9929a4ff4b1f9dbe980bf7ae713f76ba86fd6dfc798d7488b11 +5e7c30d1068322010037fc6c9bcc6182df29c66b2477dc0197f1ec7f48a7473b80a284d917378e +89b79c9aef1e975fc3edc7c60ba3afdace73264dd561ad9478a3cbbf9858978620c79c09d660b1 +048d3ff8044dc2bf02d1282e396f5c6298c30f3116285c16d3262630330c4fea515f629d480bcb +aaaeba211aabcb932c2c8c9cbb2d48092f69f11c221d9ca068c529213b4303e7db0af9ea0ecefa +45fdcfa5478312aeee4e7ab6d393b8b578cc181bde18d1386eee5ecb8aa46d9c62eb8dac91fc33 +75f93917170a43c2fb48ec5bf00d15c5fcf1d55710b6801ba04b406ad183dff76e19ee9526e6c3 +8d5c2b084e8f36b7a9b168ff8d2027c20b0d79d120a0785a35f93ac1f2a7319c65e68a4abbfd0e +e8a3e9b14b3c5f230482032323b501a82424aa3256af1f7b5de145b16848b9bdb70bccc4f83c23 +1aae58de03afd4b7d0c7cadadff17b2dbe285300606de214cef1c6241d2f2969f0b9741dc34d8d +fdd30331d8fb2b12d3bffc004ad1e9bccaf126e96a5e340a89e8a91a9eb19ea03d4886f768ce06 +3c702f25f5ccc8f473f3a4b85fad95d7a92d2e3318f635bbd79a8a4eee3f1208e9e69a7d63de37 +3262d38eb8ac6121f3df1bba843a961b5d3e7f8735ff641ea99fefd347df900b678758080fe745 +538c9d5ad428839bb6840d084a49b2cc475319a99b33a178a4b4f24e56a9c2519e8c010f81de92 +5cb895a446c87b63ecea312c8b230151adf735c6b7b158a1d655e44e2f5c77c2b461b97e494dd0 +98fdea07d1940427d2a3f2585abf264bc4edefada3790a85e0889e90e45c38482909b802b8c4c6 +78e1129af4382e479e49a9e406e7120b85955936ffd8a40f41e91238e4d11c7e660b902897e974 +6f9948ec29ed59c91597ab7d2137149edc67bc432e470a404bb9e6b1f98bcfdf1d49389f828e3f +35c203a3aededd8d4d3e6617c959d5c401fae6756223f2ff5b52134cc2804d20bb08498b8d00b0 +660fd27b4511a9a099a8487b7622184d781b47453611484da293aaa6b7e8124785991f755bbf68 +9c4cd86e573f40e4f9cce9ffb6c5a84a419e32002acd3392e677cdc2883bddc975bf0550a41e7d +5623d745f389fab05918c7cdfc9f49327eb8c4a59fad6ba52b0f075fe0a2851fde85910cf5cc95 +90b7d7bea3a1a8a251e37f6a20205b946b01da9adba76d14506850b6e634ddfdb3c853739461d4 +28ac3349438634bc2c4577fa2a337990eb7bec05a6d9b90cfe70856680d5e822480f77f580c3a1 +e82b87dc2c2c554235da6133e79ad708786a8fbfc34d813f4e4330afbd5e03f7ef0875a7a07aa2 +01c7e71ab6a818afb9399ebfd88e1d438e5d4634172f2f7000c10112692c4e9f0273a8db1c559b +2e8d66fe93521514d2896b7321372d441e1e7c63edc272b30b6735ae53ac1b08c352aced8e8e56 +9d3d3dc4e7b79c9a58310d03ac8d2760522cce13bb95bddbb22a13b13cfb78bbbf43cbee6adfb3 +33ea66621a5616442fb9eb4d5855eafb1573075650d05b9b36e80e8ee6e5a4115aa1234ef0898c +52ba94d85072c4da759dc246ae56ea526be02430ffa280c4052cf73d6eb4b62c3282d829291772 +63931d89f27b1e1dc8de5ecabd3e348ae919c983913e6ea48b21bc0827807fba233c132aaf170f +20b33ba72160f184a3df9b130e3bd09301e12d0b22dbf56ce6fc87ce156de4377498cad735a6f5 +e57840ebd7706ee44c6b44a9bc0120cd4f50720ba4b4f4002e11db98d7db9a31a5c300d81546ef +990481b7f9c278f0853c05b0fe5ef2586b4441a4218a64a07ed2bd6c547de31987297af9e99f16 +ecc361ec2650cbfc0ca9b55cb8f5b11408d58fe29e46eb31a0537fe757800abd0f34398cef4e77 +362a140d3ef6b00e83e143656b00cd218db158d44f5df1c5976fe708f15e32642e629645f1dc1a +3d22742637a809f2197ff9a60d1f58e74d0937f8d8cbf0b0de616809d91cfe77675bc90ebd5cd4 +4fc09ca0625e8cd6144b620bea2183744d33c42e535b4879ee3fc032610eab99688a6e80081516 +4ce529eab33e85cb2a84ca3cea8074039a5b0b9e2065fd72389d02c6608660adcdd7ddeb2c4feb +5a32e3276025f5c3ff65bf4b01473897164745b05a4c1e1dbaeeb2506bd306e266e613b16b93ac +0f869aecf140786a639f9856d2f47dfd448564cd6067590cd7c75bbe8b58eef67d2ea38c965e69 +c090c242e29f12dee8b4335b8ecf7ef341de3a30892c09ef87913cf5147ae0cc255e05eb602420 +ae22e8c2754976dc6ce94fc477f330dba72728449d459b2ecfecec62d0be91a9397b312a7d46b0 +cb7cfa2fc664397dbf9a50bcfbc63ed6fd6a7941d36a37a815cc9290c79fe24f19d1a5c1cf0671 +a73c6c22abb9525618c9a3a9909f1850a9f470d2fde679c254bf516641ef96f46574633a94bd7e +c9890023100e55c6a51f1124b3dec6a7fd42f63499a1066d8d27a25819a7ca1a979e04a8f1fa55 +d4aaa3eb05d8205cf93fe5d372778e809ba05dfb1e0a1e60ab6d1140d4c0bd8e2db88e7d68eca8 +e3f3e2248cb6fcd2919c12d60bf4549cad356000877d0241f7f19487e8c2b17a522bac5a91f864 +630e2ea4d2f5b7bef4d091064ed22d2f597d18c9f04e2e2e1be2ed884cafd31231da6133e79ad7 +08400c02fa56a3f3f5eb56a713fe068d4cf564e7e20dec5cd4696759cc35d34fc0efce5bdda7c0 +2eaad828006036386ca3432c6c005aee97bc884afccd3b642c0a17f418e3f2e436dd02237531cf +86ae391102b2023ae0913471b6797a8e71031412ac87b96a7173d5a56332ddc5811f8672b37795 +5fb5e39f90d787ed2b0cdb81866f79165c8353c9cc1f0924b5f4ab6bd5197ea6273aed03c838f6 +a78bd9c40eb8b71b436c0803bd569c3ae55ffac24dac9de63cf6f851c5e2199c11d9da77009e55 +3109e47298f63c44644470d768725db00743efd4b012665664b3f663da65c6329da22e64b466f5 +6bb45e7afa47e1d2e8680a49fb4948a8c424c175ebb3f221a458d75f3f84929ba4716c503c81e9 +baf81999337a0e74d9c386e2b46aa7ecaa0689e5b00465cc49f96f7003d7d0bd1463372602a5a1 +8bdf3e924493566ee07b37df97814292f85845f2cdae528cf8e6d40b1b4c7ab83c276f3d9712e7 +cd00e96f8139dee7bc8d17b48a57a39210ae727a8563319091fb1944279c1e670a24093b0d03ac +8d4a479d2eead32e98af57251eaef7a6a110493471c97c04cb50fed0461db5d0f89df4f461b67b +f479fac30463af0eb0a5c40b3ada200bda798ca929d04caca634d46b3fcfc12980c2ece9439363 +ee588b4cfbfe1adf3520a28209e59572148f5dd6e4891d9bdfb2525b5d4e16fb79c13a65e1d687 +2b1e2ef1c761d1ba66ea1ca508c2a3bb27a32515269ed4377d1ee7ddc8c94c4078bc4b352211ce +a13495326faf27554edfc3a8854cdd4ed995df19dedd2beaa478a9409a77f75fc1f4d57ac4352a +faface9ad01b0b621cd8572b3c65878ea89e3f43269e8a0c3854c1454418f7b0b385044a15ccc1 +823961ceac3f500a09579e605f92f0bbc9f2d8168c50cd9e61b84e0fc17ad73c36227e22447b9e +f1296f2fe302f23663deef0fc1eb759acc9dd1909f369fca5564adb3f0b2ed33428c75fcea4caa +c27e7e2ebcb3f50fe19f3a3904eb94baf40c40bfdf8a787e622563a2cbf6c32de0eb309ebbc440 +2e31bcdce135d63f42f314928df9816b69e19f732539bd00b5950b5aa8fc6cb3bec0f14e60c3b9 +7423680fd8158fae9b076f3d25aa3fd81bb5132f9b1f150acaf2cbc0744e0d4f42b186af0591f1 +6f51eca0ab48aa2525dbbf629b4754f23549f1f9a88d2ec444c26f28ad54c9c388ed9c00c9b2f5 +a19a32fb8880c47ebb8016ceb36d0cc17489b3e4e552984ff17ec5388f14541759ec95282eaeb4 +a2d6e3192cebedc3526f99363024cdebb040b3da29dc93b58f6090f2f5bac6c67c8ea488b3f0ef +f0d6228e20a5b865bb2d1870e9e76c9663fb32e899c3f1f03507e174af0e77433fb35748aba9dc +78f419fab7c9e508f73824460a7fe6ee23fca964a2839043a19ecb24b15b6406769b703a3b831b +d6c4c82ee4af7aad7e5295f85353c7d3476119fc664568089a3350ca986913f16002e2889a7f09 +1e02e3a400cf8b83ece85176f96fc41efa93d5d658f45ac91e6e566184cc0376ebc6c8491f09c5 +e6b12eeea7e4e445a8821e03a381c468bdef4f7074b631a3146675b53f7539b211e0b3b1885eb1 +73883c4cacf1223ef4f6dcaed8054c8738fc3dd1abf1566be3b4d8e99bc538504742ab18cb9e5c +f97c9e5284aba6f591e2d87f00952d0388baee7c3ba1182073481edc7b9c0e761a85a1643941fc +9a0bb39252d59c5b0abad385ca9296546fdb9faf2bd384db91bd96c59da8250df2b5d125d0b254 +6bc7e2c50ed4dc568d8f1b65b6f327842bed0e7e910e44ddfa302f5b30ec127aef3f02f11dc0ce +8a94bd9a77c33c8e366ad06f6675806d7f4e77282f88376fcc8e1772e41ee7f1e9cde177b826f9 +069ed5c3d3d0b4bd54bd60e01d4c768cb47b27624853fd9e3760387debdb46d9dfb6da0f7afc1e +1ca352f141b9039c6d2f3ee7ba88dbd5f1a0db0753962fad600b9245a3a6bdb0bc0dae1e65c3b2 +7bf95a52353bff9d34d860a5d9de8ea0547b9100c160b7b631ef5d8e2bdae77d7a915788bef9eb +9111d4d7f60f0329462d2ea14a7da9289922a575fd88a52c1aa98cf1f4f34b705464336e3ab7cb +09d81e427e3bf24e87bf7147dfc0c52e6c67205181ad234ca1ff0dedd467d4a13bf9afdc6167bb +6f33a47f1ab1477ab32752c256286b25438259a143a6d95b2c6e5dd38dca03fd6c803782dd7dab +0c7c0647f48a84a163cfec2ce098a2ee99d2b0d46e0afc85091c8402b36b41cc096e733d3e7252 +c2a4b58f54369f049d0924a6903f8ad3e1bbea39e25bfa05815f79eef03e1748b2d581fe117ab2 +e3c7a8f520c194b3c5e98fdea58f19b4e79d677ad54a7fa485a7a9712896618b0c33694a3c3467 +a245c52cd39fe9730e7c2634d298c1638a0c7109692cf258f8a6d0bdf578cd28f72e48176a9599 +93014e050a7d060918fde0a223eab19396cca9c895a0d5bb26eeeb9190df3f07e2d6243d7a8454 +04be35983125b133d37bf15f8c7cffc8ecf514969e23fabd262facea11fee56fa69682307719c2 +16b7881b733d24403eb703c288cbf898749f3e81fe24451e063b3c02cb2ef8dcfc36179a525662 +d47e0e6325b93d885973935c91cc6c52749a939d32fa1f5a74b689d06d161914dffba8335c544a +d8c4e83522f403157b7aa2ab15593a7583c684e472df545d24a4894038195781caa9f0f49a66fc +62fbd0a5155743b7c17c9e6ffae86076684c20a77b1e74abf75aab577b93dcde8c3ff60aed1a87 +ec006d276725215681e444c21acbb6318f41d175894c303738994c15099f4021950a8b389dc2ab +91b74ed57317f7faf10f00bda6be25467f05676ac4192c369ce03bb374617ad5bebcc56ffc07cb +5d8e7da3e17b7ab3d9d5dba4b8354c03a64f02be62ef5df14b9b9376e79a803423bc55c5cee397 +2f75faf61dc41d1e8415ca55d3cf7036121cfe771be80b697389c047c601dbb8548f0d58ed841d +25531a41d8664730f34e2b187bbeb8d9c16ae7591ca61fb2ab36ff3b6782c7e97833827a1ff866 +44ae56f463956635883b55891a08fa707e90c392ae4721fdb0b2311f8cf00a4703faad8114b615 +eef9737684a3c53ca307abd11558ce0810db4d94a444fc6babbbe40f3fca201d7f66ed5f834d33 +eeb1189648d68efefb7af5a080f1b6297b75e7815b4680e3c4b2676f3c763fd9d28fd00374c2ff +2697bb8b96ea13cebdf68eb17001c041d24e19c97d33d448f4d6aa3d50d8720b0377135f30b479 +d973ebebb93031aa1faee6c275b21cd88852d903f08e2629066e18bfcdfe736b208429a99f7420 +7fe3f156e82b16cbe3d401aaf8ea2a417a7ded77fc22d8f047685fb642b7e25a50c5141c1f1f38 +a14ce3c0b8b3447d8f48de07c687e4c4cf69204e85c6489a3cadf29ca7f2363308621dfd1f48ef +e47d7fcb3f67441ce965f04f42b78d23500d25890201f30d13dc767cebd955da0ce6f6411001fa +c35b8ff9762eeafda44864cf1fd2f8c5c006ae8afaea913ae1c75765e4d6d77b7e21a6d16daac8 +1366f23778d2c5b9a64872015bbad846c77d4a9b93353935d2b6e57b39ebbb02993c18f5228d99 +a76250b85311629162556dd299f7eb2479c705c6e477167faec90ace944145a22a5cdca376cd2f +809ee685329d8ae45f3d29b2187bd4077b1e7c5889aad999a67759f2b649e82031f198bc592bff +385a196f4b9943300e1c9557ccad85497b9f2c5ac963b85527302a1d7c42d5d5c8c58328318a64 +10df0437225eb017494c3aa09f129275a02d0f0d2d5e24fa35a6b0112435d63cc0706491cdb6e7 +5dd957789831d30481e3b8e52d678d0990202a4310255adefea4a4b48abf4b835d35c66f7f279e +1d7cd4887798ef065d2218056e2374c03c99fc2e17a24a9e70b49440f5e20c8806dc23e1b1b545 +716dcea57adc5e40d7df2a7a259b559cd37c2d8fb1eeb6731601ccb46af23de0ba8c1628f4b88a +9b2d468c96730bff946e66a5d200ae32702e6fa6c907ca03518731f60b1252a5cc2dd1f9bd9042 +8c2e0f0a7c54d92afc89333da6a83c324af0629a26996457d6374437d9cef97bb99eff0e98a22c +6b0deba1062f3c48226dbc9a5899cc864b975bd1f082d7e511a5b4cf9c0d7f30d371f689da42f1 +1df7c18389941337b9c07e1cf5f5cbf33e400feb6f4aaee60a026da25d51a3bc14d427d37fbc1e +64e938a4de50f52578768bc70ce533272db34369f898761aec2c8dda5931f895d28872ec6f209d +78be4f4ab8da35f65697c57405ce7ae5541c9ff509d5762aa6297e7a0194059eef3a2ae5d59dfa +d4e9807cba47217ba163742c83f8205052cd729ed6f8c452af9712dee557d4bf8a303209d36bdd +db790812c4e532165f955c981339226c4ccdb89f78666953c02eb2f38c4cacff62da858e16feb7 +bd379b92d9420a26fe2a0aa0c71e0640001644137c538d2da78a5403cae324705be07519a4c612 +4f4e299c41497f23ab3c479ac8e309dc694cb84b693a40ef5fa7d4eaeb84e97643474253f982e3 +8060a81ef3deb08144992e048e8fe2e9719ee6c840e5be01142f4a62ff3f2f1b1d70997f1fcd18 +3c5cdc04d67e5f46f946eefd6abbcf0170141516c7bffc2a07efb440dce1ce888a5acbe4c407c8 +fa46e28d20e7774d2e374e224b991242df5a20cf3be574c410a1a88badb054b812e7f3ea575682 +e6a86792ff4ea12bb43245bfdf588df7ceae590954e88607c4201efb34b2a9cffaaa392ab52f72 +ec08de1424d2420bf7b82f2a0dc2aa083310b4a6f2a68ab5ee8384bbbf4ed9f932308c2836d6de +8da231dee2b4f08790d1653490b71ad5d0bed41b06f659f8c3a771b82c8e58ec4e0a0c5a9f7f98 +74bc6a1479b918050f06e4d1b49232d8b44f312a5f6666b49e54389c18e7d89a399fac13ea6ea5 +84dd48ac68c0ee40d26e4f2ac32951a3aef14c503c7ca2d05701b756990b9809e25384b0815fe2 +bf5084ed21d8cdab1fe3cbe15c41e8a94336359df9ed3a43c529529a9e7f63d8de36720e108d1a +9feb0b9ec91379b57d2ac8a6799176ca5d5956ab877d87050d86ab917689c93652b729d4746f6f +34de32680d8ab5c5a4565512d6ec950a60f5b0b141f348d1934b4f3689640592f22d1c1cbcdd51 +16b204061afc1b8f15afcb06f4c44905d17b8e44ec96ca2888b34eb85465ed2b2620756ed963fb +b9312883c8aaf4870c52eb032bb5cb7458b2fbd796b670e972f576dd3941c76cbed5c40b669f0d +cb34b31d295010975536989bd2a479acfcf324fd549b3e41bdefd06f42492c36d8c0201d2526e5 +3ad34e552b2384e39836743766e384e5858f33ac842cfedc28022ee8dbefb2a2c3a408a6be8482 +6023124f7eec8295a2e1079480f30eb64479f2673da8f051267142dc54e88dc8dc4ef41036f1de +1b9a198e5006f8336b29720288c9f2e545ec8663171a280273b1626f882c6894be2fa2532c2f20 +e4e24f1406c0627f9cc4021cac67c83a60b39f1849b483757f18ca956ec3bbaea96cb0e2a10477 +e69a6040dc3b5321391af5478fdded8730e825818d960c6f4a305d653a732f1c51c2ca3e3adc4a +3722958d50a78359cabcffaae1d75ce79e72e09a354c5e805967e30ae708c109ee75ef233df667 +82bebc4c5dd9ed6ef0a50eb2693787576a19b42e70e3d0c073fdfe1b3562083bc01d9b38868e64 +b8ae90b9f3164e2a29fbb1f1bdfd8ffc2a842ac2f9801d4839c97f4bdaf1466038eb4b6975e777 +acbdd875beb59268f0eae6d3a03c2bb60bda0ffaaf77c4f33d4671ee85f301521557a6f323f1b0 +d060326fd87c60f65e4220b7fdf85de8f0ed64661fddba2efd3374fc45a507437a1ad2ab4938e4 +ccbe2411ffb70718397e156d24ab018436fecd39af252f3f9527397cdab675b261045194cb3d3b +5aca47c98f1177ca1322775aa8c29216fb7f8ce0fbe698479abd173b9541fdebbc3cde6ad12be8 +16108a049d5462572544912ecac19e6e9fc6dc06160904e7bb14796e5becff4df3a8f645825c9f +59fb419bfb5ced8e5b449a2e57adc9693e9d34e6bb5a600d16189bf146f3dbaf4f75d16b303a32 +2048ddc0de4a91156aa427f1b137a714f7ec2a4aaf612de4819dede79b78c90a86c901278125bf +2999c9acfd90a866830d00a0f1c83a87cec69cb2f382c935bc571ea3b53fb22664be95d8e95d22 +9b9985d992dbc6439f0ca52762b3cd7a4a50973c17de473f987151687da95caf1457c524a4406c +98bd025f353124da7e6389a921e56de4d1ca7901cb37f051b9d3b78709640530cd549c9907ee6d +ef35aeb2eecd1c9a1a4ae7686da61b848371dcacd9f02cc320f29ef144a10803df5dc462188709 +a85504074b8439c1a3a7c253581597d6f94289e7113c678fae406b6586ec40fe51769903be7225 +9817704579212bbe802679ef21f7578fae78d1f8c29a5438b656f13d14ac4b10d1b48d83dd1ec6 +e569358f9dce2093f319f9d3663e331744bca142ec6b901acb9b5594ea7423072d96a61e523336 +059c1091823b03dc675671cfc688bd2062830e0edf7a5d400c680784691ff46bf4d8853a42c573 +20414a0c7b838bf2bc33726d8a61d11dc42005b909b241b553c37462ba057c30e7bdeeb3ec5513 +ddca5046dbf783c780ffccdc188b5890daf03c94c8bbeadd2e33e95a8bad7cf8c890753d607e1e +74dda3963d32cd37ad2d0eb722498718df3827236d7f073eed7fc3c0ce0fb5629f08d4a71d1825 +c4d158ef364c8d69e8767acbaa92e1751a266718d67fb61b4a07c485cb887f04bd30468bd6ff51 +63e5e35c7b4bf6ead5db27d8690d7c54b2be107d8180396ba6046e8b1ec7f51a624a52ad2f9e25 +96df52d886aa0f5f0316fb0a6ff881ef3cf2a11debba86921f94b6ab18088a61ccb125c7732c36 +81373e4151094543ad56ba6b5b915ca303a753a65aa189f6d71bcd8ab565ab5696b9b0a2fb5b7f +eef3dcd6ef83f859fa4be52c24c31ec711534577aba733b4f0ec2ca742f8bd658d7646e6af057d +09b504f072b85e8ce75c51cdb3b17d34c06c1ca57b88e7ae944d289bcbc1f7ad27898a703c0b2b +9e0dfc9f87a6b2c1d9ea2cc05fef8bffb8cffd8d8246ddc86e135a8361d36748680881d25daeef +be63d60e449d50acc1718f1b6df6297849d31eecb6c990304d5e37d6bcd83c17c4ebded22db524 +66af31cdd0b576d69ed50ef359cb44507a79462b282504a48a4b8bbe2ef617837ae266e3ab90e5 +f45647ed6b99c6d8b6121f1f7771f75b76362f01bbe8a8b4268d4755c7b613b2faed3f108c29bd +a855012eea6c22210f618b416e25534efec4c0f6d0feaa4ce7517802f2ccf54c96c4b5b9227bb5 +9f8c8a49fb435d4134dce397f41f3b6d295f6453b9483c3c1d2ffd47a3199b370f2f31b3772afd +59d0135ee2970b66acc4a1a0d3a18be2b3cc0a8a261e9dfe32b1a63ebc7fd99af94cabd34eeafc +a393ab83b5ced9b4e60f868e0b7aad2c2f21b0d84919f1c1598e6143440481e3c4a97750300f7e +0660eb65416215d13d8f546f74c4700693327ae762ef61db4463f3644d534a3b4127c06741f0f3 +394f395ed384c4139b7ffc5b02185dd9e9fc4a9a790d91eda97f8262e6327779701f7440a27120 +c75242cb2e3c5570399c648b21da288d2a9d610dd11d5f9cfae9aa1caf9f05437bc137d8cf0e83 +272ecdec1c574ee6880505f03fb673c5ef5e31a674d1fa390de77900b9dc1fe4d73311032af31f +6985bf38005fd3498d569de012062aa9c4e5baee3ee651230cf8f06f9562482e56aef639b4e9b1 +4fe104d7a4e59d9f59ed573a530fead02933ab576a225e00d4681821a6bd294b96c9f73ead9eba +509ae5d9008007ece9e52dbe377e161c5b5e1fe556db12e167595ca9024a24c72e65527e3f4b94 +353bc5fd302dcd2343dfc038950bbc4c1249776e2561a90c9a5ca9c045f6812d9491a77d5f775c +caf51b81f299e1f9122c50712fcd660bf649f6282aee3c072d41bfb68cc3aec648f2201fd414d5 +0e1375dc1fa058a80f54ef48522715a66f46b0da8861864fdf8e22f6e47cb32837b34ffdc5b578 +717a1119c6e7ef7566c58b373d111eb6c2bf1e9853d13c6223fac9ad3123765163c06dfc0637c6 +92af835e89ebecf5b37620104cb8f17851abfa357e02703a059d960d9df6f7fdbb168337708e7f +919c04c802ea2e2b3a3babf74eb86e63de9b8644627d8985a1ad3b8adcca509f7a7debdd4634cb +f42715063265b57d40eb7475423c3838359cb5a1fccfcc7874c782cd859ba71d5f114991e6fe5e +557fd360943786fa161075b6783140ef30382f184be07a3b0bb616442a4a4007af227492a99485 +4b40298b9e9b4e4483dd252b2230f1f615a122918f32c3ea0a68805fcc73bca40be7f3fd94e698 +d0a65005a64dcbb7d97f60cab183acc86762873f18b673dc194cfce103f07cad09c8fd6b1f532d +72cc86ea99ea24a21a41a2e09837093d5e1f1e3d74342cc4466d9ed9c1722d8665b00a9f254950 +c286426737b8690914d856ac20f1bf27180e6f9b00981bc8b3b165c5cc7e02a689dccfd58c1b59 +39186c590a97d04d5710fb5506bcfff05f8c6241c125e300235afcd2311d997a1369783a83ee1f +040b8fe44804dd62ff7e5e77dbfd66177a934eaee33737d643092a0cf65e51b131a7ce8e502102 +aba1a8a68b8d95c5b0651c83214017ea2bd5e5a52d7e9d8afbba7cd22bbcaae087ead54739401b +0d8acda7766d94d640feecca18f1829ae75da4aba8485bc9dfc9c67409c7d465f54d740ccc8ed6 +452872082bea2773fd5d646a809f5d382b69955db3a56e61849017f103bec6de80850132002216 +dfd301110250985d8241933cab0ceac16e90d929dda1d3169a8de16ca98aeec1bf59b9b4e0ec46 +0a2631f6d3ad4f84fde5b5dcd29d53afbc6c8a3adf15d234b13de3fce52689a74f49cdf2d9affd +89a098d6fa7c0f93724a08324ed1ce7db85034bf4a67e7d0573db8e2fbd09e123156b2d066bccd +9db05b87cf61545a6e219009452b7541aba3802cade07019910ced99eccd026725d0853f9f1597 +03f449d1e1d33902148421d34f132378dd12d9e79168bcd8dec5bfe47c862adbf7883e46dcf0c2 +41e068ca77b72ce9a73a3ac40f568481f62d9f36ede5f0264c4d8f9b2ce03e19e70a94d0e467a9 +832efbacffcff9efbcbd405432914bc36fcbd16c4c2aba631f8213239b69f183482bd292247042 +7fbab78cc598a57819dae5e3321efef1fdb44e084a4402b8cee480242e42dc09bbf208042dea79 +68a643851cac21aebc1ba2740ac999d965b43feaedc8c76a32ff8d4fa30e2c8444b01716af4878 +ecf291309cbdb70c5bfbd4134ad5d044a04a057405a91e1a1ef15980383cd4aadd4190904d5528 +e55138635714ec3f9d8e0fe1bd1e8ed4b742ea90572f1ac8d01e711c074a4125ea595fbce07461 +21a5d022d16b2125a1577850f6d1010279a5dbbf135eb8edcdcfc459b7b554192149c29327f86e +477dd3d13fa5ae3a78cfb714c9a750067ba154407ef22158670059820c8041941f7ff5c27c016b +20b867f83a9c8955dcd282dbb6f72bd2067dfc1e91373a3021be2be6aec5f90c8001878077eabc +92639f5beb892bfded1d04136bb3140dafaeeeb31c21d8baf03715fc56ecc16e7dd11c12df0bb6 +8240772f2c59829482cf97f6a922c8c7deccd1a395e7ba2a1a855206c27630ae1f236606a3a9db +a1404326b8b484d5d2564b480ee7422365234cef2609605ad9e24294ce5f5bdd61a479c33fe53e +e8bf857e2d2a25dbbc9a1cfad3a5d7b2f1155d29367e318c8a82e1e0aa236b8442838e3deeb9f7 +455923cd769d28c1f780c0d3bffc275e0242ec975d2beb7f6bb5222b2cdd196e4ec94452050a76 +a4aa6352633cd58f461fd646f23758e343b5a386927b66ac6ef1acc84839f8bcca9f9f93f18cb4 +c6bbc663a951dd5211acd684ef945c10f156b5b6e1c182a7a580b3966716c334ddd9f78a404873 +a738ab6ccf8a335f1f6dc830d7d74e021bbca95d260b34d1933177e5ead9c2793d264c94e8bb27 +4b9712afb7862bbd8f74b4cce27cac9f29c086bb30162820a229e6d5c83410a8b876fad5b8a028 +abc067885b5cec60995550d35f60afd1da93e3dc14c42decd2b0800706a222e9e4b7d7bba9646f +6a3ef9489dfb011045a6444cb599c62dcae4b11de7d8d98f8e58a9a4feba0e5f7301b795f4085b +aa4a7085c5a438d0b784b21186787abf88b455f1e2ef19fbbb4501d5c69c22b219737aa5ceca0d +0475bb6916685534448816d2791a31544769ff1152c2f2cf4c7cd3f64c6830522219349254d11e +6ff2056a341648e9caf2fc5c6f48a8713448b4a0a7bf4f250df1f9ab9db798e0c5d2cad0116608 +a0a6190d8fa4f8e67f705d8dfc7cd8be652b13ec24beb159f97d22997094e38e90fe6e9f3d2a07 +ac92f1648c78e6ca7452481fb78dfefdb4399c62986b652e08ca4a9a4966cbbb572c4b2c8bedb9 +285372904836abcea90e91ee6fe8dbe8fcf718f4e0ce97b29be634db1f19b541e003409442a59e +320cd38789b76eb79e35acb1e09f5ac2eee068664a0f505c61c07da5bbf439acd52feb4caac79f +f05756472aada68f6aad5a1b760468859d9297e3ac4ceaf7830c09d3562c012d48cf7da5f93d79 +42a15fd76be4e90cb1a966503d32dd2fc39c1cd01c629b600ba6463288466ebf44a4df63fe2040 +67d6a39719ab32b5073d4bdd022933b5bb5652510919af2f90863be9f08dc507609487ebd7b796 +a3f0b029e13492cd0a1dc002d85b8da0d2ddf319f7e732ed40a87c984340976292119ab947b5f5 +b9e5e31e6023e22e6ef0d04bb9d9cbb7de56a1626c5925847300a6dbeed495b18deeb815807010 +52e4b37d9e0b346aa8a55621bf5dfe3bd1cbf2051e62ab1e1da21387b5f38c119934ffab0e8158 +107bc33f36a537996faf01870b85d5da02977ba62fb8e5d4878e1be660a6414c9084a3a5f05a4a +2c3c1f311a589ba7afc5418e3b480c92f91033ea61d0a6a1f9e209226f2cab2d9c10e2b54f22e9 +a730ab8d48548cd28c3dab415b03a59345bdfa9656cbdf48363eb233dead8ed309268a629bbc3c +6e2d85eab15f3b769afef377e4602fee3efe33401eba124aa4d4593c5356214c59db31702f44d3 +3f89196e09e6e2215687c448c9615c69d7163b2867252de9771c9ee8a39587bb374d1b705d61c4 +2da4a8a7ed8ebd24d1062d377fb37fd84d5ec3eac25d5d39bdc91f7c820b9fe806984bcfd3c375 +3a607650a6ee4460a24906f75b38f2c442a65909bb70e7383f9e90a06f13fa2acf0a37e59a3118 +5b6f9437e741571dbefb518dd564140aab4c91d37e6435ab9b2b376bf6522f90b5660ccd1d7e8b +82dc3a41d34e2c39cdfa05f1e4a7ebdd3586e69c6ccc4847e086052dc7d6cd4d8302c34f4f0402 +daf9549bdd83c24c44a68cc2e0c2635686c6e49ed8bd5675749650a79e40588f4315fdb9c187fa +7b4956151b7123f36937b06fd147eb8d51a8eeac5461441d24afbadc7922cfc237823efff1a3e7 +42b336e84702189c648ab7078d6097d8fd5f3baec947d63912156478662aeda72a501c747a325d +1227cd4093ffe9a931367dde46d65857290ade9dc3d8eb73e7149dacc34c08faa51f9fa39e0c13 +d14be58999496dfde61566e68f83135a0e43cab3cc6395217a0845423210cb032c63d56bae6877 +3991e91f186b198105f165723e08f6db39e889ef3f944d339886c0c6ed8e41c20c0743864a50d3 +7c281abf47fba6d08728aa21a833f22aead0bc38d211caf58d44d01e10c2da4065a3082e562e82 +e3f1def28b5fbab1e93cc3716256b98ce87e4dde9e93622e5ddce60981e89ad0d70d0fa8b1638f +4c12ee82a601073f84ccd47b631781ef34da22b5cd35023f23c6fff02973f993d7457efe579da6 +4d7110867d6fabe6b41b97e9e711369232f98cac9975d56d69cfc98530e76ecfa380311854170f +98cc43596a3d4019d08c25e59dbc3eb850e026cd484644353c125e38f199b22ccf0a502d675d5e +0be2a4df28ea88c3ca6b939b4b19e81ab728d3e44648830f0ccaac81f705d098c127f07f56ca85 +50ceb827722d046ce419a2504d8d6740cf9ccb5d079f1086dff76bcef4bd20e854eed00bce0858 +522818c8ebae755857d51961a9c61ab5367445b621562c645664f6df9f63a21246ca26d21aeb97 +ab2b271003d5734719358b16d2af5cbc1799c88b899e3c964c453e53de23d02bd3137cc0c1f074 +70d8ffea99d6a27e835ac56c977c149dfd13de3c0c328f8edd67c1f95953b9892e0147a245ee0d +4e5912c7eba0b810c0667b17b545543b0662d53be35c26192356522fd7db85df65e5db4d6573b6 +c0c0f5da3747e6fd6c4a041525f7e48b56ae420995938a2229c967e98c8256c28f3a99b0739f75 +f0f1d250db34ecd7abcf25b86e6b94de40d033d99710fa654fafc2adf0118f750a07aea2c94f20 +5cbd091eb2a3bbbff77253122f179395e088c6b7a9f06ad421055622a1cc0538e8b3f03c4dec14 +746a49b2a19bb161ef7d5da64a4562e3a0ad91e4ad5a052d74c1473febc4f186cd6b73d48a895e +8bbba44657d00663d5e76686f65a7ddc45ab8194be9fcd289e170994d8f17d4847080ec3f0fc36 +f811035bde3970e330f1b654034faf87b01b9981330b3513c2f32e13a280b52c92449525bd879f +684edc107a40c02da468ede54c9bf7ae72d4c314563e55d9d19496d0f80472b2f3c8398700bee6 +42314f87fcee7e0fb9e4500fc980e0a9f755c71cd9afe4b3946aa06d2ae458ea77080c3308f65b +fa253674be7182dea45a8d2c029a570cb3914f7c7cd0cf31f9b537b4f2f7cf918c5e82c6ab36f6 +5ac444ce935e1d14f413c2c005ce388ef8175d7c8e7bfd68640c650385b646fd00dbdfbb9e0ec8 +3d93f7ddd486a358a8493a4327d11568d3145b52cfbd0bd5b4d7def679906fe1297a16908e16b9 +de0a13986d5ebf4f829c4de8eeab729480c1372c0b85c462d02d8e6bb0c45fb26e4d82f0152eea +9b745e1c20e478ea50110efdc5bf0164a678ecf71278fb2bf0c8a53a99ec0f46029eaff9229108 +234fa1624b5e33f9272a232f64acd955839b942f8f8cd3f3e3871a443f685149bc5ad0083f6baf +20c14025309173788aab5568160b31e7f7ab6f10f4033345656fe958613d626e8ab7091aae62e2 +150d7f854d5740f601c0756b4d04a2f4ced1f833042b4ea609a051defe8aed847795cacc657c5d +de3331323d5f21b24154b3b2c991e3da5de77d00a4577ccc862f035c10f2c073f85b97fb489a5d +5a17d84d9ca133a018a22788ef86afd7c1c8952eadc6cf5139605b19df95546e90a839440d56ea +29625d57eba80472486cae74bd099198ab8619cbb6fcd652774b0a97806f5f83f6e8a109eeca85 +cc7bfd19ae0cf27336e00d98d130f932be9752cbb97a14a44fc6407b286744ff893a26ede2d28c +6d5a2c2369388cd6a23bf8e102d7014b3944afea96cbe03735db0e6e4922d4c5b2b6add56d317f +3532fc50a410cf19be859075abe70ed195996f9b9afc7b1f3b1a5d1602fc6d55d5171ccc345e28 +040625e4fabb286990116ff9dfdbbbd193d503d5ff027b0f4a3b124f2e0a1e428379ac6fecfa31 +3fd135a6f38da32b37b3d6b877e5148b55aa40567b78d69bfc59681acf8c871507eb03518650ee +ac7270883e7003ab9ca8a2e531516c309c31f67a059349b0f603f584c6d91770ded8a825c6f759 +cb3128bb928cb1e671c1684cfa153a41e6e775ebdf54f25d11865ad5972e8bf3193fa4044fa618 +bfb88b621420c86a5a30b4716feaa63197c6081bb1f324605d018e8c63753fab6f972670cbaa9a +4946b9c20fa42c6e52c9751654a3cd885fd9efe11600771861ae0d026fe0dbe642b073c91036ae +eecfaa08a7d85983eb13828c3903c532c0829802ff8b45e64e39079f65a5b84ff91b65688eb31e +6d5fc58cadc44319885bda8d78db2aab9fc4a268e3397d12d51ca0ffc79929344f97a6201c01a1 +83d2c0c2702049048205649b91a784c7095b8630b00c1ee0af742945873049981540863fe38a4e +57cfc8647c85c069296284599f169506c9df019471e0ca8fbc9abf7d1af1dcec4802e72bfb1d16 +138585e451918e0c421966a52ac29c21aa3822870959543691460c52ee43517b8588c365888921 +e4560d3e28f72358325197f09baea07fccf9d10b238659be645e0bcdd5e76534b21b52941a1e48 +267faae6abd43149987eb5385b44a422c0d6eb37c818542c3553452c97e016c26d38a4c0b53ae8 +b156167a5e11b861b44bc488bb2bb0cc652e3a1d12e616a502d1fcf31bfdcead55e2fdffc121ce +85f525ba491feb547a3d16d85f65ad30e430dca5c603685abe8484d4c4144f91c5595a3fda9351 +e17f00eeccfcdb2e0465566192b23a9a78c14d62fdb3b1963c780f6855dd73382089094e921ce0 +9784e1f5408c620f6481d300d9fa6f8c23876a49281b0d425090497b767614696424dcecad04f0 +767bb7601b99dd3bc5cc14d2dd77c09101fbd3b6e725e5c68ba66a5a371fd6f3ca6351ef075645 +a3c03492f7fbfa225a5f3e01b962177026d94e845953d5781064633edc41fcfed874e011304723 +a52fe8ebe5d6efc5e69b9edf735067ba2d3ef221ee6244034df2469afb6056dba877ca9850d4e2 +0ce756a79a38e74b356019de03ca6651e54d1523e12ba9d9dd8e44dae7a920eba8784607fd0eae +b0b456bad6e739e264aa0a3f4d5cc5584e072cfc8e0a2b949b4102d14fe3bbcd7ded2405718b0a +7cd41deff91bd415340d6de94797c53cd734d14a11be9fd9de57cc35175f24913e96f95315ad3c +66ee102176e59a3b669c0b992d1b6c3cc131b33119eca4fae72f7553cb977dcffbd5ea29728ac5 +a645eaab33042a0ad86e8b28fbdf79325e28c3801cd0f9d8a2b563aaebcebfc6540a3ea351caa6 +01ba8e5c5bfed770d61d7366f96d6bcde2db981b51e38ba35237ead55fa9c1cc2b3381131c1dbb +39ddd7229d06e4b9d203d066666c8ad6aaa1a6df0d346f843205c41a95f72df3027c9e6486a008 +8f8f8e7c9ef820e2e80766fc706c4df71ba5801863b29a917a16abcd824d9a94c9081208685f8a +4398004d8afa25087ba3fc81dceed46b7d6fc92d3b9557514899c5c97114ed722616b477ccfe4f +4e1faba32e6a24a091a7a1067628ecc8c421f8a967c6e0136a9c23ef642100eb91c300975d6dc2 +c09da077e24fed0aeaf2876b91d0db63c4e347afd558f61344eafd38df995bd98fcb720ce19023 +c26e0b1157e299e18d13fc7715bc1141d449f05fc6bf234ab5af157696a354495e45a818f41a00 +7e601b49500e7554695a4c21c8f6fb67d7b81379f6d3abff7b059f277e48c0291852c6af3e9d41 +2526e6b23653e0283af4ff8f06503ef135f552270f04d735cc21fbaa059025e7e32cc7f03bdc40 +6a167be87ce1442f285aba9aab9f4ff91daed0f04eadcc8e98a6ef670263a76e5b77baf65f979e +202658c06d0deaa83aeb577e4872a916a87b8aad9e9c9b22630d3b3b95b790f11f83bc6a52497a +7c03264d70f55b6ee2ecf4f5af13d45df4bcefc85437ad3d9b78e565e3dadc353e972fc21ba6f8 +ce701cb9c1aae65b2ffd89d65075bd6a5ae8f195d13165969e08c20a6143efa43a31963e81805b +584f93e1957713065d371abeef4ade6c3c024f37bcb2cc00895252332325359e6f7425665f46bf +49ed14cbdd14a659876670d962d36aab38e75c47db93978cc7c30e5e33185c437cfc260caa8c3a +f8a0fe39d955fe41a89344db01f0f33632b571832eff64ed70de8b5dcf90e73e908b274441d7e3 +4e14381c6a88bf71db27038b3037bcd5eb5a4a1c9b6067d2874b898c95dee5b114c10528dd3cd0 +4d4127190405f89ad7461b66eac5bb976c6eb433a43db8847ab105a9d6e004c1d08e17ecc40286 +2aeba56fa63297c5dab64a0f3dba016a5af5c761de36c9bdf0b375c0bd6fd3c0fa3787e362103c +3f96d6d1bca0fe0b3fa33c034f472204cef2949742fa5b973d21b8af8e7c1ede70cb40860f7a58 +ac12a7709be24ba724166770215440b0dd64dbe6af369ec9fc0d82e3c7a597884e4330109ab763 +1b8f8a62f0ce7dc17c9919d55bd997497b66e4089b640ada46e454b4cd90a45a5805eda36ec5db +69ca53cd3380d2f59c9cc4c6d52139cc33324161adabcf2eb44cfce47ba893bd4a53d4480118c1 +ba64436d3771ded3182b2de8306237012aea5d5a2995abcbc097e7535e259053c58c8de2a6c7c8 +d21ad6feec5907c2f6783541c68d00be43776a8e766d72a8fa8123d819d38f1f53f36bd48adcc6 +f7a1455cc87453f96e550903df9b819196a27061facc772b773dbd1444c37d3136a0d3158d79e9 +8a70e8e030dbaa42c202db2c0f8b03687f3d48b0f1783124024a5ebc0433990b1591459766cbea +adebafa313226d839697f97e14cad664856368583b3e2e79609cac4c81afe602dd4200cfa43fda +4f1e48d68b13f27533ab9f3fd05a0143378b81d7e5bc77eb8829d3d4e896af1c31ae42d39bc60f +687306a10107727c8fc2fb98ad61af9b7808b6afccd125519cc4f98828e53ac23388ffa188e885 +c64a97439cabfeeaca48453cf4c6ac118ff667c21300d1c1ed5fef10e03f22e69e36248e476519 +a677fd224e37ac95fa3d9aeea5b2482da91aec2f5e7a2b57a84ec7ec8699fc6581757cfc8ed6ba +b56242725f615304b3cc39c20e8dd67136f864ad32605b0ea9ab0d3662bb1ec7e317684a5a6afd +8c3e83358db1392bdfb53dcb2d1624ba8772f701a3626967c0a543a4b01d70aadea4d1c2c86383 +c6eb9a722a6c833c0c5f7e92efd67ff6bb9fc72a9933a446ac965c26c72ab279c5755890d21a9d +d9f3862582bdb1d9ccd1963a21bacd4be17d03819ab523f4473b20a87f120dcef914c72144e350 +ac33a94859de19fc1c45816ba504646b8b53410dedf6f2b1dd9b2f0b2749013aeba319bc7d2e1a +dc99cbcff5d5ca41baf982ab06c3741fdde91b5ae8601be0a690a8848d30eb1727351734daa8ec +8fc416bb425309b1232d290ad42ab8d97062812696531e1a5f6f87844f9ad9fe0cc660a7c4bb1f +7c2d37ac95214fe2cb4537f763691698dfc6cb42aee39ca3e37910a5caa4bbc93b20d756593622 +3f79d777d14971397338d791989bdd842a25eb4747a45fb3d0d853000205301a799d0f1a597a34 +fad9c47ba0c27a82251b4d460f1de01fd0066bef08e4e6139412f5f7b46ca9c6fc7b5e1bff8f5a +24498699cc168b419278a41f5eb1845b41a298cda7e237d3b149ebc1ebb02c33433c0cc8043bee +875475cc389dfdb141fa31a67c9b97f2bd9afcdcbd16b0682b3da7c3fe91ce83fa0eee07e155db +bafa26ff706798c07bc149d963c31bd8b3d9736a21d45d5d7761f6d93111a10b978b32a00b6593 +4ad2e4df3c2979ce6179aaa50a28348d3fd1743d482c16a2538eaaa17f6894fea19a21411ac73f +f779f355ae9fca182a55d43f1080a60721ec087738c87987c4699e24c19172afdf0461ef5d7582 +0f2957bf75b1b396bf2a5462c7be537e67d28ddeb130863434670c75b36878888a430846e1f656 +5c9f0f529ad4e5d2386dc61f9fb0f4782a802c853cf5830763f5b9df5d77c5c3fef77c14fa64b9 +66573237b911aa353738e3ce2683d5a4612ddce6c8f50cd2c0b11912b7b09cb60590ea9b25dad4 +1363a82751aff5a9da8e1fb9643a84f3701b518b059e935a3199685c4bf2afa07980ad93fcc6e7 +733a7c1cdb6d9ea9756eb985041297452d74381a609bb8c3f9cc4c7b9ad09e57753602e856b551 +1cda7b0c9a5e0fbab6c9ef7d7e8d6751760312ba04d771974ce5b6c18d9de0d4bf76d5e5b6cd7c +3706f94693215165881bf682231eb90cb99794177468d89877475bd1510e117d5f845f26fa3037 +62c92c2ced4c73a82afeedb275f48fb2cbbee350511004a58efd38e7002e91260e40f0792b73db +c2158d542146aac2b860479bf6046c7e0e4edc26090205846fc8e333cc60a4aa3990290508b400 +fb02f09560be4f200d272c713ba3b615e2b40b9552cd70a67a098edf6f1bc4ea86732f1fcc762f +513a1a004d39c1dd6d8d065024ad72b147c850945b7bfb29e830d4ef2e84d694422f1c89bc34ee +502aca300df3c5a3e0ff8359668b11ea33ce14ef62cca8e2dfb8e5ab68d96a9a2552bf53af81e7 +8c84d96f3b1eda14e19ac07e75e3d3e5454200592a259bc950ccb398dc20862141083a5190c1b2 +cb0aa0d8d3e8a5a3b21004fd002040c4b3ea35de14ccc1ccd679d99308409a138c1ec129a71a1b +b1443d1754f4763765b6409e8280e25b5965f444cbb2a6b78bf8541250b992e39d0a4c14ed92c7 +b9ca642ec4002946f593ae24cc194f41357a86b418c8451b342860441b9d2e33c7f8a81ecfc0e5 +630d23ee1fe752b2536ca59253c97aa18712f73f2b0335eeb0c2a26cbd98eedc669e9388314277 +18070324999284196cdbae8f16946b2bbf2b9f2a8e0a59284473831de3315a2dc0ae872b1036a8 +040d9c2a0c77a6c0c473c293ab7e19837fd227b352e8bf1bc817c76412f1d6abd47d809e989b3b +2f89285d2693824df3191a6d1bc8c9853f2bf81b8c96607a02b31d23d7495a590cf3f96d7328c4 +fd74452dc16c2818512abf981c423d146e9d7a46bf2ef32a98dda4f6695f42532fabb8abba65de +15bf69090b724a247fcdfefd6f1a83cc35a049d3324bd39816f4ee13a5a38da73a3c3fb031daf9 +bf78fde0f8f6bdd0e897a3b24e090b54a955b232b2258e5f2bfbfbe13db81f6abd243d73e3ff7d +5d21527541ce6258732e85baddc7e56dc1ae4b2bdead6d8de14b406a7234ab0c0fb1b6f129a787 +7adb864cfe9abef498ea89232ee6639c0cea4dbd515a7a09aafbe65d04ec923e1200fcce3a2f68 +4c07f202273eb6a93f7ae0771e7115385296132be25b258f67273c5a0f28f98627abf77751b786 +0e123e4a4e70422114eafe03c2606f854588ef418312650f11393ea15bfa6337b955355a183573 +948398e4438b3e13fc83a36f2d86cca04e6787f6a0b29bfbb9aba924f37f5174e3ee892d16102d +37b14d7743d9725ac7367217425cf6263f30097ef995d81458db1357305c6044d6e490d9631e9e +e456db83af1d454176174c786a03a7771b02c10daa800cfa6819f878c871f0b3230e5aeb87d8e9 +b5d0e701fc2dd78a2ef5f9e3f0e060eeafc7ebeb813d13d4d77f1130256ceda94dd73b6e7b423d +7f4c0c55f3491bbf6153b106b1316b9eb2ce72b263f05fda150b748d74b05f238982702f9564fb +43259e9fd1f77d753f7a9bee8d81b2ea2d5babfa686d76582857a1a2338a5f8c2733728fdd4adf +fcc90331d2670c260111b76369299e8f79a9570d277107c81fbd22faf23d821471f8359f13980f +282516b7800ba8ecbac26e646e401a4d7a644fc3851749c5d19a915fb046bc40de6256a24184d6 +66de6f54e6855060071896eea3338fbea7771f129a449e5bbb498f5edf7dfb723d4838d670b76a +e2842b726e757d534c24ebc471588ea38760f07b59c9af9649132871a23ce328acfee7f583f7ff +0486e07187a3d4942aaa9e6a38c61543299873ae9c902dc5777ef58d396b5feb718509da30302c +a268a9fe75e8343ea5f4da3a39445369280cfca5442939a61f82f24b15bfc71d2e1b2880df0117 +3232527227875329ab6069618cde196f153e7ec3b1dc5fa627ed4f49a43550d02cdd80cfc131d8 +6dfd6ef64530f16be3c08b2fe804ba2c8b8ef512bac05420f7809507cdc6aedbfa690c85a1bee6 +f424799dd0407f40e90051ce3d6b1020719e059187a78047533060ed3426bb7c314b179b608fdb +19f181e4cf616e0eb850c40885f2e1d9c5afa60dd6480166d7b227b731c8491237112df20561ef +85e4a3d2074bf8ba61da1e50699e9f9aba612c8a9a536cfaf7c9137b08aeee41148f96118a43a9 +01722f64a6457937eb18f9733fc353987b8e1fb9da5ec6c7229dad8d045654bcf4426f7148ef19 +42cd42954a8e5e5767ddd899f178f6f2443b93be25d5d61b9c06e68d653ba56f1506a88f9439a8 +96b3f604880099548ab0dbc39b26237dd1bbf9c1858716d858b94377708b44d800b95f77ba4f63 +577d1758719e208c1d4757e53eec9ad4e1919e197b4507d1d43355c142e91c3d4d80064fa7fd5a +85d7c08f43cdb63e507726b4378853d6ad68ecc29592eef5a8b72869a2164552ae622f3f02dfa6 +cc2b5452898943fb90521daee4b1679df0d385b9ff15c9ac6c120b30626415ed3415d06608b8b1 +82f3efd579a539771a2af421119c43a5444f187b55fb3b8668eafbcb63b919a2e2152aaa0d6261 +3c37ca5002c6daea891411cb46d58d815d7f231678185a0ea2b59ceb1030a8a9959f3426a66635 +2ce86ddb34220f43b9f0695747f65b5c7d0c0bdaf97f76a67c396f24f5c3d277a375744310a9f0 +9dbe9f4501739523f301d94e6519029352ff5c1cced0d6dd89af38c49d8fdfadbfc0baf374d259 +404ce7bfd2bf94a09a02e3463ab9c6d975190944acde85894ca4537a35ae86e26af1bd5e5bb7c7 +d24e671bd182153a5022047bd8c138016aa85f3b7b62bfdbc06ae4dbe5e37528d6c6432b3db333 +a1347a09838c17b5de1b2f9570c33e7580978d48effc18fa8e0fec834076990e0d4b8abaa2c336 +40801153c5ffc858524015d49008574cf40c060b751c67395e8d39952b5514463241e8aca46f92 +701b889780db90362ca74b5d13c5f9d3362a0b53d2dbaac2c54836e4e42422aa713a20a5124ee3 +6b353631a3933a5bbe368b0c4fd93aac08fdbdd55759778a0b7ff59a185e2f9162c0ab4ce98bd3 +7fdc7a85dddcddc3d5eba60201668eda23e041e6920b10912d251171fd3fa5bdbeae378e996d03 +9c7b07bc4aca72e81625f6f816a2bb469d2458518988aaddceb796926d796673c030554e704ff2 +6a134f1ca36bb88b0ad4b229fa59662064dff370c0cb6ec361327a9faf09f93d97e8002d4e93f4 +06f5b7ac4c823081ff2bf77e20f159ba1f91e1d11d5f5b52221cca89cfb8cca13502716df8352a +88e941027fecba9041f503953c0a4df439e212a56af4bbf445d3e334716b0020113c6bcd8ce189 +c2aeabe3253220c9b027b07b1a92b31ee5842e31da8d1def2b7f4d9c9cbfad69e25ccc1c337a5d +e90b8ef56bc380a19d68a998797a80bf9d8e68ca73af55fde9f461c0f38be9e396b0c5f77751b1 +263400088de53cd3138f4393370364c05662d96915ab7c0c4316fbd98a47851862e6d2d728294f +1426bb098197cada4647cd3d8ea31d0bd099319009cb1ab7c27cf35f340ef4085c03fe7b8a18c7 +b36ac429dfa36e70e9a0d442f1bdc8d4e7769022d0e22d7e5d2b1b7f64ae651f28cbfa5d88ac64 +7b13b1ed008437c3a2a9e2b43ef41682aa329bb43aea0c510a3d210ec9b2b77f8ed1ed75b39870 +ddd29d46cb6c749cfd266b31b42649ff297943007036344ab551166ab1481205834db71b4c458d +77c7281b24ce54f160ffa21d4601aca5e15304cb77b9f9eb0f2575feddddc92489f481237a76af +4c543f7ed8b1e73388ff0221c9693b311b2c6b3db45e2880ce0d907266679c9fa804a32fab6d5c +9747d27470b5bc3e388b327f397fe30789ad87072fc917f24d927a009e55310619c76fe7484773 +585ded0c69489294af61d36f77fbb0caa75467eab711b91701578abb70dcdbdc70049cd2699070 +d1d80ad1e7008432f2e833a52bbccff7e352739e3582637ee0c35c37ba4a5aa2fcb032da6fca75 +15f49cf3db2d45b3136842cb0b55f369053d894ca695dfc3f3382434a8eef782f9449a8994da5f +09a7fa9ba740949486e06d174c131124ee53342301e5c6447b5ea5692f731cd054faeb36e2491a +e0d34665b73a5525134cfd8e655d4a186df17f130746371e6824a2448aca6dd1d2d72dfb6cbf39 +c24d93668db60b958731f23a1f72daa840194a392cde18507d0f232860fabd9ea83f9e943e23b4 +d3f2b7c88cdcded2136c82ef5bc2b745f8beca763ba14055506b1001fbecdc75169a33e74b08d2 +dc5e26a4ca55c42bc3db41bfd959a758fcbfd0dc200dfb0d2eac41aeb7d8257cf1934c58d122c1 +8fe92f1f94c3097b5081a18e963bcd2d78aa3a418fb2af5c57432f77ea5120fe44fbc9fc4288ae +40fbd4ae63f87fd1af3aa5ff2397a32238567b0b23f031deb140829cc2ce0f0fdac0d1e0144155 +3e1698f089129ba25da5b11c0849ec820c3e18cdd0c0750174a426ce91a2805d3d50f7b2419a17 +a6af399c09407ecbb7323d8ab6e363515e2aa7b66eb8de8082306400cca47eb8838f0a3b228162 +fd45397999d9c047b1306164e63cec5b6de93a1b4895b1d075a8e79e2f987c6f8b70159905fff1 +59d52f2b0a7c53bacdb8ac7d592ef9b2073f90a20979fc1e307c445f7d08837df15a9cb87f436d +7024a32cbcf5328e4e7f82297baac4eef2d2ca1388cc364dc774f339748d0cd9894fda0a481bde +5288fdf06de5f225ea3b46b2151735738c4ec5599d85d62e36d3d30f8d75eb5d1ea29369f1a847 +4ea012234234b0dabc64f1177debbc6c568fbc49e5d49027058c6bd8c4fcfc491909822919e094 +b2cfc61f5056bf641fe88fcd7080951733203963a7d9f826e45cc552ef8bf0d3bef39482cb8a3d +ae77225e71703b6175e0432c6c5b160d27ed3613831b708d8a1f4050bd83c6cece7cec3d956886 +54cfc1a3d8d44fa69997cc57fba92c42dfa67f90765c6ed704f2028c089a60f80f9fbfa165ddfe +27bf4ac36ee8b9fd4d5120afc826b997e4273a67f07618b23ed6bc836953733e0f3331ff24c8e7 +3ceebd14805a01e2 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +0000000000000000000000000000000000000000000000000000000000000000 +cleartomark +{restore}if + +%%Page: 1 1 +%%BeginPageSetup +%%PageBoundingBox: 0 -1 212 484 +%%EndPageSetup +q 0 -1 212 485 rectclip q +0 483.095 212 -484 re W n +1 g +0 483.095 211.566 -483.094 re f +0.980392 g +28.098 266.716 158.008 -185.887 re f +0 g +1.175918 w +0 J +0 j +[] 0.0 d +4 M q 1 0 0 -1 0 483.095367 cm +28.098 216.379 158.008 185.887 re S Q +0.588235 g +0.8 w +q 1 0 0 -1 0 483.095367 cm +137.109 239.227 m 137.109 285.289 l S Q +BT +11.2 0 0 11.2 125.794472 245.486571 Tm +/f-0-0 1 Tf +[<01>-1<0203>]TJ +ET +q 1 0 0 -1 0 483.095367 cm +167.461 360.426 m 167.461 471.82 l S Q +BT +11.2 0 0 11.2 156.14778 124.28452 Tm +/f-0-0 1 Tf +[<01>-1<0203>]TJ +ET +q 1 0 0 -1 0 483.095367 cm +53.273 256.988 m 53.273 326.484 l S Q +BT +11.2 0 0 11.2 41.960955 227.724144 Tm +/f-0-0 1 Tf +[<01>-1<0203>]TJ +ET +q 1 0 0 -1 0 483.095367 cm +116.309 344.516 m 116.309 432.195 l S Q +BT +11.2 0 0 11.2 104.994472 140.195922 Tm +/f-0-0 1 Tf +[<01>-1<0203>]TJ +ET +q 1 0 0 -1 0 483.095367 cm +110.387 438.07 m 122.203 426.238 l S Q +q 1 0 0 -1 0 483.095367 cm +110.387 426.25 m 122.191 438.043 l S Q +q 1 0 0 -1 0 483.095367 cm +84.879 275.988 m 84.879 363.668 l S Q +BT +11.2 0 0 11.2 73.563314 208.723241 Tm +/f-0-0 1 Tf +[<01>-1<0203>]TJ +ET +q 1 0 0 -1 0 483.095367 cm +78.953 369.543 m 90.773 357.711 l S Q +q 1 0 0 -1 0 483.095367 cm +78.953 357.723 m 90.762 369.516 l S Q +BT +11.2 0 0 11.2 69.331636 103.965722 Tm +/f-0-0 1 Tf +[<04>-1<0506>-1<02>-1<0708>]TJ +ET +BT +11.2 0 0 11.2 63.414449 92.765722 Tm +/f-0-0 1 Tf +[<02>-1<09070a0b>-1<02>-1<09>]TJ +ET +BT +11.2 0 0 11.2 30.283202 83.907836 Tm +/f-0-0 1 Tf +[<06>-1<0c0d>-1<0b>]TJ +ET +0.941176 g +28.098 452.615 158.008 -185.887 re f +0 g +1.175918 w +q 1 0 0 -1 0 483.095367 cm +28.098 30.48 158.008 185.887 re S Q +0.8 w +q 1 0 0 -1 0 483.095367 cm +137.109 53.324 m 137.109 99.387 l S Q +BT +11.2 0 0 11.2 125.794472 431.386577 Tm +/f-0-0 1 Tf +[<01>-1<0203>]TJ +ET +0.666667 g +114.16 318.041 m 114.16 314.806 111.539 312.185 108.305 312.185 c +105.066 312.185 102.445 314.806 102.445 318.041 c 102.445 321.275 +105.066 323.9 108.305 323.9 c 111.539 323.9 114.16 321.275 114.16 +318.041 c h +114.16 318.041 m f +0 g +q 1 0 0 -1 0 483.095367 cm +114.16 165.055 m 114.16 168.289 111.539 170.91 108.305 170.91 c 105.066 +170.91 102.445 168.289 102.445 165.055 c 102.445 161.82 105.066 159.195 +108.305 159.195 c 111.539 159.195 114.16 161.82 114.16 165.055 c h +114.16 165.055 m S Q +BT +11.2 0 0 11.2 105.115627 314.328197 Tm +/f-0-0 1 Tf +<0e>Tj +ET +q 1 0 0 -1 0 483.095367 cm +167.461 174.527 m 167.461 285.922 l S Q +BT +11.2 0 0 11.2 156.14778 310.184508 Tm +/f-0-0 1 Tf +[<01>-1<0203>]TJ +ET +q 1 0 0 -1 0 483.095367 cm +53.273 71.086 m 53.273 140.586 l S Q +BT +11.2 0 0 11.2 41.960955 413.624138 Tm +/f-0-0 1 Tf +[<01>-1<0203>]TJ +ET +q 1 0 0 -1 0 483.095367 cm +116.309 158.617 m 116.309 246.297 l S Q +BT +11.2 0 0 11.2 104.994472 326.095922 Tm +/f-0-0 1 Tf +[<01>-1<0203>]TJ +ET +q 1 0 0 -1 0 483.095367 cm +110.387 252.168 m 122.203 240.336 l S Q +q 1 0 0 -1 0 483.095367 cm +110.387 240.352 m 122.191 252.145 l S Q +q 1 0 0 -1 0 483.095367 cm +84.879 90.09 m 84.879 177.77 l S Q +BT +11.2 0 0 11.2 73.563314 394.623247 Tm +/f-0-0 1 Tf +[<01>-1<0203>]TJ +ET +q 1 0 0 -1 0 483.095367 cm +78.953 183.641 m 90.773 171.809 l S Q +q 1 0 0 -1 0 483.095367 cm +78.953 171.824 m 90.762 183.617 l S Q +BT +11.2 0 0 11.2 69.331636 289.865709 Tm +/f-0-0 1 Tf +[<04>-1<0506>-1<02>-1<0708>]TJ +ET +BT +11.2 0 0 11.2 63.414449 278.665709 Tm +/f-0-0 1 Tf +[<02>-1<09070a0b>-1<02>-1<09>]TJ +ET +0.666667 g +49.816 405.685 m 49.816 402.451 47.195 399.826 43.961 399.826 c 40.727 +399.826 38.102 402.451 38.102 405.685 c 38.102 408.92 40.727 411.541 +43.961 411.541 c 47.195 411.541 49.816 408.92 49.816 405.685 c h +49.816 405.685 m f +0 g +q 1 0 0 -1 0 483.095367 cm +49.816 77.41 m 49.816 80.645 47.195 83.27 43.961 83.27 c 40.727 83.27 +38.102 80.645 38.102 77.41 c 38.102 74.176 40.727 71.555 43.961 71.555 +c 47.195 71.555 49.816 74.176 49.816 77.41 c h +49.816 77.41 m S Q +BT +11.2 0 0 11.2 40.772768 401.97105 Tm +/f-0-0 1 Tf +<0f>Tj +ET +0.666667 g +134.945 423.4 m 134.945 420.162 132.324 417.541 129.09 417.541 c +125.855 417.541 123.23 420.162 123.23 423.4 c 123.23 426.634 125.855 +429.256 129.09 429.256 c 132.324 429.256 134.945 426.634 134.945 423.4 +c h +134.945 423.4 m f +0 g +q 1 0 0 -1 0 483.095367 cm +134.945 59.695 m 134.945 62.934 132.324 65.555 129.09 65.555 c 125.855 +65.555 123.23 62.934 123.23 59.695 c 123.23 56.461 125.855 53.84 129.09 +53.84 c 132.324 53.84 134.945 56.461 134.945 59.695 c h +134.945 59.695 m S Q +BT +11.2 0 0 11.2 125.901332 419.685344 Tm +/f-0-0 1 Tf +<0f>Tj +ET +0.666667 g +82.09 386.541 m 82.09 383.306 79.465 380.685 76.23 380.685 c 72.996 +380.685 70.375 383.306 70.375 386.541 c 70.375 389.775 72.996 392.4 +76.23 392.4 c 79.465 392.4 82.09 389.775 82.09 386.541 c h +82.09 386.541 m f +0 g +q 1 0 0 -1 0 483.095367 cm +82.09 96.555 m 82.09 99.789 79.465 102.41 76.23 102.41 c 72.996 102.41 +70.375 99.789 70.375 96.555 c 70.375 93.32 72.996 90.695 76.23 90.695 c +79.465 90.695 82.09 93.32 82.09 96.555 c h +82.09 96.555 m S Q +BT +11.2 0 0 11.2 73.044197 382.828197 Tm +/f-0-0 1 Tf +<10>Tj +ET +0.666667 g +165.305 302.185 m 165.305 298.951 162.68 296.326 159.445 296.326 c +156.211 296.326 153.59 298.951 153.59 302.185 c 153.59 305.42 156.211 +308.041 159.445 308.041 c 162.68 308.041 165.305 305.42 165.305 302.185 +c h +165.305 302.185 m f +0 g +q 1 0 0 -1 0 483.095367 cm +165.305 180.91 m 165.305 184.145 162.68 186.77 159.445 186.77 c 156.211 +186.77 153.59 184.145 153.59 180.91 c 153.59 177.676 156.211 175.055 +159.445 175.055 c 162.68 175.055 165.305 177.676 165.305 180.91 c h +165.305 180.91 m S Q +BT +11.2 0 0 11.2 156.258486 298.471056 Tm +/f-0-0 1 Tf +<11>Tj +ET +BT +11.2 0 0 11.2 26.997481 455.580001 Tm +/f-0-0 1 Tf +[<12>-1<0404>-1<0b>-1<13>1<0908>1<0a>-1<14>-1<08>]TJ +ET +BT +11.2 0 0 11.2 30.283202 269.790648 Tm +/f-0-0 1 Tf +[<06>-1<0c0d>-1<0b>]TJ +ET +q 1 0 0 -1 0 483.095367 cm +11.531 34.758 m 11.598 116.398 l S Q +11.59 374.697 m 8.387 377.896 l 11.598 366.697 l 14.789 377.9 l h +11.59 374.697 m f* +0.8 w +q -0.00080459 1 1 0.00080459 0 483.095367 cm +-108.408 11.503 m -105.206 8.302 l -116.408 11.504 l -105.207 14.704 l +h +-108.408 11.503 m S Q +Q Q +showpage +%%Trailer +count op_count sub {pop} repeat +countdictstack dict_count sub {end} repeat +cairo_eps_state restore +%%EOF + +%%EndDocument + @endspecial -2000 33588 26568 37 v 3674 34717 a Fs(Figur)-18 +b(e)249 b(3.)499 b FC(Object)250 b(Lifetimes)e(in)h(a)h(T)-35 +b(race)p Black -2000 37832 a(the)400 b(interpreter)-40 +b(,)401 b(should)e(the)i(guard)f(f)-10 b(ail.)400 b(This)f(means)h +(that)g(too)g(man)-15 b(y)-2000 38939 y(guard)250 b(operations)f(also)g +(consume)h(a)f(lot)g(of)g(memory)-65 b(.)-672 40046 y(In)471 +b(the)g(rest)f(of)h(the)g(paper)h(we)f(will)g(see)g(ho)-25 +b(w)471 b(this)g(trace)g(can)h(be)-2000 41153 y(optimized)250 +b(using)f(partial)g(e)-25 b(v)g(aluation.)-2000 43714 +y FA(3.)1218 b(Object)305 b(Lifetimes)h(in)f(a)f(T)-90 +b(racing)304 b(JIT)-2000 45264 y FC(T)-80 b(o)197 b(understand)h(the)g +(problems)f(that)g(this)g(paper)h(is)f(trying)g(to)g(solv)-15 +b(e)197 b(in)h(more)-2000 46371 y(detail,)261 b(we)g(\002rst)e(need)j +(to)e(understand)h(v)-25 b(arious)260 b(cases)h(of)f(object)h +(lifetimes)-2000 47478 y(that)249 b(can)h(occur)g(in)f(a)h(tracing)f +(JIT)f(compiler)-55 b(.)-672 48585 y(Figure)357 b(3)g(sho)-25 +b(ws)357 b(a)g(trace)h(before)f(optimization,)h(together)f(with)h(the) +-2000 49692 y(lifetime)326 b(of)f(v)-25 b(arious)326 +b(kinds)g(of)f(objects)h(created)h(in)f(the)g(trace.)g(It)f(is)g(e)-15 +b(x)g(e-)-2000 50799 y(cuted)254 b(from)g(top)g(to)f(bottom.)h(At)g +(the)g(bottom,)g(a)g(jump)g(is)f(used)h(to)f(e)-15 b(x)g(ecute)-2000 +51906 y(the)269 b(same)g(loop)g(another)g(time)g(\(for)f(clarity)-65 +b(,)270 b(the)f(\002gure)g(sho)-25 b(ws)268 b(tw)-10 +b(o)269 b(itera-)-2000 53013 y(tions)198 b(of)g(the)g(loop\).)g(The)h +(loop)f(is)f(e)-15 b(x)g(ecuted)200 b(until)e(one)h(of)f(the)g(guards)h +(in)f(the)-2000 54120 y(trace)246 b(f)-10 b(ails,)244 +b(and)i(the)f(e)-15 b(x)g(ecution)247 b(is)d(aborted)i(and)g +(interpretation)f(resumes.)-672 55227 y(Some)405 b(of)e(the)i +(operations)f(within)g(this)g(trace)g(are)g Fr(new)p +Black Black 405 w FC(operations,)-2000 56334 y(which)345 +b(each)f(create)h(a)f(ne)-25 b(w)345 b(instance)f(of)g(some)g(class.)f +(These)h(instances)-2000 57441 y(are)338 b(used)f(for)h(some)f(time,)h +(e.g.,)g(by)g(calling)g(methods)g(on)f(them)h(\(which)-2000 +58548 y(are)242 b(inlined)h(into)f(the)g(trace\),)h(reading)f(and)h +(writing)f(their)g(\002elds.)g(Some)g(of)-2000 59655 +y(these)321 b(instances)f Fx(escape)p FC(,)h(which)g(means)g(that)g +(the)-15 b(y)321 b(are)f(stored)h(in)f(some)-2000 60762 +y(globally)284 b(accessible)g(place)h(or)e(are)h(passed)g(into)f(a)h +(non-inlined)h(function)-2000 61869 y(via)249 b(a)h(residual)f(call.) +-672 62976 y(T)-80 b(ogether)226 b(with)f(the)g Fr(new)p +Black Black 225 w FC(operations,)g(the)h(\002gure)f(sho)-25 +b(ws)224 b(the)h(lifetimes)-2000 64082 y(of)335 b(the)h(created)h +(objects.)e(The)h(objects)g(that)f(are)h(created)h(within)f(a)f(trace) +-2000 65189 y(using)249 b Fr(new)p Black Black 249 w +FC(f)-10 b(all)249 b(into)h(one)f(of)g(se)-25 b(v)-15 +b(eral)250 b(cate)-15 b(gories:)p Black -1696 67082 a(1.)p +Black 388 w(Objects)294 b(that)f(li)-25 b(v)-15 b(e)294 +b(for)f(some)g(time,)g(and)h(are)f(then)h(just)f(not)g(used)g(an)-15 +b(y)-561 68189 y(more)249 b(afterw)-10 b(ards.)p Black +-1696 69738 a(2.)p Black 388 w(Objects)250 b(that)f(li)-25 +b(v)-15 b(e)250 b(for)e(some)i(time)f(and)h(then)f(escape.)p +Black -1696 71288 a(3.)p Black 388 w(Objects)292 b(that)g(li)-25 +b(v)-15 b(e)293 b(for)e(some)h(time,)g(survi)-25 b(v)-15 +b(e)292 b(across)g(the)g(jump)g(to)g(the)-561 72395 y(be)-15 +b(ginning)250 b(of)f(the)h(loop,)f(and)h(are)f(then)h(not)f(used)g(an) +-15 b(y)250 b(more.)p Black Black Black 27528 886 a(4.)p +Black 388 w(Objects)352 b(that)g(li)-25 b(v)-15 b(e)352 +b(for)f(some)g(time,)h(survi)-25 b(v)-15 b(e)351 b(across)h(the)f +(jump,)h(and)28663 1993 y(then)271 b(escape.)g(T)-80 +b(o)271 b(these)g(we)g(also)f(count)i(the)e(objects)h(that)g(li)-25 +b(v)-15 b(e)271 b(across)28663 3099 y(se)-25 b(v)-15 +b(eral)249 b(jumps)g(and)h(then)g(either)f(escape)h(or)f(stop)g(being)h +(used.)28552 4945 y(The)309 b(objects)g(that)g(are)h(allocated)g(in)f +(the)g(e)-15 b(xample)310 b(trace)g(in)f(Figure)g(2)27224 +6052 y(f)-10 b(all)378 b(into)g(cate)-15 b(gories)379 +b(1)g(and)f(3.)h(Objects)g(stored)f(in)g Fo(p)46971 6163 +y Fn(5)47432 6052 y FC(,)g Fo(p)48573 6163 y Fn(6)49035 +6052 y FC(,)g Fo(p)50176 6163 y Fn(11)51421 6052 y FC(are)h(in)27224 +7159 y(cate)-15 b(gory)250 b(1,)f(objects)h(in)f Fo(p)36494 +7270 y Fn(10)37361 7159 y FC(,)g Fo(p)38373 7270 y Fn(15)39489 +7159 y FC(are)h(in)f(cate)-15 b(gory)250 b(3.)28552 8266 +y(The)357 b(creation)h(of)e(objects)h(in)g(cate)-15 b(gory)358 +b(1)f(is)f(remo)-15 b(v)g(ed)358 b(by)f(the)g(opti-)27224 +9373 y(mization)310 b(described)h(in)f(Sections)g(4)g(and)g(5.)g +(Objects)h(in)f(the)g(other)g(cate-)27224 10518 y(gories)249 +b(are)g(partially)h(optimized)g(by)f(this)g(approach)h(as)f(well.)49054 +10095 y Fz(6)27224 13016 y FA(4.)1218 b(Allocation)305 +b(Remo)-12 b(v)g(al)304 b(in)h(T)-90 b(races)27224 14566 +y FC(The)284 b(main)g(insight)f(to)h(impro)-15 b(v)g(e)284 +b(the)g(code)h(sho)-25 b(wn)284 b(in)g(Section)g(2.3)g(is)f(that)27224 +15673 y(objects)258 b(in)g(cate)-15 b(gory)259 b(1)g(do)f(not)g(survi) +-25 b(v)-15 b(e)258 b(v)-15 b(ery)259 b(long)f(\226)g(the)-15 +b(y)259 b(are)f(used)g(only)27224 16780 y(inside)405 +b(the)h(loop)g(and)g(there)g(is)f(no)h(other)f(outside)h(reference)h +(to)e(them.)27224 17887 y(The)448 b(optimizer)h(identi\002es)f(objects) +h(in)f(cate)-15 b(gory)450 b(1)e(and)h(remo)-15 b(v)g(es)448 +b(the)27224 18993 y(allocation)250 b(of)f(these)g(objects,)h(and)f(all) +h(operations)f(manipulating)i(them.)28552 20100 y(This)407 +b(is)f(a)i(process)f(that)g(is)g(usually)g(called)h Fx(escape)g +(analysis)f FC([18].)27224 21207 y(In)460 b(this)f(paper)h(we)h(will)f +(perform)g(escape)h(analysis)e(by)i(using)e(partial)27224 +22314 y(e)-25 b(v)g(aluation.)386 b(The)e(use)g(of)h(partial)f(e)-25 +b(v)g(aluation)386 b(is)e(a)h(bit)f(peculiar)h(in)f(that)27224 +23421 y(it)221 b(recei)-25 b(v)-15 b(es)222 b(no)g(static)f(input)g(ar) +-18 b(guments)222 b(for)f(the)h(trace,)g(b)-20 b(ut)221 +b(it)g(is)g(only)h(used)27224 24528 y(to)344 b(optimize)i(operations)f +(within)g(the)g(trace.)g(This)f(section)h(will)g(gi)-25 +b(v)-15 b(e)345 b(an)27224 25635 y(informal)230 b(account)i(of)e(this)g +(process)h(by)f(e)-15 b(xamining)232 b(the)f(e)-15 b(xample)232 +b(trace)f(in)27224 26742 y(Figure)261 b(2.)g(The)g(\002nal)h(trace)f +(after)h(optimization)g(can)f(be)h(seen)f(in)g(Figure)h(4)27224 +27849 y(\(the)287 b(line)h(numbers)g(are)f(the)h(lines)f(of)h(the)f +(unoptimized)i(trace)f(where)g(the)27224 28956 y(operation)250 +b(originates\).)28552 30063 y(T)-80 b(o)266 b(optimize)i(the)e(trace,)h +(it)f(is)g(tra)-20 b(v)-15 b(ersed)267 b(from)f(be)-15 +b(ginning)267 b(to)f(end)h(and)27224 31170 y(an)342 b(output)g(trace)g +(is)f(produced.)h(Ev)-15 b(ery)342 b(operation)g(in)g(the)f(input)h +(trace)g(is)27224 32277 y(either)422 b(remo)-15 b(v)g(ed)422 +b(or)g(copied)h(into)e(the)i(output)f(trace.)g(Sometimes)g(ne)-25 +b(w)27224 33384 y(operations)380 b(need)g(to)g(be)g(produced)h(as)f +(well.)g(The)g(optimizer)g(can)g(only)27224 34491 y(remo)-15 +b(v)g(e)238 b(operations)f(that)h(manipulate)g(objects)g(that)f(ha)-20 +b(v)-15 b(e)238 b(been)g(allocated)27224 35598 y(within)255 +b(the)g(trace,)g(while)g(all)g(other)g(operations)f(are)h(copied)h(to)e +(the)h(output)27224 36705 y(trace)250 b(unchanged.)28552 +37812 y(Looking)221 b(at)g(the)g(e)-15 b(xample)222 b(trace)f(of)g +(Figure)f(2,)h(the)g(operations)g(in)g(lines)27224 38919 +y(1\2269)378 b(are)f(manipulating)i(objects)f(which)g(e)-15 +b(xisted)378 b(before)g(the)g(trace)g(and)27224 40026 +y(that)257 b(are)g(passed)g(in)g(as)g(ar)-18 b(guments:)258 +b(therefore)f(the)g(optimizer)h(just)e(copies)27224 41133 +y(them)249 b(into)h(the)f(output)h(trace.)28552 42240 +y(The)f(follo)-25 b(wing)250 b(operations)f(\(lines)g(10\22617\))h(are) +f(more)g(interesting:)51577 43786 y Fl(10)-22693 b Fj(p)29894 +43897 y Fi(5)30790 43786 y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o(o)o(x)o(e)o +(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)o(\))51577 44716 y Fl(12)-22694 +b Fp(s)o(e)o(t)o(\()o Fj(p)31753 44827 y Fi(5)32185 44716 +y Fp(,)463 b(i)o(n)o(t)o(v)o(a)o(l)o(,)i Fj(i)37148 44827 +y Fi(4)37579 44716 y Fp(\))51577 45645 y Fl(15)-22693 +b Fj(p)29894 45756 y Fi(6)30790 45645 y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o +(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)o(\))51577 +46575 y Fl(17)-22694 b Fp(s)o(e)o(t)o(\()o Fj(p)31753 +46686 y Fi(6)32185 46575 y Fp(,)463 b(i)o(n)o(t)o(v)o(a)o(l)o(,)g(-)o +(1)o(0)o(0)o(\))p Black Black 28552 48299 a FC(When)322 +b(the)f(optimizer)h(encounters)g(a)f Fr(new)p Black Black +FC(,)g(it)g(remo)-15 b(v)g(es)322 b(it)e(optimisti-)27224 +49405 y(cally)-65 b(,)263 b(and)f(assumes)g(that)g(the)g(object)h(is)f +(in)g(cate)-15 b(gory)263 b(1.)f(If)f(later)i(the)f(opti-)27224 +50512 y(mizer)231 b(\002nds)g(that)g(the)g(object)h(escapes,)f(it)g +(will)g(be)h(allocated)g(at)f(that)g(point.)27224 51619 +y(The)370 b(optimizer)h(needs)g(to)f(k)-10 b(eep)371 +b(track)g(of)f(the)h(state)f(of)h(the)f(object)h(that)27224 +52726 y(the)385 b(operation)g(w)-10 b(ould)385 b(ha)-20 +b(v)-15 b(e)385 b(created.)h(This)e(is)f(done)j(with)e(the)h(help)g(of) +27224 53853 y(a)f Fx(static)f(object)33044 53430 y Fz(7)33433 +53853 y FC(.)g(The)h(static)g(object)h(describes)f(the)g(shape)g(of)g +(the)g(ob-)27224 54960 y(ject)272 b(that)g(w)-10 b(ould)273 +b(ha)-20 b(v)-15 b(e)272 b(been)h(allocated,)g(i.e.,)f(the)h(type)f(of) +f(the)i(object)f(and)27224 56067 y(where)f(the)h(v)-25 +b(alues)271 b(that)g(w)-10 b(ould)272 b(be)f(stored)g(in)f(the)i +(\002elds)e(of)h(the)g(allocated)27224 57174 y(object)250 +b(come)g(from.)28552 58281 y(In)315 b(the)h(snippet)f(abo)-15 +b(v)g(e,)317 b(the)e(tw)-10 b(o)316 b Fr(new)p Black +Black 316 w FC(operations)f(are)h(remo)-15 b(v)g(ed)316 +b(and)27224 59388 y(tw)-10 b(o)297 b(static)f(objects)h(are)g +(constructed.)h(The)f Fr(set)p Black Black 296 w FC(operations)g +(manipulate)27224 60495 y(static)251 b(objects,)g(therefore)h(the)-15 +b(y)252 b(can)f(be)h(remo)-15 b(v)g(ed)252 b(as)f(well;)h(their)f(ef) +-25 b(fect)251 b(is)27224 61602 y(remembered)f(in)f(the)h(static)f +(objects.)28552 62709 y(After)299 b(the)h(operations)f(the)h(static)f +(object)h(associated)f(with)h Fo(p)50550 62820 y Fn(5)51310 +62709 y FC(w)-10 b(ould)27224 63816 y(store)418 b(the)i(kno)-25 +b(wledge)420 b(that)f(it)g(is)f(a)i Fr(BoxedInteger)p +Black Black 420 w FC(whose)f Fr(intval)p Black Black +27224 64923 a FC(\002eld)255 b(contains)h Fo(i)33176 +65034 y Fn(4)33637 64923 y FC(;)g(the)f(one)h(associated)g(with)f +Fo(p)44226 65034 y Fn(6)44943 64923 y FC(w)-10 b(ould)255 +b(store)g(that)h(it)f(is)f(a)27224 66030 y Fr(BoxedInteger)p +Black Black 250 w FC(whose)c Fr(intval)p Black Black +249 w FC(\002eld)g(contains)f(the)h(constant)f(-100.)p +Black 27224 67115 13284 37 v 27224 67799 a Fz(6)27722 +68111 y Fw(W)-71 b(e)202 b(also)g(started)h(to)f(w)-9 +b(ork)202 b(on)f(optimizing)h(objects)h(in)e(cate)-13 +b(gory)202 b(3,)g(which)g(will)h(be)27224 69108 y(the)221 +b(subject)h(of)g(a)f(later)h(paper)-49 b(.)27224 70090 +y Fz(7)27722 70403 y Fw(Here)338 b(\223static\224)h(is)g(meant)e(in)h +(the)g(sense)g(of)f(partial)i(e)-22 b(v)g(aluation,)337 +b(i.e.,)j(kno)-22 b(wn)336 b(at)27224 71399 y(partial)378 +b(e)-22 b(v)g(aluation)378 b(time,)h(not)e(in)h(the)g(sense)h(of)f +(\223static)h(allocation\224)g(or)e(\223static)27224 +72395 y(method\224.)p Black Black Black eop end +%%Page: 5 5 +TeXDict begin 5 4 bop Black Black -672 886 a FC(The)287 +b(subsequent)g(operations)f(\(line)h(20\22626\))f(in)h(Figure)f(2,)h +(which)g(use)-2000 1993 y Fo(p)-1486 2104 y Fn(5)-776 +1993 y FC(and)250 b Fo(p)1426 2104 y Fn(6)1887 1993 y +FC(,)f(can)h(then)g(be)f(optimized)h(using)f(that)h(kno)-25 +b(wledge:)22353 3438 y Fl(20)-22694 b Fp(g)o(u)o(a)o(r)o(d)2538 +3255 y(_)3003 3438 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)6249 +3549 y Fi(5)6680 3438 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)o(\))22353 4368 y Fl(22)-22693 b Fj(i)527 +4479 y Fi(7)1423 4368 y Fp(=)464 b(g)o(e)o(t)o(\()p Fj(p)4670 +4479 y Fi(5)5101 4368 y Fp(,)g(i)o(n)o(t)o(v)o(a)o(l)o(\))22353 +5298 y Fl(23)-22694 b Fp(g)o(u)o(a)o(r)o(d)2538 5115 +y(_)3003 5298 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)6249 5409 +y Fi(6)6680 5298 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o +(e)o(r)o(\))22353 6227 y Fl(25)-22693 b Fj(i)527 6338 +y Fi(8)1423 6227 y Fp(=)464 b(g)o(e)o(t)o(\()p Fj(p)4670 +6338 y Fi(6)5101 6227 y Fp(,)g(i)o(n)o(t)o(v)o(a)o(l)o(\))22353 +7157 y Fl(26)-22693 b Fj(i)527 7268 y Fi(9)1423 7157 +y Fp(=)464 b(i)o(n)o(t)3748 6974 y(_)4213 7157 y(a)o(d)o(d)o(\()o +Fj(i)6386 7268 y Fi(7)6818 7157 y Fp(,)g Fj(i)8061 7268 +y Fi(8)8493 7157 y Fp(\))p Black Black -672 8780 a FC(The)336 +b Fr(guard)3878 8571 y(_)4411 8780 y(class)p Black Black +338 w FC(operations)g(can)h(be)f(remo)-15 b(v)g(ed,)337 +b(since)f(their)g(ar)-20 b(-)-2000 9887 y(guments)249 +b(are)g(static)g(objects)g(with)g(the)h(matching)f(type)h +Fr(BoxedInteger)p Black Black 1 w FC(.)-2000 10994 y(The)263 +b Fr(get)p Black Black 264 w FC(operations)g(can)h(be)f(remo)-15 +b(v)g(ed)265 b(as)e(well,)g(because)h(each)h(of)e(them)-2000 +12101 y(reads)288 b(a)g(\002eld)g(out)g(of)g(a)g(static)g(object.)g +(The)g(results)f(of)h(the)g(get)g(operation)-2000 13208 +y(are)227 b(replaced)h(with)e(what)h(the)g(static)g(object)g(stores)f +(in)g(these)h(\002elds:)f(all)h(the)-2000 14315 y(occurences)213 +b(of)g Fo(i)4030 14426 y Fn(7)4703 14315 y FC(and)g Fo(i)6704 +14426 y Fn(8)7378 14315 y FC(in)f(the)g(trace)h(are)g(just)f(replaced)h +(by)f Fo(i)20167 14426 y Fn(4)20841 14315 y FC(and)h(-100.)-2000 +15422 y(The)249 b(only)h(operation)g(copied)g(into)f(the)g(optimized)h +(trace)g(is)f(the)g(addition:)22353 16867 y Fl(26)-22693 +b Fj(i)527 16978 y Fi(9)1423 16867 y Fp(=)464 b(i)o(n)o(t)3748 +16684 y(_)4213 16867 y(a)o(d)o(d)o(\()o Fj(i)6386 16978 +y Fi(4)6818 16867 y Fp(,)e(-)o(1)o(0)o(0)o(\))p Black +Black -672 18489 a FC(The)204 b(rest)f(of)h(the)g(trace)g(from)g +(Figure)g(2)g(is)f(optimized)h(in)g(a)g(similar)f(v)-15 +b(ein.)-2000 19596 y(The)262 b(operations)g(in)f(lines)h(27\22635)g +(produce)g(tw)-10 b(o)262 b(more)g(static)g(objects)f(and)-2000 +20703 y(are)348 b(remo)-15 b(v)g(ed.)349 b(Those)e(in)h(line)g +(36\22639)h(are)f(just)f(copied)i(into)f(the)g(output)-2000 +21810 y(trace)274 b(because)g(the)-15 b(y)274 b(manipulate)h(objects)e +(that)h(are)f(allocated)i(before)f(the)-2000 22917 y(trace.)369 +b(Lines)e(40\22642)i(are)f(remo)-15 b(v)g(ed)369 b(because)g(the)-15 +b(y)368 b(operate)h(on)f(a)g(static)-2000 24024 y(object.)213 +b(Line)e(43)h(is)f(copied)i(into)f(the)g(output)g(trace.)g(Lines)g +(44\22646)g(produce)-2000 25131 y(a)330 b(ne)-25 b(w)331 +b(static)f(object)g(and)g(are)h(remo)-15 b(v)g(ed,)331 +b(lines)e(48\22651)i(manipulate)g(that)-2000 26238 y(static)272 +b(object)i(and)f(are)g(remo)-15 b(v)g(ed)273 b(as)f(well.)h(Lines)f +(52\22654)i(are)e(copied)i(into)-2000 27345 y(the)249 +b(output)h(trace.)-672 28452 y(The)g(last)f(operation)h(\(line)f(55\))h +(is)e(an)i(interesting)f(case.)h(It)f(is)g(the)h Fr(jump)p +Black Black -2000 29559 a FC(operation)387 b(that)h(passes)e(control)h +(back)g(to)g(the)g(be)-15 b(ginning)388 b(of)e(the)h(trace.)-2000 +30666 y(The)257 b(tw)-10 b(o)257 b(ar)-18 b(guments)258 +b(to)f(this)f(operation)i(at)f(this)f(point)h(are)h(static)f(objects.) +-2000 31773 y(Ho)-25 b(we)g(v)-15 b(er)-40 b(,)223 b(because)g(the)-15 +b(y)222 b(are)g(passed)g(into)g(the)g(ne)-15 b(xt)222 +b(iteration)g(of)g(the)g(loop)-2000 32880 y(the)-15 b(y)320 +b(li)-25 b(v)-15 b(e)319 b(longer)g(than)h(the)f(trace)h(and)f +(therefore)h(cannot)f(remain)h(static.)-2000 33987 y(The)-15 +b(y)307 b(need)g(to)g(be)g(turned)g(into)f(dynamic)i(\(runtime\))e +(objects)h(before)g(the)-2000 35094 y(actual)238 b Fr(jump)p +Black Black 237 w FC(operation.)g(This)e(process)g(of)h(turning)f(a)i +(static)e(object)i(into)f(a)-2000 36201 y(dynamic)250 +b(one)g(is)e(called)i Fx(lifting)p FC(.)-672 37308 y(Lifting)427 +b(a)h(static)g(object)g(puts)f Fr(new)p Black Black 428 +w FC(and)h Fr(set)p Black Black 428 w FC(operations)g(into)g(the)-2000 +38415 y(output)373 b(trace.)g(Those)g(operations)g(produce)g(an)g +(object)h(at)f(runtime)g(that)-2000 39522 y(has)335 b(the)g(shape)g +(described)h(by)f(the)g(static)f(object.)i(This)e(process)h(is)f(a)h +(bit)-2000 40629 y(delicate,)350 b(because)f(the)g(static)f(objects)h +(could)g(form)f(an)g(arbitrary)h(graph)-2000 41736 y(structure.)249 +b(In)g(our)g(e)-15 b(xample)250 b(it)f(is)g(simple,)g(though:)22353 +43181 y Fl(44)-22693 b Fj(p)670 43292 y Fi(15)1943 43181 +y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o +(e)o(r)o(\))22353 44111 y Fl(46)-22694 b Fp(s)o(e)o(t)o(\()p +Fj(p)2530 44222 y Fi(15)3338 44111 y Fp(,)463 b(i)o(n)o(t)o(v)o(a)o(l)o +(,)i Fj(i)8301 44222 y Fi(14)9109 44111 y Fp(\))22353 +45041 y Fl(27)-22693 b Fj(p)670 45152 y Fi(10)1943 45041 +y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o +(e)o(r)o(\))22353 45970 y Fl(29)-22694 b Fp(s)o(e)o(t)o(\()p +Fj(p)2530 46081 y Fi(10)3338 45970 y Fp(,)463 b(i)o(n)o(t)o(v)o(a)o(l)o +(,)i Fj(i)8301 46081 y Fi(9)8732 45970 y Fp(\))22353 +46900 y Fl(55)-22694 b Fp(j)o(u)o(m)o(p)o(\()p Fj(p)2995 +47011 y Fi(15)3803 46900 y Fp(,)464 b Fj(p)5189 47011 +y Fi(10)5998 46900 y Fp(\))p Black Black -672 48523 a +FC(Observ)-15 b(e)416 b(ho)-25 b(w)416 b(the)f(operations)h(for)f +(creating)h(these)f(tw)-10 b(o)416 b(instances)-2000 +49630 y(ha)-20 b(v)-15 b(e)333 b(been)g(mo)-15 b(v)g(ed)332 +b(to)g(a)g(later)g(point)g(in)g(the)g(trace.)h(This)e(is)g(w)-10 +b(orthwhile)-2000 50737 y(e)-25 b(v)-15 b(en)381 b(though)f(the)g +(objects)g(ha)-20 b(v)-15 b(e)381 b(to)f(be)g(allocated)h(in)f(the)g +(end)g(because)-2000 51844 y(some)368 b Fr(get)p Black +Black 368 w FC(operations)h(and)g Fr(guard)11429 51635 +y(_)11962 51844 y(class)p Black Black 369 w FC(operations)f(on)h(the)f +(lifted)-2000 52951 y(static)249 b(objects)h(could)f(be)h(remo)-15 +b(v)g(ed.)-672 54058 y(More)311 b(generally)-65 b(,)312 +b(lifting)f(needs)g(to)g(occur)g(if)f(a)h(static)g(object)h(is)e(used) +-2000 55165 y(in)281 b(an)-15 b(y)281 b(operation)h(apart)f(from)g +Fr(get)p Black Black FC(,)g Fr(set)p Black Black 1 w +FC(,)f(and)i Fr(guard)p Black Black FC(.)f(It)g(also)g(needs)g(to)-2000 +56272 y(occur)250 b(if)f Fr(set)p Black Black 249 w FC(is)f(used)i(to)f +(store)g(a)g(static)g(object)h(into)f(a)h(non-static)f(one.)-672 +57378 y(The)198 b(\002nal)g(optimized)h(trace)f(of)g(the)g(e)-15 +b(xample)199 b(can)g(be)f(seen)g(in)g(Figure)g(4.)-2000 +58485 y(The)343 b(optimized)h(trace)f(contains)h(only)f(tw)-10 +b(o)343 b(allocations,)h(instead)f(of)g(the)-2000 59592 +y(original)c(\002)-25 b(v)-15 b(e,)339 b(and)h(only)f(three)g +Fr(guard)12363 59383 y(_)12896 59592 y(class)p Black +Black 340 w FC(operations,)g(compared)-2000 60699 y(to)249 +b(the)h(original)f(se)-25 b(v)-15 b(en.)-2000 63097 y +FA(5.)1218 b(F)-30 b(ormal)304 b(Description)g(of)h(the)g(Algorithm) +-2000 64646 y FC(In)203 b(this)f(section)h(we)h(w)-10 +b(ant)203 b(to)g(gi)-25 b(v)-15 b(e)204 b(a)f(formal)g(description)g +(of)g(the)g(semantics)-2000 65753 y(of)404 b(the)g(traces)g(and)g(of)g +(the)g(optimizer)g(and)h(lik)-10 b(en)404 b(the)g(optimization)h(to) +-2000 66860 y(partial)429 b(e)-25 b(v)g(aluation.)430 +b(W)-80 b(e)429 b(focus)g(on)f(the)h(operations)g(for)f(manipulating) +-2000 67967 y(heap)291 b(allocated)g(objects,)g(as)e(those)h(are)h(the) +f(only)g(ones)g(that)h(are)f(actually)-2000 69074 y(optimized.)197 +b(W)-80 b(e)197 b(also)e(consider)h(only)h(objects)f(with)g(tw)-10 +b(o)196 b(\002elds)g Fo(L)f FC(and)h Fo(R)203 b FC(in)-2000 +70181 y(this)190 b(section,)h(generalizing)h(to)e(arbitrary)h(man)-15 +b(y)191 b(\002elds)f(is)g(straightforw)-10 b(ard.)-2000 +71288 y(T)-35 b(races)328 b(are)g(lists)e(of)i(operations.)g(The)g +(operations)g(considered)g(here)h(are)-2000 72395 y Fr(new)p +Black Black FC(,)249 b Fr(get)p Black Black 1 w FC(,)g +Fr(set)p Black Black 249 w FC(and)h Fr(guard)8396 72186 +y(_)8929 72395 y(class)p Black Black 1 w FC(.)p Black +Black Black 51577 1315 a Fl(1)p 0.3 0.3 0.3 TeXcolorrgb +-24631 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 464 w(a)o(r)o(g)o(u)o(m)o(e)o(n)o(t)o(s)p +Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +463 w(t)o(o)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 464 w(t)o(h)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 463 w(t)o(r)o(a)o(c)o(e)p +Black 0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 465 w Fj(p)39768 1426 y Fi(0)p 0.3 0.3 0.3 +TeXcolorrgb 40199 1315 a Fk(,)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 465 w Fj(p)41586 1426 y Fi(1)p 0.3 0.3 0.3 TeXcolorrgb +Black 51577 2245 a Fl(3)-24631 b Fp(g)o(u)o(a)o(r)o(d)29548 +2062 y(_)30013 2245 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 +2356 y Fi(1)33690 2245 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)n(\))51577 3175 y Fl(5)-24630 b Fj(i)27537 +3286 y Fi(2)28433 3175 y Fp(=)464 b(g)o(e)o(t)o(\()o +Fj(p)31679 3286 y Fi(1)32111 3175 y Fp(,)f(i)o(n)o(t)o(v)o(a)o(l)o(\)) +51577 4105 y Fl(6)-24631 b Fp(g)o(u)o(a)o(r)o(d)29548 +3922 y(_)30013 4105 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 +4216 y Fi(0)33690 4105 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)n(\))51577 5034 y Fl(8)-24630 b Fj(i)27537 +5145 y Fi(3)28433 5034 y Fp(=)464 b(g)o(e)o(t)o(\()o +Fj(p)31679 5145 y Fi(0)32111 5034 y Fp(,)f(i)o(n)o(t)o(v)o(a)o(l)o(\)) +51577 5964 y Fl(9)-24630 b Fj(i)27537 6075 y Fi(4)28433 +5964 y Fp(=)463 b(i)o(n)o(t)30757 5781 y(_)31222 5964 +y(a)o(d)o(d)o(\()p Fj(i)33396 6075 y Fi(2)33827 5964 +y Fp(,)i Fj(i)35071 6075 y Fi(3)35503 5964 y Fp(\))51577 +6894 y Fl(26)-24907 b Fj(i)27537 7005 y Fi(9)28433 6894 +y Fp(=)463 b(i)o(n)o(t)30757 6711 y(_)31222 6894 y(a)o(d)o(d)o(\()p +Fj(i)33396 7005 y Fi(4)33827 6894 y Fp(,)g(-)o(1)o(0)o(0)o(\))51577 +8754 y Fl(37)-24908 b Fp(g)o(u)o(a)o(r)o(d)29548 8571 +y(_)30013 8754 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 +8865 y Fi(0)33690 8754 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o +(g)o(e)o(r)n(\))51577 9684 y Fl(39)-24907 b Fj(i)27537 +9795 y Fi(12)28810 9684 y Fp(=)463 b(g)o(e)o(t)o(\()p +Fj(p)32056 9795 y Fi(0)32487 9684 y Fp(,)h(i)o(n)o(t)o(v)o(a)o(l)o(\)) +51577 10614 y Fl(43)-24907 b Fj(i)27537 10725 y Fi(14)28810 +10614 y Fp(=)463 b(i)o(n)o(t)31134 10431 y(_)31599 10614 +y(a)o(d)o(d)o(\()p Fj(i)33773 10725 y Fi(12)34581 10614 +y Fp(,)g(-)o(1)o(\))51577 12473 y Fl(52)-24907 b Fj(i)27537 +12584 y Fi(17)28810 12473 y Fp(=)463 b(i)o(n)o(t)31134 +12290 y(_)31599 12473 y(g)o(t)o(\()p Fj(i)33308 12584 +y Fi(14)34116 12473 y Fp(,)h(0)o(\))51577 13403 y Fl(54)-24908 +b Fp(g)o(u)o(a)o(r)o(d)29548 13220 y(_)30013 13403 y(t)o(r)o(u)o(e)n +(\()p Fj(i)32651 13514 y Fi(17)33459 13403 y Fp(\))51577 +15263 y Fl(44)h Fj(p)27680 15374 y Fi(15)28953 15263 +y Fp(=)463 b(n)o(e)o(w\()n(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o +(r)o(\))51577 16193 y Fl(46)-24908 b Fp(s)o(e)o(t)o(\()o +Fj(p)29539 16304 y Fi(15)30347 16193 y Fp(,)464 b(i)o(n)o(t)o(v)o(a)o +(l)o(,)h Fj(i)35311 16304 y Fi(14)36119 16193 y Fp(\))51577 +17123 y Fl(27)-24907 b Fj(p)27680 17234 y Fi(10)28953 +17123 y Fp(=)463 b(n)o(e)o(w\()n(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g) +o(e)o(r)o(\))51577 18052 y Fl(29)-24908 b Fp(s)o(e)o(t)o(\()o +Fj(p)29539 18163 y Fi(10)30347 18052 y Fp(,)464 b(i)o(n)o(t)o(v)o(a)o +(l)o(,)h Fj(i)35311 18163 y Fi(9)35742 18052 y Fp(\))51577 +19912 y Fl(55)-24908 b Fp(j)o(u)o(m)o(p)o(\()o Fj(p)30004 +20023 y Fi(15)30812 19912 y Fp(,)465 b Fj(p)32199 20023 +y Fi(10)33008 19912 y Fp(\))p Black Black 27224 20892 +26568 37 v 29901 22021 a Fs(Figur)-18 b(e)249 b(4.)499 +b FC(Resulting)248 b(T)-35 b(race)250 b(After)f(Allocation)h(Remo)-15 +b(v)-25 b(al)p Black 28552 25493 a(The)437 b(v)-25 b(alues)437 +b(of)g(all)g(v)-25 b(ariables)437 b(are)g(locations)g(\(i.e.,)g +(pointers\).)f(Lo-)27224 26600 y(cations)422 b(are)g(mapped)i(to)e +(objects,)g(which)h(are)f(represented)h(by)f(triples)27224 +27706 y Fh(\()p Fo(T)85 b(;)172 b(l)29066 27817 y Fn(1)29526 +27706 y Fo(;)g(l)30288 27817 y Fn(2)30748 27706 y Fh(\))234 +b FC(of)e(a)i(type)f Fo(T)142 b FC(,)234 b(and)f(tw)-10 +b(o)234 b(locations)f(that)g(represent)h(the)f(\002elds)g(of)27224 +28813 y(the)310 b(object.)h(When)f(a)g(ne)-25 b(w)311 +b(object)g(is)e(created,)i(the)f(\002elds)g(are)g(initialized)27224 +29920 y(to)243 b(null,)h(b)-20 b(ut)244 b(we)g(require)g(that)g(the)-15 +b(y)245 b(are)f(initialized)g(to)g(a)g(real)g(location)h(be-)27224 +31027 y(fore)252 b(being)h(read,)g(otherwise)g(the)g(trace)g(is)f +(malformed)g(\(this)g(condition)h(is)27224 32134 y(guaranteed)d(by)g +(ho)-25 b(w)249 b(the)h(traces)f(are)h(generated)g(in)f(PyPy\).)28552 +33241 y(W)-80 b(e)234 b(use)f(some)h(abbre)-25 b(viations)234 +b(when)f(dealing)i(with)e(object)h(triples.)f(T)-80 b(o)27224 +34348 y(read)279 b(the)g(type)g(of)f(an)h(object,)g Fh(t)-28 +b(yp)28 b(e)q(\(\()p Fo(T)85 b(;)172 b(l)42199 34459 +y Fn(1)42660 34348 y Fo(;)f(l)43421 34459 y Fn(2)43882 +34348 y Fh(\)\))341 b(=)g Fo(T)421 b FC(is)278 b(used.)g(Reading)27224 +35455 y(a)335 b(\002eld)h Fo(F)477 b FC(from)335 b(an)h(object)g(is)f +(written)h Fh(\()p Fo(T)85 b(;)172 b(l)43557 35566 y +Fn(1)44017 35455 y Fo(;)f(l)44778 35566 y Fn(2)45239 +35455 y Fh(\))45637 35566 y FD(F)46650 35455 y FC(which)336 +b(either)g(is)f Fo(l)53330 35566 y Fn(1)27224 36562 y +FC(if)314 b Fo(F)550 b Fh(=)408 b Fo(L)314 b FC(or)g +Fo(l)33016 36673 y Fn(2)33791 36562 y FC(if)g Fo(F)550 +b Fh(=)409 b Fo(R)7 b FC(.)314 b(T)-80 b(o)314 b(set)g(\002eld)g +Fo(F)456 b FC(to)314 b(a)h(ne)-25 b(w)315 b(location)g +Fo(l)17 b FC(,)313 b(we)27224 37669 y(use)h(the)g(notation)h +Fh(\()p Fo(T)85 b(;)172 b(l)35819 37780 y Fn(1)36280 +37669 y Fo(;)f(l)37041 37780 y Fn(2)37502 37669 y Fh(\)!)38184 +37780 y FD(F)38862 37669 y Fo(l)17 b FC(,)313 b(which)i(yields)f(a)h +(ne)-25 b(w)315 b(triple)f Fh(\()p Fo(T)85 b(;)172 b(l)17 +b(;)170 b(l)52932 37780 y Fn(2)53393 37669 y Fh(\))27224 +38776 y FC(if)249 b Fo(F)426 b Fh(=)285 b Fo(L)249 b +FC(or)g(a)g(ne)-25 b(w)250 b(triple)f Fh(\()p Fo(T)85 +b(;)172 b(l)39037 38887 y Fn(1)39498 38776 y Fo(;)f(l)17 +b Fh(\))248 b FC(if)h Fo(F)426 b Fh(=)285 b Fo(R)7 b +FC(.)28552 39883 y(Figure)292 b(5)g(sho)-25 b(ws)291 +b(the)h(operational)g(semantics)g(for)f(traces.)h(The)g(inter)-20 +b(-)27224 40990 y(preter)333 b(formalized)h(there)f(e)-15 +b(x)g(ecutes)335 b(one)e(operation)h(at)g(a)f(time.)g(Its)g(state)27224 +42097 y(is)294 b(represented)i(by)f(an)h(en)-40 b(vironment)296 +b Fo(E)352 b FC(and)296 b(a)f(heap)h Fo(H)79 b FC(,)295 +b(which)h(may)f(be)27224 43204 y(changed)377 b(by)f(the)g(e)-15 +b(x)g(ecution)377 b(of)e(an)h(operation.)h(The)f(en)-40 +b(vironment)376 b(is)f(a)27224 44311 y(partial)239 b(function)h(from)f +(v)-25 b(ariables)240 b(to)f(locations)h(and)f(the)h(heap)g(is)f(a)g +(partial)27224 45418 y(function)326 b(from)f(locations)g(to)h(objects.) +f(Note)h(that)g(a)g(v)-25 b(ariable)326 b(can)g(ne)-25 +b(v)-15 b(er)27224 46525 y(be)361 b(null)f(in)h(the)g(en)-40 +b(vironment,)361 b(otherwise)g(the)g(trace)g(w)-10 b(ould)361 +b(ha)-20 b(v)-15 b(e)362 b(been)27224 47632 y(malformed.)351 +b(The)h(en)-40 b(vironment)352 b(could)g(not)f(directly)h(map)f(v)-25 +b(ariables)352 b(to)27224 48739 y(objects,)256 b(because)i(se)-25 +b(v)-15 b(eral)257 b(v)-25 b(ariables)256 b(can)h(point)g(to)f(the)h +Fx(same)f FC(object,)i(be-)27224 49846 y(cause)249 b(of)g(aliasing.) +28552 50953 y(W)-80 b(e)388 b(use)f(the)h(follo)-25 b(wing)387 +b(notation)h(for)e(updating)i(partial)g(functions:)27224 +52060 y Fo(E)57 b Fh([)p Fo(v)397 b Fg(7!)361 b Fo(l)17 +b Fh(])288 b FC(denotes)i(the)g(en)-40 b(vironment)290 +b(which)g(is)f(just)f(lik)-10 b(e)290 b Fo(E)57 b FC(,)289 +b(b)-20 b(ut)290 b(maps)27224 53167 y Fo(v)285 b FC(to)249 +b Fo(l)17 b FC(.)28552 54274 y(The)344 b Fr(new)p Black +Black 343 w FC(operation)h(creates)f(a)f(ne)-25 b(w)345 +b(object)f Fh(\()p Fo(T)85 b(;)172 b Fh(n)-28 b(ull)p +Fo(;)171 b Fh(n)-28 b(ull\))344 b FC(on)f(the)27224 55380 +y(heap)382 b(under)g(a)g(fresh)f(location)i Fo(l)398 +b FC(and)382 b(adds)g(the)g(result)f(v)-25 b(ariable)383 +b(to)e(the)27224 56487 y(en)-40 b(vironment,)250 b(mapping)g(it)f(to)g +(the)h(ne)-25 b(w)250 b(location)g Fo(l)17 b FC(.)28552 +57594 y(The)232 b Fr(get)p Black Black 231 w FC(operation)g(reads)f(a)h +(\002eld)g Fo(F)373 b FC(out)231 b(of)g(an)h(object,)g(and)g(adds)f +(the)27224 58701 y(result)272 b(v)-25 b(ariable)273 b(to)g(the)g(en)-40 +b(vironment,)274 b(mapping)f(it)f(to)h(the)g(read)g(location.)27224 +59808 y(The)249 b(heap)h(is)f(unchanged.)28552 60915 +y(The)362 b Fr(set)p Black Black 361 w FC(operation)g(changes)h +(\002eld)e Fo(F)504 b FC(of)361 b(an)h(object)g(stored)f(at)g(the)27224 +62022 y(location)308 b(that)h(v)-25 b(ariable)308 b Fo(v)344 +b FC(maps)308 b(to.)g(The)g(ne)-25 b(w)308 b(v)-25 b(alue)309 +b(of)f(the)g(\002eld)g(is)f(the)27224 63129 y(location)250 +b(in)f(v)-25 b(ariable)250 b Fo(u)p FC(.)f(The)h(en)-40 +b(vironment)250 b(is)e(unchanged.)28552 64236 y(The)302 +b Fr(guard)33068 64027 y(_)33601 64236 y(class)p Black +Black 304 w FC(operation)h(is)e(used)i(to)f(check)h(whether)g(the)g +(ob-)27224 65343 y(ject)329 b(stored)f(at)g(the)h(location)g(that)g(v) +-25 b(ariable)329 b Fo(v)364 b FC(maps)328 b(to)h(is)f(of)g(type)g +Fo(T)142 b FC(.)329 b(If)27224 66450 y(that)364 b(is)f(the)h(case,)g +(then)g(e)-15 b(x)g(ecution)365 b(continues)f(without)g(changing)h +(heap)27224 67557 y(and)249 b(en)-40 b(vironment.)251 +b(Otherwise,)e(e)-15 b(x)g(ecution)251 b(is)e(stopped.)27224 +69738 y Fs(5.1)996 b(Optimizing)249 b(T)-74 b(races)27224 +71288 y FC(T)-80 b(o)397 b(optimize)i(the)f(simple)f(traces)h(of)f(the) +h(last)g(section,)g(we)g(use)f(online)27224 72395 y(partial)406 +b(e)-25 b(v)g(aluation.)408 b(The)f(partial)g(e)-25 b(v)g(aluator)407 +b(optimizes)g(one)g(operation)p Black Black eop end +%%Page: 6 6 +TeXDict begin 6 5 bop Black Black Black Black Black 325 +1393 a Fx(ne)-15 b(w)14699 748 y Fo(l)187 b Fh(fresh)p +4107 1165 23818 43 v 4107 2258 a Fo(v)321 b Fh(=)284 +b Ff(new)q Fh(\()p Fo(T)142 b Fh(\))p Fo(;)172 b(E)57 +b(;)171 b(H)12257 1687 y Fn(run)12041 2258 y Fh(=)-171 +b Fg(\))286 b Fo(E)227 b Fh([)q Fo(v)320 b Fg(7!)285 +b Fo(l)17 b Fh(])170 b Fo(;)h(H)250 b Fh([)p Fo(l)301 +b Fg(7!)284 b Fh(\()p Fo(T)85 b(;)172 b Fh(n)-28 b(ull)p +Fo(;)171 b Fh(n)-28 b(ull\))q(])32192 1393 y Fx(guar)-37 +b(d)39514 747 y Fh(t)-28 b(yp)28 b(e)q(\()p Fo(H)79 b +Fh(\()p Fo(E)57 b Fh(\()p Fo(v)36 b Fh(\)\))286 b(=)e +Fo(T)p 36023 1165 15310 43 v 36023 2258 a Ff(guard)p +FC(_)q Ff(class)p Fh(\()p Fo(v)36 b(;)171 b(T)142 b Fh(\))p +Fo(;)172 b(E)57 b(;)171 b(H)47423 1687 y Fn(run)47207 +2258 y Fh(=)-171 b Fg(\))285 b Fo(E)57 b(;)171 b(H)325 +6106 y Fx(g)-10 b(et)p 5307 5878 21419 43 v 5307 6971 +a Fo(u)284 b Fh(=)h Ff(get)p Fh(\()p Fo(v)36 b(;)172 +b(F)142 b Fh(\))p Fo(;)171 b(E)57 b(;)171 b(H)14558 6401 +y Fn(run)14342 6971 y Fh(=)-171 b Fg(\))285 b Fo(E)17257 +6166 y Fe(\002)17683 6971 y Fo(u)g Fg(7!)f Fo(H)250 b +Fh(\()p Fo(E)228 b Fh(\()q Fo(v)36 b Fh(\)\))24069 7275 +y FD(F)24747 6166 y Fe(\003)25344 6971 y Fo(;)171 b(H)39514 +5461 y Fh(t)-28 b(yp)28 b(e)q(\()p Fo(H)79 b Fh(\()p +Fo(E)57 b Fh(\()p Fo(v)36 b Fh(\)\))286 b Fg(6)p Fh(=)e +Fo(T)p 36095 5878 15166 43 v 36095 6971 a Ff(guard)p +FC(_)p Ff(class)q Fh(\()p Fo(v)36 b(;)171 b(T)142 b Fh(\))p +Fo(;)172 b(E)57 b(;)171 b(H)47495 6401 y Fn(run)47279 +6971 y Fh(=)-171 b Fg(\))285 b(?)p Fo(;)171 b Fg(?)325 +10202 y Fx(set)p 3377 9974 25279 43 v 3377 11067 a Ff(set)f +Fh(\()q Fo(v)36 b(;)171 b(F)28 b(;)171 b(u)p Fh(\))g +Fo(;)g(E)57 b(;)172 b(H)11945 10497 y Fn(run)11729 11067 +y Fh(=)-171 b Fg(\))285 b Fo(E)57 b(;)172 b(H)249 b Fh([)q +Fo(E)228 b Fh(\()p Fo(v)36 b Fh(\))285 b Fg(7!)f Fh(\()p +Fo(H)250 b Fh(\()p Fo(E)228 b Fh(\()p Fo(v)36 b Fh(\))q(\)!)25099 +11178 y FD(F)25777 11067 y Fo(E)57 b Fh(\()p Fo(u)p Fh(\)\))q(])-2000 +16080 y Fx(Object)250 b(Domains:)2170 17723 y Fo(u;)171 +b(v)235 b Fg(2)199 b Fo(V)2809 b Fh(v)-57 b(ariables)250 +b(in)g(trace)3008 18830 y Fo(T)341 b Fg(2)199 b Fd(T)2740 +b Fh(run)-28 b(time)249 b(t)-28 b(yp)28 b(es)2951 19937 +y Fo(F)341 b Fg(2)199 b(f)p Fo(L;)171 b(R)7 b Fg(g)448 +b Fh(\014elds)250 b(of)327 b(ob)57 b(jects)3425 21044 +y Fo(l)215 b Fg(2)199 b Fo(L)2709 b Fh(lo)28 b(cations)250 +b(on)f(heap)20296 16272 y Fx(Semantic)i(V)-111 b(alues:)20512 +17723 y Fo(E)256 b Fg(2)199 b Fo(V)512 b(*)285 b(L)13458 +b Fh(En)-28 b(vironmen)g(t)20396 18830 y Fo(H)278 b Fg(2)199 +b Fo(L)284 b(*)h Fd(T)228 b Fg(\002)g Fh(\()p Fo(L)f +Fg([)g(f)p Fh(n)-28 b(ull)p Fg(g)p Fh(\))228 b Fg(\002)g +Fh(\()p Fo(L)g Fg([)f(f)p Fh(n)-28 b(ull)p Fg(g)p Fh(\))448 +b(Heap)p -2000 21855 55791 37 v 14148 22984 a Fs(Figur)-18 +b(e)250 b(5.)498 b FC(The)249 b(Operational)i(Semantics)e(of)g +(Simpli\002ed)h(T)-35 b(races)p Black -2000 26299 a(of)373 +b(a)g(trace)h(at)f(a)h(time.)f(Ev)-15 b(ery)373 b(operation)h(in)f(the) +h(unoptimized)g(trace)g(is)-2000 27406 y(replaced)313 +b(by)f(a)g(list)f(of)g(operations)h(in)g(the)g(optimized)g(trace.)h +(This)e(list)g(is)-2000 28513 y(empty)371 b(if)f(the)g(operation)h(can) +g(be)g(optimized)g(a)-15 b(w)-10 b(ay)-65 b(.)372 b(The)e(optimization) +-2000 29620 y(rules)481 b(can)g(be)h(seen)f(in)g(Figure)g(6.)g(Lists)f +(are)h(written)g(using)g(angular)-2000 30727 y(brack)-10 +b(ets)262 b Fo(<)309 b(:::)h(>)p FC(,)263 b(list)e(concatenation)j(is)d +(e)-15 b(xpressed)262 b(using)g(tw)-10 b(o)262 b(colons:)-2000 +31834 y Fo(l)-1694 31945 y Fn(1)-949 31834 y Fh(::)285 +b Fo(l)210 31945 y Fn(2)671 31834 y FC(.)-672 32941 y(The)358 +b(state)h(of)e(the)i(optimizer)f(is)g(stored)g(in)g(an)g(en)-40 +b(vironment)359 b Fo(E)415 b FC(and)-2000 34048 y(a)309 +b Fx(static)g(heap)g Fo(S)57 b FC(.)308 b(Each)i(step)e(of)h(the)g +(optimizer)h(tak)-10 b(es)309 b(an)g(operation,)h(an)-2000 +35155 y(en)-40 b(vironment)305 b(and)f(a)g(static)f(heap)i(and)f +(produces)g(a)g(list)e(of)i(operations,)g(a)-2000 36262 +y(ne)-25 b(w)250 b(en)-40 b(vironment)250 b(and)g(a)f(ne)-25 +b(w)250 b(static)f(heap.)-672 37369 y(The)290 b(en)-40 +b(vironment)290 b(is)f(a)g(partial)h(function)g(from)e(v)-25 +b(ariables)290 b(in)f(the)h(un-)-2000 38476 y(optimized)208 +b(trace)f Fo(V)435 b FC(to)207 b(v)-25 b(ariables)207 +b(in)g(the)g(optimized)h(trace)f Fo(V)19691 38053 y Fc(\003)20377 +38476 y FC(\(which)h(are)-2000 39583 y(themselv)-15 b(es)223 +b(written)g(with)f(a)8560 39160 y Fc(\003)9262 39583 +y FC(for)g(clarity\).)h(The)g(reason)f(for)g(introducing)-2000 +40690 y(ne)-25 b(w)345 b(v)-25 b(ariables)345 b(in)f(the)g(optimized)h +(trace)g(is)f(that)g(se)-25 b(v)-15 b(eral)345 b(v)-25 +b(ariables)345 b(that)-2000 41797 y(appear)364 b(in)f(the)g +(unoptimized)h(trace)g(can)f(turn)g(into)g(the)g(same)g(v)-25 +b(ariables)-2000 42904 y(in)308 b(the)g(optimized)g(trace.)h(The)f(en) +-40 b(vironment)309 b(of)e(the)h(optimizer)g(serv)-15 +b(es)308 b(a)-2000 44011 y(function)267 b(similar)g(to)g(that)g(of)f +(the)i(en)-40 b(vironment)268 b(in)f(the)g(semantics:)g(to)g(e)-15 +b(x-)-2000 45118 y(press)248 b(sharing.)-672 46225 y(The)207 +b(static)g(heap)g(is)f(a)h(partial)g(function)g(from)f +Fo(V)16248 45801 y Fc(\003)16934 46225 y FC(into)g(the)h(set)g(of)f +(static)-2000 47332 y(objects,)331 b(which)h(are)f(triples)f(of)h(a)g +(type)g(and)h(tw)-10 b(o)331 b(elements)g(of)g Fo(V)21958 +46908 y Fc(\003)22438 47332 y FC(.)g(The)-2000 48439 +y(object)309 b(referenced)h(by)f(a)f(v)-25 b(ariable)310 +b Fo(v)11336 48015 y Fc(\003)12124 48439 y FC(is)e(static,)h(if)f +Fo(v)17208 48015 y Fc(\003)17996 48439 y FC(is)g(in)g(the)h(domain) +-2000 49546 y(of)293 b(the)g(static)g(heap)h Fo(S)57 +b FC(.)293 b(The)g(object)h Fo(S)57 b Fh(\()p Fo(v)12612 +49122 y Fc(\003)13091 49546 y Fh(\))293 b FC(describes)g(what)h(is)f +(statically)-2000 50652 y(kno)-25 b(wn)361 b(about)f(the)g(object,)h +(i.e.,)f(its)f(type)h(and)g(its)f(\002elds.)h(The)g(\002elds)f(of)-2000 +51759 y(objects)243 b(in)g(the)g(static)g(heap)h(are)f(also)g(elements) +g(of)g Fo(V)17001 51336 y Fc(\003)17724 51759 y FC(\(or)f(null,)h(for)g +(short)-2000 52866 y(periods)249 b(of)g(time\).)-672 +53973 y(When)331 b(the)g(optimizer)g(sees)e(a)i Fr(new)p +Black Black 330 w FC(operation,)g(it)f(optimistically)g(re-)-2000 +55080 y(mo)-15 b(v)g(es)288 b(it)f(and)h(assumes)e(that)i(the)g +(resulting)f(object)h(can)g(stay)f(static.)h(The)-2000 +56187 y(optimization)309 b(for)g(all)f(further)g(operations)h(is)f +(split)g(into)h(tw)-10 b(o)309 b(cases.)f(One)-2000 57294 +y(case)233 b(is)f(for)g(when)i(the)f(in)-40 b(v)-20 b(olv)-15 +b(ed)233 b(v)-25 b(ariables)234 b(are)e(in)h(the)g(static)g(heap,)g +(which)-2000 58401 y(means)211 b(that)g(the)h(operation)f(can)h(be)f +(performed)g(at)g(optimization)h(time)f(and)-2000 59508 +y(can)353 b(be)h(remo)-15 b(v)g(ed)353 b(from)g(the)g(trace.)g(These)g +(rules)f(mirror)g(the)h(e)-15 b(x)g(ecution)-2000 60615 +y(semantics)253 b(closely)-65 b(.)253 b(The)g(other)g(case)g(is)f(for)g +(when)i(not)e(enough)i(is)e(kno)-25 b(wn)-2000 61722 +y(about)250 b(the)f(v)-25 b(ariables,)250 b(and)g(the)f(operation)h +(has)f(to)g(be)h(residualized.)-672 62829 y(If)336 b(the)h(ar)-18 +b(gument)337 b Fo(v)373 b FC(of)336 b(a)h Fr(get)p Black +Black 337 w FC(operation)g(is)f(mapped)h(to)g(something)-2000 +63936 y(in)298 b(the)g(static)g(heap,)h(the)f Fr(get)p +Black Black 298 w FC(can)g(be)h(performed)f(at)g(optimization)h(time.) +-2000 65043 y(Otherwise,)250 b(the)f Fr(get)p Black Black +250 w FC(operation)h(needs)f(to)g(be)h(residualized.)-672 +66150 y(If)327 b(the)g(\002rst)f(ar)-18 b(gument)327 +b Fo(v)363 b FC(to)327 b(a)g Fr(set)p Black Black 327 +w FC(operation)h(is)e(mapped)i(to)f(some-)-2000 67257 +y(thing)391 b(in)g(the)g(static)f(heap,)i(then)f(the)g +Fr(set)p Black Black 391 w FC(can)g(be)g(performed)g(at)g(opti-)-2000 +68364 y(mization)313 b(time)f(\(which)g(updates)g(the)g(static)g +(heap\).)g(Otherwise)h(the)f Fr(set)p Black Black -2000 +69471 a FC(operation)235 b(needs)f(to)g(be)h(residualized.)g(This)e +(needs)i(to)f(be)g(done)h(carefully)-65 b(,)-2000 70578 +y(because)222 b(the)f(ne)-25 b(w)221 b(v)-25 b(alue)222 +b(for)e(the)h(\002eld,)g(from)f(the)h(v)-25 b(ariable)221 +b Fo(u)p FC(,)g(could)h(itself)-2000 71685 y(be)250 b(static,)f(in)g +(which)h(case)f(it)g(needs)h(to)f(be)h(lifted)f(\002rst.)p +Black Black 28552 26299 a(If)214 b(a)i Fr(guard)32753 +26090 y(_)33286 26299 y(class)p Black Black 216 w FC(is)e(performed)h +(on)g(a)g(v)-25 b(ariable)216 b(that)f(is)g(in)f(the)i(static)27224 +27406 y(heap,)250 b(the)g(type)h(check)g(can)f(be)g(performed)g(at)g +(optimization)h(time,)f(which)27224 28513 y(means)255 +b(the)f(operation)i(can)f(be)g(remo)-15 b(v)g(ed)255 +b(if)f(the)h(types)f(match.)i(If)d(the)i(type)27224 29620 +y(check)345 b(f)-10 b(ails)343 b(statically)h(or)g(if)g(the)g(object)g +(is)g(not)g(in)g(the)g(static)g(heap,)g(the)27224 30727 +y Fr(guard)29889 30518 y(_)30422 30727 y(class)p Black +Black 333 w FC(is)332 b(residualized.)i(This)d(also)i(needs)f(to)h +(lift)f(the)h(v)-25 b(ariable)27224 31834 y(on)249 b(which)h(the)f +Fr(guard)35284 31625 y(_)35817 31834 y(class)p Black +Black 251 w FC(is)f(performed.)28552 32941 y(Lifting)300 +b(tak)-10 b(es)300 b(a)h(v)-25 b(ariable)301 b(and)g(turns)f(it)g(into) +g(a)h(dynamic)g(v)-25 b(ariable.)301 b(If)27224 34048 +y(the)316 b(v)-25 b(ariable)317 b(is)f(already)h(dynamic,)g(nothing)g +(needs)f(to)g(be)h(done.)g(If)e(it)h(is)27224 35155 y(in)346 +b(the)h(static)g(heap,)g(operations)g(are)g(emitted)g(that)f(construct) +h(an)g(object)27224 36262 y(with)336 b(the)f(shape)h(described)g +(there,)g(and)g(the)g(v)-25 b(ariable)337 b(is)e(remo)-15 +b(v)g(ed)336 b(from)27224 37369 y(the)249 b(static)g(heap.)28552 +38476 y(Lifting)328 b(a)g(static)g(object)h(needs)f(to)g(recursi)-25 +b(v)-15 b(ely)329 b(lift)e(its)g(\002elds.)h(Some)27224 +39583 y(care)417 b(needs)g(to)g(be)g(tak)-10 b(en)418 +b(when)f(lifting)f(a)h(static)g(object,)h(because)f(the)27224 +40690 y(structures)456 b(described)h(by)g(the)g(static)g(heap)h(can)f +(be)g(c)-15 b(yclic.)458 b(T)-80 b(o)457 b(mak)-10 b(e)27224 +41797 y(sure)286 b(that)g(the)h(same)f(static)g(object)h(is)f(not)g +(lifted)g(twice,)h(the)g Fr(liftfield)p Black Black 27224 +42904 a FC(operation)271 b(remo)-15 b(v)g(es)271 b(it)g(from)f(the)h +(static)g(heap)h Fx(befor)-37 b(e)271 b FC(recursi)-25 +b(v)-15 b(ely)271 b(lifting)27224 44011 y(its)248 b(\002elds.)28552 +45118 y(As)h(an)h(e)-15 b(xample)250 b(for)f(lifting,)g(consider)g(the) +g(static)h(heap)32088 46777 y Fg(f)p Fo(v)33133 46299 +y Fc(\003)33897 46777 y Fg(7!)284 b Fh(\()p Fo(T)36200 +46888 y Fn(1)36662 46777 y Fo(;)171 b(w)37878 46299 y +Fc(\003)38358 46777 y Fo(;)g(v)39346 46299 y Fc(\003)39825 +46777 y Fh(\))p Fo(;)h(w)41440 46299 y Fc(\003)42203 +46777 y Fg(7!)285 b Fh(\()p Fo(T)44507 46888 y Fn(2)44969 +46777 y Fo(;)171 b(u)46013 46299 y Fc(\003)46493 46777 +y Fo(;)g(u)47537 46299 y Fc(\003)48017 46777 y Fh(\))p +Fg(g)27224 48437 y FC(which)448 b(contains)g(tw)-10 b(o)448 +b(static)g(objects.)g(If)f Fo(v)43520 48014 y Fc(\003)44447 +48437 y FC(needs)h(to)g(be)g(lifted,)f(the)27224 49544 +y(follo)-25 b(wing)249 b(residual)h(operations)f(are)g(produced:)29438 +51134 y Fj(v)29909 50800 y Fb(\003)30835 51134 y Fp(=)464 +b(n)o(e)o(w)o(\()p Fj(T)34149 51245 y Fi(1)34580 51134 +y Fp(\))29438 52064 y Fj(w)30099 51730 y Fb(\003)31026 +52064 y Fp(=)f(n)o(e)o(w)o(\()p Fj(T)34339 52175 y Fi(2)34770 +52064 y Fp(\))29437 52994 y(s)o(e)o(t)o(\()o Fj(w)31958 +52660 y Fb(\003)32420 52994 y Fp(,)i Fj(L)p Fp(,)f Fj(u)35411 +52660 y Fb(\003)35874 52994 y Fp(\))29437 53924 y(s)o(e)o(t)o(\()o +Fj(w)31958 53590 y Fb(\003)32420 53924 y Fp(,)h Fj(R)s +Fp(,)g Fj(u)35475 53590 y Fb(\003)35937 53924 y Fp(\))29437 +54854 y(s)o(e)o(t)o(\()o Fj(v)31768 54520 y Fb(\003)32230 +54854 y Fp(,)g Fj(L)o Fp(,)g Fj(w)35359 54520 y Fb(\003)35821 +54854 y Fp(\))29437 55784 y(s)o(e)o(t)o(\()o Fj(v)31768 +55449 y Fb(\003)32230 55784 y Fp(,)g Fj(R)r Fp(,)g Fj(v)35232 +55449 y Fb(\003)35694 55784 y Fp(\))p Black Black 28552 +57551 a FC(After)330 b(the)g(lifting)g(the)g(static)g(heap)h(is)f(the)g +(empty)h(set,)e(because)i(both)27224 58658 y(static)223 +b(objects)g(were)h(lifted.)f(If)g(we)g(had)h(lifted)f +Fo(w)44349 58235 y Fc(\003)45052 58658 y FC(instead)g(of)g +Fo(v)49683 58235 y Fc(\003)50163 58658 y FC(,)g(then)g(the)27224 +59765 y(follo)-25 b(wing)249 b(operations)h(w)-10 b(ould)250 +b(ha)-20 b(v)-15 b(e)250 b(been)g(produced:)29438 61356 +y Fj(w)30099 61022 y Fb(\003)31026 61356 y Fp(=)463 b(n)o(e)o(w)o(\()p +Fj(T)34339 61467 y Fi(2)34770 61356 y Fp(\))29437 62285 +y(s)o(e)o(t)o(\()o Fj(w)31958 61951 y Fb(\003)32420 62285 +y Fp(,)i Fj(L)p Fp(,)f Fj(u)35411 61951 y Fb(\003)35874 +62285 y Fp(\))29437 63215 y(s)o(e)o(t)o(\()o Fj(w)31958 +62881 y Fb(\003)32420 63215 y Fp(,)h Fj(R)s Fp(,)g Fj(u)35475 +62881 y Fb(\003)35937 63215 y Fp(\))p Black Black 28552 +64983 a FC(In)249 b(this)g(case,)g(the)h(static)f(heap)h(afterw)-10 +b(ards)249 b(w)-10 b(ould)250 b(be:)36183 66642 y Fg(f)p +Fo(v)37228 66164 y Fc(\003)37993 66642 y Fg(7!)284 b +Fh(\()p Fo(T)40296 66753 y Fn(1)40758 66642 y Fo(;)171 +b(w)41974 66164 y Fc(\003)42453 66642 y Fo(;)g(v)43441 +66164 y Fc(\003)43921 66642 y Fh(\))p Fg(g)27224 68632 +y Fs(5.2)996 b(Analysis)249 b(of)g(the)g(Algorithm)27224 +70181 y FC(While)275 b(we)f(do)h(not)f(of)-25 b(fer)274 +b(a)h(formal)f(proof)g(of)g(it,)g(it)g(can)h(ar)-18 b(gue)275 +b(informally)27224 71288 y(that)309 b(the)g(algorithm)g(presented)g +(abo)-15 b(v)g(e)310 b(is)e(sound:)h(it)f(w)-10 b(orks)309 +b(by)g(delaying)27224 72395 y(\(and)323 b(often)g(completely)h(remo)-15 +b(ving\))324 b(some)f(operations.)g(The)g(algorithm)p +Black Black eop end +%%Page: 7 7 +TeXDict begin 7 6 bop Black Black Black Black Black 7379 +8835 a Fx(ne)-15 b(w)26088 8191 y Fo(v)26621 7767 y Fc(\003)27271 +8191 y Fh(fresh)p 14598 8607 26305 43 v 14598 9863 a +Fo(v)321 b Fh(=)284 b Ff(new)q Fh(\()p Fo(T)142 b Fh(\))p +Fo(;)172 b(E)57 b(;)171 b(S)22526 9252 y Fn(opt)22288 +9863 y Fh(=)-171 b Fg(\))286 b(h)170 b(i)h Fo(;)g(E)228 +b Fh([)q Fo(v)320 b Fg(7!)284 b Fo(v)29739 9577 y Fc(\003)30219 +9863 y Fh(])171 b Fo(;)g(S)227 b Fh([)q Fo(v)32800 9577 +y Fc(\003)33564 9863 y Fg(7!)284 b Fh(\()p Fo(T)85 b(;)172 +b Fh(n)-28 b(ull)p Fo(;)171 b Fh(n)-28 b(ull\))q(])7379 +13548 y Fx(g)-10 b(et)24348 12903 y Fo(E)57 b Fh(\()p +Fo(v)36 b Fh(\))285 b Fg(2)f Fh(dom\()p Fo(S)57 b Fh(\))p +16923 13320 21654 43 v 16923 14576 a Fo(u)285 b Fh(=)g +Ff(get)p Fh(\()p Fo(v)36 b(;)171 b(F)142 b Fh(\))p Fo(;)172 +b(E)57 b(;)171 b(S)25952 13966 y Fn(opt)25714 14576 y +Fh(=)-171 b Fg(\))285 b(h)171 b(i)g Fo(;)g(E)228 b Fh([)q +Fo(u)284 b Fg(7!)h Fo(S)57 b Fh(\()p Fo(E)g Fh(\()p Fo(v)36 +b Fh(\)\))36307 14687 y FD(F)36985 14576 y Fh(])171 b +Fo(;)g(S)22174 17616 y(E)57 b Fh(\()p Fo(v)36 b Fh(\))399 +b Fo(=)-626 b Fg(2)284 b Fh(dom\()p Fo(S)57 b Fh(\))p +Fo(;)683 b(u)30534 17193 y Fc(\003)31185 17616 y Fh(fresh)p +14527 18034 26447 43 v 14527 19290 a Fo(u)285 b Fh(=)f +Ff(get)p Fh(\()p Fo(v)36 b(;)172 b(F)142 b Fh(\))p Fo(;)171 +b(E)57 b(;)171 b(S)23556 18679 y Fn(opt)23318 19290 y +Fh(=)-171 b Fg(\))285 b(h)p Fo(u)26239 19004 y Fc(\003)27004 +19290 y Fh(=)f Ff(get)q Fh(\()p Fo(E)57 b Fh(\()p Fo(v)36 +b Fh(\))p Fo(;)172 b(F)142 b Fh(\))p Fg(i)171 b Fo(;)g(E)228 +b Fh([)p Fo(u)285 b Fg(7!)f Fo(u)38901 19004 y Fc(\003)39381 +19290 y Fh(])171 b Fo(;)g(S)7379 22975 y Fx(set)24348 +22330 y Fo(E)57 b Fh(\()p Fo(v)36 b Fh(\))285 b Fg(2)f +Fh(dom\()p Fo(S)57 b Fh(\))p 14851 22747 25799 43 v 14851 +24003 a Ff(set)171 b Fh(\()p Fo(v)36 b(;)171 b(F)28 b(;)171 +b(u)p Fh(\))h Fo(;)f(E)57 b(;)171 b(S)23197 23393 y Fn(opt)22960 +24003 y Fh(=)-171 b Fg(\))285 b(h)170 b(i)h Fo(;)g(E)57 +b(;)172 b(S)227 b Fh([)p Fo(E)h Fh(\()p Fo(v)36 b Fh(\))285 +b Fg(7!)f Fh(\()p Fo(S)57 b Fh(\()p Fo(E)g Fh(\()p Fo(v)36 +b Fh(\)\)!)37092 24114 y FD(F)37771 24003 y Fo(E)57 b +Fh(\()p Fo(u)p Fh(\)\))q(])18860 27438 y Fo(E)g Fh(\()p +Fo(v)36 b Fh(\))399 b Fo(=)-625 b Fg(2)284 b Fh(dom)170 +b(\()q Fo(S)57 b Fh(\))170 b Fo(;)341 b Fh(\()p Fo(E)57 +b Fh(\()p Fo(v)36 b Fh(\))p Fo(;)172 b(S)57 b Fh(\))31293 +26868 y Fn(lift)30989 27438 y Fh(=)-171 b Fg(\))285 b +Fh(\(ops)p Fo(;)171 b(S)35944 27015 y Fc(0)36242 27438 +y Fh(\))p 15352 27855 24797 43 v 15352 29112 a Ff(set)f +Fh(\()q Fo(v)36 b(;)171 b(F)28 b(;)171 b(u)p Fh(\))g +Fo(;)g(E)57 b(;)172 b(S)23698 28501 y Fn(opt)23460 29112 +y Fh(=)-171 b Fg(\))285 b Fh(ops)f(::)i Fg(h)p Ff(set)171 +b Fh(\()p Fo(E)57 b Fh(\()p Fo(v)36 b Fh(\))p Fo(;)172 +b(F)28 b(;)171 b(E)57 b Fh(\()p Fo(u)p Fh(\)\))q Fg(i)171 +b Fo(;)g(E)57 b(;)171 b(S)39851 28826 y Fc(0)7379 32797 +y Fx(guar)-37 b(d)19794 32152 y Fo(E)57 b Fh(\()p Fo(v)36 +b Fh(\))285 b Fg(2)f Fh(dom\()p Fo(S)57 b Fh(\))p Fo(;)342 +b Fh(t)-28 b(yp)28 b(e)q(\()p Fo(S)57 b Fh(\()p Fo(E)g +Fh(\()p Fo(v)36 b Fh(\)\)\))285 b(=)g Fo(T)p 19543 32569 +16414 43 v 19543 33825 a Ff(guard)p FC(_)p Ff(class)q +Fh(\()p Fo(v)36 b(;)171 b(T)142 b Fh(\))p Fo(;)172 b(E)57 +b(;)171 b(S)30720 33214 y Fn(opt)30483 33825 y Fh(=)-171 +b Fg(\))285 b(h)171 b(i)f Fo(;)i(E)57 b(;)171 b(S)14249 +37260 y(E)57 b Fh(\()p Fo(v)36 b Fh(\))399 b Fo(=)-625 +b Fg(2)284 b Fh(dom\()p Fo(S)57 b Fh(\))227 b Fg(_)g +Fh(t)-28 b(yp)28 b(e)q(\()p Fo(S)57 b Fh(\()p Fo(E)g +Fh(\()p Fo(v)36 b Fh(\)\)\))285 b Fg(6)p Fh(=)g Fo(T)85 +b(;)342 b Fh(\()p Fo(E)57 b Fh(\()p Fo(v)36 b Fh(\))p +Fo(;)172 b(S)57 b Fh(\))35904 36690 y Fn(lift)35600 37260 +y Fh(=)-171 b Fg(\))285 b Fh(\(ops)p Fo(;)171 b(S)40555 +36837 y Fc(0)40852 37260 y Fh(\))p 13068 37677 29365 +43 v 13068 38933 a Ff(guard)p FC(_)p Ff(class)p Fh(\()p +Fo(v)36 b(;)172 b(T)142 b Fh(\))p Fo(;)172 b(E)57 b(;)171 +b(S)24245 38323 y Fn(opt)24007 38933 y Fh(=)-171 b Fg(\))286 +b Fh(ops)e(::)h Fg(h)p Ff(guard)p FC(_)p Ff(class)q Fh(\()p +Fo(E)228 b Fh(\()p Fo(v)36 b Fh(\))171 b Fo(;)g(T)142 +b Fh(\))p Fg(i)171 b Fo(;)h(E)57 b(;)171 b(S)42135 38647 +y Fc(0)7379 42619 y Fx(lifting)24912 41973 y Fo(v)25445 +41550 y Fc(\003)26323 41973 y Fo(=)-626 b Fg(2)284 b +Fh(dom\()p Fo(S)57 b Fh(\))p 24428 42391 6644 43 v 24428 +43659 a Fo(v)24961 43373 y Fc(\003)25441 43659 y Fo(;)171 +b(S)27166 43089 y Fn(lift)26863 43659 y Fh(=)-171 b Fg(\))285 +b(h)171 b(i)g Fo(;)g(S)19532 47082 y(v)20065 46658 y +Fc(\003)20829 47082 y Fg(2)285 b Fh(dom\()p Fo(S)57 b +Fh(\))p Fo(;)341 b Fh(\()p Fo(v)26766 46658 y Fc(\003)27246 +47082 y Fo(;)171 b(S)57 b Fh(\))29077 46512 y Fn(lift\014elds)29066 +47082 y Fh(=)-171 b(=)g(=)g Fg(\))286 b Fh(\(ops)p Fo(;)171 +b(S)35272 46658 y Fc(0)35570 47082 y Fh(\))p 17885 47499 +19731 43 v 17885 48767 a Fo(v)18418 48481 y Fc(\003)18897 +48767 y Fo(;)g(S)20623 48197 y Fn(lift)20319 48767 y +Fh(=)-171 b Fg(\))285 b(h)p Fo(v)23184 48481 y Fc(\003)23949 +48767 y Fh(=)f Ff(new)171 b Fh(\(t)-28 b(yp)28 b(e)172 +b(\()p Fo(S)227 b Fh(\()p Fo(v)31483 48481 y Fc(\003)31963 +48767 y Fh(\)\)\))p Fg(i)285 b Fh(::)g Fo(ops;)171 b(S)37318 +48481 y Fc(0)11321 51385 y Fe(\000)11790 52190 y Fo(S)227 +b Fh(\()p Fo(v)13574 51767 y Fc(\003)14054 52190 y Fh(\))14452 +52494 y FD(L)15240 52190 y Fo(;)171 b(S)284 b Fg(n)227 +b(f)p Fo(v)18389 51767 y Fc(\003)19154 52190 y Fg(7!)284 +b Fo(S)227 b Fh(\()p Fo(v)22246 51767 y Fc(\003)22726 +52190 y Fh(\))p Fg(g)23636 51385 y Fe(\001)24693 51620 +y Fn(lift)24390 52190 y Fh(=)-171 b Fg(\))285 b Fh(\(ops)28207 +52439 y FD(L)28824 52190 y Fo(;)171 b(S)29962 51767 y +Fc(0)30260 52190 y Fh(\))f Fo(;)31454 51385 y Fe(\000)31923 +52190 y Fo(S)227 b Fh(\()p Fo(v)33707 51767 y Fc(\003)34187 +52190 y Fh(\))34585 52494 y FD(R)35426 52190 y Fo(;)171 +b(S)36564 51767 y Fc(0)36862 51385 y Fe(\001)37919 51620 +y Fn(lift)37616 52190 y Fh(=)-171 b Fg(\))285 b Fh(\(ops)41433 +52439 y FD(R)42103 52190 y Fo(;)171 b(S)43241 51767 y +Fc(00)43781 52190 y Fh(\))p 11221 52707 33058 43 v 11221 +53975 a Fo(v)11754 53689 y Fc(\003)12234 53975 y Fo(;)g(S)13667 +53405 y Fn(lift\014elds)13656 53975 y Fh(=)-171 b(=)g(=)g +Fg(\))286 b Fh(ops)18326 54224 y FD(L)19228 53975 y Fh(::)f(ops)21566 +54224 y FD(R)22520 53975 y Fh(::)23374 53170 y Fe(\012)23857 +53975 y Ff(set)25597 53170 y Fe(\000)26066 53975 y Fo(v)26599 +53689 y Fc(\003)27079 53975 y Fo(;)171 b(L;)g(S)227 b +Fh(\()p Fo(v)30470 53689 y Fc(\003)30950 53975 y Fh(\))31348 +54280 y FD(L)31965 53170 y Fe(\001)32605 53975 y Fo(;)342 +b Ff(set)34971 53170 y Fe(\000)35440 53975 y Fo(v)35973 +53689 y Fc(\003)36453 53975 y Fo(;)171 b(R)7 b(;)171 +b(S)227 b Fh(\()p Fo(v)39929 53689 y Fc(\003)40409 53975 +y Fh(\))40807 54280 y FD(R)41477 53170 y Fe(\001\013)42601 +53975 y Fo(;)171 b(S)43739 53689 y Fc(00)-2000 57817 +y Fx(Object)250 b(Domains:)417 59460 y Fo(u;)172 b(v)235 +b Fg(2)199 b Fo(V)2809 b Fh(v)-57 b(ariables)250 b(in)f(trace)-542 +60567 y Fo(u)47 60143 y Fc(\003)527 60567 y Fo(;)171 +b(v)1515 60143 y Fc(\003)2194 60567 y Fg(2)199 b Fo(V)3901 +60143 y Fc(\003)6482 60567 y Fh(v)-57 b(ariables)250 +b(in)f(optimized)i(trace)1255 61674 y Fo(T)342 b Fg(2)199 +b Fd(T)2740 b Fh(run)-28 b(time)249 b(t)-28 b(yp)28 b(es)1199 +62781 y Fo(F)341 b Fg(2)199 b(f)p Fo(L;)171 b(R)7 b Fg(g)448 +b Fh(\014elds)249 b(of)327 b(ob)57 b(jects)20296 58009 +y Fx(Semantic)251 b(V)-111 b(alues:)20396 59460 y Fo(E)256 +b Fg(2)199 b Fo(V)512 b(*)285 b(V)25529 59036 y Fc(\003)40683 +59460 y Fh(En)-28 b(vironmen)g(t)20524 60567 y Fo(S)255 +b Fg(2)199 b Fo(V)23112 60143 y Fc(\003)23876 60567 y +Fo(*)284 b Fd(T)228 b Fg(\002)g Fh(\()p Fo(V)28325 60143 +y Fc(\003)29033 60567 y Fg([)f(f)p Fh(n)-28 b(ull)p Fg(g)p +Fh(\))227 b Fg(\002)h Fh(\()p Fo(V)35517 60143 y Fc(\003)36224 +60567 y Fg([)g(f)p Fh(n)-28 b(ull)p Fg(g)p Fh(\))448 +b(Static)250 b(Heap)p -2000 63592 55791 37 v 19858 64721 +a Fs(Figur)-18 b(e)249 b(6.)498 b FC(Optimization)251 +b(Rules)p Black Black Black eop end +%%Page: 8 8 +TeXDict begin 8 7 bop Black Black -2000 886 a FC(runs)279 +b(in)g(a)h(single)f(pass)f(o)-15 b(v)g(er)280 b(the)g(list)e(of)h +(operations.)h(W)-80 b(e)280 b(can)g(check)h(that)-2000 +1993 y(although)387 b(recursi)-25 b(v)-15 b(ely)386 b(lifting)g(a)h +(static)f(object)g(is)g(not)g(a)g(constant-time)-2000 +3099 y(operation,)266 b(the)g(algorithm)g(only)f(tak)-10 +b(es)266 b(a)f(total)h(time)g(linear)f(in)h(the)f(length)-2000 +4206 y(of)337 b(the)g(trace.)g(The)g(algorithm)g(itself)g(is)f(not)h +(particularly)g(comple)-15 b(x;)338 b(our)-2000 5313 +y(focus)343 b(is)g(rather)h(that)g Fx(in)f(the)h(conte)-20 +b(xt)345 b(of)e(tr)-15 b(acing)344 b(JITs)f FC(it)g(is)g(possible)g(to) +-2000 6420 y(\002nd)249 b(a)h(simple)f(enough)h(algorithm)f(that)h +(performs)e(well.)-672 7527 y(Note)302 b(in)f(particular)h(that)g +(objects)f(in)g(cate)-15 b(gory)303 b(1)e(\(i.e.,)g(those)g(that)h(do) +-2000 8634 y(not)240 b(escape\))h(are)g(completely)g(remo)-15 +b(v)g(ed;)241 b(moreo)-15 b(v)g(er)-40 b(,)241 b(objects)g(in)f(cate) +-15 b(gory)-2000 9741 y(2)302 b(\(i.e.,)g(escaping\))h(are)f(still)g +(partially)g(optimized:)h(all)f(the)h(operations)f(in)-2000 +10848 y(between)329 b(the)f(creation)h(of)e(the)i(object)f(and)g(the)g +(point)g(where)h(it)e(escapes)-2000 11955 y(that)225 +b(in)-40 b(v)-20 b(olv)-15 b(e)226 b(the)g(object)g(are)f(remo)-15 +b(v)g(ed.)226 b(Objects)g(in)f(cate)-15 b(gory)226 b(3)f(and)h(4)f(are) +-2000 13062 y(also)330 b(partially)g(optimized,)g(their)g(allocation)h +(is)e(delayed)i(till)e(the)h(end)g(of)-2000 14169 y(the)249 +b(trace.)-672 15276 y(The)362 b(optimization)h(is)e(particularly)i(ef) +-25 b(fecti)g(v)-15 b(e)363 b(for)e(chains)h(of)g(opera-)-2000 +16383 y(tions.)355 b(F)-15 b(or)355 b(e)-15 b(xample,)357 +b(it)f(is)e(typical)j(for)e(an)g(interpreter)h(to)g(generate)g(se-) +-2000 17490 y(quences)196 b(of)e(writes-follo)-25 b(wed-by-reads,)195 +b(where)h(one)f(interpreted)h(opcode)-2000 18597 y(writes)273 +b(to)g(some)h(object')-55 b(s)273 b(\002eld)h(and)f(the)h(ne)-15 +b(xt)274 b(interpreted)g(opcode)g(reads)-2000 19704 y(it)295 +b(back,)g(possibly)f(dispatching)i(on)e(the)i(type)f(of)f(the)h(object) +h(created)f(just)-2000 20811 y(before.)355 b(A)g(typical)h(e)-15 +b(xample)356 b(w)-10 b(ould)355 b(be)g(a)g(chain)h(of)e(arithmetic)i +(opera-)-2000 21918 y(tions.)-2000 24492 y FA(6.)1218 +b(Implementation)305 b(and)g(Ev)-12 b(aluation)-2000 +26042 y FC(The)227 b(allocation)g(remo)-15 b(v)-25 b(al)228 +b(techniques)f(described)g(in)g(this)f(paper)h(were)g(im-)-2000 +27149 y(plemented)240 b(in)f(the)h(optimizer)f(of)g(PyPy')-55 +b(s)239 b(tracing)h(JIT)-74 b(.)238 b(The)h(optimization)-2000 +28256 y(is)219 b(independent)i(of)f(which)g(interpreter)g(a)g(JIT)f(is) +g(generated)i(for)-55 b(.)220 b(There)g(are)-2000 29363 +y(some)202 b(practical)h(issues)e(be)-15 b(yond)203 b(the)g(techniques) +g(described)f(in)g(this)g(paper)-55 b(.)-2000 30470 y(The)222 +b(actual)h(implementation)g(needs)f(to)g(deal)h(with)f(more)g +(operations)g(than)-2000 31577 y(described)237 b(in)g(Section)h(5,)f +(e.g.,)g(to)g(also)g(support)f(static)h(arrays)f(in)h(addition)-2000 +32684 y(to)292 b(static)g(objects.)h(The)f(implementation)i(of)e(this)g +(optimization)h(is)e(about)-2000 33791 y(400)249 b(lines)g(of)g +(RPython)g(code.)-672 34898 y(A)423 b(further)f(complication)i(is)e +(that)h(most)f(interpreters)g(written)h(with)-2000 36005 +y(PyPy)464 b(use)g(heap-allocated)i(frame)e(objects)g(to)g(store)g +(local)g(v)-25 b(ariables.)-2000 37112 y(Those)306 b(se)-25 +b(v)-15 b(erely)307 b(hinder)f(the)h(ef)-25 b(fecti)g(v)-15 +b(eness)307 b(of)e(allocation)j(remo)-15 b(v)-25 b(al,)307 +b(be-)-2000 38219 y(cause)225 b(e)-25 b(v)-15 b(ery)226 +b(time)f(an)g(object)g(is)f(stored)h(into)g(a)g(local)g(v)-25 +b(ariable,)226 b(it)e(is)g(stored)-2000 39326 y(into)343 +b(the)g(frame-object,)h(which)f(mak)-10 b(es)343 b(it)g(escape.)h(W)-80 +b(e)344 b(implemented)g(a)-2000 40433 y(technique)392 +b(to)e(treat)g(such)h(frames)f(objects)g(in)g(a)h(special)g(w)-10 +b(ay)391 b(to)f(solv)-15 b(e)-2000 41540 y(this)280 b(problem.)g(This)f +(is)h(a)g(common)h(approach)g(in)f(VM)g(implementations)-2000 +42647 y([13,)449 b(22];)f(the)h(no)-15 b(v)g(elty)449 +b(of)g(our)f(approach)i(is)e(that)h(we)g(generalized)h(it)-2000 +43754 y(enough)250 b(to)f(be)h(usable)f(for)g(dif)-25 +b(ferent)249 b(interpreters.)-672 44860 y(T)-80 b(o)386 +b(e)-25 b(v)g(aluate)387 b(our)e(allocation)h(remo)-15 +b(v)-25 b(al)386 b(algorithm,)g(we)g(look)f(at)h(the)-2000 +45967 y(ef)-25 b(fecti)g(v)-15 b(eness)433 b(when)g(used)f(in)g(the)h +(generated)h(tracing)e(JIT)f(of)h(PyPy')-55 b(s)-2000 +47074 y(Python)488 b(interpreter)-55 b(.)489 b(This)f(interpreter)g(is) +f(a)i(full)f(implementation)h(of)-2000 48181 y(Python)579 +b(2.5)f(language)i(semantics)e(and)h(is)f(about)h(30,000)g(lines)f(of) +-2000 49288 y(RPython)249 b(code.)-672 50395 y(The)476 +b(benchmarks)f(we)h(used)f(are)h(small-to-medium)f(Python)g(pro-)-2000 +51540 y(grams,)249 b(some)g(synthetic)h(benchmarks,)g(some)f(real)g +(applications.)21601 51117 y Fz(8)-672 52647 y FC(Some)410 +b(of)e(them)i(are)f(from)f(the)h(Computer)g(Language)h(Benchmark)-2000 +53791 y(Game)378 53368 y Fz(9)766 53791 y FC(:)250 b +Fs(fannkuch)p FC(,)e Fs(nbody)p FC(,)i Fs(meteor)-37 +b(-contest)p FC(,)249 b Fs(spectral-norm)p FC(.)-672 +54898 y(Furthermore)h(there)f(are)h(the)f(follo)-25 b(wing)250 +b(benchmarks:)p Black -1419 56470 a Fm(\017)p Black -561 +56568 a Fs(crypto_pyaes)p FC(:)g(An)f(AES)g(implementation.)p +Black -1419 58020 a Fm(\017)p Black -561 58118 a Fs(django)p +FC(:)519 b(The)h(templating)g(engine)h(of)e(the)h(Django)g(web)g +(frame-)-561 59263 y(w)-10 b(ork)1476 58839 y Fz(10)2196 +59263 y FC(.)p Black -1419 60714 a Fm(\017)p Black -561 +60812 a Fs(go)p FC(:)249 b(A)h(Monte-Carlo)e(Go)i(AI)9843 +60389 y Fz(11)10562 60812 y FC(.)p Black -1419 62264 +a Fm(\017)p Black -561 62362 a Fs(html5lib)p FC(:)f(An)g(HTML5)g +(parser)-55 b(.)p Black -2000 63529 13284 37 v -2000 +64212 a Fz(8)-1502 64525 y Fw(All)901 b(the)f(source)h(code)f(of)g(the) +h(benchmarks)f(can)g(be)g(found)g(at)-2000 65521 y Fr +(http://codespeak.net/svn/pypy/benchmarks/)p Fw(.)2428 +b(There)-2000 66517 y(is)766 b(also)g(a)g(website)h(that)e(monitors)h +(PyPy')-49 b(s)766 b(performance)g(nightly)f(at)-2000 +67513 y Fr(http://speed.pypy.org/)p Fw(.)-2000 68496 +y Fz(9)-1502 68809 y Fr(http://shootout.alioth.debian.org/)-2000 +69791 y Fz(10)-1170 70104 y Fr(http://www.djangoproject.com/)-2000 +71086 y Fz(11)-1170 71399 y Fr(http://shed-skin.blogspot.com/2009/07/) +-2000 72395 y(disco-elegant-python-go-player.html)p Black +Black Black Black 27805 788 a Fm(\017)p Black 28663 886 +a Fs(py\003ate-fast)p FC(:)248 b(A)h(BZ2)g(decoder)-55 +b(.)p Black 27805 2337 a Fm(\017)p Black 28663 2435 a +Fs(raytrace-simple)p FC(:)250 b(A)f(ray)g(tracer)-55 +b(.)p Black 27805 3887 a Fm(\017)p Black 28663 3985 a +Fs(richards)p FC(:)249 b(The)h(Richards)e(benchmark.)p +Black 27805 5437 a Fm(\017)p Black 28663 5535 a Fs(spambay)-10 +b(es)p FC(:)248 b(A)i(Bayesian)f(spam)g(\002lter)42932 +5112 y Fz(12)43652 5535 y FC(.)p Black 27805 7024 a Fm(\017)p +Black 28663 7122 a Fs(telco)p FC(:)451 b(A)f(Python)h(v)-15 +b(ersion)451 b(of)f(the)h(T)-70 b(elco)451 b(decimal)h(benchmark)52821 +6699 y Fz(13)53542 7122 y FC(,)28663 8229 y(using)249 +b(a)g(pure)h(Python)f(decimal)h(\003oating)g(point)f(implementation.)p +Black 27805 9681 a Fm(\017)p Black 28663 9779 a Fs(twisted_names)p +FC(:)361 b(A)h(DNS)h(serv)-15 b(er)362 b(benchmark)h(using)f(the)h(T) +-80 b(wisted)28663 10923 y(netw)-10 b(orking)250 b(frame)-25 +b(w)-10 b(ork)37775 10500 y Fz(14)38495 10923 y FC(.)28552 +13236 y(W)-80 b(e)322 b(e)-25 b(v)g(aluate)322 b(the)e(allocation)i +(remo)-15 b(v)-25 b(al)321 b(algorithm)g(along)g(tw)-10 +b(o)321 b(lines:)27224 14343 y(\002rst)361 b(we)h(w)-10 +b(ant)363 b(to)f(kno)-25 b(w)363 b(ho)-25 b(w)362 b(man)-15 +b(y)363 b(allocations)g(could)f(be)h(optimized)27224 +15450 y(a)-15 b(w)-10 b(ay)-65 b(.)230 b(On)g(the)f(other)g(hand,)h(we) +g(w)-10 b(ant)229 b(to)g(kno)-25 b(w)230 b(ho)-25 b(w)230 +b(much)g(the)f(run)g(times)27224 16557 y(of)249 b(the)g(benchmarks)h +(is)f(impro)-15 b(v)g(ed.)28552 17664 y(The)233 b(benchmarks)h(were)g +(run)f(on)h(an)f(otherwise)h(idle)f(Intel)g(Core2)g(Duo)27224 +18771 y(P8400)326 b(processor)f(with)h(2.26)g(GHz)h(and)f(3072)g(KB)f +(of)h(cache)h(on)f(a)g(ma-)27224 19878 y(chine)224 b(with)g(3GB)f(RAM)g +(running)h(Linux)f(2.6.35.)i(W)-80 b(e)224 b(compared)h(the)f(per)-20 +b(-)27224 20985 y(formance)333 b(of)f(v)-25 b(arious)332 +b(Python)h(implementations)g(on)f(the)h(benchmarks.)27224 +22092 y(As)452 b(a)h(baseline,)g(we)g(used)f(the)h(standard)f(Python)h +(implementation)h(in)27224 23236 y(C,)296 b(CPython)h(2.6.6)34211 +22813 y Fz(15)34931 23236 y FC(,)g(which)h(uses)f(a)g(bytecode-based)i +(interpreter)-55 b(.)297 b(Fur)-20 b(-)27224 24343 y(thermore)260 +b(we)g(compared)g(ag)-5 b(ainst)260 b(Psyco[25])f(1.6,)h(a)g +(\(hand-written\))g(e)-15 b(x-)27224 25450 y(tension)357 +b(module)g(to)g(CPython)f(which)i(is)e(a)h(just-in-time)f(compiler)h +(that)27224 26557 y(produces)343 b(machine)h(code)f(at)g(run-time.)g +(It)f(is)g(not)h(based)g(on)g(traces.)g(Fi-)27224 27664 +y(nally)-65 b(,)281 b(we)g(used)g(tw)-10 b(o)281 b(v)-15 +b(ersions)280 b(of)g(PyPy')-55 b(s)280 b(Python)h(interpreter)g(\(re) +-25 b(vision)27224 28809 y(77823)323 b(of)g(SVN)h(trunk)35609 +28385 y Fz(16)36329 28809 y FC(\):)e(one)i(including)g(the)f(JIT)f(b) +-20 b(ut)324 b(not)f(optimizing)27224 29916 y(the)309 +b(traces,)h(and)g(one)g(using)f(the)h(allocation)g(remo)-15 +b(v)-25 b(al)310 b(optimizations)g(\(as)27224 31022 y(well)249 +b(as)g(some)g(minor)h(other)f(optimizations,)h(such)f(as)g(constant)g +(folding\).)28552 32129 y(As)224 b(the)h(\002rst)e(step,)i(we)g +(counted)g(the)g(occurring)g(operations)g(in)f(all)g(gen-)27224 +33236 y(erated)262 b(traces)f(before)g(and)h(after)f(the)h +(optimization)g(phase)f(for)g(all)g(bench-)27224 34343 +y(marks.)393 b(The)g(resulting)f(numbers)h(can)h(be)f(seen)h(in)f +(Figure)g(7.)g(The)g(op-)27224 35450 y(timization)416 +b(remo)-15 b(v)g(es)417 b(between)g(4\045)f(and)g(90\045)g(and)h(of)f +(allocation)h(op-)27224 36557 y(erations)g(in)g(the)g(traces)g(of)g +(the)g(benchmarks.)h(All)f(benchmarks)h(tak)-10 b(en)27224 +37664 y(together)-40 b(,)332 b(the)f(optimization)h(remo)-15 +b(v)g(es)332 b(70\045)f(percent)h(of)f(allocation)h(op-)27224 +38771 y(erations.)e(The)g(numbers)f(look)i(similar)e(for)g(reading)i +(and)f(writing)g(of)g(at-)27224 39878 y(trib)-20 b(utes.)298 +b(There)i(are)f(e)-25 b(v)-15 b(en)300 b(more)f Fr(guard)p +Black Black 299 w FC(operations)g(that)g(are)h(remo)-15 +b(v)g(ed,)27224 40985 y(ho)-25 b(we)g(v)-15 b(er)356 +b(there)f(is)f(an)i(additional)g(optimization)f(that)g(remo)-15 +b(v)g(es)356 b(guards,)27224 42092 y(so)277 b(not)g(all)g(the)h(remo) +-15 b(v)g(ed)278 b(guards)g(are)f(an)h(ef)-25 b(fect)278 +b(of)f(the)g(optimization)i(de-)27224 43199 y(scribed)192 +b(here)g(\(for)f(technical)i(reasons,)f(it)f(w)-10 b(ould)193 +b(be)f(v)-15 b(ery)192 b(hard)g(to)g(separate)27224 44306 +y(the)249 b(tw)-10 b(o)250 b(ef)-25 b(fects\).)28552 +45413 y(In)327 b(addition)h(to)f(the)g(count)h(of)e(operations)i(we)f +(also)g(performed)g(time)27224 46520 y(measurements.)242 +b(All)f(benchmarks)h(were)g(run)g(50)f(times)g(in)h(the)f(same)h(pro-) +27224 47627 y(cess,)255 b(to)h(gi)-25 b(v)-15 b(e)256 +b(the)g(JIT)f(time)h(to)g(produce)h(machine)f(code.)h(The)f(arithmetic) +27224 48734 y(mean)303 b(of)g(the)g(times)g(of)g(the)g(last)f(30)h +(runs)g(were)g(used)g(as)g(the)g(result.)g(The)27224 +49841 y(errors)262 b(were)i(computed)h(using)e(a)h(con\002dence)h +(interv)-25 b(al)264 b(with)f(a)h(95\045)f(con-)27224 +50948 y(\002dence)354 b(le)-25 b(v)-15 b(el)353 b([16].)g(The)g +(results)f(are)h(reported)g(in)g(Figure)g(8.)g(F)-15 +b(or)353 b(each)27224 52055 y(implementation)327 b(the)g(table)g(also)f +(reports)f(the)i(speedup)g(that)f(PyPy)g(with)27224 53162 +y(optimization)250 b(achie)-25 b(v)-15 b(es)250 b(o)-15 +b(v)g(er)250 b(it.)28552 54269 y(W)-40 b(ith)238 b(the)f(optimization)h +(turned)f(on,)g(PyPy')-55 b(s)236 b(Python)i(interpreter)f(out-)27224 +55376 y(performs)453 b(CPython)g(in)g(all)h(benchmarks)g(e)-15 +b(xcept)455 b(spambayes)f(\(which)27224 56483 y(hea)-20 +b(vily)376 b(relies)f(on)g(re)-15 b(gular)376 b(e)-15 +b(xpression)375 b(performance)h(and)g(thus)f(is)g(not)27224 +57590 y(helped)369 b(much)h(by)f(our)f(Python)h(JIT\))e(and)j(meteor) +-20 b(-contest.)369 b(All)g(bench-)27224 58696 y(marks)306 +b(are)h(impro)-15 b(v)g(ed)307 b(by)g(the)f(allocation)i(remo)-15 +b(v)-25 b(al)307 b(optimization,)h(by)e(at)27224 59803 +y(least)249 b(20\045)g(and)h(by)f(as)g(much)h(as)f(a)g(f)-10 +b(actor)250 b(of)e(6.95.)28552 60910 y(Psyco)262 b(is)f(able)i(to)e +(outperform)h(PyPy')-55 b(s)262 b(JIT)e(in)i(\002)-25 +b(v)-15 b(e)262 b(out)g(of)g(14)g(bench-)27224 62017 +y(marks.)336 b(W)-80 b(e)337 b(hope)f(to)g(o)-15 b(v)g(ertak)-10 +b(e)337 b(Psyco)f(\(which)h(is)e(no)h(longer)h(being)f(ac-)27224 +63124 y(ti)-25 b(v)-15 b(ely)249 b(de)-25 b(v)-15 b(elopped\))251 +b(by)e(adding)h(some)g(further)e(optimizations.)p Black +27224 66219 V 27224 66902 a Fz(12)28054 67215 y Fr +(http://spambayes.sourceforge.net/)27224 68197 y Fz(13)28054 +68510 y Fr(http://speleotrove.com/decimal/telco.html)27224 +69492 y Fz(14)28054 69805 y Fr(http://twistedmatrix.com/)27224 +70788 y Fz(15)28054 71100 y Fr(http://python.org)27224 +72083 y Fz(16)28054 72395 y Fr(http://codespeak.net/svn/pypy/trunk)p +Black Black Black eop end +%%Page: 9 9 +TeXDict begin 9 8 bop Black Black Black Black Black 2853 +44 46086 45 v 2831 1041 45 997 v 9667 1041 V 9889 1041 +V 10575 742 a Fw(num)221 b(loops)p 14932 1041 V 2089 +w(ne)-22 b(w)1328 b(remo)-13 b(v)g(ed)p 22874 1041 V +2115 w(get/set)1329 b(remo)-13 b(v)g(ed)p 31702 1041 +V 1968 w(guard)1328 b(remo)-13 b(v)g(ed)p 40088 1041 +V 2090 w(all)222 b(ops)1328 b(remo)-13 b(v)g(ed)p 48916 +1041 V 2853 1085 46086 45 v 2831 2081 45 997 v 3517 1782 +a(crypto_p)k(yaes)p 9667 2081 V 9888 2081 V 5174 w(78)p +14932 2081 V 1771 w(3088)2776 b(50\045)p 22874 2081 V +2213 w(57148)g(25\045)p 31702 2081 V 2213 w(9055)h(95\045)p +40088 2081 V 1770 w(137189)f(80\045)p 48916 2081 V 2831 +3077 V 3517 2778 a(django)p 9667 3077 V 9888 3077 V 7476 +w(51)p 14932 3077 V 2213 w(673)h(54\045)p 22874 3077 +V 2213 w(19318)f(18\045)p 31702 3077 V 2213 w(3876)h(93\045)p +40088 3077 V 2213 w(55682)f(85\045)p 48916 3077 V 2831 +4074 V 3517 3775 a(f)-9 b(annkuch)p 9667 4074 V 9888 +4074 V 6600 w(43)p 14932 4074 V 2213 w(171)2777 b(49\045)p +22874 4074 V 3099 w(886)f(63\045)p 31702 4074 V 2213 +w(1159)h(81\045)p 40088 4074 V 2656 w(4935)f(45\045)p +48916 4074 V 2831 5070 V 3517 4771 a(go)p 9667 5070 V +9888 5070 V 8558 w(517)p 14932 5070 V 1328 w(12234)g(76\045)p +22874 5070 V 1770 w(200842)g(21\045)p 31702 5070 V 1771 +w(53138)g(90\045)p 40088 5070 V 1770 w(568542)g(84\045)p +48916 5070 V 2831 6066 V 3517 5767 a(html5lib)p 9667 +6066 V 9888 6066 V 6442 w(498)p 14932 6066 V 1328 w(14432)g(68\045)p +22874 6066 V 1770 w(503390)g(11\045)p 31702 6066 V 1771 +w(71592)g(94\045)p 40088 6066 V 1327 w(1405780)g(91\045)p +48916 6066 V 2831 7062 V 3517 6764 a(meteor)-18 b(-contest)p +9667 7062 V 9888 7062 V 4643 w(59)p 14932 7062 V 2213 +w(277)2777 b(36\045)p 22874 7062 V 2656 w(4402)f(31\045)p +31702 7062 V 2213 w(1078)h(83\045)p 40088 7062 V 2213 +w(12862)f(68\045)p 48916 7062 V 2831 8059 V 3517 7760 +a(nbody)p 9667 8059 V 9888 8059 V 7672 w(13)p 14932 8059 +V 2656 w(96)h(38\045)p 22874 8059 V 3099 w(443)f(69\045)p +31702 8059 V 2656 w(449)h(78\045)p 40088 8059 V 2656 +w(2107)f(38\045)p 48916 8059 V 2831 9055 V 3517 8756 +a(p)-9 b(y\003ate-f)g(ast)p 9667 9055 V 9888 9055 V 5479 +w(162)p 14932 9055 V 1771 w(2278)2776 b(55\045)p 22874 +9055 V 2213 w(39126)g(20\045)p 31702 9055 V 2213 w(8194)h(92\045)p +40088 9055 V 1770 w(112857)f(80\045)p 48916 9055 V 2831 +10051 V 3517 9752 a(raytrace-simple)p 9667 10051 V 9888 +10051 V 3937 w(120)p 14932 10051 V 1771 w(3118)g(59\045)p +22874 10051 V 2213 w(91982)g(15\045)p 31702 10051 V 1771 +w(13572)g(95\045)p 40088 10051 V 1770 w(247436)g(89\045)p +48916 10051 V 2831 11047 V 3517 10749 a(richards)p 9667 +11047 V 9888 11047 V 7035 w(87)p 14932 11047 V 2213 w(844)3220 +b(4\045)p 22874 11047 V 2213 w(49875)2776 b(22\045)p +31702 11047 V 2213 w(4130)h(91\045)p 40088 11047 V 1770 +w(133898)f(83\045)p 48916 11047 V 2831 12044 V 3517 11745 +a(spambayes)p 9667 12044 V 9888 12044 V 5559 w(314)p +14932 12044 V 1771 w(5608)g(79\045)p 22874 12044 V 1770 +w(117002)g(11\045)p 31702 12044 V 1771 w(25313)g(94\045)p +40088 12044 V 1770 w(324125)g(90\045)p 48916 12044 V +2831 13040 V 3517 12741 a(spectral-norm)p 9667 13040 +V 9888 13040 V 4969 w(38)p 14932 13040 V 2213 w(360)h(64\045)p +22874 13040 V 2656 w(5553)f(20\045)p 31702 13040 V 2213 +w(1122)h(92\045)p 40088 13040 V 2213 w(11878)f(77\045)p +48916 13040 V 2831 14036 V 3517 13737 a(telco)p 9667 +14036 V 9888 14036 V 8166 w(46)p 14932 14036 V 1771 w(1257)g(90\045)p +22874 14036 V 2213 w(37470)3219 b(3\045)p 31702 14036 +V 2213 w(6644)2777 b(99\045)p 40088 14036 V 2213 w(98590)f(97\045)p +48916 14036 V 2831 15033 V 3517 14734 a(twisted-names)p +9667 15033 V 9888 15033 V 4330 w(214)p 14932 15033 V +1771 w(5273)g(84\045)p 22874 15033 V 1770 w(100010)g(10\045)p +31702 15033 V 1771 w(23247)g(96\045)p 40088 15033 V 1770 +w(279667)g(92\045)p 48916 15033 V 2853 15077 46086 45 +v 2831 16073 45 997 v 3517 15774 a(total)p 9667 16073 +V 9888 16073 V 7427 w(2240)p 14932 16073 V 1328 w(49709)g(70\045)p +22874 16073 V 1328 w(1227447)f(14\045)p 31702 16073 V +1328 w(222569)h(93\045)p 40088 16073 V 1327 w(3395548)g(89\045)p +48916 16073 V 2853 16117 46086 45 v -2000 17372 55791 +37 v 10632 18501 a Fs(Figur)-18 b(e)249 b(7.)499 b FC(Number)249 +b(of)g(Operations)h(and)g(Percentage)g(Remo)-15 b(v)g(ed)250 +b(By)f(Optimization)p Black Black Black Black -790 20753 +53371 45 v -812 21749 45 997 v 6322 21749 V 6544 21749 +V 8773 21451 a Fw(CPython)221 b([ms])p 14360 21749 V +2589 w Fm(\002)p 17681 21749 V 17902 21749 V 4077 w Fw(Psyco)h([ms])p +25719 21749 V 2147 w Fm(\002)p 28597 21749 V 28818 21749 +V 1549 w Fw(PyPy)g(w/o)g(optim.)g([ms])p 37921 21749 +V 2146 w Fm(\002)p 40799 21749 V 41020 21749 V 1550 w +Fw(PyPy)f(w/)h(optim.)g([ms])p 49680 21749 V 2146 w Fm(\002)p +52558 21749 V -790 21794 53371 45 v -812 22790 45 997 +v -126 22491 a Fw(crypto_p)-9 b(yaes)p 6322 22790 V 6544 +22790 V 3529 w(2757.80)220 b Fm(\006)h Fw(0.98)p 14360 +22790 V 1328 w(10.33)p 17681 22790 V 17902 22790 V 3321 +w(67.90)g Fm(\006)g Fw(0.47)p 25719 22790 V 1328 w(0.25)p +28597 22790 V 28818 22790 V 3721 w(1652.00)g Fm(\006)g +Fw(4.00)p 37921 22790 V 1328 w(6.19)p 40799 22790 V 41020 +22790 V 3722 w(266.86)f Fm(\006)h Fw(5.94)p 49680 22790 +V 1329 w(1.00)p 52558 22790 V -812 23786 V -126 23487 +a(django)p 6322 23786 V 6544 23786 V 6273 w(993.19)g +Fm(\006)g Fw(0.50)p 14360 23786 V 1771 w(3.83)p 17681 +23786 V 17902 23786 V 2878 w(913.51)g Fm(\006)g Fw(4.22)p +25719 23786 V 1328 w(3.52)p 28597 23786 V 28818 23786 +V 4164 w(694.73)g Fm(\006)g Fw(2.86)p 37921 23786 V 1328 +w(2.68)p 40799 23786 V 41020 23786 V 3722 w(259.53)f +Fm(\006)h Fw(1.79)p 49680 23786 V 1329 w(1.00)p 52558 +23786 V -812 24783 V -126 24484 a(f)-9 b(annkuch)p 6322 +24783 V 6544 24783 V 4955 w(1987.22)220 b Fm(\006)h Fw(2.02)p +14360 24783 V 1771 w(4.26)p 17681 24783 V 17902 24783 +V 2878 w(944.44)g Fm(\006)g Fw(0.61)p 25719 24783 V 1328 +w(2.02)p 28597 24783 V 28818 24783 V 4164 w(566.99)g +Fm(\006)g Fw(1.06)p 37921 24783 V 1328 w(1.21)p 40799 +24783 V 41020 24783 V 3722 w(466.87)f Fm(\006)h Fw(1.85)p +49680 24783 V 1329 w(1.00)p 52558 24783 V -812 25779 +V -126 25480 a(go)p 6322 25779 V 6544 25779 V 7798 w(947.21)g +Fm(\006)g Fw(1.58)p 14360 25779 V 1771 w(3.00)p 17681 +25779 V 17902 25779 V 2878 w(445.96)g Fm(\006)g Fw(0.68)p +25719 25779 V 1328 w(1.41)p 28597 25779 V 28818 25779 +V 3279 w(2197.71)f Fm(\006)h Fw(25.21)p 37921 25779 V +1328 w(6.95)p 40799 25779 V 41020 25779 V 3722 w(316.15)f +Fm(\006)h Fw(9.33)p 49680 25779 V 1329 w(1.00)p 52558 +25779 V -812 26775 V -126 26476 a(html5lib)p 6322 26775 +V 6544 26775 V 4354 w(13987.12)f Fm(\006)h Fw(19.51)p +14360 26775 V 1771 w(1.39)p 17681 26775 V 17902 26775 +V 1550 w(17398.25)f Fm(\006)h Fw(36.50)p 25719 26775 +V 1328 w(1.72)p 28597 26775 V 28818 26775 V 2836 w(27194.45)f +Fm(\006)h Fw(46.62)p 37921 26775 V 1328 w(2.69)p 40799 +26775 V 41020 26775 V 2393 w(10092.19)f Fm(\006)i Fw(23.50)p +49680 26775 V 1328 w(1.00)p 52558 26775 V -812 27771 +V -126 27472 a(meteor)-18 b(-contest)p 6322 27771 V 6544 +27771 V 3440 w(346.98)221 b Fm(\006)g Fw(0.35)p 14360 +27771 V 1771 w(0.88)p 17681 27771 V 17902 27771 V 2878 +w(215.66)g Fm(\006)g Fw(0.23)p 25719 27771 V 1328 w(0.55)p +28597 27771 V 28818 27771 V 4164 w(433.04)g Fm(\006)g +Fw(1.45)p 37921 27771 V 1328 w(1.10)p 40799 27771 V 41020 +27771 V 3722 w(392.85)f Fm(\006)h Fw(0.87)p 49680 27771 +V 1329 w(1.00)p 52558 27771 V -812 28768 V -126 28469 +a(nbody_modi\002ed)p 6322 28768 V 6544 28768 V 2877 w(637.90)g +Fm(\006)g Fw(1.82)p 14360 28768 V 1771 w(6.14)p 17681 +28768 V 17902 28768 V 2878 w(256.78)g Fm(\006)g Fw(0.18)p +25719 28768 V 1328 w(2.47)p 28597 28768 V 28818 28768 +V 4164 w(135.55)g Fm(\006)g Fw(0.33)p 37921 28768 V 1328 +w(1.30)p 40799 28768 V 41020 28768 V 3722 w(103.93)f +Fm(\006)h Fw(0.25)p 49680 28768 V 1329 w(1.00)p 52558 +28768 V -812 29764 V -126 29465 a(p)-9 b(y\003ate-f)g(ast)p +6322 29764 V 6544 29764 V 4277 w(3169.35)220 b Fm(\006)h +Fw(1.89)p 14360 29764 V 1771 w(1.74)p 17681 29764 V 17902 +29764 V 2435 w(1278.16)g Fm(\006)g Fw(3.13)p 25719 29764 +V 1328 w(0.70)p 28597 29764 V 28818 29764 V 3721 w(3285.89)g +Fm(\006)g Fw(8.51)p 37921 29764 V 1328 w(1.80)p 40799 +29764 V 41020 29764 V 2836 w(1822.36)f Fm(\006)i Fw(11.52)p +49680 29764 V 1328 w(1.00)p 52558 29764 V -812 30760 +V -126 30461 a(raytrace-simple)p 6322 30760 V 6544 30760 +V 2292 w(2744.60)e Fm(\006)h Fw(51.72)p 14360 30760 V +1771 w(4.24)p 17681 30760 V 17902 30760 V 2435 w(1072.66)g +Fm(\006)g Fw(1.08)p 25719 30760 V 1328 w(1.66)p 28597 +30760 V 28818 30760 V 3279 w(2778.27)f Fm(\006)h Fw(15.13)p +37921 30760 V 1328 w(4.29)p 40799 30760 V 41020 30760 +V 3722 w(647.24)f Fm(\006)h Fw(5.44)p 49680 30760 V 1329 +w(1.00)p 52558 30760 V -812 31756 V -126 31458 a(richards)p +6322 31756 V 6544 31756 V 5832 w(354.06)g Fm(\006)g Fw(1.00)p +14360 31756 V 1771 w(4.01)p 17681 31756 V 17902 31756 +V 3321 w(63.48)g Fm(\006)g Fw(0.15)p 25719 31756 V 1328 +w(0.72)p 28597 31756 V 28818 31756 V 4164 w(383.93)g +Fm(\006)g Fw(3.28)p 37921 31756 V 1328 w(4.35)p 40799 +31756 V 41020 31756 V 4164 w(88.32)g Fm(\006)g Fw(0.91)p +49680 31756 V 1329 w(1.00)p 52558 31756 V -812 32753 +V -126 32454 a(spambayes)p 6322 32753 V 6544 32753 V +4799 w(299.16)g Fm(\006)g Fw(0.35)p 14360 32753 V 1771 +w(0.75)p 17681 32753 V 17902 32753 V 2878 w(338.68)g +Fm(\006)g Fw(3.14)p 25719 32753 V 1328 w(0.85)p 28597 +32753 V 28818 32753 V 3721 w(580.90)g Fm(\006)g Fw(24.68)p +37921 32753 V 1328 w(1.46)p 40799 32753 V 41020 32753 +V 3279 w(397.37)f Fm(\006)i Fw(10.60)p 49680 32753 V +1328 w(1.00)p 52558 32753 V -812 33749 V -126 33450 a(spectral-norm)p +6322 33749 V 6544 33749 V 3766 w(478.63)f Fm(\006)g Fw(0.80)p +14360 33749 V 1771 w(4.27)p 17681 33749 V 17902 33749 +V 2878 w(139.83)g Fm(\006)g Fw(1.54)p 25719 33749 V 1328 +w(1.25)p 28597 33749 V 28818 33749 V 4164 w(353.51)g +Fm(\006)g Fw(1.39)p 37921 33749 V 1328 w(3.15)p 40799 +33749 V 41020 33749 V 3722 w(112.10)f Fm(\006)h Fw(1.17)p +49680 33749 V 1329 w(1.00)p 52558 33749 V -812 34745 +V -126 34446 a(telco)p 6322 34745 V 6544 34745 V 6521 +w(1207.67)f Fm(\006)h Fw(2.03)p 14360 34745 V 1771 w(2.44)p +17681 34745 V 17902 34745 V 2878 w(730.00)g Fm(\006)g +Fw(2.66)p 25719 34745 V 1328 w(1.47)p 28597 34745 V 28818 +34745 V 3721 w(1296.08)g Fm(\006)g Fw(4.37)p 37921 34745 +V 1328 w(2.62)p 40799 34745 V 41020 34745 V 3722 w(495.23)f +Fm(\006)h Fw(2.14)p 49680 34745 V 1329 w(1.00)p 52558 +34745 V -812 35741 V -126 35443 a(twisted_names)p 6322 +35741 V 6544 35741 V 4308 w(9.58)g Fm(\006)g Fw(0.01)p +14360 35741 V 1771 w(1.34)p 17681 35741 V 17902 35741 +V 3321 w(10.43)g Fm(\006)g Fw(0.01)p 25719 35741 V 1328 +w(1.46)p 28597 35741 V 28818 35741 V 4607 w(17.99)g Fm(\006)g +Fw(0.27)p 37921 35741 V 1328 w(2.52)p 40799 35741 V 41020 +35741 V 4607 w(7.13)g Fm(\006)g Fw(0.09)p 49680 35741 +V 1329 w(1.00)p 52558 35741 V -790 35786 53371 45 v -2000 +37040 55791 37 v 6051 38169 a Fs(Figur)-18 b(e)250 b(8.)498 +b FC(Benchmark)250 b(T)-35 b(imes)248 b(in)i(Milliseconds,)e(T)-80 +b(ogether)250 b(W)-40 b(ith)250 b(F)-15 b(actor)250 b(Ov)-15 +b(er)249 b(PyPy)g(W)-40 b(ith)250 b(Optimizations)p Black +-2000 41484 a FA(7.)1218 b(Related)305 b(W)-91 b(ork)-2000 +43034 y FC(There)373 b(e)-15 b(xists)373 b(a)g(lar)-18 +b(ge)374 b(number)f(of)g(w)-10 b(orks)373 b(on)g(escape)h(analysis,)f +(which)-2000 44141 y(is)438 b(a)h(program)f(analysis)h(that)g(tries)f +(to)g(\002nd)h(an)g(upper)f(bound)i(for)e(the)-2000 45248 +y(lifetime)480 b(of)g(objects)h(allocated)g(at)g(speci\002c)f(program)h +(points)f([4,)g(11,)-2000 46355 y(18,)348 b(24].)g(This)f(information)g +(can)i(then)f(be)g(used)g(to)f(decide)i(that)f(certain)-2000 +47462 y(objects)341 b(can)h(be)g(allocated)g(on)f(the)h(stack,)f +(because)h(their)g(lifetime)f(does)-2000 48569 y(not)269 +b(e)-15 b(xceed)271 b(that)f(of)f(the)g(stack)h(frame)f(it)g(is)g +(allocated)h(in.)g(The)f(dif)-25 b(ference)-2000 49676 +y(to)303 b(our)h(w)-10 b(ork)303 b(is)g(that)h(escape)g(analysis)f(is)g +(split)f(into)i(an)g(analysis)f(and)h(an)-2000 50783 +y(optimization)327 b(phase.)f(The)g(analysis)f(can)i(be)f(a)g(lot)g +(more)g(comple)-15 b(x)326 b(than)-2000 51890 y(our)388 +b(simple)g(one-pass)g(optimization.)h(Also,)f(stack-allocation)h +(reduces)-2000 52997 y(g)-5 b(arbage-collection)293 b(pressure)d(b)-20 +b(ut)291 b(does)g(not)g(optimize)g(a)-15 b(w)-10 b(ay)292 +b(the)f(actual)-2000 54104 y(accesses)283 b(to)f(the)h(stack-allocated) +h(object.)f(In)f(our)h(case,)g(an)g(object)g(is)f(not)-2000 +55211 y(needed)250 b(at)g(all)f(an)-15 b(y)250 b(more.)-672 +56318 y(Chang)219 b Fx(et)g(al.)f FC(describe)h(a)f(tracing)h(JIT)f +(for)g(Ja)-20 b(v)-25 b(aScript)219 b(running)f(on)h(top)-2000 +57425 y(of)276 b(a)g(JVM)g([10].)f(The)-15 b(y)277 b(mention)f(in)g +(passing)g(an)h(approach)g(to)f(allocation)-2000 58531 +y(remo)-15 b(v)-25 b(al)399 b(that)g(mo)-15 b(v)g(es)398 +b(the)h(allocation)g(of)f(an)h(object)g(of)f(type)g(1)h(out)f(of)-2000 +59638 y(the)422 b(loop)g(to)g(only)g(allocate)g(it)g(once,)h(instead)f +(of)f(e)-25 b(v)-15 b(ery)422 b(iteration.)h(No)-2000 +60745 y(details)196 b(are)g(gi)-25 b(v)-15 b(en)197 b(for)e(this)h +(optimization.)g(The)g(f)-10 b(act)197 b(that)f(the)g(object)g(is)g +(still)-2000 61852 y(allocated)238 b(and)g(needs)f(to)g(be)g(written)g +(to)g(means)h(that)f(only)g(the)g(allocations)-2000 62959 +y(are)377 b(optimized)g(a)-15 b(w)-10 b(ay)-65 b(,)377 +b(b)-20 b(ut)377 b(not)f(the)h(reads)f(out)g(of)h(and)f(writes)g(into)h +(the)-2000 64066 y(object.)-672 65173 y(SPUR,)282 b(a)g(tracing)h(JIT)e +(for)g(C#)h(seems)f(to)h(be)h(able)f(to)g(remo)-15 b(v)g(e)283 +b(alloca-)-2000 66280 y(tions)294 b(in)g(a)h(similar)e(w)-10 +b(ay)295 b(to)f(the)h(approach)g(described)g(here,)g(as)f(hinted)h(at) +-2000 67387 y(in)212 b(the)f(technical)i(report)f([3].)f(Ho)-25 +b(we)g(v)-15 b(er)-40 b(,)213 b(no)f(details)g(for)f(the)h(approach)g +(and)-2000 68494 y(its)249 b(implementation)h(are)g(gi)-25 +b(v)-15 b(en.)-672 69601 y(Psyco)294 b([25])f(is)h(a)g(\(non-tracing\)) +g(JIT)e(for)h(Python)i(that)f(implements)g(a)-2000 70708 +y(more)262 b(ad-hoc)h(v)-15 b(ersion)263 b(of)f(the)g(allocation)h +(remo)-15 b(v)-25 b(al)263 b(described)g(here.)g(Our)-2000 +71815 y(static)292 b(objects)h(could)g(be)f(related)h(to)g(what)f(are)h +(called)g Fx(virtual)g FC(objects)g(in)p Black Black +27224 41484 a(Psyco.)328 b(Historically)-65 b(,)328 b(PyPy')-55 +b(s)328 b(JIT)e(can)j(be)f(seen)h(as)e(some)h(successor)g(of)27224 +42591 y(Psyco)278 b(for)g(a)h(general)g(conte)-15 b(xt)279 +b(\(one)g(of)f(the)h(authors)f(of)h(this)f(paper)h(is)e(the)27224 +43698 y(author)249 b(of)g(Psyco\).)28552 44805 y(The)493 +b(original)f(SELF)g(JIT)f(compiler)i([9])f(used)h(an)f(algorithm)h(for) +27224 45912 y(forw)-10 b(ard-propag)-5 b(ating)295 b(the)f(types)h(of)f +(v)-25 b(ariables)294 b(as)g(part)h(of)f(its)f(optimiza-)27224 +47019 y(tions.)206 b(This)f(mak)-10 b(es)207 b(it)f(possible)g(to)h +(remo)-15 b(v)g(e)207 b(all)g(type)f(checks)i(on)e(a)h(v)-25 +b(ariable)27224 48126 y(b)-20 b(ut)247 b(the)h(\002rst)f(one.)h(The)f +(optimization)i(does)e(not)h(deal)g(with)f(remo)-15 b(ving)248 +b(the)27224 49233 y(full)253 b(object,)h(if)e(it)h(is)g(short-li)-25 +b(v)-15 b(ed,)253 b(b)-20 b(ut)253 b(the)h(type)f(check)i(remo)-15 +b(v)-25 b(als)253 b(are)g(sim-)27224 50340 y(ilar)c(to)g(what)h(our)f +(optimization)h(achie)-25 b(v)-15 b(es.)28552 51447 y(P)g(artially)204 +b(kno)-25 b(wn)205 b(data)f(structures)g(are)g(b)-20 +b(uilt)204 b(directly)g(into)g(Prolog)g(\(via)27224 52554 +y(unbound)317 b(logic)h(v)-25 b(ariables\))317 b(and)g(thus)g(the)g +(treatment)h(of)e(partially)i(static)27224 53661 y(data)358 +b(structures)f(w)-10 b(as)358 b(part)g(of)g(partial)g(e)-25 +b(v)g(aluation)360 b(of)d(Prolog)h(programs)27224 54768 +y(from)239 b(the)i(early)f(stages)g([21].)g(One)h(ef)-25 +b(fect)240 b(of)g(unfolding)h(in)f(Prolog)g(is)f(that)27224 +55875 y(terms)350 b(that)h(are)g(constructed)h(and)f(immediately)h +(matched)g(ag)-5 b(ain,)351 b(com-)27224 56982 y(pletely)336 +b(disappear)f(in)h(the)f(residual)h(program.)f(This)g(is)f(similar)h +(to)g(what)27224 58089 y(our)417 b(optimization)h(does)f(for)g(an)h +(imperati)-25 b(v)-15 b(e)418 b(language.)h(In)e(functional)27224 +59196 y(programming)216 b(this)f(idea)h(w)-10 b(as)216 +b(introduced)g(as)g(constructor)f(specialisation)27224 +60303 y(by)249 b(Mogensen)h([23].)28552 61410 y(A)453 +b(related)g(optimization)g(is)f(also)g(that)h(of)f(deforestation)h +([17,)g(27])27224 62517 y(which)306 b(remo)-15 b(v)g(es)305 +b(intermediate)i(lists)d(or)h(trees)g(in)h(functional)g(languages.) +27224 63623 y(A)290 b(more)f(general)i(approach)g(is)e(boxing)h +(analysis)g([20])f(which)h(optimizes)27224 64730 y(pairs)437 +b(of)h(calls)g(to)g(box/unbox)h(in)f(a)g(functional)h(language.)g +(Similarly)-65 b(,)27224 65837 y("dynamic)464 b(typing")g([19])f(tries) +g(to)g(remo)-15 b(v)g(e)463 b(dynamic)i(type)e(coercions)27224 +66944 y(in)384 b(a)g(dynamically)h(typed)g(lambda-calculus.)g(All)f +(these)g(optimizations)27224 68051 y(w)-10 b(ork)433 +b(by)f(analyzing)i(the)f(program)g(before)g(e)-15 b(x)g(ecution,)434 +b(which)f(mak)-10 b(es)27224 69158 y(them)282 b(unsuitable)g(for)g +(dynamic)h(languages)g(lik)-10 b(e)282 b(Python,)g(where)h(almost)27224 +70265 y(nothing)249 b(can)h(be)g(inferred)f(purely)g(by)h(looking)f(at) +h(the)f(source)h(code.)p Black Black eop end +%%Page: 10 10 +TeXDict begin 10 9 bop Black Black -2000 886 a FA(8.)1218 +b(Conclusion)304 b(and)h(Futur)-22 b(e)304 b(W)-91 b(ork)-2000 +2435 y FC(In)368 b(this)g(paper)-40 b(,)370 b(we)f(used)g(an)g +(approach)h(based)e(on)h(online)g(partial)g(e)-25 b(v)g(al-)-2000 +3542 y(uation)330 b(to)g(optimize)g(a)-15 b(w)-10 b(ay)331 +b(allocations)f(and)g(type)g(guards)f(in)h(the)g(traces)-2000 +4649 y(of)273 b(a)g(tracing)h(JIT)-74 b(.)272 b(In)h(this)g(conte)-15 +b(xt)274 b(a)f(simple)g(approach)i(based)e(on)h(partial)-2000 +5756 y(e)-25 b(v)g(aluation)229 b(gi)-25 b(v)-15 b(es)228 +b(good)g(results.)f(This)g(is)g(due)i(to)e(the)h(f)-10 +b(act)228 b(that)g(the)g(tracing)-2000 6863 y(JIT)243 +b(itself)h(is)g(responsible)g(for)g(all)h(control)g(issues,)e(which)i +(are)g(usually)g(the)-2000 7970 y(hardest)327 b(part)g(of)f(partial)h +(e)-25 b(v)g(aluation:)328 b(the)f(tracing)g(JIT)f(selects)g(the)h +(parts)-2000 9077 y(of)260 b(the)h(program)g(that)g(are)g(w)-10 +b(orthwhile)261 b(to)f(optimize,)i(and)f(e)-15 b(xtracts)261 +b(linear)-2000 10184 y(paths)250 b(through)h(them,)g(inlining)f +(functions)h(as)f(necessary)-65 b(.)251 b(What)g(is)f(left)g(to)-2000 +11291 y(optimize)g(are)f(only)h(those)f(linear)h(paths.)-672 +12398 y(W)-80 b(e)326 b(e)-15 b(xpect)326 b(a)f(similar)f(result)g(for) +g(other)h(optimizations)g(that)g(usually)-2000 13505 +y(require)236 b(a)g(comple)-15 b(x)236 b(analysis)g(phase)g(and)g(are)g +(thus)f(normally)h(too)f(slo)-25 b(w)236 b(to)-2000 14612 +y(use)295 b(at)g(runtime.)g(A)g(tracing)g(JIT)f(selects)h(interesting)f +(linear)i(paths)e(by)h(it-)-2000 15719 y(self;)215 b(therefore,)g(a)g +(nai)-25 b(v)-15 b(e)216 b(v)-15 b(ersion)215 b(of)g(man)-15 +b(y)216 b(optimizations)g(on)f(such)g(paths)-2000 16826 +y(should)207 b(gi)-25 b(v)-15 b(e)207 b(mostly)f(the)h(same)g(results.) +f(F)-15 b(or)206 b(e)-15 b(xample,)208 b(we)f(e)-15 b(xperimented)-2000 +17933 y(with)324 b(\(and)g(plan)g(to)f(write)h(about\))g(store-load)f +(propag)-5 b(ation)325 b(with)f(a)g(v)-15 b(ery)-2000 +19040 y(simple)249 b(alias)g(analysis.)-2000 21902 y +FA(Ackno)-12 b(wledgements)-2000 23452 y FC(The)261 b(authors)h(w)-10 +b(ould)262 b(lik)-10 b(e)261 b(to)h(thank)f(Stef)-10 +b(an)262 b(Hallerstede,)g(Da)-20 b(vid)262 b(Schnei-)-2000 +24559 y(der)351 b(and)h(Thomas)f(Stiehl)g(for)g(fruitful)f(discussions) +g(and)i(detailed)g(feed-)-2000 25666 y(back)284 b(during)g(the)f +(writing)h(of)f(the)h(paper)-55 b(.)284 b(W)-80 b(e)284 +b(thank)g(the)g(anon)-15 b(ymous)284 b(re-)-2000 26773 +y(vie)-25 b(wers)249 b(for)g(the)h(v)-25 b(aluable)250 +b(comments.)-2000 29635 y FA(Refer)-22 b(ences)p Black +-1557 31074 a Fw([1])p Black 387 w(D.)202 b(Ancona,)g(M.)h(Ancona,)f +(A.)g(Cuni,)f(and)h(N.)g(D.)g(Matsakis.)278 b(RPython:)202 +b(a)f(step)-137 32070 y(to)-22 b(w)-9 b(ards)298 b(reconciling)g +(dynamically)g(and)f(statically)j(typed)d(OO)h(languages.)-137 +33066 y(In)239 b Fa(Pr)-40 b(oceedings)241 b(of)f(the)g(2007)f +(symposium)i(on)f(Dynamic)h(langua)-9 b(g)g(es)p Fw(,)240 +b(pages)-137 34063 y(53\22664,)220 b(Montreal,)j(Quebec,)f(Canada,)g +(2007.)f(A)-35 b(CM.)p Black -1557 35444 a([2])p Black +387 w(V)-114 b(.)377 b(Bala,)f(E.)h(Duesterw)-9 b(ald,)379 +b(and)d(S.)h(Banerjia.)815 b(Dynamo:)376 b(a)h(transparent)-137 +36440 y(dynamic)300 b(optimization)h(system.)574 b Fa(A)-27 +b(CM)301 b(SIGPLAN)f(Notices)p Fw(,)i(35\(5\):1\22612,)-137 +37436 y(2000.)p Black -1557 38817 a([3])p Black 387 w(M.)398 +b(Bebenita,)g(F)-71 b(.)398 b(Brandner)-35 b(,)397 b(M.)h(F)-13 +b(ahndrich,)398 b(F)-71 b(.)398 b(Logozzo,)g(W)-81 b(.)397 +b(Schulte,)-137 39814 y(N.)322 b(T)-31 b(illmann,)323 +b(and)e(H.)i(V)-98 b(enter)-49 b(.)640 b(SPUR:)323 b(a)f(trace-based)h +(JIT)f(compiler)f(for)-137 40810 y(CIL.)744 b(In)354 +b Fa(Pr)-40 b(oceedings)355 b(of)f(the)g(A)-27 b(CM)354 +b(international)h(confer)-33 b(ence)355 b(on)f(Ob-)-137 +41806 y(ject)199 b(oriented)h(pr)-40 b(o)-9 b(gr)c(amming)199 +b(systems)i(langua)-9 b(g)g(es)199 b(and)f(applications)p +Fw(,)i(pages)-137 42802 y(708\226725,)220 b(Reno/T)-71 +b(ahoe,)222 b(Ne)-22 b(v)g(ada,)222 b(USA,)h(2010.)d(A)-35 +b(CM.)p Black -1557 44184 a([4])p Black 387 w(B.)330 +b(Blanchet.)668 b(Escape)331 b(analysis)g(for)f(Ja)-18 +b(v)c(a:)331 b(Theory)f(and)g(practice.)668 b Fa(A)-27 +b(CM)-137 45180 y(T)-49 b(r)-13 b(ans.)222 b(Pr)-40 b(o)-9 +b(gr)c(am.)223 b(Lang)-13 b(.)221 b(Syst.)p Fw(,)i(25\(6\):713\226775,) +d(2003.)p Black -1557 46561 a([5])p Black 387 w(C.)257 +b(F)-71 b(.)258 b(Bolz,)g(A.)g(Cuni,)f(M.)h(Fija\007k)-9 +b(o)-22 b(wski,)259 b(and)e(A.)h(Rigo.)434 b(T)-31 b(racing)257 +b(the)h(meta-)-137 47557 y(le)-22 b(v)-13 b(el:)183 b(PyPy')-49 +b(s)183 b(tracing)g(JIT)g(compiler)-49 b(.)236 b(In)182 +b Fa(Pr)-40 b(oceedings)184 b(of)f(the)g(4th)f(workshop)-137 +48553 y(on)197 b(the)i(Implementation,)g(Compilation,)f(Optimization)h +(of)f(Object-Oriented)-137 49550 y(Langua)-9 b(g)g(es)342 +b(and)g(Pr)-40 b(o)-9 b(gr)c(amming)343 b(Systems)p Fw(,)h(pages)f +(18\22625,)f(Geno)-13 b(v)-22 b(a,)342 b(Italy)-58 b(,)-137 +50546 y(2009.)221 b(A)-35 b(CM.)p Black -1557 51927 a([6])p +Black 387 w(C.)180 b(F)-71 b(.)181 b(Bolz,)f(A.)h(K)-13 +b(uhn,)180 b(A.)h(Lienhard,)f(N.)h(Matsakis,)h(O.)f(Nierstrasz,)h(L.)f +(Reng-)-137 52923 y(gli,)f(A.)h(Rigo,)f(and)g(T)-66 b(.)181 +b(V)-98 b(erw)-9 b(aest.)231 b(Back)180 b(to)g(the)g(future)g(in)g(one) +g(week)g(\227)g(imple-)-137 53919 y(menting)272 b(a)h(Smalltalk)h(VM)f +(in)g(PyPy.)483 b(In)272 b Fa(Self-Sustaining)h(Systems)p +Fw(,)i(pages)-137 54916 y(123\226139.)220 b(2008.)p Black +-1557 56297 a([7])p Black 387 w(C.)343 b(F)-71 b(.)344 +b(Bolz,)g(M.)g(Leuschel,)g(and)f(D.)h(Schneider)-49 b(.)710 +b(T)-71 b(o)-22 b(w)-9 b(ards)344 b(a)g(jitting)f(VM)-137 +57293 y(for)278 b(Prolog)h(e)-13 b(x)g(ecution.)502 b(In)279 +b Fa(Pr)-40 b(oceedings)280 b(of)f(the)g(12th)f(international)h(A)-27 +b(CM)-137 58289 y(SIGPLAN)292 b(symposium)i(on)e(Principles)i(and)d(pr) +-13 b(actice)293 b(of)g(declar)-13 b(ative)293 b(pr)-40 +b(o-)-137 59286 y(gr)-13 b(amming)p Fw(,)222 b(pages)g(99\226108,)e +(Hagenber)-16 b(g,)222 b(Austria,)h(2010.)e(A)-35 b(CM.)p +Black -1557 60667 a([8])p Black 387 w(C.)338 b(Bruni)g(and)g(T)-66 +b(.)339 b(V)-98 b(erw)-9 b(aest.)695 b(PyGirl:)339 b(generating)f +(Whole-System)h(VMs)-137 61663 y(from)241 b(High-Le)-22 +b(v)-13 b(el)241 b(prototypes)g(using)h(PyPy.)383 b(In)241 +b(W)-81 b(.)241 b(Aalst,)j(J.)e(Mylopoulos,)-137 62659 +y(N.)380 b(M.)h(Sadeh,)f(M.)g(J.)h(Sha)-13 b(w)-58 b(,)381 +b(C.)e(Szyperski,)i(M.)f(Oriol,)h(and)e(B.)h(Me)-13 b(yer)-35 +b(,)-137 63655 y(editors,)394 b Fa(Objects,)h(Components,)f(Models)f +(and)g(P)-71 b(atterns)p Fw(,)395 b(v)-18 b(olume)393 +b(33)g(of)-137 64652 y Fa(Lectur)-33 b(e)364 b(Notes)g(in)e(Business)j +(Information)e(Pr)-40 b(ocessing)p Fw(,)365 b(pages)e(328\226347.)-137 +65648 y(Springer)221 b(Berlin)g(Heidelber)-16 b(g,)223 +b(2009.)318 b(10.1007/978-3-642-02571-6_19.)p Black -1557 +67029 a([9])p Black 387 w(C.)276 b(Chambers,)g(D.)g(Ung)l(ar)-35 +b(,)276 b(and)f(E.)i(Lee.)493 b(An)276 b(ef)-22 b(\002cient)277 +b(implementation)f(of)-137 68025 y(SELF)283 b(a)g(dynamically-typed)g +(object-oriented)g(language)f(based)h(on)g(proto-)-137 +69022 y(types.)319 b Fa(SIGPLAN)222 b(Not.)p Fw(,)g(24\(10\):49\22670,) +e(1989.)p Black -2000 70403 a([10])p Black 387 w(M.)230 +b(Chang,)g(M.)g(Bebenita,)g(A.)h(Y)-89 b(ermolo)-13 b(vich,)230 +b(A.)h(Gal,)g(and)e(M.)i(Franz.)346 b(Ef)-22 b(\002-)-137 +71399 y(cient)257 b(just-in-time)h(e)-13 b(x)g(ecution)256 +b(of)h(dynamically)g(typed)f(languages)h(via)g(code)-137 +72395 y(specialization)359 b(using)f(precise)h(runtime)f(type)g +(inference.)757 b(T)-62 b(echnical)358 b(Re-)p Black +Black 29087 886 a(port)c(ICS-TR-07-10,)g(Donald)g(Bren)h(School)f(of)h +(Information)f(and)g(Com-)29087 1882 y(puter)221 b(Science,)i(Uni)-22 +b(v)-13 b(ersity)221 b(of)g(California,)h(Irvine,)g(2007.)p +Black 27224 3210 a([11])p Black 387 w(J.)200 b(Choi,)g(M.)g(Gupta,)h +(M.)f(Serrano,)g(V)-114 b(.)201 b(C.)f(Sreedhar)-35 b(,)200 +b(and)f(S.)i(Midkif)-22 b(f.)272 b(Escape)29087 4206 +y(analysis)222 b(for)f(Ja)-18 b(v)c(a.)320 b Fa(SIGPLAN)222 +b(Not.)p Fw(,)g(34\(10\):1\22619,)e(1999.)p Black 27224 +5535 a([12])p Black 387 w(A.)273 b(Cuni.)483 b Fa(High)273 +b(performance)h(implementation)g(of)e(Python)h(for)g(CLI/.NET)29087 +6531 y(with)365 b(JIT)i(compiler)g(g)-9 b(ener)c(ation)366 +b(for)g(dynamic)h(langua)-9 b(g)g(es.)782 b Fw(PhD)366 +b(thesis,)29087 7527 y(Dipartimento)283 b(di)f(Informatica)h(e)g +(Scienze)h(dell'Informazione,)e(Uni)-22 b(v)-13 b(ersity)29087 +8524 y(of)221 b(Geno)-13 b(v)-22 b(a,)221 b(2010.)318 +b(T)-62 b(echnical)223 b(Report)d(DISI-TH-2010-05.)p +Black 27224 9852 a([13])p Black 387 w(A.)428 b(Gal,)g(B.)g(Eich,)g(M.)g +(Sha)-18 b(v)-13 b(er)-35 b(,)428 b(D.)g(Anderson,)g(B.)f(Kaplan,)i(G.) +f(Hoare,)29087 10848 y(D.)460 b(Mandelin,)g(B.)g(Zbarsk)-13 +b(y)-58 b(,)460 b(J.)h(Orendorf)-22 b(f,)459 b(M.)i(Bebenita,)f(M.)g +(Chang,)29087 11844 y(M.)176 b(Franz,)g(E.)g(Smith,)g(R.)g(Reitmaier) +-35 b(,)176 b(and)f(M.)h(Haghighat.)220 b(T)-31 b(race-based)176 +b(just-)29087 12841 y(in-time)221 b(type)h(specialization)h(for)e +(dynamic)g(languages.)319 b(In)222 b Fa(PLDI)p Fw(,)g(2009.)p +Black 27224 14169 a([14])p Black 387 w(A.)372 b(Gal)h(and)f(M.)g +(Franz.)801 b(Incremental)373 b(dynamic)f(code)f(generation)h(with) +29087 15165 y(trace)267 b(trees.)467 b(T)-62 b(echnical)269 +b(Report)d(ICS-TR-06-16,)h(Donald)g(Bren)g(School)g(of)29087 +16162 y(Information)297 b(and)h(Computer)g(Science,)h(Uni)-22 +b(v)-13 b(ersity)299 b(of)f(California,)h(Irvine,)29087 +17158 y(No)-13 b(v)-58 b(.)221 b(2006.)p Black 27224 +18486 a([15])p Black 387 w(A.)330 b(Gal,)h(C.)f(W)-81 +b(.)329 b(Probst,)i(and)e(M.)i(Franz.)666 b(HotpathVM:)331 +b(an)e(ef)-22 b(fecti)g(v)-13 b(e)330 b(JIT)29087 19482 +y(compiler)411 b(for)g(resource-constrained)h(de)-22 +b(vices.)929 b(In)411 b Fa(Pr)-40 b(oceedings)412 b(of)g(the)29087 +20479 y(2nd)179 b(international)h(confer)-33 b(ence)181 +b(on)e(V)-66 b(irtual)181 b(e)-18 b(xecution)181 b(en)-35 +b(vir)-40 b(onments)p Fw(,)181 b(pages)29087 21475 y(144\226153,)220 +b(Otta)-13 b(w)k(a,)223 b(Ontario,)f(Canada,)g(2006.)f(A)-35 +b(CM.)p Black 27224 22803 a([16])p Black 387 w(A.)239 +b(Geor)-16 b(ges,)239 b(D.)g(Buytaert,)g(and)f(L.)h(Eeckhout.)373 +b(Statistically)241 b(rigorous)d(Ja)-18 b(v)c(a)29087 +23800 y(performance)221 b(e)-22 b(v)g(aluation.)319 b +Fa(SIGPLAN)221 b(Not.)p Fw(,)h(42\(10\):57\22676,)e(2007.)p +Black 27224 25128 a([17])p Black 387 w(A.)287 b(Gill,)h(J.)g(Launchb) +-18 b(ury)-58 b(,)287 b(and)g(S.)h(L.)f(P)-98 b(.)287 +b(Jones.)530 b(A)287 b(short)g(cut)g(to)g(deforesta-)29087 +26124 y(tion.)603 b(In)310 b Fa(Pr)-40 b(oceedings)312 +b(of)e(the)h(confer)-33 b(ence)311 b(on)f(Functional)g(pr)-40 +b(o)-9 b(gr)c(amming)29087 27121 y(langua)k(g)g(es)209 +b(and)g(computer)i(ar)-33 b(c)-13 b(hitectur)-33 b(e)p +Fw(,)212 b(FPCA)e('93,)g(page)f(223\226232,)g(Ne)-22 +b(w)29087 28117 y(Y)-97 b(ork,)221 b(NY)-114 b(,)222 +b(USA,)h(1993.)e(A)-35 b(CM.)p Black 27224 29445 a([18])p +Black 387 w(B.)213 b(Goldber)-16 b(g)213 b(and)g(Y)-114 +b(.)214 b(G.)g(P)-13 b(ark.)301 b(Higher)214 b(order)f(escape)h +(analysis:)h(optimizing)29087 30441 y(stack)265 b(allocation)g(in)f +(functional)g(program)g(implementations.)458 b(In)264 +b Fa(Pr)-40 b(oceed-)29087 31438 y(ings)223 b(of)g(the)h(thir)-33 +b(d)224 b(Eur)-40 b(opean)223 b(symposium)i(on)d(pr)-40 +b(o)-9 b(gr)c(amming)224 b(on)f(ESOP)h('90)p Fw(,)29087 +32434 y(pages)320 b(152\226160,)f(Copenhagen,)h(Denmark,)h(1990.)e +(Springer)-18 b(-V)-98 b(erlag)321 b(Ne)-22 b(w)29087 +33430 y(Y)-97 b(ork,)221 b(Inc.)p Black 27224 34759 a([19])p +Black 387 w(F)-71 b(.)251 b(Henglein.)411 b(Dynamic)251 +b(typing:)f(syntax)g(and)g(proof)f(theory)-58 b(.)411 +b Fa(Sci.)251 b(Comput.)29087 35755 y(Pr)-40 b(o)-9 b(gr)c(am.)p +Fw(,)222 b(22:197\226230,)f(June)g(1994.)p Black 27224 +37083 a([20])p Black 387 w(J.)395 b(J\370r)-16 b(gensen.)875 +b Fa(A)394 b(Calculus)h(for)g(Boxing)g(Analysis)h(of)e(P)-71 +b(olymorphically)29087 38079 y(T)-66 b(yped)201 b(Langua)-9 +b(g)g(es)p Fw(.)274 b(Ph.D.)202 b(Thesis,)g(Uni)-22 b(v)-13 +b(ersity)201 b(of)f(Copenhapen,)g(1996.)273 b(TR)29087 +39076 y(96/28.)p Black 27224 40404 a([21])p Black 387 +w(J.)312 b(W)-81 b(.)312 b(Llo)-9 b(yd)311 b(and)g(J.)i(C.)e +(Shepherdson.)608 b(P)-13 b(artial)313 b(e)-22 b(v)g(aluation)311 +b(in)h(logic)f(pro-)29087 41400 y(gramming.)318 b Fa(J)-22 +b(.)222 b(Lo)-9 b(g)c(.)222 b(Pr)-40 b(o)-9 b(gr)c(am.)p +Fw(,)223 b(11\(3-4\):217\226242,)c(1991.)p Black 27224 +42729 a([22])p Black 387 w(E.)175 b(Miranda.)220 b(Conte)-13 +b(xt)174 b(management)i(in)f(V)-53 b(isualW)-71 b(orks)176 +b(5i.)220 b(T)-62 b(echnical)176 b(report,)29087 43725 +y(P)-13 b(arcPlace)222 b(Di)-22 b(vision,)223 b(CINCOM,)f(Inc.,)g +(1999.)p Black 27224 45053 a([23])p Black 387 w(T)-66 +b(.)413 b(Mogensen.)933 b(Constructor)412 b(specialization.)934 +b(In)413 b Fa(Pr)-40 b(oceedings)414 b(of)f(the)29087 +46050 y(1993)180 b(A)-27 b(CM)180 b(SIGPLAN)h(symposium)i(on)d(P)-71 +b(artial)182 b(e)-13 b(valuation)180 b(and)g(semantics-)29087 +47046 y(based)339 b(pr)-40 b(o)-9 b(gr)c(am)341 b(manipulation)p +Fw(,)f(pages)g(22\22632,)f(Copenhagen,)g(Denmark,)29087 +48042 y(1993.)220 b(A)-35 b(CM.)p Black 27224 49370 a([24])p +Black 387 w(Y)-114 b(.)225 b(G.)h(P)-13 b(ark)225 b(and)g(B.)g(Goldber) +-16 b(g.)330 b(Escape)226 b(analysis)g(on)f(lists.)332 +b Fa(SIGPLAN)225 b(Not.)p Fw(,)29087 50367 y(27\(7\):116\226127,)219 +b(1992.)p Black 27224 51695 a([25])p Black 387 w(A.)403 +b(Rigo.)899 b(Representation-based)403 b(just-in-time)h(specialization) +g(and)f(the)29087 52691 y(Psyco)298 b(prototype)g(for)g(Python.)565 +b(In)298 b Fa(Pr)-40 b(oceedings)299 b(of)f(the)g(2004)g(A)-27 +b(CM)298 b(SIG-)29087 53688 y(PLAN)190 b(symposium)i(on)f(P)-71 +b(artial)191 b(e)-13 b(valuation)190 b(and)g(semantics-based)i(pr)-40 +b(o)-9 b(gr)c(am)29087 54684 y(manipulation)p Fw(,)221 +b(pages)h(15\22626,)e(V)-98 b(erona,)222 b(Italy)-58 +b(,)223 b(2004.)d(A)-35 b(CM.)p Black 27224 56012 a([26])p +Black 387 w(A.)349 b(Rigo)g(and)g(S.)h(Pedroni.)728 b(PyPy')-49 +b(s)350 b(approach)f(to)g(virtual)h(machine)f(con-)29087 +57008 y(struction.)620 b(In)315 b Fa(Companion)f(to)i(the)f(21st)h(A) +-27 b(CM)315 b(SIGPLAN)h(confer)-33 b(ence)316 b(on)29087 +58005 y(Object-oriented)j(pr)-40 b(o)-9 b(gr)c(amming)318 +b(systems,)i(langua)-9 b(g)g(es,)318 b(and)f(applications)p +Fw(,)29087 59001 y(pages)221 b(944\226953,)g(Portland,)h(Ore)-13 +b(gon,)221 b(USA,)i(2006.)e(A)-35 b(CM.)p Black 27224 +60329 a([27])p Black 387 w(P)-98 b(.)316 b(W)-71 b(adler)-49 +b(.)622 b(Deforestation:)317 b(transforming)f(programs)g(to)g +(eliminate)g(trees.)29087 61326 y(In)218 b Fa(Pr)-40 +b(oceedings)220 b(of)f(the)g(Second)f(Eur)-40 b(opean)219 +b(Symposium)g(on)f(Pr)-40 b(o)-9 b(gr)c(amming)p Fw(,)29087 +62322 y(page)230 b(231\226248,)f(Amsterdam,)i(The)f(Netherlands,)i(The) +e(Netherlands,)h(1988.)29087 63318 y(North-Holland)221 +b(Publishing)h(Co.)p Black Black eop end +%%Trailer + +userdict /end-hook known{end-hook}if +%%EOF From afa at codespeak.net Thu Nov 18 17:10:15 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 18 Nov 2010 17:10:15 +0100 (CET) Subject: [pypy-svn] r79258 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101118161015.80AED282BD6@codespeak.net> Author: afa Date: Thu Nov 18 17:10:12 2010 New Revision: 79258 Modified: pypy/branch/fast-forward/pypy/module/_io/__init__.py pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/interp_io.py pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Log: _RawIOBase implements read(), based on an abstract readinto() method provided by subclasses Modified: pypy/branch/fast-forward/pypy/module/_io/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/__init__.py Thu Nov 18 17:10:12 2010 @@ -7,7 +7,7 @@ } interpleveldefs = { - 'DEFAULT_BUFFER_SIZE': 'space.wrap(interp_io.DEFAULT_BUFFER_SIZE)', + 'DEFAULT_BUFFER_SIZE': 'space.wrap(interp_iobase.DEFAULT_BUFFER_SIZE)', 'BlockingIOError': 'interp_io.W_BlockingIOError', '_IOBase': 'interp_iobase.W_IOBase', '_RawIOBase': 'interp_iobase.W_RawIOBase', Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Thu Nov 18 17:10:12 2010 @@ -10,9 +10,9 @@ from pypy.rlib.rarithmetic import r_longlong, intmask from pypy.tool.sourcetools import func_renamer from pypy.module._io.interp_iobase import ( - W_IOBase, convert_size, + W_IOBase, DEFAULT_BUFFER_SIZE, convert_size, check_readable_w, check_writable_w, check_seekable_w) -from pypy.module._io.interp_io import DEFAULT_BUFFER_SIZE, W_BlockingIOError +from pypy.module._io.interp_io import W_BlockingIOError from pypy.module.thread.os_lock import Lock STATE_ZERO, STATE_OK, STATE_DETACHED = range(3) Modified: pypy/branch/fast-forward/pypy/module/_io/interp_io.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_io.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_io.py Thu Nov 18 17:10:12 2010 @@ -5,8 +5,6 @@ from pypy.module.exceptions.interp_exceptions import W_IOError from pypy.module._io.interp_iobase import W_IOBase -DEFAULT_BUFFER_SIZE = 8192 - class W_BlockingIOError(W_IOError): def __init__(self, space): W_IOError.__init__(self, space) Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Thu Nov 18 17:10:12 2010 @@ -6,6 +6,8 @@ from pypy.interpreter.error import OperationError, operationerrfmt from pypy.rlib.rstring import StringBuilder +DEFAULT_BUFFER_SIZE = 8192 + def convert_size(space, w_size): if space.is_w(w_size, space.w_None): return -1 @@ -271,9 +273,41 @@ ) class W_RawIOBase(W_IOBase): - pass + # ________________________________________________________________ + # Abstract read methods, based on readinto() + + @unwrap_spec('self', ObjSpace, W_Root) + def read_w(self, space, w_size=None): + size = convert_size(space, w_size) + if size < 0: + return space.call_method(self, "readall") + + w_buffer = space.call_function(space.w_bytearray, w_size) + w_length = space.call_method(self, "readinto", w_buffer) + space.delslice(w_buffer, w_length, space.len(w_buffer)) + return space.str(w_buffer) + + @unwrap_spec('self', ObjSpace) + def readall_w(self, space): + builder = StringBuilder() + while True: + w_data = space.call_method(self, "read", + space.wrap(DEFAULT_BUFFER_SIZE)) + + if not space.isinstance_w(w_data, space.w_str): + raise OperationError(space.w_TypeError, space.wrap( + "read() should return bytes")) + data = space.str_w(w_data) + if not data: + break + builder.append(data) + return space.wrap(builder.build()) + W_RawIOBase.typedef = TypeDef( '_RawIOBase', W_IOBase.typedef, __new__ = generic_new_descr(W_RawIOBase), + + read = interp2app(W_RawIOBase.read_w), + readall = interp2app(W_RawIOBase.readall_w), ) Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Thu Nov 18 17:10:12 2010 @@ -103,6 +103,16 @@ ref = weakref.ref(f) assert ref() is f + def test_rawio_read(self): + import _io + class MockRawIO(_io._RawIOBase): + stack = ['abc', 'de', ''] + def readinto(self, buf): + data = self.stack.pop(0) + buf[:len(data)] = data + return len(data) + assert MockRawIO().read() == 'abcde' + class AppTestOpen: def setup_class(cls): tmpfile = udir.join('tmpfile').ensure() From antocuni at codespeak.net Thu Nov 18 17:18:41 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 18 Nov 2010 17:18:41 +0100 (CET) Subject: [pypy-svn] r79259 - pypy/trunk/pypy/tool Message-ID: <20101118161841.21EF4282BEA@codespeak.net> Author: antocuni Date: Thu Nov 18 17:18:40 2010 New Revision: 79259 Modified: pypy/trunk/pypy/tool/terminal.py Log: a bit of a hack, enough to make pypy.tool.progressbar working more or less nicely on pypy Modified: pypy/trunk/pypy/tool/terminal.py ============================================================================== --- pypy/trunk/pypy/tool/terminal.py (original) +++ pypy/trunk/pypy/tool/terminal.py Thu Nov 18 17:18:40 2010 @@ -62,9 +62,10 @@ for control in CONTROLS: # Set the control escape sequence setattr(MODULE, control, curses.tigetstr(CONTROLS[control]) or '') - for value in VALUES: - # Set terminal related values - setattr(MODULE, value, curses.tigetnum(VALUES[value])) + if hasattr(curses, 'tigetnum'): + for value in VALUES: + # Set terminal related values + setattr(MODULE, value, curses.tigetnum(VALUES[value])) def render(text): """Helper function to apply controls easily @@ -74,7 +75,16 @@ return text % MODULE.__dict__ try: - import curses + if '__pypy__' in sys.builtin_module_names: + # this is not really the whole curses, but our _minimal_curses it's + # better than nothing + import _minimal_curses as curses + # a bit of a hack: we have tigetstr but not tigetnum, so we call + # default() to have default values, then setup() will overwrite the + # ones it can + default() + else: + import curses setup() except Exception, e: # There is a failure; set all attributes to default From arigo at codespeak.net Thu Nov 18 17:58:21 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 18 Nov 2010 17:58:21 +0100 (CET) Subject: [pypy-svn] r79260 - in pypy/branch/jit-free/pypy/jit/backend: . llgraph test x86 x86/test Message-ID: <20101118165821.BF82B282BD6@codespeak.net> Author: arigo Date: Thu Nov 18 17:58:20 2010 New Revision: 79260 Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py pypy/branch/jit-free/pypy/jit/backend/model.py pypy/branch/jit-free/pypy/jit/backend/test/runner_test.py pypy/branch/jit-free/pypy/jit/backend/x86/runner.py pypy/branch/jit-free/pypy/jit/backend/x86/test/test_regalloc.py pypy/branch/jit-free/pypy/jit/backend/x86/test/test_runner.py Log: Forgot to adapt the interface of x86's compile_bridge(). Fix some x86 tests. Collect some stats on the cpu about how many bridges are made and freed. Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/llgraph/runner.py Thu Nov 18 17:58:20 2010 @@ -122,7 +122,7 @@ c = llimpl.compile_start() clt = original_loop_token.compiled_loop_token clt.loop_and_bridges.append(c) - clt.bridges_count += 1 + clt.compiling_a_bridge() self._compile_loop_or_bridge(c, inputargs, operations) old, oldindex = faildescr._compiled_fail llimpl.compile_redirect_fail(old, oldindex, c) Modified: pypy/branch/jit-free/pypy/jit/backend/model.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/model.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/model.py Thu Nov 18 17:58:20 2010 @@ -8,6 +8,10 @@ done_with_this_frame_int_v = -1 done_with_this_frame_ref_v = -1 done_with_this_frame_float_v = -1 + total_compiled_loops = 0 + total_compiled_bridges = 0 + total_freed_loops = 0 + total_freed_bridges = 0 def __init__(self): self.fail_descr_list = [] @@ -138,6 +142,8 @@ for n in compiled_loop_token.faildescr_indices: lst[n] = None self.fail_descr_free_list.extend(compiled_loop_token.faildescr_indices) + self.total_freed_loops += 1 + self.total_freed_bridges += compiled_loop_token.bridges_count # We expect 'compiled_loop_token' to be itself garbage-collected soon. @staticmethod @@ -268,6 +274,7 @@ class CompiledLoopToken(object): def __init__(self, cpu, number): + cpu.total_compiled_loops += 1 self.cpu = cpu self.number = number self.bridges_count = 0 @@ -279,6 +286,10 @@ def record_faildescr_index(self, n): self.faildescr_indices.append(n) + def compiling_a_bridge(self): + self.cpu.total_compiled_bridges += 1 + self.bridges_count += 1 + def __del__(self): debug_start("jit-free-looptoken") debug_print("Freeing loop #", self.number, 'with', Modified: pypy/branch/jit-free/pypy/jit/backend/test/runner_test.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/test/runner_test.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/test/runner_test.py Thu Nov 18 17:58:20 2010 @@ -174,6 +174,8 @@ assert not wr_i1() and not wr_guard() def test_compile_bridge(self): + self.cpu.total_compiled_loops = 0 + self.cpu.total_compiled_bridges = 0 i0 = BoxInt() i1 = BoxInt() i2 = BoxInt() @@ -207,6 +209,9 @@ res = self.cpu.get_latest_value_int(0) assert res == 20 + assert self.cpu.total_compiled_loops == 1 + assert self.cpu.total_compiled_bridges == 1 + def test_compile_bridge_with_holes(self): i0 = BoxInt() i1 = BoxInt() Modified: pypy/branch/jit-free/pypy/jit/backend/x86/runner.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/x86/runner.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/x86/runner.py Thu Nov 18 17:58:20 2010 @@ -53,7 +53,10 @@ self.assembler.assemble_loop(inputargs, operations, looptoken, log=log) - def compile_bridge(self, faildescr, inputargs, operations, log=True): + def compile_bridge(self, faildescr, inputargs, operations, + original_loop_token, log=True): + clt = original_loop_token.compiled_loop_token + clt.compiling_a_bridge() self.assembler.assemble_bridge(faildescr, inputargs, operations, log=log) Modified: pypy/branch/jit-free/pypy/jit/backend/x86/test/test_regalloc.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/x86/test/test_regalloc.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/x86/test/test_regalloc.py Thu Nov 18 17:58:20 2010 @@ -166,7 +166,8 @@ assert ([box.type for box in bridge.inputargs] == [box.type for box in guard_op.getfailargs()]) faildescr = guard_op.getdescr() - self.cpu.compile_bridge(faildescr, bridge.inputargs, bridge.operations) + self.cpu.compile_bridge(faildescr, bridge.inputargs, bridge.operations, + loop.token) return bridge def run(self, loop): Modified: pypy/branch/jit-free/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/x86/test/test_runner.py Thu Nov 18 17:58:20 2010 @@ -371,7 +371,7 @@ ] bridge[1].setfailargs([i1b]) - self.cpu.compile_bridge(faildescr1, [i1b], bridge) + self.cpu.compile_bridge(faildescr1, [i1b], bridge, looptoken) name, address, size = agent.functions[1] assert name == "Bridge # 0: bye" # Would be exactly ==, but there are some guard failure recovery From arigo at codespeak.net Thu Nov 18 18:12:44 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 18 Nov 2010 18:12:44 +0100 (CET) Subject: [pypy-svn] r79261 - pypy/branch/jit-free/pypy/jit/backend Message-ID: <20101118171244.58786282BEA@codespeak.net> Author: arigo Date: Thu Nov 18 18:12:42 2010 New Revision: 79261 Modified: pypy/branch/jit-free/pypy/jit/backend/model.py Log: No-op change. Modified: pypy/branch/jit-free/pypy/jit/backend/model.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/model.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/model.py Thu Nov 18 18:12:42 2010 @@ -142,8 +142,6 @@ for n in compiled_loop_token.faildescr_indices: lst[n] = None self.fail_descr_free_list.extend(compiled_loop_token.faildescr_indices) - self.total_freed_loops += 1 - self.total_freed_bridges += compiled_loop_token.bridges_count # We expect 'compiled_loop_token' to be itself garbage-collected soon. @staticmethod @@ -295,4 +293,6 @@ debug_print("Freeing loop #", self.number, 'with', self.bridges_count, 'attached bridges') self.cpu.free_loop_and_bridges(self) + self.cpu.total_freed_loops += 1 + self.cpu.total_freed_bridges += self.bridges_count debug_stop("jit-free-looptoken") From afa at codespeak.net Thu Nov 18 18:16:45 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 18 Nov 2010 18:16:45 +0100 (CET) Subject: [pypy-svn] r79262 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101118171645.887DD282BEA@codespeak.net> Author: afa Date: Thu Nov 18 18:16:44 2010 New Revision: 79262 Modified: pypy/branch/fast-forward/pypy/objspace/std/inttype.py pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py Log: Implement bytearray->int conversion Modified: pypy/branch/fast-forward/pypy/objspace/std/inttype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/inttype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/inttype.py Thu Nov 18 18:16:44 2010 @@ -1,5 +1,6 @@ from pypy.interpreter import gateway, typedef from pypy.interpreter.error import OperationError +from pypy.interpreter.buffer import Buffer from pypy.objspace.std.register_all import register_all from pypy.objspace.std.stdtypedef import StdTypeDef, SMM from pypy.objspace.std.strutil import (string_to_int, string_to_bigint, @@ -62,6 +63,18 @@ # ____________________________________________________________ +def string_to_int_or_long(space, string, base=10): + w_longval = None + value = 0 + try: + value = string_to_int(string, base) + except ParseStringError, e: + raise OperationError(space.w_ValueError, + space.wrap(e.msg)) + except ParseStringOverflowError, e: + w_longval = retry_to_w_long(space, e.parser) + return value, w_longval + def retry_to_w_long(space, parser, base=0): parser.rewind() try: @@ -78,31 +91,35 @@ w_value = w_x # 'x' is the keyword argument name in CPython value = 0 if w_base is None: + ok = False # check for easy cases if type(w_value) is W_IntObject: value = w_value.intval + ok = True elif space.is_true(space.isinstance(w_value, space.w_str)): - try: - value = string_to_int(space.str_w(w_value)) - except ParseStringError, e: - raise OperationError(space.w_ValueError, - space.wrap(e.msg)) - except ParseStringOverflowError, e: - w_longval = retry_to_w_long(space, e.parser) + value, w_longval = string_to_int_or_long(space, space.str_w(w_value)) + ok = True elif space.is_true(space.isinstance(w_value, space.w_unicode)): if space.config.objspace.std.withropeunicode: from pypy.objspace.std.ropeunicodeobject import unicode_to_decimal_w else: from pypy.objspace.std.unicodeobject import unicode_to_decimal_w string = unicode_to_decimal_w(space, w_value) - try: - value = string_to_int(string) - except ParseStringError, e: - raise OperationError(space.w_ValueError, - space.wrap(e.msg)) - except ParseStringOverflowError, e: - w_longval = retry_to_w_long(space, e.parser) + value, w_longval = string_to_int_or_long(space, string) + ok = True else: + # If object supports the buffer interface + try: + w_buffer = space.buffer(w_value) + except OperationError, e: + if not e.match(space, space.w_TypeError): + raise + else: + buf = space.interp_w(Buffer, w_buffer) + value, w_longval = string_to_int_or_long(space, buf.as_str()) + ok = True + + if not ok: # otherwise, use the __int__() then __trunc__() methods try: w_obj = space.int(w_value) @@ -139,13 +156,8 @@ raise OperationError(space.w_TypeError, space.wrap("int() can't convert non-string " "with explicit base")) - try: - value = string_to_int(s, base) - except ParseStringError, e: - raise OperationError(space.w_ValueError, - space.wrap(e.msg)) - except ParseStringOverflowError, e: - w_longval = retry_to_w_long(space, e.parser, base) + + value, w_longval = string_to_int_or_long(space, s, base) if w_longval is not None: if not space.is_w(w_inttype, space.w_int): Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py Thu Nov 18 18:16:44 2010 @@ -192,3 +192,6 @@ u = b.decode('utf-8') assert isinstance(u, unicode) assert u == u'abcdefghi' + + def test_int(self): + assert int(bytearray('-1234')) == -1234 From antocuni at codespeak.net Thu Nov 18 18:21:13 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 18 Nov 2010 18:21:13 +0100 (CET) Subject: [pypy-svn] r79263 - in pypy/branch/jit-free/pypy/jit: backend metainterp Message-ID: <20101118172113.94307282BEF@codespeak.net> Author: antocuni Date: Thu Nov 18 18:21:12 2010 New Revision: 79263 Modified: pypy/branch/jit-free/pypy/jit/backend/model.py pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py Log: rename the log categories, and keep track of when we allocate loops and bridges Modified: pypy/branch/jit-free/pypy/jit/backend/model.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/model.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/model.py Thu Nov 18 18:21:12 2010 @@ -280,6 +280,9 @@ # that belong to this loop or to a bridge attached to it. # Filled by the frontend calling record_faildescr_index(). self.faildescr_indices = [] + debug_start("jit-mem-looptoken-alloc") + debug_print("allocating Loop #", self.number) + debug_stop("jit-mem-looptoken-alloc") def record_faildescr_index(self, n): self.faildescr_indices.append(n) @@ -287,12 +290,15 @@ def compiling_a_bridge(self): self.cpu.total_compiled_bridges += 1 self.bridges_count += 1 + debug_start("jit-mem-looptoken-alloc") + debug_print("allocating Bridge #", self.bridges_count, "of Loop #", self.number) + debug_stop("jit-mem-looptoken-alloc") def __del__(self): - debug_start("jit-free-looptoken") - debug_print("Freeing loop #", self.number, 'with', + debug_start("jit-mem-looptoken-free") + debug_print("freeing Loop #", self.number, 'with', self.bridges_count, 'attached bridges') self.cpu.free_loop_and_bridges(self) self.cpu.total_freed_loops += 1 self.cpu.total_freed_bridges += self.bridges_count - debug_stop("jit-free-looptoken") + debug_stop("jit-mem-looptoken-free") Modified: pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py Thu Nov 18 18:21:12 2010 @@ -58,7 +58,7 @@ self.alive_loops[looptoken] = None def _kill_old_loops_now(self): - debug_start("jit-free-memmgr") + debug_start("jit-mem-collect") oldtotal = len(self.alive_loops) #print self.alive_loops.keys() debug_print("Current generation:", self.current_generation) @@ -76,4 +76,4 @@ from pypy.rlib import rgc # a single one is not enough for all tests :-( rgc.collect(); rgc.collect(); rgc.collect() - debug_stop("jit-free-memmgr") + debug_stop("jit-mem-collect") From arigo at codespeak.net Thu Nov 18 18:29:54 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 18 Nov 2010 18:29:54 +0100 (CET) Subject: [pypy-svn] r79264 - pypy/branch/jit-free/pypy/jit/backend/x86 Message-ID: <20101118172954.4D1FB282BEF@codespeak.net> Author: arigo Date: Thu Nov 18 18:29:52 2010 New Revision: 79264 Modified: pypy/branch/jit-free/pypy/jit/backend/x86/assembler.py Log: Don't use open_file_as_stream(). It may be annotated from the main program as taking mode='r', which is not compatible with calling it with mode='w'. Modified: pypy/branch/jit-free/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/x86/assembler.py Thu Nov 18 18:29:52 2010 @@ -34,7 +34,6 @@ from pypy.rlib.debug import debug_print from pypy.rlib import rgc from pypy.jit.backend.x86.jump import remap_frame_layout -from pypy.rlib.streamio import open_file_as_stream from pypy.jit.metainterp.history import ConstInt, BoxInt # darwin requires the stack to be 16 bytes aligned on calls. Same for gcc 4.5.0, @@ -244,11 +243,11 @@ if self._debug: output_log = self._output_loop_log assert output_log is not None - f = open_file_as_stream(output_log, "w") + fd = os.open(output_log, os.O_WRONLY | os.O_CREAT, 0666) for i in range(len(self.loop_run_counters)): name, struct = self.loop_run_counters[i] - f.write(str(name) + ":" + str(struct.i) + "\n") - f.close() + os.write(fd, str(name) + ":" + str(struct.i) + "\n") + os.close(fd) def _build_float_constants(self): # 44 bytes: 32 bytes for the data, and up to 12 bytes for alignment From arigo at codespeak.net Thu Nov 18 18:47:23 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 18 Nov 2010 18:47:23 +0100 (CET) Subject: [pypy-svn] r79266 - in pypy/branch/jit-free/pypy/jit: metainterp tool tool/test Message-ID: <20101118174723.6BFBD282BEC@codespeak.net> Author: arigo Date: Thu Nov 18 18:47:21 2010 New Revision: 79266 Modified: pypy/branch/jit-free/pypy/jit/metainterp/jitprof.py pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py pypy/branch/jit-free/pypy/jit/tool/jitoutput.py pypy/branch/jit-free/pypy/jit/tool/test/test_jitoutput.py Log: Print the cpu.total_{compiled,freed}_{loops,bridges} at the end too. Modified: pypy/branch/jit-free/pypy/jit/metainterp/jitprof.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/jitprof.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/jitprof.py Thu Nov 18 18:47:21 2010 @@ -24,6 +24,10 @@ NVIRTUALS NVHOLES NVREUSED +TOTAL_COMPILED_LOOPS +TOTAL_COMPILED_BRIDGES +TOTAL_FREED_LOOPS +TOTAL_FREED_BRIDGES """ def _setup(): @@ -91,6 +95,7 @@ calls = 0 current = None printing = True + cpu = None def start(self): self.starttime = self.timer() @@ -176,6 +181,16 @@ self._print_intline("nvirtuals", cnt[NVIRTUALS]) self._print_intline("nvholes", cnt[NVHOLES]) self._print_intline("nvreused", cnt[NVREUSED]) + cpu = self.cpu + if cpu is not None: # for some tests + self._print_intline("Total # of loops", + cpu.total_compiled_loops) + self._print_intline("Total # of bridges", + cpu.total_compiled_bridges) + self._print_intline("Freed # of loops", + cpu.total_freed_loops) + self._print_intline("Freed # of bridges", + cpu.total_freed_bridges) def _print_line_time(self, string, i, tim): final = "%s:%s\t%d\t%f\n" % (string, " " * max(0, 13-len(string)), i, tim) Modified: pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/pyjitpl.py Thu Nov 18 18:47:21 2010 @@ -1206,6 +1206,7 @@ self.logger_ops = Logger(self, guard_number=True) self.profiler = ProfilerClass() + self.profiler.cpu = cpu self.warmrunnerdesc = warmrunnerdesc backendmodule = self.cpu.__module__ Modified: pypy/branch/jit-free/pypy/jit/tool/jitoutput.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/tool/jitoutput.py (original) +++ pypy/branch/jit-free/pypy/jit/tool/jitoutput.py Thu Nov 18 18:47:21 2010 @@ -27,6 +27,10 @@ (('nvirtuals',), '^nvirtuals:\s+(\d+)$'), (('nvholes',), '^nvholes:\s+(\d+)$'), (('nvreused',), '^nvreused:\s+(\d+)$'), + (('total_compiled_loops',), '^Total # of loops:\s+(\d+)$'), + (('total_compiled_bridges',), '^Total # of bridges:\s+(\d+)$'), + (('total_freed_loops',), '^Freed # of loops:\s+(\d+)$'), + (('total_freed_bridges',), '^Freed # of bridges:\s+(\d+)$'), ] class Ops(object): Modified: pypy/branch/jit-free/pypy/jit/tool/test/test_jitoutput.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/tool/test/test_jitoutput.py (original) +++ pypy/branch/jit-free/pypy/jit/tool/test/test_jitoutput.py Thu Nov 18 18:47:21 2010 @@ -60,6 +60,10 @@ nvirtuals: 13 nvholes: 14 nvreused: 15 +Total # of loops: 100 +Total # of bridges: 300 +Freed # of loops: 99 +Freed # of bridges: 299 ''' def test_parse(): From antocuni at codespeak.net Thu Nov 18 19:16:11 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 18 Nov 2010 19:16:11 +0100 (CET) Subject: [pypy-svn] r79267 - in pypy/trunk/pypy/jit/tool: . test Message-ID: <20101118181611.29CFD282BEB@codespeak.net> Author: antocuni Date: Thu Nov 18 19:16:09 2010 New Revision: 79267 Added: pypy/trunk/pypy/jit/tool/test/test_loopcounter.py (contents, props changed) Modified: pypy/trunk/pypy/jit/tool/loopcounter.py Log: adapt to the new log format that is being used in the jit-free branch Modified: pypy/trunk/pypy/jit/tool/loopcounter.py ============================================================================== --- pypy/trunk/pypy/jit/tool/loopcounter.py (original) +++ pypy/trunk/pypy/jit/tool/loopcounter.py Thu Nov 18 19:16:09 2010 @@ -7,32 +7,41 @@ import py import sys import optparse +import re def get_timestamp(line): - import re match = re.match(r'\[([0-9a-f]*)\] .*', line) return int(match.group(1), 16) -def main(logfile, options): - log = open(logfile) +def count_loops_and_bridges(log): loops = 0 bridges = 0 time0 = None - print 'timestamp,total,loops,bridges' - for line in log: + lines = iter(log) + for line in lines: if time0 is None and line.startswith('['): time0 = get_timestamp(line) - if '{jit-log-opt-' in line: - time_now = get_timestamp(line) - if '{jit-log-opt-loop' in line: + if '{jit-mem-looptoken-' in line: + time_now = get_timestamp(line) - time0 + text = lines.next() + if text.startswith('allocating Loop #'): loops += 1 - elif '{jit-log-opt-bridge' in line: + elif text.startswith('allocating Bridge #'): bridges += 1 + elif text.startswith('freeing Loop #'): + match = re.match('freeing Loop # .* with ([0-9]*) attached bridges\n', text) + loops -=1 + bridges -= int(match.group(1)) else: - assert False, 'unknown category %s' % line + assert False, 'unknown line' % line total = loops+bridges - timestamp = time_now - time0 - print '%d,%d,%d,%d' % (timestamp, total, loops, bridges) + yield (time_now, total, loops, bridges) + +def main(logfile, options): + print 'timestamp,total,loops,bridges' + log = open(logfile) + for timestamp, total, loops, bridges in count_loops_and_bridges(log): + print '%d,%d,%d,%d' % (timestamp, total, loops, bridges) if __name__ == '__main__': parser = optparse.OptionParser(usage="%prog loopfile [options]") @@ -40,5 +49,4 @@ if len(args) != 1: parser.print_help() sys.exit(2) - main(args[0], options) Added: pypy/trunk/pypy/jit/tool/test/test_loopcounter.py ============================================================================== --- (empty file) +++ pypy/trunk/pypy/jit/tool/test/test_loopcounter.py Thu Nov 18 19:16:09 2010 @@ -0,0 +1,33 @@ +from cStringIO import StringIO +from pypy.jit.tool.loopcounter import count_loops_and_bridges + +def test_loopcounter(): + log = StringIO(""" +[1200] {stuff +... +[1201] stuff} +[120a] {jit-mem-looptoken-alloc +allocating Loop # 0 +[120b] jit-mem-looptoken-alloc} +[1300] {jit-mem-looptoken-alloc +allocating Bridge # 1 of Loop # 0 +[1301] jit-mem-looptoken-alloc} +[1400] {jit-mem-looptoken-alloc +allocating Bridge # 2 of Loop # 0 +[1401] jit-mem-looptoken-alloc} +[1500] {jit-mem-looptoken-alloc +allocating Loop # 1 +[1501] jit-mem-looptoken-alloc} +[1600] {jit-mem-looptoken-free +freeing Loop # 0 with 2 attached bridges +[1601] jit-mem-looptoken-free} +""") + lines = list(count_loops_and_bridges(log)) + assert lines == [ + # time total loops bridges + (0x00a, 1, 1, 0), + (0x100, 2, 1, 1), + (0x200, 3, 1, 2), + (0x300, 4, 2, 2), + (0x400, 1, 1, 0), + ] From wlav at codespeak.net Fri Nov 19 04:14:21 2010 From: wlav at codespeak.net (wlav at codespeak.net) Date: Fri, 19 Nov 2010 04:14:21 +0100 (CET) Subject: [pypy-svn] r79268 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101119031421.36C0F282B9D@codespeak.net> Author: wlav Date: Fri Nov 19 04:14:18 2010 New Revision: 79268 Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py Log: further rpython fixes for arrays Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/converter.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/converter.py Fri Nov 19 04:14:18 2010 @@ -1,9 +1,11 @@ import sys from pypy.interpreter.error import OperationError +from pypy.interpreter.buffer import Buffer from pypy.rpython.lltypesystem import rffi, lltype from pypy.rlib.rarithmetic import r_singlefloat from pypy.module._rawffi.interp_rawffi import unpack_simple_shape +from pypy.module._rawffi.array import W_Array from pypy.module.cppyy import helper, capi @@ -126,15 +128,13 @@ class ShortConverter(LongConverter): def from_memory(self, space, w_obj, offset): fieldptr = self._get_fieldptr(space, w_obj, offset) - intptr = rffi.cast(rffi.SHORTP, fieldptr) - return space.wrap(intptr[0]) + shortptr = rffi.cast(rffi.SHORTP, fieldptr) + return space.wrap(shortptr[0]) def to_memory(self, space, w_obj, w_value, offset): - import struct fieldptr = self._get_fieldptr(space, w_obj, offset) - pack = struct.pack('h', space.unwrap(w_value)) # unchecked - fieldptr[0] = pack[0] - fieldptr[1] = pack[1] + shortptr = rffi.cast(rffi.SHORTP, fieldptr) + shortptr[0] = rffi.cast(rffi.SHORT, space.c_int_w(w_value)) class FloatConverter(TypeConverter): def convert_argument(self, space, w_obj): @@ -190,8 +190,8 @@ # read access, so no copy needed fieldptr = self._get_fieldptr(space, w_obj, offset) ptrval = rffi.cast(rffi.UINT, fieldptr) - w_array = unpack_simple_shape(space, space.wrap('h')) - return w_array.fromaddress(space, ptrval, self.size) + arr = space.interp_w(W_Array, unpack_simple_shape(space, space.wrap('h'))) + return arr.fromaddress(space, ptrval, self.size) def to_memory(self, space, w_obj, w_value, offset): # copy only the pointer value @@ -204,10 +204,10 @@ def to_memory(self, space, w_obj, w_value, offset): # copy the full array (uses byte copy for now) fieldptr = self._get_fieldptr(space, w_obj, offset) - value = w_value.getslotvalue(2) + buf = space.interp_w(Buffer, w_value.getslotvalue(2)) # TODO: get sizeof(short) from system - for i in range(min(self.size*2, value.getlength())): - fieldptr[i] = value.getitem(i) + for i in range(min(self.size*2, buf.getlength())): + fieldptr[i] = buf.getitem(i) class LongPtrConverter(TypeConverter): _immutable_ = True @@ -221,8 +221,8 @@ # read access, so no copy needed fieldptr = self._get_fieldptr(space, w_obj, offset) ptrval = rffi.cast(rffi.UINT, fieldptr) - w_array = unpack_simple_shape(space, space.wrap('l')) - return w_array.fromaddress(space, ptrval, self.size) + arr = space.interp_w(W_Array, unpack_simple_shape(space, space.wrap('l'))) + return arr.fromaddress(space, ptrval, self.size) def to_memory(self, space, w_obj, w_value, offset): # copy only the pointer value @@ -235,10 +235,10 @@ def to_memory(self, space, w_obj, w_value, offset): # copy the full array (uses byte copy for now) fieldptr = self._get_fieldptr(space, w_obj, offset) - value = w_value.getslotvalue(2) + buf = space.interp_w(Buffer, w_value.getslotvalue(2)) # TODO: get sizeof(long) from system - for i in range(min(self.size*4, value.getlength())): - fieldptr[i] = value.getitem(i) + for i in range(min(self.size*4, buf.getlength())): + fieldptr[i] = buf.getitem(i) class InstancePtrConverter(TypeConverter): From afa at codespeak.net Fri Nov 19 10:43:56 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 19 Nov 2010 10:43:56 +0100 (CET) Subject: [pypy-svn] r79269 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101119094356.7C95C282B9D@codespeak.net> Author: afa Date: Fri Nov 19 10:43:53 2010 New Revision: 79269 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Log: Some progress in TextIOWrapper Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Fri Nov 19 10:43:53 2010 @@ -6,6 +6,8 @@ from pypy.interpreter.baseobjspace import ObjSpace, W_Root from pypy.interpreter.error import OperationError, operationerrfmt +STATE_ZERO, STATE_OK, STATE_DETACHED = range(3) + class W_TextIOBase(W_IOBase): w_encoding = None @@ -34,11 +36,37 @@ ) class W_TextIOWrapper(W_TextIOBase): + def __init__(self, space): + W_TextIOBase.__init__(self, space) + self.state = STATE_ZERO + @unwrap_spec('self', ObjSpace, W_Root, W_Root, W_Root, W_Root, int) def descr_init(self, space, w_buffer, w_encoding=None, w_errors=None, w_newline=None, line_buffering=0): + self.state = STATE_ZERO + self.w_buffer = w_buffer - self.w_encoding = w_encoding + + # Set encoding + self.w_encoding = None + if space.is_w(w_encoding, space.w_None): + try: + w_locale = space.call_method(space.builtin, '__import__', + space.wrap("locale")) + self.w_encoding = space.call_method(w_locale, + "getpreferredencoding") + except OperationError, e: + # getpreferredencoding() may also raise ImportError + if not space.match(space, space.w_ImportError): + raise + self.w_encoding = space.wrap("ascii") + if self.w_encoding: + pass + elif not space.is_w(w_encoding, space.w_None): + self.w_encoding = w_encoding + else: + raise OperationError(space.w_IOError, space.wrap( + "could not determine default encoding")) if space.is_w(w_newline, space.w_None): newline = None @@ -50,6 +78,31 @@ self.line_buffering = line_buffering + self.state = STATE_OK + + def _check_init(self, space): + if self.state == STATE_ZERO: + raise OperationError(space.w_ValueError, space.wrap( + "I/O operation on uninitialized object")) + elif self.state == STATE_DETACHED: + raise OperationError(space.w_ValueError, space.wrap( + "underlying buffer has been detached")) + + @unwrap_spec('self', ObjSpace) + def readable_w(self, space): + self._check_init(space) + return space.call_method(self.w_buffer, "readable") + + @unwrap_spec('self', ObjSpace) + def writable_w(self, space): + self._check_init(space) + return space.call_method(self.w_buffer, "writable") + + @unwrap_spec('self', ObjSpace) + def seekable_w(self, space): + self._check_init(space) + return space.call_method(self.w_buffer, "seekable") + @unwrap_spec('self', ObjSpace, W_Root) def read_w(self, space, w_size=None): # XXX w_size? @@ -71,4 +124,7 @@ readline = interp2app(W_TextIOWrapper.readline_w), line_buffering = interp_attrproperty("line_buffering", W_TextIOWrapper), + readable = interp2app(W_TextIOWrapper.readable_w), + writable = interp2app(W_TextIOWrapper.writable_w), + seekable = interp2app(W_TextIOWrapper.seekable_w), ) Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Fri Nov 19 10:43:53 2010 @@ -18,4 +18,13 @@ assert t.readline() == u"\xe9\n" raises(TypeError, t.__init__, b, newline=42) raises(ValueError, t.__init__, b, newline='xyzzy') + t = _io.TextIOWrapper(b) + assert t.encoding + def test_properties(self): + import _io + r = _io.BytesIO(b"\xc3\xa9\n\n") + b = _io.BufferedReader(r, 1000) + t = _io.TextIOWrapper(b) + assert t.readable() + assert t.seekable() From afa at codespeak.net Fri Nov 19 13:08:33 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 19 Nov 2010 13:08:33 +0100 (CET) Subject: [pypy-svn] r79270 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101119120833.37F91282B9D@codespeak.net> Author: afa Date: Fri Nov 19 13:08:31 2010 New Revision: 79270 Modified: pypy/branch/fast-forward/pypy/module/_io/__init__.py pypy/branch/fast-forward/pypy/module/_io/interp_textio.py pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Log: io.IncrementalNewlineDecoder Modified: pypy/branch/fast-forward/pypy/module/_io/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/__init__.py Fri Nov 19 13:08:31 2010 @@ -24,7 +24,7 @@ 'TextIOWrapper': 'interp_textio.W_TextIOWrapper', 'open': 'interp_io.open', - 'IncrementalNewlineDecoder': 'space.w_None', + 'IncrementalNewlineDecoder': 'interp_textio.W_IncrementalNewlineDecoder', } def init(self, space): Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Fri Nov 19 13:08:31 2010 @@ -3,11 +3,185 @@ TypeDef, GetSetProperty, interp_attrproperty_w, interp_attrproperty, generic_new_descr) from pypy.interpreter.gateway import interp2app, unwrap_spec -from pypy.interpreter.baseobjspace import ObjSpace, W_Root +from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root from pypy.interpreter.error import OperationError, operationerrfmt +from pypy.rlib.rstring import UnicodeBuilder STATE_ZERO, STATE_OK, STATE_DETACHED = range(3) +SEEN_CR = 1 +SEEN_LF = 2 +SEEN_CRLF = 4 +SEEN_ALL = SEEN_CR | SEEN_LF | SEEN_CRLF + +class W_IncrementalNewlineDecoder(Wrappable): + seennl = 0 + pendingcr = False + w_decoder = None + + def __init__(self, space): + pass + + @unwrap_spec('self', ObjSpace, W_Root, int, W_Root) + def descr_init(self, space, w_decoder, translate, w_errors=None): + self.w_decoder = w_decoder + self.translate = translate + if space.is_w(w_errors, space.w_None): + self.w_errors = space.wrap("strict") + else: + self.w_errors = w_errors + + self.seennl = 0 + pendingcr = False + + def newlines_get_w(space, self): + return { + SEEN_CR: space.wrap("\r"), + SEEN_LF: space.wrap("\n"), + SEEN_CRLF: space.wrap("\r\n"), + SEEN_CR | SEEN_LF: space.wrap(("\r", "\n")), + SEEN_CR | SEEN_CRLF: space.wrap(("\r", "\r\n")), + SEEN_LF | SEEN_CRLF: space.wrap(("\n", "\r\n")), + SEEN_CR | SEEN_LF | SEEN_CRLF: space.wrap(("\r", "\n", "\r\n")), + }.get(self.seennl) + + @unwrap_spec('self', ObjSpace, W_Root, int) + def decode_w(self, space, w_input, final=False): + if self.w_decoder is None: + raise OperationError(space.w_ValueError, space.wrap( + "IncrementalNewlineDecoder.__init__ not called")) + + # decode input (with the eventual \r from a previous pass) + if not space.is_w(self.w_decoder, space.w_None): + w_output = space.call_method(self.w_decoder, "decode", + w_input, space.wrap(final)) + else: + w_output = w_input + + if not space.isinstance_w(w_output, space.w_unicode): + raise OperationError(space.w_TypeError, space.wrap( + "decoder should return a string result")) + + output = space.unicode_w(w_output) + output_len = len(output) + if self.pendingcr and (final or output_len): + output = u'\r' + output + self.pendingcr = False + output_len += 1 + + # retain last \r even when not translating data: + # then readline() is sure to get \r\n in one pass + if not final and output_len > 0: + last = output_len - 1 + assert last >= 0 + if output[last] == u'\r': + output = output[:last] + self.pendingcr = True + output_len -= 1 + + if output_len == 0: + return space.wrap(u"") + + # Record which newlines are read and do newline translation if + # desired, all in one pass. + seennl = self.seennl + + # If, up to now, newlines are consistently \n, do a quick check + # for the \r + only_lf = False + if seennl == SEEN_LF or seennl == 0: + only_lf = (output.find(u'\r') < 0) + + if only_lf: + # If not already seen, quick scan for a possible "\n" character. + # (there's nothing else to be done, even when in translation mode) + if seennl == 0 and output.find(u'\n') >= 0: + seennl |= SEEN_LF + # Finished: we have scanned for newlines, and none of them + # need translating. + elif not self.translate: + i = 0 + while i < output_len: + if seennl == SEEN_ALL: + break + c = output[i] + i += 1 + if c == u'\n': + seennl |= SEEN_LF + elif c == u'\r': + if i < output_len and output[i] == u'\n': + seennl |= SEEN_CRLF + i += 1 + else: + seennl |= SEEN_CR + elif output.find(u'\r') >= 0: + # Translate! + builder = UnicodeBuilder(output_len) + i = 0 + while i < output_len: + c = output[i] + i += 1 + if c == u'\n': + seennl |= SEEN_LF + elif c == u'\r': + if i < output_len and output[i] == u'\n': + seennl |= SEEN_CRLF + i += 1 + else: + seennl |= SEEN_CR + builder.append(u'\n') + continue + builder.append(c) + output = builder.build() + + self.seennl |= seennl + return space.wrap(output) + + @unwrap_spec('self', ObjSpace) + def reset_w(self, space): + self.seennl = 0 + self.pendingcr = False + if self.w_decoder and not space.is_w(self.w_decoder, space.w_None): + space.call_method(self.w_decoder, "reset") + + @unwrap_spec('self', ObjSpace) + def getstate_w(self, space): + if self.w_decoder and not space.is_w(self.w_decoder, space.w_None): + w_state = space.call_method(self.w_decoder, "getstate") + w_buffer, w_flag = space.unpackiterable(w_state, 2) + flag = space.r_longlong_w(w_flag) + else: + w_buffer = space.wrap("") + flag = 0 + flag <<= 1 + if self.pendingcr: + flag |= 1 + return space.newtuple([w_buffer, space.wrap(flag)]) + + @unwrap_spec('self', ObjSpace, W_Root) + def setstate_w(self, space, w_state): + w_buffer, w_flag = space.unpackiterable(w_state, 2) + flag = space.r_longlong_w(w_flag) + self.pendingcr = (flag & 1) + flag >>= 1 + + if self.w_decoder and not space.is_w(self.w_decoder, space.w_None): + w_state = space.newtuple([w_buffer, space.wrap(flag)]) + space.call_method(self.w_decoder, "setstate", w_state) + +W_IncrementalNewlineDecoder.typedef = TypeDef( + 'TextIOWrapper', + __new__ = generic_new_descr(W_IncrementalNewlineDecoder), + __init__ = interp2app(W_IncrementalNewlineDecoder.descr_init), + + decode = interp2app(W_IncrementalNewlineDecoder.decode_w), + reset = interp2app(W_IncrementalNewlineDecoder.reset_w), + getstate = interp2app(W_IncrementalNewlineDecoder.getstate_w), + setstate = interp2app(W_IncrementalNewlineDecoder.setstate_w), + + newlines = GetSetProperty(W_IncrementalNewlineDecoder.newlines_get_w), + ) + class W_TextIOBase(W_IOBase): w_encoding = None Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Fri Nov 19 13:08:31 2010 @@ -28,3 +28,112 @@ t = _io.TextIOWrapper(b) assert t.readable() assert t.seekable() + +class AppTestIncrementalNewlineDecoder: + + def test_newline_decoder(self): + import _io + def check_newline_decoding_utf8(decoder): + # UTF-8 specific tests for a newline decoder + def _check_decode(b, s, **kwargs): + # We exercise getstate() / setstate() as well as decode() + state = decoder.getstate() + assert decoder.decode(b, **kwargs) == s + decoder.setstate(state) + assert decoder.decode(b, **kwargs) == s + + _check_decode(b'\xe8\xa2\x88', u"\u8888") + + _check_decode(b'\xe8', "") + _check_decode(b'\xa2', "") + _check_decode(b'\x88', u"\u8888") + + _check_decode(b'\xe8', "") + _check_decode(b'\xa2', "") + _check_decode(b'\x88', u"\u8888") + + _check_decode(b'\xe8', "") + raises(UnicodeDecodeError, decoder.decode, b'', final=True) + + decoder.reset() + _check_decode(b'\n', "\n") + _check_decode(b'\r', "") + _check_decode(b'', "\n", final=True) + _check_decode(b'\r', "\n", final=True) + + _check_decode(b'\r', "") + _check_decode(b'a', "\na") + + _check_decode(b'\r\r\n', "\n\n") + _check_decode(b'\r', "") + _check_decode(b'\r', "\n") + _check_decode(b'\na', "\na") + + _check_decode(b'\xe8\xa2\x88\r\n', u"\u8888\n") + _check_decode(b'\xe8\xa2\x88', u"\u8888") + _check_decode(b'\n', "\n") + _check_decode(b'\xe8\xa2\x88\r', u"\u8888") + _check_decode(b'\n', "\n") + + def check_newline_decoding(decoder, encoding): + result = [] + if encoding is not None: + encoder = codecs.getincrementalencoder(encoding)() + def _decode_bytewise(s): + # Decode one byte at a time + for b in encoder.encode(s): + result.append(decoder.decode(b)) + else: + encoder = None + def _decode_bytewise(s): + # Decode one char at a time + for c in s: + result.append(decoder.decode(c)) + assert decoder.newlines == None + _decode_bytewise(u"abc\n\r") + assert decoder.newlines == '\n' + _decode_bytewise(u"\nabc") + assert decoder.newlines == ('\n', '\r\n') + _decode_bytewise(u"abc\r") + assert decoder.newlines == ('\n', '\r\n') + _decode_bytewise(u"abc") + assert decoder.newlines == ('\r', '\n', '\r\n') + _decode_bytewise(u"abc\r") + assert "".join(result) == "abc\n\nabcabc\nabcabc" + decoder.reset() + input = u"abc" + if encoder is not None: + encoder.reset() + input = encoder.encode(input) + assert decoder.decode(input) == "abc" + assert decoder.newlines is None + + encodings = ( + # None meaning the IncrementalNewlineDecoder takes unicode input + # rather than bytes input + None, 'utf-8', 'latin-1', + 'utf-16', 'utf-16-le', 'utf-16-be', + 'utf-32', 'utf-32-le', 'utf-32-be', + ) + import codecs + for enc in encodings: + decoder = enc and codecs.getincrementaldecoder(enc)() + decoder = _io.IncrementalNewlineDecoder(decoder, translate=True) + check_newline_decoding(decoder, enc) + decoder = codecs.getincrementaldecoder("utf-8")() + decoder = _io.IncrementalNewlineDecoder(decoder, translate=True) + check_newline_decoding_utf8(decoder) + + def test_newline_bytes(self): + import _io + # Issue 5433: Excessive optimization in IncrementalNewlineDecoder + def _check(dec): + assert dec.newlines is None + assert dec.decode(u"\u0D00") == u"\u0D00" + assert dec.newlines is None + assert dec.decode(u"\u0A00") == u"\u0A00" + assert dec.newlines is None + dec = _io.IncrementalNewlineDecoder(None, translate=False) + _check(dec) + dec = _io.IncrementalNewlineDecoder(None, translate=True) + _check(dec) From david at codespeak.net Fri Nov 19 13:26:25 2010 From: david at codespeak.net (david at codespeak.net) Date: Fri, 19 Nov 2010 13:26:25 +0100 (CET) Subject: [pypy-svn] r79271 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101119122625.20F485080C@codespeak.net> Author: david Date: Fri Nov 19 13:26:23 2010 New Revision: 79271 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Log: Fix decoding of frame and location information when building a bridge Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Fri Nov 19 13:26:23 2010 @@ -56,6 +56,8 @@ regs = rffi.cast(rffi.CCHARP, frame_loc - (frame_depth + len(r.all_regs))*WORD) i = 3 fail_index = -1 + if self.debug: + import pdb; pdb.set_trace() while(True): i += 1 fail_index += 1 @@ -94,8 +96,10 @@ self.fail_boxes_count = fail_index return descr - def update_bridge_bindings(self, enc, inputargs, regalloc): + def decode_inputargs_and_frame_depth(self, enc, inputargs, regalloc): + locs = [] # first word contains frame depth + frame_depth = self.decode32(enc, 0) + 1 j = 4 for i in range(len(inputargs)): res = enc[j] @@ -113,12 +117,14 @@ assert 0, 'fail' elif res == '\xFC': # stack location stack_loc = self.decode32(enc, j+1) - loc = regalloc.make_sure_var_in_reg(inputargs[i]) - self.mc.LDR_ri(loc.value, r.fp.value, -stack_loc) + loc = regalloc.frame_manager.frame_pos(stack_loc, INT) j += 4 else: # reg location - regalloc.force_allocate_reg(inputargs[i], selected_reg=r.all_regs[ord(res)]) + loc = r.all_regs[ord(res)] j += 1 + locs.append(loc) + return frame_depth, locs + def decode32(self, mem, index): highval = ord(mem[index+3]) if highval >= 128: @@ -283,6 +289,7 @@ # cpu interface def assemble_loop(self, inputargs, operations, looptoken): + self.debug = False longevity = compute_vars_longevity(inputargs, operations) regalloc = ARMRegisterManager(longevity, assembler=self, frame_manager=ARMFrameManager()) self.align() @@ -374,12 +381,14 @@ return False def assemble_bridge(self, faildescr, inputargs, operations): + self.debug = False enc = rffi.cast(rffi.CCHARP, faildescr._failure_recovery_code) longevity = compute_vars_longevity(inputargs, operations) regalloc = ARMRegisterManager(longevity, assembler=self, frame_manager=ARMFrameManager()) bridge_head = self.mc.curraddr() - self.update_bridge_bindings(enc, inputargs, regalloc) + frame_depth, locs = self.decode_inputargs_and_frame_depth(enc, inputargs, regalloc) + regalloc.update_bindings(locs, frame_depth, inputargs) print 'Bridge', inputargs, operations self._walk_operations(operations, regalloc) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/regalloc.py Fri Nov 19 13:26:23 2010 @@ -23,6 +23,28 @@ def call_result_location(self, v): return r.r0 + def update_bindings(self, locs, frame_depth, inputargs): + used = {} + i = 0 + self.frame_manager.frame_depth = frame_depth + for loc in locs: + arg = inputargs[i] + i += 1 + if loc.is_reg(): + self.reg_bindings[arg] = loc + else: + self.frame_manager.frame_bindings[arg] = loc + used[loc] = None + + # XXX combine with x86 code and move to llsupport + self.free_regs = [] + for reg in self.all_regs: + if reg not in used: + self.free_regs.append(reg) + # note: we need to make a copy of inputargs because possibly_free_vars + # is also used on op args, which is a non-resizable list + self.possibly_free_vars(list(inputargs)) + class ARMFrameManager(FrameManager): def __init__(self): FrameManager.__init__(self) From david at codespeak.net Fri Nov 19 13:27:07 2010 From: david at codespeak.net (david at codespeak.net) Date: Fri, 19 Nov 2010 13:27:07 +0100 (CET) Subject: [pypy-svn] r79272 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101119122707.16A03282B9E@codespeak.net> Author: david Date: Fri Nov 19 13:27:05 2010 New Revision: 79272 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Fix an issue when emit a call Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Fri Nov 19 13:27:05 2010 @@ -181,6 +181,7 @@ return self._emit_guard(op, regalloc, c.EQ) def emit_op_guard_false(self, op, regalloc, fcond): + print 'Failargs: ', op.getfailargs() a0 = op.getarg(0) l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) self.mc.CMP_ri(l0.value, 0) @@ -569,7 +570,8 @@ assert isinstance(descr, LoopToken) resbox = TempBox() - self._emit_call(descr._arm_direct_bootstrap_code, op.getarglist(), regalloc, fcond, resbox) + self._emit_call(descr._arm_direct_bootstrap_code, op.getarglist(), + regalloc, fcond, result=resbox) #self.mc.ensure_bytes_available(256) if op.result is None: value = self.cpu.done_with_this_frame_void_v From antocuni at codespeak.net Fri Nov 19 14:23:26 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 19 Nov 2010 14:23:26 +0100 (CET) Subject: [pypy-svn] r79273 - in pypy/branch/jit-free/pypy/module/__pypy__: . test Message-ID: <20101119132326.2F6C65080C@codespeak.net> Author: antocuni Date: Fri Nov 19 14:23:24 2010 New Revision: 79273 Added: pypy/branch/jit-free/pypy/module/__pypy__/interp_debug.py (contents, props changed) pypy/branch/jit-free/pypy/module/__pypy__/test/test_debug.py (contents, props changed) Modified: pypy/branch/jit-free/pypy/module/__pypy__/__init__.py Log: make debug_{start,stop,print} available at app-level Modified: pypy/branch/jit-free/pypy/module/__pypy__/__init__.py ============================================================================== --- pypy/branch/jit-free/pypy/module/__pypy__/__init__.py (original) +++ pypy/branch/jit-free/pypy/module/__pypy__/__init__.py Fri Nov 19 14:23:24 2010 @@ -11,6 +11,10 @@ 'internal_repr' : 'interp_magic.internal_repr', 'bytebuffer' : 'bytebuffer.bytebuffer', 'identity_dict' : 'interp_identitydict.W_IdentityDict', + 'debug_start' : 'interp_debug.debug_start', + 'debug_print' : 'interp_debug.debug_print', + 'debug_stop' : 'interp_debug.debug_stop', + 'debug_print_once' : 'interp_debug.debug_print_once', } def setup_after_space_initialization(self): Added: pypy/branch/jit-free/pypy/module/__pypy__/interp_debug.py ============================================================================== --- (empty file) +++ pypy/branch/jit-free/pypy/module/__pypy__/interp_debug.py Fri Nov 19 14:23:24 2010 @@ -0,0 +1,23 @@ +from pypy.interpreter.gateway import interp2app, NoneNotWrapped, unwrap_spec, ObjSpace +from pypy.interpreter.error import OperationError +from pypy.rlib import debug + + at unwrap_spec(ObjSpace, str) +def debug_start(space, category): + debug.debug_start(category) + + at unwrap_spec(ObjSpace, 'args_w') +def debug_print(space, args_w): + parts = [space.str_w(space.str(w_item)) for w_item in args_w] + debug.debug_print(' '.join(parts)) + + at unwrap_spec(ObjSpace, str) +def debug_stop(space, category): + debug.debug_stop(category) + + + at unwrap_spec(ObjSpace, str, 'args_w') +def debug_print_once(space, category, args_w): + debug_start(space, category) + debug_print(space, args_w) + debug_stop(space, category) Added: pypy/branch/jit-free/pypy/module/__pypy__/test/test_debug.py ============================================================================== --- (empty file) +++ pypy/branch/jit-free/pypy/module/__pypy__/test/test_debug.py Fri Nov 19 14:23:24 2010 @@ -0,0 +1,41 @@ +import py +from pypy.conftest import gettestobjspace +from pypy.rlib import debug + +class AppTestDebug: + def setup_class(cls): + cls.space = gettestobjspace(usemodules=['__pypy__']) + space = cls.space + cls.w_check_log = cls.space.wrap(cls.check_log) + + def setup_method(self, meth): + debug._log = debug.DebugLog() + + def teardown_method(self, meth): + debug._log = None + + @classmethod + def check_log(cls, expected): + assert list(debug._log) == expected + + def test_debug_print(self): + from __pypy__ import debug_start, debug_stop, debug_print + debug_start('my-category') + debug_print('one') + debug_print('two', 3, []) + debug_stop('my-category') + self.check_log([ + ('my-category', [ + ('debug_print', 'one'), + ('debug_print', 'two 3 []'), + ]) + ]) + + def test_debug_print_once(self): + from __pypy__ import debug_print_once + debug_print_once('foobar', 'hello world') + self.check_log([ + ('foobar', [ + ('debug_print', 'hello world'), + ]) + ]) From arigo at codespeak.net Fri Nov 19 14:37:53 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 19 Nov 2010 14:37:53 +0100 (CET) Subject: [pypy-svn] r79274 - pypy/trunk/pypy/translator/c/src Message-ID: <20101119133753.BAD9C282B9D@codespeak.net> Author: arigo Date: Fri Nov 19 14:37:52 2010 New Revision: 79274 Added: pypy/trunk/pypy/translator/c/src/asm_gcc_x86_64.h Modified: pypy/trunk/pypy/translator/c/src/g_include.h Log: Add the high-res timestamp to x86-64 too. Added: pypy/trunk/pypy/translator/c/src/asm_gcc_x86_64.h ============================================================================== --- (empty file) +++ pypy/trunk/pypy/translator/c/src/asm_gcc_x86_64.h Fri Nov 19 14:37:52 2010 @@ -0,0 +1,8 @@ +/* This optional file only works for GCC on an x86-64. + */ + +#define READ_TIMESTAMP(val) do { \ + unsigned int _eax, _edx; \ + asm volatile("rdtsc" : "=a" (_eax), "=d" (_edx)); \ + val = (((unsigned long) _edx) << 32) | _eax; \ +} while (0) Modified: pypy/trunk/pypy/translator/c/src/g_include.h ============================================================================== --- pypy/trunk/pypy/translator/c/src/g_include.h (original) +++ pypy/trunk/pypy/translator/c/src/g_include.h Fri Nov 19 14:37:52 2010 @@ -43,6 +43,10 @@ # include "src/asm_gcc_x86.h" #endif +#if defined(__GNUC__) && defined(__amd64__) +# include "src/asm_gcc_x86_64.h" +#endif + #if defined(__GNUC__) && defined(__ppc__) # include "src/asm_ppc.h" #endif From afa at codespeak.net Fri Nov 19 14:45:14 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 19 Nov 2010 14:45:14 +0100 (CET) Subject: [pypy-svn] r79275 - pypy/branch/fast-forward/pypy/module/_io Message-ID: <20101119134514.43F1A282B9E@codespeak.net> Author: afa Date: Fri Nov 19 14:45:12 2010 New Revision: 79275 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Log: Fix translation Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Fri Nov 19 14:45:12 2010 @@ -20,7 +20,19 @@ w_decoder = None def __init__(self, space): - pass + self.w_newlines_dict = { + SEEN_CR: space.wrap("\r"), + SEEN_LF: space.wrap("\n"), + SEEN_CRLF: space.wrap("\r\n"), + SEEN_CR | SEEN_LF: space.newtuple( + [space.wrap("\r"), space.wrap("\n")]), + SEEN_CR | SEEN_CRLF: space.newtuple( + [space.wrap("\r"), space.wrap("\r\n")]), + SEEN_LF | SEEN_CRLF: space.newtuple( + [space.wrap("\n"), space.wrap("\r\n")]), + SEEN_CR | SEEN_LF | SEEN_CRLF: space.newtuple( + [space.wrap("\r"), space.wrap("\n"), space.wrap("\r\n")]), + } @unwrap_spec('self', ObjSpace, W_Root, int, W_Root) def descr_init(self, space, w_decoder, translate, w_errors=None): @@ -35,15 +47,7 @@ pendingcr = False def newlines_get_w(space, self): - return { - SEEN_CR: space.wrap("\r"), - SEEN_LF: space.wrap("\n"), - SEEN_CRLF: space.wrap("\r\n"), - SEEN_CR | SEEN_LF: space.wrap(("\r", "\n")), - SEEN_CR | SEEN_CRLF: space.wrap(("\r", "\r\n")), - SEEN_LF | SEEN_CRLF: space.wrap(("\n", "\r\n")), - SEEN_CR | SEEN_LF | SEEN_CRLF: space.wrap(("\r", "\n", "\r\n")), - }.get(self.seennl) + return self.w_newlines_dict.get(self.seennl, space.w_None) @unwrap_spec('self', ObjSpace, W_Root, int) def decode_w(self, space, w_input, final=False): @@ -231,7 +235,7 @@ "getpreferredencoding") except OperationError, e: # getpreferredencoding() may also raise ImportError - if not space.match(space, space.w_ImportError): + if not e.match(space, space.w_ImportError): raise self.w_encoding = space.wrap("ascii") if self.w_encoding: From afa at codespeak.net Fri Nov 19 15:09:21 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 19 Nov 2010 15:09:21 +0100 (CET) Subject: [pypy-svn] r79276 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101119140921.CE8BD282BDC@codespeak.net> Author: afa Date: Fri Nov 19 15:09:20 2010 New Revision: 79276 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Log: Use IncrementalNewlineDecoder in TextIOWrapper Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Fri Nov 19 15:09:20 2010 @@ -6,6 +6,7 @@ from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root from pypy.interpreter.error import OperationError, operationerrfmt from pypy.rlib.rstring import UnicodeBuilder +from pypy.module._codecs import interp_codecs STATE_ZERO, STATE_OK, STATE_DETACHED = range(3) @@ -246,6 +247,9 @@ raise OperationError(space.w_IOError, space.wrap( "could not determine default encoding")) + if space.is_w(w_errors, space.w_None): + w_errors = space.wrap("strict") + if space.is_w(w_newline, space.w_None): newline = None else: @@ -256,6 +260,21 @@ self.line_buffering = line_buffering + # XXX self.writenl + readuniversal = not newline # null or empty + readtranslate = newline is None + + # build the decoder object + if space.is_true(space.call_method(w_buffer, "readable")): + w_codec = interp_codecs.lookup_codec(space, + space.str_w(self.w_encoding)) + self.w_decoder = space.call_method(w_codec, + "incrementaldecoder", w_errors) + if readuniversal: + self.w_decoder = space.call_function( + space.gettypeobject(W_IncrementalNewlineDecoder.typedef), + self.w_decoder, space.wrap(readtranslate)) + self.state = STATE_OK def _check_init(self, space): @@ -283,15 +302,17 @@ @unwrap_spec('self', ObjSpace, W_Root) def read_w(self, space, w_size=None): + self._check_init(space) # XXX w_size? w_bytes = space.call_method(self.w_buffer, "read") - return space.call_method(w_bytes, "decode", self.w_encoding) + return space.call_method(self.w_decoder, "decode", w_bytes) @unwrap_spec('self', ObjSpace, W_Root) def readline_w(self, space, w_limit=None): + self._check_init(space) # XXX w_limit? w_bytes = space.call_method(self.w_buffer, "readline") - return space.call_method(w_bytes, "decode", self.w_encoding) + return space.call_method(self.w_decoder, "decode", w_bytes) W_TextIOWrapper.typedef = TypeDef( 'TextIOWrapper', W_TextIOBase.typedef, Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Fri Nov 19 15:09:20 2010 @@ -29,6 +29,13 @@ assert t.readable() assert t.seekable() + def test_newlinetranslate(self): + import _io + r = _io.BytesIO(b"abc\r\ndef\rg") + b = _io.BufferedReader(r, 1000) + t = _io.TextIOWrapper(b) + assert t.read() == u"abc\ndef\ng" + class AppTestIncrementalNewlineDecoder: def test_newline_decoder(self): From afa at codespeak.net Fri Nov 19 16:04:04 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 19 Nov 2010 16:04:04 +0100 (CET) Subject: [pypy-svn] r79277 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101119150404.91CA65080B@codespeak.net> Author: afa Date: Fri Nov 19 16:04:02 2010 New Revision: 79277 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Log: Fix a crash when underlying buffer is not readable Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Fri Nov 19 16:04:02 2010 @@ -218,6 +218,7 @@ def __init__(self, space): W_TextIOBase.__init__(self, space) self.state = STATE_ZERO + self.w_decoder = None @unwrap_spec('self', ObjSpace, W_Root, W_Root, W_Root, W_Root, int) def descr_init(self, space, w_buffer, w_encoding=None, @@ -285,6 +286,10 @@ raise OperationError(space.w_ValueError, space.wrap( "underlying buffer has been detached")) + def _check_closed(self, space, message=None): + self._check_init(space) + W_TextIOBase._check_closed(self, space, message) + @unwrap_spec('self', ObjSpace) def readable_w(self, space): self._check_init(space) @@ -302,7 +307,10 @@ @unwrap_spec('self', ObjSpace, W_Root) def read_w(self, space, w_size=None): - self._check_init(space) + self._check_closed(space) + if not self.w_decoder: + raise OperationError(space.w_IOError, space.wrap("not readable")) + # XXX w_size? w_bytes = space.call_method(self.w_buffer, "read") return space.call_method(self.w_decoder, "decode", w_bytes) Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Fri Nov 19 16:04:02 2010 @@ -29,6 +29,14 @@ assert t.readable() assert t.seekable() + def test_unreadable(self): + import _io + class UnReadable(_io.BytesIO): + def readable(self): + return False + txt = _io.TextIOWrapper(UnReadable()) + raises(IOError, txt.read) + def test_newlinetranslate(self): import _io r = _io.BytesIO(b"abc\r\ndef\rg") From antocuni at codespeak.net Fri Nov 19 16:13:31 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 19 Nov 2010 16:13:31 +0100 (CET) Subject: [pypy-svn] r79278 - in pypy/branch/jit-free/pypy/translator/c: . test Message-ID: <20101119151331.3A55B282BE3@codespeak.net> Author: antocuni Date: Fri Nov 19 16:13:29 2010 New Revision: 79278 Modified: pypy/branch/jit-free/pypy/translator/c/funcgen.py pypy/branch/jit-free/pypy/translator/c/test/test_standalone.py Log: (arigo, antocuni): make debug_{start,stop} working also with non-constant strings (needed by __pypy__.debug*) Modified: pypy/branch/jit-free/pypy/translator/c/funcgen.py ============================================================================== --- pypy/branch/jit-free/pypy/translator/c/funcgen.py (original) +++ pypy/branch/jit-free/pypy/translator/c/funcgen.py Fri Nov 19 16:13:29 2010 @@ -769,17 +769,20 @@ "if (PYPY_HAVE_DEBUG_PRINTS) { fprintf(PYPY_DEBUG_FILE, %s); %s}" % (', '.join(argv), free_line)) + def _op_debug(self, opname, arg): + if isinstance(arg, Constant): + string_literal = c_string_constant(''.join(arg.value.chars)) + return "%s(%s);" % (opname, string_literal) + else: + x = "%s(RPyString_AsCharP(%s));\n" % (opname, self.expr(arg)) + x += "RPyString_FreeCache();" + return x + def OP_DEBUG_START(self, op): - arg = op.args[0] - assert isinstance(arg, Constant) - return "PYPY_DEBUG_START(%s);" % ( - c_string_constant(''.join(arg.value.chars)),) + return self._op_debug('PYPY_DEBUG_START', op.args[0]) def OP_DEBUG_STOP(self, op): - arg = op.args[0] - assert isinstance(arg, Constant) - return "PYPY_DEBUG_STOP(%s);" % ( - c_string_constant(''.join(arg.value.chars)),) + return self._op_debug('PYPY_DEBUG_STOP', op.args[0]) def OP_DEBUG_ASSERT(self, op): return 'RPyAssert(%s, %s);' % (self.expr(op.args[0]), Modified: pypy/branch/jit-free/pypy/translator/c/test/test_standalone.py ============================================================================== --- pypy/branch/jit-free/pypy/translator/c/test/test_standalone.py (original) +++ pypy/branch/jit-free/pypy/translator/c/test/test_standalone.py Fri Nov 19 16:13:29 2010 @@ -388,6 +388,20 @@ assert not err assert path.check(file=0) + def test_debug_print_start_stop_nonconst(self): + def entry_point(argv): + debug_start(argv[1]) + debug_print(argv[2]) + debug_stop(argv[1]) + return 0 + t, cbuilder = self.compile(entry_point) + out, err = cbuilder.cmdexec("foo bar", err=True, env={'PYPYLOG': ':-'}) + lines = err.splitlines() + assert '{foo' in lines[0] + assert 'bar' == lines[1] + assert 'foo}' in lines[2] + + def test_fatal_error(self): def g(x): if x == 1: From david at codespeak.net Fri Nov 19 17:04:10 2010 From: david at codespeak.net (david at codespeak.net) Date: Fri, 19 Nov 2010 17:04:10 +0100 (CET) Subject: [pypy-svn] r79279 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101119160410.C8F94282B9D@codespeak.net> Author: david Date: Fri Nov 19 17:04:08 2010 New Revision: 79279 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Log: Merge int_mul_ovf and guard_(no_)overflow operations and remove the hack used in that case before Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Fri Nov 19 17:04:08 2010 @@ -165,6 +165,7 @@ location: \xFC = stack location \xFD = imm location + emtpy = reg location \xFE = Empty arg """ @@ -173,7 +174,7 @@ reg = regalloc.force_allocate_reg(box) # XXX free this memory # XXX allocate correct amount of memory - mem = lltype.malloc(rffi.CArray(lltype.Char), (len(args)+5)*4, flavor='raw') + mem = lltype.malloc(rffi.CArray(lltype.Char), len(args)*6+9, flavor='raw') # Note, the actual frame depth is one less than the value stored in # regalloc.frame_manager.frame_depth self.encode32(mem, 0, regalloc.frame_manager.frame_depth - 1) @@ -378,6 +379,10 @@ if op.getopnum() == rop.CALL_MAY_FORCE or op.getopnum() == rop.CALL_ASSEMBLER: assert operations[i + 1].getopnum() == rop.GUARD_NOT_FORCED return True + if op.getopnum() == rop.INT_MUL_OVF: + opnum = operations[i + 1].getopnum() + assert opnum == rop.GUARD_OVERFLOW or opnum == rop.GUARD_NO_OVERFLOW + return True return False def assemble_bridge(self, faildescr, inputargs, operations): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Fri Nov 19 17:04:08 2010 @@ -92,7 +92,8 @@ return fcond #ref: http://blogs.arm.com/software-enablement/detecting-overflow-from-mul/ - def emit_op_int_mul_ovf(self, op, regalloc, fcond): + f = False + def emit_guard_int_mul_ovf(self, op, guard, regalloc, fcond): a0 = op.getarg(0) a1 = op.getarg(1) reg1 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) @@ -101,7 +102,10 @@ self.mc.SMULL(res.value, r.ip.value, reg1.value, reg2.value, cond=fcond) self.mc.CMP_rr(r.ip.value, res.value, shifttype=shift.ASR, imm=31, cond=fcond) regalloc.possibly_free_vars_for_op(op) - return 0xF # XXX Remove: hack to show that the prev operation was a mul_ovf + if guard.getopnum() == rop.GUARD_OVERFLOW: + return self._emit_guard(guard, regalloc, c.EQ) + else: + return self._emit_guard(guard, regalloc, c.NE) emit_op_int_floordiv = gen_emit_op_by_helper_call('DIV') emit_op_int_mod = gen_emit_op_by_helper_call('MOD') @@ -165,6 +169,8 @@ def _emit_guard(self, op, regalloc, fcond): descr = op.getdescr() assert isinstance(descr, BasicFailDescr) + if hasattr(op, 'getfailargs'): + print 'Failargs: ', op.getfailargs() descr._arm_guard_code = self.mc.curraddr() memaddr = self._gen_path_to_exit_path(op, op.getfailargs(), regalloc, fcond) descr._failure_recovery_code = memaddr @@ -181,7 +187,6 @@ return self._emit_guard(op, regalloc, c.EQ) def emit_op_guard_false(self, op, regalloc, fcond): - print 'Failargs: ', op.getfailargs() a0 = op.getarg(0) l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) self.mc.CMP_ri(l0.value, 0) @@ -204,13 +209,9 @@ emit_op_guard_isnull = emit_op_guard_false def emit_op_guard_no_overflow(self, op, regalloc, fcond): - if fcond == 0xF: # XXX: hack to check if the prev op was a mul_ovf - return self._emit_guard(op, regalloc, c.NE) return self._emit_guard(op, regalloc, c.VS) def emit_op_guard_overflow(self, op, regalloc, fcond): - if fcond == 0xF: # XXX: hack to check if the prev op was a mul_ovf - return self._emit_guard(op, regalloc, c.EQ) return self._emit_guard(op, regalloc, c.VC) class OpAssembler(object): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Fri Nov 19 17:04:08 2010 @@ -41,6 +41,8 @@ pass def execute_token(self, executable_token): + i = [self.get_latest_value_int(x) for x in range(10)] + print 'Inputargs: ', i addr = executable_token._arm_bootstrap_code assert addr % 8 == 0 func = rffi.cast(lltype.Ptr(self.BOOTSTRAP_TP), addr) From antocuni at codespeak.net Fri Nov 19 17:33:43 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 19 Nov 2010 17:33:43 +0100 (CET) Subject: [pypy-svn] r79280 - pypy/branch/jit-free/pypy/translator Message-ID: <20101119163343.99ADA282B9D@codespeak.net> Author: antocuni Date: Fri Nov 19 17:33:42 2010 New Revision: 79280 Modified: pypy/branch/jit-free/pypy/translator/driver.py Log: print markers about the various translation phases in the log Modified: pypy/branch/jit-free/pypy/translator/driver.py ============================================================================== --- pypy/branch/jit-free/pypy/translator/driver.py (original) +++ pypy/branch/jit-free/pypy/translator/driver.py Fri Nov 19 17:33:42 2010 @@ -19,6 +19,11 @@ log = py.log.Producer("translation") py.log.setconsumer("translation", ansi_log) +try: + from __pypy__ import debug_print_once +except ImportError: + def debug_print_once(*args): + pass def taskdef(taskfunc, deps, title, new_state=None, expected_states=[], idemp=False, earlycheck=None): @@ -283,6 +288,7 @@ return else: self.log.info("%s..." % title) + debug_print_once('gc-collect-task', 'starting', goal) self.timer.start_event(goal) try: instrument = False From afa at codespeak.net Fri Nov 19 18:06:07 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 19 Nov 2010 18:06:07 +0100 (CET) Subject: [pypy-svn] r79281 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101119170607.3F3D8282B9D@codespeak.net> Author: afa Date: Fri Nov 19 18:06:05 2010 New Revision: 79281 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Log: Test and fix Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Fri Nov 19 18:06:05 2010 @@ -801,7 +801,7 @@ tell = interp2app(W_BufferedWriter.tell_w), close = interp2app(W_BufferedWriter.close_w), fileno = interp2app(W_BufferedWriter.fileno_w), - isatty = interp2app(W_BufferedWriter.fileno_w), + isatty = interp2app(W_BufferedWriter.isatty_w), detach = interp2app(W_BufferedWriter.detach_w), truncate = interp2app(W_BufferedWriter.truncate_w), closed = GetSetProperty(W_BufferedWriter.closed_get_w), Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Fri Nov 19 18:06:05 2010 @@ -345,6 +345,7 @@ assert not pair.closed assert pair.readable() assert pair.writable() + assert not pair.isatty() assert pair.read() == "abc" assert pair.write("abc") == 3 From afa at codespeak.net Fri Nov 19 18:50:57 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 19 Nov 2010 18:50:57 +0100 (CET) Subject: [pypy-svn] r79282 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101119175057.76050282B9D@codespeak.net> Author: afa Date: Fri Nov 19 18:50:56 2010 New Revision: 79282 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py pypy/branch/fast-forward/pypy/module/_io/interp_textio.py pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Log: Respect limit of TextIOWrapper.read(limit); buffer decoded chars. Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py Fri Nov 19 18:50:56 2010 @@ -49,6 +49,10 @@ return space.wrap(output) @unwrap_spec('self', ObjSpace, W_Root) + def read1_w(self, space, w_size): + return self.read_w(space, w_size) + + @unwrap_spec('self', ObjSpace, W_Root) def readinto_w(self, space, w_buffer): rwbuffer = space.rwbuffer_w(w_buffer) size = rwbuffer.getlength() @@ -171,6 +175,7 @@ __init__ = interp2app(W_BytesIO.descr_init), read = interp2app(W_BytesIO.read_w), + read1 = interp2app(W_BytesIO.read1_w), readinto = interp2app(W_BytesIO.readinto_w), write = interp2app(W_BytesIO.write_w), truncate = interp2app(W_BytesIO.truncate_w), Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Fri Nov 19 18:50:56 2010 @@ -7,6 +7,7 @@ from pypy.interpreter.error import OperationError, operationerrfmt from pypy.rlib.rstring import UnicodeBuilder from pypy.module._codecs import interp_codecs +from pypy.module._io.interp_iobase import convert_size STATE_ZERO, STATE_OK, STATE_DETACHED = range(3) @@ -220,6 +221,10 @@ self.state = STATE_ZERO self.w_decoder = None + self.decoded_chars = None # buffer for text returned from decoder + self.decoded_chars_used = 0 # offset into _decoded_chars for read() + self.chunk_size = 8192 + @unwrap_spec('self', ObjSpace, W_Root, W_Root, W_Root, W_Root, int) def descr_init(self, space, w_buffer, w_encoding=None, w_errors=None, w_newline=None, line_buffering=0): @@ -305,15 +310,90 @@ self._check_init(space) return space.call_method(self.w_buffer, "seekable") + # _____________________________________________________________ + + def _set_decoded_chars(self, chars): + self.decoded_chars = chars + self.decoded_chars_used = 0 + + def _get_decoded_chars(self, size): + if self.decoded_chars is None: + return u"" + + available = len(self.decoded_chars) - self.decoded_chars_used + if available < 0 or size > available: + size = available + + if self.decoded_chars_used > 0 or size < available: + chars = self.decoded_chars[self.decoded_chars_used: + self.decoded_chars_used + size] + else: + chars = self.decoded_chars + + self.decoded_chars_used += size + return chars + + def _read_chunk(self, space): + """Read and decode the next chunk of data from the BufferedReader. + The return value is True unless EOF was reached. The decoded string + is placed in self._decoded_chars (replacing its previous value). + The entire input chunk is sent to the decoder, though some of it may + remain buffered in the decoder, yet to be converted.""" + + if not self.w_decoder: + raise OperationError(space.w_IOError, space.wrap("not readable")) + + # XXX + # if self.telling... + + # Read a chunk, decode it, and put the result in self._decoded_chars + w_input = space.call_method(self.w_buffer, "read1", + space.wrap(self.chunk_size)) + eof = space.int_w(space.len(w_input)) == 0 + w_decoded = space.call_method(self.w_decoder, "decode", + w_input, space.wrap(eof)) + self._set_decoded_chars(space.unicode_w(w_decoded)) + if space.int_w(space.len(w_decoded)) > 0: + eof = False + + # XXX + # if self.telling... + + return not eof + @unwrap_spec('self', ObjSpace, W_Root) def read_w(self, space, w_size=None): self._check_closed(space) if not self.w_decoder: raise OperationError(space.w_IOError, space.wrap("not readable")) - # XXX w_size? - w_bytes = space.call_method(self.w_buffer, "read") - return space.call_method(self.w_decoder, "decode", w_bytes) + size = convert_size(space, w_size) + if size < 0: + # Read everything + w_bytes = space.call_method(self.w_buffer, "read") + w_decoded = space.call_method(self.w_decoder, "decode", w_bytes) + w_result = space.wrap(self._get_decoded_chars(-1)) + w_final = space.add(w_result, w_decoded) + self.snapshot = None + return w_final + + remaining = size + builder = UnicodeBuilder(size) + + # Keep reading chunks until we have n characters to return + while True: + data = self._get_decoded_chars(remaining) + builder.append(data) + remaining -= len(data) + + if remaining <= 0: # Done + break + + if not self._read_chunk(space): + # EOF + break + + return space.wrap(builder.build()) @unwrap_spec('self', ObjSpace, W_Root) def readline_w(self, space, w_limit=None): Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Fri Nov 19 18:50:56 2010 @@ -44,6 +44,19 @@ t = _io.TextIOWrapper(b) assert t.read() == u"abc\ndef\ng" + def test_one_by_one(self): + import _io + r = _io.BytesIO(b"abc\r\ndef\rg") + t = _io.TextIOWrapper(r) + reads = [] + while True: + c = t.read(1) + assert len(c) <= 1 + if not c: + break + reads.append(c) + assert u''.join(reads) == u"abc\ndef\ng" + class AppTestIncrementalNewlineDecoder: def test_newline_decoder(self): From fijal at codespeak.net Fri Nov 19 19:13:08 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 19 Nov 2010 19:13:08 +0100 (CET) Subject: [pypy-svn] r79283 - in pypy/trunk/pypy: objspace/flow objspace/flow/test translator translator/backendopt translator/backendopt/test Message-ID: <20101119181308.6B474282BE3@codespeak.net> Author: fijal Date: Fri Nov 19 19:13:06 2010 New Revision: 79283 Modified: pypy/trunk/pypy/objspace/flow/model.py pypy/trunk/pypy/objspace/flow/test/test_objspace.py pypy/trunk/pypy/translator/backendopt/mallocprediction.py pypy/trunk/pypy/translator/backendopt/mallocv.py pypy/trunk/pypy/translator/backendopt/raisingop2direct_call.py pypy/trunk/pypy/translator/backendopt/removenoops.py pypy/trunk/pypy/translator/backendopt/storesink.py pypy/trunk/pypy/translator/backendopt/support.py pypy/trunk/pypy/translator/backendopt/test/test_ssa.py pypy/trunk/pypy/translator/exceptiontransform.py pypy/trunk/pypy/translator/simplify.py Log: make SpaceOperations consume less space by making them specialized on exact number of arguments. Saves a bit of space and a bit of time when annotating. Modified: pypy/trunk/pypy/objspace/flow/model.py ============================================================================== --- pypy/trunk/pypy/objspace/flow/model.py (original) +++ pypy/trunk/pypy/objspace/flow/model.py Fri Nov 19 19:13:06 2010 @@ -226,7 +226,7 @@ assert isinstance(a, Variable), a self.inputargs = [mapping.get(a, a) for a in self.inputargs] for op in self.operations: - op.args = [mapping.get(a, a) for a in op.args] + op.args = tuple([mapping.get(a, a) for a in op.args]) op.result = mapping.get(op.result, op.result) self.exitswitch = mapping.get(self.exitswitch, self.exitswitch) for link in self.exits: @@ -329,13 +329,17 @@ """Attempted wrapping of a type that cannot sanely appear in flow graph or during its construction""" - class SpaceOperation(object): __slots__ = "opname args result offset".split() + def __new__(cls, opname, args, result, offset=-1): + if opname == 'direct_call': + assert isinstance(args[0], Constant) + cls = globals().get('SpaceOperation' + str(len(args)), SpaceOperationN) + return object.__new__(cls, opname, args, result, offset) def __init__(self, opname, args, result, offset=-1): self.opname = intern(opname) # operation name - self.args = list(args) # mixed list of var/const + self.args = tuple(args) # mixed list of var/const self.result = result # either Variable or Constant instance self.offset = offset # offset in code string @@ -355,6 +359,58 @@ return "%r = %s(%s)" % (self.result, self.opname, ", ".join(map(repr, self.args))) + def getargs(self): + raise NotImplementedError + def setargs(self, args): + raise NotImplementedError + args = property(getargs, setargs) + +class SpaceOperation0(SpaceOperation): + __slots__ = "opname result offset".split() + + def getargs(self): + return () + def setargs(self, args): + assert not args + args = property(getargs, setargs) + +class SpaceOperation1(SpaceOperation): + __slots__ = "opname result offset arg0".split() + + def getargs(self): + return (self.arg0,) + def setargs(self, args): + self.arg0, = args + args = property(getargs, setargs) + +class SpaceOperation2(SpaceOperation): + __slots__ = "opname result offset arg0 arg1".split() + + def getargs(self): + return (self.arg0, self.arg1) + def setargs(self, args): + self.arg0, self.arg1 = args + args = property(getargs, setargs) + +class SpaceOperation3(SpaceOperation): + __slots__ = "opname result offset arg0 arg1 arg2".split() + + def getargs(self): + return (self.arg0, self.arg1, self.arg2) + def setargs(self, args): + self.arg0, self.arg1, self.arg2 = args + args = property(getargs, setargs) + +class SpaceOperationN(SpaceOperation): + __slots__ = "opname result offset _args".split() + + def getargs(self): + return self._args + def setargs(self, args): + assert isinstance(args, tuple) + self._args = args + args = property(getargs, setargs) + class Atom(object): def __init__(self, name): self.__name__ = name # make save_global happy Modified: pypy/trunk/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/trunk/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/trunk/pypy/objspace/flow/test/test_objspace.py Fri Nov 19 19:13:06 2010 @@ -373,9 +373,9 @@ ops = x.startblock.operations assert len(ops) == 2 assert ops[0].opname == 'simple_call' - assert ops[0].args == [Constant(IndexError)] + assert ops[0].args == (Constant(IndexError),) assert ops[1].opname == 'type' - assert ops[1].args == [ops[0].result] + assert ops[1].args == (ops[0].result,) assert x.startblock.exits[0].args == [ops[1].result, ops[0].result] assert x.startblock.exits[0].target is x.exceptblock Modified: pypy/trunk/pypy/translator/backendopt/mallocprediction.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/mallocprediction.py (original) +++ pypy/trunk/pypy/translator/backendopt/mallocprediction.py Fri Nov 19 19:13:06 2010 @@ -80,7 +80,7 @@ graphvars = [None] * len(op.args) else: graphvars = called_graph.getargs() + [called_graph.getreturnvar()] - for var, graphvar in zip(op.args[1:] + [op.result], graphvars): + for var, graphvar in zip(op.args[1:] + (op.result,), graphvars): varstate = adi.getstate(var) if varstate is None: #print "no varstate" Modified: pypy/trunk/pypy/translator/backendopt/mallocv.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/mallocv.py (original) +++ pypy/trunk/pypy/translator/backendopt/mallocv.py Fri Nov 19 19:13:06 2010 @@ -980,7 +980,7 @@ def handle_op_indirect_call(self, op): v_func = self.rename_nonvirtual(op.args[0], op) if isinstance(v_func, Constant): - op = SpaceOperation('direct_call', [v_func] + op.args[1:-1], + op = SpaceOperation('direct_call', (v_func,) + op.args[1:-1], op.result) return self.handle_op_direct_call(op) else: Modified: pypy/trunk/pypy/translator/backendopt/raisingop2direct_call.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/raisingop2direct_call.py (original) +++ pypy/trunk/pypy/translator/backendopt/raisingop2direct_call.py Fri Nov 19 19:13:06 2010 @@ -1,5 +1,6 @@ from pypy.translator.backendopt.support import log, all_operations, annotate import pypy.rpython.raisingops.raisingops +from pypy.objspace.flow.model import SpaceOperation log = log.raisingop2directcall def is_raisingop(op): @@ -26,7 +27,7 @@ log('starting') seen = {} - for op in all_operations(graphs): + for block, i, op in all_operations(graphs): if not is_raisingop(op): continue func = getattr(pypy.rpython.raisingops.raisingops, op.opname, None) @@ -36,8 +37,10 @@ if op.opname not in seen: seen[op.opname] = 0 seen[op.opname] += 1 - op.args.insert(0, annotate(translator, func, op.result, op.args)) - op.opname = 'direct_call' + arg = annotate(translator, func, op.result, op.args) + block.operations[i] = SpaceOperation('direct_call', + (arg,) + op.args, + op.result, i) #statistics... for k, v in seen.iteritems(): @@ -55,7 +58,7 @@ # op.opname += '_' #selfdiagnostics... assert that there are no more raisingops - for op in all_operations(graphs): + for block, i, op in all_operations(graphs): if is_raisingop(op): log.warning("%s not transformed" % op.opname) Modified: pypy/trunk/pypy/translator/backendopt/removenoops.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/removenoops.py (original) +++ pypy/trunk/pypy/translator/backendopt/removenoops.py Fri Nov 19 19:13:06 2010 @@ -23,7 +23,7 @@ if op is not None: for i in range(len(op.args)): if op.args[i] == op_result: - op.args[i] = op_arg + op.args = op.args[:i] + (op_arg,) + op.args[i + 1:] if (op.opname == "indirect_call" and isinstance(op.args[0], Constant)): op.opname = "direct_call" Modified: pypy/trunk/pypy/translator/backendopt/storesink.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/storesink.py (original) +++ pypy/trunk/pypy/translator/backendopt/storesink.py Fri Nov 19 19:13:06 2010 @@ -1,6 +1,7 @@ from pypy.rpython.lltypesystem.lloperation import llop from pypy.translator.backendopt import removenoops +from pypy.objspace.flow.model import SpaceOperation def has_side_effects(op): if op.opname == 'debug_assert': @@ -22,13 +23,12 @@ for block in graph.iterblocks(): newops = [] cache = {} - for op in block.operations: + for i, op in enumerate(block.operations): if op.opname == 'getfield': tup = (op.args[0], op.args[1].value) res = cache.get(tup, None) if res is not None: - op.opname = 'same_as' - op.args = [res] + op = SpaceOperation('same_as', [res], op.result, i) added_some_same_as = True else: cache[tup] = op.result Modified: pypy/trunk/pypy/translator/backendopt/support.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/support.py (original) +++ pypy/trunk/pypy/translator/backendopt/support.py Fri Nov 19 19:13:06 2010 @@ -21,8 +21,8 @@ def all_operations(graphs): for graph in graphs: for block in graph.iterblocks(): - for op in block.operations: - yield op + for i, op in enumerate(block.operations): + yield block, i, op def annotate(translator, func, result, args): args = [arg.concretetype for arg in args] Modified: pypy/trunk/pypy/translator/backendopt/test/test_ssa.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/test/test_ssa.py (original) +++ pypy/trunk/pypy/translator/backendopt/test/test_ssa.py Fri Nov 19 19:13:06 2010 @@ -75,7 +75,7 @@ assert len(b2.inputargs) == 2 assert len(b3.inputargs) == 2 - assert b2.inputargs == b2.operations[0].args + assert b2.inputargs == list(b2.operations[0].args) assert len(b1.exits[0].args) == 2 assert b1.exits[0].args[1] is c assert len(b2.exits[0].args) == 2 Modified: pypy/trunk/pypy/translator/exceptiontransform.py ============================================================================== --- pypy/trunk/pypy/translator/exceptiontransform.py (original) +++ pypy/trunk/pypy/translator/exceptiontransform.py Fri Nov 19 19:13:06 2010 @@ -234,33 +234,39 @@ def replace_stack_unwind(self, block): for i in range(len(block.operations)): - if block.operations[i].opname == 'stack_unwind': + op = block.operations[i] + if op.opname == 'stack_unwind': # if there are stack_unwind ops left, # the graph was not stackless-transformed # so we need to raise a StackOverflow in any # case - block.operations[i].opname = "direct_call" - block.operations[i].args = [self.rpyexc_raise_stack_overflow_ptr] + args = [self.rpyexc_raise_stack_overflow_ptr] + block.operations[i] = SpaceOperation('direct_call', + args, op.result, i) def replace_fetch_restore_operations(self, block): # the gctransformer will create these operations. It looks as if the # order of transformations is important - but the gctransformer will # put them in a new graph, so all transformations will run again. for i in range(len(block.operations)): - opname = block.operations[i].opname + op = block.operations[i] + opname = op.opname if opname == 'gc_fetch_exception': - block.operations[i].opname = "direct_call" - block.operations[i].args = [self.rpyexc_fetch_exception_ptr] - + args = [self.rpyexc_fetch_exception_ptr] + block.operations[i] = SpaceOperation('direct_call', args, + op.result, i) elif opname == 'gc_restore_exception': - block.operations[i].opname = "direct_call" - block.operations[i].args.insert(0, self.rpyexc_restore_exception_ptr) + args = (self.rpyexc_restore_exception_ptr,) + op.args + block.operations[i] = SpaceOperation('direct_call', args, + op.result, i) elif opname == 'get_exception_addr': # only for lltype - block.operations[i].opname = "direct_call" - block.operations[i].args.insert(0, self.rpyexc_get_exception_addr_ptr) + args = (self.rpyexc_get_exception_addr_ptr,) + op.args + block.operations[i] = SpaceOperation('direct_call', args, + op.result, i) elif opname == 'get_exc_value_addr': # only for lltype - block.operations[i].opname = "direct_call" - block.operations[i].args.insert(0, self.rpyexc_get_exc_value_addr_ptr) + args = (self.rpyexc_get_exc_value_addr_ptr,) + op.args + block.operations[i] = SpaceOperation('direct_call', args, + op.result, i) def transform_block(self, graph, block): need_exc_matching = False Modified: pypy/trunk/pypy/translator/simplify.py ============================================================================== --- pypy/trunk/pypy/translator/simplify.py (original) +++ pypy/trunk/pypy/translator/simplify.py Fri Nov 19 19:13:06 2010 @@ -146,8 +146,9 @@ del block.operations[i] block.renamevariables({op.result: op1.result}) elif op.args[0] == covfls: - op.opname = 'lshift_ovf' - del op.args[0] + block.operations[i] = SpaceOperation('lshift_ovf', + op.args[1:], op.result, + i) def simplify_exceptions(graph): """The exception handling caused by non-implicit exceptions @@ -293,14 +294,13 @@ def rename(v): return renaming.get(v, v) def rename_op(op): + opname = op.opname args = [rename(a) for a in op.args] - op = SpaceOperation(op.opname, args, rename(op.result), op.offset) - # special case... - if op.opname == 'indirect_call': - if isinstance(op.args[0], Constant): - assert isinstance(op.args[-1], Constant) - del op.args[-1] - op.opname = 'direct_call' + if (opname == 'indirect_call' and + isinstance(args[0], Constant)): + opname = 'direct_call' + del args[-1] + op = SpaceOperation(opname, args, rename(op.result), op.offset) return op for op in link.target.operations: link.prevblock.operations.append(rename_op(op)) From afa at codespeak.net Fri Nov 19 19:27:41 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 19 Nov 2010 19:27:41 +0100 (CET) Subject: [pypy-svn] r79284 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101119182741.01EAC282BE3@codespeak.net> Author: afa Date: Fri Nov 19 19:27:40 2010 New Revision: 79284 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Log: Add detach(), flush(), close() Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Fri Nov 19 19:27:40 2010 @@ -207,11 +207,21 @@ def readline_w(self, space, w_limit=None): self._unsupportedoperation(space, "readline") + @unwrap_spec('self', ObjSpace, W_Root) + def write_w(self, space, w_data): + self._unsupportedoperation(space, "write") + + @unwrap_spec('self', ObjSpace) + def detach_w(self, space): + self._unsupportedoperation(space, "detach") + W_TextIOBase.typedef = TypeDef( '_TextIOBase', W_IOBase.typedef, __new__ = generic_new_descr(W_TextIOBase), read = interp2app(W_TextIOBase.read_w), + readline = interp2app(W_TextIOBase.readline_w), + detach = interp2app(W_TextIOBase.detach_w), encoding = interp_attrproperty_w("w_encoding", W_TextIOBase) ) @@ -310,6 +320,25 @@ self._check_init(space) return space.call_method(self.w_buffer, "seekable") + @unwrap_spec('self', ObjSpace) + def fileno_w(self, space): + self._check_init(space) + return space.call_method(self.w_buffer, "fileno") + + @unwrap_spec('self', ObjSpace) + def flush_w(self, space): + self._check_closed(space) + # XXX self.telling = self.seekable + # XXX self._writeflush(space) + space.call_method(self.w_buffer, "flush") + + @unwrap_spec('self', ObjSpace) + def close_w(self, space): + self._check_init(space) + if not self._closed(space): + return space.call_method(self, "flush") + return space.call_method(self.w_buffer, "close") + # _____________________________________________________________ def _set_decoded_chars(self, chars): @@ -402,6 +431,15 @@ w_bytes = space.call_method(self.w_buffer, "readline") return space.call_method(self.w_decoder, "decode", w_bytes) + @unwrap_spec('self', ObjSpace) + def detach_w(self, space): + self._check_init(space) + space.call_method(self, "flush") + w_buffer = self.w_buffer + self.w_buffer = None + self.state = STATE_DETACHED + return w_buffer + W_TextIOWrapper.typedef = TypeDef( 'TextIOWrapper', W_TextIOBase.typedef, __new__ = generic_new_descr(W_TextIOWrapper), @@ -409,9 +447,13 @@ read = interp2app(W_TextIOWrapper.read_w), readline = interp2app(W_TextIOWrapper.readline_w), + detach = interp2app(W_TextIOWrapper.detach_w), + flush = interp2app(W_TextIOWrapper.flush_w), + close = interp2app(W_TextIOWrapper.close_w), line_buffering = interp_attrproperty("line_buffering", W_TextIOWrapper), readable = interp2app(W_TextIOWrapper.readable_w), writable = interp2app(W_TextIOWrapper.writable_w), seekable = interp2app(W_TextIOWrapper.seekable_w), + fileno = interp2app(W_TextIOWrapper.fileno_w), ) Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Fri Nov 19 19:27:40 2010 @@ -37,6 +37,18 @@ txt = _io.TextIOWrapper(UnReadable()) raises(IOError, txt.read) + def test_detach(self): + import _io + b = _io.BytesIO() + f = _io.TextIOWrapper(b) + assert f.detach() is b + raises(ValueError, f.fileno) + raises(ValueError, f.close) + raises(ValueError, f.detach) + raises(ValueError, f.flush) + assert not b.closed + b.close() + def test_newlinetranslate(self): import _io r = _io.BytesIO(b"abc\r\ndef\rg") From afa at codespeak.net Fri Nov 19 19:44:10 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 19 Nov 2010 19:44:10 +0100 (CET) Subject: [pypy-svn] r79285 - pypy/branch/fast-forward/pypy/module/_io Message-ID: <20101119184410.30D075080B@codespeak.net> Author: afa Date: Fri Nov 19 19:44:08 2010 New Revision: 79285 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Log: Translation fix Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Fri Nov 19 19:44:08 2010 @@ -354,8 +354,11 @@ size = available if self.decoded_chars_used > 0 or size < available: - chars = self.decoded_chars[self.decoded_chars_used: - self.decoded_chars_used + size] + start = self.decoded_chars_used + end = self.decoded_chars_used + size + assert start >= 0 + assert end >= 0 + chars = self.decoded_chars[start:end] else: chars = self.decoded_chars From fijal at codespeak.net Fri Nov 19 20:00:14 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 19 Nov 2010 20:00:14 +0100 (CET) Subject: [pypy-svn] r79286 - pypy/branch/out-of-line-guards Message-ID: <20101119190014.E9237282B9D@codespeak.net> Author: fijal Date: Fri Nov 19 20:00:13 2010 New Revision: 79286 Added: pypy/branch/out-of-line-guards/ (props changed) - copied from r79285, pypy/trunk/ Log: Create a branch to play with out-of-line guards (maybe) From fijal at codespeak.net Fri Nov 19 20:28:33 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 19 Nov 2010 20:28:33 +0100 (CET) Subject: [pypy-svn] r79287 - pypy/trunk/pypy/tool/release Message-ID: <20101119192833.56965282BE3@codespeak.net> Author: fijal Date: Fri Nov 19 20:28:31 2010 New Revision: 79287 Modified: pypy/trunk/pypy/tool/release/make_release.py Log: update script Modified: pypy/trunk/pypy/tool/release/make_release.py ============================================================================== --- pypy/trunk/pypy/tool/release/make_release.py (original) +++ pypy/trunk/pypy/tool/release/make_release.py Fri Nov 19 20:28:31 2010 @@ -4,7 +4,7 @@ into release packages. Note: you must run apropriate buildbots first and make sure there are no failures. Use force-builds.py from the same directory. -Usage: make_release.py release/ +Usage: make_release.py release/ release_version """ import autopath @@ -76,7 +76,7 @@ t.add('pypy-%s' % release) alltars.append(name) t.close() - shutil.rmtree(str(tmpdir.join('pypy-1.3'))) + shutil.rmtree(str(tmpdir.join('pypy-' + release))) for name in alltars: print "Uploading %s" % name os.system('scp %s codespeak.net:/www/pypy.org/htdocs/download' % name) @@ -84,8 +84,8 @@ os.chdir(olddir) if __name__ == '__main__': - if len(sys.argv) != 2: + if len(sys.argv) != 3: print __doc__ sys.exit(1) - main(sys.argv[1], release='1.3') + main(sys.argv[1], release=sys.argv[2]) From fijal at codespeak.net Fri Nov 19 21:17:40 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 19 Nov 2010 21:17:40 +0100 (CET) Subject: [pypy-svn] r79288 - in pypy/trunk/pypy/jit/codewriter: . test Message-ID: <20101119201740.F279C282B9D@codespeak.net> Author: fijal Date: Fri Nov 19 21:17:28 2010 New Revision: 79288 Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py Log: Fix codewriter Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/jtransform.py (original) +++ pypy/trunk/pypy/jit/codewriter/jtransform.py Fri Nov 19 21:17:28 2010 @@ -87,17 +87,19 @@ def _do_renaming(self, rename, op): op = SpaceOperation(op.opname, op.args[:], op.result) + args = list(op.args) for i, v in enumerate(op.args): if isinstance(v, Variable): if v in rename: - op.args[i] = rename[v] + args[i] = rename[v] elif isinstance(v, ListOfKind): newlst = [] for x in v: if x in rename: x = rename[x] newlst.append(x) - op.args[i] = ListOfKind(v.kind, newlst) + args[i] = ListOfKind(v.kind, newlst) + op.args = tuple(args) return op def _do_renaming_on_link(self, rename, link): @@ -203,7 +205,7 @@ 'float_ge': 'float_le', }.get(op.opname, op.opname) return SpaceOperation(reversename, - [op.args[1], op.args[0]] + op.args[2:], + (op.args[1], op.args[0],) + op.args[2:], op.result) else: return op @@ -377,7 +379,7 @@ c_func, TP = support.builtin_func_for_spec(self.cpu.rtyper, oopspec_name, argtypes, resulttype, extra, extrakey) - return SpaceOperation('direct_call', [c_func] + args, op.result) + return SpaceOperation('direct_call', (c_func,) + args, op.result) def _do_builtin_call(self, op, oopspec_name=None, args=None, extra=None, extrakey=None): @@ -419,25 +421,25 @@ if op.args[1].value['flavor'] == 'raw': ARRAY = op.args[0].value return self._do_builtin_call(op, 'raw_malloc', - [op.args[2]], + (op.args[2],), extra = (ARRAY,), extrakey = ARRAY) if op.args[0].value == rstr.STR: - return SpaceOperation('newstr', [op.args[2]], op.result) + return SpaceOperation('newstr', (op.args[2],), op.result) elif op.args[0].value == rstr.UNICODE: - return SpaceOperation('newunicode', [op.args[2]], op.result) + return SpaceOperation('newunicode', (op.args[2],), op.result) else: # XXX only strings or simple arrays for now ARRAY = op.args[0].value arraydescr = self.cpu.arraydescrof(ARRAY) - return SpaceOperation('new_array', [arraydescr, op.args[2]], + return SpaceOperation('new_array', (arraydescr, op.args[2]), op.result) def rewrite_op_free(self, op): flags = op.args[1].value assert flags['flavor'] == 'raw' ARRAY = op.args[0].concretetype.TO - return self._do_builtin_call(op, 'raw_free', [op.args[0]], + return self._do_builtin_call(op, 'raw_free', (op.args[0],), extra = (ARRAY,), extrakey = ARRAY) def rewrite_op_getarrayitem(self, op): @@ -626,7 +628,7 @@ if hasattr(rtti._obj, 'destructor_funcptr'): RESULT = lltype.Ptr(STRUCT) assert RESULT == op.result.concretetype - return self._do_builtin_call(op, 'alloc_with_del', [], + return self._do_builtin_call(op, 'alloc_with_del', (), extra = (RESULT, vtable), extrakey = STRUCT) heaptracker.register_known_gctype(self.cpu, vtable, STRUCT) Modified: pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py (original) +++ pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py Fri Nov 19 21:17:28 2010 @@ -220,11 +220,11 @@ if isinstance(name2, str): name2 = name2, name2 if isinstance(v1, Constant) and isinstance(v2, Variable): - assert op1.args == [v2, v1] + assert op1.args == (v2, v1) assert op1.result == v3 assert op1.opname == name2[1] else: - assert op1.args == [v1, v2] + assert op1.args == (v1, v2) assert op1.result == v3 assert op1.opname == name2[0] @@ -237,13 +237,13 @@ op0, op1 = oplist assert op0.opname == 'int_add_ovf' if isinstance(v1, Constant) and isinstance(v2, Variable): - assert op0.args == [v2, v1] + assert op0.args == (v2, v1) assert op0.result == v3 else: - assert op0.args == [v1, v2] + assert op0.args == (v1, v2) assert op0.result == v3 assert op1.opname == '-live-' - assert op1.args == [] + assert op1.args == () assert op1.result is None def test_calls(): @@ -296,7 +296,7 @@ kind = getkind(v.concretetype) assert kind == 'void' or kind[0] in expectedkind assert op1.opname == '-live-' - assert op1.args == [] + assert op1.args == () def direct_call_test(argtypes, restype, expectedkind): op = get_direct_call_op(argtypes, restype) @@ -317,15 +317,16 @@ kind = getkind(v.concretetype) assert kind == 'void' or kind[0] in expectedkind assert op1.opname == '-live-' - assert op1.args == [] + assert op1.args == () def indirect_residual_call_test(argtypes, restype, expectedkind): # an indirect call that is residual in all cases is very similar to # a residual direct call op = get_direct_call_op(argtypes, restype) - op.opname = 'indirect_call' - op.args[0] = varoftype(op.args[0].concretetype) - op.args.append(Constant(['somegraph1', 'somegraph2'], lltype.Void)) + op = SpaceOperation('indirect_call', (varoftype(op.args[0].concretetype),) + + op.args[1:] + (Constant(['somegraph1', 'somegraph2'], + lltype.Void),), + op.result) tr = Transformer(FakeCPU(), FakeResidualIndirectCallControl()) tr.graph = 'someinitialgraph' oplist = tr.rewrite_operation(op) @@ -344,24 +345,25 @@ kind = getkind(v.concretetype) assert kind == 'void' or kind[0] in expectedkind assert op1.opname == '-live-' - assert op1.args == [] + assert op1.args == () def indirect_regular_call_test(argtypes, restype, expectedkind): # a regular indirect call is preceded by a guard_value on the # function address, so that pyjitpl can know which jitcode to follow from pypy.jit.codewriter.flatten import IndirectCallTargets op = get_direct_call_op(argtypes, restype) - op.opname = 'indirect_call' - op.args[0] = varoftype(op.args[0].concretetype) - op.args.append(Constant(['somegraph1', 'somegraph2'], lltype.Void)) + op = SpaceOperation('indirect_call', (varoftype(op.args[0].concretetype),) + + op.args[1:] + (Constant(['somegraph1', 'somegraph2'], + lltype.Void),), + op.result) tr = Transformer(FakeCPU(), FakeRegularIndirectCallControl()) tr.graph = 'someinitialgraph' oplist = tr.rewrite_operation(op) op0gv, op1gv, op0, op1 = oplist assert op0gv.opname == '-live-' - assert op0gv.args == [] + assert op0gv.args == () assert op1gv.opname == 'int_guard_value' - assert op1gv.args == [op.args[0]] + assert op1gv.args == (op.args[0],) assert op1gv.result is None # reskind = getkind(restype)[0] @@ -383,7 +385,7 @@ # False, because this 'residual_call' will likely call further jitcodes # which can do e.g. guard_class or other stuff requiring anyway a -live-. assert op1.opname == '-live-' - assert op1.args == [] + assert op1.args == () def test_getfield(): # XXX a more compact encoding would be possible, something along @@ -412,7 +414,7 @@ op1 = Transformer(FakeCPU()).rewrite_operation(op) assert op1.opname == 'getfield_gc_' + suffix fielddescr = ('fielddescr', S, name) - assert op1.args == [v_parent, fielddescr] + assert op1.args == (v_parent, fielddescr) assert op1.result == v_result def test_getfield_typeptr(): @@ -423,9 +425,9 @@ oplist = Transformer(FakeCPU()).rewrite_operation(op) op0, op1 = oplist assert op0.opname == '-live-' - assert op0.args == [] + assert op0.args == () assert op1.opname == 'guard_class' - assert op1.args == [v_parent] + assert op1.args == (v_parent,) assert op1.result == v_result def test_setfield(): @@ -454,7 +456,7 @@ op1 = Transformer(FakeCPU()).rewrite_operation(op) assert op1.opname == 'setfield_gc_' + suffix fielddescr = ('fielddescr', S, name) - assert op1.args == [v_parent, fielddescr, v_newvalue] + assert op1.args == (v_parent, fielddescr, v_newvalue) assert op1.result is None def test_malloc_new(): @@ -464,7 +466,7 @@ Constant({'flavor': 'gc'}, lltype.Void)], v) op1 = Transformer(FakeCPU()).rewrite_operation(op) assert op1.opname == 'new' - assert op1.args == [('sizedescr', S)] + assert op1.args == (('sizedescr', S),) def test_malloc_new_with_vtable(): vtable = lltype.malloc(rclass.OBJECT_VTABLE, immortal=True) @@ -476,7 +478,7 @@ cpu = FakeCPU() op1 = Transformer(cpu).rewrite_operation(op) assert op1.opname == 'new_with_vtable' - assert op1.args == [('sizedescr', S)] + assert op1.args == (('sizedescr', S),) #assert heaptracker.descr2vtable(cpu, op1.args[0]) == vtable [type check] vtable_int = heaptracker.adr2int(llmemory.cast_ptr_to_adr(vtable)) assert heaptracker.vtable2descr(cpu, vtable_int) == op1.args[0] @@ -498,7 +500,7 @@ assert op0.args[0].value == 'alloc_with_del' # pseudo-function as a str assert list(op0.args[2]) == [] assert op1.opname == '-live-' - assert op1.args == [] + assert op1.args == () def test_rename_on_links(): v1 = Variable() @@ -525,17 +527,17 @@ op = SpaceOperation(opname, [v1, v2], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == opname - assert op1.args == [v1, v2] + assert op1.args == (v1, v2) # op = SpaceOperation(opname, [v1, c0], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == reducedname - assert op1.args == [v1] + assert op1.args == (v1,) # op = SpaceOperation(opname, [c0, v2], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == reducedname - assert op1.args == [v2] + assert op1.args == (v2,) def test_ptr_eq(): v1 = varoftype(rclass.OBJECTPTR) @@ -548,17 +550,17 @@ op = SpaceOperation(opname, [v1, v2], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == opname - assert op1.args == [v1, v2] + assert op1.args == (v1, v2) # op = SpaceOperation(opname, [v1, c0], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == reducedname - assert op1.args == [v1] + assert op1.args == (v1,) # op = SpaceOperation(opname, [c0, v2], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == reducedname - assert op1.args == [v2] + assert op1.args == (v2,) def test_nongc_ptr_eq(): v1 = varoftype(rclass.NONGCOBJECTPTR) @@ -571,27 +573,27 @@ op = SpaceOperation(opname, [v1, v2], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == opname.replace('ptr_', 'int_') - assert op1.args == [v1, v2] + assert op1.args == (v1, v2) # op = SpaceOperation(opname, [v1, c0], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == reducedname - assert op1.args == [v1] + assert op1.args == (v1,) # op = SpaceOperation(opname, [c0, v2], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == reducedname - assert op1.args == [v2] + assert op1.args == (v2,) # op = SpaceOperation('ptr_iszero', [v1], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'int_is_zero' - assert op1.args == [v1] + assert op1.args == (v1,) # op = SpaceOperation('ptr_nonzero', [v1], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'int_is_true' - assert op1.args == [v1] + assert op1.args == (v1,) def test_str_getinteriorarraysize(): v = varoftype(lltype.Ptr(rstr.STR)) @@ -601,7 +603,7 @@ v_result) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'strlen' - assert op1.args == [v] + assert op1.args == (v,) assert op1.result == v_result def test_unicode_getinteriorarraysize(): @@ -612,7 +614,7 @@ v_result) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'unicodelen' - assert op1.args == [v] + assert op1.args == (v,) assert op1.result == v_result def test_str_getinteriorfield(): @@ -624,7 +626,7 @@ v_result) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'strgetitem' - assert op1.args == [v, v_index] + assert op1.args == (v, v_index,) assert op1.result == v_result def test_unicode_getinteriorfield(): @@ -636,7 +638,7 @@ v_result) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'unicodegetitem' - assert op1.args == [v, v_index] + assert op1.args == (v, v_index,) assert op1.result == v_result def test_str_setinteriorfield(): @@ -649,7 +651,7 @@ v_void) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'strsetitem' - assert op1.args == [v, v_index, v_newchr] + assert op1.args == (v, v_index, v_newchr,) assert op1.result == v_void def test_unicode_setinteriorfield(): @@ -662,7 +664,7 @@ v_void) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'unicodesetitem' - assert op1.args == [v, v_index, v_newchr] + assert op1.args == (v, v_index, v_newchr,) assert op1.result == v_void def test_promote_1(): @@ -674,9 +676,9 @@ oplist = Transformer().rewrite_operation(op) op0, op1, op2 = oplist assert op0.opname == '-live-' - assert op0.args == [] + assert op0.args == () assert op1.opname == 'int_guard_value' - assert op1.args == [v1] + assert op1.args == (v1,) assert op1.result is None assert op2 is None @@ -694,9 +696,9 @@ Transformer().optimize_block(block) assert len(block.operations) == 2 assert block.operations[0].opname == '-live-' - assert block.operations[0].args == [] + assert block.operations[0].args == () assert block.operations[1].opname == 'int_guard_value' - assert block.operations[1].args == [v1] + assert block.operations[1].args == (v1,) assert block.operations[1].result is None assert block.exits[0].args == [v1] @@ -725,10 +727,10 @@ assert len(oplist) == 6 assert oplist[0].opname == '-live-' assert oplist[1].opname == 'int_guard_value' - assert oplist[1].args == [v1] + assert oplist[1].args == (v1,) assert oplist[2].opname == '-live-' assert oplist[3].opname == 'int_guard_value' - assert oplist[3].args == [v2] + assert oplist[3].args == (v2,) assert oplist[4].opname == 'jit_merge_point' assert oplist[4].args[0].value == 42 assert list(oplist[4].args[1]) == [v1, v2] @@ -742,7 +744,7 @@ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2) op1 = Transformer(FakeCPU()).rewrite_operation(op) assert op1.opname == 'getfield_gc_i' - assert op1.args == [v1, ('fielddescr', S, 'x')] + assert op1.args == (v1, ('fielddescr', S, 'x'),) assert op1.result == v2 def test_getfield_gc_pure(): @@ -753,7 +755,7 @@ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2) op1 = Transformer(FakeCPU()).rewrite_operation(op) assert op1.opname == 'getfield_gc_i_pure' - assert op1.args == [v1, ('fielddescr', S, 'x')] + assert op1.args == (v1, ('fielddescr', S, 'x'),) assert op1.result == v2 def test_getfield_gc_greenfield(): @@ -771,7 +773,7 @@ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2) op1 = Transformer(FakeCPU(), FakeCC()).rewrite_operation(op) assert op1.opname == 'getfield_gc_i_greenfield' - assert op1.args == [v1, ('fielddescr', S, 'x')] + assert op1.args == (v1, ('fielddescr', S, 'x'),) assert op1.result == v2 def test_int_abs(): @@ -792,7 +794,7 @@ op = SpaceOperation('malloc_varsize', [c_STR, c_flavor, v1], v2) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'newstr' - assert op1.args == [v1] + assert op1.args == (v1,) assert op1.result == v2 def test_str_concat(): From fijal at codespeak.net Fri Nov 19 21:20:40 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 19 Nov 2010 21:20:40 +0100 (CET) Subject: [pypy-svn] r79289 - pypy/branch/spaceop-sep-classes Message-ID: <20101119202040.886E35080B@codespeak.net> Author: fijal Date: Fri Nov 19 21:20:38 2010 New Revision: 79289 Added: pypy/branch/spaceop-sep-classes/ (props changed) - copied from r79288, pypy/trunk/ Log: It would be better to do such changes on branches, bad me, create a branch From fijal at codespeak.net Fri Nov 19 21:25:03 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 19 Nov 2010 21:25:03 +0100 (CET) Subject: [pypy-svn] r79290 - in pypy/trunk/pypy: jit/codewriter jit/codewriter/test objspace/flow objspace/flow/test translator translator/backendopt translator/backendopt/test Message-ID: <20101119202503.97CBF282BDC@codespeak.net> Author: fijal Date: Fri Nov 19 21:25:01 2010 New Revision: 79290 Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py pypy/trunk/pypy/objspace/flow/model.py pypy/trunk/pypy/objspace/flow/test/test_objspace.py pypy/trunk/pypy/translator/backendopt/mallocprediction.py pypy/trunk/pypy/translator/backendopt/mallocv.py pypy/trunk/pypy/translator/backendopt/raisingop2direct_call.py pypy/trunk/pypy/translator/backendopt/removenoops.py pypy/trunk/pypy/translator/backendopt/storesink.py pypy/trunk/pypy/translator/backendopt/support.py pypy/trunk/pypy/translator/backendopt/test/test_ssa.py pypy/trunk/pypy/translator/exceptiontransform.py pypy/trunk/pypy/translator/simplify.py Log: Revert SpaceOperation changes, they're parked on a branch for now Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/jtransform.py (original) +++ pypy/trunk/pypy/jit/codewriter/jtransform.py Fri Nov 19 21:25:01 2010 @@ -87,19 +87,17 @@ def _do_renaming(self, rename, op): op = SpaceOperation(op.opname, op.args[:], op.result) - args = list(op.args) for i, v in enumerate(op.args): if isinstance(v, Variable): if v in rename: - args[i] = rename[v] + op.args[i] = rename[v] elif isinstance(v, ListOfKind): newlst = [] for x in v: if x in rename: x = rename[x] newlst.append(x) - args[i] = ListOfKind(v.kind, newlst) - op.args = tuple(args) + op.args[i] = ListOfKind(v.kind, newlst) return op def _do_renaming_on_link(self, rename, link): @@ -205,7 +203,7 @@ 'float_ge': 'float_le', }.get(op.opname, op.opname) return SpaceOperation(reversename, - (op.args[1], op.args[0],) + op.args[2:], + [op.args[1], op.args[0]] + op.args[2:], op.result) else: return op @@ -379,7 +377,7 @@ c_func, TP = support.builtin_func_for_spec(self.cpu.rtyper, oopspec_name, argtypes, resulttype, extra, extrakey) - return SpaceOperation('direct_call', (c_func,) + args, op.result) + return SpaceOperation('direct_call', [c_func] + args, op.result) def _do_builtin_call(self, op, oopspec_name=None, args=None, extra=None, extrakey=None): @@ -421,25 +419,25 @@ if op.args[1].value['flavor'] == 'raw': ARRAY = op.args[0].value return self._do_builtin_call(op, 'raw_malloc', - (op.args[2],), + [op.args[2]], extra = (ARRAY,), extrakey = ARRAY) if op.args[0].value == rstr.STR: - return SpaceOperation('newstr', (op.args[2],), op.result) + return SpaceOperation('newstr', [op.args[2]], op.result) elif op.args[0].value == rstr.UNICODE: - return SpaceOperation('newunicode', (op.args[2],), op.result) + return SpaceOperation('newunicode', [op.args[2]], op.result) else: # XXX only strings or simple arrays for now ARRAY = op.args[0].value arraydescr = self.cpu.arraydescrof(ARRAY) - return SpaceOperation('new_array', (arraydescr, op.args[2]), + return SpaceOperation('new_array', [arraydescr, op.args[2]], op.result) def rewrite_op_free(self, op): flags = op.args[1].value assert flags['flavor'] == 'raw' ARRAY = op.args[0].concretetype.TO - return self._do_builtin_call(op, 'raw_free', (op.args[0],), + return self._do_builtin_call(op, 'raw_free', [op.args[0]], extra = (ARRAY,), extrakey = ARRAY) def rewrite_op_getarrayitem(self, op): @@ -628,7 +626,7 @@ if hasattr(rtti._obj, 'destructor_funcptr'): RESULT = lltype.Ptr(STRUCT) assert RESULT == op.result.concretetype - return self._do_builtin_call(op, 'alloc_with_del', (), + return self._do_builtin_call(op, 'alloc_with_del', [], extra = (RESULT, vtable), extrakey = STRUCT) heaptracker.register_known_gctype(self.cpu, vtable, STRUCT) Modified: pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py (original) +++ pypy/trunk/pypy/jit/codewriter/test/test_jtransform.py Fri Nov 19 21:25:01 2010 @@ -220,11 +220,11 @@ if isinstance(name2, str): name2 = name2, name2 if isinstance(v1, Constant) and isinstance(v2, Variable): - assert op1.args == (v2, v1) + assert op1.args == [v2, v1] assert op1.result == v3 assert op1.opname == name2[1] else: - assert op1.args == (v1, v2) + assert op1.args == [v1, v2] assert op1.result == v3 assert op1.opname == name2[0] @@ -237,13 +237,13 @@ op0, op1 = oplist assert op0.opname == 'int_add_ovf' if isinstance(v1, Constant) and isinstance(v2, Variable): - assert op0.args == (v2, v1) + assert op0.args == [v2, v1] assert op0.result == v3 else: - assert op0.args == (v1, v2) + assert op0.args == [v1, v2] assert op0.result == v3 assert op1.opname == '-live-' - assert op1.args == () + assert op1.args == [] assert op1.result is None def test_calls(): @@ -296,7 +296,7 @@ kind = getkind(v.concretetype) assert kind == 'void' or kind[0] in expectedkind assert op1.opname == '-live-' - assert op1.args == () + assert op1.args == [] def direct_call_test(argtypes, restype, expectedkind): op = get_direct_call_op(argtypes, restype) @@ -317,16 +317,15 @@ kind = getkind(v.concretetype) assert kind == 'void' or kind[0] in expectedkind assert op1.opname == '-live-' - assert op1.args == () + assert op1.args == [] def indirect_residual_call_test(argtypes, restype, expectedkind): # an indirect call that is residual in all cases is very similar to # a residual direct call op = get_direct_call_op(argtypes, restype) - op = SpaceOperation('indirect_call', (varoftype(op.args[0].concretetype),) - + op.args[1:] + (Constant(['somegraph1', 'somegraph2'], - lltype.Void),), - op.result) + op.opname = 'indirect_call' + op.args[0] = varoftype(op.args[0].concretetype) + op.args.append(Constant(['somegraph1', 'somegraph2'], lltype.Void)) tr = Transformer(FakeCPU(), FakeResidualIndirectCallControl()) tr.graph = 'someinitialgraph' oplist = tr.rewrite_operation(op) @@ -345,25 +344,24 @@ kind = getkind(v.concretetype) assert kind == 'void' or kind[0] in expectedkind assert op1.opname == '-live-' - assert op1.args == () + assert op1.args == [] def indirect_regular_call_test(argtypes, restype, expectedkind): # a regular indirect call is preceded by a guard_value on the # function address, so that pyjitpl can know which jitcode to follow from pypy.jit.codewriter.flatten import IndirectCallTargets op = get_direct_call_op(argtypes, restype) - op = SpaceOperation('indirect_call', (varoftype(op.args[0].concretetype),) - + op.args[1:] + (Constant(['somegraph1', 'somegraph2'], - lltype.Void),), - op.result) + op.opname = 'indirect_call' + op.args[0] = varoftype(op.args[0].concretetype) + op.args.append(Constant(['somegraph1', 'somegraph2'], lltype.Void)) tr = Transformer(FakeCPU(), FakeRegularIndirectCallControl()) tr.graph = 'someinitialgraph' oplist = tr.rewrite_operation(op) op0gv, op1gv, op0, op1 = oplist assert op0gv.opname == '-live-' - assert op0gv.args == () + assert op0gv.args == [] assert op1gv.opname == 'int_guard_value' - assert op1gv.args == (op.args[0],) + assert op1gv.args == [op.args[0]] assert op1gv.result is None # reskind = getkind(restype)[0] @@ -385,7 +383,7 @@ # False, because this 'residual_call' will likely call further jitcodes # which can do e.g. guard_class or other stuff requiring anyway a -live-. assert op1.opname == '-live-' - assert op1.args == () + assert op1.args == [] def test_getfield(): # XXX a more compact encoding would be possible, something along @@ -414,7 +412,7 @@ op1 = Transformer(FakeCPU()).rewrite_operation(op) assert op1.opname == 'getfield_gc_' + suffix fielddescr = ('fielddescr', S, name) - assert op1.args == (v_parent, fielddescr) + assert op1.args == [v_parent, fielddescr] assert op1.result == v_result def test_getfield_typeptr(): @@ -425,9 +423,9 @@ oplist = Transformer(FakeCPU()).rewrite_operation(op) op0, op1 = oplist assert op0.opname == '-live-' - assert op0.args == () + assert op0.args == [] assert op1.opname == 'guard_class' - assert op1.args == (v_parent,) + assert op1.args == [v_parent] assert op1.result == v_result def test_setfield(): @@ -456,7 +454,7 @@ op1 = Transformer(FakeCPU()).rewrite_operation(op) assert op1.opname == 'setfield_gc_' + suffix fielddescr = ('fielddescr', S, name) - assert op1.args == (v_parent, fielddescr, v_newvalue) + assert op1.args == [v_parent, fielddescr, v_newvalue] assert op1.result is None def test_malloc_new(): @@ -466,7 +464,7 @@ Constant({'flavor': 'gc'}, lltype.Void)], v) op1 = Transformer(FakeCPU()).rewrite_operation(op) assert op1.opname == 'new' - assert op1.args == (('sizedescr', S),) + assert op1.args == [('sizedescr', S)] def test_malloc_new_with_vtable(): vtable = lltype.malloc(rclass.OBJECT_VTABLE, immortal=True) @@ -478,7 +476,7 @@ cpu = FakeCPU() op1 = Transformer(cpu).rewrite_operation(op) assert op1.opname == 'new_with_vtable' - assert op1.args == (('sizedescr', S),) + assert op1.args == [('sizedescr', S)] #assert heaptracker.descr2vtable(cpu, op1.args[0]) == vtable [type check] vtable_int = heaptracker.adr2int(llmemory.cast_ptr_to_adr(vtable)) assert heaptracker.vtable2descr(cpu, vtable_int) == op1.args[0] @@ -500,7 +498,7 @@ assert op0.args[0].value == 'alloc_with_del' # pseudo-function as a str assert list(op0.args[2]) == [] assert op1.opname == '-live-' - assert op1.args == () + assert op1.args == [] def test_rename_on_links(): v1 = Variable() @@ -527,17 +525,17 @@ op = SpaceOperation(opname, [v1, v2], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == opname - assert op1.args == (v1, v2) + assert op1.args == [v1, v2] # op = SpaceOperation(opname, [v1, c0], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == reducedname - assert op1.args == (v1,) + assert op1.args == [v1] # op = SpaceOperation(opname, [c0, v2], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == reducedname - assert op1.args == (v2,) + assert op1.args == [v2] def test_ptr_eq(): v1 = varoftype(rclass.OBJECTPTR) @@ -550,17 +548,17 @@ op = SpaceOperation(opname, [v1, v2], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == opname - assert op1.args == (v1, v2) + assert op1.args == [v1, v2] # op = SpaceOperation(opname, [v1, c0], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == reducedname - assert op1.args == (v1,) + assert op1.args == [v1] # op = SpaceOperation(opname, [c0, v2], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == reducedname - assert op1.args == (v2,) + assert op1.args == [v2] def test_nongc_ptr_eq(): v1 = varoftype(rclass.NONGCOBJECTPTR) @@ -573,27 +571,27 @@ op = SpaceOperation(opname, [v1, v2], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == opname.replace('ptr_', 'int_') - assert op1.args == (v1, v2) + assert op1.args == [v1, v2] # op = SpaceOperation(opname, [v1, c0], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == reducedname - assert op1.args == (v1,) + assert op1.args == [v1] # op = SpaceOperation(opname, [c0, v2], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == reducedname - assert op1.args == (v2,) + assert op1.args == [v2] # op = SpaceOperation('ptr_iszero', [v1], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'int_is_zero' - assert op1.args == (v1,) + assert op1.args == [v1] # op = SpaceOperation('ptr_nonzero', [v1], v3) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'int_is_true' - assert op1.args == (v1,) + assert op1.args == [v1] def test_str_getinteriorarraysize(): v = varoftype(lltype.Ptr(rstr.STR)) @@ -603,7 +601,7 @@ v_result) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'strlen' - assert op1.args == (v,) + assert op1.args == [v] assert op1.result == v_result def test_unicode_getinteriorarraysize(): @@ -614,7 +612,7 @@ v_result) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'unicodelen' - assert op1.args == (v,) + assert op1.args == [v] assert op1.result == v_result def test_str_getinteriorfield(): @@ -626,7 +624,7 @@ v_result) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'strgetitem' - assert op1.args == (v, v_index,) + assert op1.args == [v, v_index] assert op1.result == v_result def test_unicode_getinteriorfield(): @@ -638,7 +636,7 @@ v_result) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'unicodegetitem' - assert op1.args == (v, v_index,) + assert op1.args == [v, v_index] assert op1.result == v_result def test_str_setinteriorfield(): @@ -651,7 +649,7 @@ v_void) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'strsetitem' - assert op1.args == (v, v_index, v_newchr,) + assert op1.args == [v, v_index, v_newchr] assert op1.result == v_void def test_unicode_setinteriorfield(): @@ -664,7 +662,7 @@ v_void) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'unicodesetitem' - assert op1.args == (v, v_index, v_newchr,) + assert op1.args == [v, v_index, v_newchr] assert op1.result == v_void def test_promote_1(): @@ -676,9 +674,9 @@ oplist = Transformer().rewrite_operation(op) op0, op1, op2 = oplist assert op0.opname == '-live-' - assert op0.args == () + assert op0.args == [] assert op1.opname == 'int_guard_value' - assert op1.args == (v1,) + assert op1.args == [v1] assert op1.result is None assert op2 is None @@ -696,9 +694,9 @@ Transformer().optimize_block(block) assert len(block.operations) == 2 assert block.operations[0].opname == '-live-' - assert block.operations[0].args == () + assert block.operations[0].args == [] assert block.operations[1].opname == 'int_guard_value' - assert block.operations[1].args == (v1,) + assert block.operations[1].args == [v1] assert block.operations[1].result is None assert block.exits[0].args == [v1] @@ -727,10 +725,10 @@ assert len(oplist) == 6 assert oplist[0].opname == '-live-' assert oplist[1].opname == 'int_guard_value' - assert oplist[1].args == (v1,) + assert oplist[1].args == [v1] assert oplist[2].opname == '-live-' assert oplist[3].opname == 'int_guard_value' - assert oplist[3].args == (v2,) + assert oplist[3].args == [v2] assert oplist[4].opname == 'jit_merge_point' assert oplist[4].args[0].value == 42 assert list(oplist[4].args[1]) == [v1, v2] @@ -744,7 +742,7 @@ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2) op1 = Transformer(FakeCPU()).rewrite_operation(op) assert op1.opname == 'getfield_gc_i' - assert op1.args == (v1, ('fielddescr', S, 'x'),) + assert op1.args == [v1, ('fielddescr', S, 'x')] assert op1.result == v2 def test_getfield_gc_pure(): @@ -755,7 +753,7 @@ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2) op1 = Transformer(FakeCPU()).rewrite_operation(op) assert op1.opname == 'getfield_gc_i_pure' - assert op1.args == (v1, ('fielddescr', S, 'x'),) + assert op1.args == [v1, ('fielddescr', S, 'x')] assert op1.result == v2 def test_getfield_gc_greenfield(): @@ -773,7 +771,7 @@ op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2) op1 = Transformer(FakeCPU(), FakeCC()).rewrite_operation(op) assert op1.opname == 'getfield_gc_i_greenfield' - assert op1.args == (v1, ('fielddescr', S, 'x'),) + assert op1.args == [v1, ('fielddescr', S, 'x')] assert op1.result == v2 def test_int_abs(): @@ -794,7 +792,7 @@ op = SpaceOperation('malloc_varsize', [c_STR, c_flavor, v1], v2) op1 = Transformer().rewrite_operation(op) assert op1.opname == 'newstr' - assert op1.args == (v1,) + assert op1.args == [v1] assert op1.result == v2 def test_str_concat(): Modified: pypy/trunk/pypy/objspace/flow/model.py ============================================================================== --- pypy/trunk/pypy/objspace/flow/model.py (original) +++ pypy/trunk/pypy/objspace/flow/model.py Fri Nov 19 21:25:01 2010 @@ -226,7 +226,7 @@ assert isinstance(a, Variable), a self.inputargs = [mapping.get(a, a) for a in self.inputargs] for op in self.operations: - op.args = tuple([mapping.get(a, a) for a in op.args]) + op.args = [mapping.get(a, a) for a in op.args] op.result = mapping.get(op.result, op.result) self.exitswitch = mapping.get(self.exitswitch, self.exitswitch) for link in self.exits: @@ -329,17 +329,13 @@ """Attempted wrapping of a type that cannot sanely appear in flow graph or during its construction""" + class SpaceOperation(object): __slots__ = "opname args result offset".split() - def __new__(cls, opname, args, result, offset=-1): - if opname == 'direct_call': - assert isinstance(args[0], Constant) - cls = globals().get('SpaceOperation' + str(len(args)), SpaceOperationN) - return object.__new__(cls, opname, args, result, offset) def __init__(self, opname, args, result, offset=-1): self.opname = intern(opname) # operation name - self.args = tuple(args) # mixed list of var/const + self.args = list(args) # mixed list of var/const self.result = result # either Variable or Constant instance self.offset = offset # offset in code string @@ -359,58 +355,6 @@ return "%r = %s(%s)" % (self.result, self.opname, ", ".join(map(repr, self.args))) - def getargs(self): - raise NotImplementedError - def setargs(self, args): - raise NotImplementedError - args = property(getargs, setargs) - -class SpaceOperation0(SpaceOperation): - __slots__ = "opname result offset".split() - - def getargs(self): - return () - def setargs(self, args): - assert not args - args = property(getargs, setargs) - -class SpaceOperation1(SpaceOperation): - __slots__ = "opname result offset arg0".split() - - def getargs(self): - return (self.arg0,) - def setargs(self, args): - self.arg0, = args - args = property(getargs, setargs) - -class SpaceOperation2(SpaceOperation): - __slots__ = "opname result offset arg0 arg1".split() - - def getargs(self): - return (self.arg0, self.arg1) - def setargs(self, args): - self.arg0, self.arg1 = args - args = property(getargs, setargs) - -class SpaceOperation3(SpaceOperation): - __slots__ = "opname result offset arg0 arg1 arg2".split() - - def getargs(self): - return (self.arg0, self.arg1, self.arg2) - def setargs(self, args): - self.arg0, self.arg1, self.arg2 = args - args = property(getargs, setargs) - -class SpaceOperationN(SpaceOperation): - __slots__ = "opname result offset _args".split() - - def getargs(self): - return self._args - def setargs(self, args): - assert isinstance(args, tuple) - self._args = args - args = property(getargs, setargs) - class Atom(object): def __init__(self, name): self.__name__ = name # make save_global happy Modified: pypy/trunk/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/trunk/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/trunk/pypy/objspace/flow/test/test_objspace.py Fri Nov 19 21:25:01 2010 @@ -373,9 +373,9 @@ ops = x.startblock.operations assert len(ops) == 2 assert ops[0].opname == 'simple_call' - assert ops[0].args == (Constant(IndexError),) + assert ops[0].args == [Constant(IndexError)] assert ops[1].opname == 'type' - assert ops[1].args == (ops[0].result,) + assert ops[1].args == [ops[0].result] assert x.startblock.exits[0].args == [ops[1].result, ops[0].result] assert x.startblock.exits[0].target is x.exceptblock Modified: pypy/trunk/pypy/translator/backendopt/mallocprediction.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/mallocprediction.py (original) +++ pypy/trunk/pypy/translator/backendopt/mallocprediction.py Fri Nov 19 21:25:01 2010 @@ -80,7 +80,7 @@ graphvars = [None] * len(op.args) else: graphvars = called_graph.getargs() + [called_graph.getreturnvar()] - for var, graphvar in zip(op.args[1:] + (op.result,), graphvars): + for var, graphvar in zip(op.args[1:] + [op.result], graphvars): varstate = adi.getstate(var) if varstate is None: #print "no varstate" Modified: pypy/trunk/pypy/translator/backendopt/mallocv.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/mallocv.py (original) +++ pypy/trunk/pypy/translator/backendopt/mallocv.py Fri Nov 19 21:25:01 2010 @@ -980,7 +980,7 @@ def handle_op_indirect_call(self, op): v_func = self.rename_nonvirtual(op.args[0], op) if isinstance(v_func, Constant): - op = SpaceOperation('direct_call', (v_func,) + op.args[1:-1], + op = SpaceOperation('direct_call', [v_func] + op.args[1:-1], op.result) return self.handle_op_direct_call(op) else: Modified: pypy/trunk/pypy/translator/backendopt/raisingop2direct_call.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/raisingop2direct_call.py (original) +++ pypy/trunk/pypy/translator/backendopt/raisingop2direct_call.py Fri Nov 19 21:25:01 2010 @@ -1,6 +1,5 @@ from pypy.translator.backendopt.support import log, all_operations, annotate import pypy.rpython.raisingops.raisingops -from pypy.objspace.flow.model import SpaceOperation log = log.raisingop2directcall def is_raisingop(op): @@ -27,7 +26,7 @@ log('starting') seen = {} - for block, i, op in all_operations(graphs): + for op in all_operations(graphs): if not is_raisingop(op): continue func = getattr(pypy.rpython.raisingops.raisingops, op.opname, None) @@ -37,10 +36,8 @@ if op.opname not in seen: seen[op.opname] = 0 seen[op.opname] += 1 - arg = annotate(translator, func, op.result, op.args) - block.operations[i] = SpaceOperation('direct_call', - (arg,) + op.args, - op.result, i) + op.args.insert(0, annotate(translator, func, op.result, op.args)) + op.opname = 'direct_call' #statistics... for k, v in seen.iteritems(): @@ -58,7 +55,7 @@ # op.opname += '_' #selfdiagnostics... assert that there are no more raisingops - for block, i, op in all_operations(graphs): + for op in all_operations(graphs): if is_raisingop(op): log.warning("%s not transformed" % op.opname) Modified: pypy/trunk/pypy/translator/backendopt/removenoops.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/removenoops.py (original) +++ pypy/trunk/pypy/translator/backendopt/removenoops.py Fri Nov 19 21:25:01 2010 @@ -23,7 +23,7 @@ if op is not None: for i in range(len(op.args)): if op.args[i] == op_result: - op.args = op.args[:i] + (op_arg,) + op.args[i + 1:] + op.args[i] = op_arg if (op.opname == "indirect_call" and isinstance(op.args[0], Constant)): op.opname = "direct_call" Modified: pypy/trunk/pypy/translator/backendopt/storesink.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/storesink.py (original) +++ pypy/trunk/pypy/translator/backendopt/storesink.py Fri Nov 19 21:25:01 2010 @@ -1,7 +1,6 @@ from pypy.rpython.lltypesystem.lloperation import llop from pypy.translator.backendopt import removenoops -from pypy.objspace.flow.model import SpaceOperation def has_side_effects(op): if op.opname == 'debug_assert': @@ -23,12 +22,13 @@ for block in graph.iterblocks(): newops = [] cache = {} - for i, op in enumerate(block.operations): + for op in block.operations: if op.opname == 'getfield': tup = (op.args[0], op.args[1].value) res = cache.get(tup, None) if res is not None: - op = SpaceOperation('same_as', [res], op.result, i) + op.opname = 'same_as' + op.args = [res] added_some_same_as = True else: cache[tup] = op.result Modified: pypy/trunk/pypy/translator/backendopt/support.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/support.py (original) +++ pypy/trunk/pypy/translator/backendopt/support.py Fri Nov 19 21:25:01 2010 @@ -21,8 +21,8 @@ def all_operations(graphs): for graph in graphs: for block in graph.iterblocks(): - for i, op in enumerate(block.operations): - yield block, i, op + for op in block.operations: + yield op def annotate(translator, func, result, args): args = [arg.concretetype for arg in args] Modified: pypy/trunk/pypy/translator/backendopt/test/test_ssa.py ============================================================================== --- pypy/trunk/pypy/translator/backendopt/test/test_ssa.py (original) +++ pypy/trunk/pypy/translator/backendopt/test/test_ssa.py Fri Nov 19 21:25:01 2010 @@ -75,7 +75,7 @@ assert len(b2.inputargs) == 2 assert len(b3.inputargs) == 2 - assert b2.inputargs == list(b2.operations[0].args) + assert b2.inputargs == b2.operations[0].args assert len(b1.exits[0].args) == 2 assert b1.exits[0].args[1] is c assert len(b2.exits[0].args) == 2 Modified: pypy/trunk/pypy/translator/exceptiontransform.py ============================================================================== --- pypy/trunk/pypy/translator/exceptiontransform.py (original) +++ pypy/trunk/pypy/translator/exceptiontransform.py Fri Nov 19 21:25:01 2010 @@ -234,39 +234,33 @@ def replace_stack_unwind(self, block): for i in range(len(block.operations)): - op = block.operations[i] - if op.opname == 'stack_unwind': + if block.operations[i].opname == 'stack_unwind': # if there are stack_unwind ops left, # the graph was not stackless-transformed # so we need to raise a StackOverflow in any # case - args = [self.rpyexc_raise_stack_overflow_ptr] - block.operations[i] = SpaceOperation('direct_call', - args, op.result, i) + block.operations[i].opname = "direct_call" + block.operations[i].args = [self.rpyexc_raise_stack_overflow_ptr] def replace_fetch_restore_operations(self, block): # the gctransformer will create these operations. It looks as if the # order of transformations is important - but the gctransformer will # put them in a new graph, so all transformations will run again. for i in range(len(block.operations)): - op = block.operations[i] - opname = op.opname + opname = block.operations[i].opname if opname == 'gc_fetch_exception': - args = [self.rpyexc_fetch_exception_ptr] - block.operations[i] = SpaceOperation('direct_call', args, - op.result, i) + block.operations[i].opname = "direct_call" + block.operations[i].args = [self.rpyexc_fetch_exception_ptr] + elif opname == 'gc_restore_exception': - args = (self.rpyexc_restore_exception_ptr,) + op.args - block.operations[i] = SpaceOperation('direct_call', args, - op.result, i) + block.operations[i].opname = "direct_call" + block.operations[i].args.insert(0, self.rpyexc_restore_exception_ptr) elif opname == 'get_exception_addr': # only for lltype - args = (self.rpyexc_get_exception_addr_ptr,) + op.args - block.operations[i] = SpaceOperation('direct_call', args, - op.result, i) + block.operations[i].opname = "direct_call" + block.operations[i].args.insert(0, self.rpyexc_get_exception_addr_ptr) elif opname == 'get_exc_value_addr': # only for lltype - args = (self.rpyexc_get_exc_value_addr_ptr,) + op.args - block.operations[i] = SpaceOperation('direct_call', args, - op.result, i) + block.operations[i].opname = "direct_call" + block.operations[i].args.insert(0, self.rpyexc_get_exc_value_addr_ptr) def transform_block(self, graph, block): need_exc_matching = False Modified: pypy/trunk/pypy/translator/simplify.py ============================================================================== --- pypy/trunk/pypy/translator/simplify.py (original) +++ pypy/trunk/pypy/translator/simplify.py Fri Nov 19 21:25:01 2010 @@ -146,9 +146,8 @@ del block.operations[i] block.renamevariables({op.result: op1.result}) elif op.args[0] == covfls: - block.operations[i] = SpaceOperation('lshift_ovf', - op.args[1:], op.result, - i) + op.opname = 'lshift_ovf' + del op.args[0] def simplify_exceptions(graph): """The exception handling caused by non-implicit exceptions @@ -294,13 +293,14 @@ def rename(v): return renaming.get(v, v) def rename_op(op): - opname = op.opname args = [rename(a) for a in op.args] - if (opname == 'indirect_call' and - isinstance(args[0], Constant)): - opname = 'direct_call' - del args[-1] - op = SpaceOperation(opname, args, rename(op.result), op.offset) + op = SpaceOperation(op.opname, args, rename(op.result), op.offset) + # special case... + if op.opname == 'indirect_call': + if isinstance(op.args[0], Constant): + assert isinstance(op.args[-1], Constant) + del op.args[-1] + op.opname = 'direct_call' return op for op in link.target.operations: link.prevblock.operations.append(rename_op(op)) From wlav at codespeak.net Fri Nov 19 22:11:24 2010 From: wlav at codespeak.net (wlav at codespeak.net) Date: Fri, 19 Nov 2010 22:11:24 +0100 (CET) Subject: [pypy-svn] r79291 - in pypy/branch/reflex-support/pypy/module/cppyy: . test Message-ID: <20101119211124.E12E0282BDC@codespeak.net> Author: wlav Date: Fri Nov 19 22:11:17 2010 New Revision: 79291 Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py pypy/branch/reflex-support/pypy/module/cppyy/helper.py pypy/branch/reflex-support/pypy/module/cppyy/test/test_helper.py Log: reduce number of possible call points in get_converter Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/converter.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/converter.py Fri Nov 19 22:11:17 2010 @@ -19,7 +19,7 @@ class TypeConverter(object): - def __init__(self, space, extra=-1): + def __init__(self, space, array_size): pass def _get_fieldptr(self, space, w_obj, offset): @@ -43,6 +43,14 @@ lltype.free(arg, flavor='raw') +class ArrayTypeConverter(TypeConverter): + def __init__(self, space, array_size): + if array_size <= 0: + self.size = sys.maxint + else: + self.size = array_size + + class VoidConverter(TypeConverter): def __init__(self, space, name): self.name = name @@ -178,11 +186,8 @@ return rffi.cast(rffi.VOIDP, x) -class ShortPtrConverter(TypeConverter): +class ShortPtrConverter(ArrayTypeConverter): _immutable_ = True - def __init__(self, space, detail=sys.maxint): - self.size = detail - def convert_argument(self, space, w_obj): assert 0, "not yet implemented" @@ -209,11 +214,8 @@ for i in range(min(self.size*2, buf.getlength())): fieldptr[i] = buf.getitem(i) -class LongPtrConverter(TypeConverter): +class LongPtrConverter(ArrayTypeConverter): _immutable_ = True - def __init__(self, space, detail=sys.maxint): - self.size = detail - def convert_argument(self, space, w_obj): assert 0, "not yet implemented" @@ -276,7 +278,7 @@ # 1) full, exact match try: - return _converters[name](space) + return _converters[name](space, -1) except KeyError, k: pass @@ -284,17 +286,20 @@ compound = helper.compound(name) clean_name = helper.clean_type(name) try: - array_index = helper.array_index(name) - if array_index: - return _converters[clean_name+compound](space, array_index) - return _converters[clean_name+compound](space) + # array_index may be negative to indicate no size or no size found + array_size = helper.array_size(name) + return _converters[clean_name+compound](space, array_size) except KeyError, k: pass - + + # 5) generalized cases (covers basically all user classes) cpptype = interp_cppyy.type_byname(space, clean_name) if compound == "*": return InstancePtrConverter(space, cpptype) + + # 6) void converter, which fails on use + # # return a void converter here, so that the class can be build even # when some types are unknown; this overload will simply fail on use return VoidConverter(space, name) Modified: pypy/branch/reflex-support/pypy/module/cppyy/helper.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/helper.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/helper.py Fri Nov 19 22:11:17 2010 @@ -7,7 +7,7 @@ i = _find_qualifier_index(name) return "".join(name[i:].split(" ")) -def array_index(name): +def array_size(name): name = "".join(rstring.split(name, "const")) # poor man's replace if name[-1] == "]": # array type i = _find_qualifier_index(name) @@ -15,7 +15,7 @@ c = name[i] if c == "[": return int(name[i+1:-1]) - return 0 + return -1 def _find_qualifier_index(name): i = len(name) Modified: pypy/branch/reflex-support/pypy/module/cppyy/test/test_helper.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/test/test_helper.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/test/test_helper.py Fri Nov 19 22:11:17 2010 @@ -5,7 +5,7 @@ assert helper.compound("int* const *&") == "**&" assert helper.compound("std::vector*") == "*" assert helper.compound("unsigned long int[5]") == "[]" - assert helper.array_index("unsigned long int[5]") == 5 + assert helper.array_size("unsigned long int[5]") == 5 def test_clean_type(): From wlav at codespeak.net Sat Nov 20 03:25:46 2010 From: wlav at codespeak.net (wlav at codespeak.net) Date: Sat, 20 Nov 2010 03:25:46 +0100 (CET) Subject: [pypy-svn] r79292 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101120022546.40941282B9D@codespeak.net> Author: wlav Date: Sat Nov 20 03:25:43 2010 New Revision: 79292 Modified: pypy/branch/reflex-support/pypy/module/cppyy/helper.py Log: fixes for rtyper: make sure iterator stop is non-neg Modified: pypy/branch/reflex-support/pypy/module/cppyy/helper.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/helper.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/helper.py Sat Nov 20 03:25:43 2010 @@ -2,24 +2,25 @@ def compound(name): name = "".join(rstring.split(name, "const")) # poor man's replace - if name[-1] == "]": # array type + if name.endswith("]"): # array type? return "[]" i = _find_qualifier_index(name) return "".join(name[i:].split(" ")) def array_size(name): name = "".join(rstring.split(name, "const")) # poor man's replace - if name[-1] == "]": # array type - i = _find_qualifier_index(name) - for i in range(len(name) - 1, -1, -1): - c = name[i] - if c == "[": - return int(name[i+1:-1]) + if name.endswith("]"): # array type? + idx = name.rfind("[") + if 0 < idx: + end = len(name)-1 # len rather than -1 for rpython + if 0 < end and (idx+1) < end: # guarantee non-neg for rpython + return int(name[idx+1:end]) return -1 def _find_qualifier_index(name): i = len(name) - for i in range(len(name) - 1, -1, -1): + # search from the back; note len(name) > 0 (so rtyper can use uint) + for i in range(len(name) - 1, 0, -1): c = name[i] if c.isalnum() or c == ">" or c == "]": break @@ -29,9 +30,8 @@ assert name.find("const") == -1 i = _find_qualifier_index(name) name = name[:i].strip(' ') - if name[-1] == "]": - for i in range(len(name) - 1, -1, -1): - c = name[i] - if c == "[": - return name[:i] + if name.endswith("]"): # array type? + idx = name.rfind("[") + if 0 < idx: + return name[:idx] return name From fijal at codespeak.net Sat Nov 20 09:53:55 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sat, 20 Nov 2010 09:53:55 +0100 (CET) Subject: [pypy-svn] r79293 - pypy/trunk/pypy/doc Message-ID: <20101120085355.F2744282B9D@codespeak.net> Author: fijal Date: Sat Nov 20 09:53:53 2010 New Revision: 79293 Modified: pypy/trunk/pypy/doc/release-1.4.0.txt Log: I think 64bit is also stable - I use it for everyday work Modified: pypy/trunk/pypy/doc/release-1.4.0.txt ============================================================================== --- pypy/trunk/pypy/doc/release-1.4.0.txt (original) +++ pypy/trunk/pypy/doc/release-1.4.0.txt Sat Nov 20 09:53:53 2010 @@ -10,9 +10,9 @@ own development. Among other features, this release includes numerous performance improvements -(which made fast self-hosting possible), a 64-bit JIT backend, as well as serious -stabilization. As of now, we can consider the 32-bit version of PyPy stable enough -to run in production. +(which made fast self-hosting possible), a 64-bit JIT backend, as well +as serious stabilization. As of now, we can consider the 32-bit and 64-bit +linux versions of PyPy stable enoughto run in production. More highlights =============== From david at codespeak.net Sat Nov 20 12:46:19 2010 From: david at codespeak.net (david at codespeak.net) Date: Sat, 20 Nov 2010 12:46:19 +0100 (CET) Subject: [pypy-svn] r79294 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101120114619.438D7282B9D@codespeak.net> Author: david Date: Sat Nov 20 12:46:15 2010 New Revision: 79294 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Correct imm value loading in guard_value Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Sat Nov 20 12:46:15 2010 @@ -196,8 +196,9 @@ def emit_op_guard_value(self, op, regalloc, fcond): a0 = op.getarg(0) a1 = op.getarg(1) + imm_a1 = self._check_imm_arg(a1) l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) - l1 = regalloc.make_sure_var_in_reg(a1) + l1 = regalloc.make_sure_var_in_reg(a1, imm_fine=imm_a1) if l1.is_imm(): self.mc.CMP_ri(l0.value, l1.getint()) else: From arigo at codespeak.net Sat Nov 20 14:58:57 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 20 Nov 2010 14:58:57 +0100 (CET) Subject: [pypy-svn] r79295 - pypy/trunk/pypy/module/pypyjit Message-ID: <20101120135857.6BD58282B9E@codespeak.net> Author: arigo Date: Sat Nov 20 14:58:54 2010 New Revision: 79295 Modified: pypy/trunk/pypy/module/pypyjit/__init__.py Log: Remove the XXX comment, which is wrong nowadays. Modified: pypy/trunk/pypy/module/pypyjit/__init__.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/__init__.py (original) +++ pypy/trunk/pypy/module/pypyjit/__init__.py Sat Nov 20 14:58:54 2010 @@ -15,6 +15,5 @@ # add the 'defaults' attribute from pypy.rlib.jit import PARAMETERS space = self.space - # XXX this is not really the default compiled into a pypy-c-jit XXX w_obj = space.wrap(PARAMETERS) space.setattr(space.wrap(self), space.wrap('defaults'), w_obj) From arigo at codespeak.net Sat Nov 20 15:03:43 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 20 Nov 2010 15:03:43 +0100 (CET) Subject: [pypy-svn] r79296 - pypy/trunk/pypy/translator/goal Message-ID: <20101120140343.CFF95282BD6@codespeak.net> Author: arigo Date: Sat Nov 20 15:03:41 2010 New Revision: 79296 Modified: pypy/trunk/pypy/translator/goal/targetpypystandalone.py Log: This lets the file produced by PYPYJITLOG contain the name of the executable. Useful for loading symbols. Modified: pypy/trunk/pypy/translator/goal/targetpypystandalone.py ============================================================================== --- pypy/trunk/pypy/translator/goal/targetpypystandalone.py (original) +++ pypy/trunk/pypy/translator/goal/targetpypystandalone.py Sat Nov 20 15:03:41 2010 @@ -28,9 +28,14 @@ w_call_finish_gateway = space.wrap(gateway.interp2app(call_finish)) w_call_startup_gateway = space.wrap(gateway.interp2app(call_startup)) w_os = setup_nanos(space) + withjit = space.config.objspace.usemodules.pypyjit def entry_point(argv): space.timer.start("Entrypoint") + if withjit: + from pypy.jit.backend.hlinfo import highleveljitinfo + highleveljitinfo.sys_executable = argv[0] + #debug("entry point starting") #for arg in argv: # debug(" argv -> " + arg) From arigo at codespeak.net Sat Nov 20 15:08:50 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 20 Nov 2010 15:08:50 +0100 (CET) Subject: [pypy-svn] r79297 - in pypy/trunk/pypy/translator/c: src test Message-ID: <20101120140850.A30A45080B@codespeak.net> Author: arigo Date: Sat Nov 20 15:08:49 2010 New Revision: 79297 Modified: pypy/trunk/pypy/translator/c/src/debug_print.h pypy/trunk/pypy/translator/c/test/test_standalone.py Log: We can now say 'PYPYLOG=gc-collect,jit-log-opt:logfile'. Modified: pypy/trunk/pypy/translator/c/src/debug_print.h ============================================================================== --- pypy/trunk/pypy/translator/c/src/debug_print.h (original) +++ pypy/trunk/pypy/translator/c/src/debug_print.h Sat Nov 20 15:08:49 2010 @@ -10,6 +10,7 @@ but not any nested debug_print :fname full logging prefix:fname conditional logging + prefix1,prefix2:fname conditional logging with multiple selections Conditional logging means that it only includes the debug_start/debug_stop sections whose name match 'prefix'. Other sections are ignored, including @@ -70,6 +71,8 @@ static void pypy_debug_open(void) { char *filename = getenv("PYPYLOG"); + if (filename) + unsetenv("PYPYLOG"); /* don't pass it to subprocesses */ if (filename && filename[0]) { char *colon = strchr(filename, ':'); @@ -139,12 +142,22 @@ #endif -static bool_t startswith(const char *str, const char *substr) +static bool_t startswithoneof(const char *str, const char *substr) { - while (*substr) - if (*str++ != *substr++) - return 0; - return 1; + const char *p = str; + for (; *substr; substr++) + { + if (*substr != ',') + { + if (p && *p++ != *substr) + p = NULL; /* mismatch */ + } + else if (p != NULL) + return 1; /* match */ + else + p = str; /* mismatched, retry with the next */ + } + return p != NULL; } #if defined(_MSC_VER) || defined(__MINGW32__) @@ -175,7 +188,7 @@ if (!debug_profile) { /* non-profiling version */ - if (!debug_prefix || !startswith(category, debug_prefix)) + if (!debug_prefix || !startswithoneof(category, debug_prefix)) { /* wrong section name, or no PYPYLOG at all, skip it */ return; Modified: pypy/trunk/pypy/translator/c/test/test_standalone.py ============================================================================== --- pypy/trunk/pypy/translator/c/test/test_standalone.py (original) +++ pypy/trunk/pypy/translator/c/test/test_standalone.py Sat Nov 20 15:08:49 2010 @@ -368,12 +368,27 @@ assert not err assert path.check(file=1) data = path.read() - assert 'toplevel' in path.read() - assert 'mycat' not in path.read() - assert 'foo 2 bar 3' not in path.read() + assert 'toplevel' in data + assert 'mycat' not in data + assert 'foo 2 bar 3' not in data assert 'cat2' in data assert 'baz' in data assert 'bok' not in data + # check with PYPYLOG=myc,cat2:somefilename (includes mycat and cat2) + path = udir.join('test_debug_xxx_myc_cat2.log') + out, err = cbuilder.cmdexec("", err=True, + env={'PYPYLOG': 'myc,cat2:%s' % path}) + assert out.strip() == 'got:bcda.' + assert not err + assert path.check(file=1) + data = path.read() + assert 'toplevel' in data + assert '{mycat' in data + assert 'mycat}' in data + assert 'foo 2 bar 3' in data + assert 'cat2' in data + assert 'baz' in data + assert 'bok' in data # # finally, check compiling with logging disabled from pypy.config.pypyoption import get_pypy_config From arigo at codespeak.net Sat Nov 20 15:09:54 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 20 Nov 2010 15:09:54 +0100 (CET) Subject: [pypy-svn] r79298 - in pypy/branch/jit-free/pypy/translator/c: . test Message-ID: <20101120140954.CEB4A282BDC@codespeak.net> Author: arigo Date: Sat Nov 20 15:09:53 2010 New Revision: 79298 Modified: pypy/branch/jit-free/pypy/translator/c/funcgen.py pypy/branch/jit-free/pypy/translator/c/test/test_standalone.py Log: Win32 support, and test. Modified: pypy/branch/jit-free/pypy/translator/c/funcgen.py ============================================================================== --- pypy/branch/jit-free/pypy/translator/c/funcgen.py (original) +++ pypy/branch/jit-free/pypy/translator/c/funcgen.py Sat Nov 20 15:09:53 2010 @@ -1,3 +1,4 @@ +import sys from pypy.translator.c.support import USESLOTS # set to False if necessary while refactoring from pypy.translator.c.support import cdecl from pypy.translator.c.support import llvalue_from_constant, gen_assignments @@ -758,9 +759,15 @@ argv.append('(%s) ? "True" : "False"' % self.expr(arg)) continue elif T == SignedLongLong: - format.append('%lld') + if sys.platform == 'win32': + format.append('%I64d') + else: + format.append('%lld') elif T == UnsignedLongLong: - format.append('%llu') + if sys.platform == 'win32': + format.append('%I64u') + else: + format.append('%llu') else: raise Exception("don't know how to debug_print %r" % (T,)) argv.append(self.expr(arg)) Modified: pypy/branch/jit-free/pypy/translator/c/test/test_standalone.py ============================================================================== --- pypy/branch/jit-free/pypy/translator/c/test/test_standalone.py (original) +++ pypy/branch/jit-free/pypy/translator/c/test/test_standalone.py Sat Nov 20 15:09:53 2010 @@ -272,7 +272,7 @@ x = "got:" debug_start ("mycat") if have_debug_prints(): x += "b" - debug_print ("foo", 2, "bar", 3) + debug_print ("foo", r_longlong(2), "bar", 3) debug_start ("cat2") if have_debug_prints(): x += "c" debug_print ("baz") From arigo at codespeak.net Sat Nov 20 15:12:46 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 20 Nov 2010 15:12:46 +0100 (CET) Subject: [pypy-svn] r79299 - pypy/branch/jit-free/pypy/rpython/memory/gc Message-ID: <20101120141246.79C2A282BDC@codespeak.net> Author: arigo Date: Sat Nov 20 15:12:44 2010 New Revision: 79299 Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/base.py pypy/branch/jit-free/pypy/rpython/memory/gc/inspector.py pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py pypy/branch/jit-free/pypy/rpython/memory/gc/semispace.py Log: If dump_rpy_heap(), for the common GCs, don't use an AddressDict containing all objects visited so far, but use a flag in the object header itself. Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/base.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/base.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/base.py Sat Nov 20 15:12:44 2010 @@ -19,6 +19,7 @@ malloc_zero_filled = False prebuilt_gc_objects_are_static_roots = True object_minimal_size = 0 + gcflag_extra = 0 # or a real GC flag that is always 0 when not collecting def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, translated_to_c=True): Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/inspector.py Sat Nov 20 15:12:44 2010 @@ -107,15 +107,20 @@ def __init__(self, gc, fd): self.gc = gc + self.gcflag = gc.gcflag_extra self.fd = rffi.cast(rffi.INT, fd) self.writebuffer = lltype.malloc(rffi.LONGP.TO, self.BUFSIZE, flavor='raw') self.buf_count = 0 - self.seen = AddressDict() + if self.gcflag == 0: + self.seen = AddressDict() self.pending = AddressStack() def delete(self): - self.seen.delete() + if self.gcflag == 0: + self.seen.delete() + else: + self.clear_gcflag_again() self.pending.delete() lltype.free(self.writebuffer, flavor='raw') free_non_gc_object(self) @@ -161,9 +166,15 @@ self.add(obj) def add(self, obj): - if not self.seen.contains(obj): - self.seen.setitem(obj, obj) - self.pending.append(obj) + if self.gcflag == 0: + if not self.seen.contains(obj): + self.seen.setitem(obj, obj) + self.pending.append(obj) + else: + hdr = self.gc.header(obj) + if (hdr.tid & self.gcflag) == 0: + hdr.tid |= self.gcflag + self.pending.append(obj) def add_roots(self): self.gc.enumerate_all_roots(_hd_add_root, self) @@ -177,9 +188,25 @@ while pending.non_empty(): self.writeobj(pending.pop()) + def clear_gcflag_again(self): + self.gc.enumerate_all_roots(_hd_clear_root, self) + pending = self.pending + while pending.non_empty(): + self.clear(pending.pop()) + + def clear(self, obj): + assert self.gcflag != 0 + hdr = self.gc.header(obj) + if (hdr.tid & self.gcflag) != 0: + hdr.tid &= ~self.gcflag + self.pending.append(obj) + def _hd_add_root(obj, heap_dumper): heap_dumper.add(obj) +def _hd_clear_root(obj, heap_dumper): + heap_dumper.clear(obj) + def dump_rpy_heap(gc, fd): heapdumper = HeapDumper(gc, fd) heapdumper.add_roots() Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py Sat Nov 20 15:12:44 2010 @@ -63,6 +63,7 @@ needs_write_barrier = True prebuilt_gc_objects_are_static_roots = False malloc_zero_filled = True # xxx experiment with False + gcflag_extra = GCFLAG_FINALIZATION_ORDERING # All objects start with a HDR, i.e. with a field 'tid' which contains # a word. This word is divided in two halves: the lower half contains Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/semispace.py Sat Nov 20 15:12:44 2010 @@ -42,6 +42,7 @@ inline_simple_malloc_varsize = True malloc_zero_filled = True first_unused_gcflag = first_gcflag << 5 + gcflag_extra = GCFLAG_FINALIZATION_ORDERING HDR = lltype.Struct('header', ('tid', lltype.Signed)) # XXX or rffi.INT? typeid_is_in_field = 'tid' From arigo at codespeak.net Sat Nov 20 15:14:01 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 20 Nov 2010 15:14:01 +0100 (CET) Subject: [pypy-svn] r79300 - in pypy/branch/jit-free/pypy: interpreter module/_pickle_support Message-ID: <20101120141401.04717282BEA@codespeak.net> Author: arigo Date: Sat Nov 20 15:14:00 2010 New Revision: 79300 Modified: pypy/branch/jit-free/pypy/interpreter/baseobjspace.py pypy/branch/jit-free/pypy/interpreter/generator.py pypy/branch/jit-free/pypy/module/_pickle_support/maker.py Log: Try to make generators more friendly towards the GC. In the common case where the generator is run to completion, set 'self.frame' to None (as in CPython actually). Should lower the risks of seeing chains of objects with a __del__. Modified: pypy/branch/jit-free/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/jit-free/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/jit-free/pypy/interpreter/baseobjspace.py Sat Nov 20 15:14:00 2010 @@ -147,7 +147,7 @@ __already_enqueued_for_destruction = False - def _enqueue_for_destruction(self, space): + def _enqueue_for_destruction(self, space, call_user_del=True): """Put the object in the destructor queue of the space. At a later, safe point in time, UserDelAction will use space.userdel() to call the object's app-level __del__ method. @@ -160,7 +160,8 @@ return self.__already_enqueued_for_destruction = True self.clear_all_weakrefs() - space.user_del_action.register_dying_object(self) + if call_user_del: + space.user_del_action.register_dying_object(self) def _call_builtin_destructor(self): pass # method overridden in typedef.py Modified: pypy/branch/jit-free/pypy/interpreter/generator.py ============================================================================== --- pypy/branch/jit-free/pypy/interpreter/generator.py (original) +++ pypy/branch/jit-free/pypy/interpreter/generator.py Sat Nov 20 15:14:00 2010 @@ -10,7 +10,7 @@ def __init__(self, frame): self.space = frame.space - self.frame = frame + self.frame = frame # turned into None when frame_finished_execution self.running = False def descr__reduce__(self, space): @@ -19,9 +19,13 @@ mod = space.interp_w(MixedModule, w_mod) new_inst = mod.get('generator_new') w = space.wrap + if self.frame: + w_frame = w(self.frame) + else: + w_frame = space.w_None tup = [ - w(self.frame), + w_frame, w(self.running), ] @@ -41,7 +45,8 @@ if self.running: raise OperationError(space.w_ValueError, space.wrap('generator already executing')) - if self.frame.frame_finished_execution: + frame = self.frame + if frame is None: # xxx a bit ad-hoc, but we don't want to go inside # execute_generator_frame() if the frame is actually finished if operr is None: @@ -49,7 +54,7 @@ raise operr # XXX it's not clear that last_instr should be promoted at all # but as long as it is necessary for call_assembler, let's do it early - last_instr = jit.hint(self.frame.last_instr, promote=True) + last_instr = jit.hint(frame.last_instr, promote=True) if last_instr == -1: if w_arg and not space.is_w(w_arg, space.w_None): msg = "can't send non-None value to a just-started generator" @@ -60,18 +65,19 @@ self.running = True try: try: - w_result = self.frame.execute_generator_frame(w_arg, operr) + w_result = frame.execute_generator_frame(w_arg, operr) except OperationError: # errors finish a frame - self.frame.frame_finished_execution = True + self.frame = None raise # if the frame is now marked as finished, it was RETURNed from - if self.frame.frame_finished_execution: + if frame.frame_finished_execution: + self.frame = None raise OperationError(space.w_StopIteration, space.w_None) else: return w_result # YIELDed finally: - self.frame.f_backref = jit.vref_None + frame.f_backref = jit.vref_None self.running = False def descr_throw(self, w_type, w_val=None, w_tb=None): @@ -115,7 +121,7 @@ raise OperationError(space.w_RuntimeError, space.wrap(msg)) def descr_gi_frame(space, self): - if not self.frame.frame_finished_execution: + if self.frame is not None and not self.frame.frame_finished_execution: return self.frame else: return space.w_None @@ -125,15 +131,17 @@ applevel __del__, which is called at a safe point after the interp-level __del__ enqueued the object for destruction """ - # Only bother raising an exception if the frame is still not - # finished and finally or except blocks are present. - if not self.frame.frame_finished_execution: + self.descr_close() + + def __del__(self): + # Only bother enqueuing self to raise an exception if the frame is + # still not finished and finally or except blocks are present. + must_call_close = False + if self.frame is not None: block = self.frame.lastblock while block is not None: if not isinstance(block, LoopBlock): - self.descr_close() - return + must_call_close = True + break block = block.previous - - def __del__(self): - self._enqueue_for_destruction(self.space) + self._enqueue_for_destruction(self.space, must_call_close) Modified: pypy/branch/jit-free/pypy/module/_pickle_support/maker.py ============================================================================== --- pypy/branch/jit-free/pypy/module/_pickle_support/maker.py (original) +++ pypy/branch/jit-free/pypy/module/_pickle_support/maker.py Sat Nov 20 15:14:00 2010 @@ -67,11 +67,12 @@ return space.wrap(tb) traceback_new.unwrap_spec = [ObjSpace] -def generator_new(space, frame, running): +def generator_new(space, w_frame, running): + frame = space.interp_w(PyFrame, w_frame, can_be_None=True) new_generator = GeneratorIterator(frame) new_generator.running = running return space.wrap(new_generator) -generator_new.unwrap_spec = [ObjSpace, PyFrame, int] +generator_new.unwrap_spec = [ObjSpace, W_Root, int] def xrangeiter_new(space, current, remaining, step): from pypy.module.__builtin__.functional import W_XRangeIterator From arigo at codespeak.net Sat Nov 20 15:18:37 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 20 Nov 2010 15:18:37 +0100 (CET) Subject: [pypy-svn] r79301 - in pypy/branch/jit-free/pypy/translator/c: src test Message-ID: <20101120141837.B619D282BEB@codespeak.net> Author: arigo Date: Sat Nov 20 15:18:36 2010 New Revision: 79301 Modified: pypy/branch/jit-free/pypy/translator/c/src/debug_print.h pypy/branch/jit-free/pypy/translator/c/test/test_standalone.py Log: Merge r79297 from trunk. Modified: pypy/branch/jit-free/pypy/translator/c/src/debug_print.h ============================================================================== --- pypy/branch/jit-free/pypy/translator/c/src/debug_print.h (original) +++ pypy/branch/jit-free/pypy/translator/c/src/debug_print.h Sat Nov 20 15:18:36 2010 @@ -10,6 +10,7 @@ but not any nested debug_print :fname full logging prefix:fname conditional logging + prefix1,prefix2:fname conditional logging with multiple selections Conditional logging means that it only includes the debug_start/debug_stop sections whose name match 'prefix'. Other sections are ignored, including @@ -70,6 +71,8 @@ static void pypy_debug_open(void) { char *filename = getenv("PYPYLOG"); + if (filename) + unsetenv("PYPYLOG"); /* don't pass it to subprocesses */ if (filename && filename[0]) { char *colon = strchr(filename, ':'); @@ -139,12 +142,22 @@ #endif -static bool_t startswith(const char *str, const char *substr) +static bool_t startswithoneof(const char *str, const char *substr) { - while (*substr) - if (*str++ != *substr++) - return 0; - return 1; + const char *p = str; + for (; *substr; substr++) + { + if (*substr != ',') + { + if (p && *p++ != *substr) + p = NULL; /* mismatch */ + } + else if (p != NULL) + return 1; /* match */ + else + p = str; /* mismatched, retry with the next */ + } + return p != NULL; } #if defined(_MSC_VER) || defined(__MINGW32__) @@ -175,7 +188,7 @@ if (!debug_profile) { /* non-profiling version */ - if (!debug_prefix || !startswith(category, debug_prefix)) + if (!debug_prefix || !startswithoneof(category, debug_prefix)) { /* wrong section name, or no PYPYLOG at all, skip it */ return; Modified: pypy/branch/jit-free/pypy/translator/c/test/test_standalone.py ============================================================================== --- pypy/branch/jit-free/pypy/translator/c/test/test_standalone.py (original) +++ pypy/branch/jit-free/pypy/translator/c/test/test_standalone.py Sat Nov 20 15:18:36 2010 @@ -368,12 +368,27 @@ assert not err assert path.check(file=1) data = path.read() - assert 'toplevel' in path.read() - assert 'mycat' not in path.read() - assert 'foo 2 bar 3' not in path.read() + assert 'toplevel' in data + assert 'mycat' not in data + assert 'foo 2 bar 3' not in data assert 'cat2' in data assert 'baz' in data assert 'bok' not in data + # check with PYPYLOG=myc,cat2:somefilename (includes mycat and cat2) + path = udir.join('test_debug_xxx_myc_cat2.log') + out, err = cbuilder.cmdexec("", err=True, + env={'PYPYLOG': 'myc,cat2:%s' % path}) + assert out.strip() == 'got:bcda.' + assert not err + assert path.check(file=1) + data = path.read() + assert 'toplevel' in data + assert '{mycat' in data + assert 'mycat}' in data + assert 'foo 2 bar 3' in data + assert 'cat2' in data + assert 'baz' in data + assert 'bok' in data # # finally, check compiling with logging disabled from pypy.config.pypyoption import get_pypy_config From david at codespeak.net Sat Nov 20 16:02:22 2010 From: david at codespeak.net (david at codespeak.net) Date: Sat, 20 Nov 2010 16:02:22 +0100 (CET) Subject: [pypy-svn] r79303 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101120150222.0160F282B9D@codespeak.net> Author: david Date: Sat Nov 20 16:02:19 2010 New Revision: 79303 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Log: Ensure stack pointer patch location fits in the current piece of memory Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Sat Nov 20 16:02:19 2010 @@ -321,8 +321,10 @@ def _prepare_sp_patch_location(self): """Generate NOPs as placeholder to patch the instruction(s) to update the sp according to the number of spilled variables""" + size = (self.mc.size_of_gen_load_int+WORD) + self.mc.ensure_can_fit(size) l = self.mc.curraddr() - for _ in range((self.mc.size_of_gen_load_int+WORD)//WORD): + for _ in range(size//WORD): self.mc.MOV_rr(r.r0.value, r.r0.value) return l From fijal at codespeak.net Sat Nov 20 16:03:51 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sat, 20 Nov 2010 16:03:51 +0100 (CET) Subject: [pypy-svn] r79304 - pypy/trunk/pypy/tool/release Message-ID: <20101120150351.EECE9282B9D@codespeak.net> Author: fijal Date: Sat Nov 20 16:03:50 2010 New Revision: 79304 Modified: pypy/trunk/pypy/tool/release/force-builds.py Log: Add x86-64 buildbot to stuff we want to run automatically Modified: pypy/trunk/pypy/tool/release/force-builds.py ============================================================================== --- pypy/trunk/pypy/tool/release/force-builds.py (original) +++ pypy/trunk/pypy/tool/release/force-builds.py Sat Nov 20 16:03:50 2010 @@ -25,6 +25,7 @@ 'pypy-c-stackless-app-level-linux-x86-32', 'pypy-c-app-level-win-x86-32', 'pypy-c-jit-linux-x86-32', + 'pypy-c-jit-linux-x86-64', # 'pypy-c-jit-macosx-x86-32', 'pypy-c-jit-win-x86-32', ] From arigo at codespeak.net Sat Nov 20 16:34:07 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sat, 20 Nov 2010 16:34:07 +0100 (CET) Subject: [pypy-svn] r79305 - in pypy/branch/jit-free/pypy: rpython/memory/gc translator/c/test Message-ID: <20101120153407.CBA9C282B9D@codespeak.net> Author: arigo Date: Sat Nov 20 16:34:06 2010 New Revision: 79305 Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/inspector.py pypy/branch/jit-free/pypy/translator/c/test/test_newgc.py Log: Improve the test, and fix. Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/inspector.py Sat Nov 20 16:34:06 2010 @@ -119,8 +119,6 @@ def delete(self): if self.gcflag == 0: self.seen.delete() - else: - self.clear_gcflag_again() self.pending.delete() lltype.free(self.writebuffer, flavor='raw') free_non_gc_object(self) @@ -145,6 +143,8 @@ self.flush() write._always_inline_ = True + # ---------- + def write_marker(self): self.write(0) self.write(0) @@ -188,30 +188,50 @@ while pending.non_empty(): self.writeobj(pending.pop()) - def clear_gcflag_again(self): - self.gc.enumerate_all_roots(_hd_clear_root, self) - pending = self.pending - while pending.non_empty(): - self.clear(pending.pop()) + # ---------- + # A simplified copy of the above, to make sure we walk again all the + # objects to clear the 'gcflag'. + + def unwriteobj(self, obj): + gc = self.gc + gc.trace(obj, self._unwriteref, None) + + def _unwriteref(self, pointer, _): + obj = pointer.address[0] + self.unadd(obj) - def clear(self, obj): + def unadd(self, obj): assert self.gcflag != 0 hdr = self.gc.header(obj) if (hdr.tid & self.gcflag) != 0: hdr.tid &= ~self.gcflag self.pending.append(obj) + def clear_gcflag_again(self): + self.gc.enumerate_all_roots(_hd_unadd_root, self) + pendingroots = self.pending + self.pending = AddressStack() + self.unwalk(pendingroots) + pendingroots.delete() + + def unwalk(self, pending): + while pending.non_empty(): + self.unwriteobj(pending.pop()) + def _hd_add_root(obj, heap_dumper): heap_dumper.add(obj) -def _hd_clear_root(obj, heap_dumper): - heap_dumper.clear(obj) +def _hd_unadd_root(obj, heap_dumper): + heap_dumper.unadd(obj) def dump_rpy_heap(gc, fd): heapdumper = HeapDumper(gc, fd) heapdumper.add_roots() heapdumper.walk(heapdumper.pending) heapdumper.flush() + if heapdumper.gcflag != 0: + heapdumper.clear_gcflag_again() + heapdumper.unwalk(heapdumper.pending) heapdumper.delete() return True Modified: pypy/branch/jit-free/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/branch/jit-free/pypy/translator/c/test/test_newgc.py (original) +++ pypy/branch/jit-free/pypy/translator/c/test/test_newgc.py Sat Nov 20 16:34:06 2010 @@ -1064,14 +1064,16 @@ def test_get_rpy_type_index(self): self.run("get_rpy_type_index") - filename_dump = str(udir.join('test_dump_rpy_heap')) + filename1_dump = str(udir.join('test_dump_rpy_heap.1')) + filename2_dump = str(udir.join('test_dump_rpy_heap.2')) def define_dump_rpy_heap(self): U = lltype.GcForwardReference() U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)), ('x', lltype.Signed))) S = lltype.GcStruct('S', ('u', lltype.Ptr(U))) A = lltype.GcArray(lltype.Ptr(S)) - filename = self.filename_dump + filename1 = self.filename1_dump + filename2 = self.filename2_dump def fn(): s = lltype.malloc(S) @@ -1081,20 +1083,31 @@ a = lltype.malloc(A, 1000) s2 = lltype.malloc(S) # - fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666) - rgc.dump_rpy_heap(fd) + fd1 = os.open(filename1, os.O_WRONLY | os.O_CREAT, 0666) + fd2 = os.open(filename2, os.O_WRONLY | os.O_CREAT, 0666) + rgc.dump_rpy_heap(fd1) + rgc.dump_rpy_heap(fd2) # try twice in a row keepalive_until_here(s2) keepalive_until_here(s) keepalive_until_here(a) - os.close(fd) + os.close(fd1) + os.close(fd2) return 0 return fn def test_dump_rpy_heap(self): self.run("dump_rpy_heap") - assert os.path.exists(self.filename_dump) - assert os.path.getsize(self.filename_dump) > 64 + for fn in [self.filename1_dump, self.filename2_dump]: + assert os.path.exists(fn) + assert os.path.getsize(fn) > 64 + f = open(self.filename1_dump) + data1 = f.read() + f.close() + f = open(self.filename2_dump) + data2 = f.read() + f.close() + assert data1 == data2 filename_dump_typeids_z = str(udir.join('test_typeids_z')) def define_write_typeids_z(self): From david at codespeak.net Sat Nov 20 18:57:11 2010 From: david at codespeak.net (david at codespeak.net) Date: Sat, 20 Nov 2010 18:57:11 +0100 (CET) Subject: [pypy-svn] r79306 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . helper Message-ID: <20101120175711.DD7D4282B9D@codespeak.net> Author: david Date: Sat Nov 20 18:57:08 2010 New Revision: 79306 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Refactor how guards are generated Now a jump a the head checks the condition and skips the guard if the conditions holds Fixes an issue with spilling withing a guard which was overwritten when patching the guard code from a bridge Also free result vars for the case that they are not used. Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Sat Nov 20 18:57:08 2010 @@ -170,8 +170,7 @@ """ descr = op.getdescr() - box = TempBox() - reg = regalloc.force_allocate_reg(box) + reg = r.lr # XXX free this memory # XXX allocate correct amount of memory mem = lltype.malloc(rffi.CArray(lltype.Char), len(args)*6+9, flavor='raw') @@ -214,14 +213,10 @@ n = self.cpu.get_fail_descr_number(descr) self.encode32(mem, j+1, n) + self.mc.ensure_can_fit(self.mc.size_of_gen_load_int) self.mc.gen_load_int(r.lr.value, memaddr, cond=fcond) # use lr to pass an argument - self.mc.B(self._exit_code_addr, fcond, reg) + self.mc.B(self._exit_code_addr) - # This register is used for patching when assembling a bridge - # guards going to be patched are allways conditional - if fcond != c.AL: - descr._arm_guard_reg = reg - regalloc.possibly_free_var(box) return memaddr def align(self): @@ -367,7 +362,6 @@ i = 0 while i < len(operations): op = operations[i] - # XXX consider merging ops with next one if it is an adecuate guard opnum = op.getopnum() if self.can_merge_with_next_guard(op, i, operations): fcond = self.operations_with_guard[opnum](self, op, @@ -401,7 +395,7 @@ self._walk_operations(operations, regalloc) print 'Done building bridges' - self.patch_trace(faildescr, bridge_head) + self.patch_trace(faildescr, bridge_head, regalloc) print 'Done patching trace' if self._debug_asm: self._dump_trace('bridge.asm') @@ -418,11 +412,12 @@ #XXX check ranges for different operations return isinstance(arg, ConstInt) and arg.getint() <= size and lower_bound - def patch_trace(self, faildescr, bridge_addr): - # XXX make sure there is enough space at patch target - fcond = faildescr._arm_guard_cond - b = ARMv7InMemoryBuilder(faildescr._arm_guard_code, faildescr._arm_guard_size) - b.B(bridge_addr, fcond, some_reg=faildescr._arm_guard_reg) + def patch_trace(self, faildescr, bridge_addr, regalloc): + # The first instruction (word) is not overwritten, because it is the + # one that actually checks the condition + b = ARMv7InMemoryBuilder(faildescr._arm_guard_code, + self.guard_size-WORD) + b.B(bridge_addr, some_reg=r.lr) # regalloc support def regalloc_mov(self, prev_loc, loc): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Sat Nov 20 18:57:08 2010 @@ -12,6 +12,8 @@ self.mc.MOV_ri(res.value, 1, true_cond) self.mc.MOV_ri(res.value, 0, false_cond) regalloc.possibly_free_vars_for_op(op) + if op.result: + regalloc.possibly_free_var(op.result) return fcond return f @@ -41,6 +43,8 @@ res = regalloc.force_allocate_reg(op.result, [arg0, arg1]) rr_op(res.value, l0.value, l1.value) regalloc.possibly_free_vars_for_op(op) + if op.result: + regalloc.possibly_free_var(op.result) return fcond return f @@ -56,7 +60,10 @@ regalloc.before_call() getattr(self.mc, opname)(fcond) regalloc.after_call(op.result) + regalloc.possibly_free_vars_for_op(op) + if op.result: + regalloc.possibly_free_var(op.result) return fcond return f @@ -87,5 +94,7 @@ self.mc.MOV_ri(res.value, 1, cond=condition) self.mc.MOV_ri(res.value, 0, cond=inv) regalloc.possibly_free_vars_for_op(op) + if op.result: + regalloc.possibly_free_var(op.result) return fcond return f Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Sat Nov 20 18:57:08 2010 @@ -49,6 +49,8 @@ res = regalloc.force_allocate_reg(op.result, forbidden_vars=[a0, a1]) self.mc.ADD_rr(res.value, l0.value, l1.value, s=1) regalloc.possibly_free_vars_for_op(op) + if op.result: + regalloc.possibly_free_var(op.result) return fcond def emit_op_int_sub(self, op, regalloc, fcond): @@ -79,6 +81,8 @@ self.mc.SUB_rr(res.value, l0.value, l1.value, s=1) regalloc.possibly_free_vars_for_op(op) + if op.result: + regalloc.possibly_free_var(op.result) return fcond def emit_op_int_mul(self, op, regalloc, fcond): @@ -89,6 +93,7 @@ res = regalloc.force_allocate_reg(op.result, [a0, a1]) self.mc.MUL(res.value, reg1.value, reg2.value) regalloc.possibly_free_vars_for_op(op) + regalloc.possibly_free_var(op.result) return fcond #ref: http://blogs.arm.com/software-enablement/detecting-overflow-from-mul/ @@ -102,10 +107,12 @@ self.mc.SMULL(res.value, r.ip.value, reg1.value, reg2.value, cond=fcond) self.mc.CMP_rr(r.ip.value, res.value, shifttype=shift.ASR, imm=31, cond=fcond) regalloc.possibly_free_vars_for_op(op) + if op.result: + regalloc.possibly_free_var(op.result) if guard.getopnum() == rop.GUARD_OVERFLOW: - return self._emit_guard(guard, regalloc, c.EQ) - else: return self._emit_guard(guard, regalloc, c.NE) + else: + return self._emit_guard(guard, regalloc, c.EQ) emit_op_int_floordiv = gen_emit_op_by_helper_call('DIV') emit_op_int_mod = gen_emit_op_by_helper_call('MOD') @@ -150,6 +157,8 @@ self.mc.MVN_rr(res.value, reg.value) regalloc.possibly_free_vars_for_op(op) + if op.result: + regalloc.possibly_free_var(op.result) return fcond #XXX check for a better way of doing this @@ -166,17 +175,19 @@ _mixin_ = True + guard_size = ARMv7Builder.size_of_gen_load_int + 2*WORD def _emit_guard(self, op, regalloc, fcond): descr = op.getdescr() assert isinstance(descr, BasicFailDescr) if hasattr(op, 'getfailargs'): print 'Failargs: ', op.getfailargs() + self.mc.ensure_can_fit(self.guard_size) + self.mc.ADD_ri(r.pc.value, r.pc.value, self.guard_size, cond=fcond) descr._arm_guard_code = self.mc.curraddr() - memaddr = self._gen_path_to_exit_path(op, op.getfailargs(), regalloc, fcond) + memaddr = self._gen_path_to_exit_path(op, op.getfailargs(), regalloc) descr._failure_recovery_code = memaddr - descr._arm_guard_cond = fcond - descr._arm_guard_size = self.mc.curraddr() - descr._arm_guard_code regalloc.possibly_free_vars_for_op(op) + self.mc.NOP() return c.AL def emit_op_guard_true(self, op, regalloc, fcond): @@ -184,14 +195,14 @@ l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) self.mc.CMP_ri(l0.value, 0) regalloc.possibly_free_var(l0) - return self._emit_guard(op, regalloc, c.EQ) + return self._emit_guard(op, regalloc, c.NE) def emit_op_guard_false(self, op, regalloc, fcond): a0 = op.getarg(0) l0 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) self.mc.CMP_ri(l0.value, 0) regalloc.possibly_free_var(l0) - return self._emit_guard(op, regalloc, c.NE) + return self._emit_guard(op, regalloc, c.EQ) def emit_op_guard_value(self, op, regalloc, fcond): a0 = op.getarg(0) @@ -204,16 +215,16 @@ else: self.mc.CMP_rr(l0.value, l1.value) regalloc.possibly_free_vars_for_op(op) - return self._emit_guard(op, regalloc, c.NE) + return self._emit_guard(op, regalloc, c.EQ) emit_op_guard_nonnull = emit_op_guard_true emit_op_guard_isnull = emit_op_guard_false def emit_op_guard_no_overflow(self, op, regalloc, fcond): - return self._emit_guard(op, regalloc, c.VS) + return self._emit_guard(op, regalloc, c.VC) def emit_op_guard_overflow(self, op, regalloc, fcond): - return self._emit_guard(op, regalloc, c.VC) + return self._emit_guard(op, regalloc, c.VS) class OpAssembler(object): @@ -301,6 +312,8 @@ else: self.mc.MOV_rr(resloc.value, argloc.value) regalloc.possibly_free_vars_for_op(op) + if op.result: + regalloc.possibly_free_var(op.result) return fcond class FieldOpAssembler(object): @@ -373,6 +386,8 @@ self.mc.LDR_ri(res.value, base_loc.value, ofs) regalloc.possibly_free_var(op.getarg(0)) + if op.result: + regalloc.possibly_free_var(op.result) def emit_op_setarrayitem_gc(self, op, regalloc, fcond): a0 = op.getarg(0) @@ -443,6 +458,8 @@ l0 = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) regalloc.possibly_free_vars_for_op(op) res = regalloc.force_allocate_reg(op.result) + if op.result: + regalloc.possibly_free_var(op.result) basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.STR, self.cpu.translate_support_code) l1 = regalloc.make_sure_var_in_reg(ConstInt(ofs_length)) @@ -460,6 +477,8 @@ res = regalloc.force_allocate_reg(op.result) regalloc.possibly_free_vars_for_op(op) regalloc.possibly_free_var(t) + if op.result: + regalloc.possibly_free_var(op.result) basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.STR, self.cpu.translate_support_code) @@ -480,6 +499,8 @@ temp = regalloc.force_allocate_reg(t) regalloc.possibly_free_vars_for_op(op) regalloc.possibly_free_var(t) + if op.result: + regalloc.possibly_free_var(op.result) basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.STR, self.cpu.translate_support_code) @@ -499,6 +520,8 @@ l0 = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) regalloc.possibly_free_vars_for_op(op) res = regalloc.force_allocate_reg(op.result) + if op.result: + regalloc.possibly_free_var(op.result) basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.UNICODE, self.cpu.translate_support_code) l1 = regalloc.make_sure_var_in_reg(ConstInt(ofs_length)) @@ -516,6 +539,8 @@ res = regalloc.force_allocate_reg(op.result) regalloc.possibly_free_vars_for_op(op) regalloc.possibly_free_var(t) + if op.result: + regalloc.possibly_free_var(op.result) basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.UNICODE, self.cpu.translate_support_code) @@ -539,6 +564,8 @@ temp = regalloc.force_allocate_reg(t) regalloc.possibly_free_vars_for_op(op) regalloc.possibly_free_var(t) + if op.result: + regalloc.possibly_free_var(op.result) basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.UNICODE, self.cpu.translate_support_code) @@ -661,8 +688,10 @@ self.mc.CMP_ri(l0.value, 0) regalloc.possibly_free_var(t) regalloc.possibly_free_vars_for_op(op) + if op.result: + regalloc.possibly_free_var(op.result) - self._emit_guard(guard_op, regalloc, c.LT) + self._emit_guard(guard_op, regalloc, c.GE) return fcond def emit_guard_call_may_force(self, op, guard_op, regalloc, fcond): @@ -679,7 +708,7 @@ self.mc.CMP_ri(l0.value, 0) regalloc.possibly_free_var(t) - self._emit_guard(guard_op, regalloc, c.LT) + self._emit_guard(guard_op, regalloc, c.GE) return fcond def _write_fail_index(self, fail_index, regalloc): From hakanardo at codespeak.net Sun Nov 21 11:16:14 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Sun, 21 Nov 2010 11:16:14 +0100 (CET) Subject: [pypy-svn] r79307 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test Message-ID: <20101121101614.E7064282B9D@codespeak.net> Author: hakanardo Date: Sun Nov 21 11:16:12 2010 New Revision: 79307 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py Log: Duplicated changes from the merge in test_optimizeopt Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizebasic.py Sun Nov 21 11:16:12 2010 @@ -39,7 +39,7 @@ b1 = BoxInt() opt = optimizeopt.Optimizer(FakeMetaInterpStaticData(LLtypeMixin.cpu), None) - fdescr = ResumeGuardDescr(None, None) + fdescr = ResumeGuardDescr(None) op = ResOperation(rop.GUARD_TRUE, ['dummy'], None, descr=fdescr) # setup rd data fi0 = resume.FrameInfo(None, "code0", 11) @@ -49,12 +49,12 @@ # opt.store_final_boxes_in_guard(op) if op.getfailargs() == [b0, b1]: - assert fdescr.rd_numb.nums == [tag(1, TAGBOX)] - assert fdescr.rd_numb.prev.nums == [tag(0, TAGBOX)] + assert list(fdescr.rd_numb.nums) == [tag(1, TAGBOX)] + assert list(fdescr.rd_numb.prev.nums) == [tag(0, TAGBOX)] else: assert op.getfailargs() == [b1, b0] - assert fdescr.rd_numb.nums == [tag(0, TAGBOX)] - assert fdescr.rd_numb.prev.nums == [tag(1, TAGBOX)] + assert list(fdescr.rd_numb.nums) == [tag(0, TAGBOX)] + assert list(fdescr.rd_numb.prev.nums) == [tag(1, TAGBOX)] assert fdescr.rd_virtuals is None assert fdescr.rd_consts == [] @@ -252,6 +252,8 @@ metainterp_sd = FakeMetaInterpStaticData(self.cpu) if hasattr(self, 'vrefinfo'): metainterp_sd.virtualref_info = self.vrefinfo + if hasattr(self, 'callinfocollection'): + metainterp_sd.callinfocollection = self.callinfocollection # # XXX list the exact optimizations that are needed for each test from pypy.jit.metainterp.optimizeopt import (OptIntBounds, @@ -1404,7 +1406,7 @@ ops = """ [p1] i1 = getfield_gc(p1, descr=valuedescr) - debug_merge_point(15) + debug_merge_point(15, 0) i2 = getfield_gc(p1, descr=valuedescr) escape(i1) escape(i2) @@ -1413,7 +1415,7 @@ expected = """ [p1] i1 = getfield_gc(p1, descr=valuedescr) - debug_merge_point(15) + debug_merge_point(15, 0) escape(i1) escape(i1) jump(p1) @@ -4203,22 +4205,20 @@ # ---------- def optimize_strunicode_loop_extradescrs(self, ops, optops): from pypy.jit.metainterp.optimizeopt import string - def my_callinfo_for_oopspec(oopspecindex): - calldescrtype = type(LLtypeMixin.strequaldescr) - for value in LLtypeMixin.__dict__.values(): - if isinstance(value, calldescrtype): - if (value.get_extra_info() and - value.get_extra_info().oopspecindex == oopspecindex): - # returns 0 for 'func' in this test - return value, 0 - raise AssertionError("not found: oopspecindex=%d" % oopspecindex) + class FakeCallInfoCollection: + def callinfo_for_oopspec(self, oopspecindex): + calldescrtype = type(LLtypeMixin.strequaldescr) + for value in LLtypeMixin.__dict__.values(): + if isinstance(value, calldescrtype): + extra = value.get_extra_info() + if extra and extra.oopspecindex == oopspecindex: + # returns 0 for 'func' in this test + return value, 0 + raise AssertionError("not found: oopspecindex=%d" % + oopspecindex) # - saved = string.callinfo_for_oopspec - try: - string.callinfo_for_oopspec = my_callinfo_for_oopspec - self.optimize_strunicode_loop(ops, optops) - finally: - string.callinfo_for_oopspec = saved + self.callinfocollection = FakeCallInfoCollection() + self.optimize_strunicode_loop(ops, optops) def test_str_equal_noop1(self): ops = """ From fijal at codespeak.net Sun Nov 21 15:27:08 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 21 Nov 2010 15:27:08 +0100 (CET) Subject: [pypy-svn] r79308 - pypy/branch/out-of-line-guards Message-ID: <20101121142708.B3B8F282B9D@codespeak.net> Author: fijal Date: Sun Nov 21 15:27:06 2010 New Revision: 79308 Removed: pypy/branch/out-of-line-guards/ Log: will recreate branch, branched at wrong point From fijal at codespeak.net Sun Nov 21 15:28:34 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Sun, 21 Nov 2010 15:28:34 +0100 (CET) Subject: [pypy-svn] r79309 - pypy/branch/ouf-of-line-guards Message-ID: <20101121142834.AF9A3282B9D@codespeak.net> Author: fijal Date: Sun Nov 21 15:28:33 2010 New Revision: 79309 Added: pypy/branch/ouf-of-line-guards/ (props changed) - copied from r79308, pypy/trunk/ Log: (Re)Create a branch for out-of-line guards From commits-noreply at bitbucket.org Sun Nov 21 15:52:02 2010 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Sun, 21 Nov 2010 08:52:02 -0600 (CST) Subject: [pypy-svn] jitviewer commit ea21641d1ba5: Rename baseviewer.py to jitviewer.py Message-ID: <20101121145202.090511E129C@bitbucket02.managed.contegix.com> # HG changeset patch -- Bitbucket.org # Project jitviewer # URL http://bitbucket.org/pypy/jitviewer/overview # User Maciej Fijalkowski # Date 1290351107 -7200 # Node ID ea21641d1ba518dd1c17a6635341430d2a91608c # Parent 8ddd79f6882a00b1c0f0ea44283eb7d89e5058a6 Rename baseviewer.py to jitviewer.py --- /dev/null +++ b/jitviewer.py @@ -0,0 +1,127 @@ +#!/usr/bin/env pypy-c +""" A web-based browser of your log files. Run by + +baseviewer.py + +and point your browser to http://localhost:5000 + +Demo logfile available in this directory as 'log'. +""" + +import sys +import os +import cgi +import flask +import inspect +from pypy.tool.logparser import parse_log_file, extract_category +from loops import (parse, slice_debug_merge_points, adjust_bridges, + parse_log_counts) +from storage import LoopStorage +from display import CodeRepr, CodeReprNoFile + +from pygments import highlight +from pygments.lexers import PythonLexer +from pygments.formatters import HtmlFormatter + +CUTOFF = 30 + +class Server(object): + def __init__(self, storage): + self.storage = storage + + def index(self): + all = flask.request.args.get('all', None) + loops = [] + for index, loop in enumerate(self.storage.loops): + if 'entry bridge' in loop.comment: + is_entry = True + else: + is_entry = False + func = slice_debug_merge_points(loop.operations, self.storage, + limit=1) + func.count = loop.count + loops.append((is_entry, index, func)) + loops.sort(lambda a, b: cmp(b[2].count, a[2].count)) + if len(loops) > CUTOFF: + extra_data = "Show all (%d) loops" % len(loops) + else: + extra_data = "" + if not all: + loops = loops[:CUTOFF] + return flask.render_template('index.html', loops=loops, + extra_data=extra_data) + + def loop(self): + no = int(flask.request.args.get('no', '0')) + orig_loop = self.storage.loops[no] + ops = adjust_bridges(orig_loop, flask.request.args) + loop = slice_debug_merge_points(ops, self.storage) + path = flask.request.args.get('path', '').split(',') + if path: + up = '"' + ','.join(path[:-1]) + '"' + else: + up = '""' + callstack = [] + path_so_far = [] + for e in path: + if e: + callstack.append((','.join(path_so_far), + '%s in %s at %d' % (loop.name, + loop.filename, + loop.startlineno))) + loop = loop.chunks[int(e)] + path_so_far.append(e) + callstack.append((','.join(path_so_far), '%s in %s at %d' % (loop.name, + loop.filename, loop.startlineno))) + + startline, endline = loop.linerange + if loop.filename is not None: + code = self.storage.load_code(loop.filename)[loop.startlineno] + source = CodeRepr(inspect.getsource(code), code, loop) + else: + source = CodeReprNoFile(loop) + d = {'html': flask.render_template('loop.html', + source=source, + current_loop=no, + upper_path=up, + show_upper_path=bool(path)), + 'scrollto': startline, + 'callstack': callstack} + return flask.jsonify(d) + +def start_browser(url): + import time + import webbrowser + import threading + def run(): + time.sleep(0.5) # give the server some time to start + webbrowser.open(url) + th = threading.Thread(target=run) + th.start() + return th + +def main(): + + if not '__pypy__' in sys.builtin_module_names: + print "Please run it using pypy-c" + sys.exit(1) + if len(sys.argv) != 2: + print __doc__ + sys.exit(1) + log = parse_log_file(sys.argv[1]) + extra_path = os.path.dirname(sys.argv[1]) + storage = LoopStorage(extra_path) + loops = [parse(l) for l in extract_category(log, "jit-log-opt-")] + parse_log_counts(open(sys.argv[1] + '.count').readlines(), loops) + storage.reconnect_loops(loops) + app = flask.Flask(__name__) + server = Server(storage) + app.debug = True + app.route('/')(server.index) + app.route('/loop')(server.loop) + #th = start_browser('http://localhost:5000/') + app.run(use_reloader=False, host='0.0.0.0') + #th.join() + +if __name__ == '__main__': + main() --- a/baseviewer.py +++ /dev/null @@ -1,127 +0,0 @@ -#!/usr/bin/env pypy-c -""" A web-based browser of your log files. Run by - -baseviewer.py - -and point your browser to http://localhost:5000 - -Demo logfile available in this directory as 'log'. -""" - -import sys -import os -import cgi -import flask -import inspect -from pypy.tool.logparser import parse_log_file, extract_category -from loops import (parse, slice_debug_merge_points, adjust_bridges, - parse_log_counts) -from storage import LoopStorage -from display import CodeRepr, CodeReprNoFile - -from pygments import highlight -from pygments.lexers import PythonLexer -from pygments.formatters import HtmlFormatter - -CUTOFF = 30 - -class Server(object): - def __init__(self, storage): - self.storage = storage - - def index(self): - all = flask.request.args.get('all', None) - loops = [] - for index, loop in enumerate(self.storage.loops): - if 'entry bridge' in loop.comment: - is_entry = True - else: - is_entry = False - func = slice_debug_merge_points(loop.operations, self.storage, - limit=1) - func.count = loop.count - loops.append((is_entry, index, func)) - loops.sort(lambda a, b: cmp(b[2].count, a[2].count)) - if len(loops) > CUTOFF: - extra_data = "Show all (%d) loops" % len(loops) - else: - extra_data = "" - if not all: - loops = loops[:CUTOFF] - return flask.render_template('index.html', loops=loops, - extra_data=extra_data) - - def loop(self): - no = int(flask.request.args.get('no', '0')) - orig_loop = self.storage.loops[no] - ops = adjust_bridges(orig_loop, flask.request.args) - loop = slice_debug_merge_points(ops, self.storage) - path = flask.request.args.get('path', '').split(',') - if path: - up = '"' + ','.join(path[:-1]) + '"' - else: - up = '""' - callstack = [] - path_so_far = [] - for e in path: - if e: - callstack.append((','.join(path_so_far), - '%s in %s at %d' % (loop.name, - loop.filename, - loop.startlineno))) - loop = loop.chunks[int(e)] - path_so_far.append(e) - callstack.append((','.join(path_so_far), '%s in %s at %d' % (loop.name, - loop.filename, loop.startlineno))) - - startline, endline = loop.linerange - if loop.filename is not None: - code = self.storage.load_code(loop.filename)[loop.startlineno] - source = CodeRepr(inspect.getsource(code), code, loop) - else: - source = CodeReprNoFile(loop) - d = {'html': flask.render_template('loop.html', - source=source, - current_loop=no, - upper_path=up, - show_upper_path=bool(path)), - 'scrollto': startline, - 'callstack': callstack} - return flask.jsonify(d) - -def start_browser(url): - import time - import webbrowser - import threading - def run(): - time.sleep(0.5) # give the server some time to start - webbrowser.open(url) - th = threading.Thread(target=run) - th.start() - return th - -def main(): - - if not '__pypy__' in sys.builtin_module_names: - print "Please run it using pypy-c" - sys.exit(1) - if len(sys.argv) != 2: - print __doc__ - sys.exit(1) - log = parse_log_file(sys.argv[1]) - extra_path = os.path.dirname(sys.argv[1]) - storage = LoopStorage(extra_path) - loops = [parse(l) for l in extract_category(log, "jit-log-opt-")] - parse_log_counts(open(sys.argv[1] + '.count').readlines(), loops) - storage.reconnect_loops(loops) - app = flask.Flask(__name__) - server = Server(storage) - app.debug = True - app.route('/')(server.index) - app.route('/loop')(server.loop) - #th = start_browser('http://localhost:5000/') - app.run(use_reloader=False, host='0.0.0.0') - #th.join() - -if __name__ == '__main__': - main() From arigo at codespeak.net Sun Nov 21 16:44:59 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 21 Nov 2010 16:44:59 +0100 (CET) Subject: [pypy-svn] r79310 - pypy/branch/jit-free/pypy/jit/metainterp Message-ID: <20101121154459.DABBA282B9D@codespeak.net> Author: arigo Date: Sun Nov 21 16:44:58 2010 New Revision: 79310 Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Log: An essential fix for a rare case. Unfortunately, it's obviously a bug fix, but writing a test is a total mess :-( Modified: pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/warmstate.py Sun Nov 21 16:44:58 2010 @@ -258,9 +258,9 @@ def attach_unoptimized_bridge_from_interp(self, greenkey, entry_loop_token): cell = self.jit_cell_at_key(greenkey) - cell.counter = -1 old_token = cell.get_entry_loop_token() cell.set_entry_loop_token(entry_loop_token) + cell.counter = -1 # valid entry bridge attached if old_token is not None: self.cpu.redirect_call_assembler(old_token, entry_loop_token) # entry_loop_token is also kept alive by any loop that used @@ -574,6 +574,8 @@ entry_loop_token = cell.get_entry_loop_token() if entry_loop_token is None: from pypy.jit.metainterp.compile import compile_tmp_callback + if cell.counter == -1: # used to be a valid entry bridge, + cell.counter = 0 # but was freed in the meantime. memmgr = warmrunnerdesc.memory_manager entry_loop_token = compile_tmp_callback(cpu, jd, greenkey, redboxes, memmgr) From arigo at codespeak.net Sun Nov 21 16:45:31 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 21 Nov 2010 16:45:31 +0100 (CET) Subject: [pypy-svn] r79311 - in pypy/trunk/pypy/jit: codewriter metainterp metainterp/test Message-ID: <20101121154531.BF2FC282B9D@codespeak.net> Author: arigo Date: Sun Nov 21 16:45:29 2010 New Revision: 79311 Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py pypy/trunk/pypy/jit/metainterp/pyjitpl.py pypy/trunk/pypy/jit/metainterp/test/test_recursive.py Log: Minor bug and fix. Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/jtransform.py (original) +++ pypy/trunk/pypy/jit/codewriter/jtransform.py Sun Nov 21 16:45:29 2010 @@ -843,9 +843,16 @@ "general mix-up of jitdrivers?") ops = self.promote_greens(op.args[2:], jitdriver) num_green_args = len(jitdriver.greens) + redlists = self.make_three_lists(op.args[2+num_green_args:]) + for redlist in redlists: + for v in redlist: + assert isinstance(v, Variable), ( + "Constant specified red in jit_merge_point()") + assert len(dict.fromkeys(redlist)) == len(list(redlist)), ( + "duplicate red variable on jit_merge_point()") args = ([Constant(self.portal_jd.index, lltype.Signed)] + self.make_three_lists(op.args[2:2+num_green_args]) + - self.make_three_lists(op.args[2+num_green_args:])) + redlists) op1 = SpaceOperation('jit_merge_point', args, None) op2 = SpaceOperation('-live-', [], None) # ^^^ we need a -live- for the case of do_recursive_call() Modified: pypy/trunk/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/trunk/pypy/jit/metainterp/pyjitpl.py Sun Nov 21 16:45:29 2010 @@ -94,6 +94,18 @@ else: raise AssertionError(argcode) outvalue[startindex+i] = reg + def _put_back_list_of_boxes(self, outvalue, startindex, position): + code = self.bytecode + length = ord(code[position]) + position += 1 + for i in range(length): + index = ord(code[position+i]) + box = outvalue[startindex+i] + if box.type == history.INT: self.registers_i[index] = box + elif box.type == history.REF: self.registers_r[index] = box + elif box.type == history.FLOAT: self.registers_f[index] = box + else: raise AssertionError(box.type) + def get_current_position_info(self): return self.jitcode.get_live_vars_info(self.pc) @@ -814,8 +826,9 @@ for i in range(num_green_args): assert isinstance(varargs[i], Const) - @arguments("orgpc", "int", "boxes3", "boxes3") - def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, redboxes): + @arguments("orgpc", "int", "boxes3", "jitcode_position", "boxes3") + def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, + jcposition, redboxes): any_operation = len(self.metainterp.history.operations) > 0 jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex] self.verify_green_args(jitdriver_sd, greenboxes) @@ -843,6 +856,10 @@ self.pc = orgpc self.metainterp.reached_loop_header(greenboxes, redboxes) self.pc = saved_pc + # no exception, which means that the jit_merge_point did not + # close the loop. We have to put the possibly-modified list + # 'redboxes' back into the registers where it comes from. + put_back_list_of_boxes3(self, jcposition, redboxes) else: # warning! careful here. We have to return from the current # frame containing the jit_merge_point, and then use @@ -2293,6 +2310,8 @@ else: raise AssertionError("bad argcode") position += 1 + elif argtype == "jitcode_position": + value = position else: raise AssertionError("bad argtype: %r" % (argtype,)) args += (value,) @@ -2337,3 +2356,15 @@ argtypes = unrolling_iterable(unboundmethod.argtypes) handler.func_name = 'handler_' + name return handler + +def put_back_list_of_boxes3(frame, position, newvalue): + code = frame.bytecode + length1 = ord(code[position]) + position2 = position + 1 + length1 + length2 = ord(code[position2]) + position3 = position2 + 1 + length2 + length3 = ord(code[position3]) + assert len(newvalue) == length1 + length2 + length3 + frame._put_back_list_of_boxes(newvalue, 0, position) + frame._put_back_list_of_boxes(newvalue, length1, position2) + frame._put_back_list_of_boxes(newvalue, length1 + length2, position3) Modified: pypy/trunk/pypy/jit/metainterp/test/test_recursive.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_recursive.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_recursive.py Sun Nov 21 16:45:29 2010 @@ -1142,6 +1142,19 @@ res = self.meta_interp(main, [], inline=True, trace_limit=tlimit) assert ''.join(res.chars) == 'ABCDEFGHIabcdefghijJ' * 5 + def test_no_duplicates_bug(self): + driver = JitDriver(greens = ['codeno'], reds = ['i'], + get_printable_location = lambda codeno: str(codeno)) + def portal(codeno, i): + while i > 0: + driver.can_enter_jit(codeno=codeno, i=i) + driver.jit_merge_point(codeno=codeno, i=i) + if codeno > 0: + break + portal(i, i) + i -= 1 + self.meta_interp(portal, [0, 10], inline=True) + class TestLLtype(RecursiveTests, LLJitMixin): pass From commits-noreply at bitbucket.org Sun Nov 21 16:49:03 2010 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Sun, 21 Nov 2010 09:49:03 -0600 (CST) Subject: [pypy-svn] jitviewer commit 53403054c7d9: setuptoolize jitviewer Message-ID: <20101121154903.1E43D1E0FC0@bitbucket02.managed.contegix.com> # HG changeset patch -- Bitbucket.org # Project jitviewer # URL http://bitbucket.org/pypy/jitviewer/overview # User Maciej Fijalkowski # Date 1290354523 -7200 # Node ID 53403054c7d926925f697d6d519e7f42d1776541 # Parent ea21641d1ba518dd1c17a6635341430d2a91608c setuptoolize jitviewer --- a/module_finder.py +++ /dev/null @@ -1,25 +0,0 @@ - -import os, sys, marshal, types, struct, imp - -def _all_codes_from(code): - res = {} - more = [code] - while more: - next = more.pop() - res[next.co_firstlineno] = next - more += [co for co in next.co_consts - if isinstance(co, types.CodeType)] - return res - -def gather_all_code_objs(fname): - """ Gathers all code objects from a give fname and sorts them by - starting lineno - """ - fname = str(fname) - if fname.endswith('.pyc'): - code = compile(open(fname[:-1]).read(), fname, 'exec') - elif fname.endswith('.py'): - code = compile(open(fname).read(), fname, 'exec') - else: - raise Exception("Unknown file extension: %s" % fname) - return _all_codes_from(code) --- /dev/null +++ b/_jitviewer/disassembler.py @@ -0,0 +1,297 @@ +"""Disassembler of Python byte code into mnemonics. + +Comes from standard library, modified for the purpose of having a structured +view on things +""" + +import sys +import types +import inspect + +from opcode import * +from opcode import __all__ as _opcodes_all + +__all__ = ["dis","disassemble","distb","disco"] + _opcodes_all +del _opcodes_all + +class Opcode(object): + """ An abstract base class for all opcode implementations + """ + def __init__(self, pos, lineno, arg=None, argstr=''): + self.pos = pos + self.arg = arg + self.argstr = argstr + self.lineno = lineno + self.line_starts_here = False + + def __repr__(self): + if self.arg is None: + return "<%s at %d>" % (self.__class__.__name__, self.pos) + return "<%s (%s) at %d>" % (self.__class__.__name__, self.arg, self.pos) + +class CodeRepresentation(object): + """ Representation of opcodes + """ + def __init__(self, opcodes, source): + self.opcodes = opcodes + self.map = {} + current_lineno = None + for opcode in opcodes: + self.map[opcode.pos] = opcode + if opcode.lineno != current_lineno: + opcode.line_starts_here = True + current_lineno = opcode.lineno + self.source = source.split("\n") + +def _setup(): + for opcode in opname: + if not opcode.startswith('<'): + class O(Opcode): + pass + opcode = opcode.replace('+', '_') + O.__name__ = opcode + globals()[opcode] = O + +_setup() + +def dis(x=None): + """Disassemble classes, methods, functions, or code. + + With no argument, disassemble the last traceback. + + """ + if x is None: + distb() + return + if type(x) is types.InstanceType: + x = x.__class__ + if hasattr(x, 'im_func'): + x = x.im_func + if hasattr(x, 'func_code'): + x = x.func_code + if hasattr(x, '__dict__'): + xxx + items = x.__dict__.items() + items.sort() + for name, x1 in items: + if type(x1) in (types.MethodType, + types.FunctionType, + types.CodeType, + types.ClassType): + print "Disassembly of %s:" % name + try: + dis(x1) + except TypeError, msg: + print "Sorry:", msg + print + elif hasattr(x, 'co_code'): + return disassemble(x) + elif isinstance(x, str): + return disassemble_string(x) + else: + raise TypeError, \ + "don't know how to disassemble %s objects" % \ + type(x).__name__ + +def distb(tb=None): + """Disassemble a traceback (default: last traceback).""" + if tb is None: + try: + tb = sys.last_traceback + except AttributeError: + raise RuntimeError, "no last traceback to disassemble" + while tb.tb_next: tb = tb.tb_next + disassemble(tb.tb_frame.f_code, tb.tb_lasti) + +def disassemble(co, lasti=-1): + """Disassemble a code object.""" + source = inspect.getsource(co) + code = co.co_code + labels = findlabels(code) + linestarts = dict(findlinestarts(co)) + n = len(code) + i = 0 + extended_arg = 0 + free = None + res = [] + lastline = co.co_firstlineno + while i < n: + c = code[i] + op = ord(c) + if i in linestarts: + lastline = linestarts[i] + + #if i == lasti: + # xxx + # print '-->', + #else: + # xxx + # print ' ', + #if i in labels: + # xxx + # print '>>', + #else: + # xxx + # print ' ', + #xxx + pos = i + i = i + 1 + if op >= HAVE_ARGUMENT: + oparg = ord(code[i]) + ord(code[i+1])*256 + extended_arg + opargstr = str(oparg) + extended_arg = 0 + i = i+2 + if op == EXTENDED_ARG: + extended_arg = oparg*65536L + if op in hasconst: + opargstr = repr(co.co_consts[oparg]) + elif op in hasname: + opargstr = co.co_names[oparg] + elif op in hasjrel: + opargstr = 'to ' + repr(i + oparg) + elif op in haslocal: + opargstr = co.co_varnames[oparg] + elif op in hascompare: + opargstr = cmp_op[oparg] + elif op in hasfree: + if free is None: + free = co.co_cellvars + co.co_freevars + opargstr = free[oparg] + else: + oparg = None + opargstr = '' + opcls = globals()[opname[op].replace('+', '_')] + res.append(opcls(pos, lastline, oparg, opargstr)) + return CodeRepresentation(res, source) + +def disassemble_string(code, lasti=-1, varnames=None, names=None, + constants=None): + labels = findlabels(code) + n = len(code) + i = 0 + while i < n: + c = code[i] + op = ord(c) + if i == lasti: + xxx + print '-->', + else: + xxx + print ' ', + if i in labels: + xxx + print '>>', + else: + xxx + print ' ', + xxxx + print repr(i).rjust(4), + print opname[op].ljust(15), + i = i+1 + if op >= HAVE_ARGUMENT: + oparg = ord(code[i]) + ord(code[i+1])*256 + i = i+2 + xxx + print repr(oparg).rjust(5), + if op in hasconst: + if constants: + xxx + print '(' + repr(constants[oparg]) + ')', + else: + xxx + print '(%d)'%oparg, + elif op in hasname: + if names is not None: + xxx + print '(' + names[oparg] + ')', + else: + xxx + print '(%d)'%oparg, + elif op in hasjrel: + xxx + print '(to ' + repr(i + oparg) + ')', + elif op in haslocal: + if varnames: + xxx + print '(' + varnames[oparg] + ')', + else: + xxx + print '(%d)' % oparg, + elif op in hascompare: + xxx + print '(' + cmp_op[oparg] + ')', + xxx + print + +disco = disassemble # XXX For backwards compatibility + +def findlabels(code): + """Detect all offsets in a byte code which are jump targets. + + Return the list of offsets. + + """ + labels = [] + n = len(code) + i = 0 + while i < n: + c = code[i] + op = ord(c) + i = i+1 + if op >= HAVE_ARGUMENT: + oparg = ord(code[i]) + ord(code[i+1])*256 + i = i+2 + label = -1 + if op in hasjrel: + label = i+oparg + elif op in hasjabs: + label = oparg + if label >= 0: + if label not in labels: + labels.append(label) + return labels + +def findlinestarts(code): + """Find the offsets in a byte code which are start of lines in the source. + + Generate pairs (offset, lineno) as described in Python/compile.c. + + """ + byte_increments = [ord(c) for c in code.co_lnotab[0::2]] + line_increments = [ord(c) for c in code.co_lnotab[1::2]] + + lastlineno = None + lineno = code.co_firstlineno + addr = 0 + for byte_incr, line_incr in zip(byte_increments, line_increments): + if byte_incr: + if lineno != lastlineno: + yield (addr, lineno) + lastlineno = lineno + addr += byte_incr + lineno += line_incr + if lineno != lastlineno: + yield (addr, lineno) + +def _test(): + """Simple test program to disassemble a file.""" + if sys.argv[1:]: + if sys.argv[2:]: + sys.stderr.write("usage: python dis.py [-|file]\n") + sys.exit(2) + fn = sys.argv[1] + if not fn or fn == "-": + fn = None + else: + fn = None + if fn is None: + f = sys.stdin + else: + f = open(fn) + source = f.read() + if fn is not None: + f.close() + else: + fn = "" + code = compile(source, fn, "exec") + dis(code) --- /dev/null +++ b/_jitviewer/source.py @@ -0,0 +1,39 @@ + +import re + +def f(): + i = 0 + while i < 1003: + i += 1 + +f() + +def inner(i): + return i + 1 + +def inlined_call(): + i = 0 + while i < 1003: + i = inner(i) + +inlined_call() + +def bridge(): + s = 0 + i = 0 + while i < 10000: + if i % 2: + s += 1 + else: + s += 2 + i += 1 + return s + +bridge() + +def inlined_str_stuff(): + s = [str(i) for i in range(3000)] + for elem in s: + re.search('3', elem) + +inlined_str_stuff() --- a/test/test_module.py +++ /dev/null @@ -1,21 +0,0 @@ - -import py -from module_finder import gather_all_code_objs -import re, sys - -def setup_module(mod): - if sys.version_info[:2] != (2.6): - py.test.skip("Specific python 2.6 tests") - -def test_gather_code_py(): - fname = re.__file__ - codes = gather_all_code_objs(fname) - assert len(codes) == 21 - assert sorted(codes.keys()) == [102, 134, 139, 144, 153, 164, 169, 181, 188, 192, 197, 206, 229, 251, 266, 271, 277, 285, 293, 294, 308] - -def test_load_code(): - fname = re.__file__ - code = gather_all_code_objs(fname)[144] - assert code.co_name == 'sub' - assert code.co_filename == '/usr/lib/python2.6/re.py' - assert code.co_firstlineno == 144 --- a/test/test_loops.py +++ /dev/null @@ -1,203 +0,0 @@ - -from pypy.jit.metainterp.resoperation import ResOperation, rop -from pypy.jit.metainterp.history import ConstInt, Const -from loops import parse, Bytecode, Function, slice_debug_merge_points,\ - adjust_bridges, parse_log_counts -import py -from storage import LoopStorage - -def test_parse(): - ops = parse(''' - [i7] - i9 = int_lt(i7, 1003) - guard_true(i9, descr=) [] - i13 = getfield_raw(151937600, descr=) - ''').operations - assert len(ops) == 3 - assert ops[0].name == 'int_lt' - assert ops[1].name == 'guard_true' - assert ops[1].descr is not None - assert ops[0].res == 'i9' - assert ops[0].html_repr().plaintext() == 'i9 = i7 < 1003' - assert ops[2].descr is not None - assert len(ops[2].args) == 1 - assert ops[2].html_repr().plaintext() == 'i13 = ((pypysig_long_struct)151937600).value' - -def test_parse_non_code(): - ops = parse(''' - [] - debug_merge_point("SomeRandomStuff", 0) - ''') - res = slice_debug_merge_points(ops.operations, LoopStorage()) - assert len(res.chunks) == 1 - assert res.chunks[0].html_repr() - -def test_split(): - ops = parse(''' - [i0] - debug_merge_point(" #10 ADD", 0) - debug_merge_point(" #11 SUB", 0) - i1 = int_add(i0, 1) - debug_merge_point(" #11 SUB", 0) - i2 = int_add(i1, 1) - ''') - res = slice_debug_merge_points(ops.operations, LoopStorage()) - assert len(res.chunks) == 3 - assert len(res.chunks[0].operations) == 1 - assert len(res.chunks[1].operations) == 2 - assert len(res.chunks[2].operations) == 2 - assert res.chunks[2].bytecode_no == 11 - -def test_inlined_call(): - ops = parse(""" - [] - debug_merge_point(' #28 CALL_FUNCTION', 0) - i18 = getfield_gc(p0, descr=) - debug_merge_point(' #0 LOAD_FAST', 1) - debug_merge_point(' #3 LOAD_CONST', 1) - debug_merge_point(' #7 RETURN_VALUE', 1) - debug_merge_point(' #31 STORE_FAST', 0) - """) - res = slice_debug_merge_points(ops.operations, LoopStorage()) - assert len(res.chunks) == 3 # two chunks + inlined call - assert isinstance(res.chunks[0], Bytecode) - assert isinstance(res.chunks[1], Function) - assert isinstance(res.chunks[2], Bytecode) - assert res.chunks[1].path == "1" - assert len(res.chunks[1].chunks) == 3 - -def test_name(): - ops = parse(''' - [i0] - debug_merge_point(" #10 ADD", 0) - debug_merge_point(" #11 SUB", 0) - i1 = int_add(i0, 1) - debug_merge_point(" #11 SUB", 0) - i2 = int_add(i1, 1) - ''') - res = slice_debug_merge_points(ops.operations, LoopStorage()) - assert res.repr() == res.chunks[0].repr() - assert res.repr() == "stuff, file '/tmp/x.py', line 200" - assert res.startlineno == 200 - assert res.filename == '/tmp/x.py' - assert res.name == 'stuff' - -def test_name_no_first(): - ops = parse(''' - [i0] - i3 = int_add(i0, 1) - debug_merge_point(" #10 ADD", 0) - debug_merge_point(" #11 SUB", 0) - i1 = int_add(i0, 1) - debug_merge_point(" #11 SUB", 0) - i2 = int_add(i1, 1) - ''') - res = slice_debug_merge_points(ops.operations, LoopStorage()) - assert res.repr() == res.chunks[1].repr() - -def test_lineno(): - fname = str(py.path.local(__file__).join('..', 'x.py')) - ops = parse(''' - [i0, i1] - debug_merge_point(" #0 LOAD_FAST", 0) - debug_merge_point(" #3 LOAD_FAST", 0) - debug_merge_point(" #6 BINARY_ADD", 0) - debug_merge_point(" #7 RETURN_VALUE", 0) - ''' % locals()) - res = slice_debug_merge_points(ops.operations, LoopStorage()) - assert res.chunks[1].lineno == 3 - -def test_linerange(): - fname = str(py.path.local(__file__).join('..', 'x.py')) - ops = parse(''' - [i0, i1] - debug_merge_point(" #9 LOAD_FAST", 0) - debug_merge_point(" #12 LOAD_CONST", 0) - debug_merge_point(" #22 LOAD_CONST", 0) - debug_merge_point(" #28 LOAD_CONST", 0) - debug_merge_point(" #6 SETUP_LOOP", 0) - ''' % locals()) - res = slice_debug_merge_points(ops.operations, LoopStorage()) - assert res.linerange == (7, 9) - assert res.lineset == set([7, 8, 9]) - -def test_linerange_notstarts(): - fname = str(py.path.local(__file__).join('..', 'x.py')) - ops = parse(""" - [p6, p1] - debug_merge_point(' #17 FOR_ITER', 0) - guard_class(p6, 144264192, descr=) - p12 = getfield_gc(p6, descr=) - """ % locals()) - res = slice_debug_merge_points(ops.operations, LoopStorage()) - assert res.lineset - -def test_reassign_loops(): - main = parse(''' - [v0] - guard_false(v0, descr=) [] - ''') - main.count = 10 - bridge = parse(''' - # bridge out of Guard 18 with 13 ops - [i0, i1] - int_add(i0, i1) - ''') - bridge.count = 3 - entry_bridge = parse(''' - # Loop 3 : entry bridge - [] - ''') - loops = LoopStorage().reconnect_loops([main, bridge, entry_bridge]) - assert len(loops) == 2 - assert len(loops[0].operations[0].bridge.operations) == 1 - assert loops[0].operations[0].bridge.no == 18 - assert loops[0].operations[0].percentage == 30 - -def test_adjust_bridges(): - main = parse(''' - [v0] - guard_false(v0, descr=) - guard_true(v0, descr=) - ''') - bridge = parse(''' - # bridge out of Guard 13 - [] - int_add(0, 1) - ''') - loops = LoopStorage().reconnect_loops([main, bridge]) - assert adjust_bridges(main, {})[1].name == 'guard_true' - assert adjust_bridges(main, {'loop-13': True})[1].name == 'int_add' - -def test_parsing_strliteral(): - loop = parse(""" - debug_merge_point('StrLiteralSearch at 11/51 [17, 8, 3, 1, 1, 1, 1, 51, 0, 19, 51, 1]', 0) - """) - ops = slice_debug_merge_points(loop.operations, LoopStorage()) - chunk = ops.chunks[0] - assert chunk.bytecode_name == 'StrLiteralSearch' - -LINES = ''' -0:3 -1:3 -2:604 -3:396 -4:102 -5:2000 -6:3147 -7:2445 -8:2005 -9:2000 -10:1420 -11:40 -12:0 -'''.split("\n") - -def test_parse_log_count(): - class Loop(object): - pass - - loops = [Loop() for i in range(13)] - nums = parse_log_counts(LINES, loops) - assert nums[5] == 2000 - assert loops[9].count == 2000 --- a/test/test_disassembler.py +++ /dev/null @@ -1,33 +0,0 @@ - -import disassembler, sys -import py - -def f(a, b): - return a + b - -def g(a, b): - c = a+b - return c - -def test_disassembler(): - res = disassembler.dis(f) - if sys.version_info[:2] != (2, 6): - py.test.skip("2.6 only test") - assert len(res.opcodes) == 4 - assert [x.__class__.__name__ for x in res.opcodes] == [ - 'LOAD_FAST', 'LOAD_FAST', 'BINARY_ADD', 'RETURN_VALUE'] - for i in range(4): - assert res.opcodes[i].lineno == f.func_code.co_firstlineno + 1 - assert res.opcodes[0].argstr == 'a' - -def test_line_starting_opcodes(): - if sys.version_info[:2] != (2, 6): - py.test.skip("2.6 only test") - res = disassembler.dis(g) - assert len(res.opcodes) == 6 - for i, opcode in enumerate(res.opcodes): - if i in (0, 4): - assert opcode.__class__.__name__ == 'LOAD_FAST' - assert opcode.line_starts_here - else: - assert not opcode.line_starts_here --- /dev/null +++ b/_jitviewer/storage.py @@ -0,0 +1,56 @@ + +""" This file represents a storage mechanism that let us invent unique names +for all loops and bridges, so http requests can refer to them by name +""" + +import os +from loops import Function, Bytecode +from module_finder import gather_all_code_objs + +class LoopStorage(object): + def __init__(self, extrapath=None): + self.loops = None + self.functions = {} + self.codes = {} + self.extrapath = extrapath + + def load_code(self, fname): + try: + return self.codes[fname] + except KeyError: + if os.path.isabs(fname): + res = gather_all_code_objs(fname) + else: + if self.extrapath is None: + raise IOError("Cannot find %s" % fname) + res = gather_all_code_objs(os.path.join(self.extrapath, fname)) + self.codes[fname] = res + return res + + def reconnect_loops(self, loops): + """ Re-connect loops in a way that entry bridges are filtered out + and normal bridges are associated with guards. Returning list of + normal loops. + """ + res = [] + guard_dict = {} + for loop_no, loop in enumerate(loops): + for op in loop.operations: + if op.name.startswith('guard_'): + guard_dict[int(op.descr[len(' + +and point your browser to http://localhost:5000 + +Demo logfile available in this directory as 'log'. +""" + +import sys +import os +import cgi +import flask +import inspect +from pypy.tool.logparser import parse_log_file, extract_category +from _jitviewer.loops import (parse, slice_debug_merge_points, adjust_bridges, + parse_log_counts) +from _jitviewer.storage import LoopStorage +from _jitviewer.display import CodeRepr, CodeReprNoFile + +from pygments import highlight +from pygments.lexers import PythonLexer +from pygments.formatters import HtmlFormatter + +CUTOFF = 30 + +class Server(object): + def __init__(self, storage): + self.storage = storage + + def index(self): + all = flask.request.args.get('all', None) + loops = [] + for index, loop in enumerate(self.storage.loops): + if 'entry bridge' in loop.comment: + is_entry = True + else: + is_entry = False + func = slice_debug_merge_points(loop.operations, self.storage, + limit=1) + func.count = loop.count + loops.append((is_entry, index, func)) + loops.sort(lambda a, b: cmp(b[2].count, a[2].count)) + if len(loops) > CUTOFF: + extra_data = "Show all (%d) loops" % len(loops) + else: + extra_data = "" + if not all: + loops = loops[:CUTOFF] + return flask.render_template('index.html', loops=loops, + extra_data=extra_data) + + def loop(self): + no = int(flask.request.args.get('no', '0')) + orig_loop = self.storage.loops[no] + ops = adjust_bridges(orig_loop, flask.request.args) + loop = slice_debug_merge_points(ops, self.storage) + path = flask.request.args.get('path', '').split(',') + if path: + up = '"' + ','.join(path[:-1]) + '"' + else: + up = '""' + callstack = [] + path_so_far = [] + for e in path: + if e: + callstack.append((','.join(path_so_far), + '%s in %s at %d' % (loop.name, + loop.filename, + loop.startlineno))) + loop = loop.chunks[int(e)] + path_so_far.append(e) + callstack.append((','.join(path_so_far), '%s in %s at %d' % (loop.name, + loop.filename, loop.startlineno))) + + startline, endline = loop.linerange + if loop.filename is not None: + code = self.storage.load_code(loop.filename)[loop.startlineno] + source = CodeRepr(inspect.getsource(code), code, loop) + else: + source = CodeReprNoFile(loop) + d = {'html': flask.render_template('loop.html', + source=source, + current_loop=no, + upper_path=up, + show_upper_path=bool(path)), + 'scrollto': startline, + 'callstack': callstack} + return flask.jsonify(d) + +def start_browser(url): + import time + import webbrowser + import threading + def run(): + time.sleep(0.5) # give the server some time to start + webbrowser.open(url) + th = threading.Thread(target=run) + th.start() + return th + +def main(): + + if not '__pypy__' in sys.builtin_module_names: + print "Please run it using pypy-c" + sys.exit(1) + if len(sys.argv) != 2: + print __doc__ + sys.exit(1) + log = parse_log_file(sys.argv[1]) + extra_path = os.path.dirname(sys.argv[1]) + storage = LoopStorage(extra_path) + loops = [parse(l) for l in extract_category(log, "jit-log-opt-")] + parse_log_counts(open(sys.argv[1] + '.count').readlines(), loops) + storage.reconnect_loops(loops) + app = flask.Flask(__name__) + server = Server(storage) + app.debug = True + app.route('/')(server.index) + app.route('/loop')(server.loop) + #th = start_browser('http://localhost:5000/') + app.run(use_reloader=False, host='0.0.0.0') + #th.join() + +if __name__ == '__main__': + main() --- /dev/null +++ b/_jitviewer/test/test_loops.py @@ -0,0 +1,204 @@ + +from pypy.jit.metainterp.resoperation import ResOperation, rop +from pypy.jit.metainterp.history import ConstInt, Const +from _jitviewer.loops import parse, Bytecode, Function,\ + slice_debug_merge_points,\ + adjust_bridges, parse_log_counts +from _jitviewer.storage import LoopStorage +import py + +def test_parse(): + ops = parse(''' + [i7] + i9 = int_lt(i7, 1003) + guard_true(i9, descr=) [] + i13 = getfield_raw(151937600, descr=) + ''').operations + assert len(ops) == 3 + assert ops[0].name == 'int_lt' + assert ops[1].name == 'guard_true' + assert ops[1].descr is not None + assert ops[0].res == 'i9' + assert ops[0].html_repr().plaintext() == 'i9 = i7 < 1003' + assert ops[2].descr is not None + assert len(ops[2].args) == 1 + assert ops[2].html_repr().plaintext() == 'i13 = ((pypysig_long_struct)151937600).value' + +def test_parse_non_code(): + ops = parse(''' + [] + debug_merge_point("SomeRandomStuff", 0) + ''') + res = slice_debug_merge_points(ops.operations, LoopStorage()) + assert len(res.chunks) == 1 + assert res.chunks[0].html_repr() + +def test_split(): + ops = parse(''' + [i0] + debug_merge_point(" #10 ADD", 0) + debug_merge_point(" #11 SUB", 0) + i1 = int_add(i0, 1) + debug_merge_point(" #11 SUB", 0) + i2 = int_add(i1, 1) + ''') + res = slice_debug_merge_points(ops.operations, LoopStorage()) + assert len(res.chunks) == 3 + assert len(res.chunks[0].operations) == 1 + assert len(res.chunks[1].operations) == 2 + assert len(res.chunks[2].operations) == 2 + assert res.chunks[2].bytecode_no == 11 + +def test_inlined_call(): + ops = parse(""" + [] + debug_merge_point(' #28 CALL_FUNCTION', 0) + i18 = getfield_gc(p0, descr=) + debug_merge_point(' #0 LOAD_FAST', 1) + debug_merge_point(' #3 LOAD_CONST', 1) + debug_merge_point(' #7 RETURN_VALUE', 1) + debug_merge_point(' #31 STORE_FAST', 0) + """) + res = slice_debug_merge_points(ops.operations, LoopStorage()) + assert len(res.chunks) == 3 # two chunks + inlined call + assert isinstance(res.chunks[0], Bytecode) + assert isinstance(res.chunks[1], Function) + assert isinstance(res.chunks[2], Bytecode) + assert res.chunks[1].path == "1" + assert len(res.chunks[1].chunks) == 3 + +def test_name(): + ops = parse(''' + [i0] + debug_merge_point(" #10 ADD", 0) + debug_merge_point(" #11 SUB", 0) + i1 = int_add(i0, 1) + debug_merge_point(" #11 SUB", 0) + i2 = int_add(i1, 1) + ''') + res = slice_debug_merge_points(ops.operations, LoopStorage()) + assert res.repr() == res.chunks[0].repr() + assert res.repr() == "stuff, file '/tmp/x.py', line 200" + assert res.startlineno == 200 + assert res.filename == '/tmp/x.py' + assert res.name == 'stuff' + +def test_name_no_first(): + ops = parse(''' + [i0] + i3 = int_add(i0, 1) + debug_merge_point(" #10 ADD", 0) + debug_merge_point(" #11 SUB", 0) + i1 = int_add(i0, 1) + debug_merge_point(" #11 SUB", 0) + i2 = int_add(i1, 1) + ''') + res = slice_debug_merge_points(ops.operations, LoopStorage()) + assert res.repr() == res.chunks[1].repr() + +def test_lineno(): + fname = str(py.path.local(__file__).join('..', 'x.py')) + ops = parse(''' + [i0, i1] + debug_merge_point(" #0 LOAD_FAST", 0) + debug_merge_point(" #3 LOAD_FAST", 0) + debug_merge_point(" #6 BINARY_ADD", 0) + debug_merge_point(" #7 RETURN_VALUE", 0) + ''' % locals()) + res = slice_debug_merge_points(ops.operations, LoopStorage()) + assert res.chunks[1].lineno == 3 + +def test_linerange(): + fname = str(py.path.local(__file__).join('..', 'x.py')) + ops = parse(''' + [i0, i1] + debug_merge_point(" #9 LOAD_FAST", 0) + debug_merge_point(" #12 LOAD_CONST", 0) + debug_merge_point(" #22 LOAD_CONST", 0) + debug_merge_point(" #28 LOAD_CONST", 0) + debug_merge_point(" #6 SETUP_LOOP", 0) + ''' % locals()) + res = slice_debug_merge_points(ops.operations, LoopStorage()) + assert res.linerange == (7, 9) + assert res.lineset == set([7, 8, 9]) + +def test_linerange_notstarts(): + fname = str(py.path.local(__file__).join('..', 'x.py')) + ops = parse(""" + [p6, p1] + debug_merge_point(' #17 FOR_ITER', 0) + guard_class(p6, 144264192, descr=) + p12 = getfield_gc(p6, descr=) + """ % locals()) + res = slice_debug_merge_points(ops.operations, LoopStorage()) + assert res.lineset + +def test_reassign_loops(): + main = parse(''' + [v0] + guard_false(v0, descr=) [] + ''') + main.count = 10 + bridge = parse(''' + # bridge out of Guard 18 with 13 ops + [i0, i1] + int_add(i0, i1) + ''') + bridge.count = 3 + entry_bridge = parse(''' + # Loop 3 : entry bridge + [] + ''') + loops = LoopStorage().reconnect_loops([main, bridge, entry_bridge]) + assert len(loops) == 2 + assert len(loops[0].operations[0].bridge.operations) == 1 + assert loops[0].operations[0].bridge.no == 18 + assert loops[0].operations[0].percentage == 30 + +def test_adjust_bridges(): + main = parse(''' + [v0] + guard_false(v0, descr=) + guard_true(v0, descr=) + ''') + bridge = parse(''' + # bridge out of Guard 13 + [] + int_add(0, 1) + ''') + loops = LoopStorage().reconnect_loops([main, bridge]) + assert adjust_bridges(main, {})[1].name == 'guard_true' + assert adjust_bridges(main, {'loop-13': True})[1].name == 'int_add' + +def test_parsing_strliteral(): + loop = parse(""" + debug_merge_point('StrLiteralSearch at 11/51 [17, 8, 3, 1, 1, 1, 1, 51, 0, 19, 51, 1]', 0) + """) + ops = slice_debug_merge_points(loop.operations, LoopStorage()) + chunk = ops.chunks[0] + assert chunk.bytecode_name == 'StrLiteralSearch' + +LINES = ''' +0:3 +1:3 +2:604 +3:396 +4:102 +5:2000 +6:3147 +7:2445 +8:2005 +9:2000 +10:1420 +11:40 +12:0 +'''.split("\n") + +def test_parse_log_count(): + class Loop(object): + pass + + loops = [Loop() for i in range(13)] + nums = parse_log_counts(LINES, loops) + assert nums[5] == 2000 + assert loops[9].count == 2000 --- /dev/null +++ b/_jitviewer/loops.py @@ -0,0 +1,360 @@ + +import re, sys +from pypy.jit.metainterp.resoperation import rop, opname +from disassembler import dis +from pypy.jit.tool.oparser import OpParser + +class Html(str): + def __html__(self): + return self + + def plaintext(self): + # This is not a general way to strip tags, but it's good enough to use + # in tests + import re + s = re.sub('<.*?>', '', self) + s = s.replace("<", "<") + s = s.replace(">", ">") + s = s.replace("&", "&") + return s + + +def cssclass(cls, s, **kwds): + attrs = ['%s="%s"' % (name, value) for name, value in kwds.iteritems()] + return '%s' % (cls, ' '.join(attrs), s) + + +def _new_binop(name): + import cgi + name = cgi.escape(name) + def f(self): + return '%s = %s %s %s' % (self.getres(), self.getarg(0), name, self.getarg(1)) + return f + +class Op(object): + bridge = None + + def __init__(self, name, args, res, descr): + self.name = name + self.args = args + self.res = res + self.descr = descr + self._is_guard = name.startswith('guard_') + if self._is_guard: + self.guard_no = int(self.descr[len('', 'int_gt'), + ('<=', 'int_le'), + ('>=', 'int_ge'), + ('+', 'int_add'), + ('+', 'float_add'), + ('-', 'int_sub'), + ('-', 'float_sub'), + ('&', 'int_and')]: + locals()['repr_' + name] = _new_binop(bin_op) + + def repr_guard_true(self): + return '%s is true' % self.getarg(0) + + def repr_guard_false(self): + return '%s is false' % self.getarg(0) + + def repr_guard_value(self): + return '%s is %s' % (self.getarg(0), self.getarg(1)) + + def repr_guard_isnull(self): + return '%s is null' % self.getarg(0) + + def repr_getfield_raw(self): + name, field = self.descr.split(' ')[1].rsplit('.', 1) + return '%s = ((%s)%s).%s' % (self.getres(), name, self.getarg(0), field[2:]) + + def repr_getfield_gc(self): + fullname, field = self.descr.split(' ')[1].rsplit('.', 1) + names = fullname.rsplit('.', 1) + if len(names) == 2: + namespace, classname = names + else: + namespace = '' + classname = names[0] + namespace = cssclass('namespace', namespace) + classname = cssclass('classname', classname) + field = cssclass('fieldname', field) + + obj = self.getarg(0) + return '%s = ((%s.%s)%s).%s' % (self.getres(), namespace, classname, obj, field) + repr_getfield_gc_pure = repr_getfield_gc + + def repr_setfield_raw(self): + name, field = self.descr.split(' ')[1].rsplit('.', 1) + return '((%s)%s).%s = %s' % (name, self.getarg(0), field[2:], self.getarg(1)) + + def repr_setfield_gc(self): + name, field = self.descr.split(' ')[1].rsplit('.', 1) + return '((%s)%s).%s = %s' % (name, self.getarg(0), field, self.getarg(1)) + + def generic_repr(self): + if self.res is not None: + return '%s = %s(%s)' % (self.getres(), self.name, ', '.join(self.args)) + else: + return '%s(%s)' % (self.name, ', '.join(self.args)) + + def __repr__(self): + return '<%s (%s)>' % (self.name, ', '.join([repr(a) + for a in self.args])) + +class SimpleParser(OpParser): + def parse_args(self, opname, argspec): + if not argspec.strip(): + return [], None + if opname == 'debug_merge_point': + return argspec.rsplit(", ", 1), None + else: + args = argspec.split(', ') + descr = None + if args[-1].startswith('descr='): + descr = args[-1][len('descr='):] + args = args[:-1] + return (args, descr) + + def box_for_var(self, res): + return res + + def create_op(self, opnum, args, res, descr): + return Op(intern(opname[opnum].lower()), args, res, descr) + +class NonCodeError(Exception): + pass + +class Bytecode(object): + filename = None + startlineno = 0 + name = None + code = None + bytecode_no = 0 + bytecode_name = None + is_bytecode = True + inline_level = None + + def __init__(self, operations, storage): + if operations[0].name == 'debug_merge_point': + self.inline_level = int(operations[0].args[1]) + m = re.search('\w]+), file \'(.+?)\', line (\d+)> #(\d+) (\w+)', + operations[0].getarg(0)) + if m is None: + # a non-code loop, like StrLiteralSearch or something + self.bytecode_name = operations[0].args[0].split(" ")[0][1:] + else: + self.name, self.filename, lineno, bytecode_no, self.bytecode_name = m.groups() + self.startlineno = int(lineno) + self.bytecode_no = int(bytecode_no) + self.operations = operations + self.storage = storage + + def repr(self): + if self.filename is None: + return "Unknown" + return "%s, file '%s', line %d" % (self.name, self.filename, + self.startlineno) + + def getcode(self): + if self.code is None: + self.code = dis(self.storage.load_code(self.filename)[self.startlineno]) + return self.code + + def getlineno(self): + code = self.getcode() + return code.map[self.bytecode_no].lineno + lineno = property(getlineno) + + def getline_starts_here(self): + code = self.getcode() + return code.map[self.bytecode_no].line_starts_here + line_starts_here = property(getline_starts_here) + + def __repr__(self): + return "[%s]" % ", ".join([repr(op) for op in self.operations]) + + def pretty_print(self, out): + pass + + def html_repr(self): + if self.filename is not None: + code = self.getcode() + opcode = self.code.map[self.bytecode_no] + return '%s %s' % (self.bytecode_name, opcode.argstr) + else: + return self.bytecode_name + +class Function(object): + filename = None + name = None + startlineno = 0 + _linerange = None + _lineset = None + is_bytecode = False + inline_level = None + + def __init__(self, chunks, path, storage): + self.path = path + self.chunks = chunks + for chunk in self.chunks: + if chunk.filename is not None: + self.startlineno = chunk.startlineno + self.filename = chunk.filename + self.name = chunk.name + self.inline_level = chunk.inline_level + break + self.storage = storage + + def getlinerange(self): + if self._linerange is None: + self._compute_linerange() + return self._linerange + linerange = property(getlinerange) + + def getlineset(self): + if self._lineset is None: + self._compute_linerange() + return self._lineset + lineset = property(getlineset) + + def _compute_linerange(self): + self._lineset = set() + minline = sys.maxint + maxline = -1 + for chunk in self.chunks: + if chunk.is_bytecode and chunk.filename is not None: + lineno = chunk.lineno + minline = min(minline, lineno) + maxline = max(maxline, lineno) + if chunk.line_starts_here or len(chunk.operations) > 1: + self._lineset.add(lineno) + if minline == sys.maxint: + minline = 0 + maxline = 0 + self._linerange = minline, maxline + + def html_repr(self): + return "inlined call to %s in %s" % (self.name, self.filename) + + def repr(self): + if self.filename is None: + return "Unknown" + return "%s, file '%s', line %d" % (self.name, self.filename, + self.startlineno) + + def __repr__(self): + return "[%s]" % ", ".join([repr(chunk) for chunk in self.chunks]) + + def pretty_print(self, out): + print >>out, "Loop starting at %s in %s at %d" % (self.name, + self.filename, self.startlineno) + lineno = -1 + for chunk in self.chunks: + if chunk.filename is not None and chunk.lineno != lineno: + lineno = chunk.lineno + source = chunk.getcode().source[chunk.lineno - + chunk.startlineno] + print >>out, " ", source + chunk.pretty_print(out) + +def parse_log_counts(lines, loops): + nums = [] + i = 0 + for line in lines: + if line: + num, count = line.split(':') + assert int(num) == i + count = int(count) + nums.append(count) + loops[i].count = count + i += 1 + return nums + +def parse(input): + return SimpleParser(input, None, {}, 'lltype', None, + nonstrict=True).parse() + +def slice_debug_merge_points(operations, storage, limit=None): + """ Slice given operation list into a chain of Bytecode chunks. + Also detect inlined functions and make them Function + """ + stack = [] + + def getpath(stack): + return ",".join([str(len(v)) for v in stack]) + + def append_to_res(bc): + if not stack: + stack.append([]) + else: + if bc.inline_level + 1 != len(stack): + if bc.inline_level < len(stack): + last = stack.pop() + stack[-1].append(Function(last, getpath(stack), storage)) + else: + stack.append([]) + stack[-1].append(bc) + + so_far = [] + stack = [] + for op in operations: + if op.name == 'debug_merge_point': + if so_far: + append_to_res(Bytecode(so_far, storage)) + if limit: + break + so_far = [] + so_far.append(op) + if so_far: + append_to_res(Bytecode(so_far, storage)) + # wrap stack back up + if not stack: + # no ops whatsoever + return Function([], getpath(stack), storage) + while True: + next = stack.pop() + if not stack: + return Function(next, getpath(stack), storage) + stack[-1].append(Function(next, getpath(stack), storage)) + +def adjust_bridges(loop, bridges): + """ Slice given loop according to given bridges to follow. Returns a plain + list of operations. + """ + ops = loop.operations + res = [] + i = 0 + while i < len(ops): + op = ops[i] + if op.is_guard() and bridges.get('loop-' + str(op.guard_no), None): + res.append(op) + i = 0 + ops = op.bridge.operations + else: + res.append(op) + i += 1 + return res --- a/README +++ b/README @@ -8,11 +8,15 @@ Now install the dependencies: pip install flask pygments simplejson +or + + python setup.py develop + It also requires pypy to be importable (as in source code). Finally, run it: - python baseviewer.py log + jitviewer.py log where log is a logfile generated by PYPYLOG=jit-log-opt:log pypy-c . An example log file comes with a checkout. --- /dev/null +++ b/setup.py @@ -0,0 +1,16 @@ +#!/usr/bin/env pypy-c + +from setuptools import setup + +setup(name='JitViewer', + version='0.1', + description="Viewer for pypy's jit traces", + author='Maciej Fijalkowski, Antonio Cuni and the PyPy team', + author_email='fijall at gmail.com', + url='http://pypy.org', + packages=['jitviewer'], + scripts=['bin/jitviewer.py'], + requires=['flask', 'pygments', 'simplejson'], + include_package_data=True, + package_data={'': ['templates/*.html', 'static/*']}, + ) --- /dev/null +++ b/_jitviewer/test/test_disassembler.py @@ -0,0 +1,34 @@ + +from _jitviewer import disassembler +import sys +import py + +def f(a, b): + return a + b + +def g(a, b): + c = a+b + return c + +def test_disassembler(): + res = disassembler.dis(f) + if sys.version_info[:2] != (2, 6): + py.test.skip("2.6 only test") + assert len(res.opcodes) == 4 + assert [x.__class__.__name__ for x in res.opcodes] == [ + 'LOAD_FAST', 'LOAD_FAST', 'BINARY_ADD', 'RETURN_VALUE'] + for i in range(4): + assert res.opcodes[i].lineno == f.func_code.co_firstlineno + 1 + assert res.opcodes[0].argstr == 'a' + +def test_line_starting_opcodes(): + if sys.version_info[:2] != (2, 6): + py.test.skip("2.6 only test") + res = disassembler.dis(g) + assert len(res.opcodes) == 6 + for i, opcode in enumerate(res.opcodes): + if i in (0, 4): + assert opcode.__class__.__name__ == 'LOAD_FAST' + assert opcode.line_starts_here + else: + assert not opcode.line_starts_here --- a/source.py +++ /dev/null @@ -1,39 +0,0 @@ - -import re - -def f(): - i = 0 - while i < 1003: - i += 1 - -f() - -def inner(i): - return i + 1 - -def inlined_call(): - i = 0 - while i < 1003: - i = inner(i) - -inlined_call() - -def bridge(): - s = 0 - i = 0 - while i < 10000: - if i % 2: - s += 1 - else: - s += 2 - i += 1 - return s - -bridge() - -def inlined_str_stuff(): - s = [str(i) for i in range(3000)] - for elem in s: - re.search('3', elem) - -inlined_str_stuff() --- a/storage.py +++ /dev/null @@ -1,56 +0,0 @@ - -""" This file represents a storage mechanism that let us invent unique names -for all loops and bridges, so http requests can refer to them by name -""" - -import os -from loops import Function, Bytecode -from module_finder import gather_all_code_objs - -class LoopStorage(object): - def __init__(self, extrapath=None): - self.loops = None - self.functions = {} - self.codes = {} - self.extrapath = extrapath - - def load_code(self, fname): - try: - return self.codes[fname] - except KeyError: - if os.path.isabs(fname): - res = gather_all_code_objs(fname) - else: - if self.extrapath is None: - raise IOError("Cannot find %s" % fname) - res = gather_all_code_objs(os.path.join(self.extrapath, fname)) - self.codes[fname] = res - return res - - def reconnect_loops(self, loops): - """ Re-connect loops in a way that entry bridges are filtered out - and normal bridges are associated with guards. Returning list of - normal loops. - """ - res = [] - guard_dict = {} - for loop_no, loop in enumerate(loops): - for op in loop.operations: - if op.name.startswith('guard_'): - guard_dict[int(op.descr[len('', '', self) - s = s.replace("<", "<") - s = s.replace(">", ">") - s = s.replace("&", "&") - return s - - -def cssclass(cls, s, **kwds): - attrs = ['%s="%s"' % (name, value) for name, value in kwds.iteritems()] - return '%s' % (cls, ' '.join(attrs), s) - - -def _new_binop(name): - import cgi - name = cgi.escape(name) - def f(self): - return '%s = %s %s %s' % (self.getres(), self.getarg(0), name, self.getarg(1)) - return f - -class Op(object): - bridge = None - - def __init__(self, name, args, res, descr): - self.name = name - self.args = args - self.res = res - self.descr = descr - self._is_guard = name.startswith('guard_') - if self._is_guard: - self.guard_no = int(self.descr[len('', 'int_gt'), - ('<=', 'int_le'), - ('>=', 'int_ge'), - ('+', 'int_add'), - ('+', 'float_add'), - ('-', 'int_sub'), - ('-', 'float_sub'), - ('&', 'int_and')]: - locals()['repr_' + name] = _new_binop(bin_op) - - def repr_guard_true(self): - return '%s is true' % self.getarg(0) - - def repr_guard_false(self): - return '%s is false' % self.getarg(0) - - def repr_guard_value(self): - return '%s is %s' % (self.getarg(0), self.getarg(1)) - - def repr_guard_isnull(self): - return '%s is null' % self.getarg(0) - - def repr_getfield_raw(self): - name, field = self.descr.split(' ')[1].rsplit('.', 1) - return '%s = ((%s)%s).%s' % (self.getres(), name, self.getarg(0), field[2:]) - - def repr_getfield_gc(self): - fullname, field = self.descr.split(' ')[1].rsplit('.', 1) - names = fullname.rsplit('.', 1) - if len(names) == 2: - namespace, classname = names - else: - namespace = '' - classname = names[0] - namespace = cssclass('namespace', namespace) - classname = cssclass('classname', classname) - field = cssclass('fieldname', field) - - obj = self.getarg(0) - return '%s = ((%s.%s)%s).%s' % (self.getres(), namespace, classname, obj, field) - repr_getfield_gc_pure = repr_getfield_gc - - def repr_setfield_raw(self): - name, field = self.descr.split(' ')[1].rsplit('.', 1) - return '((%s)%s).%s = %s' % (name, self.getarg(0), field[2:], self.getarg(1)) - - def repr_setfield_gc(self): - name, field = self.descr.split(' ')[1].rsplit('.', 1) - return '((%s)%s).%s = %s' % (name, self.getarg(0), field, self.getarg(1)) - - def generic_repr(self): - if self.res is not None: - return '%s = %s(%s)' % (self.getres(), self.name, ', '.join(self.args)) - else: - return '%s(%s)' % (self.name, ', '.join(self.args)) - - def __repr__(self): - return '<%s (%s)>' % (self.name, ', '.join([repr(a) - for a in self.args])) - -class SimpleParser(OpParser): - def parse_args(self, opname, argspec): - if not argspec.strip(): - return [], None - if opname == 'debug_merge_point': - return argspec.rsplit(", ", 1), None - else: - args = argspec.split(', ') - descr = None - if args[-1].startswith('descr='): - descr = args[-1][len('descr='):] - args = args[:-1] - return (args, descr) - - def box_for_var(self, res): - return res - - def create_op(self, opnum, args, res, descr): - return Op(intern(opname[opnum].lower()), args, res, descr) - -class NonCodeError(Exception): - pass - -class Bytecode(object): - filename = None - startlineno = 0 - name = None - code = None - bytecode_no = 0 - bytecode_name = None - is_bytecode = True - inline_level = None - - def __init__(self, operations, storage): - if operations[0].name == 'debug_merge_point': - self.inline_level = int(operations[0].args[1]) - m = re.search('\w]+), file \'(.+?)\', line (\d+)> #(\d+) (\w+)', - operations[0].getarg(0)) - if m is None: - # a non-code loop, like StrLiteralSearch or something - self.bytecode_name = operations[0].args[0].split(" ")[0][1:] - else: - self.name, self.filename, lineno, bytecode_no, self.bytecode_name = m.groups() - self.startlineno = int(lineno) - self.bytecode_no = int(bytecode_no) - self.operations = operations - self.storage = storage - - def repr(self): - if self.filename is None: - return "Unknown" - return "%s, file '%s', line %d" % (self.name, self.filename, - self.startlineno) - - def getcode(self): - if self.code is None: - self.code = dis(self.storage.load_code(self.filename)[self.startlineno]) - return self.code - - def getlineno(self): - code = self.getcode() - return code.map[self.bytecode_no].lineno - lineno = property(getlineno) - - def getline_starts_here(self): - code = self.getcode() - return code.map[self.bytecode_no].line_starts_here - line_starts_here = property(getline_starts_here) - - def __repr__(self): - return "[%s]" % ", ".join([repr(op) for op in self.operations]) - - def pretty_print(self, out): - pass - - def html_repr(self): - if self.filename is not None: - code = self.getcode() - opcode = self.code.map[self.bytecode_no] - return '%s %s' % (self.bytecode_name, opcode.argstr) - else: - return self.bytecode_name - -class Function(object): - filename = None - name = None - startlineno = 0 - _linerange = None - _lineset = None - is_bytecode = False - inline_level = None - - def __init__(self, chunks, path, storage): - self.path = path - self.chunks = chunks - for chunk in self.chunks: - if chunk.filename is not None: - self.startlineno = chunk.startlineno - self.filename = chunk.filename - self.name = chunk.name - self.inline_level = chunk.inline_level - break - self.storage = storage - - def getlinerange(self): - if self._linerange is None: - self._compute_linerange() - return self._linerange - linerange = property(getlinerange) - - def getlineset(self): - if self._lineset is None: - self._compute_linerange() - return self._lineset - lineset = property(getlineset) - - def _compute_linerange(self): - self._lineset = set() - minline = sys.maxint - maxline = -1 - for chunk in self.chunks: - if chunk.is_bytecode and chunk.filename is not None: - lineno = chunk.lineno - minline = min(minline, lineno) - maxline = max(maxline, lineno) - if chunk.line_starts_here or len(chunk.operations) > 1: - self._lineset.add(lineno) - if minline == sys.maxint: - minline = 0 - maxline = 0 - self._linerange = minline, maxline - - def html_repr(self): - return "inlined call to %s in %s" % (self.name, self.filename) - - def repr(self): - if self.filename is None: - return "Unknown" - return "%s, file '%s', line %d" % (self.name, self.filename, - self.startlineno) - - def __repr__(self): - return "[%s]" % ", ".join([repr(chunk) for chunk in self.chunks]) - - def pretty_print(self, out): - print >>out, "Loop starting at %s in %s at %d" % (self.name, - self.filename, self.startlineno) - lineno = -1 - for chunk in self.chunks: - if chunk.filename is not None and chunk.lineno != lineno: - lineno = chunk.lineno - source = chunk.getcode().source[chunk.lineno - - chunk.startlineno] - print >>out, " ", source - chunk.pretty_print(out) - -def parse_log_counts(lines, loops): - nums = [] - i = 0 - for line in lines: - if line: - num, count = line.split(':') - assert int(num) == i - count = int(count) - nums.append(count) - loops[i].count = count - i += 1 - return nums - -def parse(input): - return SimpleParser(input, None, {}, 'lltype', None, - nonstrict=True).parse() - -def slice_debug_merge_points(operations, storage, limit=None): - """ Slice given operation list into a chain of Bytecode chunks. - Also detect inlined functions and make them Function - """ - stack = [] - - def getpath(stack): - return ",".join([str(len(v)) for v in stack]) - - def append_to_res(bc): - if not stack: - stack.append([]) - else: - if bc.inline_level + 1 != len(stack): - if bc.inline_level < len(stack): - last = stack.pop() - stack[-1].append(Function(last, getpath(stack), storage)) - else: - stack.append([]) - stack[-1].append(bc) - - so_far = [] - stack = [] - for op in operations: - if op.name == 'debug_merge_point': - if so_far: - append_to_res(Bytecode(so_far, storage)) - if limit: - break - so_far = [] - so_far.append(op) - if so_far: - append_to_res(Bytecode(so_far, storage)) - # wrap stack back up - if not stack: - # no ops whatsoever - return Function([], getpath(stack), storage) - while True: - next = stack.pop() - if not stack: - return Function(next, getpath(stack), storage) - stack[-1].append(Function(next, getpath(stack), storage)) - -def adjust_bridges(loop, bridges): - """ Slice given loop according to given bridges to follow. Returns a plain - list of operations. - """ - ops = loop.operations - res = [] - i = 0 - while i < len(ops): - op = ops[i] - if op.is_guard() and bridges.get('loop-' + str(op.guard_no), None): - res.append(op) - i = 0 - ops = op.bridge.operations - else: - res.append(op) - i += 1 - return res --- a/jitviewer.py +++ /dev/null @@ -1,127 +0,0 @@ -#!/usr/bin/env pypy-c -""" A web-based browser of your log files. Run by - -baseviewer.py - -and point your browser to http://localhost:5000 - -Demo logfile available in this directory as 'log'. -""" - -import sys -import os -import cgi -import flask -import inspect -from pypy.tool.logparser import parse_log_file, extract_category -from loops import (parse, slice_debug_merge_points, adjust_bridges, - parse_log_counts) -from storage import LoopStorage -from display import CodeRepr, CodeReprNoFile - -from pygments import highlight -from pygments.lexers import PythonLexer -from pygments.formatters import HtmlFormatter - -CUTOFF = 30 - -class Server(object): - def __init__(self, storage): - self.storage = storage - - def index(self): - all = flask.request.args.get('all', None) - loops = [] - for index, loop in enumerate(self.storage.loops): - if 'entry bridge' in loop.comment: - is_entry = True - else: - is_entry = False - func = slice_debug_merge_points(loop.operations, self.storage, - limit=1) - func.count = loop.count - loops.append((is_entry, index, func)) - loops.sort(lambda a, b: cmp(b[2].count, a[2].count)) - if len(loops) > CUTOFF: - extra_data = "Show all (%d) loops" % len(loops) - else: - extra_data = "" - if not all: - loops = loops[:CUTOFF] - return flask.render_template('index.html', loops=loops, - extra_data=extra_data) - - def loop(self): - no = int(flask.request.args.get('no', '0')) - orig_loop = self.storage.loops[no] - ops = adjust_bridges(orig_loop, flask.request.args) - loop = slice_debug_merge_points(ops, self.storage) - path = flask.request.args.get('path', '').split(',') - if path: - up = '"' + ','.join(path[:-1]) + '"' - else: - up = '""' - callstack = [] - path_so_far = [] - for e in path: - if e: - callstack.append((','.join(path_so_far), - '%s in %s at %d' % (loop.name, - loop.filename, - loop.startlineno))) - loop = loop.chunks[int(e)] - path_so_far.append(e) - callstack.append((','.join(path_so_far), '%s in %s at %d' % (loop.name, - loop.filename, loop.startlineno))) - - startline, endline = loop.linerange - if loop.filename is not None: - code = self.storage.load_code(loop.filename)[loop.startlineno] - source = CodeRepr(inspect.getsource(code), code, loop) - else: - source = CodeReprNoFile(loop) - d = {'html': flask.render_template('loop.html', - source=source, - current_loop=no, - upper_path=up, - show_upper_path=bool(path)), - 'scrollto': startline, - 'callstack': callstack} - return flask.jsonify(d) - -def start_browser(url): - import time - import webbrowser - import threading - def run(): - time.sleep(0.5) # give the server some time to start - webbrowser.open(url) - th = threading.Thread(target=run) - th.start() - return th - -def main(): - - if not '__pypy__' in sys.builtin_module_names: - print "Please run it using pypy-c" - sys.exit(1) - if len(sys.argv) != 2: - print __doc__ - sys.exit(1) - log = parse_log_file(sys.argv[1]) - extra_path = os.path.dirname(sys.argv[1]) - storage = LoopStorage(extra_path) - loops = [parse(l) for l in extract_category(log, "jit-log-opt-")] - parse_log_counts(open(sys.argv[1] + '.count').readlines(), loops) - storage.reconnect_loops(loops) - app = flask.Flask(__name__) - server = Server(storage) - app.debug = True - app.route('/')(server.index) - app.route('/loop')(server.loop) - #th = start_browser('http://localhost:5000/') - app.run(use_reloader=False, host='0.0.0.0') - #th.join() - -if __name__ == '__main__': - main() --- a/display.py +++ /dev/null @@ -1,52 +0,0 @@ - -from pypy.jit.metainterp.resoperation import rop -from loops import Bytecode - -class LineRepr(object): - """ A representation of a single line - """ - def __init__(self, line, in_loop, chunks=None): - self.line = line - self.in_loop = in_loop - if chunks is None: - self.chunks = [] - else: - self.chunks = chunks - -class CodeReprNoFile(object): - firstlineno = 0 - - def __init__(self, loop): - self.lines = [LineRepr('', True, loop.chunks)] - -class CodeRepr(object): - """ A representation of a single code object suitable for display - """ - def __init__(self, source, code, loop): - lineset = loop.lineset - self.lines = [] - self.firstlineno = code.co_firstlineno - for i, line in enumerate(source.split("\n")): - no = i + code.co_firstlineno - in_loop = no in lineset - self.lines.append(LineRepr(line, in_loop)) - - last_lineno = -1 - for chunk in loop.chunks: - if chunk.is_bytecode: - chunk.cssclass = 'dmp ' - if len(chunk.operations) <= 1: - chunk.cssclass += 'empty' - else: - chunk.cssclass += 'nonempty' - no = chunk.lineno - if no < last_lineno: - no = last_lineno - else: - last_lineno = no - else: - no = last_lineno - self.lines[no - self.firstlineno].chunks.append(chunk) - - - --- /dev/null +++ b/_jitviewer/module_finder.py @@ -0,0 +1,25 @@ + +import os, sys, marshal, types, struct, imp + +def _all_codes_from(code): + res = {} + more = [code] + while more: + next = more.pop() + res[next.co_firstlineno] = next + more += [co for co in next.co_consts + if isinstance(co, types.CodeType)] + return res + +def gather_all_code_objs(fname): + """ Gathers all code objects from a give fname and sorts them by + starting lineno + """ + fname = str(fname) + if fname.endswith('.pyc'): + code = compile(open(fname[:-1]).read(), fname, 'exec') + elif fname.endswith('.py'): + code = compile(open(fname).read(), fname, 'exec') + else: + raise Exception("Unknown file extension: %s" % fname) + return _all_codes_from(code) --- a/test/test_storage.py +++ /dev/null @@ -1,9 +0,0 @@ - -import py -from storage import LoopStorage - -def test_load_codes(): - tmppath = py.test.ensuretemp('load_codes') - tmppath.join("x.py").write("def f(): pass") # one code - s = LoopStorage(str(tmppath)) - assert s.load_code(str(tmppath.join('x.py'))) == s.load_code('x.py') --- a/disassembler.py +++ /dev/null @@ -1,297 +0,0 @@ -"""Disassembler of Python byte code into mnemonics. - -Comes from standard library, modified for the purpose of having a structured -view on things -""" - -import sys -import types -import inspect - -from opcode import * -from opcode import __all__ as _opcodes_all - -__all__ = ["dis","disassemble","distb","disco"] + _opcodes_all -del _opcodes_all - -class Opcode(object): - """ An abstract base class for all opcode implementations - """ - def __init__(self, pos, lineno, arg=None, argstr=''): - self.pos = pos - self.arg = arg - self.argstr = argstr - self.lineno = lineno - self.line_starts_here = False - - def __repr__(self): - if self.arg is None: - return "<%s at %d>" % (self.__class__.__name__, self.pos) - return "<%s (%s) at %d>" % (self.__class__.__name__, self.arg, self.pos) - -class CodeRepresentation(object): - """ Representation of opcodes - """ - def __init__(self, opcodes, source): - self.opcodes = opcodes - self.map = {} - current_lineno = None - for opcode in opcodes: - self.map[opcode.pos] = opcode - if opcode.lineno != current_lineno: - opcode.line_starts_here = True - current_lineno = opcode.lineno - self.source = source.split("\n") - -def _setup(): - for opcode in opname: - if not opcode.startswith('<'): - class O(Opcode): - pass - opcode = opcode.replace('+', '_') - O.__name__ = opcode - globals()[opcode] = O - -_setup() - -def dis(x=None): - """Disassemble classes, methods, functions, or code. - - With no argument, disassemble the last traceback. - - """ - if x is None: - distb() - return - if type(x) is types.InstanceType: - x = x.__class__ - if hasattr(x, 'im_func'): - x = x.im_func - if hasattr(x, 'func_code'): - x = x.func_code - if hasattr(x, '__dict__'): - xxx - items = x.__dict__.items() - items.sort() - for name, x1 in items: - if type(x1) in (types.MethodType, - types.FunctionType, - types.CodeType, - types.ClassType): - print "Disassembly of %s:" % name - try: - dis(x1) - except TypeError, msg: - print "Sorry:", msg - print - elif hasattr(x, 'co_code'): - return disassemble(x) - elif isinstance(x, str): - return disassemble_string(x) - else: - raise TypeError, \ - "don't know how to disassemble %s objects" % \ - type(x).__name__ - -def distb(tb=None): - """Disassemble a traceback (default: last traceback).""" - if tb is None: - try: - tb = sys.last_traceback - except AttributeError: - raise RuntimeError, "no last traceback to disassemble" - while tb.tb_next: tb = tb.tb_next - disassemble(tb.tb_frame.f_code, tb.tb_lasti) - -def disassemble(co, lasti=-1): - """Disassemble a code object.""" - source = inspect.getsource(co) - code = co.co_code - labels = findlabels(code) - linestarts = dict(findlinestarts(co)) - n = len(code) - i = 0 - extended_arg = 0 - free = None - res = [] - lastline = co.co_firstlineno - while i < n: - c = code[i] - op = ord(c) - if i in linestarts: - lastline = linestarts[i] - - #if i == lasti: - # xxx - # print '-->', - #else: - # xxx - # print ' ', - #if i in labels: - # xxx - # print '>>', - #else: - # xxx - # print ' ', - #xxx - pos = i - i = i + 1 - if op >= HAVE_ARGUMENT: - oparg = ord(code[i]) + ord(code[i+1])*256 + extended_arg - opargstr = str(oparg) - extended_arg = 0 - i = i+2 - if op == EXTENDED_ARG: - extended_arg = oparg*65536L - if op in hasconst: - opargstr = repr(co.co_consts[oparg]) - elif op in hasname: - opargstr = co.co_names[oparg] - elif op in hasjrel: - opargstr = 'to ' + repr(i + oparg) - elif op in haslocal: - opargstr = co.co_varnames[oparg] - elif op in hascompare: - opargstr = cmp_op[oparg] - elif op in hasfree: - if free is None: - free = co.co_cellvars + co.co_freevars - opargstr = free[oparg] - else: - oparg = None - opargstr = '' - opcls = globals()[opname[op].replace('+', '_')] - res.append(opcls(pos, lastline, oparg, opargstr)) - return CodeRepresentation(res, source) - -def disassemble_string(code, lasti=-1, varnames=None, names=None, - constants=None): - labels = findlabels(code) - n = len(code) - i = 0 - while i < n: - c = code[i] - op = ord(c) - if i == lasti: - xxx - print '-->', - else: - xxx - print ' ', - if i in labels: - xxx - print '>>', - else: - xxx - print ' ', - xxxx - print repr(i).rjust(4), - print opname[op].ljust(15), - i = i+1 - if op >= HAVE_ARGUMENT: - oparg = ord(code[i]) + ord(code[i+1])*256 - i = i+2 - xxx - print repr(oparg).rjust(5), - if op in hasconst: - if constants: - xxx - print '(' + repr(constants[oparg]) + ')', - else: - xxx - print '(%d)'%oparg, - elif op in hasname: - if names is not None: - xxx - print '(' + names[oparg] + ')', - else: - xxx - print '(%d)'%oparg, - elif op in hasjrel: - xxx - print '(to ' + repr(i + oparg) + ')', - elif op in haslocal: - if varnames: - xxx - print '(' + varnames[oparg] + ')', - else: - xxx - print '(%d)' % oparg, - elif op in hascompare: - xxx - print '(' + cmp_op[oparg] + ')', - xxx - print - -disco = disassemble # XXX For backwards compatibility - -def findlabels(code): - """Detect all offsets in a byte code which are jump targets. - - Return the list of offsets. - - """ - labels = [] - n = len(code) - i = 0 - while i < n: - c = code[i] - op = ord(c) - i = i+1 - if op >= HAVE_ARGUMENT: - oparg = ord(code[i]) + ord(code[i+1])*256 - i = i+2 - label = -1 - if op in hasjrel: - label = i+oparg - elif op in hasjabs: - label = oparg - if label >= 0: - if label not in labels: - labels.append(label) - return labels - -def findlinestarts(code): - """Find the offsets in a byte code which are start of lines in the source. - - Generate pairs (offset, lineno) as described in Python/compile.c. - - """ - byte_increments = [ord(c) for c in code.co_lnotab[0::2]] - line_increments = [ord(c) for c in code.co_lnotab[1::2]] - - lastlineno = None - lineno = code.co_firstlineno - addr = 0 - for byte_incr, line_incr in zip(byte_increments, line_increments): - if byte_incr: - if lineno != lastlineno: - yield (addr, lineno) - lastlineno = lineno - addr += byte_incr - lineno += line_incr - if lineno != lastlineno: - yield (addr, lineno) - -def _test(): - """Simple test program to disassemble a file.""" - if sys.argv[1:]: - if sys.argv[2:]: - sys.stderr.write("usage: python dis.py [-|file]\n") - sys.exit(2) - fn = sys.argv[1] - if not fn or fn == "-": - fn = None - else: - fn = None - if fn is None: - f = sys.stdin - else: - f = open(fn) - source = f.read() - if fn is not None: - f.close() - else: - fn = "" - code = compile(source, fn, "exec") - dis(code) --- /dev/null +++ b/_jitviewer/test/test_display.py @@ -0,0 +1,43 @@ + +from _jitviewer.display import CodeRepr + +class MockLoop(object): + pass + +class MockCode(object): + pass + +class MockChunk(object): + is_bytecode = True + + def __init__(self, operations, lineno): + self.operations = operations + self.lineno = lineno + +class Op(object): + bridge = None + + def __init__(self, a): + self.a = a + +SOURCE = """def f(): +return a + b +""" + +def test_code_repr(): + loop = MockLoop() + loop.chunks = [MockChunk([], 3), MockChunk([Op('a'), Op('b'), Op('c')], 4), + MockChunk([Op('a'), Op('b')], 4)] + MockLoop.linerange = (4, 5) + MockLoop.lineset = set([4, 5]) + code = MockCode() + code.co_firstlineno = 3 + repr = CodeRepr(SOURCE, code, loop) + assert len(repr.lines) == 3 + assert repr.lines[1].in_loop + assert not repr.lines[0].in_loop + assert repr.lines[0].chunks == [loop.chunks[0]] + assert repr.lines[1].chunks == [loop.chunks[1], loop.chunks[2]] + +def test_code_repr_bridgestate(): + pass Binary file test/xre.pyc has changed --- a/test/test_display.py +++ /dev/null @@ -1,43 +0,0 @@ - -from display import CodeRepr - -class MockLoop(object): - pass - -class MockCode(object): - pass - -class MockChunk(object): - is_bytecode = True - - def __init__(self, operations, lineno): - self.operations = operations - self.lineno = lineno - -class Op(object): - bridge = None - - def __init__(self, a): - self.a = a - -SOURCE = """def f(): -return a + b -""" - -def test_code_repr(): - loop = MockLoop() - loop.chunks = [MockChunk([], 3), MockChunk([Op('a'), Op('b'), Op('c')], 4), - MockChunk([Op('a'), Op('b')], 4)] - MockLoop.linerange = (4, 5) - MockLoop.lineset = set([4, 5]) - code = MockCode() - code.co_firstlineno = 3 - repr = CodeRepr(SOURCE, code, loop) - assert len(repr.lines) == 3 - assert repr.lines[1].in_loop - assert not repr.lines[0].in_loop - assert repr.lines[0].chunks == [loop.chunks[0]] - assert repr.lines[1].chunks == [loop.chunks[1], loop.chunks[2]] - -def test_code_repr_bridgestate(): - pass --- /dev/null +++ b/_jitviewer/display.py @@ -0,0 +1,52 @@ + +from pypy.jit.metainterp.resoperation import rop +from loops import Bytecode + +class LineRepr(object): + """ A representation of a single line + """ + def __init__(self, line, in_loop, chunks=None): + self.line = line + self.in_loop = in_loop + if chunks is None: + self.chunks = [] + else: + self.chunks = chunks + +class CodeReprNoFile(object): + firstlineno = 0 + + def __init__(self, loop): + self.lines = [LineRepr('', True, loop.chunks)] + +class CodeRepr(object): + """ A representation of a single code object suitable for display + """ + def __init__(self, source, code, loop): + lineset = loop.lineset + self.lines = [] + self.firstlineno = code.co_firstlineno + for i, line in enumerate(source.split("\n")): + no = i + code.co_firstlineno + in_loop = no in lineset + self.lines.append(LineRepr(line, in_loop)) + + last_lineno = -1 + for chunk in loop.chunks: + if chunk.is_bytecode: + chunk.cssclass = 'dmp ' + if len(chunk.operations) <= 1: + chunk.cssclass += 'empty' + else: + chunk.cssclass += 'nonempty' + no = chunk.lineno + if no < last_lineno: + no = last_lineno + else: + last_lineno = no + else: + no = last_lineno + self.lines[no - self.firstlineno].chunks.append(chunk) + + + From commits-noreply at bitbucket.org Sun Nov 21 16:57:40 2010 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Sun, 21 Nov 2010 09:57:40 -0600 (CST) Subject: [pypy-svn] jitviewer commit c8c87cffc932: essential change to test pushing Message-ID: <20101121155740.ED86C24128A@bitbucket01.managed.contegix.com> # HG changeset patch -- Bitbucket.org # Project jitviewer # URL http://bitbucket.org/pypy/jitviewer/overview # User Maciej Fijalkowski # Date 1290354802 -7200 # Node ID c8c87cffc932b2e6a8a613e7c244363aefd3136f # Parent 53403054c7d926925f697d6d519e7f42d1776541 essential change to test pushing --- a/setup.py +++ b/setup.py @@ -12,5 +12,4 @@ setup(name='JitViewer', scripts=['bin/jitviewer.py'], requires=['flask', 'pygments', 'simplejson'], include_package_data=True, - package_data={'': ['templates/*.html', 'static/*']}, - ) + package_data={'': ['templates/*.html', 'static/*']}) From arigo at codespeak.net Sun Nov 21 17:11:26 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 21 Nov 2010 17:11:26 +0100 (CET) Subject: [pypy-svn] r79312 - pypy/trunk/pypy/translator/c/gcc Message-ID: <20101121161126.DD75A282B9E@codespeak.net> Author: arigo Date: Sun Nov 21 17:11:25 2010 New Revision: 79312 Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py Log: Tentative. Remove the version of find_functions() for Mac OS/X, and replace it with the newer version for Win32, which seems to have been more debugged. Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py ============================================================================== --- pypy/trunk/pypy/translator/c/gcc/trackgcroot.py (original) +++ pypy/trunk/pypy/translator/c/gcc/trackgcroot.py Sun Nov 21 17:11:25 2010 @@ -1412,20 +1412,14 @@ in_function = False for n, line in enumerate(iterlines): if self.r_textstart.match(line): - assert not in_text, "unexpected repeated .text start: %d" % n in_text = True elif self.r_sectionstart.match(line): - if in_function: - yield in_function, functionlines - functionlines = [] in_text = False - in_function = False elif in_text and self.FunctionGcRootTracker.r_functionstart.match(line): yield in_function, functionlines functionlines = [] in_function = True functionlines.append(line) - if functionlines: yield in_function, functionlines @@ -1442,23 +1436,6 @@ format = "mingw32" FunctionGcRootTracker = Mingw32FunctionGcRootTracker - def find_functions(self, iterlines): - functionlines = [] - in_text = False - in_function = False - for n, line in enumerate(iterlines): - if self.r_textstart.match(line): - in_text = True - elif self.r_sectionstart.match(line): - in_text = False - elif in_text and self.FunctionGcRootTracker.r_functionstart.match(line): - yield in_function, functionlines - functionlines = [] - in_function = True - functionlines.append(line) - if functionlines: - yield in_function, functionlines - class MsvcAssemblerParser(AssemblerParser): format = "msvc" FunctionGcRootTracker = MsvcFunctionGcRootTracker From arigo at codespeak.net Sun Nov 21 17:17:23 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 21 Nov 2010 17:17:23 +0100 (CET) Subject: [pypy-svn] r79313 - pypy/release/1.4.x/pypy/translator/platform Message-ID: <20101121161723.3A28D5080F@codespeak.net> Author: arigo Date: Sun Nov 21 17:17:21 2010 New Revision: 79313 Modified: pypy/release/1.4.x/pypy/translator/platform/linux.py pypy/release/1.4.x/pypy/translator/platform/posix.py Log: Revert r76937, and add the workaround for Linux 32 only -- not e.g. Darwin. Modified: pypy/release/1.4.x/pypy/translator/platform/linux.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/platform/linux.py (original) +++ pypy/release/1.4.x/pypy/translator/platform/linux.py Sun Nov 21 17:17:21 2010 @@ -27,6 +27,8 @@ class Linux(BaseLinux): + shared_only = () # it seems that on 32-bit linux, compiling with -fPIC + # gives assembler that asmgcc is not happy about. def library_dirs_for_libffi_a(self): # places where we need to look for libffi.a return self.library_dirs_for_libffi() + ['/usr/lib'] Modified: pypy/release/1.4.x/pypy/translator/platform/posix.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/platform/posix.py (original) +++ pypy/release/1.4.x/pypy/translator/platform/posix.py Sun Nov 21 17:17:21 2010 @@ -104,12 +104,10 @@ else: target_name = exe_name.basename - cflags = self.cflags - if sys.maxint > 2147483647: # XXX XXX XXX sort this out - if shared: - cflags = self.cflags + self.shared_only - else: - cflags = self.cflags + self.standalone_only + if shared: + cflags = self.cflags + self.shared_only + else: + cflags = self.cflags + self.standalone_only m = GnuMakefile(path) m.exe_name = exe_name From arigo at codespeak.net Sun Nov 21 17:22:48 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 21 Nov 2010 17:22:48 +0100 (CET) Subject: [pypy-svn] r79314 - pypy/trunk/pypy/translator/platform Message-ID: <20101121162248.41A95282B9E@codespeak.net> Author: arigo Date: Sun Nov 21 17:22:46 2010 New Revision: 79314 Modified: pypy/trunk/pypy/translator/platform/linux.py pypy/trunk/pypy/translator/platform/posix.py Log: Revert r76937, and add the workaround for Linux 32 only -- not e.g. Darwin. Modified: pypy/trunk/pypy/translator/platform/linux.py ============================================================================== --- pypy/trunk/pypy/translator/platform/linux.py (original) +++ pypy/trunk/pypy/translator/platform/linux.py Sun Nov 21 17:22:46 2010 @@ -27,6 +27,8 @@ class Linux(BaseLinux): + shared_only = () # it seems that on 32-bit linux, compiling with -fPIC + # gives assembler that asmgcc is not happy about. def library_dirs_for_libffi_a(self): # places where we need to look for libffi.a return self.library_dirs_for_libffi() + ['/usr/lib'] Modified: pypy/trunk/pypy/translator/platform/posix.py ============================================================================== --- pypy/trunk/pypy/translator/platform/posix.py (original) +++ pypy/trunk/pypy/translator/platform/posix.py Sun Nov 21 17:22:46 2010 @@ -104,12 +104,10 @@ else: target_name = exe_name.basename - cflags = self.cflags - if sys.maxint > 2147483647: # XXX XXX XXX sort this out - if shared: - cflags = self.cflags + self.shared_only - else: - cflags = self.cflags + self.standalone_only + if shared: + cflags = self.cflags + self.shared_only + else: + cflags = self.cflags + self.standalone_only m = GnuMakefile(path) m.exe_name = exe_name From arigo at codespeak.net Sun Nov 21 17:34:39 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 21 Nov 2010 17:34:39 +0100 (CET) Subject: [pypy-svn] r79315 - pypy/trunk/pypy/translator/c/gcc Message-ID: <20101121163439.84B8E282B9D@codespeak.net> Author: arigo Date: Sun Nov 21 17:34:37 2010 New Revision: 79315 Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py Log: Hacking more. Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py ============================================================================== --- pypy/trunk/pypy/translator/c/gcc/trackgcroot.py (original) +++ pypy/trunk/pypy/translator/c/gcc/trackgcroot.py Sun Nov 21 17:34:37 2010 @@ -1405,6 +1405,7 @@ 'const_data' ] r_sectionstart = re.compile(r"\t\.("+'|'.join(OTHERSECTIONS)+").*$") + sections_doesnt_end_function = {'cstring': True, 'const': True} def find_functions(self, iterlines): functionlines = [] @@ -1414,6 +1415,12 @@ if self.r_textstart.match(line): in_text = True elif self.r_sectionstart.match(line): + sectionname = self.r_sectionstart.match(line).group(1) + if (in_function and + sectionname not in self.sections_doesnt_end_function): + yield in_function, functionlines + functionlines = [] + in_function = False in_text = False elif in_text and self.FunctionGcRootTracker.r_functionstart.match(line): yield in_function, functionlines From arigo at codespeak.net Sun Nov 21 17:45:43 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 21 Nov 2010 17:45:43 +0100 (CET) Subject: [pypy-svn] r79316 - pypy/trunk/pypy/translator/c/gcc/test Message-ID: <20101121164543.C64F6282B9D@codespeak.net> Author: arigo Date: Sun Nov 21 17:45:42 2010 New Revision: 79316 Modified: pypy/trunk/pypy/translator/c/gcc/test/test_trackgcroot.py Log: Fix the test. I *think* that it's ok. Modified: pypy/trunk/pypy/translator/c/gcc/test/test_trackgcroot.py ============================================================================== --- pypy/trunk/pypy/translator/c/gcc/test/test_trackgcroot.py (original) +++ pypy/trunk/pypy/translator/c/gcc/test/test_trackgcroot.py Sun Nov 21 17:45:42 2010 @@ -98,14 +98,13 @@ """ lines = source.splitlines(True) parts = list(DarwinAssemblerParser().find_functions(iter(lines))) - assert len(parts) == 7 + assert len(parts) == 6 assert parts[0] == (False, lines[:3]) assert parts[1] == (True, lines[3:7]) assert parts[2] == (True, lines[7:11]) - assert parts[3] == (True, lines[11:13]) - assert parts[4] == (False, lines[13:18]) - assert parts[5] == (True, lines[18:20]) - assert parts[6] == (False, lines[20:]) + assert parts[3] == (True, lines[11:18]) + assert parts[4] == (True, lines[18:20]) + assert parts[5] == (False, lines[20:]) def test_computegcmaptable(): tests = [] From arigo at codespeak.net Sun Nov 21 17:52:03 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 21 Nov 2010 17:52:03 +0100 (CET) Subject: [pypy-svn] r79317 - in pypy/release/1.4.x/pypy/translator/c/gcc: . test Message-ID: <20101121165203.4C5DC282B9D@codespeak.net> Author: arigo Date: Sun Nov 21 17:52:01 2010 New Revision: 79317 Modified: pypy/release/1.4.x/pypy/translator/c/gcc/test/test_trackgcroot.py pypy/release/1.4.x/pypy/translator/c/gcc/trackgcroot.py Log: Merge r79312 to r79316 from trunk. Modified: pypy/release/1.4.x/pypy/translator/c/gcc/test/test_trackgcroot.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/c/gcc/test/test_trackgcroot.py (original) +++ pypy/release/1.4.x/pypy/translator/c/gcc/test/test_trackgcroot.py Sun Nov 21 17:52:01 2010 @@ -98,14 +98,13 @@ """ lines = source.splitlines(True) parts = list(DarwinAssemblerParser().find_functions(iter(lines))) - assert len(parts) == 7 + assert len(parts) == 6 assert parts[0] == (False, lines[:3]) assert parts[1] == (True, lines[3:7]) assert parts[2] == (True, lines[7:11]) - assert parts[3] == (True, lines[11:13]) - assert parts[4] == (False, lines[13:18]) - assert parts[5] == (True, lines[18:20]) - assert parts[6] == (False, lines[20:]) + assert parts[3] == (True, lines[11:18]) + assert parts[4] == (True, lines[18:20]) + assert parts[5] == (False, lines[20:]) def test_computegcmaptable(): tests = [] Modified: pypy/release/1.4.x/pypy/translator/c/gcc/trackgcroot.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/c/gcc/trackgcroot.py (original) +++ pypy/release/1.4.x/pypy/translator/c/gcc/trackgcroot.py Sun Nov 21 17:52:01 2010 @@ -1405,6 +1405,7 @@ 'const_data' ] r_sectionstart = re.compile(r"\t\.("+'|'.join(OTHERSECTIONS)+").*$") + sections_doesnt_end_function = {'cstring': True, 'const': True} def find_functions(self, iterlines): functionlines = [] @@ -1412,20 +1413,20 @@ in_function = False for n, line in enumerate(iterlines): if self.r_textstart.match(line): - assert not in_text, "unexpected repeated .text start: %d" % n in_text = True elif self.r_sectionstart.match(line): - if in_function: + sectionname = self.r_sectionstart.match(line).group(1) + if (in_function and + sectionname not in self.sections_doesnt_end_function): yield in_function, functionlines functionlines = [] + in_function = False in_text = False - in_function = False elif in_text and self.FunctionGcRootTracker.r_functionstart.match(line): yield in_function, functionlines functionlines = [] in_function = True functionlines.append(line) - if functionlines: yield in_function, functionlines @@ -1442,23 +1443,6 @@ format = "mingw32" FunctionGcRootTracker = Mingw32FunctionGcRootTracker - def find_functions(self, iterlines): - functionlines = [] - in_text = False - in_function = False - for n, line in enumerate(iterlines): - if self.r_textstart.match(line): - in_text = True - elif self.r_sectionstart.match(line): - in_text = False - elif in_text and self.FunctionGcRootTracker.r_functionstart.match(line): - yield in_function, functionlines - functionlines = [] - in_function = True - functionlines.append(line) - if functionlines: - yield in_function, functionlines - class MsvcAssemblerParser(AssemblerParser): format = "msvc" FunctionGcRootTracker = MsvcFunctionGcRootTracker From arigo at codespeak.net Sun Nov 21 18:11:40 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 21 Nov 2010 18:11:40 +0100 (CET) Subject: [pypy-svn] r79318 - pypy/trunk/pypy/tool/release Message-ID: <20101121171140.93107282B9D@codespeak.net> Author: arigo Date: Sun Nov 21 18:11:39 2010 New Revision: 79318 Modified: pypy/trunk/pypy/tool/release/package.py Log: Improve the docstring. Modified: pypy/trunk/pypy/tool/release/package.py ============================================================================== --- pypy/trunk/pypy/tool/release/package.py (original) +++ pypy/trunk/pypy/tool/release/package.py Sun Nov 21 18:11:39 2010 @@ -1,8 +1,12 @@ #!/usr/bin/env python """ A sample script that packages PyPy, provided that it's already built. -Usage: +It uses 'pypy/translator/goal/pypy-c' and parts of the rest of the working +copy. Usage: -package.py pypydir [name-of-archive] [name-of-pypy-c] + package.py root-pypy-dir [name-of-archive] [name-of-pypy-c] + +Usually you would do: package.py ../../.. pypy-VER-PLATFORM. +The output is found in the directory /tmp/usession-YOURNAME/build/. """ import autopath From commits-noreply at bitbucket.org Sun Nov 21 18:32:00 2010 From: commits-noreply at bitbucket.org (commits-noreply at bitbucket.org) Date: Sun, 21 Nov 2010 11:32:00 -0600 (CST) Subject: [pypy-svn] jitviewer commit b47851b49b66: Finish static files dispatching correctly Message-ID: <20101121173200.425CC6C12B1@bitbucket03.managed.contegix.com> # HG changeset patch -- Bitbucket.org # Project jitviewer # URL http://bitbucket.org/pypy/jitviewer/overview # User Maciej Fijalkowski # Date 1290360709 -7200 # Node ID b47851b49b6644d3f8345729ecc2ab957f28f13f # Parent c8c87cffc932b2e6a8a613e7c244363aefd3136f Finish static files dispatching correctly --- a/bin/jitviewer.py +++ b/bin/jitviewer.py @@ -23,6 +23,11 @@ from pygments import highlight from pygments.lexers import PythonLexer from pygments.formatters import HtmlFormatter +from jinja2 import Environment, FileSystemLoader + +from werkzeug import Response +from flask.helpers import send_from_directory + CUTOFF = 30 class Server(object): @@ -49,7 +54,7 @@ class Server(object): if not all: loops = loops[:CUTOFF] return flask.render_template('index.html', loops=loops, - extra_data=extra_data) + extra_data=extra_data) def loop(self): no = int(flask.request.args.get('no', '0')) @@ -100,8 +105,14 @@ def start_browser(url): th.start() return th +class OverrideFlask(flask.Flask): + def __init__(self, *args, **kwargs): + self.root_path = kwargs.pop('root_path') + flask.Flask.__init__(self, *args, **kwargs) + def main(): - + PATH = os.path.join(os.path.dirname( + os.path.dirname(__file__))) if not '__pypy__' in sys.builtin_module_names: print "Please run it using pypy-c" sys.exit(1) @@ -114,7 +125,7 @@ def main(): loops = [parse(l) for l in extract_category(log, "jit-log-opt-")] parse_log_counts(open(sys.argv[1] + '.count').readlines(), loops) storage.reconnect_loops(loops) - app = flask.Flask(__name__) + app = OverrideFlask('__name__', root_path=PATH) server = Server(storage) app.debug = True app.route('/')(server.index) --- a/_jitviewer/source.py +++ /dev/null @@ -1,39 +0,0 @@ - -import re - -def f(): - i = 0 - while i < 1003: - i += 1 - -f() - -def inner(i): - return i + 1 - -def inlined_call(): - i = 0 - while i < 1003: - i = inner(i) - -inlined_call() - -def bridge(): - s = 0 - i = 0 - while i < 10000: - if i % 2: - s += 1 - else: - s += 2 - i += 1 - return s - -bridge() - -def inlined_str_stuff(): - s = [str(i) for i in range(3000)] - for elem in s: - re.search('3', elem) - -inlined_str_stuff() --- /dev/null +++ b/source.py @@ -0,0 +1,39 @@ + +import re + +def f(): + i = 0 + while i < 1003: + i += 1 + +f() + +def inner(i): + return i + 1 + +def inlined_call(): + i = 0 + while i < 1003: + i = inner(i) + +inlined_call() + +def bridge(): + s = 0 + i = 0 + while i < 10000: + if i % 2: + s += 1 + else: + s += 2 + i += 1 + return s + +bridge() + +def inlined_str_stuff(): + s = [str(i) for i in range(3000)] + for elem in s: + re.search('3', elem) + +inlined_str_stuff() From arigo at codespeak.net Sun Nov 21 19:05:46 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 21 Nov 2010 19:05:46 +0100 (CET) Subject: [pypy-svn] r79319 - pypy/trunk/pypy/module/sys Message-ID: <20101121180546.1A53E282B9D@codespeak.net> Author: arigo Date: Sun Nov 21 19:05:45 2010 New Revision: 79319 Modified: pypy/trunk/pypy/module/sys/state.py Log: Adds the path "lib-python/2.5.2/plat-SYS.PLATFORM" to sys.path too (but not plat-win32, as this does not exists, and on Darwin, also adds plat-mac and plat-mac/lib-scriptpackages). Modified: pypy/trunk/pypy/module/sys/state.py ============================================================================== --- pypy/trunk/pypy/module/sys/state.py (original) +++ pypy/trunk/pypy/module/sys/state.py Sun Nov 21 19:05:45 2010 @@ -33,6 +33,8 @@ raise OSError(errno.ENOTDIR, path) +platform = sys.platform + def getinitialpath(prefix): from pypy.module.sys.version import CPYTHON_VERSION dirname = '%d.%d.%d' % (CPYTHON_VERSION[0], @@ -51,6 +53,16 @@ importlist.append(lib_pypy) importlist.append(python_std_lib_modified) importlist.append(python_std_lib) + # + # List here the extra platform-specific paths. + if platform != 'win32': + importlist.append(os.path.join(python_std_lib, 'plat-'+platform)) + if platform == 'darwin': + importlist.append(os.path.join(python_std_lib, 'plat-mac')) + importlist.append(os.path.join(python_std_lib, + 'plat-mac', + 'lib-scriptpackages')) + # return importlist def pypy_initial_path(space, srcdir): From arigo at codespeak.net Sun Nov 21 19:12:22 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 21 Nov 2010 19:12:22 +0100 (CET) Subject: [pypy-svn] r79320 - pypy/release/1.4.x/lib-python/modified-2.5.2 Message-ID: <20101121181222.7DC6D282B9D@codespeak.net> Author: arigo Date: Sun Nov 21 19:12:21 2010 New Revision: 79320 Modified: pypy/release/1.4.x/lib-python/modified-2.5.2/site.py Log: A temporary workaround for the release branch. Modified: pypy/release/1.4.x/lib-python/modified-2.5.2/site.py ============================================================================== --- pypy/release/1.4.x/lib-python/modified-2.5.2/site.py (original) +++ pypy/release/1.4.x/lib-python/modified-2.5.2/site.py Sun Nov 21 19:12:21 2010 @@ -380,6 +380,10 @@ def main(): abs__file__() + # TMP + if sys.platform == "darwin": + sys.path.append(sys.path[-1] + "/plat-mac") + # /TMP paths_in_sys = removeduppaths() if (os.name == "posix" and sys.path and os.path.basename(sys.path[-1]) == "Modules"): From arigo at codespeak.net Sun Nov 21 19:32:08 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 21 Nov 2010 19:32:08 +0100 (CET) Subject: [pypy-svn] r79321 - pypy/release/1.4.x/pypy/tool/release Message-ID: <20101121183208.203A2282B9D@codespeak.net> Author: arigo Date: Sun Nov 21 19:32:06 2010 New Revision: 79321 Modified: pypy/release/1.4.x/pypy/tool/release/package.py Log: * Rename pypy-c to pypy by default. * Also put the site-packages directory. Modified: pypy/release/1.4.x/pypy/tool/release/package.py ============================================================================== --- pypy/release/1.4.x/pypy/tool/release/package.py (original) +++ pypy/release/1.4.x/pypy/tool/release/package.py Sun Nov 21 19:32:06 2010 @@ -32,7 +32,7 @@ class PyPyCNotFound(Exception): pass -def package(basedir, name='pypy-nightly', rename_pypy_c='pypy-c', +def package(basedir, name='pypy-nightly', rename_pypy_c='pypy', copy_to_dir = None, override_pypy_c = None): basedir = py.path.local(basedir) if sys.platform == 'win32': @@ -64,6 +64,10 @@ headers = includedir.listdir('*.h') + includedir.listdir('*.inl') for n in headers: shutil.copy(str(n), str(pypydir.join('include'))) + # + spdir = pypydir.ensure('site-packages', dir=True) + shutil.copy(str(basedir.join('site-packages', 'README')), str(spdir)) + # pypydir.ensure('bin', dir=True) archive_pypy_c = pypydir.join('bin', rename_pypy_c) shutil.copy(str(pypy_c), str(archive_pypy_c)) From arigo at codespeak.net Sun Nov 21 19:32:48 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Sun, 21 Nov 2010 19:32:48 +0100 (CET) Subject: [pypy-svn] r79322 - pypy/trunk/pypy/tool/release Message-ID: <20101121183248.053D6282B9D@codespeak.net> Author: arigo Date: Sun Nov 21 19:32:46 2010 New Revision: 79322 Modified: pypy/trunk/pypy/tool/release/package.py Log: Merge r79321. Modified: pypy/trunk/pypy/tool/release/package.py ============================================================================== --- pypy/trunk/pypy/tool/release/package.py (original) +++ pypy/trunk/pypy/tool/release/package.py Sun Nov 21 19:32:46 2010 @@ -36,7 +36,7 @@ class PyPyCNotFound(Exception): pass -def package(basedir, name='pypy-nightly', rename_pypy_c='pypy-c', +def package(basedir, name='pypy-nightly', rename_pypy_c='pypy', copy_to_dir = None, override_pypy_c = None): basedir = py.path.local(basedir) if sys.platform == 'win32': @@ -68,6 +68,10 @@ headers = includedir.listdir('*.h') + includedir.listdir('*.inl') for n in headers: shutil.copy(str(n), str(pypydir.join('include'))) + # + spdir = pypydir.ensure('site-packages', dir=True) + shutil.copy(str(basedir.join('site-packages', 'README')), str(spdir)) + # pypydir.ensure('bin', dir=True) archive_pypy_c = pypydir.join('bin', rename_pypy_c) shutil.copy(str(pypy_c), str(archive_pypy_c)) From hakanardo at codespeak.net Sun Nov 21 19:57:57 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Sun, 21 Nov 2010 19:57:57 +0100 (CET) Subject: [pypy-svn] r79323 - in pypy/branch/jit-unroll-loops/pypy/jit/metainterp: . optimizeopt test Message-ID: <20101121185757.5E887282B9D@codespeak.net> Author: hakanardo Date: Sun Nov 21 19:57:54 2010 New Revision: 79323 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/history.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/simple_optimize.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_greenfield.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py Log: When possible, create a short preamble as the operations in the preamble but not in the loop, making this short preamble followed by one iteration of the loop equivalent to the original preamble. The short preamble is stored in the loop_token and inlined at the end of each bridge, which then can jump to the loop directly instead of jumping to the preamble. When it was not possible to create such a short preamble the bridge will fall back to jumping to the original preamble. Bridges inherits virtuals from the loop and this approach now allows them to stay virtual and part of the inlined short preamble to be optimized away. However other loop invariant operations are currently not inherited which results in them being recalculated at the end of each guard. Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py Sun Nov 21 19:57:54 2010 @@ -541,7 +541,8 @@ op = new_loop.operations[-1] if not isinstance(target_loop_token, TerminatingLoopToken): # normal case - op.setdescr(target_loop_token) # patch the jump target + #op.setdescr(target_loop_token) # patch the jump target + pass else: # The target_loop_token is a pseudo loop token, # e.g. loop_tokens_done_with_this_frame_void[0] Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/history.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/history.py Sun Nov 21 19:57:54 2010 @@ -490,6 +490,9 @@ def _get_str(self): # for debugging only return self.constbox()._get_str() + def forget_value(self): + raise NotImplementedError + class BoxInt(Box): type = INT _attrs_ = ('value',) @@ -502,6 +505,9 @@ assert isinstance(value, Symbolic) self.value = value + def forget_value(self): + self.value = 0 + def clonebox(self): return BoxInt(self.value) @@ -537,6 +543,9 @@ assert isinstance(floatval, float) self.value = floatval + def forget_value(self): + self.value = 0.0 + def clonebox(self): return BoxFloat(self.value) @@ -569,6 +578,9 @@ assert lltype.typeOf(value) == llmemory.GCREF self.value = value + def forget_value(self): + self.value = lltype.nullptr(llmemory.GCREF.TO) + def clonebox(self): return BoxPtr(self.value) @@ -613,6 +625,9 @@ assert ootype.typeOf(value) is ootype.Object self.value = value + def forget_value(self): + self.value = ootype.NULL + def clonebox(self): return BoxObj(self.value) @@ -726,6 +741,7 @@ was compiled; but the LoopDescr remains alive and points to the generated assembler. """ + short_preamble = None terminating = False # see TerminatingLoopToken in compile.py outermost_jitdriver_sd = None # and more data specified by the backend when the loop is compiled Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimize.py Sun Nov 21 19:57:54 2010 @@ -37,6 +37,7 @@ bridge.operations[-1].setdescr(old_loop_token) # patch jump target optimize_bridge_1(metainterp_sd, bridge) return old_loop_tokens[0] + #return bridge.operations[-1].getdescr() return None # ____________________________________________________________ Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/__init__.py Sun Nov 21 19:57:54 2010 @@ -5,19 +5,21 @@ from pypy.jit.metainterp.optimizeopt.heap import OptHeap from pypy.jit.metainterp.optimizeopt.fficall import OptFfiCall from pypy.jit.metainterp.optimizeopt.string import OptString -from pypy.jit.metainterp.optimizeopt.unroll import optimize_unroll +from pypy.jit.metainterp.optimizeopt.unroll import optimize_unroll, OptInlineShortPreamble def optimize_loop_1(metainterp_sd, loop, unroll=True): """Optimize loop.operations to remove internal overheadish operations. """ opt_str = OptString() - optimizations = [OptIntBounds(), + optimizations = [OptInlineShortPreamble(), + OptIntBounds(), OptRewrite(), OptVirtualize(), opt_str, OptHeap(), OptFfiCall(), ] + if unroll: opt_str.enabled = False # FIXME: Workaround to disable string optimisation # during preamble but to keep it during the loop Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Sun Nov 21 19:57:54 2010 @@ -2,6 +2,7 @@ from pypy.jit.metainterp.resoperation import rop, ResOperation from pypy.jit.metainterp.compile import ResumeGuardDescr from pypy.jit.metainterp.resume import Snapshot +from pypy.jit.metainterp.history import TreeLoop, LoopToken # FXIME: Introduce some VirtualOptimizer super class instead @@ -17,9 +18,11 @@ def __init__(self, metainterp_sd, loop, optimizations): self.optimizer = Optimizer(metainterp_sd, loop, optimizations) self.cloned_operations = [] + self.originalop = {} for op in self.optimizer.loop.operations: - self.cloned_operations.append(op.clone()) - + newop = op.clone() + self.cloned_operations.append(newop) + self.originalop[newop] = op def propagate_all_forward(self): loop = self.optimizer.loop @@ -49,6 +52,52 @@ loop.operations = self.optimizer.newoperations + short = self.create_short_preamble(loop.preamble.operations, + loop.preamble.inputargs, + loop.operations, + loop.inputargs, + loop.token) + if short: + if False: + # FIXME: This should save some memory but requires + # a lot of tests to be fixed... + loop.preamble.operations = short + short_loop = loop.preamble + else: + short_loop = TreeLoop('short preamble') + short_loop.inputargs = loop.preamble.inputargs[:] + short_loop.operations = short + + assert isinstance(loop.preamble.token, LoopToken) + loop.preamble.token.short_preamble = short_loop + + # Clone ops and boxes to get private versions and forget the + # values to allow them to be freed + boxmap = {} + for i in range(len(short_loop.inputargs)): + box = short_loop.inputargs[i] + newbox = box.clonebox() + boxmap[box] = newbox + newbox.forget_value() + short_loop.inputargs[i] = newbox + for i in range(len(short)): + oldop = short[i] + op = oldop.clone() + args = [] + for a in op.getarglist(): + if not isinstance(a, Const): + a = boxmap[a] + args.append(a) + op.initarglist(args) + box = op.result + if box: + newbox = box.clonebox() + boxmap[box] = newbox + newbox.forget_value() + op.result = newbox + short[i] = op + + def inline(self, loop_operations, loop_args, jump_args): self.argmap = argmap = {} assert len(loop_args) == len(jump_args) @@ -71,17 +120,18 @@ # This loop is equivalent to the main optimization loop in # Optimizer.propagate_all_forward for newop in loop_operations: + #print 'N:', newop if newop.getopnum() == rop.JUMP: args = inputargs else: args = newop.getarglist() newop.initarglist([self.inline_arg(a) for a in args]) - #print 'P:', newop if newop.result: old_result = newop.result newop.result = newop.result.clonebox() argmap[old_result] = newop.result + #print 'P:', newop descr = newop.getdescr() if isinstance(descr, ResumeGuardDescr): @@ -131,3 +181,127 @@ new_snapshot = Snapshot(self.inline_snapshot(snapshot.prev), boxes) self.snapshot_map[snapshot] = new_snapshot return new_snapshot + + def sameop(self, preambleop, loopop): + #if preambleop.getopnum() != loopop.getopnum(): + # return False + #pargs = preambleop.getarglist() + #largs = loopop.getarglist() + #if len(pargs) != len(largs): + # return False + try: + return self.originalop[loopop] is preambleop + except KeyError: + return False + + def create_short_preamble(self, preamble, preambleargs, + loop, inputargs, token): + #return None # Dissable + + short_preamble = [] + loop_i = preamble_i = 0 + while loop_i < len(loop)-1 and preamble_i < len(preamble)-1: + if self.sameop(preamble[preamble_i], loop[loop_i]): + loop_i += 1 + preamble_i += 1 + else: + short_preamble.append(preamble[preamble_i]) + preamble_i += 1 + + if loop_i < len(loop)-1: + print "Loop contains ops not in preamble???" + return None + while preamble_i < len(preamble)-1: + short_preamble.append(preamble[preamble_i]) + preamble_i += 1 + + jumpargs = [None] * len(inputargs) + allboxes = preambleargs[:] + for op in short_preamble: + if op.result: + allboxes.append(op.result) + + for result in allboxes: + box = self.inline_arg(result) + for i in range(len(inputargs)): + b = inputargs[i] + if self.optimizer.getvalue(box) is self.optimizer.getvalue(b): + jumpargs[i] = result + break + + for a in jumpargs: + if a is None: + print "Unable to find all input arguments???" + return None + + jmp = ResOperation(rop.JUMP, jumpargs[:], None) + jmp.setdescr(token) + short_preamble.append(jmp) + + # Make sure it is safe to move the instrucions in short_preamble + # to the top making short_preamble followed by loop equvivalent + # to preamble + for op in short_preamble: + opnum = op.getopnum() + if (op.is_always_pure() or + opnum == rop.GETFIELD_GC or + opnum == rop.GETARRAYITEM_GC or + opnum == rop.JUMP): + continue + return None + # FIXME: Turn guards into conditional jumps to the preamble + + # Check that boxes used as arguemts are produced. Might not be + # needed, but let's play it safe. + seen = {} + for box in preambleargs: + seen[box] = True + for op in short_preamble: + for box in op.getarglist(): + if box not in seen: + print "Op arguments not produced???" + return None + if op.result: + seen[op.result] = True + + + return short_preamble + +class OptInlineShortPreamble(Optimization): + def reconstruct_for_next_iteration(self, optimizer, valuemap): + return self + + def propagate_forward(self, op): + if op.getopnum() == rop.JUMP: + descr = op.getdescr() + assert isinstance(descr, LoopToken) + short = descr.short_preamble + if short: + self.inline(short.operations, short.inputargs, op.getarglist()) + return + self.emit_operation(op) + + + + def inline(self, loop_operations, loop_args, jump_args): + self.argmap = argmap = {} + assert len(loop_args) == len(jump_args) + for i in range(len(loop_args)): + argmap[loop_args[i]] = jump_args[i] + + for op in loop_operations: + newop = op.clone() + args = newop.getarglist() + newop.initarglist([self.inline_arg(a) for a in args]) + + if newop.result: + old_result = newop.result + newop.result = newop.result.clonebox() + argmap[old_result] = newop.result + + self.emit_operation(newop) + + def inline_arg(self, arg): + if isinstance(arg, Const): + return arg + return self.argmap[arg] Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py Sun Nov 21 19:57:54 2010 @@ -1804,6 +1804,8 @@ old_loop_tokens = self.get_compiled_merge_points(greenkey) if len(old_loop_tokens) == 0: return + #if self.resumekey.guard_opnum == rop.GUARD_CLASS: + # return # Kepp tracing for another iteration self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None) target_loop_token = compile.compile_new_bridge(self, old_loop_tokens, self.resumekey) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/simple_optimize.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/simple_optimize.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/simple_optimize.py Sun Nov 21 19:57:54 2010 @@ -42,8 +42,14 @@ descr.store_final_boxes(op, newboxes) newoperations.extend(transform(op)) loop.operations = newoperations + jumpop = newoperations[-1] + if jumpop.getopnum() == rop.JUMP: + jumpop.setdescr(loop.token) return None def optimize_bridge(metainterp_sd, old_loops, loop): optimize_loop(metainterp_sd, [], loop) + jumpop = loop.operations[-1] + if jumpop.getopnum() == rop.JUMP: + jumpop.setdescr(old_loops[0]) return old_loops[0] Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py Sun Nov 21 19:57:54 2010 @@ -356,7 +356,7 @@ return res res = self.meta_interp(f, [6, 32]) assert res == 3427 - self.check_loop_count(2) + self.check_loop_count(3) def test_loop_invariant_mul_bridge_maintaining1(self): myjitdriver = JitDriver(greens = [], reds = ['y', 'res', 'x']) @@ -372,10 +372,11 @@ return res res = self.meta_interp(f, [6, 32]) assert res == 1167 - self.check_loop_count(2) - self.check_loops({'int_add': 1, 'int_lt': 1, - 'int_sub': 1, 'guard_false': 1, - 'jump': 1}) + self.check_loop_count(3) + self.check_loops({'int_add': 2, 'int_lt': 1, + 'int_sub': 2, 'guard_false': 1, + 'jump': 2, + 'int_gt': 1, 'guard_true': 1, 'int_mul': 1}) def test_loop_invariant_mul_bridge_maintaining2(self): @@ -387,18 +388,17 @@ myjitdriver.jit_merge_point(x=x, y=y, res=res) z = x * x res += z - if y<8: + if y<16: res += z y -= 1 return res - res = self.meta_interp(f, [6, 16]) - assert res == 828 - self.check_loop_count(2) - self.check_loops({'int_add': 1, 'int_lt': 1, - 'int_sub': 1, 'guard_false': 1, - 'jump': 1}) - - + res = self.meta_interp(f, [6, 32]) + assert res == 1692 + self.check_loop_count(3) + self.check_loops({'int_add': 2, 'int_lt': 1, + 'int_sub': 2, 'guard_false': 1, + 'jump': 2, + 'int_gt': 1, 'guard_true': 1, 'int_mul': 1}) def test_loop_invariant_intbox(self): myjitdriver = JitDriver(greens = [], reds = ['y', 'res', 'x']) @@ -450,7 +450,9 @@ assert res == f(6, 15) gc.collect() - assert not [wr for wr in wr_loops if wr()] + #assert not [wr for wr in wr_loops if wr()] + for loop in [wr for wr in wr_loops if wr()]: + assert loop().name == 'short preamble' def test_string(self): def f(n): @@ -1804,7 +1806,8 @@ assert res == 8 py.test.raises(AssertGreenFailed, self.interp_operations, f, [8, 0]) - def test_mutiple_specialied_versions(self): + def test_multiple_specialied_versions(self): + py.test.skip('Not supported yet') myjitdriver = JitDriver(greens = [], reds = ['y', 'x', 'res']) class Base: def __init__(self, val): Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_compile.py Sun Nov 21 19:57:54 2010 @@ -85,7 +85,7 @@ metainterp.history.inputargs = loop.inputargs[:] # loop_tokens = [] - loop_token = compile_new_loop(metainterp, loop_tokens, 0) + loop_token = compile_new_loop(metainterp, loop_tokens, [], 0) assert loop_tokens == [loop_token] assert loop_token.number == 1 assert staticdata.globaldata.loopnumbering == 2 @@ -101,7 +101,7 @@ metainterp.history.operations = loop.operations[:] metainterp.history.inputargs = loop.inputargs[:] # - loop_token_2 = compile_new_loop(metainterp, loop_tokens, 0) + loop_token_2 = compile_new_loop(metainterp, loop_tokens, [], 0) assert loop_token_2 is loop_token assert loop_tokens == [loop_token] assert len(cpu.seen) == 0 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_greenfield.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_greenfield.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_greenfield.py Sun Nov 21 19:57:54 2010 @@ -49,7 +49,7 @@ # res = self.meta_interp(g, [7]) assert res == -22 - self.check_loop_count(4) + self.check_loop_count(6) self.check_loops(guard_value=0) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_loop.py Sun Nov 21 19:57:54 2010 @@ -87,7 +87,10 @@ return res * 2 res = self.meta_interp(f, [6, 33], policy=StopAtXPolicy(l)) assert res == f(6, 33) - self.check_loop_count(2) + if self.optimizer == OPTIMIZER_FULL: + self.check_loop_count(3) + else: + self.check_loop_count(2) def test_alternating_loops(self): myjitdriver = JitDriver(greens = [], reds = ['pattern']) @@ -101,8 +104,11 @@ pass pattern >>= 1 return 42 - self.meta_interp(f, [0xF0F0]) - self.check_loop_count(2) + self.meta_interp(f, [0xF0F0F0]) + if self.optimizer == OPTIMIZER_FULL: + self.check_loop_count(3) + else: + self.check_loop_count(2) def test_interp_simple(self): myjitdriver = JitDriver(greens = ['i'], reds = ['x', 'y']) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_optimizeopt.py Sun Nov 21 19:57:54 2010 @@ -7,7 +7,8 @@ import pypy.jit.metainterp.optimizeopt.virtualize as virtualize from pypy.jit.metainterp.optimizeopt import optimize_loop_1 from pypy.jit.metainterp.optimizeutil import InvalidLoop -from pypy.jit.metainterp.history import AbstractDescr, ConstInt, BoxInt, TreeLoop +from pypy.jit.metainterp.history import AbstractDescr, ConstInt, BoxInt +from pypy.jit.metainterp.history import TreeLoop, LoopToken from pypy.jit.metainterp.jitprof import EmptyProfiler from pypy.jit.metainterp import executor, compile, resume, history from pypy.jit.metainterp.resoperation import rop, opname, ResOperation @@ -167,6 +168,7 @@ self.loop = loop loop.preamble = TreeLoop('preamble') loop.preamble.inputargs = loop.inputargs + loop.preamble.token = LoopToken() metainterp_sd = FakeMetaInterpStaticData(self.cpu) if hasattr(self, 'vrefinfo'): metainterp_sd.virtualref_info = self.vrefinfo Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_recursive.py Sun Nov 21 19:57:54 2010 @@ -1016,7 +1016,7 @@ res = self.meta_interp(portal, [2, 0], inline=True, policy=StopAtXPolicy(residual)) assert res == portal(2, 0) - self.check_loops(call_assembler=3, everywhere=True) + self.check_loops(call_assembler=4, everywhere=True) def test_inline_without_hitting_the_loop(self): driver = JitDriver(greens = ['codeno'], reds = ['i'], Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_send.py Sun Nov 21 19:57:54 2010 @@ -162,6 +162,7 @@ y -= 1 return 42 policy = StopAtXPolicy(externfn) + for j in range(69, 75): res = self.meta_interp(f, [j], policy=policy) assert res == 42 @@ -169,8 +170,8 @@ self.check_enter_count(3) self.check_loop_count(3) else: - self.check_enter_count(5) - self.check_loop_count(5) + self.check_enter_count_at_most(5) + self.check_loop_count_at_most(5) def test_oosend_guard_failure(self): myjitdriver = JitDriver(greens = [], reds = ['x', 'y', 'w']) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_virtual.py Sun Nov 21 19:57:54 2010 @@ -164,7 +164,7 @@ getfield_gc=0, setfield_gc=0) def test_two_loops_with_virtual(self): - py.test.skip("We don't know how to virtualize across bridges right now") + #py.test.skip("We don't know how to virtualize across bridges right now") myjitdriver = JitDriver(greens = [], reds = ['n', 'node']) def f(n): node = self._new() From hakanardo at codespeak.net Sun Nov 21 22:03:26 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Sun, 21 Nov 2010 22:03:26 +0100 (CET) Subject: [pypy-svn] r79324 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt Message-ID: <20101121210326.87626282B9D@codespeak.net> Author: hakanardo Date: Sun Nov 21 22:03:23 2010 New Revision: 79324 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Log: killed some debug prints Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Sun Nov 21 22:03:23 2010 @@ -209,7 +209,7 @@ preamble_i += 1 if loop_i < len(loop)-1: - print "Loop contains ops not in preamble???" + #print "Loop contains ops not in preamble???" return None while preamble_i < len(preamble)-1: short_preamble.append(preamble[preamble_i]) @@ -231,7 +231,7 @@ for a in jumpargs: if a is None: - print "Unable to find all input arguments???" + #print "Unable to find all input arguments???" return None jmp = ResOperation(rop.JUMP, jumpargs[:], None) @@ -259,7 +259,7 @@ for op in short_preamble: for box in op.getarglist(): if box not in seen: - print "Op arguments not produced???" + #print "Op arguments not produced???" return None if op.result: seen[op.result] = True From dan at codespeak.net Sun Nov 21 23:39:07 2010 From: dan at codespeak.net (dan at codespeak.net) Date: Sun, 21 Nov 2010 23:39:07 +0100 (CET) Subject: [pypy-svn] r79325 - pypy/branch/psycopg2compatibility Message-ID: <20101121223907.974BC282B9C@codespeak.net> Author: dan Date: Sun Nov 21 23:39:04 2010 New Revision: 79325 Added: pypy/branch/psycopg2compatibility/ (props changed) - copied from r79324, pypy/trunk/ Log: I don't like mucking about on trunk. From dan at codespeak.net Mon Nov 22 01:36:16 2010 From: dan at codespeak.net (dan at codespeak.net) Date: Mon, 22 Nov 2010 01:36:16 +0100 (CET) Subject: [pypy-svn] r79326 - in pypy/branch/psycopg2compatibility/pypy: module/cpyext module/cpyext/test tool Message-ID: <20101122003616.015C1282B9C@codespeak.net> Author: dan Date: Mon Nov 22 01:36:13 2010 New Revision: 79326 Added: pypy/branch/psycopg2compatibility/pypy/tool/call_logger.py Modified: pypy/branch/psycopg2compatibility/pypy/module/cpyext/api.py pypy/branch/psycopg2compatibility/pypy/module/cpyext/intobject.py pypy/branch/psycopg2compatibility/pypy/module/cpyext/pyobject.py pypy/branch/psycopg2compatibility/pypy/module/cpyext/pystate.py pypy/branch/psycopg2compatibility/pypy/module/cpyext/stubs.py pypy/branch/psycopg2compatibility/pypy/module/cpyext/test/test_pystate.py Log: Importing my changes from before. Modified: pypy/branch/psycopg2compatibility/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/psycopg2compatibility/pypy/module/cpyext/api.py (original) +++ pypy/branch/psycopg2compatibility/pypy/module/cpyext/api.py Mon Nov 22 01:36:13 2010 @@ -36,6 +36,42 @@ DEBUG_WRAPPER = True +if not we_are_translated() and DEBUG_WRAPPER: + from functools import wraps +else: + def wraps(f): return lambda f: f + +from pypy.tool.call_logger import CallLogger, SkipArgument + +class CPyExtCallLogger(CallLogger): + def format_arg(self, arg, args, kwargs, name=''): + from pypy.module.cpyext.object import PyObject_Repr + from pypy.rpython.lltypesystem.rffi import charp2str, unicode_from_buffer + from pypy.rpython.lltypesystem.lltype import direct_arrayitems + from pypy.module.cpyext.pyobject import from_ref, InvalidPointerException + from pypy.objspace.std.model import UnwrapError + + if isinstance(arg, ObjSpace): + raise SkipArgument() + + if is_PyObject(arg): + space = args[0] + return space.unwrap(PyObject_Repr(space, arg)) + + return repr(arg) + + def log(self, logstr, depth=0): + from sys import stderr + print >>stderr, (' ' * depth) + logstr + + def log_call(self, f): + if not we_are_translated() and DEBUG_WRAPPER: + return CallLogger.log_call(self, f) + else: + return f + +log_call = CPyExtCallLogger().log_call + # update these for other platforms Py_ssize_t = lltype.Signed Py_ssize_tP = rffi.CArrayPtr(Py_ssize_t) @@ -321,6 +357,8 @@ 'PyCObject_Type', 'init_pycobject', 'PyObject_AsReadBuffer', 'PyObject_AsWriteBuffer', 'PyObject_CheckReadBuffer', + 'PyThread_Get', + 'PyInterpreter_Head', 'PyInterpreter_Next', ] TYPES = {} GLOBALS = { # this needs to include all prebuilt pto, otherwise segfaults occur @@ -470,7 +508,10 @@ [name.startswith("w_") for name in names]))) fatal_value = callable.api_func.restype._defl() + logged_callable = log_call(callable) + @specialize.ll() + @wraps(callable) def wrapper(*args): from pypy.module.cpyext.pyobject import make_ref, from_ref from pypy.module.cpyext.pyobject import Reference @@ -481,8 +522,6 @@ retval = fatal_value boxed_args = () try: - if not we_are_translated() and DEBUG_WRAPPER: - print >>sys.stderr, callable, assert len(args) == len(callable.api_func.argtypes) for i, (typ, is_wrapped) in argtypes_enum_ui: arg = args[i] @@ -496,9 +535,7 @@ boxed_args += (arg_conv, ) state = space.fromcache(State) try: - result = callable(space, *boxed_args) - if not we_are_translated() and DEBUG_WRAPPER: - print >>sys.stderr, " DONE" + result = logged_callable(space, *boxed_args) except OperationError, e: failed = True state.set_exception(e) Modified: pypy/branch/psycopg2compatibility/pypy/module/cpyext/intobject.py ============================================================================== --- pypy/branch/psycopg2compatibility/pypy/module/cpyext/intobject.py (original) +++ pypy/branch/psycopg2compatibility/pypy/module/cpyext/intobject.py Mon Nov 22 01:36:13 2010 @@ -3,7 +3,6 @@ from pypy.module.cpyext.api import (cpython_api, PyObject, CANNOT_FAIL, build_type_checkers, Py_ssize_t) - PyInt_Check, PyInt_CheckExact = build_type_checkers("Int") @cpython_api([lltype.Signed], PyObject) @@ -49,3 +48,31 @@ """ return space.wrap(ival) # XXX this is wrong on win64 + at cpython_api([rffi.CCHARP, rffi.CCHARPP, rffi.INT_real], PyObject) +def PyInt_FromString(space, str, pend, base): + """Return a new PyIntObject or PyLongObject based on the string + value in str, which is interpreted according to the radix in base. If + pend is non-NULL, *pend will point to the first character in str which + follows the representation of the number. If base is 0, the radix will be + determined based on the leading characters of str: if str starts with + '0x' or '0X', radix 16 will be used; if str starts with '0', radix + 8 will be used; otherwise radix 10 will be used. If base is not 0, it + must be between 2 and 36, inclusive. Leading spaces are ignored. If + there are no digits, ValueError will be raised. If the string represents + a number too large to be contained within the machine's long int type + and overflow warnings are being suppressed, a PyLongObject will be + returned. If overflow warnings are not being suppressed, NULL will be + returned in this case.""" + # TODO: test! + if pend: + raise NotImplementedError + len = pend - str # FIXME: can sub pointers? right function name? + str = rffi.charp2strn(str, len) + else: + str = rffi.charp2str(str) + + base = int(base) + + value = int(str, base) + + return space.wrap(value) Modified: pypy/branch/psycopg2compatibility/pypy/module/cpyext/pyobject.py ============================================================================== --- pypy/branch/psycopg2compatibility/pypy/module/cpyext/pyobject.py (original) +++ pypy/branch/psycopg2compatibility/pypy/module/cpyext/pyobject.py Mon Nov 22 01:36:13 2010 @@ -171,7 +171,7 @@ lifeline = self.lifeline_dict.get(w_obj) if lifeline is not None: # make old PyObject ready for use in C code py_obj = lifeline.pyo - assert py_obj.c_ob_refcnt == 0 + assert py_obj.c_ob_refcnt == 0, "%r refcount %d" % (py_obj, py_obj.c_ob_refcnt) return py_obj else: return lltype.nullptr(PyObject.TO) Modified: pypy/branch/psycopg2compatibility/pypy/module/cpyext/pystate.py ============================================================================== --- pypy/branch/psycopg2compatibility/pypy/module/cpyext/pystate.py (original) +++ pypy/branch/psycopg2compatibility/pypy/module/cpyext/pystate.py Mon Nov 22 01:36:13 2010 @@ -3,8 +3,8 @@ from pypy.rpython.lltypesystem import rffi, lltype -PyThreadState = lltype.Ptr(cpython_struct("PyThreadState", ())) PyInterpreterState = lltype.Ptr(cpython_struct("PyInterpreterState", ())) +PyThreadState = lltype.Ptr(cpython_struct("PyThreadState", [('interp', PyInterpreterState)])) @cpython_api([], PyThreadState, error=CANNOT_FAIL) def PyEval_SaveThread(space): @@ -35,4 +35,55 @@ def PyEval_ThreadsInitialized(space): return 1 +# XXX: might be generally useful +def encapsulator(T, flavor='raw'): + class MemoryCapsule(object): + def __init__(self): + self.memory = lltype.malloc(T, flavor=flavor) + def __del__(self): + lltype.free(self.memory, flavor=flavor) + return MemoryCapsule + +ThreadStateCapsule = encapsulator(PyThreadState.TO) + +class InterpreterState(object): + def __init__(self, space): + self.interpreter_state = lltype.malloc(PyInterpreterState.TO, flavor='raw', immortal=True) + + def new_thread_state(self): + capsule = ThreadStateCapsule() + ts = capsule.memory + ts.c_interp = self.interpreter_state + return capsule + + def get_thread_state(self, space): + ec = space.getexecutioncontext() + return self._get_thread_state(ec).memory + + def _get_thread_state(self, ec): + try: + ts = ec.cpyext_threadstate + if not ts: + ec.cpyext_threadstate = self.new_thread_state() + except AttributeError, e: + ec.cpyext_threadstate = self.new_thread_state() + return ec.cpyext_threadstate + + at cpython_api([], PyThreadState, error=CANNOT_FAIL) +def PyThreadState_Get(space, ): + state = space.fromcache(InterpreterState) + return state.get_thread_state(space) + + at cpython_api([], PyInterpreterState, error=CANNOT_FAIL) +def PyInterpreterState_Head(space, ): + """Return the interpreter state object at the head of the list of all such objects. + """ + return space.fromcache(InterpreterState).interpreter_state + + at cpython_api([PyInterpreterState], PyInterpreterState, error=CANNOT_FAIL) +def PyInterpreterState_Next(space, interp): + """Return the next interpreter state object after interp from the list of all + such objects. + """ + return lltype.nullptr(PyInterpreterState.TO) Modified: pypy/branch/psycopg2compatibility/pypy/module/cpyext/stubs.py ============================================================================== --- pypy/branch/psycopg2compatibility/pypy/module/cpyext/stubs.py (original) +++ pypy/branch/psycopg2compatibility/pypy/module/cpyext/stubs.py Mon Nov 22 01:36:13 2010 @@ -1679,13 +1679,6 @@ PyInterpreterState_Clear().""" raise NotImplementedError - at cpython_api([], PyThreadState) -def PyThreadState_Get(space, ): - """Return the current thread state. The global interpreter lock must be held. - When the current thread state is NULL, this issues a fatal error (so that - the caller needn't check for NULL).""" - raise NotImplementedError - @cpython_api([], PyObject) def PyThreadState_GetDict(space, ): """Return a dictionary in which extensions can store thread-specific state @@ -1822,19 +1815,6 @@ defined.""" raise NotImplementedError - at cpython_api([], PyInterpreterState) -def PyInterpreterState_Head(space, ): - """Return the interpreter state object at the head of the list of all such objects. - """ - raise NotImplementedError - - at cpython_api([PyInterpreterState], PyInterpreterState) -def PyInterpreterState_Next(space, interp): - """Return the next interpreter state object after interp from the list of all - such objects. - """ - raise NotImplementedError - @cpython_api([PyInterpreterState], PyThreadState) def PyInterpreterState_ThreadHead(space, interp): """Return the a pointer to the first PyThreadState object in the list of @@ -1849,23 +1829,6 @@ """ raise NotImplementedError - at cpython_api([rffi.CCHARP, rffi.CCHARPP, rffi.INT_real], PyObject) -def PyInt_FromString(space, str, pend, base): - """Return a new PyIntObject or PyLongObject based on the string - value in str, which is interpreted according to the radix in base. If - pend is non-NULL, *pend will point to the first character in str which - follows the representation of the number. If base is 0, the radix will be - determined based on the leading characters of str: if str starts with - '0x' or '0X', radix 16 will be used; if str starts with '0', radix - 8 will be used; otherwise radix 10 will be used. If base is not 0, it - must be between 2 and 36, inclusive. Leading spaces are ignored. If - there are no digits, ValueError will be raised. If the string represents - a number too large to be contained within the machine's long int type - and overflow warnings are being suppressed, a PyLongObject will be - returned. If overflow warnings are not being suppressed, NULL will be - returned in this case.""" - raise NotImplementedError - @cpython_api([rffi.SIZE_T], PyObject) def PyInt_FromSize_t(space, ival): """Create a new integer object with a value of ival. If the value exceeds Modified: pypy/branch/psycopg2compatibility/pypy/module/cpyext/test/test_pystate.py ============================================================================== --- pypy/branch/psycopg2compatibility/pypy/module/cpyext/test/test_pystate.py (original) +++ pypy/branch/psycopg2compatibility/pypy/module/cpyext/test/test_pystate.py Mon Nov 22 01:36:13 2010 @@ -1,4 +1,7 @@ from pypy.module.cpyext.test.test_cpyext import AppTestCpythonExtensionBase +from pypy.module.cpyext.test.test_api import BaseApiTest +from pypy.rpython.lltypesystem.lltype import nullptr +from pypy.module.cpyext.pystate import PyInterpreterState, PyThreadState class AppTestThreads(AppTestCpythonExtensionBase): def test_allow_threads(self): @@ -17,3 +20,26 @@ # Should compile at least module.test() +class TestInterpreterState(BaseApiTest): + def test_interpreter_head(self, space, api): + state = api.PyInterpreterState_Head() + assert state != nullptr(PyInterpreterState.TO) + + def test_interpreter_next(self, space, api): + state = api.PyInterpreterState_Head() + assert nullptr(PyInterpreterState.TO) == api.PyInterpreterState_Next(state) + +def clear_threadstate(space): + # XXX: this should collect the ThreadState memory + del space.getexecutioncontext().cpyext_threadstate + +class TestThreadState(BaseApiTest): + def test_thread_state_get(self, space, api): + ts = api.PyThreadState_Get() + assert ts != nullptr(PyThreadState.TO) + clear_threadstate(space) + + def test_thread_state_interp(self, space, api): + ts = api.PyThreadState_Get() + assert ts.c_interp == api.PyInterpreterState_Head() + clear_threadstate(space) Added: pypy/branch/psycopg2compatibility/pypy/tool/call_logger.py ============================================================================== --- (empty file) +++ pypy/branch/psycopg2compatibility/pypy/tool/call_logger.py Mon Nov 22 01:36:13 2010 @@ -0,0 +1,47 @@ +from functools import wraps + +class SkipArgument(Exception): pass + +class CallLogger(object): + def __init__(self): + self.indentation = 0 + + def format_arg(self, arg, args=[], kwargs={}, name=''): + return repr(arg) + + def argstrs(self, args, kwargs): + argstrs = [] + for arg in args: + try: + argstrs.append(self.format_arg(arg, args, kwargs)) + except SkipArgument, e: continue + argstr = ', '.join(argstrs) + + kwargstrs = [] + for name, arg in kwargs.iteritems(): + try: + kwargstrs.append("%s=%s" % (name, self.format_arg(arg, args, kwargs, name=name))) + except SkipArgument, e: continue + kwargstr = ', '.join(kwargstrs) + + if argstr and kwargstr: + return ', '.join([argstr, kwargstr]) + elif argstr: + return argstr + else: + return kwargstr + + def log(self, logstr, depth=0): + print (' ' * depth) + logstr + + def log_call(self, f): + @wraps(f) + def wrapped(*args, **kwargs): + argstr = self.argstrs(args, kwargs) + self.log("%s(%s)" % (f.__name__, argstr), depth=self.indentation) + self.indentation += 1 + result = f(*args, **kwargs) + self.indentation -= 1 + self.log("%s(%s)->%r" % (f.__name__, argstr, result), depth=self.indentation) + return result + return wrapped From fijal at codespeak.net Mon Nov 22 08:05:51 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 22 Nov 2010 08:05:51 +0100 (CET) Subject: [pypy-svn] r79327 - pypy/build/bot2/pypybuildbot Message-ID: <20101122070551.40155282B9D@codespeak.net> Author: fijal Date: Mon Nov 22 08:05:48 2010 New Revision: 79327 Modified: pypy/build/bot2/pypybuildbot/pypylist.py Log: Properly escape URLs Modified: pypy/build/bot2/pypybuildbot/pypylist.py ============================================================================== --- pypy/build/bot2/pypybuildbot/pypylist.py (original) +++ pypy/build/bot2/pypybuildbot/pypylist.py Mon Nov 22 08:05:48 2010 @@ -3,6 +3,7 @@ import itertools import re import py +import cgi from twisted.web import resource from twisted.web.static import File, DirectoryLister @@ -189,7 +190,7 @@ if branch == 'trunk': branch = '%3Ctrunk%3E' # if category: - href = '/summary?category=%s&branch=%s&recentrev=%s' % (category, branch, rev) + href = cgi.escape('/summary?category=%s&branch=%s&recentrev=%s' % (category, branch, rev)) str_summary = '%s' % (href, summary) else: str_summary = str(summary) From fijal at codespeak.net Mon Nov 22 08:49:16 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 22 Nov 2010 08:49:16 +0100 (CET) Subject: [pypy-svn] r79328 - pypy/trunk/pypy/tool/release Message-ID: <20101122074916.EBC1D5080F@codespeak.net> Author: fijal Date: Mon Nov 22 08:49:14 2010 New Revision: 79328 Modified: pypy/trunk/pypy/tool/release/make_release.py Log: Fix for new html Modified: pypy/trunk/pypy/tool/release/make_release.py ============================================================================== --- pypy/trunk/pypy/tool/release/make_release.py (original) +++ pypy/trunk/pypy/tool/release/make_release.py Mon Nov 22 08:49:14 2010 @@ -30,7 +30,8 @@ else: xml = override_xml dom = minidom.parseString(xml) - refs = [node.getAttribute('href') for node in dom.getElementsByTagName('a')] + refs = [node.getAttribute('href') for node in dom.getElementsByTagName('a') + if 'pypy' in node.getAttribute('href')] # all refs are of form: pypy-{type}-{revision}-{platform}.tar.bz2 r = re.compile('pypy-c-([\w\d]+)-(\d+)-([\w\d]+).tar.bz2$') d = {} From dan at codespeak.net Mon Nov 22 09:21:20 2010 From: dan at codespeak.net (dan at codespeak.net) Date: Mon, 22 Nov 2010 09:21:20 +0100 (CET) Subject: [pypy-svn] r79329 - pypy/branch/psycopg2compatibility/pypy/module/cpyext Message-ID: <20101122082120.B3A9C5080F@codespeak.net> Author: dan Date: Mon Nov 22 09:21:19 2010 New Revision: 79329 Modified: pypy/branch/psycopg2compatibility/pypy/module/cpyext/api.py Log: Trying to RPython-ize CPyExtCallLogger Modified: pypy/branch/psycopg2compatibility/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/psycopg2compatibility/pypy/module/cpyext/api.py (original) +++ pypy/branch/psycopg2compatibility/pypy/module/cpyext/api.py Mon Nov 22 09:21:19 2010 @@ -33,6 +33,7 @@ from py.builtin import BaseException from pypy.tool.sourcetools import func_with_new_name from pypy.rpython.lltypesystem.lloperation import llop +from pypy.rlib.debug import debug_print DEBUG_WRAPPER = True @@ -42,35 +43,66 @@ def wraps(f): return lambda f: f from pypy.tool.call_logger import CallLogger, SkipArgument +from pypy.rlib.objectmodel import specialize -class CPyExtCallLogger(CallLogger): - def format_arg(self, arg, args, kwargs, name=''): - from pypy.module.cpyext.object import PyObject_Repr +class CPyExtCallLogger(object): + def __init__(self): + self.indentation = 0 + + @specialize.argtype(2) + def format_arg(self, space, arg, argtype): from pypy.rpython.lltypesystem.rffi import charp2str, unicode_from_buffer from pypy.rpython.lltypesystem.lltype import direct_arrayitems from pypy.module.cpyext.pyobject import from_ref, InvalidPointerException from pypy.objspace.std.model import UnwrapError + from pypy.module.cpyext.object import PyObject_Repr if isinstance(arg, ObjSpace): raise SkipArgument() - if is_PyObject(arg): - space = args[0] - return space.unwrap(PyObject_Repr(space, arg)) + if is_PyObject(argtype): + return space.str_w(PyObject_Repr(space, arg)) return repr(arg) def log(self, logstr, depth=0): - from sys import stderr - print >>stderr, (' ' * depth) + logstr + debug_print((' ' * depth) + logstr) - def log_call(self, f): + def log_cpyext_call(self, f): if not we_are_translated() and DEBUG_WRAPPER: - return CallLogger.log_call(self, f) + api_function = f.api_func + name = f.__name__ + argtypes = api_function.argtypes + result_type = api_function.restype + def format_args(space, args): + argstrs = [] + for i in unrolling_iterable(range(len(argtypes))): + try: + argtype = argtypes[i] + arg = args[i] + argstrs.append(self.format_arg(space, arg, argtype)) + except SkipArgument, e: continue + return ', '.join(argstrs) + + @wraps(f) + def wrapped(*args): + space = args[0] + argstr = format_args(space, args[1:]) + self.log("%s(%s)" % (name, argstr), depth=self.indentation) + self.indentation += 1 + result = f(*args) + self.indentation -= 1 + try: + resultformat = self.format_arg(result, space, result_type) + except SkipArgument: + resultformat = '' + self.log("%s(%s)->%s" % (name, argstr, resultformat), depth=self.indentation) + return result + return wrapped else: return f -log_call = CPyExtCallLogger().log_call +log_call = CPyExtCallLogger().log_cpyext_call # update these for other platforms Py_ssize_t = lltype.Signed From hakanardo at codespeak.net Mon Nov 22 09:40:24 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Mon, 22 Nov 2010 09:40:24 +0100 (CET) Subject: [pypy-svn] r79330 - in pypy/branch/jit-unroll-loops/pypy/jit/metainterp: optimizeopt test Message-ID: <20101122084024.77D4D282B9D@codespeak.net> Author: hakanardo Date: Mon Nov 22 09:40:23 2010 New Revision: 79330 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_greenfield.py Log: beeing more carefull about moving getitem ops Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Mon Nov 22 09:40:23 2010 @@ -197,22 +197,31 @@ def create_short_preamble(self, preamble, preambleargs, loop, inputargs, token): #return None # Dissable - + + state = ExeState() short_preamble = [] loop_i = preamble_i = 0 while loop_i < len(loop)-1 and preamble_i < len(preamble)-1: if self.sameop(preamble[preamble_i], loop[loop_i]): loop_i += 1 - preamble_i += 1 else: + if not state.safe_to_move(preamble[preamble_i]): + #print "Unsafe heap operation." + return None short_preamble.append(preamble[preamble_i]) - preamble_i += 1 + state.update(preamble[preamble_i]) + preamble_i += 1 + if loop_i < len(loop)-1: #print "Loop contains ops not in preamble???" return None while preamble_i < len(preamble)-1: + if not state.safe_to_move(preamble[preamble_i]): + #print "Unsafe heap operation." + return None short_preamble.append(preamble[preamble_i]) + state.update(preamble[preamble_i]) preamble_i += 1 jumpargs = [None] * len(inputargs) @@ -238,17 +247,6 @@ jmp.setdescr(token) short_preamble.append(jmp) - # Make sure it is safe to move the instrucions in short_preamble - # to the top making short_preamble followed by loop equvivalent - # to preamble - for op in short_preamble: - opnum = op.getopnum() - if (op.is_always_pure() or - opnum == rop.GETFIELD_GC or - opnum == rop.GETARRAYITEM_GC or - opnum == rop.JUMP): - continue - return None # FIXME: Turn guards into conditional jumps to the preamble # Check that boxes used as arguemts are produced. Might not be @@ -267,6 +265,43 @@ return short_preamble +class ExeState(object): + def __init__(self): + self.heap_dirty = False + self.unsafe_getitem = {} + + # Make sure it is safe to move the instrucions in short_preamble + # to the top making short_preamble followed by loop equvivalent + # to preamble + def safe_to_move(self, op): + opnum = op.getopnum() + if op.is_always_pure(): + return True + elif opnum == rop.JUMP: + return True + elif (opnum == rop.GETFIELD_GC or + opnum == rop.GETFIELD_RAW): + if self.heap_dirty: + return False + descr = op.getdescr() + if descr in self.unsafe_getitem: + return False + return True + return False + + def update(self, op): + if (op.has_no_side_effect() or + op.is_ovf() or + op.is_guard()): + return + opnum = op.getopnum() + if (opnum == rop.SETFIELD_GC or + opnum == rop.SETFIELD_RAW): + descr = op.getdescr() + self.unsafe_getitem[descr] = True + return + self.heap_dirty = True + class OptInlineShortPreamble(Optimization): def reconstruct_for_next_iteration(self, optimizer, valuemap): return self Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_greenfield.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_greenfield.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_greenfield.py Mon Nov 22 09:40:23 2010 @@ -49,7 +49,7 @@ # res = self.meta_interp(g, [7]) assert res == -22 - self.check_loop_count(6) + self.check_loop_count(4) self.check_loops(guard_value=0) From afa at codespeak.net Mon Nov 22 10:42:20 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 22 Nov 2010 10:42:20 +0100 (CET) Subject: [pypy-svn] r79331 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101122094220.B2C4B5080F@codespeak.net> Author: afa Date: Mon Nov 22 10:42:18 2010 New Revision: 79331 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Log: Test and fix Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Mon Nov 22 10:42:18 2010 @@ -350,7 +350,7 @@ return u"" available = len(self.decoded_chars) - self.decoded_chars_used - if available < 0 or size > available: + if size < 0 or size > available: size = available if self.decoded_chars_used > 0 or size < available: Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Mon Nov 22 10:42:18 2010 @@ -69,6 +69,14 @@ reads.append(c) assert u''.join(reads) == u"abc\ndef\ng" + def test_read_some_then_all(self): + import _io + r = _io.BytesIO("abc\ndef\n") + t = _io.TextIOWrapper(r) + reads = t.read(4) + reads += t.read() + assert reads == u"abc\ndef\n" + class AppTestIncrementalNewlineDecoder: def test_newline_decoder(self): From arigo at codespeak.net Mon Nov 22 11:49:50 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 11:49:50 +0100 (CET) Subject: [pypy-svn] r79332 - pypy/trunk/pypy/module/sys/test Message-ID: <20101122104950.83E00282BD6@codespeak.net> Author: arigo Date: Mon Nov 22 11:49:48 2010 New Revision: 79332 Modified: pypy/trunk/pypy/module/sys/test/test_initialpath.py Log: Fix test. Modified: pypy/trunk/pypy/module/sys/test/test_initialpath.py ============================================================================== --- pypy/trunk/pypy/module/sys/test/test_initialpath.py (original) +++ pypy/trunk/pypy/module/sys/test/test_initialpath.py Mon Nov 22 11:49:48 2010 @@ -15,4 +15,5 @@ def test_stdlib_in_prefix(tmpdir): dirs = build_hierarchy(tmpdir) path = getinitialpath(str(tmpdir)) - assert path == map(str, dirs) + # we get at least 'dirs', and maybe more (e.g. plat-linux2) + assert path[:len(dirs)] == map(str, dirs) From arigo at codespeak.net Mon Nov 22 11:52:35 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 11:52:35 +0100 (CET) Subject: [pypy-svn] r79333 - pypy/trunk/pypy/tool/release/test Message-ID: <20101122105235.7AA505080F@codespeak.net> Author: arigo Date: Mon Nov 22 11:52:33 2010 New Revision: 79333 Modified: pypy/trunk/pypy/tool/release/test/test_package.py Log: Fix test. Modified: pypy/trunk/pypy/tool/release/test/test_package.py ============================================================================== --- pypy/trunk/pypy/tool/release/test/test_package.py (original) +++ pypy/trunk/pypy/tool/release/test/test_package.py Mon Nov 22 11:52:33 2010 @@ -18,7 +18,7 @@ prefix = builddir.join(test) cpyver = '%d.%d.%d' % CPYTHON_VERSION[:3] assert prefix.join('lib-python', cpyver, 'test').check() - assert prefix.join('bin', 'pypy-c').check() + assert prefix.join('bin', 'pypy').check() assert prefix.join('lib_pypy', 'syslog.py').check() assert not prefix.join('lib_pypy', 'py').check() assert not prefix.join('lib_pypy', 'ctypes_configure').check() From arigo at codespeak.net Mon Nov 22 11:55:04 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 11:55:04 +0100 (CET) Subject: [pypy-svn] r79334 - pypy/trunk/pypy/translator/goal/test2 Message-ID: <20101122105504.57E37282BF1@codespeak.net> Author: arigo Date: Mon Nov 22 11:55:02 2010 New Revision: 79334 Modified: pypy/trunk/pypy/translator/goal/test2/test_app_main.py Log: Fix tests. Modified: pypy/trunk/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/trunk/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/trunk/pypy/translator/goal/test2/test_app_main.py Mon Nov 22 11:55:02 2010 @@ -534,7 +534,8 @@ newpath = app_main.get_library_path('/tmp/pypy-c') # stdlib not found assert newpath == sys.path newpath = app_main.get_library_path(self.fake_exe) - assert newpath == self.expected_path + # we get at least 'expected_path', and maybe more (e.g.plat-linux2) + assert newpath[:len(self.expected_path)] == self.expected_path finally: sys.path.pop() @@ -547,7 +548,9 @@ app_main.os = os pypy_c = os.path.join(self.trunkdir, 'pypy', 'translator', 'goal', 'pypy-c') newpath = app_main.get_library_path(pypy_c) - assert len(newpath) == 3 + # we get at least lib_pypy, lib-python/modified-X.Y.Z, + # lib-python/X.Y.Z, and maybe more (e.g. plat-linux2) + assert len(newpath) >= 3 for p in newpath: assert p.startswith(self.trunkdir) finally: From arigo at codespeak.net Mon Nov 22 11:58:02 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 11:58:02 +0100 (CET) Subject: [pypy-svn] r79335 - pypy/branch/jit-free/pypy/translator Message-ID: <20101122105802.7D39450811@codespeak.net> Author: arigo Date: Mon Nov 22 11:58:01 2010 New Revision: 79335 Modified: pypy/branch/jit-free/pypy/translator/driver.py Log: * Rename gc-collect-task into translation-task. * A (commented out) way to dump the heap after each step, provided we are running on pypy. Modified: pypy/branch/jit-free/pypy/translator/driver.py ============================================================================== --- pypy/branch/jit-free/pypy/translator/driver.py (original) +++ pypy/branch/jit-free/pypy/translator/driver.py Mon Nov 22 11:58:01 2010 @@ -288,7 +288,7 @@ return else: self.log.info("%s..." % title) - debug_print_once('gc-collect-task', 'starting', goal) + debug_print_once('translation-task', 'starting', goal) self.timer.start_event(goal) try: instrument = False @@ -311,6 +311,7 @@ raise except: pass + #import gc; gc.dump_rpy_heap('rpyheap-after-%s.dump' % goal) return res def task_annotate(self): From arigo at codespeak.net Mon Nov 22 13:47:55 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 13:47:55 +0100 (CET) Subject: [pypy-svn] r79337 - pypy/trunk/pypy/module/sys Message-ID: <20101122124755.4EDE9282BF6@codespeak.net> Author: arigo Date: Mon Nov 22 13:47:53 2010 New Revision: 79337 Modified: pypy/trunk/pypy/module/sys/state.py Log: Translation fix on OS/X. Modified: pypy/trunk/pypy/module/sys/state.py ============================================================================== --- pypy/trunk/pypy/module/sys/state.py (original) +++ pypy/trunk/pypy/module/sys/state.py Mon Nov 22 13:47:53 2010 @@ -58,10 +58,9 @@ if platform != 'win32': importlist.append(os.path.join(python_std_lib, 'plat-'+platform)) if platform == 'darwin': - importlist.append(os.path.join(python_std_lib, 'plat-mac')) - importlist.append(os.path.join(python_std_lib, - 'plat-mac', - 'lib-scriptpackages')) + platmac = os.path.join(python_std_lib, 'plat-mac') + importlist.append(platmac) + importlist.append(os.path.join(platmac, 'lib-scriptpackages')) # return importlist From cfbolz at codespeak.net Mon Nov 22 14:01:32 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 22 Nov 2010 14:01:32 +0100 (CET) Subject: [pypy-svn] r79338 - in pypy/trunk/pypy/interpreter: . test Message-ID: <20101122130132.2A16B282BF6@codespeak.net> Author: cfbolz Date: Mon Nov 22 14:01:30 2010 New Revision: 79338 Modified: pypy/trunk/pypy/interpreter/argument.py pypy/trunk/pypy/interpreter/test/test_argument.py Log: fix test in module/_stackless. Modified: pypy/trunk/pypy/interpreter/argument.py ============================================================================== --- pypy/trunk/pypy/interpreter/argument.py (original) +++ pypy/trunk/pypy/interpreter/argument.py Mon Nov 22 14:01:30 2010 @@ -402,9 +402,10 @@ space = self.space w_args = space.newtuple(self.arguments_w) w_kwds = space.newdict() - for i in range(len(self.keywords)): - space.setitem(w_kwds, space.wrap(self.keywords[i]), - self.keywords_w[i]) + if self.keywords is not None: + for i in range(len(self.keywords)): + space.setitem(w_kwds, space.wrap(self.keywords[i]), + self.keywords_w[i]) return w_args, w_kwds class ArgumentsForTranslation(Arguments): Modified: pypy/trunk/pypy/interpreter/test/test_argument.py ============================================================================== --- pypy/trunk/pypy/interpreter/test/test_argument.py (original) +++ pypy/trunk/pypy/interpreter/test/test_argument.py Mon Nov 22 14:01:30 2010 @@ -453,7 +453,11 @@ assert set(args.keywords) == set(['a', 'b']) assert args.keywords_w[args.keywords.index('a')] == 2 assert args.keywords_w[args.keywords.index('b')] == 3 - + + args = Arguments(space, [1]) + w_args, w_kwds = args.topacked() + assert w_args == (1, ) + assert not w_kwds class TestErrorHandling(object): def test_missing_args(self): From fijal at codespeak.net Mon Nov 22 14:09:22 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 22 Nov 2010 14:09:22 +0100 (CET) Subject: [pypy-svn] r79340 - pypy/trunk/pypy/doc Message-ID: <20101122130922.6B823282BFB@codespeak.net> Author: fijal Date: Mon Nov 22 14:09:20 2010 New Revision: 79340 Added: pypy/trunk/pypy/doc/release-1.4.0beta.txt Log: Write down a draft of 1.4beta release Added: pypy/trunk/pypy/doc/release-1.4.0beta.txt ============================================================================== --- (empty file) +++ pypy/trunk/pypy/doc/release-1.4.0beta.txt Mon Nov 22 14:09:20 2010 @@ -0,0 +1,25 @@ +==================================== +PyPy 1.4beta - towards 1.4.0 release +==================================== + +Hello. + +As we head towards 1.4 release, which should be considered the very first PyPy +release ready to substitute CPython for at least some of us, we're pleased to +announce 1.4 beta release. + +This release contains all major features from upcoming 1.4, the only thing +missing being improved memory footprint. + +Highlights: + +* x86_64 JIT backend + +* rewritten fast and jitted regular expressions + +* improvements all across the board (for example faster map calls) + +* virtualenv support (virtualenv 0.5 or later) + +Cheers, +The PyPy team From fijal at codespeak.net Mon Nov 22 14:10:18 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 22 Nov 2010 14:10:18 +0100 (CET) Subject: [pypy-svn] r79341 - pypy/trunk/pypy/doc Message-ID: <20101122131018.3A0A250811@codespeak.net> Author: fijal Date: Mon Nov 22 14:10:16 2010 New Revision: 79341 Modified: pypy/trunk/pypy/doc/release-1.4.0beta.txt Log: Disclaimer Modified: pypy/trunk/pypy/doc/release-1.4.0beta.txt ============================================================================== --- pypy/trunk/pypy/doc/release-1.4.0beta.txt (original) +++ pypy/trunk/pypy/doc/release-1.4.0beta.txt Mon Nov 22 14:10:16 2010 @@ -11,6 +11,8 @@ This release contains all major features from upcoming 1.4, the only thing missing being improved memory footprint. +However, this is a beta release and might still contain some issues. + Highlights: * x86_64 JIT backend From fijal at codespeak.net Mon Nov 22 14:19:47 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 22 Nov 2010 14:19:47 +0100 (CET) Subject: [pypy-svn] r79342 - pypy/trunk/pypy/doc Message-ID: <20101122131947.EEAB6282BFC@codespeak.net> Author: fijal Date: Mon Nov 22 14:19:40 2010 New Revision: 79342 Modified: pypy/trunk/pypy/doc/release-1.4.0beta.txt Log: mention debugging msgs Modified: pypy/trunk/pypy/doc/release-1.4.0beta.txt ============================================================================== --- pypy/trunk/pypy/doc/release-1.4.0beta.txt (original) +++ pypy/trunk/pypy/doc/release-1.4.0beta.txt Mon Nov 22 14:19:40 2010 @@ -11,7 +11,9 @@ This release contains all major features from upcoming 1.4, the only thing missing being improved memory footprint. -However, this is a beta release and might still contain some issues. +However, this is a beta release and might still contain some issues. One of +those issues is that, like on nightly builds, pypy might print some debugging +output at the end of your program run. Highlights: From fijal at codespeak.net Mon Nov 22 14:21:25 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 22 Nov 2010 14:21:25 +0100 (CET) Subject: [pypy-svn] r79343 - pypy/trunk/pypy/doc Message-ID: <20101122132125.6C82050813@codespeak.net> Author: fijal Date: Mon Nov 22 14:21:23 2010 New Revision: 79343 Modified: pypy/trunk/pypy/doc/release-1.4.0beta.txt Log: mention cpyext Modified: pypy/trunk/pypy/doc/release-1.4.0beta.txt ============================================================================== --- pypy/trunk/pypy/doc/release-1.4.0beta.txt (original) +++ pypy/trunk/pypy/doc/release-1.4.0beta.txt Mon Nov 22 14:21:23 2010 @@ -19,6 +19,9 @@ * x86_64 JIT backend +* since PyPy 1.3 we have an experimental support for CPython C extensions. + Those has to be recompiled using `pypy setup.py build` + * rewritten fast and jitted regular expressions * improvements all across the board (for example faster map calls) From fijal at codespeak.net Mon Nov 22 14:23:32 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 22 Nov 2010 14:23:32 +0100 (CET) Subject: [pypy-svn] r79344 - pypy/trunk/pypy/doc Message-ID: <20101122132332.794F5282BFD@codespeak.net> Author: fijal Date: Mon Nov 22 14:23:30 2010 New Revision: 79344 Modified: pypy/trunk/pypy/doc/release-1.4.0beta.txt Log: point to patches Modified: pypy/trunk/pypy/doc/release-1.4.0beta.txt ============================================================================== --- pypy/trunk/pypy/doc/release-1.4.0beta.txt (original) +++ pypy/trunk/pypy/doc/release-1.4.0beta.txt Mon Nov 22 14:23:30 2010 @@ -20,7 +20,9 @@ * x86_64 JIT backend * since PyPy 1.3 we have an experimental support for CPython C extensions. - Those has to be recompiled using `pypy setup.py build` + Those has to be recompiled using `pypy setup.py build`. Extensions usually + have to be tweaked for e.g. refcounting bugs that don't manifest on CPython. + There is a `list of patches`_ available for some extensions. * rewritten fast and jitted regular expressions @@ -30,3 +32,5 @@ Cheers, The PyPy team + +.. _`list of patches`: http://codespeak.net/svn/pypy/trunk/pypy/module/cpyext/patches/ From antocuni at codespeak.net Mon Nov 22 15:09:23 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Mon, 22 Nov 2010 15:09:23 +0100 (CET) Subject: [pypy-svn] r79345 - pypy/trunk/pypy/doc Message-ID: <20101122140923.4872F282BF9@codespeak.net> Author: antocuni Date: Mon Nov 22 15:09:20 2010 New Revision: 79345 Modified: pypy/trunk/pypy/doc/release-1.4.0beta.txt Log: fix virtualenv version Modified: pypy/trunk/pypy/doc/release-1.4.0beta.txt ============================================================================== --- pypy/trunk/pypy/doc/release-1.4.0beta.txt (original) +++ pypy/trunk/pypy/doc/release-1.4.0beta.txt Mon Nov 22 15:09:20 2010 @@ -28,7 +28,7 @@ * improvements all across the board (for example faster map calls) -* virtualenv support (virtualenv 0.5 or later) +* virtualenv support (virtualenv 1.5 or later) Cheers, The PyPy team From agaynor at codespeak.net Mon Nov 22 15:26:03 2010 From: agaynor at codespeak.net (agaynor at codespeak.net) Date: Mon, 22 Nov 2010 15:26:03 +0100 (CET) Subject: [pypy-svn] r79346 - in pypy/release/1.4.x: . pypy pypy/jit/metainterp/optimizeopt pypy/module/array/benchmark pypy/module/array/test pypy/rlib pypy/rlib/test pypy/translator/goal pypy/translator/goal/test2 Message-ID: <20101122142603.21566282BFB@codespeak.net> Author: agaynor Date: Mon Nov 22 15:26:02 2010 New Revision: 79346 Modified: pypy/release/1.4.x/ (props changed) pypy/release/1.4.x/pypy/ (props changed) pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/Makefile (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimg.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/loop.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sum.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.c (props changed) pypy/release/1.4.x/pypy/module/array/benchmark/sumtst.py (props changed) pypy/release/1.4.x/pypy/module/array/test/test_array_old.py (props changed) pypy/release/1.4.x/pypy/rlib/rerased.py (props changed) pypy/release/1.4.x/pypy/rlib/test/test_rerased.py (props changed) pypy/release/1.4.x/pypy/translator/goal/app_main.py pypy/release/1.4.x/pypy/translator/goal/test2/test_app_main.py Log: Merged 79161 from trunk. Modified: pypy/release/1.4.x/pypy/translator/goal/app_main.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/goal/app_main.py (original) +++ pypy/release/1.4.x/pypy/translator/goal/app_main.py Mon Nov 22 15:26:02 2010 @@ -362,6 +362,9 @@ try: if run_command: # handle the "-c" command + # Put '' on sys.path + sys.path.insert(0, '') + def run_it(): exec cmd in mainmodule.__dict__ success = run_toplevel(run_it) Modified: pypy/release/1.4.x/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/release/1.4.x/pypy/translator/goal/test2/test_app_main.py Mon Nov 22 15:26:02 2010 @@ -496,6 +496,9 @@ assert repr(str(tmpdir.join('otherpath'))) in data assert "''" not in data + data = self.run('-c "import sys; print sys.path"') + assert data.startswith("[''") + class AppTestAppMain: From arigo at codespeak.net Mon Nov 22 16:28:54 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 16:28:54 +0100 (CET) Subject: [pypy-svn] r79347 - in pypy/extradoc: planning sprintinfo/post-ep2008 talk/bern2007 talk/ep2006 talk/roadshow-ibm talk/roadshow-ibm/ui/default Message-ID: <20101122152854.638ED5080F@codespeak.net> Author: arigo Date: Mon Nov 22 16:28:51 2010 New Revision: 79347 Modified: pypy/extradoc/planning/jit.txt pypy/extradoc/sprintinfo/post-ep2008/planning.txt pypy/extradoc/talk/bern2007/rpython-overview.txt pypy/extradoc/talk/ep2006/jit_lightningtalk.py pypy/extradoc/talk/roadshow-ibm/overview2.png (props changed) pypy/extradoc/talk/roadshow-ibm/ui/default/pretty.css Log: Fix an issue, open another one. Modified: pypy/extradoc/planning/jit.txt ============================================================================== --- pypy/extradoc/planning/jit.txt (original) +++ pypy/extradoc/planning/jit.txt Mon Nov 22 16:28:51 2010 @@ -89,8 +89,8 @@ - guard_true(frame.is_being_profiled) all over the place -- 64-bit issues: in test_pypy_c, test_array_intimg, the STORE_SUBSCR turns - into strange code (truncate the int to 32 bits). +- "op = getattr(llop, spaceop.opname)" at app-level ends up in a long chain + of calls to 'll_streq__rpy_stringPtr_rpy_stringPtr' - foo(*args) generates sub-optimal code even if foo takes a fixed number of arguments, because we always allocate the Arguments class in that case Modified: pypy/extradoc/sprintinfo/post-ep2008/planning.txt ============================================================================== --- pypy/extradoc/sprintinfo/post-ep2008/planning.txt (original) +++ pypy/extradoc/sprintinfo/post-ep2008/planning.txt Mon Nov 22 16:28:51 2010 @@ -19,7 +19,7 @@ * started http://code.djangoproject.com/wiki/DjangoAndPyPy - support for profiling the calls to builtins with - sys.setprofile(): TESTING + sys.setprofile(): DONE - fix the default options of "translate.py", make "--help" show only the reasonable options, remove "--faassen", Modified: pypy/extradoc/talk/bern2007/rpython-overview.txt ============================================================================== --- pypy/extradoc/talk/bern2007/rpython-overview.txt (original) +++ pypy/extradoc/talk/bern2007/rpython-overview.txt Mon Nov 22 16:28:51 2010 @@ -86,9 +86,9 @@ Numerical Types =============== - - Python ``int``s are C ``long``s + - Python ``int`` are C ``long`` - have kind of the obvious C/Java-like behaviour - - difference between Python and RPython: RPython ``int``s wrap around silently + - difference between Python and RPython: RPython ``int`` wrap around silently Strings Modified: pypy/extradoc/talk/ep2006/jit_lightningtalk.py ============================================================================== --- pypy/extradoc/talk/ep2006/jit_lightningtalk.py (original) +++ pypy/extradoc/talk/ep2006/jit_lightningtalk.py Mon Nov 22 16:28:51 2010 @@ -195,9 +195,9 @@ timings = [] print '%-20s %12s %12s' %('', '', 'magnitudes') print '%-20s %12s %12s' %('', 'us/call', 'speedup') - for name, fn, repeat in (('Python interpreter', f, 1000), - ('RPython interpreter', fc, 100000), - ('Compiled', fjc, 10000000)): - timings.append(measure(fn, 41, repeat)) + for name, fn, repeat in (#('Python interpreter', f, 1000), + #('RPython interpreter', fc, 100000), + ('Compiled', fjc, 1)): + timings.append(measure(fn, 99999999, repeat)) print '%-20s: %12f %12f'%(name, timings[-1]*10**6, math.log10(timings[0] / timings[-1])) Modified: pypy/extradoc/talk/roadshow-ibm/ui/default/pretty.css ============================================================================== --- pypy/extradoc/talk/roadshow-ibm/ui/default/pretty.css (original) +++ pypy/extradoc/talk/roadshow-ibm/ui/default/pretty.css Mon Nov 22 16:28:51 2010 @@ -79,7 +79,7 @@ .huge pre, .big pre, .small pre, .tiny pre {font-size: 115%;} .maroon {color: maroon;} -.red {color: red;} +.red {color: #F88;} .magenta {color: magenta;} .fuchsia {color: fuchsia;} .pink {color: #FAA;} From afa at codespeak.net Mon Nov 22 17:29:44 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 22 Nov 2010 17:29:44 +0100 (CET) Subject: [pypy-svn] r79348 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101122162944.4D01D282B9D@codespeak.net> Author: afa Date: Mon Nov 22 17:29:41 2010 New Revision: 79348 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Log: Looong code to implement TextIOWrapper.readline() Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Mon Nov 22 17:29:41 2010 @@ -235,6 +235,10 @@ self.decoded_chars_used = 0 # offset into _decoded_chars for read() self.chunk_size = 8192 + self.readuniversal = False + self.readtranslate = False + self.readnl = None + @unwrap_spec('self', ObjSpace, W_Root, W_Root, W_Root, W_Root, int) def descr_init(self, space, w_buffer, w_encoding=None, w_errors=None, w_newline=None, line_buffering=0): @@ -269,16 +273,18 @@ if space.is_w(w_newline, space.w_None): newline = None else: - newline = space.str_w(w_newline) - if newline and newline not in ('\n', '\r\n', '\r'): + newline = space.unicode_w(w_newline) + if newline and newline not in (u'\n', u'\r\n', u'\r'): + r = space.str_w(space.repr(w_newline)) raise OperationError(space.w_ValueError, space.wrap( - "illegal newline value: %s" % (newline,))) + "illegal newline value: %s" % (r,))) self.line_buffering = line_buffering + self.readuniversal = not newline # null or empty + self.readtranslate = newline is None + self.readnl = newline # XXX self.writenl - readuniversal = not newline # null or empty - readtranslate = newline is None # build the decoder object if space.is_true(space.call_method(w_buffer, "readable")): @@ -286,10 +292,10 @@ space.str_w(self.w_encoding)) self.w_decoder = space.call_method(w_codec, "incrementaldecoder", w_errors) - if readuniversal: + if self.readuniversal: self.w_decoder = space.call_function( space.gettypeobject(W_IncrementalNewlineDecoder.typedef), - self.w_decoder, space.wrap(readtranslate)) + self.w_decoder, space.wrap(self.readtranslate)) self.state = STATE_OK @@ -352,6 +358,7 @@ available = len(self.decoded_chars) - self.decoded_chars_used if size < 0 or size > available: size = available + assert size >= 0 if self.decoded_chars_used > 0 or size < available: start = self.decoded_chars_used @@ -427,12 +434,137 @@ return space.wrap(builder.build()) + def _find_line_ending(self, line, start): + end = len(line) + size = end - start + if self.readtranslate: + + # Newlines are already translated, only search for \n + pos = line.find(u'\n', start, end) + if pos >= 0: + return pos - start + 1, 0 + else: + return -1, size + elif self.readuniversal: + # Universal newline search. Find any of \r, \r\n, \n + # The decoder ensures that \r\n are not split in two pieces + i = 0 + while True: + # Fast path for non-control chars. The loop always ends + # since the Py_UNICODE storage is NUL-terminated. + while line[start + i] > '\r': + i += 1 + if i >= size: + return -1, size + ch = line[start + i] + i += 1 + if ch == '\n': + return i, 0 + if ch == '\r': + if line[start + i] == '\n': + return i + 1, 0 + else: + return i, 0 + else: + # Non-universal mode. + pos = line.find(self.readnl, start, end) + if pos >= 0: + return pos - start + len(self.readnl), 0 + else: + restart = end - len(self.readnl) + if restart >= 0: + pos = line.find(self.readnl[0], restart, end) + if pos >= 0: + return -1, pos - start + return -1, size + @unwrap_spec('self', ObjSpace, W_Root) def readline_w(self, space, w_limit=None): - self._check_init(space) - # XXX w_limit? - w_bytes = space.call_method(self.w_buffer, "readline") - return space.call_method(self.w_decoder, "decode", w_bytes) + self._check_closed(space) + # XXX self._writeflush(space) + + limit = convert_size(space, w_limit) + chunked = 0 + + line = None + remaining = None + chunks = [] + + while True: + # First, get some data if necessary + has_data = True + while not self.decoded_chars: + if not self._read_chunk(space): + has_data = False + break + if not has_data: + # end of file + self._set_decoded_chars(None) + self.snapshot = None + start = endpos = offset_to_buffer = 0 + break + + if not remaining: + line = self.decoded_chars + start = self.decoded_chars_used + offset_to_buffer = 0 + else: + assert self.decoded_chars_used == 0 + line = remaining + self.decoded_chars + start = 0 + offset_to_buffer = len(remaining) + remaining = None + + line_len = len(line) + endpos, consumed = self._find_line_ending(line, start) + if endpos >= 0: + endpos += start + if limit >= 0 and endpos >= start + limit - chunked: + endpos = start + limit - chunked + assert endpos >= 0 + break + assert consumed >= 0 + + # We can put aside up to `endpos` + endpos = consumed + start + if limit >= 0 and endpos >= start + limit - chunked: + # Didn't find line ending, but reached length limit + endpos = start + limit - chunked + assert endpos >= 0 + break + + # No line ending seen yet - put aside current data + if endpos > start: + s = line[start:endpos] + chunks.append(s) + chunked += len(s) + # There may be some remaining bytes we'll have to prepend to the + # next chunk of data + if endpos < line_len: + remaining = line[endpos:] + line = None + # We have consumed the buffer + self._set_decoded_chars(None) + + if line: + # Our line ends in the current buffer + decoded_chars_used = endpos - offset_to_buffer + assert decoded_chars_used >= 0 + self.decoded_chars_used = decoded_chars_used + if start > 0 or endpos < len(line): + line = line[start:endpos] + if remaining: + chunks.append(remaining) + remaining = None + if chunks: + if line: + chunks.append(line) + line = u''.join(chunks) + + if line: + return space.wrap(line) + else: + return space.wrap(u'') @unwrap_spec('self', ObjSpace) def detach_w(self, space): Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Mon Nov 22 17:29:41 2010 @@ -77,6 +77,14 @@ reads += t.read() assert reads == u"abc\ndef\n" + def test_read_some_then_readline(self): + import _io + r = _io.BytesIO("abc\ndef\n") + t = _io.TextIOWrapper(r) + reads = t.read(4) + reads += t.readline() + assert reads == u"abc\ndef\n" + class AppTestIncrementalNewlineDecoder: def test_newline_decoder(self): From arigo at codespeak.net Mon Nov 22 17:48:23 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 17:48:23 +0100 (CET) Subject: [pypy-svn] r79349 - pypy/trunk/pypy/jit/metainterp/test Message-ID: <20101122164823.CFFEF5080F@codespeak.net> Author: arigo Date: Mon Nov 22 17:48:22 2010 New Revision: 79349 Modified: pypy/trunk/pypy/jit/metainterp/test/test_exception.py Log: (antocuni, arigo) Phew. Found out the origin of the bug we were tracking. Modified: pypy/trunk/pypy/jit/metainterp/test/test_exception.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_exception.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_exception.py Mon Nov 22 17:48:22 2010 @@ -1,6 +1,6 @@ import py, sys from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin -from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE +from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE, dont_look_inside from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask from pypy.jit.codewriter.policy import StopAtXPolicy @@ -588,6 +588,33 @@ res = self.interp_operations(f, [99]) assert res == 21 + def test_bug_exc1_noexc_exc2(self): + myjitdriver = JitDriver(greens=[], reds=['i']) + @dont_look_inside + def rescall(i): + if i < 10: + raise KeyError + if i < 20: + return None + raise ValueError + def f(i): + while i < 30: + myjitdriver.can_enter_jit(i=i) + myjitdriver.jit_merge_point(i=i) + try: + rescall(i) + except KeyError: + assert i < 10 + except ValueError: + assert i >= 20 + else: + assert 10 <= i < 20 + i += 1 + return i + res = self.meta_interp(f, [0], inline=True) + assert res == 30 + + class MyError(Exception): def __init__(self, n): self.n = n From arigo at codespeak.net Mon Nov 22 17:54:28 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 17:54:28 +0100 (CET) Subject: [pypy-svn] r79350 - in pypy/trunk/pypy/jit/metainterp: optimizeopt test Message-ID: <20101122165428.99319282BD4@codespeak.net> Author: arigo Date: Mon Nov 22 17:54:26 2010 New Revision: 79350 Modified: pypy/trunk/pypy/jit/metainterp/optimizeopt/optimizer.py pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py Log: (antocuni, arigo) Add another test. Fix for r79349. Modified: pypy/trunk/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/trunk/pypy/jit/metainterp/optimizeopt/optimizer.py Mon Nov 22 17:54:26 2010 @@ -299,7 +299,9 @@ return CVAL_ZERO def propagate_all_forward(self): - self.exception_might_have_happened = False + self.exception_might_have_happened = True + # ^^^ at least at the start of bridges. For loops, we could set + # it to False, but we probably don't care self.newoperations = [] self.i = 0 while self.i < len(self.loop.operations): Modified: pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py Mon Nov 22 17:54:26 2010 @@ -789,8 +789,12 @@ i3 = call(i2, descr=nonwritedescr) jump(i1) # the exception is considered lost when we loop back """ + # note that 'guard_no_exception' at the very start must be kept + # around: bridges may start with one. (In case of loops we could + # remove it, but we probably don't care.) expected = """ [i] + guard_no_exception() [] i1 = int_add(i, 3) i2 = call(i1, descr=nonwritedescr) guard_no_exception() [i1, i2] From arigo at codespeak.net Mon Nov 22 17:58:17 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 17:58:17 +0100 (CET) Subject: [pypy-svn] r79351 - in pypy/branch/jit-free/pypy/jit/metainterp: optimizeopt test Message-ID: <20101122165817.69A1C50810@codespeak.net> Author: arigo Date: Mon Nov 22 17:58:15 2010 New Revision: 79351 Modified: pypy/branch/jit-free/pypy/jit/metainterp/optimizeopt/optimizer.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_exception.py pypy/branch/jit-free/pypy/jit/metainterp/test/test_optimizeopt.py Log: Merge r79349-r79350 from trunk. Modified: pypy/branch/jit-free/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/optimizeopt/optimizer.py Mon Nov 22 17:58:15 2010 @@ -299,7 +299,9 @@ return CVAL_ZERO def propagate_all_forward(self): - self.exception_might_have_happened = False + self.exception_might_have_happened = True + # ^^^ at least at the start of bridges. For loops, we could set + # it to False, but we probably don't care self.newoperations = [] self.i = 0 while self.i < len(self.loop.operations): Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_exception.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_exception.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_exception.py Mon Nov 22 17:58:15 2010 @@ -1,6 +1,6 @@ import py, sys from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin -from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE +from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE, dont_look_inside from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask from pypy.jit.codewriter.policy import StopAtXPolicy @@ -588,6 +588,33 @@ res = self.interp_operations(f, [99]) assert res == 21 + def test_bug_exc1_noexc_exc2(self): + myjitdriver = JitDriver(greens=[], reds=['i']) + @dont_look_inside + def rescall(i): + if i < 10: + raise KeyError + if i < 20: + return None + raise ValueError + def f(i): + while i < 30: + myjitdriver.can_enter_jit(i=i) + myjitdriver.jit_merge_point(i=i) + try: + rescall(i) + except KeyError: + assert i < 10 + except ValueError: + assert i >= 20 + else: + assert 10 <= i < 20 + i += 1 + return i + res = self.meta_interp(f, [0], inline=True) + assert res == 30 + + class MyError(Exception): def __init__(self, n): self.n = n Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_optimizeopt.py Mon Nov 22 17:58:15 2010 @@ -789,8 +789,12 @@ i3 = call(i2, descr=nonwritedescr) jump(i1) # the exception is considered lost when we loop back """ + # note that 'guard_no_exception' at the very start must be kept + # around: bridges may start with one. (In case of loops we could + # remove it, but we probably don't care.) expected = """ [i] + guard_no_exception() [] i1 = int_add(i, 3) i2 = call(i1, descr=nonwritedescr) guard_no_exception() [i1, i2] From arigo at codespeak.net Mon Nov 22 17:59:11 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 17:59:11 +0100 (CET) Subject: [pypy-svn] r79352 - pypy/branch/jit-free/pypy/jit/backend/llgraph Message-ID: <20101122165911.7AC8D282BF1@codespeak.net> Author: arigo Date: Mon Nov 22 17:59:09 2010 New Revision: 79352 Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py Log: Fix bug (op.fail_args can be None to mean []). Modified: pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/llgraph/llimpl.py Mon Nov 22 17:59:09 2010 @@ -449,7 +449,10 @@ _stats.exec_conditional_jumps += 1 if op.jump_target is not None: # a patched guard, pointing to further code - args = [self.getenv(v) for v in op.fail_args if v] + if op.fail_args: + args = [self.getenv(v) for v in op.fail_args if v] + else: + args = [] assert len(op.jump_target.inputargs) == len(args) self.env = dict(zip(op.jump_target.inputargs, args)) self.loop = op.jump_target From arigo at codespeak.net Mon Nov 22 18:06:35 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 18:06:35 +0100 (CET) Subject: [pypy-svn] r79353 - in pypy/release/1.4.x/pypy/jit/metainterp: optimizeopt test Message-ID: <20101122170635.7E86250811@codespeak.net> Author: arigo Date: Mon Nov 22 18:06:33 2010 New Revision: 79353 Modified: pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py pypy/release/1.4.x/pypy/jit/metainterp/test/test_exception.py pypy/release/1.4.x/pypy/jit/metainterp/test/test_optimizeopt.py Log: Merge r79349-r79350 from trunk. Modified: pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/release/1.4.x/pypy/jit/metainterp/optimizeopt/optimizer.py Mon Nov 22 18:06:33 2010 @@ -299,7 +299,9 @@ return CVAL_ZERO def propagate_all_forward(self): - self.exception_might_have_happened = False + self.exception_might_have_happened = True + # ^^^ at least at the start of bridges. For loops, we could set + # it to False, but we probably don't care self.newoperations = [] self.i = 0 while self.i < len(self.loop.operations): Modified: pypy/release/1.4.x/pypy/jit/metainterp/test/test_exception.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/metainterp/test/test_exception.py (original) +++ pypy/release/1.4.x/pypy/jit/metainterp/test/test_exception.py Mon Nov 22 18:06:33 2010 @@ -1,6 +1,6 @@ import py, sys from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin -from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE +from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE, dont_look_inside from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask from pypy.jit.codewriter.policy import StopAtXPolicy @@ -588,6 +588,33 @@ res = self.interp_operations(f, [99]) assert res == 21 + def test_bug_exc1_noexc_exc2(self): + myjitdriver = JitDriver(greens=[], reds=['i']) + @dont_look_inside + def rescall(i): + if i < 10: + raise KeyError + if i < 20: + return None + raise ValueError + def f(i): + while i < 30: + myjitdriver.can_enter_jit(i=i) + myjitdriver.jit_merge_point(i=i) + try: + rescall(i) + except KeyError: + assert i < 10 + except ValueError: + assert i >= 20 + else: + assert 10 <= i < 20 + i += 1 + return i + res = self.meta_interp(f, [0], inline=True) + assert res == 30 + + class MyError(Exception): def __init__(self, n): self.n = n Modified: pypy/release/1.4.x/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/release/1.4.x/pypy/jit/metainterp/test/test_optimizeopt.py Mon Nov 22 18:06:33 2010 @@ -789,8 +789,12 @@ i3 = call(i2, descr=nonwritedescr) jump(i1) # the exception is considered lost when we loop back """ + # note that 'guard_no_exception' at the very start must be kept + # around: bridges may start with one. (In case of loops we could + # remove it, but we probably don't care.) expected = """ [i] + guard_no_exception() [] i1 = int_add(i, 3) i2 = call(i1, descr=nonwritedescr) guard_no_exception() [i1, i2] From arigo at codespeak.net Mon Nov 22 18:12:54 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 18:12:54 +0100 (CET) Subject: [pypy-svn] r79354 - in pypy/release/1.4.x: lib-python/modified-2.5.2 pypy/module/sys pypy/module/sys/test Message-ID: <20101122171254.4556E282BD6@codespeak.net> Author: arigo Date: Mon Nov 22 18:12:52 2010 New Revision: 79354 Modified: pypy/release/1.4.x/lib-python/modified-2.5.2/site.py pypy/release/1.4.x/pypy/module/sys/state.py pypy/release/1.4.x/pypy/module/sys/test/test_initialpath.py Log: Revert the temporary fix in r79320, and add the proper fixes from trunk (r79319, r79332, r79337). Modified: pypy/release/1.4.x/lib-python/modified-2.5.2/site.py ============================================================================== --- pypy/release/1.4.x/lib-python/modified-2.5.2/site.py (original) +++ pypy/release/1.4.x/lib-python/modified-2.5.2/site.py Mon Nov 22 18:12:52 2010 @@ -380,10 +380,6 @@ def main(): abs__file__() - # TMP - if sys.platform == "darwin": - sys.path.append(sys.path[-1] + "/plat-mac") - # /TMP paths_in_sys = removeduppaths() if (os.name == "posix" and sys.path and os.path.basename(sys.path[-1]) == "Modules"): Modified: pypy/release/1.4.x/pypy/module/sys/state.py ============================================================================== --- pypy/release/1.4.x/pypy/module/sys/state.py (original) +++ pypy/release/1.4.x/pypy/module/sys/state.py Mon Nov 22 18:12:52 2010 @@ -33,6 +33,8 @@ raise OSError(errno.ENOTDIR, path) +platform = sys.platform + def getinitialpath(prefix): from pypy.module.sys.version import CPYTHON_VERSION dirname = '%d.%d.%d' % (CPYTHON_VERSION[0], @@ -51,6 +53,15 @@ importlist.append(lib_pypy) importlist.append(python_std_lib_modified) importlist.append(python_std_lib) + # + # List here the extra platform-specific paths. + if platform != 'win32': + importlist.append(os.path.join(python_std_lib, 'plat-'+platform)) + if platform == 'darwin': + platmac = os.path.join(python_std_lib, 'plat-mac') + importlist.append(platmac) + importlist.append(os.path.join(platmac, 'lib-scriptpackages')) + # return importlist def pypy_initial_path(space, srcdir): Modified: pypy/release/1.4.x/pypy/module/sys/test/test_initialpath.py ============================================================================== --- pypy/release/1.4.x/pypy/module/sys/test/test_initialpath.py (original) +++ pypy/release/1.4.x/pypy/module/sys/test/test_initialpath.py Mon Nov 22 18:12:52 2010 @@ -15,4 +15,5 @@ def test_stdlib_in_prefix(tmpdir): dirs = build_hierarchy(tmpdir) path = getinitialpath(str(tmpdir)) - assert path == map(str, dirs) + # we get at least 'dirs', and maybe more (e.g. plat-linux2) + assert path[:len(dirs)] == map(str, dirs) From arigo at codespeak.net Mon Nov 22 18:14:52 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 18:14:52 +0100 (CET) Subject: [pypy-svn] r79355 - pypy/trunk/pypy/doc Message-ID: <20101122171452.260BB50812@codespeak.net> Author: arigo Date: Mon Nov 22 18:14:50 2010 New Revision: 79355 Modified: pypy/trunk/pypy/doc/release-1.4.0beta.txt Log: Typo. Modified: pypy/trunk/pypy/doc/release-1.4.0beta.txt ============================================================================== --- pypy/trunk/pypy/doc/release-1.4.0beta.txt (original) +++ pypy/trunk/pypy/doc/release-1.4.0beta.txt Mon Nov 22 18:14:50 2010 @@ -20,7 +20,7 @@ * x86_64 JIT backend * since PyPy 1.3 we have an experimental support for CPython C extensions. - Those has to be recompiled using `pypy setup.py build`. Extensions usually + Those have to be recompiled using `pypy setup.py build`. Extensions usually have to be tweaked for e.g. refcounting bugs that don't manifest on CPython. There is a `list of patches`_ available for some extensions. From antocuni at codespeak.net Mon Nov 22 19:00:57 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Mon, 22 Nov 2010 19:00:57 +0100 (CET) Subject: [pypy-svn] r79356 - pypy/trunk/pypy/jit/tool Message-ID: <20101122180057.3AF4F282BD4@codespeak.net> Author: antocuni Date: Mon Nov 22 19:00:55 2010 New Revision: 79356 Added: pypy/trunk/pypy/jit/tool/log-template.gnumeric (contents, props changed) pypy/trunk/pypy/jit/tool/log2gnumeric.py (contents, props changed) Log: a tool to automatically convert a logfile into a gnumeric spreadsheet with a graph Added: pypy/trunk/pypy/jit/tool/log-template.gnumeric ============================================================================== Binary file. No diff available. Added: pypy/trunk/pypy/jit/tool/log2gnumeric.py ============================================================================== --- (empty file) +++ pypy/trunk/pypy/jit/tool/log2gnumeric.py Mon Nov 22 19:00:55 2010 @@ -0,0 +1,81 @@ +#! /usr/bin/env python +""" +Usage: log2gnumeric logfile + +Produces a logfile.gnumeric file which contains the data extracted from the +logfile generated with the PYPYLOG env variable. + +Currently, it expects log to contain the translation-task and gc-collect +categories. + +You can freely edit the graph in log-template.gnumeric: this script will +create a new file replacing the 'translation-task' and 'gc-collect' sheets. +""" + +import re, sys +import gzip + +def main(): + logname = sys.argv[1] + outname = logname + '.gnumeric' + data = open(logname).read() + data = data.replace('\n', '') + xml = gzip.open('log-template.gnumeric').read() + # + xml = replace_sheet(xml, 'translation-task', tasks_rows(data)) + xml = replace_sheet(xml, 'gc-collect', gc_collect_rows(data)) + # + out = gzip.open(outname, 'wb') + out.write(xml) + out.close() + + +def replace_sheet(xml, sheet_name, data): + pattern = '(%s.*?).*?' + regex = re.compile(pattern % sheet_name, re.DOTALL) + cells = gen_cells(data) + xml2 = regex.sub(r'\1%s' % cells, xml) + assert xml != xml2 + return xml2 + +def gen_cells(data): + parts = [] + parts.append(' ') + for i, row in enumerate(data): + for j, val in enumerate(row): + cell = ' %s' + parts.append(cell % (i, j, val)) + parts.append(' ') + return ''.join(parts) + + +def gc_collect_rows(data, show_header=True): + s = r""" +----------- Full collection ------------------ +\| used before collection: +\| in ArenaCollection: (\d+) bytes +\| raw_malloced: (\d+) bytes +\| used after collection: +\| in ArenaCollection: (\d+) bytes +\| raw_malloced: (\d+) bytes +\| number of major collects: (\d+) +`---------------------------------------------- +\[([0-9a-f]+)\] gc-collect\}""" + # + r = re.compile(s.replace('\n', '')) + for a,b,c,d,e,f in r.findall(data): + yield int(f, 16), int(a)+int(b), int(c)+int(d) + +def tasks_rows(data): + s = r""" +\{translation-task +starting ([\w-]+) +\[([0-9a-f]+)\] translation-task\}""" + # + r = re.compile(s.replace('\n', '')) + for a,b in r.findall(data): + yield int(b, 16), 1, a + + +if __name__ == '__main__': + main() From arigo at codespeak.net Mon Nov 22 19:11:10 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 19:11:10 +0100 (CET) Subject: [pypy-svn] r79357 - pypy/branch/jit-free/pypy/jit/metainterp Message-ID: <20101122181110.3F13850810@codespeak.net> Author: arigo Date: Mon Nov 22 19:11:08 2010 New Revision: 79357 Modified: pypy/branch/jit-free/pypy/jit/metainterp/jitprof.py Log: Fix for test_jitprof. Modified: pypy/branch/jit-free/pypy/jit/metainterp/jitprof.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/jitprof.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/jitprof.py Mon Nov 22 19:11:08 2010 @@ -39,6 +39,7 @@ _setup() JITPROF_LINES = ncounters + 1 + 1 # one for TOTAL, 1 for calls, update if needed +_CPU_LINES = 4 # the last 4 lines are stored on the cpu class BaseProfiler(object): pass @@ -101,7 +102,7 @@ self.starttime = self.timer() self.t1 = self.starttime self.times = [0, 0] - self.counters = [0] * ncounters + self.counters = [0] * (ncounters - _CPU_LINES) self.calls = 0 self.current = [] From arigo at codespeak.net Mon Nov 22 19:11:22 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 19:11:22 +0100 (CET) Subject: [pypy-svn] r79358 - pypy/branch/jit-free/pypy/jit/backend/test Message-ID: <20101122181122.B6ED3282BD6@codespeak.net> Author: arigo Date: Mon Nov 22 19:11:21 2010 New Revision: 79358 Modified: pypy/branch/jit-free/pypy/jit/backend/test/test_random.py Log: Fix test_random. Modified: pypy/branch/jit-free/pypy/jit/backend/test/test_random.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/test/test_random.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/test/test_random.py Mon Nov 22 19:11:21 2010 @@ -524,7 +524,8 @@ self.prebuilt_ptr_consts = [] self.r = r self.build_random_loop(cpu, builder_factory, r, startvars) - + self.keepalive_loop_tokens = [] + def build_random_loop(self, cpu, builder_factory, r, startvars): loop = TreeLoop('test_random_function') @@ -686,10 +687,11 @@ self.guard_op = rl.guard_op self.prebuilt_ptr_consts += rl.prebuilt_ptr_consts self.dont_generate_more = True + self.keepalive_loop_tokens.append(rl.loop.token) if r.random() < .05: return False self.builder.cpu.compile_bridge(fail_descr, fail_args, - subloop.operations) + subloop.operations, self.loop.token) return True def check_random_function(cpu, BuilderClass, r, num=None, max=None): From arigo at codespeak.net Mon Nov 22 19:13:26 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 22 Nov 2010 19:13:26 +0100 (CET) Subject: [pypy-svn] r79359 - pypy/branch/jit-free/pypy/jit/backend/test Message-ID: <20101122181326.E829A50810@codespeak.net> Author: arigo Date: Mon Nov 22 19:13:25 2010 New Revision: 79359 Modified: pypy/branch/jit-free/pypy/jit/backend/test/test_random.py Log: Use the official interface to keep the looptoken alive. Modified: pypy/branch/jit-free/pypy/jit/backend/test/test_random.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/backend/test/test_random.py (original) +++ pypy/branch/jit-free/pypy/jit/backend/test/test_random.py Mon Nov 22 19:13:25 2010 @@ -524,7 +524,6 @@ self.prebuilt_ptr_consts = [] self.r = r self.build_random_loop(cpu, builder_factory, r, startvars) - self.keepalive_loop_tokens = [] def build_random_loop(self, cpu, builder_factory, r, startvars): @@ -686,8 +685,8 @@ subloop.operations[-1] = jump_op self.guard_op = rl.guard_op self.prebuilt_ptr_consts += rl.prebuilt_ptr_consts + self.loop.token.record_jump_to(rl.loop.token) self.dont_generate_more = True - self.keepalive_loop_tokens.append(rl.loop.token) if r.random() < .05: return False self.builder.cpu.compile_bridge(fail_descr, fail_args, From david at codespeak.net Mon Nov 22 19:52:38 2010 From: david at codespeak.net (david at codespeak.net) Date: Mon, 22 Nov 2010 19:52:38 +0100 (CET) Subject: [pypy-svn] r79361 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . test Message-ID: <20101122185238.948A4282BF1@codespeak.net> Author: david Date: Mon Nov 22 19:52:36 2010 New Revision: 79361 Added: pypy/branch/arm-backend/pypy/jit/backend/arm/jump.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_jump.py Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Finish implementing the jump operation, based on the x86 jump code, adapted for the ARM backend Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Mon Nov 22 19:52:36 2010 @@ -235,7 +235,6 @@ self.mc.MOV_rr(r.fp.value, r.sp.value) def gen_bootstrap_code(self, inputargs, regalloc, looptoken): - regs = [] for i in range(len(inputargs)): loc = inputargs[i] reg = regalloc.force_allocate_reg(loc) @@ -247,10 +246,10 @@ raise ValueError self.mc.gen_load_int(reg.value, addr) self.mc.LDR_ri(reg.value, reg.value) - regs.append(reg) regalloc.possibly_free_var(reg) - looptoken._arm_arglocs = regs - return regs + arglocs = [regalloc.loc(arg) for arg in inputargs] + looptoken._arm_arglocs = arglocs + return arglocs direct_bootstrap_code_size=100*WORD def gen_direct_bootstrap_code(self, arglocs, loop_head, regalloc): @@ -265,7 +264,7 @@ for i in range(reg_args): loc = arglocs[i] - self.mc.MOV_rr(loc.value, i) + self.mov_loc_loc(r.all_regs[i], loc) for i in range(stack_locs): loc = arglocs[reg_args + i] @@ -273,10 +272,8 @@ if loc.is_reg(): self.mc.LDR_ri(loc.value, r.fp.value, stack_position) elif loc.is_stack(): - self.mc.PUSH([r.r0.value]) - self.mc.LDR_ri(r.ip, r.fp.value, stack_position) + self.mc.LDR_ri(r.ip.value, r.fp.value, stack_position) self.mov_loc_loc(r.ip, loc) - self.mc.POP([r.r0.value]) else: assert 0, 'invalid location' sp_patch_location = self._prepare_sp_patch_location() @@ -432,6 +429,24 @@ self.mc.MOV_rr(loc.value, prev_loc.value) mov_loc_loc = regalloc_mov + def regalloc_push(self, loc): + if loc.is_stack(): + self.mc.LDR_ri(r.ip.value, r.fp.value, loc.position*-WORD) + self.mc.PUSH([r.ip.value]) + elif loc.is_reg(): + self.mc.PUSH([loc.value]) + else: + assert 0, 'ffuu' + + def regalloc_pop(self, loc): + if loc.is_stack(): + self.mc.POP([r.ip.value]) + self.mc.STR_ri(r.ip.value, r.fp.value, loc.position*-WORD) + elif loc.is_reg(): + self.mc.POP([loc.value]) + else: + assert 0, 'ffuu' + def leave_jitted_hook(self): pass Added: pypy/branch/arm-backend/pypy/jit/backend/arm/jump.py ============================================================================== --- (empty file) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/jump.py Mon Nov 22 19:52:36 2010 @@ -0,0 +1,70 @@ +# ../x86/jump.py +# XXX combine with ../x86/jump.py and move to llsupport +import sys +from pypy.tool.pairtype import extendabletype + +def remap_frame_layout(assembler, src_locations, dst_locations, tmpreg): + pending_dests = len(dst_locations) + srccount = {} # maps dst_locations to how many times the same + # location appears in src_locations + for dst in dst_locations: + srccount[dst.as_key()] = 0 + for i in range(len(dst_locations)): + src = src_locations[i] + if src.is_imm(): + continue + key = src.as_key() + if key in srccount: + if key == dst_locations[i].as_key(): + srccount[key] = -sys.maxint # ignore a move "x = x" + pending_dests -= 1 + else: + srccount[key] += 1 + + while pending_dests > 0: + progress = False + for i in range(len(dst_locations)): + dst = dst_locations[i] + key = dst.as_key() + if srccount[key] == 0: + srccount[key] = -1 # means "it's done" + pending_dests -= 1 + src = src_locations[i] + if not src.is_imm(): + key = src.as_key() + if key in srccount: + srccount[key] -= 1 + _move(assembler, src, dst, tmpreg) + progress = True + if not progress: + # we are left with only pure disjoint cycles + sources = {} # maps dst_locations to src_locations + for i in range(len(dst_locations)): + src = src_locations[i] + dst = dst_locations[i] + sources[dst.as_key()] = src + # + for i in range(len(dst_locations)): + dst = dst_locations[i] + originalkey = dst.as_key() + if srccount[originalkey] >= 0: + assembler.regalloc_push(dst) + while True: + key = dst.as_key() + assert srccount[key] == 1 + # ^^^ because we are in a simple cycle + srccount[key] = -1 + pending_dests -= 1 + src = sources[key] + if src.as_key() == originalkey: + break + _move(assembler, src, dst, tmpreg) + dst = src + assembler.regalloc_pop(dst) + assert pending_dests == 0 + +def _move(assembler, src, dst, tmpreg): + if dst.is_stack() and src.is_stack(): + assembler.regalloc_mov(src, tmpreg) + src = tmpreg + assembler.regalloc_mov(src, dst) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/locations.py Mon Nov 22 19:52:36 2010 @@ -1,7 +1,8 @@ from pypy.jit.metainterp.history import INT from pypy.jit.backend.arm.arch import WORD class AssemblerLocation(object): - pass + _immutable_ = True + def is_imm(self): return False @@ -11,7 +12,11 @@ def is_reg(self): return False + def as_key(self): + raise NotImplementedError + class RegisterLocation(AssemblerLocation): + _immutable_ = True def __init__(self, value): self.value = value @@ -22,8 +27,12 @@ def is_reg(self): return True + def as_key(self): + return self.value + class ImmLocation(AssemblerLocation): _immutable_ = True + def __init__(self, value): self.value = value @@ -36,8 +45,12 @@ def is_imm(self): return True + def as_key(self): + return self.value + 20 + class StackLocation(AssemblerLocation): _immutable_ = True + def __init__(self, position, num_words=1, type=INT): self.position = position self.width = num_words * WORD @@ -50,7 +63,7 @@ return self.width // WORD def __repr__(self): - return 'SP+%d' % (self.position,) + return 'FP+%d' % (self.position,) def location_code(self): return 'b' @@ -60,3 +73,6 @@ def is_stack(self): return True + + def as_key(self): + return -self.position Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Mon Nov 22 19:52:36 2010 @@ -10,6 +10,7 @@ gen_emit_op_unary_cmp, gen_emit_op_ri, gen_emit_cmp_op) from pypy.jit.backend.arm.codebuilder import ARMv7Builder, ARMv7InMemoryBuilder +from pypy.jit.backend.arm.jump import remap_frame_layout from pypy.jit.backend.arm.regalloc import ARMRegisterManager from pypy.jit.backend.llsupport import symbolic from pypy.jit.backend.llsupport.descr import BaseFieldDescr, BaseArrayDescr @@ -231,12 +232,9 @@ _mixin_ = True def emit_op_jump(self, op, regalloc, fcond): - registers = op.getdescr()._arm_arglocs - for i in range(op.numargs()): - # avoid moving stuff twice - loc = registers[i] - prev_loc = regalloc.loc(op.getarg(i)) - self.mov_loc_loc(prev_loc, loc) + destlocs = op.getdescr()._arm_arglocs + srclocs = [regalloc.loc(op.getarg(i)) for i in range(op.numargs())] + remap_frame_layout(self, srclocs, destlocs, r.ip) loop_code = op.getdescr()._arm_loop_code self.mc.B(loop_code, fcond) Added: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_jump.py ============================================================================== --- (empty file) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_jump.py Mon Nov 22 19:52:36 2010 @@ -0,0 +1,116 @@ +import py + +from pypy.jit.backend.x86.test.test_jump import MockAssembler +from pypy.jit.backend.arm.registers import * +from pypy.jit.backend.arm.locations import * +from pypy.jit.backend.arm.regalloc import ARMFrameManager +from pypy.jit.backend.arm.jump import remap_frame_layout +from pypy.jit.metainterp.history import INT + +frame_pos = ARMFrameManager.frame_pos + +class TestJump(object): + def setup_method(self, m): + self.assembler = MockAssembler() + + def test_trivial(self): + remap_frame_layout(self.assembler, [], [], '?') + assert self.assembler.ops == [] + remap_frame_layout(self.assembler, [r0, r1, r3, r5, r6, r7, r9], + [r0, r1, r3, r5, r6, r7, r9], '?') + assert self.assembler.ops == [] + s8 = frame_pos(1, INT) + s12 = frame_pos(31, INT) + s20 = frame_pos(6, INT) + remap_frame_layout(self.assembler, [r0, r1, s20, s8, r3, r5, r6, s12, r7, r9], + [r0, r1, s20, s8, r3, r5, r6, s12, r7, r9], + '?') + assert self.assembler.ops == [] + + def test_simple_registers(self): + remap_frame_layout(self.assembler, [r0, r1, r2], [r3, r4, r5], '?') + assert self.assembler.ops == [('mov', r0, r3), + ('mov', r1, r4), + ('mov', r2, r5)] + + def test_simple_framelocs(self): + s8 = frame_pos(0, INT) + s12 = frame_pos(13, INT) + s20 = frame_pos(20, INT) + s24 = frame_pos(221, INT) + remap_frame_layout(self.assembler, [s8, r7, s12], [s20, s24, r9], ip) + assert self.assembler.ops == [('mov', s8, ip), + ('mov', ip, s20), + ('mov', r7, s24), + ('mov', s12, r9)] + + def test_reordering(self): + s8 = frame_pos(8, INT) + s12 = frame_pos(12, INT) + s20 = frame_pos(19, INT) + s24 = frame_pos(1, INT) + remap_frame_layout(self.assembler, [r7, s8, s20, r4], + [s8, r4, r7, r2], '?') + assert self.assembler.got([('mov', r4, r2), + ('mov', s8, r4), + ('mov', r7, s8), + ('mov', s20, r7)]) + + def test_cycle(self): + s8 = frame_pos(8, INT) + s12 = frame_pos(12, INT) + s20 = frame_pos(19, INT) + s24 = frame_pos(1, INT) + remap_frame_layout(self.assembler, [r4, s8, s20, r7], + [s8, r7, r4, s20], '?') + assert self.assembler.got([('push', s8), + ('mov', r4, s8), + ('mov', s20, r4), + ('mov', r7, s20), + ('pop', r7)]) + + def test_cycle_2(self): + s8 = frame_pos(8, INT) + s12 = frame_pos(12, INT) + s20 = frame_pos(19, INT) + s24 = frame_pos(1, INT) + s2 = frame_pos(2, INT) + s3 = frame_pos(3, INT) + remap_frame_layout(self.assembler, + [r0, s8, r1, s20, r0, s20, s24, r3, s2, s3], + [s8, s20, r1, r0, r4, s24, r5, s12, s3, s2], + ip) + assert self.assembler.got([('mov', r0, r4), + ('mov', s24, r5), + ('mov', r3, s12), + ('mov', s20, ip), + ('mov', ip, s24), + ('push', s8), + ('mov', r0, s8), + ('mov', s20, r0), + ('pop', s20), + ('push', s3), + ('mov', s2, ip), + ('mov', ip, s3), + ('pop', s2)]) + + def test_constants(self): + c3 = ImmLocation(3) + remap_frame_layout(self.assembler, [c3], [r0], '?') + assert self.assembler.ops == [('mov', c3, r0)] + + def test_constants2(self): + c3 = ImmLocation(3) + s12 = frame_pos(12, INT) + remap_frame_layout(self.assembler, [c3], [s12], '?') + assert self.assembler.ops == [('mov', c3, s12)] + + def test_constants_and_cycle(self): + c3 = ImmLocation(3) + s12 = frame_pos(13, INT) + remap_frame_layout(self.assembler, [r5, c3, s12], + [s12, r0, r5], r1) + assert self.assembler.ops == [('mov', c3, r0), + ('push', s12), + ('mov', r5, s12), + ('pop', r5)] From fijal at codespeak.net Mon Nov 22 21:32:19 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 22 Nov 2010 21:32:19 +0100 (CET) Subject: [pypy-svn] r79362 - pypy/release/1.4.x/pypy/config Message-ID: <20101122203219.387295080B@codespeak.net> Author: fijal Date: Mon Nov 22 21:32:14 2010 New Revision: 79362 Modified: pypy/release/1.4.x/pypy/config/translationoption.py Log: As we're going to rebuild those anyway, let's turn off jit debug Modified: pypy/release/1.4.x/pypy/config/translationoption.py ============================================================================== --- pypy/release/1.4.x/pypy/config/translationoption.py (original) +++ pypy/release/1.4.x/pypy/config/translationoption.py Mon Nov 22 21:32:14 2010 @@ -115,7 +115,7 @@ default="auto", cmdline="--jit-backend"), ChoiceOption("jit_debug", "the amount of debugging dumps made by the JIT", ["off", "profile", "steps", "detailed"], - default="profile", # XXX for now + default="off", # XXX for now cmdline="--jit-debug"), ChoiceOption("jit_profiler", "integrate profiler support into the JIT", ["off", "oprofile"], From afa at codespeak.net Mon Nov 22 23:08:12 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 22 Nov 2010 23:08:12 +0100 (CET) Subject: [pypy-svn] r79365 - pypy/branch/fast-forward/pypy/module/sys Message-ID: <20101122220812.22283282BD6@codespeak.net> Author: afa Date: Mon Nov 22 23:08:10 2010 New Revision: 79365 Modified: pypy/branch/fast-forward/pypy/module/sys/version.py Log: Show GCC version in sys.version; fixes the test. Modified: pypy/branch/fast-forward/pypy/module/sys/version.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/sys/version.py (original) +++ pypy/branch/fast-forward/pypy/module/sys/version.py Mon Nov 22 23:08:10 2010 @@ -2,6 +2,7 @@ Version numbers exposed by PyPy through the 'sys' module. """ import os +import re from pypy.translator.platform import platform #XXX # the release serial 42 is not in range(16) @@ -18,6 +19,13 @@ if platform.name == 'msvc': COMPILER_INFO = 'MSC v.%d 32 bit' % (platform.version * 10 + 600) +elif platform.cc == 'gcc': + out = platform.execute(platform.cc, '--version').out + match = re.search(' (\d+\.\d+(\.\d+)*)', out) + if match: + COMPILER_INFO = "GCC " + match.group(1) + else: + COMPILER_INFO = "GCC" else: COMPILER_INFO = "" From wlav at codespeak.net Mon Nov 22 23:15:32 2010 From: wlav at codespeak.net (wlav at codespeak.net) Date: Mon, 22 Nov 2010 23:15:32 +0100 (CET) Subject: [pypy-svn] r79366 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101122221532.2513D5080B@codespeak.net> Author: wlav Date: Mon Nov 22 23:15:31 2010 New Revision: 79366 Modified: pypy/branch/reflex-support/pypy/module/cppyy/executor.py Log: for rtyper: make code paths more consistent Modified: pypy/branch/reflex-support/pypy/module/cppyy/executor.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/executor.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/executor.py Mon Nov 22 23:15:31 2010 @@ -6,47 +6,59 @@ _executors = {} class FunctionExecutor(object): + _immutable_ = True + def __init__(self, space, cpptype): + pass + def execute(self, space, func, cppthis, num_args, args): raise NotImplementedError class VoidExecutor(FunctionExecutor): + _immutable_ = True def execute(self, space, func, cppthis, num_args, args): capi.c_call_v(func.cpptype.handle, func.method_index, cppthis, num_args, args) return space.w_None class BoolExecutor(FunctionExecutor): + _immutable_ = True def execute(self, space, func, cppthis, num_args, args): result = capi.c_call_b(func.cpptype.handle, func.method_index, cppthis, num_args, args) return space.wrap(result) class CharExecutor(FunctionExecutor): - def execute(self, space, func, cppthis, num_args, args): + _immutable_ = True + def execute(self, space, func, cppthis, num_args, args): result = capi.c_call_c(func.cpptype.handle, func.method_index, cppthis, num_args, args) return space.wrap(result) class ShortExecutor(FunctionExecutor): + _immutable_ = True def execute(self, space, func, cppthis, num_args, args): result = capi.c_call_h(func.cpptype.handle, func.method_index, cppthis, num_args, args) return space.wrap(result) class LongExecutor(FunctionExecutor): + _immutable_ = True def execute(self, space, func, cppthis, num_args, args): result = capi.c_call_l(func.cpptype.handle, func.method_index, cppthis, num_args, args) return space.wrap(result) class FloatExecutor(FunctionExecutor): + _immutable_ = True def execute(self, space, func, cppthis, num_args, args): result = capi.c_call_f(func.cpptype.handle, func.method_index, cppthis, num_args, args) return space.wrap(result) class DoubleExecutor(FunctionExecutor): + _immutable_ = True def execute(self, space, func, cppthis, num_args, args): result = capi.c_call_d(func.cpptype.handle, func.method_index, cppthis, num_args, args) return space.wrap(result) class CStringExecutor(FunctionExecutor): + _immutable_ = True def execute(self, space, func, cppthis, num_args, args): lresult = capi.c_call_l(func.cpptype.handle, func.method_index, cppthis, num_args, args) ccpresult = rffi.cast(rffi.CCHARP, lresult) @@ -70,7 +82,7 @@ from pypy.module.cppyy import interp_cppyy try: - return _executors[name] + return _executors[name](space, None) except KeyError: pass @@ -79,20 +91,21 @@ if compound == "*": return InstancePtrExecutor(space, cpptype) - return None # currently used until proper lazy instantiation available in interp_cppyy + # currently used until proper lazy instantiation available in interp_cppyy + return FunctionExecutor(space, None) # raise TypeError("no clue what %s is" % name) -_executors["void"] = VoidExecutor() -_executors["bool"] = BoolExecutor() -_executors["char"] = CharExecutor() -_executors["unsigned char"] = CharExecutor() -_executors["short int"] = ShortExecutor() -_executors["unsigned short int"] = ShortExecutor() -_executors["int"] = LongExecutor() -_executors["unsigned int"] = LongExecutor() -_executors["long int"] = LongExecutor() -_executors["unsigned long int"] = LongExecutor() -_executors["float"] = FloatExecutor() -_executors["double"] = DoubleExecutor() -_executors["char*"] = CStringExecutor() +_executors["void"] = VoidExecutor +_executors["bool"] = BoolExecutor +_executors["char"] = CharExecutor +_executors["unsigned char"] = CharExecutor +_executors["short int"] = ShortExecutor +_executors["unsigned short int"] = ShortExecutor +_executors["int"] = LongExecutor +_executors["unsigned int"] = LongExecutor +_executors["long int"] = LongExecutor +_executors["unsigned long int"] = LongExecutor +_executors["float"] = FloatExecutor +_executors["double"] = DoubleExecutor +_executors["char*"] = CStringExecutor From wlav at codespeak.net Mon Nov 22 23:16:04 2010 From: wlav at codespeak.net (wlav at codespeak.net) Date: Mon, 22 Nov 2010 23:16:04 +0100 (CET) Subject: [pypy-svn] r79367 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101122221604.369F2282BDC@codespeak.net> Author: wlav Date: Mon Nov 22 23:16:02 2010 New Revision: 79367 Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Log: cosmetics and for rtyper: explicit str Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Mon Nov 22 23:16:02 2010 @@ -39,7 +39,7 @@ cpptype._find_data_members() return cpptype - raise OperationError(space.w_TypeError, space.wrap("no such C++ class %s" % name)) + raise OperationError(space.w_TypeError, space.wrap(str("no such C++ class %s" % name))) type_byname.unwrap_spec = [ObjSpace, str] @@ -217,7 +217,6 @@ class W_CPPDataMember(Wrappable): _immutable_fields_ = ["converter", "offset"] - def __init__(self, space, cpptype, offset): self.space = space self.converter = converter.get_converter(self.space, cpptype) @@ -239,7 +238,6 @@ class W_CPPType(Wrappable): _immutable_fields_ = ["name", "handle"] - def __init__(self, space, name, handle): self.space = space self.name = name From wlav at codespeak.net Mon Nov 22 23:16:31 2010 From: wlav at codespeak.net (wlav at codespeak.net) Date: Mon, 22 Nov 2010 23:16:31 +0100 (CET) Subject: [pypy-svn] r79368 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101122221631.7DC8F5080C@codespeak.net> Author: wlav Date: Mon Nov 22 23:16:30 2010 New Revision: 79368 Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py Log: for rtyper: use rffi cast rather than chr Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/converter.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/converter.py Mon Nov 22 23:16:30 2010 @@ -91,10 +91,12 @@ def _from_space(self, space, w_value): # allow int to pass to char and make sure that str is of length 1 if space.isinstance_w(w_value, space.w_int): - try: - value = chr(space.c_int_w(w_value)) - except ValueError, e: - raise OperationError(space.w_TypeError, space.wrap(str(e))) + ival = space.c_int_w(w_value) + if ival < 0 or 256 <= ival: + raise OperationError(space.w_TypeError, + space.wrap("char arg not in range(256)")) + + value = rffi.cast(rffi.CHAR, space.c_int_w(w_value)) else: value = space.str_w(w_value) From afa at codespeak.net Tue Nov 23 00:04:14 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 23 Nov 2010 00:04:14 +0100 (CET) Subject: [pypy-svn] r79377 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101122230414.747B1282BDC@codespeak.net> Author: afa Date: Tue Nov 23 00:04:12 2010 New Revision: 79377 Modified: pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py pypy/branch/fast-forward/pypy/objspace/std/dicttype.py pypy/branch/fast-forward/pypy/objspace/std/model.py pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py Log: Start implementing dictionary views Modified: pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py Tue Nov 23 00:04:12 2010 @@ -756,6 +756,15 @@ def dict_itervalues__DictMulti(space, w_self): return W_DictMultiIterObject(space, w_self.iter(), VALUESITER) +def dict_viewitems__DictMulti(space, w_self): + return W_DictMultiViewItemsObject(space, w_self) + +def dict_viewkeys__DictMulti(space, w_self): + return W_DictMultiViewKeysObject(space, w_self) + +def dict_viewvalues__DictMulti(space, w_self): + return W_DictMultiViewValuesObject(space, w_self) + def dict_clear__DictMulti(space, w_self): w_self.clear() @@ -874,6 +883,29 @@ raise OperationError(space.w_StopIteration, space.w_None) # ____________________________________________________________ +# Views + +class W_DictViewObject(W_Object): + def __init__(w_self, space, w_dict): + w_self.w_dict = w_dict + +class W_DictMultiViewKeysObject(W_DictViewObject): + from pypy.objspace.std.dicttype import dict_keys_typedef as typedef +registerimplementation(W_DictMultiViewKeysObject) + +class W_DictMultiViewItemsObject(W_DictViewObject): + from pypy.objspace.std.dicttype import dict_items_typedef as typedef +registerimplementation(W_DictMultiViewItemsObject) + +class W_DictMultiViewValuesObject(W_DictViewObject): + from pypy.objspace.std.dicttype import dict_values_typedef as typedef +registerimplementation(W_DictMultiViewValuesObject) + +def len__DictMultiViewKeys(space, w_dictview): + return space.len(w_dictview.w_dict) +len__DictMultiViewItems = len__DictMultiViewValues = len__DictMultiViewKeys + +# ____________________________________________________________ from pypy.objspace.std import dicttype register_all(vars(), dicttype) Modified: pypy/branch/fast-forward/pypy/objspace/std/dicttype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/dicttype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/dicttype.py Tue Nov 23 00:04:12 2010 @@ -44,6 +44,12 @@ doc='D.iterkeys() -> an iterator over the keys of D') dict_itervalues = SMM('itervalues', 1, doc='D.itervalues() -> an iterator over the values of D') +dict_viewkeys = SMM('viewkeys', 1, + doc="D.viewkeys() -> a set-like object providing a view on D's keys") +dict_viewitems = SMM('viewitems', 1, + doc="D.viewitems() -> a set-like object providing a view on D's items") +dict_viewvalues = SMM('viewvalues', 1, + doc="D.viewvalues() -> an object providing a view on D's values") dict_reversed = SMM('__reversed__', 1) def dict_reversed__ANY(space, w_dict): @@ -160,3 +166,29 @@ __reduce__ = gateway.interp2app(descr_dictiter__reduce__, unwrap_spec=[gateway.W_Root, gateway.ObjSpace]), ) + +# ____________________________________________________________ +# Dict views + + + at gateway.unwrap_spec(gateway.ObjSpace, gateway.W_Root, gateway.W_Root) +def descr_view__new__(space, w_viewtype, w_dict): + from pypy.objspace.std.dictmultiobject import W_DictView + w_obj = space.allocate_instance(W_DictView, w_viewtype) + w_obj.__init__(space, w_dict) + return w_obj + +dict_keys_typedef = StdTypeDef( + "dict_keys", + __new__ = descr_view__new__, + ) + +dict_items_typedef = StdTypeDef( + "dict_items", + __new__ = descr_view__new__, + ) + +dict_values_typedef = StdTypeDef( + "dict_values", + __new__ = descr_view__new__, + ) Modified: pypy/branch/fast-forward/pypy/objspace/std/model.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/model.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/model.py Tue Nov 23 00:04:12 2010 @@ -122,6 +122,9 @@ iterobject.W_ReverseSeqIterObject: [], unicodeobject.W_UnicodeObject: [], dictproxyobject.W_DictProxyObject: [], + dictmultiobject.W_DictMultiViewKeysObject: [], + dictmultiobject.W_DictMultiViewItemsObject: [], + dictmultiobject.W_DictMultiViewValuesObject: [], pypy.interpreter.pycode.PyCode: [], pypy.interpreter.special.Ellipsis: [], } Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py Tue Nov 23 00:04:12 2010 @@ -500,6 +500,12 @@ iterable = {} raises(TypeError, len, iter(iterable)) + def test_dictview(self): + d = {1: 2, 3: 4} + assert len(d.viewkeys()) == 2 + assert len(d.viewitems()) == 2 + assert len(d.viewvalues()) == 2 + class AppTest_DictMultiObject(AppTest_DictObject): From afa at codespeak.net Tue Nov 23 01:18:47 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 23 Nov 2010 01:18:47 +0100 (CET) Subject: [pypy-svn] r79379 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101123001847.B10925080B@codespeak.net> Author: afa Date: Tue Nov 23 01:18:46 2010 New Revision: 79379 Modified: pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py pypy/branch/fast-forward/pypy/objspace/std/dicttype.py pypy/branch/fast-forward/pypy/objspace/std/model.py pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py Log: More progress on dictviews Modified: pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py Tue Nov 23 01:18:46 2010 @@ -7,6 +7,7 @@ from pypy.module.__builtin__.__init__ import BUILTIN_TO_INDEX, OPTIMIZED_BUILTINS from pypy.rlib.objectmodel import r_dict, we_are_translated +from pypy.objspace.std.settype import set_typedef as settypedef def _is_str(space, w_key): return space.is_w(space.type(w_key), space.w_str) @@ -757,13 +758,13 @@ return W_DictMultiIterObject(space, w_self.iter(), VALUESITER) def dict_viewitems__DictMulti(space, w_self): - return W_DictMultiViewItemsObject(space, w_self) + return W_DictViewItemsObject(space, w_self) def dict_viewkeys__DictMulti(space, w_self): - return W_DictMultiViewKeysObject(space, w_self) + return W_DictViewKeysObject(space, w_self) def dict_viewvalues__DictMulti(space, w_self): - return W_DictMultiViewValuesObject(space, w_self) + return W_DictViewValuesObject(space, w_self) def dict_clear__DictMulti(space, w_self): w_self.clear() @@ -889,21 +890,62 @@ def __init__(w_self, space, w_dict): w_self.w_dict = w_dict -class W_DictMultiViewKeysObject(W_DictViewObject): +class W_DictViewKeysObject(W_DictViewObject): from pypy.objspace.std.dicttype import dict_keys_typedef as typedef -registerimplementation(W_DictMultiViewKeysObject) +registerimplementation(W_DictViewKeysObject) -class W_DictMultiViewItemsObject(W_DictViewObject): +class W_DictViewItemsObject(W_DictViewObject): from pypy.objspace.std.dicttype import dict_items_typedef as typedef -registerimplementation(W_DictMultiViewItemsObject) +registerimplementation(W_DictViewItemsObject) -class W_DictMultiViewValuesObject(W_DictViewObject): +class W_DictViewValuesObject(W_DictViewObject): from pypy.objspace.std.dicttype import dict_values_typedef as typedef -registerimplementation(W_DictMultiViewValuesObject) +registerimplementation(W_DictViewValuesObject) -def len__DictMultiViewKeys(space, w_dictview): +def len__DictViewKeys(space, w_dictview): return space.len(w_dictview.w_dict) -len__DictMultiViewItems = len__DictMultiViewValues = len__DictMultiViewKeys +len__DictViewItems = len__DictViewValues = len__DictViewKeys + +def iter__DictViewKeys(space, w_dictview): + return dict_iterkeys__DictMulti(space, w_dictview.w_dict) +def iter__DictViewItems(space, w_dictview): + return dict_iteritems__DictMulti(space, w_dictview.w_dict) +def iter__DictViewValues(space, w_dictview): + return dict_itervalues__DictMulti(space, w_dictview.w_dict) + +def all_contained_in(space, w_dictview, w_otherview): + w_iter = space.iter(w_dictview) + assert isinstance(w_iter, W_DictMultiIterObject) + + while True: + try: + w_item = space.next(w_iter) + except OperationError, e: + if not e.match(space, space.w_StopIteration): + raise + break + if not space.is_true(space.contains(w_otherview, w_item)): + return space.w_False + + return space.w_True + +def eq__DictViewKeys_DictViewKeys(space, w_dictview, w_otherview): + if space.eq_w(space.len(w_dictview), space.len(w_otherview)): + return all_contained_in(space, w_dictview, w_otherview) + return space.w_False +eq__DictViewKeys_settypedef = eq__DictViewKeys_DictViewKeys + +eq__DictViewKeys_DictViewItems = eq__DictViewKeys_DictViewKeys +eq__DictViewItems_DictViewItems = eq__DictViewKeys_DictViewKeys +eq__DictViewItems_settypedef = eq__DictViewItems_DictViewItems + +def repr__DictViewKeys(space, w_dictview): + w_seq = space.call_function(space.w_list, w_dictview) + w_repr = space.repr(w_seq) + return space.wrap("%s(%s)" % (space.type(w_dictview).getname(space, "?"), + space.str_w(w_repr))) +repr__DictViewItems = repr__DictViewKeys +repr__DictViewValues = repr__DictViewKeys # ____________________________________________________________ Modified: pypy/branch/fast-forward/pypy/objspace/std/dicttype.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/dicttype.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/dicttype.py Tue Nov 23 01:18:46 2010 @@ -170,25 +170,14 @@ # ____________________________________________________________ # Dict views - - at gateway.unwrap_spec(gateway.ObjSpace, gateway.W_Root, gateway.W_Root) -def descr_view__new__(space, w_viewtype, w_dict): - from pypy.objspace.std.dictmultiobject import W_DictView - w_obj = space.allocate_instance(W_DictView, w_viewtype) - w_obj.__init__(space, w_dict) - return w_obj - dict_keys_typedef = StdTypeDef( "dict_keys", - __new__ = descr_view__new__, ) dict_items_typedef = StdTypeDef( "dict_items", - __new__ = descr_view__new__, ) dict_values_typedef = StdTypeDef( "dict_values", - __new__ = descr_view__new__, ) Modified: pypy/branch/fast-forward/pypy/objspace/std/model.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/model.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/model.py Tue Nov 23 01:18:46 2010 @@ -122,9 +122,9 @@ iterobject.W_ReverseSeqIterObject: [], unicodeobject.W_UnicodeObject: [], dictproxyobject.W_DictProxyObject: [], - dictmultiobject.W_DictMultiViewKeysObject: [], - dictmultiobject.W_DictMultiViewItemsObject: [], - dictmultiobject.W_DictMultiViewValuesObject: [], + dictmultiobject.W_DictViewKeysObject: [], + dictmultiobject.W_DictViewItemsObject: [], + dictmultiobject.W_DictViewValuesObject: [], pypy.interpreter.pycode.PyCode: [], pypy.interpreter.special.Ellipsis: [], } Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py Tue Nov 23 01:18:46 2010 @@ -500,13 +500,6 @@ iterable = {} raises(TypeError, len, iter(iterable)) - def test_dictview(self): - d = {1: 2, 3: 4} - assert len(d.viewkeys()) == 2 - assert len(d.viewitems()) == 2 - assert len(d.viewvalues()) == 2 - - class AppTest_DictMultiObject(AppTest_DictObject): def test_emptydict_unhashable(self): @@ -526,6 +519,148 @@ assert type(key) is str assert getattr(a, s) == 42 +class AppTestDictViews: + def test_dictview(self): + d = {1: 2, 3: 4} + assert len(d.viewkeys()) == 2 + assert len(d.viewitems()) == 2 + assert len(d.viewvalues()) == 2 + + def test_constructors_not_callable(self): + kt = type({}.viewkeys()) + raises(TypeError, kt, {}) + raises(TypeError, kt) + it = type({}.viewitems()) + raises(TypeError, it, {}) + raises(TypeError, it) + vt = type({}.viewvalues()) + raises(TypeError, vt, {}) + raises(TypeError, vt) + + def test_dict_keys(self): + d = {1: 10, "a": "ABC"} + keys = d.viewkeys() + assert len(keys) == 2 + assert set(keys) == set([1, "a"]) + assert keys == set([1, "a"]) + assert keys != set([1, "a", "b"]) + assert keys != set([1, "b"]) + assert keys != set([1]) + assert keys != 42 + assert 1 in keys + assert "a" in keys + assert 10 not in keys + assert "Z" not in keys + assert d.viewkeys() == d.viewkeys() + e = {1: 11, "a": "def"} + assert d.viewkeys() == e.viewkeys() + del e["a"] + assert d.viewkeys() != e.viewkeys() + + def test_dict_items(self): + d = {1: 10, "a": "ABC"} + items = d.viewitems() + assert len(items) == 2 + assert set(items) == set([(1, 10), ("a", "ABC")]) + assert items == set([(1, 10), ("a", "ABC")]) + assert items != set([(1, 10), ("a", "ABC"), "junk"]) + assert items != set([(1, 10), ("a", "def")]) + assert items != set([(1, 10)]) + assert items != 42 + assert (1, 10) in items + assert ("a", "ABC") in items + assert (1, 11) not in items + assert 1 not in items + assert () not in items + assert (1,) not in items + assert (1, 2, 3) not in items + assert d.viewitems() == d.viewitems() + e = d.copy() + assert d.viewitems() == e.viewitems() + e["a"] = "def" + assert d.viewitems() != e.viewitems() + + def test_dict_mixed_keys_items(self): + d = {(1, 1): 11, (2, 2): 22} + e = {1: 1, 2: 2} + assert d.viewkeys() == e.viewitems() + assert d.viewitems() != e.viewkeys() + + def test_dict_values(self): + d = {1: 10, "a": "ABC"} + values = d.viewvalues() + assert set(values) == set([10, "ABC"]) + assert len(values) == 2 + + def test_dict_repr(self): + d = {1: 10, "a": "ABC"} + assert isinstance(repr(d), str) + r = repr(d.viewitems()) + assert isinstance(r, str) + assert (r == "dict_items([('a', 'ABC'), (1, 10)])" or + r == "dict_items([(1, 10), ('a', 'ABC')])") + r = repr(d.viewkeys()) + assert isinstance(r, str) + assert (r == "dict_keys(['a', 1])" or + r == "dict_keys([1, 'a'])") + r = repr(d.viewvalues()) + assert isinstance(r, str) + assert (r == "dict_values(['ABC', 10])" or + r == "dict_values([10, 'ABC'])") + + def test_keys_set_operations(self): + d1 = {'a': 1, 'b': 2} + d2 = {'b': 3, 'c': 2} + d3 = {'d': 4, 'e': 5} + assert d1.viewkeys() & d1.viewkeys() == {'a', 'b'} + assert d1.viewkeys() & d2.viewkeys() == {'b'} + assert d1.viewkeys() & d3.viewkeys() == set() + assert d1.viewkeys() & set(d1.viewkeys()) == {'a', 'b'} + assert d1.viewkeys() & set(d2.viewkeys()) == {'b'} + assert d1.viewkeys() & set(d3.viewkeys()) == set() + + assert d1.viewkeys() | d1.viewkeys() == {'a', 'b'} + assert d1.viewkeys() | d2.viewkeys() == {'a', 'b', 'c'} + assert d1.viewkeys() | d3.viewkeys() == {'a', 'b', 'd', 'e'} + assert d1.viewkeys() | set(d1.viewkeys()) == {'a', 'b'} + assert d1.viewkeys() | set(d2.viewkeys()) == {'a', 'b', 'c'} + assert d1.viewkeys() | set(d3.viewkeys()) == {'a', 'b', 'd', 'e'} + + assert d1.viewkeys() ^ d1.viewkeys() == set() + assert d1.viewkeys() ^ d2.viewkeys() == {'a', 'c'} + assert d1.viewkeys() ^ d3.viewkeys() == {'a', 'b', 'd', 'e'} + assert d1.viewkeys() ^ set(d1.viewkeys()) == set() + assert d1.viewkeys() ^ set(d2.viewkeys()) == {'a', 'c'} + assert d1.viewkeys() ^ set(d3.viewkeys()) == {'a', 'b', 'd', 'e'} + + def test_items_set_operations(self): + d1 = {'a': 1, 'b': 2} + d2 = {'a': 2, 'b': 2} + d3 = {'d': 4, 'e': 5} + assert d1.viewitems() & d1.viewitems() == {('a', 1), ('b', 2)} + assert d1.viewitems() & d2.viewitems() == {('b', 2)} + assert d1.viewitems() & d3.viewitems() == set() + assert d1.viewitems() & set(d1.viewitems()) == {('a', 1), ('b', 2)} + assert d1.viewitems() & set(d2.viewitems()) == {('b', 2)} + assert d1.viewitems() & set(d3.viewitems()) == set() + + assert d1.viewitems() | d1.viewitems() == {('a', 1), ('b', 2)} + assert (d1.viewitems() | d2.viewitems() == + {('a', 1), ('a', 2), ('b', 2)}) + assert (d1.viewitems() | d3.viewitems() == + {('a', 1), ('b', 2), ('d', 4), ('e', 5)}) + assert (d1.viewitems() | set(d1.viewitems()) == + {('a', 1), ('b', 2)}) + assert (d1.viewitems() | set(d2.viewitems()) == + {('a', 1), ('a', 2), ('b', 2)}) + assert (d1.viewitems() | set(d3.viewitems()) == + {('a', 1), ('b', 2), ('d', 4), ('e', 5)}) + + assert d1.viewitems() ^ d1.viewitems() == set() + assert d1.viewitems() ^ d2.viewitems() == {('a', 1), ('a', 2)} + assert (d1.viewitems() ^ d3.viewitems() == + {('a', 1), ('b', 2), ('d', 4), ('e', 5)}) + class AppTestModuleDict(object): def setup_class(cls): From afa at codespeak.net Tue Nov 23 01:26:44 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 23 Nov 2010 01:26:44 +0100 (CET) Subject: [pypy-svn] r79380 - pypy/branch/fast-forward/pypy/objspace/std Message-ID: <20101123002644.42AE9282BD4@codespeak.net> Author: afa Date: Tue Nov 23 01:26:42 2010 New Revision: 79380 Modified: pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py Log: Fix remaining dictview tests Modified: pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/dictmultiobject.py Tue Nov 23 01:26:42 2010 @@ -947,6 +947,30 @@ repr__DictViewItems = repr__DictViewKeys repr__DictViewValues = repr__DictViewKeys +def and__DictViewKeys_DictViewKeys(space, w_dictview, w_otherview): + w_set = space.call_function(space.w_set, w_dictview) + space.call_method(w_set, "intersection_update", w_otherview) + return w_set +and__DictViewKeys_settypedef = and__DictViewKeys_DictViewKeys +and__DictViewItems_DictViewItems = and__DictViewKeys_DictViewKeys +and__DictViewItems_settypedef = and__DictViewKeys_DictViewKeys + +def or__DictViewKeys_DictViewKeys(space, w_dictview, w_otherview): + w_set = space.call_function(space.w_set, w_dictview) + space.call_method(w_set, "update", w_otherview) + return w_set +or__DictViewKeys_settypedef = or__DictViewKeys_DictViewKeys +or__DictViewItems_DictViewItems = or__DictViewKeys_DictViewKeys +or__DictViewItems_settypedef = or__DictViewKeys_DictViewKeys + +def xor__DictViewKeys_DictViewKeys(space, w_dictview, w_otherview): + w_set = space.call_function(space.w_set, w_dictview) + space.call_method(w_set, "symmetric_difference_update", w_otherview) + return w_set +xor__DictViewKeys_settypedef = xor__DictViewKeys_DictViewKeys +xor__DictViewItems_DictViewItems = xor__DictViewKeys_DictViewKeys +xor__DictViewItems_settypedef = xor__DictViewKeys_DictViewKeys + # ____________________________________________________________ from pypy.objspace.std import dicttype From fijal at codespeak.net Tue Nov 23 08:02:46 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 23 Nov 2010 08:02:46 +0100 (CET) Subject: [pypy-svn] r79381 - pypy/trunk/pypy/tool/release Message-ID: <20101123070246.A341E282BD6@codespeak.net> Author: fijal Date: Tue Nov 23 08:02:43 2010 New Revision: 79381 Modified: pypy/trunk/pypy/tool/release/force-builds.py Log: Comment out non-jit builds to not overload the server Modified: pypy/trunk/pypy/tool/release/force-builds.py ============================================================================== --- pypy/trunk/pypy/tool/release/force-builds.py (original) +++ pypy/trunk/pypy/tool/release/force-builds.py Tue Nov 23 08:02:43 2010 @@ -20,8 +20,8 @@ 'own-linux-x86-32', 'own-linux-x86-64', # 'own-macosx-x86-32', - 'pypy-c-app-level-linux-x86-32', - 'pypy-c-app-level-linux-x86-64', +# 'pypy-c-app-level-linux-x86-32', +# 'pypy-c-app-level-linux-x86-64', 'pypy-c-stackless-app-level-linux-x86-32', 'pypy-c-app-level-win-x86-32', 'pypy-c-jit-linux-x86-32', From fijal at codespeak.net Tue Nov 23 08:18:58 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 23 Nov 2010 08:18:58 +0100 (CET) Subject: [pypy-svn] r79382 - pypy/trunk/pypy/config Message-ID: <20101123071858.17721282BD6@codespeak.net> Author: fijal Date: Tue Nov 23 08:18:57 2010 New Revision: 79382 Modified: pypy/trunk/pypy/config/translationoption.py Log: Disable jit debugging by default Modified: pypy/trunk/pypy/config/translationoption.py ============================================================================== --- pypy/trunk/pypy/config/translationoption.py (original) +++ pypy/trunk/pypy/config/translationoption.py Tue Nov 23 08:18:57 2010 @@ -115,7 +115,7 @@ default="auto", cmdline="--jit-backend"), ChoiceOption("jit_debug", "the amount of debugging dumps made by the JIT", ["off", "profile", "steps", "detailed"], - default="profile", # XXX for now + default="off", cmdline="--jit-debug"), ChoiceOption("jit_profiler", "integrate profiler support into the JIT", ["off", "oprofile"], From david at codespeak.net Tue Nov 23 09:05:35 2010 From: david at codespeak.net (david at codespeak.net) Date: Tue, 23 Nov 2010 09:05:35 +0100 (CET) Subject: [pypy-svn] r79383 - pypy/branch/arm-backend/pypy/jit/backend Message-ID: <20101123080535.7B247282B90@codespeak.net> Author: david Date: Tue Nov 23 09:05:31 2010 New Revision: 79383 Modified: pypy/branch/arm-backend/pypy/jit/backend/detect_cpu.py Log: Add ARM to the cpu detection Modified: pypy/branch/arm-backend/pypy/jit/backend/detect_cpu.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/detect_cpu.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/detect_cpu.py Tue Nov 23 09:05:31 2010 @@ -36,7 +36,8 @@ 'i86pc': 'x86', # Solaris/Intel 'x86': 'x86', # Apple 'Power Macintosh': 'ppc', - 'x86_64': 'x86_64', + 'x86_64': 'x86_64', + 'armv7l': 'arm', }[mach] except KeyError: raise ProcessorAutodetectError, "unsupported processor '%s'" % mach @@ -62,6 +63,8 @@ return "pypy.jit.backend.cli.runner", "CliCPU" elif backend_name == 'llvm': return "pypy.jit.backend.llvm.runner", "LLVMCPU" + elif backend_name == 'arm': + return "pypy.jit.backend.arm.runner", "ArmCPU" else: raise ProcessorAutodetectError, ( "we have no JIT backend for this cpu: '%s'" % backend_name) From david at codespeak.net Tue Nov 23 09:11:24 2010 From: david at codespeak.net (david at codespeak.net) Date: Tue, 23 Nov 2010 09:11:24 +0100 (CET) Subject: [pypy-svn] r79384 - pypy/branch/arm-backend/pypy/jit/backend/test Message-ID: <20101123081124.25A4B282BD6@codespeak.net> Author: david Date: Tue Nov 23 09:11:22 2010 New Revision: 79384 Modified: pypy/branch/arm-backend/pypy/jit/backend/test/runner_test.py Log: Extend test_assembler_call to test fast and slow path and mark some more tests as requiring floats Modified: pypy/branch/arm-backend/pypy/jit/backend/test/runner_test.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/test/runner_test.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/test/runner_test.py Tue Nov 23 09:11:22 2010 @@ -1549,6 +1549,8 @@ assert values == [1, 10] def test_force_operations_returning_float(self): + if not self.cpu.supports_floats: + py.test.skip("requires floats") values = [] def maybe_force(token, flag): if flag: @@ -1797,6 +1799,7 @@ loop = parse(ops) looptoken = LoopToken() looptoken.outermost_jitdriver_sd = FakeJitDriverSD() + done_number = self.cpu.get_fail_descr_number(loop.operations[-1].getdescr()) self.cpu.compile_loop(loop.inputargs, loop.operations, looptoken) ARGS = [lltype.Signed] * 10 RES = lltype.Signed @@ -1822,7 +1825,23 @@ assert self.cpu.get_latest_value_int(0) == 13 assert called + # test the fast path, which should not call assembler_helper() + del called[:] + self.cpu.done_with_this_frame_int_v = done_number + try: + othertoken = LoopToken() + self.cpu.compile_loop(loop.inputargs, loop.operations, othertoken) + for i in range(10): + self.cpu.set_future_value_int(i, i+1) + res = self.cpu.execute_token(othertoken) + assert self.cpu.get_latest_value_int(0) == 97 + assert not called + finally: + del self.cpu.done_with_this_frame_int_v + def test_assembler_call_float(self): + if not self.cpu.supports_floats: + py.test.skip("requires floats") called = [] def assembler_helper(failindex, virtualizable): assert self.cpu.get_latest_value_float(0) == 1.2 + 3.2 @@ -1909,6 +1928,8 @@ lltype.free(a, flavor='raw') def test_redirect_call_assembler(self): + if not self.cpu.supports_floats: + py.test.skip("requires floats") called = [] def assembler_helper(failindex, virtualizable): assert self.cpu.get_latest_value_float(0) == 1.25 + 3.25 From david at codespeak.net Tue Nov 23 09:25:36 2010 From: david at codespeak.net (david at codespeak.net) Date: Tue, 23 Nov 2010 09:25:36 +0100 (CET) Subject: [pypy-svn] r79385 - pypy/branch/arm-backend/pypy/jit/backend/arm/test Message-ID: <20101123082536.CF7CA5080B@codespeak.net> Author: david Date: Tue Nov 23 09:25:35 2010 New Revision: 79385 Added: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_zll_random.py Log: Random tests for ARM backend with a custom lists of operations to test Added: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_zll_random.py ============================================================================== --- (empty file) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_zll_random.py Tue Nov 23 09:25:35 2010 @@ -0,0 +1,52 @@ +from pypy.jit.backend.detect_cpu import getcpuclass +from pypy.jit.backend.test import test_ll_random +from pypy.jit.backend.test import test_random +from pypy.jit.backend.test.test_ll_random import LLtypeOperationBuilder +from pypy.jit.backend.test.test_random import check_random_function, Random +from pypy.jit.metainterp.resoperation import rop + +# XXX Remove this list from here once all operations work and use the default +# one +OPERATIONS = test_random.OPERATIONS[:] + +#for i in range(4): # make more common +# OPERATIONS.append(test_ll_random.GetFieldOperation(rop.GETFIELD_GC)) +# OPERATIONS.append(test_ll_random.GetFieldOperation(rop.GETFIELD_GC)) +# OPERATIONS.append(test_ll_random.SetFieldOperation(rop.SETFIELD_GC)) +# #OPERATIONS.append(test_ll_random.NewOperation(rop.NEW)) +# #OPERATIONS.append(test_ll_random.NewOperation(rop.NEW_WITH_VTABLE)) +# +# OPERATIONS.append(test_ll_random.GetArrayItemOperation(rop.GETARRAYITEM_GC)) +# OPERATIONS.append(test_ll_random.GetArrayItemOperation(rop.GETARRAYITEM_GC)) +# OPERATIONS.append(test_ll_random.SetArrayItemOperation(rop.SETARRAYITEM_GC)) +# #OPERATIONS.append(test_ll_random.NewArrayOperation(rop.NEW_ARRAY)) +# OPERATIONS.append(test_ll_random.ArrayLenOperation(rop.ARRAYLEN_GC)) +# #OPERATIONS.append(test_ll_random.NewStrOperation(rop.NEWSTR)) +# #OPERATIONS.append(test_ll_random.NewUnicodeOperation(rop.NEWUNICODE)) +# OPERATIONS.append(test_ll_random.StrGetItemOperation(rop.STRGETITEM)) +# OPERATIONS.append(test_ll_random.UnicodeGetItemOperation(rop.UNICODEGETITEM)) +# OPERATIONS.append(test_ll_random.StrSetItemOperation(rop.STRSETITEM)) +# OPERATIONS.append(test_ll_random.UnicodeSetItemOperation(rop.UNICODESETITEM)) +# OPERATIONS.append(test_ll_random.StrLenOperation(rop.STRLEN)) +# OPERATIONS.append(test_ll_random.UnicodeLenOperation(rop.UNICODELEN)) +# +#for i in range(2): + #OPERATIONS.append(test_ll_random.GuardClassOperation(rop.GUARD_CLASS)) + #OPERATIONS.append(test_ll_random.CallOperation(rop.CALL)) + #OPERATIONS.append(test_ll_random.RaisingCallOperation(rop.CALL)) + #OPERATIONS.append(test_ll_random.RaisingCallOperationGuardNoException(rop.CALL)) + #OPERATIONS.append(test_ll_random.RaisingCallOperationWrongGuardException(rop.CALL)) + #OPERATIONS.append(test_ll_random.CallOperationException(rop.CALL)) +#OPERATIONS.append(test_ll_random.GuardNonNullClassOperation(rop.GUARD_NONNULL_CLASS)) + + + +LLtypeOperationBuilder.OPERATIONS = OPERATIONS + +CPU = getcpuclass() + +def test_stress(): + cpu = CPU(None, None) + r = Random() + for i in range(1000): + check_random_function(cpu, LLtypeOperationBuilder, r, i, 1000) From david at codespeak.net Tue Nov 23 10:13:07 2010 From: david at codespeak.net (david at codespeak.net) Date: Tue, 23 Nov 2010 10:13:07 +0100 (CET) Subject: [pypy-svn] r79386 - in pypy/branch/arm-backend: . dotviewer lib-python lib-python/modified-2.5.2/distutils lib-python/modified-2.5.2/test lib_pypy/_ctypes lib_pypy/ctypes_config_cache lib_pypy/ctypes_config_cache/test lib_pypy/pypy_test pypy pypy/annotation pypy/annotation/test pypy/config pypy/config/test pypy/doc pypy/doc/config pypy/interpreter pypy/interpreter/astcompiler/test pypy/interpreter/pyparser pypy/interpreter/pyparser/test pypy/interpreter/test pypy/jit/backend pypy/jit/backend/arm pypy/jit/backend/llgraph pypy/jit/backend/llsupport pypy/jit/backend/llsupport/test pypy/jit/backend/test pypy/jit/backend/x86 pypy/jit/backend/x86/test pypy/jit/backend/x86/tool pypy/jit/codewriter pypy/jit/codewriter/test pypy/jit/metainterp pypy/jit/metainterp/optimizeopt pypy/jit/metainterp/test pypy/jit/tl pypy/jit/tl/spli pypy/jit/tool pypy/jit/tool/test pypy/module/__builtin__ pypy/module/__builtin__/test pypy/module/__pypy__ pypy/module/__pypy__/test pypy/module/_bisect pypy/module/_bisect/test pypy/module/_ffi pypy/module/_ffi/test pypy/module/_rawffi pypy/module/_rawffi/test pypy/module/_socket/test pypy/module/_sre pypy/module/_stackless/test pypy/module/_weakref pypy/module/_winreg pypy/module/array pypy/module/array/benchmark pypy/module/array/test pypy/module/bz2 pypy/module/cpyext pypy/module/cpyext/test pypy/module/gc pypy/module/gc/test pypy/module/imp pypy/module/imp/test pypy/module/parser/test pypy/module/posix pypy/module/pypyjit pypy/module/pypyjit/test pypy/module/rctime pypy/module/signal pypy/module/signal/test pypy/module/sys pypy/module/sys/test pypy/module/test_lib_pypy/ctypes_tests pypy/module/thread pypy/module/thread/test pypy/objspace pypy/objspace/flow pypy/objspace/flow/test pypy/objspace/std pypy/objspace/std/test pypy/objspace/test pypy/rlib pypy/rlib/rsre pypy/rlib/rsre/test pypy/rlib/test pypy/rpython pypy/rpython/lltypesystem pypy/rpython/lltypesystem/test pypy/rpython/memory pypy/rpython/memory/gc pypy/rpython/memory/gc/test pypy/rpython/memory/gctransform pypy/rpython/memory/test pypy/rpython/module pypy/rpython/test pypy/tool pypy/tool/release pypy/tool/release/test pypy/tool/test pypy/translator pypy/translator/c pypy/translator/c/gcc pypy/translator/c/gcc/test pypy/translator/c/gcc/test/darwin64 pypy/translator/c/gcc/test/elf pypy/translator/c/gcc/test/elf64 pypy/translator/c/src pypy/translator/c/test pypy/translator/cli/src pypy/translator/goal pypy/translator/goal/test2 pypy/translator/jvm/test pypy/translator/oosupport/test_template pypy/translator/platform Message-ID: <20101123091307.6875336C220@codespeak.net> Author: david Date: Tue Nov 23 10:12:47 2010 New Revision: 79386 Added: pypy/branch/arm-backend/lib-python/modified-2.5.2/test/test_dict.py - copied unchanged from r79385, pypy/trunk/lib-python/modified-2.5.2/test/test_dict.py pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/dumpcache.py - copied unchanged from r79385, pypy/trunk/lib_pypy/ctypes_config_cache/dumpcache.py pypy/branch/arm-backend/pypy/doc/config/objspace.soabi.txt - copied unchanged from r79385, pypy/trunk/pypy/doc/config/objspace.soabi.txt pypy/branch/arm-backend/pypy/doc/config/objspace.std.withmapdict.txt - copied unchanged from r79385, pypy/trunk/pypy/doc/config/objspace.std.withmapdict.txt pypy/branch/arm-backend/pypy/doc/config/objspace.std.withstrbuf.txt - copied unchanged from r79385, pypy/trunk/pypy/doc/config/objspace.std.withstrbuf.txt pypy/branch/arm-backend/pypy/doc/config/objspace.translationmodules.txt - copied unchanged from r79385, pypy/trunk/pypy/doc/config/objspace.translationmodules.txt pypy/branch/arm-backend/pypy/doc/config/objspace.usemodules._bisect.txt - copied unchanged from r79385, pypy/trunk/pypy/doc/config/objspace.usemodules._bisect.txt pypy/branch/arm-backend/pypy/doc/config/objspace.usemodules._ffi.txt - copied unchanged from r79385, pypy/trunk/pypy/doc/config/objspace.usemodules._ffi.txt pypy/branch/arm-backend/pypy/doc/release-1.4.0.txt - copied unchanged from r79385, pypy/trunk/pypy/doc/release-1.4.0.txt pypy/branch/arm-backend/pypy/doc/release-1.4.0beta.txt - copied unchanged from r79385, pypy/trunk/pypy/doc/release-1.4.0beta.txt pypy/branch/arm-backend/pypy/interpreter/pyparser/autopath.py - copied unchanged from r79385, pypy/trunk/pypy/interpreter/pyparser/autopath.py pypy/branch/arm-backend/pypy/interpreter/pyparser/genpytokenize.py - copied unchanged from r79385, pypy/trunk/pypy/interpreter/pyparser/genpytokenize.py pypy/branch/arm-backend/pypy/interpreter/pyparser/pylexer.py - copied unchanged from r79385, pypy/trunk/pypy/interpreter/pyparser/pylexer.py pypy/branch/arm-backend/pypy/jit/backend/conftest.py - copied unchanged from r79385, pypy/trunk/pypy/jit/backend/conftest.py pypy/branch/arm-backend/pypy/jit/backend/llsupport/ffisupport.py - copied unchanged from r79385, pypy/trunk/pypy/jit/backend/llsupport/ffisupport.py pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_ffisupport.py - copied unchanged from r79385, pypy/trunk/pypy/jit/backend/llsupport/test/test_ffisupport.py pypy/branch/arm-backend/pypy/jit/codewriter/test/test_void_list.py - copied unchanged from r79385, pypy/trunk/pypy/jit/codewriter/test/test_void_list.py pypy/branch/arm-backend/pypy/jit/metainterp/greenfield.py - copied unchanged from r79385, pypy/trunk/pypy/jit/metainterp/greenfield.py pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/fficall.py - copied unchanged from r79385, pypy/trunk/pypy/jit/metainterp/optimizeopt/fficall.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_fficall.py - copied unchanged from r79385, pypy/trunk/pypy/jit/metainterp/test/test_fficall.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_greenfield.py - copied unchanged from r79385, pypy/trunk/pypy/jit/metainterp/test/test_greenfield.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizefficall.py - copied unchanged from r79385, pypy/trunk/pypy/jit/metainterp/test/test_optimizefficall.py pypy/branch/arm-backend/pypy/jit/tl/jittest.py - copied unchanged from r79385, pypy/trunk/pypy/jit/tl/jittest.py pypy/branch/arm-backend/pypy/jit/tool/log-template.gnumeric - copied unchanged from r79385, pypy/trunk/pypy/jit/tool/log-template.gnumeric pypy/branch/arm-backend/pypy/jit/tool/log2gnumeric.py - copied unchanged from r79385, pypy/trunk/pypy/jit/tool/log2gnumeric.py pypy/branch/arm-backend/pypy/jit/tool/loopcounter.py - copied unchanged from r79385, pypy/trunk/pypy/jit/tool/loopcounter.py pypy/branch/arm-backend/pypy/jit/tool/oparser.py - copied unchanged from r79385, pypy/trunk/pypy/jit/tool/oparser.py pypy/branch/arm-backend/pypy/jit/tool/pypytrace.vim - copied unchanged from r79385, pypy/trunk/pypy/jit/tool/pypytrace.vim pypy/branch/arm-backend/pypy/jit/tool/test/test_loopcounter.py - copied unchanged from r79385, pypy/trunk/pypy/jit/tool/test/test_loopcounter.py pypy/branch/arm-backend/pypy/jit/tool/test/test_oparser.py - copied unchanged from r79385, pypy/trunk/pypy/jit/tool/test/test_oparser.py pypy/branch/arm-backend/pypy/module/_bisect/ (props changed) - copied from r79385, pypy/trunk/pypy/module/_bisect/ pypy/branch/arm-backend/pypy/module/_ffi/ (props changed) - copied from r79385, pypy/trunk/pypy/module/_ffi/ pypy/branch/arm-backend/pypy/module/_stackless/test/conftest.py - copied unchanged from r79385, pypy/trunk/pypy/module/_stackless/test/conftest.py pypy/branch/arm-backend/pypy/objspace/std/mapdict.py - copied unchanged from r79385, pypy/trunk/pypy/objspace/std/mapdict.py pypy/branch/arm-backend/pypy/objspace/std/strbufobject.py - copied unchanged from r79385, pypy/trunk/pypy/objspace/std/strbufobject.py pypy/branch/arm-backend/pypy/objspace/std/test/test_mapdict.py - copied unchanged from r79385, pypy/trunk/pypy/objspace/std/test/test_mapdict.py pypy/branch/arm-backend/pypy/objspace/std/test/test_methodcache.py - copied unchanged from r79385, pypy/trunk/pypy/objspace/std/test/test_methodcache.py pypy/branch/arm-backend/pypy/objspace/std/test/test_strbufobject.py - copied unchanged from r79385, pypy/trunk/pypy/objspace/std/test/test_strbufobject.py pypy/branch/arm-backend/pypy/rlib/clibffi.py - copied unchanged from r79385, pypy/trunk/pypy/rlib/clibffi.py pypy/branch/arm-backend/pypy/rlib/libffi.py - copied unchanged from r79385, pypy/trunk/pypy/rlib/libffi.py pypy/branch/arm-backend/pypy/rlib/rerased.py - copied unchanged from r79385, pypy/trunk/pypy/rlib/rerased.py pypy/branch/arm-backend/pypy/rlib/rsre/rsre_jit.py - copied unchanged from r79385, pypy/trunk/pypy/rlib/rsre/rsre_jit.py pypy/branch/arm-backend/pypy/rlib/rsre/test/conftest.py - copied unchanged from r79385, pypy/trunk/pypy/rlib/rsre/test/conftest.py pypy/branch/arm-backend/pypy/rlib/rsre/test/test_zjit.py - copied unchanged from r79385, pypy/trunk/pypy/rlib/rsre/test/test_zjit.py pypy/branch/arm-backend/pypy/rlib/test/test_clibffi.py - copied unchanged from r79385, pypy/trunk/pypy/rlib/test/test_clibffi.py pypy/branch/arm-backend/pypy/rlib/test/test_libffi.py - copied unchanged from r79385, pypy/trunk/pypy/rlib/test/test_libffi.py pypy/branch/arm-backend/pypy/rlib/test/test_rerased.py - copied unchanged from r79385, pypy/trunk/pypy/rlib/test/test_rerased.py pypy/branch/arm-backend/pypy/tool/gcdump.py - copied unchanged from r79385, pypy/trunk/pypy/tool/gcdump.py pypy/branch/arm-backend/pypy/tool/leakfinder.py - copied unchanged from r79385, pypy/trunk/pypy/tool/leakfinder.py pypy/branch/arm-backend/pypy/tool/test/test_leakfinder.py - copied unchanged from r79385, pypy/trunk/pypy/tool/test/test_leakfinder.py pypy/branch/arm-backend/pypy/translator/c/gcc/test/darwin64/ - copied from r79385, pypy/trunk/pypy/translator/c/gcc/test/darwin64/ pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf/track10.s - copied unchanged from r79385, pypy/trunk/pypy/translator/c/gcc/test/elf/track10.s pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf/track11.s - copied unchanged from r79385, pypy/trunk/pypy/translator/c/gcc/test/elf/track11.s pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf64/track_loadconst.s - copied unchanged from r79385, pypy/trunk/pypy/translator/c/gcc/test/elf64/track_loadconst.s pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf64/track_rpyassertfailed.s - copied unchanged from r79385, pypy/trunk/pypy/translator/c/gcc/test/elf64/track_rpyassertfailed.s pypy/branch/arm-backend/pypy/translator/c/src/asm_gcc_x86_64.h - copied unchanged from r79385, pypy/trunk/pypy/translator/c/src/asm_gcc_x86_64.h pypy/branch/arm-backend/pypy/translator/c/src/debug_alloc.h - copied unchanged from r79385, pypy/trunk/pypy/translator/c/src/debug_alloc.h Removed: pypy/branch/arm-backend/pypy/doc/config/objspace.std.withinlineddict.txt pypy/branch/arm-backend/pypy/doc/config/objspace.std.withshadowtracking.txt pypy/branch/arm-backend/pypy/doc/config/objspace.std.withsharingdict.txt pypy/branch/arm-backend/pypy/jit/backend/test/conftest.py pypy/branch/arm-backend/pypy/jit/metainterp/test/oparser.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_oparser.py pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_find.py pypy/branch/arm-backend/pypy/objspace/std/inlinedict.py pypy/branch/arm-backend/pypy/objspace/std/sharingdict.py pypy/branch/arm-backend/pypy/objspace/std/test/test_inlinedict.py pypy/branch/arm-backend/pypy/objspace/std/test/test_shadowtracking.py pypy/branch/arm-backend/pypy/objspace/std/test/test_sharingdict.py pypy/branch/arm-backend/pypy/rpython/rspecialcase.py pypy/branch/arm-backend/pypy/rpython/test/test_rspecialcase.py Modified: pypy/branch/arm-backend/ (props changed) pypy/branch/arm-backend/dotviewer/drawgraph.py pypy/branch/arm-backend/lib-python/conftest.py pypy/branch/arm-backend/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py pypy/branch/arm-backend/lib-python/modified-2.5.2/test/mapping_tests.py pypy/branch/arm-backend/lib-python/modified-2.5.2/test/test_descr.py pypy/branch/arm-backend/lib_pypy/_ctypes/function.py pypy/branch/arm-backend/lib_pypy/_ctypes/structure.py pypy/branch/arm-backend/lib_pypy/_ctypes/union.py pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/ (props changed) pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/hashlib.ctc.py pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/locale.ctc.py pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/pyexpat.ctc.py pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/rebuild.py pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/resource.ctc.py pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/syslog.ctc.py pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/test/test_cache.py pypy/branch/arm-backend/lib_pypy/pypy_test/test_ctypes_support.py pypy/branch/arm-backend/pypy/ (props changed) pypy/branch/arm-backend/pypy/annotation/bookkeeper.py pypy/branch/arm-backend/pypy/annotation/builtin.py pypy/branch/arm-backend/pypy/annotation/classdef.py pypy/branch/arm-backend/pypy/annotation/description.py pypy/branch/arm-backend/pypy/annotation/dictdef.py pypy/branch/arm-backend/pypy/annotation/listdef.py pypy/branch/arm-backend/pypy/annotation/model.py pypy/branch/arm-backend/pypy/annotation/policy.py pypy/branch/arm-backend/pypy/annotation/specialize.py pypy/branch/arm-backend/pypy/annotation/test/test_annrpython.py pypy/branch/arm-backend/pypy/annotation/test/test_model.py pypy/branch/arm-backend/pypy/annotation/unaryop.py pypy/branch/arm-backend/pypy/config/pypyoption.py pypy/branch/arm-backend/pypy/config/test/test_pypyoption.py pypy/branch/arm-backend/pypy/config/translationoption.py pypy/branch/arm-backend/pypy/conftest.py pypy/branch/arm-backend/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt pypy/branch/arm-backend/pypy/doc/docindex.txt pypy/branch/arm-backend/pypy/doc/getting-started.txt pypy/branch/arm-backend/pypy/doc/interpreter-optimizations.txt pypy/branch/arm-backend/pypy/interpreter/argument.py pypy/branch/arm-backend/pypy/interpreter/astcompiler/test/test_compiler.py pypy/branch/arm-backend/pypy/interpreter/baseobjspace.py pypy/branch/arm-backend/pypy/interpreter/executioncontext.py pypy/branch/arm-backend/pypy/interpreter/function.py pypy/branch/arm-backend/pypy/interpreter/gateway.py pypy/branch/arm-backend/pypy/interpreter/pycode.py pypy/branch/arm-backend/pypy/interpreter/pyframe.py pypy/branch/arm-backend/pypy/interpreter/pyopcode.py pypy/branch/arm-backend/pypy/interpreter/pyparser/pytokenize.py pypy/branch/arm-backend/pypy/interpreter/pyparser/test/test_pyparse.py pypy/branch/arm-backend/pypy/interpreter/test/test_argument.py pypy/branch/arm-backend/pypy/interpreter/test/test_executioncontext.py pypy/branch/arm-backend/pypy/interpreter/test/test_function.py pypy/branch/arm-backend/pypy/interpreter/typedef.py pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/detect_cpu.py pypy/branch/arm-backend/pypy/jit/backend/llgraph/llimpl.py pypy/branch/arm-backend/pypy/jit/backend/llgraph/runner.py pypy/branch/arm-backend/pypy/jit/backend/llsupport/descr.py pypy/branch/arm-backend/pypy/jit/backend/llsupport/gc.py pypy/branch/arm-backend/pypy/jit/backend/llsupport/llmodel.py pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py pypy/branch/arm-backend/pypy/jit/backend/llsupport/symbolic.py pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_descr.py pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_gc.py pypy/branch/arm-backend/pypy/jit/backend/model.py pypy/branch/arm-backend/pypy/jit/backend/test/runner_test.py pypy/branch/arm-backend/pypy/jit/backend/test/test_ll_random.py pypy/branch/arm-backend/pypy/jit/backend/test/test_random.py pypy/branch/arm-backend/pypy/jit/backend/x86/assembler.py pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py pypy/branch/arm-backend/pypy/jit/backend/x86/regloc.py pypy/branch/arm-backend/pypy/jit/backend/x86/runner.py pypy/branch/arm-backend/pypy/jit/backend/x86/rx86.py pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_assembler.py pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_gc_integration.py pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_regalloc.py pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_runner.py pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_string.py pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_zrpy_gc.py pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_ztranslation.py pypy/branch/arm-backend/pypy/jit/backend/x86/tool/viewcode.py pypy/branch/arm-backend/pypy/jit/codewriter/assembler.py pypy/branch/arm-backend/pypy/jit/codewriter/call.py pypy/branch/arm-backend/pypy/jit/codewriter/codewriter.py pypy/branch/arm-backend/pypy/jit/codewriter/effectinfo.py pypy/branch/arm-backend/pypy/jit/codewriter/format.py pypy/branch/arm-backend/pypy/jit/codewriter/jtransform.py pypy/branch/arm-backend/pypy/jit/codewriter/support.py pypy/branch/arm-backend/pypy/jit/codewriter/test/test_codewriter.py pypy/branch/arm-backend/pypy/jit/codewriter/test/test_effectinfo.py pypy/branch/arm-backend/pypy/jit/codewriter/test/test_jtransform.py pypy/branch/arm-backend/pypy/jit/codewriter/test/test_list.py pypy/branch/arm-backend/pypy/jit/metainterp/blackhole.py pypy/branch/arm-backend/pypy/jit/metainterp/compile.py pypy/branch/arm-backend/pypy/jit/metainterp/executor.py pypy/branch/arm-backend/pypy/jit/metainterp/graphpage.py pypy/branch/arm-backend/pypy/jit/metainterp/history.py pypy/branch/arm-backend/pypy/jit/metainterp/jitdriver.py pypy/branch/arm-backend/pypy/jit/metainterp/logger.py pypy/branch/arm-backend/pypy/jit/metainterp/optimize_nopspec.py (contents, props changed) pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/__init__.py pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/intbounds.py pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/optimizer.py (contents, props changed) pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/rewrite.py pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/string.py pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/virtualize.py pypy/branch/arm-backend/pypy/jit/metainterp/pyjitpl.py pypy/branch/arm-backend/pypy/jit/metainterp/resoperation.py pypy/branch/arm-backend/pypy/jit/metainterp/resume.py pypy/branch/arm-backend/pypy/jit/metainterp/simple_optimize.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_basic.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_compile.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_del.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_exception.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_jitdriver.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_logger.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_loop_nopspec.py (props changed) pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizefindnode.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizeopt.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_recursive.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resoperation.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resume.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_string.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_virtualref.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_warmspot.py pypy/branch/arm-backend/pypy/jit/metainterp/test/test_ztranslation.py pypy/branch/arm-backend/pypy/jit/metainterp/virtualizable.py pypy/branch/arm-backend/pypy/jit/metainterp/virtualref.py pypy/branch/arm-backend/pypy/jit/metainterp/warmspot.py pypy/branch/arm-backend/pypy/jit/metainterp/warmstate.py pypy/branch/arm-backend/pypy/jit/tl/pypyjit.py pypy/branch/arm-backend/pypy/jit/tl/pypyjit_child.py pypy/branch/arm-backend/pypy/jit/tl/spli/interpreter.py pypy/branch/arm-backend/pypy/jit/tool/loopviewer.py pypy/branch/arm-backend/pypy/jit/tool/pypytrace-mode.el pypy/branch/arm-backend/pypy/jit/tool/showstats.py pypy/branch/arm-backend/pypy/jit/tool/test/test_traceviewer.py pypy/branch/arm-backend/pypy/jit/tool/traceviewer.py pypy/branch/arm-backend/pypy/module/__builtin__/descriptor.py pypy/branch/arm-backend/pypy/module/__builtin__/interp_classobj.py pypy/branch/arm-backend/pypy/module/__builtin__/test/test_classobj.py pypy/branch/arm-backend/pypy/module/__pypy__/__init__.py pypy/branch/arm-backend/pypy/module/__pypy__/interp_magic.py pypy/branch/arm-backend/pypy/module/__pypy__/test/test_special.py pypy/branch/arm-backend/pypy/module/_bisect/test/ (props changed) pypy/branch/arm-backend/pypy/module/_ffi/test/ (props changed) pypy/branch/arm-backend/pypy/module/_rawffi/__init__.py pypy/branch/arm-backend/pypy/module/_rawffi/array.py pypy/branch/arm-backend/pypy/module/_rawffi/callback.py pypy/branch/arm-backend/pypy/module/_rawffi/interp_rawffi.py pypy/branch/arm-backend/pypy/module/_rawffi/structure.py pypy/branch/arm-backend/pypy/module/_rawffi/test/test__rawffi.py pypy/branch/arm-backend/pypy/module/_rawffi/test/test_nested.py pypy/branch/arm-backend/pypy/module/_socket/test/test_sock_app.py pypy/branch/arm-backend/pypy/module/_sre/interp_sre.py pypy/branch/arm-backend/pypy/module/_weakref/interp__weakref.py pypy/branch/arm-backend/pypy/module/_winreg/interp_winreg.py pypy/branch/arm-backend/pypy/module/array/benchmark/Makefile (props changed) pypy/branch/arm-backend/pypy/module/array/benchmark/intimg.c (props changed) pypy/branch/arm-backend/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/branch/arm-backend/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/branch/arm-backend/pypy/module/array/benchmark/loop.c (props changed) pypy/branch/arm-backend/pypy/module/array/benchmark/sum.c (props changed) pypy/branch/arm-backend/pypy/module/array/benchmark/sumtst.c (props changed) pypy/branch/arm-backend/pypy/module/array/benchmark/sumtst.py (props changed) pypy/branch/arm-backend/pypy/module/array/interp_array.py pypy/branch/arm-backend/pypy/module/array/test/test_array_old.py (props changed) pypy/branch/arm-backend/pypy/module/bz2/interp_bz2.py pypy/branch/arm-backend/pypy/module/cpyext/api.py pypy/branch/arm-backend/pypy/module/cpyext/cdatetime.py pypy/branch/arm-backend/pypy/module/cpyext/classobject.py pypy/branch/arm-backend/pypy/module/cpyext/presetup.py pypy/branch/arm-backend/pypy/module/cpyext/pyobject.py pypy/branch/arm-backend/pypy/module/cpyext/test/test_borrow.py pypy/branch/arm-backend/pypy/module/cpyext/test/test_cpyext.py pypy/branch/arm-backend/pypy/module/cpyext/test/test_unicodeobject.py pypy/branch/arm-backend/pypy/module/cpyext/typeobject.py pypy/branch/arm-backend/pypy/module/gc/__init__.py pypy/branch/arm-backend/pypy/module/gc/app_referents.py pypy/branch/arm-backend/pypy/module/gc/interp_gc.py pypy/branch/arm-backend/pypy/module/gc/referents.py pypy/branch/arm-backend/pypy/module/gc/test/test_gc.py pypy/branch/arm-backend/pypy/module/imp/importing.py pypy/branch/arm-backend/pypy/module/imp/interp_imp.py pypy/branch/arm-backend/pypy/module/imp/test/test_app.py pypy/branch/arm-backend/pypy/module/imp/test/test_import.py pypy/branch/arm-backend/pypy/module/parser/test/test_parser.py pypy/branch/arm-backend/pypy/module/posix/interp_posix.py pypy/branch/arm-backend/pypy/module/pypyjit/__init__.py pypy/branch/arm-backend/pypy/module/pypyjit/interp_jit.py pypy/branch/arm-backend/pypy/module/pypyjit/policy.py pypy/branch/arm-backend/pypy/module/pypyjit/test/test_policy.py pypy/branch/arm-backend/pypy/module/pypyjit/test/test_pypy_c.py pypy/branch/arm-backend/pypy/module/rctime/interp_time.py pypy/branch/arm-backend/pypy/module/signal/__init__.py pypy/branch/arm-backend/pypy/module/signal/interp_signal.py pypy/branch/arm-backend/pypy/module/signal/test/test_signal.py pypy/branch/arm-backend/pypy/module/sys/__init__.py pypy/branch/arm-backend/pypy/module/sys/state.py pypy/branch/arm-backend/pypy/module/sys/test/test_initialpath.py pypy/branch/arm-backend/pypy/module/sys/version.py pypy/branch/arm-backend/pypy/module/sys/vm.py pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py pypy/branch/arm-backend/pypy/module/thread/gil.py pypy/branch/arm-backend/pypy/module/thread/ll_thread.py pypy/branch/arm-backend/pypy/module/thread/test/test_gil.py pypy/branch/arm-backend/pypy/objspace/descroperation.py pypy/branch/arm-backend/pypy/objspace/flow/flowcontext.py pypy/branch/arm-backend/pypy/objspace/flow/model.py pypy/branch/arm-backend/pypy/objspace/flow/objspace.py pypy/branch/arm-backend/pypy/objspace/flow/test/test_objspace.py pypy/branch/arm-backend/pypy/objspace/std/callmethod.py pypy/branch/arm-backend/pypy/objspace/std/celldict.py pypy/branch/arm-backend/pypy/objspace/std/dictmultiobject.py pypy/branch/arm-backend/pypy/objspace/std/dicttype.py pypy/branch/arm-backend/pypy/objspace/std/fake.py pypy/branch/arm-backend/pypy/objspace/std/frozensettype.py pypy/branch/arm-backend/pypy/objspace/std/inttype.py pypy/branch/arm-backend/pypy/objspace/std/listobject.py pypy/branch/arm-backend/pypy/objspace/std/longtype.py pypy/branch/arm-backend/pypy/objspace/std/model.py pypy/branch/arm-backend/pypy/objspace/std/objspace.py pypy/branch/arm-backend/pypy/objspace/std/proxyobject.py pypy/branch/arm-backend/pypy/objspace/std/setobject.py pypy/branch/arm-backend/pypy/objspace/std/settype.py pypy/branch/arm-backend/pypy/objspace/std/smallintobject.py pypy/branch/arm-backend/pypy/objspace/std/stringobject.py pypy/branch/arm-backend/pypy/objspace/std/stringtype.py pypy/branch/arm-backend/pypy/objspace/std/strutil.py pypy/branch/arm-backend/pypy/objspace/std/test/test_dictmultiobject.py pypy/branch/arm-backend/pypy/objspace/std/test/test_longobject.py pypy/branch/arm-backend/pypy/objspace/std/test/test_setobject.py pypy/branch/arm-backend/pypy/objspace/std/test/test_strutil.py pypy/branch/arm-backend/pypy/objspace/std/test/test_typeobject.py pypy/branch/arm-backend/pypy/objspace/std/test/test_versionedtype.py pypy/branch/arm-backend/pypy/objspace/std/typeobject.py pypy/branch/arm-backend/pypy/objspace/test/test_descroperation.py pypy/branch/arm-backend/pypy/rlib/_rsocket_rffi.py pypy/branch/arm-backend/pypy/rlib/jit.py pypy/branch/arm-backend/pypy/rlib/objectmodel.py pypy/branch/arm-backend/pypy/rlib/rarithmetic.py pypy/branch/arm-backend/pypy/rlib/rbigint.py pypy/branch/arm-backend/pypy/rlib/rgc.py pypy/branch/arm-backend/pypy/rlib/rmmap.py pypy/branch/arm-backend/pypy/rlib/rsre/rsre_char.py pypy/branch/arm-backend/pypy/rlib/rsre/rsre_core.py pypy/branch/arm-backend/pypy/rlib/rsre/test/test_match.py pypy/branch/arm-backend/pypy/rlib/rstring.py pypy/branch/arm-backend/pypy/rlib/streamio.py pypy/branch/arm-backend/pypy/rlib/test/test_jit.py pypy/branch/arm-backend/pypy/rlib/test/test_rarithmetic.py pypy/branch/arm-backend/pypy/rlib/test/test_rbigint.py pypy/branch/arm-backend/pypy/rlib/test/test_rdynload.py pypy/branch/arm-backend/pypy/rlib/test/test_rsocket.py pypy/branch/arm-backend/pypy/rlib/test/test_rstring.py pypy/branch/arm-backend/pypy/rlib/test/test_runicode.py pypy/branch/arm-backend/pypy/rlib/test/test_rzlib.py pypy/branch/arm-backend/pypy/rpython/annlowlevel.py pypy/branch/arm-backend/pypy/rpython/llinterp.py pypy/branch/arm-backend/pypy/rpython/lltypesystem/ll2ctypes.py pypy/branch/arm-backend/pypy/rpython/lltypesystem/llarena.py pypy/branch/arm-backend/pypy/rpython/lltypesystem/llmemory.py pypy/branch/arm-backend/pypy/rpython/lltypesystem/lloperation.py pypy/branch/arm-backend/pypy/rpython/lltypesystem/lltype.py pypy/branch/arm-backend/pypy/rpython/lltypesystem/rbuilder.py pypy/branch/arm-backend/pypy/rpython/lltypesystem/rclass.py pypy/branch/arm-backend/pypy/rpython/lltypesystem/rdict.py pypy/branch/arm-backend/pypy/rpython/lltypesystem/rffi.py pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_ll2ctypes.py pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_llmemory.py pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_lltype.py pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_rffi.py pypy/branch/arm-backend/pypy/rpython/memory/gc/base.py pypy/branch/arm-backend/pypy/rpython/memory/gc/generation.py pypy/branch/arm-backend/pypy/rpython/memory/gc/inspector.py pypy/branch/arm-backend/pypy/rpython/memory/gc/markcompact.py pypy/branch/arm-backend/pypy/rpython/memory/gc/marksweep.py pypy/branch/arm-backend/pypy/rpython/memory/gc/minimark.py pypy/branch/arm-backend/pypy/rpython/memory/gc/minimarkpage.py pypy/branch/arm-backend/pypy/rpython/memory/gc/semispace.py pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_direct.py pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_minimark.py pypy/branch/arm-backend/pypy/rpython/memory/gctransform/asmgcroot.py pypy/branch/arm-backend/pypy/rpython/memory/gctransform/framework.py pypy/branch/arm-backend/pypy/rpython/memory/gctransform/transform.py pypy/branch/arm-backend/pypy/rpython/memory/gcwrapper.py pypy/branch/arm-backend/pypy/rpython/memory/support.py pypy/branch/arm-backend/pypy/rpython/memory/test/test_gc.py pypy/branch/arm-backend/pypy/rpython/memory/test/test_transformed_gc.py pypy/branch/arm-backend/pypy/rpython/module/ll_time.py pypy/branch/arm-backend/pypy/rpython/rbuilder.py pypy/branch/arm-backend/pypy/rpython/rbuiltin.py pypy/branch/arm-backend/pypy/rpython/rmodel.py pypy/branch/arm-backend/pypy/rpython/rpbc.py pypy/branch/arm-backend/pypy/rpython/rtyper.py pypy/branch/arm-backend/pypy/rpython/test/test_llinterp.py pypy/branch/arm-backend/pypy/rpython/test/test_nongc.py pypy/branch/arm-backend/pypy/rpython/test/test_rbuilder.py pypy/branch/arm-backend/pypy/rpython/test/test_rclass.py pypy/branch/arm-backend/pypy/rpython/test/test_rdict.py pypy/branch/arm-backend/pypy/rpython/test/test_rpbc.py pypy/branch/arm-backend/pypy/rpython/test/test_rptr.py pypy/branch/arm-backend/pypy/tool/alarm.py pypy/branch/arm-backend/pypy/tool/readdictinfo.py pypy/branch/arm-backend/pypy/tool/release/ (props changed) pypy/branch/arm-backend/pypy/tool/release/__init__.py (props changed) pypy/branch/arm-backend/pypy/tool/release/force-builds.py (contents, props changed) pypy/branch/arm-backend/pypy/tool/release/make_release.py (contents, props changed) pypy/branch/arm-backend/pypy/tool/release/package.py pypy/branch/arm-backend/pypy/tool/release/test/ (props changed) pypy/branch/arm-backend/pypy/tool/release/test/__init__.py (props changed) pypy/branch/arm-backend/pypy/tool/release/test/test_make_release.py (props changed) pypy/branch/arm-backend/pypy/tool/release/test/test_package.py pypy/branch/arm-backend/pypy/tool/rundictbenchmarks.py pypy/branch/arm-backend/pypy/tool/statistic_irc_log.py pypy/branch/arm-backend/pypy/tool/terminal.py pypy/branch/arm-backend/pypy/tool/watchdog.py pypy/branch/arm-backend/pypy/tool/watchdog_nt.py pypy/branch/arm-backend/pypy/translator/c/funcgen.py pypy/branch/arm-backend/pypy/translator/c/gcc/instruction.py pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf/track5.s pypy/branch/arm-backend/pypy/translator/c/gcc/test/test_trackgcroot.py pypy/branch/arm-backend/pypy/translator/c/gcc/trackgcroot.py pypy/branch/arm-backend/pypy/translator/c/genc.py pypy/branch/arm-backend/pypy/translator/c/node.py pypy/branch/arm-backend/pypy/translator/c/src/asm_gcc_x86.h pypy/branch/arm-backend/pypy/translator/c/src/debug_print.h pypy/branch/arm-backend/pypy/translator/c/src/g_include.h pypy/branch/arm-backend/pypy/translator/c/src/int.h pypy/branch/arm-backend/pypy/translator/c/src/main.h pypy/branch/arm-backend/pypy/translator/c/src/signals.h pypy/branch/arm-backend/pypy/translator/c/src/stack.h pypy/branch/arm-backend/pypy/translator/c/test/test_genc.py pypy/branch/arm-backend/pypy/translator/c/test/test_lltyped.py pypy/branch/arm-backend/pypy/translator/c/test/test_newgc.py pypy/branch/arm-backend/pypy/translator/c/test/test_standalone.py pypy/branch/arm-backend/pypy/translator/c/test/test_typed.py pypy/branch/arm-backend/pypy/translator/cli/src/pypylib.cs pypy/branch/arm-backend/pypy/translator/driver.py pypy/branch/arm-backend/pypy/translator/goal/ann_override.py pypy/branch/arm-backend/pypy/translator/goal/app_main.py pypy/branch/arm-backend/pypy/translator/goal/targetpypystandalone.py pypy/branch/arm-backend/pypy/translator/goal/test2/test_app_main.py pypy/branch/arm-backend/pypy/translator/goal/translate.py pypy/branch/arm-backend/pypy/translator/jvm/test/test_class.py pypy/branch/arm-backend/pypy/translator/oosupport/test_template/class_.py pypy/branch/arm-backend/pypy/translator/platform/darwin.py pypy/branch/arm-backend/pypy/translator/platform/linux.py pypy/branch/arm-backend/pypy/translator/platform/posix.py pypy/branch/arm-backend/pypy/translator/transform.py Log: merged trunk changes r77502 through r79385 into arm backend branch Modified: pypy/branch/arm-backend/dotviewer/drawgraph.py ============================================================================== --- pypy/branch/arm-backend/dotviewer/drawgraph.py (original) +++ pypy/branch/arm-backend/dotviewer/drawgraph.py Tue Nov 23 10:12:47 2010 @@ -423,20 +423,43 @@ else: for line in lines: raw_line = line.replace('\\l','').replace('\r','') or ' ' - img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor) - w, h = img.get_size() - if w>wmax: wmax = w - if raw_line.strip(): - if line.endswith('\\l'): - def cmd(img=img, y=hmax): - img.draw(xleft, ytop+y) - elif line.endswith('\r'): - def cmd(img=img, y=hmax, w=w): - img.draw(xright-w, ytop+y) - else: - def cmd(img=img, y=hmax, w=w): - img.draw(xcenter-w//2, ytop+y) + if '\f' in raw_line: # grayed out parts of the line + imgs = [] + graytext = True + h = 16 + w_total = 0 + for linepart in raw_line.split('\f'): + graytext = not graytext + if not linepart.strip(): + continue + if graytext: + fgcolor = (128, 160, 160) + else: + fgcolor = (0, 0, 0) + img = TextSnippet(self, linepart, fgcolor, bgcolor) + imgs.append((w_total, img)) + w, h = img.get_size() + w_total += w + if w_total > wmax: wmax = w_total + def cmd(imgs=imgs, y=hmax): + for x, img in imgs: + img.draw(xleft+x, ytop+y) commands.append(cmd) + else: + img = TextSnippet(self, raw_line, (0, 0, 0), bgcolor) + w, h = img.get_size() + if w>wmax: wmax = w + if raw_line.strip(): + if line.endswith('\\l'): + def cmd(img=img, y=hmax): + img.draw(xleft, ytop+y) + elif line.endswith('\r'): + def cmd(img=img, y=hmax, w=w): + img.draw(xright-w, ytop+y) + else: + def cmd(img=img, y=hmax, w=w): + img.draw(xcenter-w//2, ytop+y) + commands.append(cmd) hmax += h #hmax += 8 Modified: pypy/branch/arm-backend/lib-python/conftest.py ============================================================================== --- pypy/branch/arm-backend/lib-python/conftest.py (original) +++ pypy/branch/arm-backend/lib-python/conftest.py Tue Nov 23 10:12:47 2010 @@ -144,7 +144,7 @@ RegrTest('test_binhex.py'), RegrTest('test_binop.py', core=True), - RegrTest('test_bisect.py', core=True), + RegrTest('test_bisect.py', core=True, usemodules='_bisect'), RegrTest('test_bool.py', core=True), RegrTest('test_bsddb.py', skip="unsupported extension module"), RegrTest('test_bsddb185.py', skip="unsupported extension module"), @@ -464,7 +464,12 @@ RegrTest('test_coding.py'), RegrTest('test_complex_args.py'), RegrTest('test_contextlib.py', usemodules="thread"), - RegrTest('test_ctypes.py', usemodules="_rawffi"), + # we skip test ctypes, since we adapted it massively in order + # to test what we want to support. There are real failures, + # but it's about missing features that we don't want to support + # now + RegrTest('test_ctypes.py', usemodules="_rawffi", + skip="missing features that we don't want to support now"), RegrTest('test_defaultdict.py'), RegrTest('test_email_renamed.py'), RegrTest('test_exception_variations.py'), Modified: pypy/branch/arm-backend/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py ============================================================================== --- pypy/branch/arm-backend/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py (original) +++ pypy/branch/arm-backend/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py Tue Nov 23 10:12:47 2010 @@ -3,6 +3,7 @@ import sys import os +import imp from distutils.errors import DistutilsPlatformError @@ -47,11 +48,17 @@ _config_vars = None +def _get_so_extension(): + for ext, mod, typ in imp.get_suffixes(): + if typ == imp.C_EXTENSION: + return ext + def _init_posix(): """Initialize the module as appropriate for POSIX systems.""" g = {} g['EXE'] = "" - g['SO'] = ".so" + g['SO'] = _get_so_extension() or ".so" + g['SOABI'] = g['SO'].rsplit('.')[0] global _config_vars _config_vars = g @@ -61,7 +68,8 @@ """Initialize the module as appropriate for NT""" g = {} g['EXE'] = ".exe" - g['SO'] = ".pyd" + g['SO'] = _get_so_extension() or ".pyd" + g['SOABI'] = g['SO'].rsplit('.')[0] global _config_vars _config_vars = g Modified: pypy/branch/arm-backend/lib-python/modified-2.5.2/test/mapping_tests.py ============================================================================== --- pypy/branch/arm-backend/lib-python/modified-2.5.2/test/mapping_tests.py (original) +++ pypy/branch/arm-backend/lib-python/modified-2.5.2/test/mapping_tests.py Tue Nov 23 10:12:47 2010 @@ -1,6 +1,7 @@ # tests common to dict and UserDict import unittest import UserDict +from test import test_support class BasicTestMappingProtocol(unittest.TestCase): @@ -525,7 +526,8 @@ self.assertEqual(va, int(ka)) kb, vb = tb = b.popitem() self.assertEqual(vb, int(kb)) - self.assert_(not(copymode < 0 and ta != tb)) + if test_support.check_impl_detail(): + self.assert_(not(copymode < 0 and ta != tb)) self.assert_(not a) self.assert_(not b) Modified: pypy/branch/arm-backend/lib-python/modified-2.5.2/test/test_descr.py ============================================================================== --- pypy/branch/arm-backend/lib-python/modified-2.5.2/test/test_descr.py (original) +++ pypy/branch/arm-backend/lib-python/modified-2.5.2/test/test_descr.py Tue Nov 23 10:12:47 2010 @@ -2028,7 +2028,9 @@ except TypeError, msg: verify(str(msg).find("weak reference") >= 0) else: - verify(0, "weakref.ref(no) should be illegal") + # in PyPy it is (sometimes) possible to take a weakref here + #verify(0, "weakref.ref(no) should be illegal") + pass class Weak(object): __slots__ = ['foo', '__weakref__'] yes = Weak() Modified: pypy/branch/arm-backend/lib_pypy/_ctypes/function.py ============================================================================== --- pypy/branch/arm-backend/lib_pypy/_ctypes/function.py (original) +++ pypy/branch/arm-backend/lib_pypy/_ctypes/function.py Tue Nov 23 10:12:47 2010 @@ -171,7 +171,7 @@ return self._build_result(restype, resbuffer, argtypes, argsandobjs) def _getfuncptr(self, argtypes, restype, thisarg=None): - if self._ptr is not None: + if self._ptr is not None and argtypes is self._argtypes_: return self._ptr if restype is None or not isinstance(restype, _CDataMeta): import ctypes Modified: pypy/branch/arm-backend/lib_pypy/_ctypes/structure.py ============================================================================== --- pypy/branch/arm-backend/lib_pypy/_ctypes/structure.py (original) +++ pypy/branch/arm-backend/lib_pypy/_ctypes/structure.py Tue Nov 23 10:12:47 2010 @@ -7,7 +7,7 @@ def round_up(size, alignment): return (size + alignment - 1) & -alignment -def size_alignment_pos(fields): +def size_alignment_pos(fields, is_union=False): import ctypes size = 0 alignment = 1 @@ -15,14 +15,19 @@ for fieldname, ctype in fields: fieldsize = ctypes.sizeof(ctype) fieldalignment = ctypes.alignment(ctype) - size = round_up(size, fieldalignment) alignment = max(alignment, fieldalignment) - pos.append(size) - size += fieldsize + if is_union: + pos.append(0) + size = max(size, fieldsize) + else: + size = round_up(size, fieldalignment) + pos.append(size) + size += fieldsize size = round_up(size, alignment) return size, alignment, pos -def names_and_fields(_fields_, superclass, zero_offset=False, anon=None): +def names_and_fields(_fields_, superclass, zero_offset=False, anon=None, + is_union=False): if isinstance(_fields_, tuple): _fields_ = list(_fields_) for _, tp in _fields_: @@ -36,7 +41,7 @@ rawfields = [(name, ctype._ffishape) for name, ctype in all_fields] if not zero_offset: - _, _, pos = size_alignment_pos(all_fields) + _, _, pos = size_alignment_pos(all_fields, is_union) else: pos = [0] * len(all_fields) fields = {} @@ -73,8 +78,8 @@ # ________________________________________________________________ -def _set_shape(tp, rawfields): - tp._ffistruct = _rawffi.Structure(rawfields) +def _set_shape(tp, rawfields, is_union=False): + tp._ffistruct = _rawffi.Structure(rawfields, is_union) tp._ffiargshape = tp._ffishape = (tp._ffistruct, 1) tp._fficompositesize = tp._ffistruct.size @@ -92,13 +97,14 @@ raise AttributeError("Structure or union cannot contain itself") self._names, rawfields, self._fieldtypes = names_and_fields( value, self.__bases__[0], False, - self.__dict__.get('_anonymous_', None)) + self.__dict__.get('_anonymous_', None), self._is_union) _CDataMeta.__setattr__(self, '_fields_', value) - _set_shape(self, rawfields) + _set_shape(self, rawfields, self._is_union) return _CDataMeta.__setattr__(self, name, value) -class StructureMeta(_CDataMeta): +class StructOrUnionMeta(_CDataMeta): + def __new__(self, name, cls, typedict): res = type.__new__(self, name, cls, typedict) if '_fields_' in typedict: @@ -109,8 +115,8 @@ raise AttributeError("Anonymous field not found") res._names, rawfields, res._fieldtypes = names_and_fields( typedict['_fields_'], cls[0], False, - typedict.get('_anonymous_', None)) - _set_shape(res, rawfields) + typedict.get('_anonymous_', None), self._is_union) + _set_shape(res, rawfields, self._is_union) return res @@ -150,8 +156,8 @@ res.__dict__['_index'] = -1 return res -class Structure(_CData): - __metaclass__ = StructureMeta +class StructOrUnion(_CData): + __metaclass__ = StructOrUnionMeta def __new__(cls, *args, **kwds): if not hasattr(cls, '_ffistruct'): @@ -213,3 +219,10 @@ def _get_buffer_value(self): return self._buffer.buffer + + +class StructureMeta(StructOrUnionMeta): + _is_union = False + +class Structure(StructOrUnion): + __metaclass__ = StructureMeta Modified: pypy/branch/arm-backend/lib_pypy/_ctypes/union.py ============================================================================== --- pypy/branch/arm-backend/lib_pypy/_ctypes/union.py (original) +++ pypy/branch/arm-backend/lib_pypy/_ctypes/union.py Tue Nov 23 10:12:47 2010 @@ -1,118 +1,7 @@ +from _ctypes import structure +class UnionMeta(structure.StructOrUnionMeta): + _is_union = True -import _rawffi -from _ctypes.basics import _CData, _CDataMeta, store_reference, keepalive_key -from _ctypes.basics import ensure_objects -from _ctypes.structure import round_up, names_and_fields, struct_getattr,\ - struct_setattr - - -def _set_shape(tp): - size = tp._sizeofinstances() - alignment = tp._alignmentofinstances() - tp._ffiopaque = _rawffi.Structure((size, alignment)) # opaque - tp._ffiargshape = tp._ffishape = (tp._ffiopaque, 1) - tp._fficompositesize = tp._ffiopaque.size - # we need to create an array of size one for each - # of our elements - tp._ffiarrays = {} - for name, field in tp._fieldtypes.iteritems(): - tp._ffiarrays[name] = _rawffi.Array(field.ctype._ffishape) - -class UnionMeta(_CDataMeta): - def __new__(self, name, cls, typedict): - res = type.__new__(self, name, cls, typedict) - if '_fields_' in typedict: - res._names, rawfields, res._fieldtypes = names_and_fields( - typedict['_fields_'], cls[0], True, - typedict.get('_anonymous_', None)) - _set_shape(res) - - def __init__(self): # don't allow arguments by now - if not hasattr(self, '_ffiarrays'): - raise TypeError("Cannot instantiate union, has no type") - # malloc size - size = self.__class__._sizeofinstances() - self.__dict__['_objects'] = {} - self.__dict__['_buffer'] = self._ffiopaque(autofree=True) - res.__init__ = __init__ - return res - - def _sizeofinstances(self): - if not hasattr(self, '_size_'): - self._size_ = max([field.size for field in - self._fieldtypes.values()] + [0]) - return self._size_ - - def _alignmentofinstances(self): - from ctypes import alignment - if not hasattr(self, '_alignment_'): - self._alignment_ = max([alignment(field.ctype) for field in - self._fieldtypes.values()] + [1]) - return self._alignment_ - - __getattr__ = struct_getattr - - def __setattr__(self, name, value): - if name == '_fields_': - if self.__dict__.get('_fields_', None): - raise AttributeError("_fields_ is final") - if self in [v for k, v in value]: - raise AttributeError("Union cannot contain itself") - self._names, rawfields, self._fieldtypes = names_and_fields( - value, self.__bases__[0], True, - self.__dict__.get('_anonymous_', None)) - _CDataMeta.__setattr__(self, '_fields_', value) - _set_shape(self) - _CDataMeta.__setattr__(self, name, value) - - def _CData_output(self, resarray, base=None, index=-1): - res = self.__new__(self) - ffiopaque = self._ffiopaque.fromaddress(resarray.buffer) - res.__dict__['_buffer'] = ffiopaque - res.__dict__['_base'] = base - res.__dict__['_index'] = index - return res - - def _CData_retval(self, resbuffer): - res = self.__new__(self) - res.__dict__['_buffer'] = resbuffer - res.__dict__['_base'] = None - res.__dict__['_index'] = -1 - return res - - -class Union(_CData): +class Union(structure.StructOrUnion): __metaclass__ = UnionMeta - - def __getattr__(self, name): - try: - field = self._fieldtypes[name] - except KeyError: - raise AttributeError(name) - fieldtype = field.ctype - val = self._ffiarrays[name].fromaddress(self._buffer.buffer, 1) - offset = field.num - return fieldtype._CData_output(val, self, offset) - - def __setattr__(self, name, value): - try: - field = self._fieldtypes[name] - except KeyError: - raise AttributeError(name) - fieldtype = field.ctype - cobj = fieldtype.from_param(value) - if ensure_objects(cobj) is not None: - key = keepalive_key(field.num) - store_reference(self, key, cobj._objects) - arg = cobj._get_buffer_value() - if fieldtype._fficompositesize is not None: - from ctypes import memmove - dest = self._buffer.buffer - memmove(dest, arg, fieldtype._fficompositesize) - else: - buf = self._ffiarrays[name].fromaddress(self._buffer.buffer, 1) - buf[0] = arg - - def _get_buffer_value(self): - return self._buffer.buffer Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/hashlib.ctc.py ============================================================================== --- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/hashlib.ctc.py (original) +++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/hashlib.ctc.py Tue Nov 23 10:12:47 2010 @@ -4,7 +4,8 @@ """ from ctypes import * -from ctypes_configure import configure, dumpcache +from ctypes_configure import configure +import dumpcache class CConfig: @@ -17,4 +18,4 @@ [('digest', c_void_p)]) config = configure.configure(CConfig) -dumpcache.dumpcache(__file__, '_hashlib_cache.py', config) +dumpcache.dumpcache2('hashlib', config) Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/locale.ctc.py ============================================================================== --- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/locale.ctc.py (original) +++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/locale.ctc.py Tue Nov 23 10:12:47 2010 @@ -5,7 +5,7 @@ from ctypes_configure.configure import (configure, ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger, SimpleType, check_eci) -from ctypes_configure.dumpcache import dumpcache +import dumpcache # ____________________________________________________________ @@ -70,4 +70,4 @@ config['ALL_CONSTANTS'] = tuple(_CONSTANTS) config['HAS_LANGINFO'] = HAS_LANGINFO -dumpcache(__file__, '_locale_cache.py', config) +dumpcache.dumpcache2('locale', config) Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/pyexpat.ctc.py ============================================================================== --- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/pyexpat.ctc.py (original) +++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/pyexpat.ctc.py Tue Nov 23 10:12:47 2010 @@ -5,7 +5,8 @@ import ctypes from ctypes import c_char_p, c_int, c_void_p, c_char -from ctypes_configure import configure, dumpcache +from ctypes_configure import configure +import dumpcache class CConfigure: @@ -41,4 +42,4 @@ config = configure.configure(CConfigure) -dumpcache.dumpcache(__file__, '_pyexpat_cache.py', config) +dumpcache.dumpcache2('pyexpat', config) Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/rebuild.py ============================================================================== --- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/rebuild.py (original) +++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/rebuild.py Tue Nov 23 10:12:47 2010 @@ -31,10 +31,25 @@ sys.path[:] = path def try_rebuild(): + from pypy.jit.backend import detect_cpu + model = detect_cpu.autodetect_main_model_and_size() + # remove the files '_*_model_.py' + left = {} + for p in os.listdir(_dirpath): + if p.startswith('_') and (p.endswith('_%s_.py' % model) or + p.endswith('_%s_.pyc' % model)): + os.unlink(os.path.join(_dirpath, p)) + elif p.startswith('_') and (p.endswith('_.py') or + p.endswith('_.pyc')): + for i in range(2, len(p)-4): + left[p[:i]] = True + # remove the files '_*_cache.py' if there is no '_*_*_.py' left around for p in os.listdir(_dirpath): if p.startswith('_') and (p.endswith('_cache.py') or p.endswith('_cache.pyc')): - os.unlink(os.path.join(_dirpath, p)) + if p[:-9] not in left: + os.unlink(os.path.join(_dirpath, p)) + # for p in os.listdir(_dirpath): if p.endswith('.ctc.py'): try: Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/resource.ctc.py ============================================================================== --- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/resource.ctc.py (original) +++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/resource.ctc.py Tue Nov 23 10:12:47 2010 @@ -4,7 +4,7 @@ """ from ctypes import sizeof -from ctypes_configure.dumpcache import dumpcache +import dumpcache from ctypes_configure.configure import (configure, ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger, SimpleType) @@ -58,4 +58,4 @@ del config[key] config['ALL_CONSTANTS'] = _CONSTANTS + tuple(optional_constants) -dumpcache(__file__, '_resource_cache.py', config) +dumpcache.dumpcache2('resource', config) Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/syslog.ctc.py ============================================================================== --- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/syslog.ctc.py (original) +++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/syslog.ctc.py Tue Nov 23 10:12:47 2010 @@ -5,7 +5,7 @@ from ctypes_configure.configure import (configure, ExternalCompilationInfo, ConstantInteger, DefinedConstantInteger) -from ctypes_configure.dumpcache import dumpcache +import dumpcache _CONSTANTS = ( @@ -72,4 +72,4 @@ all_constants = config.keys() all_constants.sort() config['ALL_CONSTANTS'] = tuple(all_constants) -dumpcache(__file__, '_syslog_cache.py', config) +dumpcache.dumpcache2('syslog', config) Modified: pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/test/test_cache.py ============================================================================== --- pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/test/test_cache.py (original) +++ pypy/branch/arm-backend/lib_pypy/ctypes_config_cache/test/test_cache.py Tue Nov 23 10:12:47 2010 @@ -7,19 +7,26 @@ def run(filename, outputname): filepath = dirpath.join(filename) - tmpdir = udir.ensure('testcache-' + filename, dir=True) - outputpath = tmpdir.join(outputname) - d = {'__file__': str(outputpath)} + tmpdir = udir.ensure('testcache-' + os.path.splitext(filename)[0], + dir=True) + tmpdir.join('dumpcache.py').write(dirpath.join('dumpcache.py').read()) path = sys.path[:] try: - sys.path.insert(0, str(dirpath)) - execfile(str(filepath), d) + sys.path.insert(0, str(tmpdir)) + execfile(str(filepath), {}) finally: sys.path[:] = path + sys.modules.pop('dumpcache', None) # + outputpath = tmpdir.join(outputname) assert outputpath.check(exists=1) - d = {} - execfile(str(outputpath), d) + modname = os.path.splitext(outputname)[0] + try: + sys.path.insert(0, str(tmpdir)) + d = {} + execfile(str(outputpath), d) + finally: + sys.path[:] = path return d Modified: pypy/branch/arm-backend/lib_pypy/pypy_test/test_ctypes_support.py ============================================================================== --- pypy/branch/arm-backend/lib_pypy/pypy_test/test_ctypes_support.py (original) +++ pypy/branch/arm-backend/lib_pypy/pypy_test/test_ctypes_support.py Tue Nov 23 10:12:47 2010 @@ -22,12 +22,11 @@ assert get_errno() == 0 def test_argument_conversion_and_checks(): - import ctypes - libc = ctypes.cdll.LoadLibrary("libc.so.6") - libc.strlen.argtypes = ctypes.c_char_p, - libc.strlen.restype = ctypes.c_size_t - assert libc.strlen("eggs") == 4 - + strlen = standard_c_lib.strlen + strlen.argtypes = [c_char_p] + strlen.restype = c_size_t + assert strlen("eggs") == 4 + # Should raise ArgumentError, not segfault - py.test.raises(ctypes.ArgumentError, libc.strlen, False) + py.test.raises(ArgumentError, strlen, False) Modified: pypy/branch/arm-backend/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/branch/arm-backend/pypy/annotation/bookkeeper.py (original) +++ pypy/branch/arm-backend/pypy/annotation/bookkeeper.py Tue Nov 23 10:12:47 2010 @@ -24,7 +24,7 @@ from pypy.rpython import extregistry from pypy.tool.identity_dict import identity_dict -class Stats: +class Stats(object): def __init__(self, bookkeeper): self.bookkeeper = bookkeeper @@ -87,13 +87,6 @@ else: return obj.knowntype.__name__ - def consider_tuple_iter(self, tup): - ctxt = "[%s]" % sys._getframe(4).f_code.co_name - if tup.is_constant(): - return ctxt, tup.const - else: - return ctxt, tuple([self.typerepr(x) for x in tup.items]) - def consider_tuple_random_getitem(self, tup): return tuple([self.typerepr(x) for x in tup.items]) @@ -137,7 +130,7 @@ def consider_dict_delitem(self, dic): return dic -class Bookkeeper: +class Bookkeeper(object): """The log of choices that have been made while analysing the operations. It ensures that the same 'choice objects' will be returned if we ask again during reflowing. Like ExecutionContext, there is an implicit @@ -736,7 +729,7 @@ return True # for parsing call arguments -class RPythonCallsSpace: +class RPythonCallsSpace(object): """Pseudo Object Space providing almost no real operation. For the Arguments class: if it really needs other operations, it means that the call pattern is too complex for R-Python. Modified: pypy/branch/arm-backend/pypy/annotation/builtin.py ============================================================================== --- pypy/branch/arm-backend/pypy/annotation/builtin.py (original) +++ pypy/branch/arm-backend/pypy/annotation/builtin.py Tue Nov 23 10:12:47 2010 @@ -423,7 +423,7 @@ from pypy.annotation.model import SomePtr from pypy.rpython.lltypesystem import lltype -def malloc(s_T, s_n=None, s_flavor=None, s_zero=None): +def malloc(s_T, s_n=None, s_flavor=None, s_zero=None, s_track_allocation=None): assert (s_n is None or s_n.knowntype == int or issubclass(s_n.knowntype, pypy.rlib.rarithmetic.base_int)) assert s_T.is_constant() @@ -438,19 +438,24 @@ r = SomePtr(lltype.typeOf(p)) else: assert s_flavor.is_constant() + assert s_track_allocation is None or s_track_allocation.is_constant() # not sure how to call malloc() for the example 'p' in the # presence of s_extraargs r = SomePtr(lltype.Ptr(s_T.const)) return r -def free(s_p, s_flavor): +def free(s_p, s_flavor, s_track_allocation=None): assert s_flavor.is_constant() + assert s_track_allocation is None or s_track_allocation.is_constant() # same problem as in malloc(): some flavors are not easy to # malloc-by-example #T = s_p.ll_ptrtype.TO #p = lltype.malloc(T, flavor=s_flavor.const) #lltype.free(p, flavor=s_flavor.const) +def render_immortal(s_p, s_track_allocation=None): + assert s_track_allocation is None or s_track_allocation.is_constant() + def typeOf(s_val): lltype = annotation_to_lltype(s_val, info="in typeOf(): ") return immutablevalue(lltype) @@ -518,6 +523,7 @@ BUILTIN_ANALYZERS[lltype.malloc] = malloc BUILTIN_ANALYZERS[lltype.free] = free +BUILTIN_ANALYZERS[lltype.render_immortal] = render_immortal BUILTIN_ANALYZERS[lltype.typeOf] = typeOf BUILTIN_ANALYZERS[lltype.cast_primitive] = cast_primitive BUILTIN_ANALYZERS[lltype.nullptr] = nullptr Modified: pypy/branch/arm-backend/pypy/annotation/classdef.py ============================================================================== --- pypy/branch/arm-backend/pypy/annotation/classdef.py (original) +++ pypy/branch/arm-backend/pypy/annotation/classdef.py Tue Nov 23 10:12:47 2010 @@ -58,7 +58,7 @@ # same name in all subclasses of A, if any. (Parent class attributes can # be visible in reads from instances of subclasses.) -class Attribute: +class Attribute(object): # readonly-ness # SomeThing-ness # NB. an attribute is readonly if it is a constant class attribute. @@ -402,7 +402,7 @@ # ____________________________________________________________ -class InstanceSource: +class InstanceSource(object): instance_level = True def __init__(self, bookkeeper, obj): Modified: pypy/branch/arm-backend/pypy/annotation/description.py ============================================================================== --- pypy/branch/arm-backend/pypy/annotation/description.py (original) +++ pypy/branch/arm-backend/pypy/annotation/description.py Tue Nov 23 10:12:47 2010 @@ -6,7 +6,7 @@ from pypy.tool.sourcetools import valid_identifier from pypy.tool.pairtype import extendabletype -class CallFamily: +class CallFamily(object): """A family of Desc objects that could be called from common call sites. The call families are conceptually a partition of all (callable) Desc objects, where the equivalence relation is the transitive closure of @@ -51,7 +51,7 @@ self.total_calltable_size += 1 -class FrozenAttrFamily: +class FrozenAttrFamily(object): """A family of FrozenDesc objects that have any common 'getattr' sites. The attr families are conceptually a partition of FrozenDesc objects, where the equivalence relation is the transitive closure of: @@ -80,7 +80,7 @@ self.attrs[attrname] = s_value -class ClassAttrFamily: +class ClassAttrFamily(object): """A family of ClassDesc objects that have common 'getattr' sites for a given attribute name. The attr families are conceptually a partition of ClassDesc objects, where the equivalence relation is the transitive Modified: pypy/branch/arm-backend/pypy/annotation/dictdef.py ============================================================================== --- pypy/branch/arm-backend/pypy/annotation/dictdef.py (original) +++ pypy/branch/arm-backend/pypy/annotation/dictdef.py Tue Nov 23 10:12:47 2010 @@ -77,7 +77,7 @@ dictdef.dictvalue = self -class DictDef: +class DictDef(object): """A dict definition remembers how general the keys and values in that particular dict have to be. Every dict creation makes a new DictDef, and the union of two dicts merges the DictKeys and DictValues that each Modified: pypy/branch/arm-backend/pypy/annotation/listdef.py ============================================================================== --- pypy/branch/arm-backend/pypy/annotation/listdef.py (original) +++ pypy/branch/arm-backend/pypy/annotation/listdef.py Tue Nov 23 10:12:47 2010 @@ -6,7 +6,7 @@ class TooLateForChange(Exception): pass -class ListItem: +class ListItem(object): mutated = False # True for lists mutated after creation resized = False # True for lists resized after creation range_step = None # the step -- only for lists only created by a range() @@ -117,7 +117,7 @@ return updated -class ListDef: +class ListDef(object): """A list definition remembers how general the items in that particular list have to be. Every list creation makes a new ListDef, and the union of two lists merges the ListItems that each ListDef stores.""" Modified: pypy/branch/arm-backend/pypy/annotation/model.py ============================================================================== --- pypy/branch/arm-backend/pypy/annotation/model.py (original) +++ pypy/branch/arm-backend/pypy/annotation/model.py Tue Nov 23 10:12:47 2010 @@ -574,11 +574,11 @@ NUMBER = object() annotation_to_ll_map = [ + (SomeSingleFloat(), lltype.SingleFloat), (s_None, lltype.Void), # also matches SomeImpossibleValue() (s_Bool, lltype.Bool), (SomeInteger(knowntype=r_ulonglong), NUMBER), (SomeFloat(), lltype.Float), - (SomeSingleFloat(), lltype.SingleFloat), (SomeChar(), lltype.Char), (SomeUnicodeCodePoint(), lltype.UniChar), (SomeAddress(), llmemory.Address), Modified: pypy/branch/arm-backend/pypy/annotation/policy.py ============================================================================== --- pypy/branch/arm-backend/pypy/annotation/policy.py (original) +++ pypy/branch/arm-backend/pypy/annotation/policy.py Tue Nov 23 10:12:47 2010 @@ -1,4 +1,4 @@ -# base annotation policy for overrides and specialization +# base annotation policy for specialization from pypy.annotation.specialize import default_specialize as default from pypy.annotation.specialize import specialize_argvalue, specialize_argtype, specialize_arglistitemtype from pypy.annotation.specialize import memo @@ -41,7 +41,7 @@ if directive is None: return pol.default_specialize - # specialize|override:name[(args)] + # specialize[(args)] directive_parts = directive.split('(', 1) if len(directive_parts) == 1: [name] = directive_parts @@ -60,14 +60,6 @@ except AttributeError: raise AttributeError("%r specialize tag not defined in annotation" "policy %s" % (name, pol)) - if directive.startswith('override:'): - # different signature: override__xyz(*args_s) - if parms: - raise Exception, "override:* specialisations don't support parameters" - def specialize_override(funcdesc, args_s): - funcdesc.overridden = True - return specializer(*args_s) - return specialize_override else: if not parms: return specializer @@ -92,9 +84,5 @@ from pypy.rpython.annlowlevel import LowLevelAnnotatorPolicy return LowLevelAnnotatorPolicy.specialize__ll_and_arg(*args) - def override__ignore(pol, *args): - bk = getbookkeeper() - return bk.immutablevalue(None) - class StrictAnnotatorPolicy(AnnotatorPolicy): allow_someobjects = False Modified: pypy/branch/arm-backend/pypy/annotation/specialize.py ============================================================================== --- pypy/branch/arm-backend/pypy/annotation/specialize.py (original) +++ pypy/branch/arm-backend/pypy/annotation/specialize.py Tue Nov 23 10:12:47 2010 @@ -100,7 +100,7 @@ # ____________________________________________________________________________ # specializations -class MemoTable: +class MemoTable(object): def __init__(self, funcdesc, args, value): self.funcdesc = funcdesc self.table = {args: value} Modified: pypy/branch/arm-backend/pypy/annotation/test/test_annrpython.py ============================================================================== --- pypy/branch/arm-backend/pypy/annotation/test/test_annrpython.py (original) +++ pypy/branch/arm-backend/pypy/annotation/test/test_annrpython.py Tue Nov 23 10:12:47 2010 @@ -1,4 +1,4 @@ - +from __future__ import with_statement import autopath import py.test import sys @@ -766,28 +766,6 @@ s = a.build_types(f, [list]) assert s.classdef is a.bookkeeper.getuniqueclassdef(IndexError) # KeyError ignored because l is a list - def test_overrides(self): - excs = [] - def record_exc(e): - """NOT_RPYTHON""" - excs.append(sys.exc_info) - record_exc._annspecialcase_ = "override:record_exc" - def g(): - pass - def f(): - try: - g() - except Exception, e: - record_exc(e) - class MyAnnotatorPolicy(policy.AnnotatorPolicy): - - def override__record_exc(pol, s_e): - return a.bookkeeper.immutablevalue(None) - - a = self.RPythonAnnotator(policy=MyAnnotatorPolicy()) - s = a.build_types(f, []) - assert s.const is None - def test_freeze_protocol(self): class Stuff: def __init__(self, flag): @@ -3359,6 +3337,26 @@ s = a.build_types(f, []) assert isinstance(s, annmodel.SomeChar) + def test_context_manager(self): + class C: + def __init__(self): + pass + def __enter__(self): + self.x = 1 + def __exit__(self, *args): + self.x = 3 + def f(): + c = C() + with c: + pass + return c.x + + a = self.RPythonAnnotator() + s = a.build_types(f, []) + assert isinstance(s, annmodel.SomeInteger) + # not a constant: both __enter__ and __exit__ have been annotated + assert not s.is_constant() + def g(n): return [0,1,2,n] Modified: pypy/branch/arm-backend/pypy/annotation/test/test_model.py ============================================================================== --- pypy/branch/arm-backend/pypy/annotation/test/test_model.py (original) +++ pypy/branch/arm-backend/pypy/annotation/test/test_model.py Tue Nov 23 10:12:47 2010 @@ -128,7 +128,7 @@ assert isinstance(s_p, SomeOOInstance) and s_p.ootype == C def test_annotation_to_lltype(): - from pypy.rlib.rarithmetic import r_uint + from pypy.rlib.rarithmetic import r_uint, r_singlefloat s_i = SomeInteger() s_pos = SomeInteger(nonneg=True) s_1 = SomeInteger(nonneg=True); s_1.const = 1 @@ -151,6 +151,9 @@ C = ootype.Instance('C', ROOT, {}) ref = SomeOOInstance(C) assert annotation_to_lltype(ref) == C + s_singlefloat = SomeSingleFloat() + s_singlefloat.const = r_singlefloat(0.0) + assert annotation_to_lltype(s_singlefloat) == lltype.SingleFloat def test_ll_union(): PS1 = lltype.Ptr(lltype.GcStruct('s')) Modified: pypy/branch/arm-backend/pypy/annotation/unaryop.py ============================================================================== --- pypy/branch/arm-backend/pypy/annotation/unaryop.py (original) +++ pypy/branch/arm-backend/pypy/annotation/unaryop.py Tue Nov 23 10:12:47 2010 @@ -434,6 +434,9 @@ def method_clear(dct): pass + def method_popitem(dct): + return dct.getanyitem('items') + def _can_only_throw(dic, *ignore): if dic1.dictdef.dictkey.custom_eq_hash: return None # r_dict: can throw anything Modified: pypy/branch/arm-backend/pypy/config/pypyoption.py ============================================================================== --- pypy/branch/arm-backend/pypy/config/pypyoption.py (original) +++ pypy/branch/arm-backend/pypy/config/pypyoption.py Tue Nov 23 10:12:47 2010 @@ -30,9 +30,15 @@ "rctime" , "select", "zipimport", "_lsprof", "crypt", "signal", "_rawffi", "termios", "zlib", "struct", "md5", "sha", "bz2", "_minimal_curses", "cStringIO", - "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array"] + "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array", + "_bisect"] )) +translation_modules = default_modules.copy() +translation_modules.update(dict.fromkeys( + ["fcntl", "rctime", "select", "signal", "_rawffi", "zlib", + "struct", "md5", "cStringIO", "array"])) + working_oo_modules = default_modules.copy() working_oo_modules.update(dict.fromkeys( ["md5", "sha", "cStringIO", "itertools"] @@ -73,9 +79,10 @@ } module_import_dependencies = { - # no _rawffi if importing pypy.rlib.libffi raises ImportError + # no _rawffi if importing pypy.rlib.clibffi raises ImportError # or CompilationError - "_rawffi" : ["pypy.rlib.libffi"], + "_rawffi" : ["pypy.rlib.clibffi"], + "_ffi" : ["pypy.rlib.clibffi"], "zlib" : ["pypy.rlib.rzlib"], "bz2" : ["pypy.module.bz2.interp_bz2"], @@ -147,6 +154,12 @@ cmdline="--allworkingmodules", negation=True), + BoolOption("translationmodules", + "use only those modules that are needed to run translate.py on pypy", + default=False, + cmdline="--translationmodules", + suggests=[("objspace.allworkingmodules", False)]), + BoolOption("geninterp", "specify whether geninterp should be used", cmdline=None, default=True), @@ -162,6 +175,11 @@ default=False, requires=[("objspace.usepycfiles", True)]), + StrOption("soabi", + "Tag to differentiate extension modules built for different Python interpreters", + cmdline="--soabi", + default=None), + BoolOption("honor__builtins__", "Honor the __builtins__ key of a module dictionary", default=False), @@ -198,6 +216,9 @@ BoolOption("withstrslice", "use strings optimized for slicing", default=False), + BoolOption("withstrbuf", "use strings optimized for addition (ver 2)", + default=False), + BoolOption("withprebuiltchar", "use prebuilt single-character string objects", default=False), @@ -210,7 +231,8 @@ BoolOption("withrope", "use ropes as the string implementation", default=False, requires=[("objspace.std.withstrslice", False), - ("objspace.std.withstrjoin", False)], + ("objspace.std.withstrjoin", False), + ("objspace.std.withstrbuf", False)], suggests=[("objspace.std.withprebuiltchar", True), ("objspace.std.sharesmallstr", True)]), @@ -224,19 +246,17 @@ requires=[("objspace.opcodes.CALL_LIKELY_BUILTIN", False), ("objspace.honor__builtins__", False)]), - BoolOption("withsharingdict", - "use dictionaries that share the keys part", - default=False), - BoolOption("withdictmeasurement", "create huge files with masses of information " "about dictionaries", default=False), - BoolOption("withinlineddict", - "make instances more compact by revoming a level of indirection", + BoolOption("withmapdict", + "make instances really small but slow without the JIT", default=False, - requires=[("objspace.std.withshadowtracking", False)]), + requires=[("objspace.std.getattributeshortcut", True), + ("objspace.std.withtypeversion", True), + ]), BoolOption("withrangelist", "enable special range list implementation that does not " @@ -251,12 +271,6 @@ # weakrefs needed, because of get_subclasses() requires=[("translation.rweakref", True)]), - BoolOption("withshadowtracking", - "track whether an instance attribute shadows a type" - " attribute", - default=False, - requires=[("objspace.std.withtypeversion", True), - ("translation.rweakref", True)]), BoolOption("withmethodcache", "try to cache method lookups", default=False, @@ -328,9 +342,6 @@ config.objspace.std.suggest(optimized_list_getitem=True) config.objspace.std.suggest(getattributeshortcut=True) config.objspace.std.suggest(newshortcut=True) - if type_system != 'ootype': - config.objspace.std.suggest(withsharingdict=True) - config.objspace.std.suggest(withinlineddict=True) # extra costly optimizations only go in level 3 if level == '3': @@ -343,7 +354,7 @@ config.objspace.std.suggest(withprebuiltint=True) config.objspace.std.suggest(withrangelist=True) config.objspace.std.suggest(withprebuiltchar=True) - config.objspace.std.suggest(withinlineddict=True) + config.objspace.std.suggest(withmapdict=True) config.objspace.std.suggest(withstrslice=True) config.objspace.std.suggest(withstrjoin=True) # xxx other options? ropes maybe? @@ -359,6 +370,7 @@ # extra optimizations with the JIT if level == 'jit': config.objspace.std.suggest(withcelldict=True) + config.objspace.std.suggest(withmapdict=True) def enable_allworkingmodules(config): @@ -373,6 +385,11 @@ modules = [name for name in modules if name not in essential_modules] config.objspace.usemodules.suggest(**dict.fromkeys(modules, True)) +def enable_translationmodules(config): + modules = translation_modules + modules = [name for name in modules if name not in essential_modules] + config.objspace.usemodules.suggest(**dict.fromkeys(modules, True)) + if __name__ == '__main__': config = get_pypy_config() Modified: pypy/branch/arm-backend/pypy/config/test/test_pypyoption.py ============================================================================== --- pypy/branch/arm-backend/pypy/config/test/test_pypyoption.py (original) +++ pypy/branch/arm-backend/pypy/config/test/test_pypyoption.py Tue Nov 23 10:12:47 2010 @@ -47,7 +47,7 @@ def test_set_pypy_opt_level(): conf = get_pypy_config() set_pypy_opt_level(conf, '2') - assert conf.objspace.std.withsharingdict + assert conf.objspace.std.newshortcut conf = get_pypy_config() set_pypy_opt_level(conf, '0') assert not conf.objspace.std.newshortcut @@ -59,7 +59,6 @@ assert not conf.objspace.std.withtypeversion assert not conf.objspace.std.withmethodcache - assert not conf.objspace.std.withshadowtracking def test_check_documentation(): def check_file_exists(fn): Modified: pypy/branch/arm-backend/pypy/config/translationoption.py ============================================================================== --- pypy/branch/arm-backend/pypy/config/translationoption.py (original) +++ pypy/branch/arm-backend/pypy/config/translationoption.py Tue Nov 23 10:12:47 2010 @@ -115,7 +115,7 @@ default="auto", cmdline="--jit-backend"), ChoiceOption("jit_debug", "the amount of debugging dumps made by the JIT", ["off", "profile", "steps", "detailed"], - default="profile", # XXX for now + default="off", cmdline="--jit-debug"), ChoiceOption("jit_profiler", "integrate profiler support into the JIT", ["off", "oprofile"], Modified: pypy/branch/arm-backend/pypy/conftest.py ============================================================================== --- pypy/branch/arm-backend/pypy/conftest.py (original) +++ pypy/branch/arm-backend/pypy/conftest.py Tue Nov 23 10:12:47 2010 @@ -7,6 +7,7 @@ from inspect import isclass, getmro from pypy.tool.udir import udir from pypy.tool.autopath import pypydir +from pypy.tool import leakfinder # pytest settings pytest_plugins = "resultlog", @@ -326,6 +327,31 @@ class PyPyTestFunction(py.test.collect.Function): # All PyPy test items catch and display OperationErrors specially. # + def runtest(self): + self.runtest_open() + try: + self.runtest_perform() + finally: + self.runtest_close() + self.runtest_finish() + + def runtest_open(self): + leakfinder.start_tracking_allocations() + + def runtest_perform(self): + super(PyPyTestFunction, self).runtest() + + def runtest_close(self): + if leakfinder.TRACK_ALLOCATIONS: + self._pypytest_leaks = leakfinder.stop_tracking_allocations(False) + else: # stop_tracking_allocations() already called + self._pypytest_leaks = None + + def runtest_finish(self): + # check for leaks, but only if the test passed so far + if self._pypytest_leaks: + raise leakfinder.MallocMismatch(self._pypytest_leaks) + def execute_appex(self, space, target, *args): try: target(*args) @@ -352,9 +378,9 @@ def _keywords(self): return super(IntTestFunction, self)._keywords() + ['interplevel'] - def runtest(self): + def runtest_perform(self): try: - super(IntTestFunction, self).runtest() + super(IntTestFunction, self).runtest_perform() except OperationError, e: check_keyboard_interrupt(e) raise @@ -367,12 +393,15 @@ py.test.skip('%s: %s' % (e.__class__.__name__, e)) cls = cls.__bases__[0] raise + + def runtest_finish(self): if 'pygame' in sys.modules: global _pygame_imported if not _pygame_imported: _pygame_imported = True assert option.view, ("should not invoke Pygame " "if conftest.option.view is False") + super(IntTestFunction, self).runtest_finish() class AppTestFunction(PyPyTestFunction): def _prunetraceback(self, traceback): @@ -385,7 +414,7 @@ def _keywords(self): return ['applevel'] + super(AppTestFunction, self)._keywords() - def runtest(self): + def runtest_perform(self): target = self.obj if option.runappdirect: return target() @@ -417,7 +446,7 @@ space.setattr(w_instance, space.wrap(name[2:]), getattr(instance, name)) - def runtest(self): + def runtest_perform(self): target = self.obj if option.runappdirect: return target() Modified: pypy/branch/arm-backend/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt ============================================================================== --- pypy/branch/arm-backend/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt (original) +++ pypy/branch/arm-backend/pypy/doc/config/objspace.opcodes.CALL_METHOD.txt Tue Nov 23 10:12:47 2010 @@ -5,8 +5,6 @@ case. So far, this only works for calls with no keyword, no ``*arg`` and no ``**arg`` but it would be easy to extend. -Gives the best results combined with :config:`objspace.std.withshadowtracking`. - For more information, see the section in `Standard Interpreter Optimizations`_. .. _`Standard Interpreter Optimizations`: ../interpreter-optimizations.html#lookup-method-call-method Modified: pypy/branch/arm-backend/pypy/doc/docindex.txt ============================================================================== --- pypy/branch/arm-backend/pypy/doc/docindex.txt (original) +++ pypy/branch/arm-backend/pypy/doc/docindex.txt Tue Nov 23 10:12:47 2010 @@ -84,7 +84,7 @@ PyPy's own tests `summary`_, daily updated, run through BuildBot infrastructure. You can also find CPython's compliance tests run with compiled ``pypy-c`` -exeuctables there. +executables there. information dating from early 2007: Modified: pypy/branch/arm-backend/pypy/doc/getting-started.txt ============================================================================== --- pypy/branch/arm-backend/pypy/doc/getting-started.txt (original) +++ pypy/branch/arm-backend/pypy/doc/getting-started.txt Tue Nov 23 10:12:47 2010 @@ -18,6 +18,7 @@ translation process - as opposed to encoding low level details into the language implementation itself. `more...`_ + .. _Python: http://docs.python.org/ref .. _`more...`: architecture.html Modified: pypy/branch/arm-backend/pypy/doc/interpreter-optimizations.txt ============================================================================== --- pypy/branch/arm-backend/pypy/doc/interpreter-optimizations.txt (original) +++ pypy/branch/arm-backend/pypy/doc/interpreter-optimizations.txt Tue Nov 23 10:12:47 2010 @@ -153,8 +153,8 @@ dicts: the representation of the instance dict contains only a list of values. -You can enable this feature with the :config:`objspace.std.withsharingdict` -option. +A more advanced version of sharing dicts, called *map dicts,* is available +with the :config:`objspace.std.withmapdict` option. Builtin-Shadowing +++++++++++++++++ @@ -219,8 +219,7 @@ shadowing the class attribute. If we know that there is no shadowing (since instance dict tells us that) we can save this lookup on the instance dictionary. -You can enable this feature with the :config:`objspace.std.withshadowtracking` -option. +*This was deprecated and is no longer available.* Method Caching Modified: pypy/branch/arm-backend/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/argument.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/argument.py Tue Nov 23 10:12:47 2010 @@ -64,7 +64,7 @@ return not self == other - # make it look tuply for the annotator + # make it look tuply for its use in the annotator def __len__(self): return 3 @@ -103,10 +103,11 @@ make_sure_not_resized(self.keywords_w) make_sure_not_resized(self.arguments_w) - if ((w_stararg is not None and w_stararg) or - (w_starstararg is not None and w_starstararg)): - self._combine_wrapped(w_stararg, w_starstararg) - # if we have a call where * or ** args are used at the callsite + if w_stararg is not None and space.is_true(w_stararg): + self._combine_starargs_wrapped(w_stararg) + if w_starstararg is not None and space.is_true(w_starstararg): + self._combine_starstarargs_wrapped(w_starstararg) + # if we have a call where **args are used at the callsite # we shouldn't let the JIT see the argument matching self._dont_jit = True else: @@ -142,42 +143,48 @@ def _combine_wrapped(self, w_stararg, w_starstararg): "unpack the *arg and **kwd into arguments_w and keywords_w" - # unpack the * arguments if w_stararg is not None: - self.arguments_w = (self.arguments_w + - self.space.fixedview(w_stararg)) - # unpack the ** arguments + self._combine_starargs_wrapped(w_stararg) if w_starstararg is not None: - space = self.space - if not space.is_true(space.isinstance(w_starstararg, space.w_dict)): + self._combine_starstarargs_wrapped(w_starstararg) + + def _combine_starargs_wrapped(self, w_stararg): + # unpack the * arguments + self.arguments_w = (self.arguments_w + + self.space.fixedview(w_stararg)) + + def _combine_starstarargs_wrapped(self, w_starstararg): + # unpack the ** arguments + space = self.space + if not space.is_true(space.isinstance(w_starstararg, space.w_dict)): + raise OperationError(space.w_TypeError, + space.wrap("argument after ** must be " + "a dictionary")) + keywords_w = [None] * space.int_w(space.len(w_starstararg)) + keywords = [None] * space.int_w(space.len(w_starstararg)) + i = 0 + for w_key in space.unpackiterable(w_starstararg): + try: + key = space.str_w(w_key) + except OperationError, e: + if not e.match(space, space.w_TypeError): + raise raise OperationError(space.w_TypeError, - space.wrap("argument after ** must be " - "a dictionary")) - keywords_w = [None] * space.int_w(space.len(w_starstararg)) - keywords = [None] * space.int_w(space.len(w_starstararg)) - i = 0 - for w_key in space.unpackiterable(w_starstararg): - try: - key = space.str_w(w_key) - except OperationError, e: - if not e.match(space, space.w_TypeError): - raise - raise OperationError(space.w_TypeError, - space.wrap("keywords must be strings")) - if self.keywords and key in self.keywords: - raise operationerrfmt(self.space.w_TypeError, - "got multiple values " - "for keyword argument " - "'%s'", key) - keywords[i] = key - keywords_w[i] = space.getitem(w_starstararg, w_key) - i += 1 - if self.keywords is None: - self.keywords = keywords - self.keywords_w = keywords_w - else: - self.keywords = self.keywords + keywords - self.keywords_w = self.keywords_w + keywords_w + space.wrap("keywords must be strings")) + if self.keywords and key in self.keywords: + raise operationerrfmt(self.space.w_TypeError, + "got multiple values " + "for keyword argument " + "'%s'", key) + keywords[i] = key + keywords_w[i] = space.getitem(w_starstararg, w_key) + i += 1 + if self.keywords is None: + self.keywords = keywords + self.keywords_w = keywords_w + else: + self.keywords = self.keywords + keywords + self.keywords_w = self.keywords_w + keywords_w def fixedunpack(self, argcount): """The simplest argument parsing: get the 'argcount' arguments, @@ -226,6 +233,10 @@ # argnames = list of formal parameter names # scope_w = resulting list of wrapped values # + + # some comments about the JIT: it assumes that signature is a constant, + # so all values coming from there can be assumed constant. It assumes + # that the length of the defaults_w does not vary too much. co_argcount = signature.num_argnames() # expected formal arguments, without */** has_vararg = signature.has_vararg() has_kwarg = signature.has_kwarg() @@ -245,12 +256,6 @@ args_w = self.arguments_w num_args = len(args_w) - keywords = self.keywords - keywords_w = self.keywords_w - num_kwds = 0 - if keywords is not None: - num_kwds = len(keywords) - avail = num_args + upfront if input_argcount < co_argcount: @@ -260,15 +265,24 @@ else: take = num_args + # letting the JIT unroll this loop is safe, because take is always + # smaller than co_argcount for i in range(take): scope_w[i + input_argcount] = args_w[i] input_argcount += take + keywords = self.keywords + keywords_w = self.keywords_w + num_kwds = 0 + if keywords is not None: + num_kwds = len(keywords) # the code assumes that keywords can potentially be large, but that # argnames is typically not too large num_remainingkwds = num_kwds used_keywords = None if keywords: + # letting JIT unroll the loop is *only* safe if the callsite didn't + # use **args because num_kwds can be arbitrarily large otherwise. used_keywords = [False] * num_kwds for i in range(num_kwds): name = keywords[i] @@ -276,7 +290,7 @@ if j < 0: continue elif j < input_argcount: - # check that no keyword argument conflicts with these note + # check that no keyword argument conflicts with these. note # that for this purpose we ignore the first blindargs, # which were put into place by prepend(). This way, # keywords do not conflict with the hidden extra argument @@ -388,9 +402,10 @@ space = self.space w_args = space.newtuple(self.arguments_w) w_kwds = space.newdict() - for i in range(len(self.keywords)): - space.setitem(w_kwds, space.wrap(self.keywords[i]), - self.keywords_w[i]) + if self.keywords is not None: + for i in range(len(self.keywords)): + space.setitem(w_kwds, space.wrap(self.keywords[i]), + self.keywords_w[i]) return w_args, w_kwds class ArgumentsForTranslation(Arguments): Modified: pypy/branch/arm-backend/pypy/interpreter/astcompiler/test/test_compiler.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/astcompiler/test/test_compiler.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/astcompiler/test/test_compiler.py Tue Nov 23 10:12:47 2010 @@ -41,7 +41,7 @@ co_expr = compile(evalexpr, '', 'eval') space = self.space pyco_expr = PyCode._from_code(space, co_expr) - w_res = pyco_expr.exec_code(space, w_dict, w_dict) + w_res = pyco_expr.exec_host_bytecode(space, w_dict, w_dict) res = space.str_w(space.repr(w_res)) if not isinstance(expected, float): assert res == repr(expected) Modified: pypy/branch/arm-backend/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/baseobjspace.py Tue Nov 23 10:12:47 2010 @@ -36,13 +36,10 @@ return space.finditem_str(w_dict, attr) return None - def getdictvalue_attr_is_in_class(self, space, attr): - return self.getdictvalue(space, attr) - - def setdictvalue(self, space, attr, w_value, shadows_type=True): + def setdictvalue(self, space, attr, w_value): w_dict = self.getdict() if w_dict is not None: - space.setitem_str(w_dict, attr, w_value, shadows_type) + space.setitem_str(w_dict, attr, w_value) return True return False @@ -168,6 +165,20 @@ def _call_builtin_destructor(self): pass # method overridden in typedef.py + # hooks that the mapdict implementations needs: + def _get_mapdict_map(self): + return None + def _set_mapdict_map(self, map): + raise NotImplementedError + def _mapdict_read_storage(self, index): + raise NotImplementedError + def _mapdict_write_storage(self, index, value): + raise NotImplementedError + def _mapdict_storage_length(self): + raise NotImplementedError + def _set_mapdict_storage_and_map(self, storage, map): + raise NotImplementedError + class Wrappable(W_Root): """A subclass of Wrappable is an internal, interpreter-level class @@ -247,10 +258,9 @@ self.interned_strings = {} self.actionflag = ActionFlag() # changed by the signal module + self.check_signal_action = None # changed by the signal module self.user_del_action = UserDelAction(self) self.frame_trace_action = FrameTraceAction(self) - self.actionflag.register_action(self.user_del_action) - self.actionflag.register_action(self.frame_trace_action) from pypy.interpreter.pycode import cpython_magic, default_magic self.our_magic = default_magic @@ -643,7 +653,7 @@ """shortcut for space.int_w(space.hash(w_obj))""" return self.int_w(self.hash(w_obj)) - def setitem_str(self, w_obj, key, w_value, shadows_type=True): + def setitem_str(self, w_obj, key, w_value): return self.setitem(w_obj, self.wrap(key), w_value) def finditem_str(self, w_obj, key): @@ -725,7 +735,7 @@ def unpackiterable(self, w_iterable, expected_length=-1): """Unpack an iterable object into a real (interpreter-level) list. - Raise a real (subclass of) ValueError if the length is wrong.""" + Raise an OperationError(w_ValueError) if the length is wrong.""" w_iterator = self.iter(w_iterable) items = [] while True: Modified: pypy/branch/arm-backend/pypy/interpreter/executioncontext.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/executioncontext.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/executioncontext.py Tue Nov 23 10:12:47 2010 @@ -5,6 +5,8 @@ from pypy.rlib.unroll import unrolling_iterable from pypy.rlib import jit +TICK_COUNTER_STEP = 100 + def app_profile_call(space, w_callable, frame, event, w_arg): space.call_function(w_callable, space.wrap(frame), @@ -19,6 +21,9 @@ # XXX self.w_tracefunc, self.profilefunc # XXX frame.is_being_profiled + # XXX [fijal] but they're not. is_being_profiled is guarded a bit all + # over the place as well as w_tracefunc + def __init__(self, space): self.space = space self.topframeref = jit.vref_None @@ -163,24 +168,19 @@ if self.w_tracefunc is not None: self._trace(frame, 'return', w_retval) - def bytecode_trace(self, frame): + def bytecode_trace(self, frame, decr_by=TICK_COUNTER_STEP): "Trace function called before each bytecode." # this is split into a fast path and a slower path that is # not invoked every time bytecode_trace() is. actionflag = self.space.actionflag - ticker = actionflag.get() - if actionflag.has_bytecode_counter: # this "if" is constant-folded - ticker += 1 - actionflag.set(ticker) - if ticker & actionflag.interesting_bits: # fast check + if actionflag.decrement_ticker(decr_by) < 0: actionflag.action_dispatcher(self, frame) # slow path bytecode_trace._always_inline_ = True def bytecode_trace_after_exception(self, frame): "Like bytecode_trace(), but without increasing the ticker." actionflag = self.space.actionflag - ticker = actionflag.get() - if ticker & actionflag.interesting_bits: # fast check + if actionflag.get_ticker() < 0: actionflag.action_dispatcher(self, frame) # slow path bytecode_trace_after_exception._always_inline_ = True @@ -311,125 +311,109 @@ frame.last_exception = last_exception self.is_tracing -= 1 + def checksignals(self): + """Similar to PyErr_CheckSignals(). If called in the main thread, + and if signals are pending for the process, deliver them now + (i.e. call the signal handlers).""" + if self.space.check_signal_action is not None: + self.space.check_signal_action.perform(self, None) + def _freeze_(self): raise Exception("ExecutionContext instances should not be seen during" " translation. Now is a good time to inspect the" " traceback and see where this one comes from :-)") -class AbstractActionFlag: - """This holds the global 'action flag'. It is a single bitfield - integer, with bits corresponding to AsyncAction objects that need to - be immediately triggered. The correspondance from bits to - AsyncAction instances is built at translation time. We can quickly - check if there is anything at all to do by checking if any of the - relevant bits is set. If threads are enabled, they consume the 20 - lower bits to hold a counter incremented at each bytecode, to know - when to release the GIL. +class AbstractActionFlag(object): + """This holds in an integer the 'ticker'. If threads are enabled, + it is decremented at each bytecode; when it reaches zero, we release + the GIL. And whether we have threads or not, it is forced to zero + whenever we fire any of the asynchronous actions. """ def __init__(self): self._periodic_actions = [] self._nonperiodic_actions = [] - self.unused_bits = self.FREE_BITS[:] self.has_bytecode_counter = False - self.interesting_bits = 0 + self.fired_actions = None + self.checkinterval_scaled = 100 * TICK_COUNTER_STEP self._rebuild_action_dispatcher() def fire(self, action): - """Request for the action to be run before the next opcode. - The action must have been registered at space initalization time.""" - ticker = self.get() - self.set(ticker | action.bitmask) - - def register_action(self, action): - "NOT_RPYTHON" - assert isinstance(action, AsyncAction) - if action.bitmask == 0: - while True: - action.bitmask = self.unused_bits.pop(0) - if not (action.bitmask & self.interesting_bits): - break - self.interesting_bits |= action.bitmask - if action.bitmask & self.BYTECODE_COUNTER_OVERFLOW_BIT: - assert action.bitmask == self.BYTECODE_COUNTER_OVERFLOW_BIT - self._periodic_actions.append(action) + """Request for the action to be run before the next opcode.""" + if not action._fired: + action._fired = True + if self.fired_actions is None: + self.fired_actions = [] + self.fired_actions.append(action) + # set the ticker to -1 in order to force action_dispatcher() + # to run at the next possible bytecode + self.reset_ticker(-1) + + def register_periodic_action(self, action, use_bytecode_counter): + """NOT_RPYTHON: + Register the PeriodicAsyncAction action to be called whenever the + tick counter becomes smaller than 0. If 'use_bytecode_counter' is + True, make sure that we decrease the tick counter at every bytecode. + This is needed for threads. Note that 'use_bytecode_counter' can be + False for signal handling, because whenever the process receives a + signal, the tick counter is set to -1 by C code in signals.h. + """ + assert isinstance(action, PeriodicAsyncAction) + self._periodic_actions.append(action) + if use_bytecode_counter: self.has_bytecode_counter = True - self.force_tick_counter() - else: - self._nonperiodic_actions.append((action, action.bitmask)) self._rebuild_action_dispatcher() - def setcheckinterval(self, space, interval): - if interval < self.CHECK_INTERVAL_MIN: - interval = self.CHECK_INTERVAL_MIN - elif interval > self.CHECK_INTERVAL_MAX: - interval = self.CHECK_INTERVAL_MAX - space.sys.checkinterval = interval - self.force_tick_counter() - - def force_tick_counter(self): - # force the tick counter to a valid value -- this actually forces - # it to reach BYTECODE_COUNTER_OVERFLOW_BIT at the next opcode. - ticker = self.get() - ticker &= ~ self.BYTECODE_COUNTER_OVERFLOW_BIT - ticker |= self.BYTECODE_COUNTER_MASK - self.set(ticker) + def getcheckinterval(self): + return self.checkinterval_scaled // TICK_COUNTER_STEP + + def setcheckinterval(self, interval): + MAX = sys.maxint // TICK_COUNTER_STEP + if interval < 1: + interval = 1 + elif interval > MAX: + interval = MAX + self.checkinterval_scaled = interval * TICK_COUNTER_STEP def _rebuild_action_dispatcher(self): periodic_actions = unrolling_iterable(self._periodic_actions) - nonperiodic_actions = unrolling_iterable(self._nonperiodic_actions) - has_bytecode_counter = self.has_bytecode_counter @jit.dont_look_inside def action_dispatcher(ec, frame): - # periodic actions - if has_bytecode_counter: - ticker = self.get() - if ticker & self.BYTECODE_COUNTER_OVERFLOW_BIT: - # We must run the periodic actions now, but first - # reset the bytecode counter (the following line - # works by assuming that we just overflowed the - # counter, i.e. BYTECODE_COUNTER_OVERFLOW_BIT is - # set but none of the BYTECODE_COUNTER_MASK bits - # are). - ticker -= ec.space.sys.checkinterval - self.set(ticker) - for action in periodic_actions: - action.perform(ec, frame) + # periodic actions (first reset the bytecode counter) + self.reset_ticker(self.checkinterval_scaled) + for action in periodic_actions: + action.perform(ec, frame) # nonperiodic actions - for action, bitmask in nonperiodic_actions: - ticker = self.get() - if ticker & bitmask: - self.set(ticker & ~ bitmask) + list = self.fired_actions + if list is not None: + self.fired_actions = None + for action in list: + action._fired = False action.perform(ec, frame) action_dispatcher._dont_inline_ = True self.action_dispatcher = action_dispatcher - # Bits reserved for the bytecode counter, if used - BYTECODE_COUNTER_MASK = (1 << 20) - 1 - BYTECODE_COUNTER_OVERFLOW_BIT = (1 << 20) - - # Free bits - FREE_BITS = [1 << _b for _b in range(21, LONG_BIT-1)] - - # The acceptable range of values for sys.checkinterval, so that - # the bytecode_counter fits in 20 bits - CHECK_INTERVAL_MIN = 1 - CHECK_INTERVAL_MAX = BYTECODE_COUNTER_OVERFLOW_BIT - class ActionFlag(AbstractActionFlag): """The normal class for space.actionflag. The signal module provides a different one.""" - _flags = 0 + _ticker = 0 + + def get_ticker(self): + return self._ticker - def get(self): - return self._flags + def reset_ticker(self, value): + self._ticker = value - def set(self, value): - self._flags = value + def decrement_ticker(self, by): + value = self._ticker + if self.has_bytecode_counter: # this 'if' is constant-folded + value -= by + self._ticker = value + return value class AsyncAction(object): @@ -437,7 +421,7 @@ asynchronously with regular bytecode execution, but that still need to occur between two opcodes, not at a completely random time. """ - bitmask = 0 # means 'please choose one bit automatically' + _fired = False def __init__(self, space): self.space = space @@ -450,10 +434,11 @@ def fire_after_thread_switch(self): """Bit of a hack: fire() the action but only the next time the GIL is released and re-acquired (i.e. after a potential thread switch). - Don't call this if threads are not enabled. + Don't call this if threads are not enabled. Currently limited to + one action (i.e. reserved for CheckSignalAction from module/signal). """ from pypy.module.thread.gil import spacestate - spacestate.set_actionflag_bit_after_thread_switch |= self.bitmask + spacestate.action_after_thread_switch = self def perform(self, executioncontext, frame): """To be overridden.""" @@ -463,7 +448,6 @@ """Abstract base class for actions that occur automatically every sys.checkinterval bytecodes. """ - bitmask = ActionFlag.BYTECODE_COUNTER_OVERFLOW_BIT class UserDelAction(AsyncAction): Modified: pypy/branch/arm-backend/pypy/interpreter/function.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/function.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/function.py Tue Nov 23 10:12:47 2010 @@ -430,8 +430,11 @@ self.w_class = w_class # possibly space.w_None def descr_method__new__(space, w_subtype, w_function, w_instance, w_class=None): - if space.is_w( w_instance, space.w_None ): + if space.is_w(w_instance, space.w_None): w_instance = None + if w_instance is None and space.is_w(w_class, space.w_None): + raise OperationError(space.w_TypeError, + space.wrap("unbound methods must have class")) method = space.allocate_instance(Method, w_subtype) Method.__init__(method, space, w_function, w_instance, w_class) return space.wrap(method) @@ -586,12 +589,14 @@ w_klass = space.type(w_obj) return space.wrap(Method(space, self.w_function, w_klass, space.w_None)) - def descr_classmethod__new__(space, w_type, w_function): + def descr_classmethod__new__(space, w_subtype, w_function): if not space.is_true(space.callable(w_function)): typename = space.type(w_function).getname(space, '?') raise operationerrfmt(space.w_TypeError, "'%s' object is not callable", typename) - return space.wrap(ClassMethod(w_function)) + instance = space.allocate_instance(ClassMethod, w_subtype) + instance.__init__(w_function) + return space.wrap(instance) class FunctionWithFixedCode(Function): can_change_code = False Modified: pypy/branch/arm-backend/pypy/interpreter/gateway.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/gateway.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/gateway.py Tue Nov 23 10:12:47 2010 @@ -28,7 +28,7 @@ # internal non-translatable parts: import py -class SignatureBuilder: +class SignatureBuilder(object): "NOT_RPYTHON" def __init__(self, func=None, argnames=None, varargname=None, kwargname=None, name = None): @@ -51,7 +51,7 @@ #________________________________________________________________ -class UnwrapSpecRecipe: +class UnwrapSpecRecipe(object): "NOT_RPYTHON" bases_order = [Wrappable, W_Root, ObjSpace, Arguments, object] Modified: pypy/branch/arm-backend/pypy/interpreter/pycode.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/pycode.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/pycode.py Tue Nov 23 10:12:47 2010 @@ -117,6 +117,10 @@ self._compute_flatcall() + if self.space.config.objspace.std.withmapdict: + from pypy.objspace.std.mapdict import init_mapdict_cache + init_mapdict_cache(self) + def _freeze_(self): if (self.magic == cpython_magic and '__pypy__' not in sys.builtin_module_names): @@ -253,6 +257,12 @@ tuple(self.co_freevars), tuple(self.co_cellvars) ) + def exec_host_bytecode(self, w_dict, w_globals, w_locals): + from pypy.interpreter.pyframe import CPythonFrame + frame = CPythonFrame(self.space, self, w_globals, None) + frame.setdictscope(w_locals) + return frame.run() + def dump(self): """A dis.dis() dump of the code object.""" co = self._to_code() Modified: pypy/branch/arm-backend/pypy/interpreter/pyframe.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/pyframe.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/pyframe.py Tue Nov 23 10:12:47 2010 @@ -13,6 +13,7 @@ from pypy.rlib.rarithmetic import intmask from pypy.rlib import jit, rstack from pypy.tool import stdlib_opcode +from pypy.tool.stdlib_opcode import host_bytecode_spec # Define some opcodes used g = globals() @@ -140,7 +141,8 @@ # the following 'assert' is an annotation hint: it hides from # the annotator all methods that are defined in PyFrame but # overridden in the {,Host}FrameClass subclasses of PyFrame. - assert isinstance(self, self.space.FrameClass) + assert (isinstance(self, self.space.FrameClass) or + not self.space.config.translating) executioncontext = self.space.getexecutioncontext() executioncontext.enter(self) try: @@ -634,6 +636,18 @@ return space.wrap(self.builtin is not space.builtin) return space.w_False +class CPythonFrame(PyFrame): + """ + Execution of host (CPython) opcodes. + """ + + bytecode_spec = host_bytecode_spec + opcode_method_names = host_bytecode_spec.method_names + opcodedesc = host_bytecode_spec.opcodedesc + opdescmap = host_bytecode_spec.opdescmap + HAVE_ARGUMENT = host_bytecode_spec.HAVE_ARGUMENT + + # ____________________________________________________________ def get_block_class(opname): Modified: pypy/branch/arm-backend/pypy/interpreter/pyopcode.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/pyopcode.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/pyopcode.py Tue Nov 23 10:12:47 2010 @@ -164,6 +164,9 @@ next_instr = block.handle(self, unroller) return next_instr + def call_contextmanager_exit_function(self, w_func, w_typ, w_val, w_tb): + return self.space.call_function(w_func, w_typ, w_val, w_tb) + @jit.unroll_safe def dispatch_bytecode(self, co_code, next_instr, ec): space = self.space @@ -710,9 +713,14 @@ def LOAD_ATTR(self, nameindex, next_instr): "obj.attributename" - w_attributename = self.getname_w(nameindex) w_obj = self.popvalue() - w_value = self.space.getattr(w_obj, w_attributename) + if (self.space.config.objspace.std.withmapdict + and not jit.we_are_jitted()): + from pypy.objspace.std.mapdict import LOAD_ATTR_caching + w_value = LOAD_ATTR_caching(self.getcode(), w_obj, nameindex) + else: + w_attributename = self.getname_w(nameindex) + w_value = self.space.getattr(w_obj, w_attributename) self.pushvalue(w_value) LOAD_ATTR._always_inline_ = True @@ -868,23 +876,43 @@ def WITH_CLEANUP(self, oparg, next_instr): # see comment in END_FINALLY for stack state - w_exitfunc = self.popvalue() - w_unroller = self.peekvalue(2) + # This opcode changed a lot between CPython versions + if (self.pycode.magic >= 0xa0df2ef + # Implementation since 2.7a0: 62191 (introduce SETUP_WITH) + or self.pycode.magic >= 0xa0df2d1): + # implementation since 2.6a1: 62161 (WITH_CLEANUP optimization) + self.popvalue() + self.popvalue() + w_unroller = self.popvalue() + w_exitfunc = self.popvalue() + self.pushvalue(w_unroller) + self.pushvalue(self.space.w_None) + self.pushvalue(self.space.w_None) + elif self.pycode.magic >= 0xa0df28c: + # Implementation since 2.5a0: 62092 (changed WITH_CLEANUP opcode) + w_exitfunc = self.popvalue() + w_unroller = self.peekvalue(2) + else: + raise NotImplementedError("WITH_CLEANUP for CPython <= 2.4") + unroller = self.space.interpclass_w(w_unroller) if isinstance(unroller, SApplicationException): operr = unroller.operr - w_result = self.space.call_function(w_exitfunc, - operr.w_type, - operr.get_w_value(self.space), - operr.application_traceback) + w_traceback = self.space.wrap(operr.application_traceback) + w_result = self.call_contextmanager_exit_function( + w_exitfunc, + operr.w_type, + operr.get_w_value(self.space), + w_traceback) if self.space.is_true(w_result): # __exit__() returned True -> Swallow the exception. self.settopvalue(self.space.w_None, 2) else: - self.space.call_function(w_exitfunc, - self.space.w_None, - self.space.w_None, - self.space.w_None) + self.call_contextmanager_exit_function( + w_exitfunc, + self.space.w_None, + self.space.w_None, + self.space.w_None) @jit.unroll_safe def call_function(self, oparg, w_star=None, w_starstar=None): @@ -1112,7 +1140,7 @@ state_pack_variables = staticmethod(state_pack_variables) -class FrameBlock: +class FrameBlock(object): """Abstract base class for frame blocks from the blockstack, used by the SETUP_XXX and POP_BLOCK opcodes.""" @@ -1225,10 +1253,18 @@ frame.pushvalue(frame.space.w_None) return self.handlerposition # jump to the handler +class WithBlock(FinallyBlock): + + def really_handle(self, frame, unroller): + if (frame.space.full_exceptions and + isinstance(unroller, SApplicationException)): + unroller.operr.normalize_exception(frame.space) + return FinallyBlock.really_handle(self, frame, unroller) block_classes = {'SETUP_LOOP': LoopBlock, 'SETUP_EXCEPT': ExceptBlock, - 'SETUP_FINALLY': FinallyBlock} + 'SETUP_FINALLY': FinallyBlock, + 'SETUP_WITH': WithBlock} ### helpers written at the application-level ### # Some of these functions are expected to be generally useful if other Modified: pypy/branch/arm-backend/pypy/interpreter/pyparser/pytokenize.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/pyparser/pytokenize.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/pyparser/pytokenize.py Tue Nov 23 10:12:47 2010 @@ -21,40 +21,42 @@ __all__ = [ "tokenize" ] # ______________________________________________________________________ -# Automatically generated DFA's (with one or two hand tweeks): -pseudoStatesAccepts = [True, True, True, True, True, True, True, True, - True, True, False, True, True, True, False, False, - False, False, True, False, False, True, True, False, - True, False, True, False, True, False, True, False, - False, False, True, False, False, False, True] - -pseudoStates = [ - {'\t': 0, '\n': 13, '\x0c': 0, '\r': 14, ' ': 0, '!': 10, - '"': 16, '#': 18, '%': 12, '&': 12, - "'": 15, '(': 13, ')': 13, '*': 7, - '+': 12, ',': 13, '-': 12, '.': 6, - '/': 11, '0': 4, '1': 5, '2': 5, - '3': 5, '4': 5, '5': 5, '6': 5, - '7': 5, '8': 5, '9': 5, ':': 13, - ';': 13, '<': 9, '=': 12, '>': 8, - '@': 13, 'A': 1, - 'B': 1, 'C': 1, 'D': 1, 'E': 1, - 'F': 1, 'G': 1, 'H': 1, 'I': 1, - 'J': 1, 'K': 1, 'L': 1, 'M': 1, - 'N': 1, 'O': 1, 'P': 1, 'Q': 1, - 'R': 2, 'S': 1, 'T': 1, 'U': 3, - 'V': 1, 'W': 1, 'X': 1, 'Y': 1, - 'Z': 1, '[': 13, '\\': 17, ']': 13, - '^': 12, '_': 1, '`': 13, 'a': 1, - 'b': 1, 'c': 1, 'd': 1, 'e': 1, - 'f': 1, 'g': 1, 'h': 1, 'i': 1, - 'j': 1, 'k': 1, 'l': 1, 'm': 1, - 'n': 1, 'o': 1, 'p': 1, 'q': 1, - 'r': 2, 's': 1, 't': 1, 'u': 3, - 'v': 1, 'w': 1, 'x': 1, 'y': 1, - 'z': 1, '{': 13, '|': 12, '}': 13, - '~': 13}, +# Automatically generated DFA's +accepts = [True, True, True, True, True, True, True, True, + True, True, False, True, True, True, True, False, + False, False, True, True, True, False, True, + False, True, False, True, False, False, True, + False, False, False, False, True, False, False, + False, True] +states = [ + # 0 + {'\t': 0, '\n': 13, '\x0c': 0, + '\r': 14, ' ': 0, '!': 10, '"': 16, + '#': 18, '%': 12, '&': 12, "'": 15, + '(': 13, ')': 13, '*': 7, '+': 12, + ',': 13, '-': 12, '.': 6, '/': 11, + '0': 4, '1': 5, '2': 5, '3': 5, + '4': 5, '5': 5, '6': 5, '7': 5, + '8': 5, '9': 5, ':': 13, ';': 13, + '<': 9, '=': 12, '>': 8, '@': 13, + 'A': 1, 'B': 1, 'C': 1, 'D': 1, + 'E': 1, 'F': 1, 'G': 1, 'H': 1, + 'I': 1, 'J': 1, 'K': 1, 'L': 1, + 'M': 1, 'N': 1, 'O': 1, 'P': 1, + 'Q': 1, 'R': 2, 'S': 1, 'T': 1, + 'U': 3, 'V': 1, 'W': 1, 'X': 1, + 'Y': 1, 'Z': 1, '[': 13, '\\': 17, + ']': 13, '^': 12, '_': 1, '`': 13, + 'a': 1, 'b': 1, 'c': 1, 'd': 1, + 'e': 1, 'f': 1, 'g': 1, 'h': 1, + 'i': 1, 'j': 1, 'k': 1, 'l': 1, + 'm': 1, 'n': 1, 'o': 1, 'p': 1, + 'q': 1, 'r': 2, 's': 1, 't': 1, + 'u': 3, 'v': 1, 'w': 1, 'x': 1, + 'y': 1, 'z': 1, '{': 13, '|': 12, + '}': 13, '~': 13}, + # 1 {'0': 1, '1': 1, '2': 1, '3': 1, '4': 1, '5': 1, '6': 1, '7': 1, '8': 1, '9': 1, 'A': 1, 'B': 1, @@ -71,7 +73,7 @@ 'p': 1, 'q': 1, 'r': 1, 's': 1, 't': 1, 'u': 1, 'v': 1, 'w': 1, 'x': 1, 'y': 1, 'z': 1}, - + # 2 {'"': 16, "'": 15, '0': 1, '1': 1, '2': 1, '3': 1, '4': 1, '5': 1, '6': 1, '7': 1, '8': 1, '9': 1, @@ -89,7 +91,7 @@ 'r': 1, 's': 1, 't': 1, 'u': 1, 'v': 1, 'w': 1, 'x': 1, 'y': 1, 'z': 1}, - + # 3 {'"': 16, "'": 15, '0': 1, '1': 1, '2': 1, '3': 1, '4': 1, '5': 1, '6': 1, '7': 1, '8': 1, '9': 1, @@ -107,158 +109,187 @@ 'r': 2, 's': 1, 't': 1, 'u': 1, 'v': 1, 'w': 1, 'x': 1, 'y': 1, 'z': 1}, - - {'.': 24, '0': 22, '1': 22, '2': 22, - '3': 22, '4': 22, '5': 22, '6': 22, - '7': 22, '8': 23, '9': 23, 'E': 25, - 'J': 13, 'L': 13, 'X': 21, 'e': 25, - 'j': 13, 'l': 13, 'x': 21}, - - {'.': 24, '0': 5, '1': 5, '2': 5, + # 4 + {'.': 22, '0': 20, '1': 20, '2': 20, + '3': 20, '4': 20, '5': 20, '6': 20, + '7': 20, '8': 21, '9': 21, 'E': 23, + 'J': 13, 'L': 13, 'X': 19, 'e': 23, + 'j': 13, 'l': 13, 'x': 19}, + # 5 + {'.': 22, '0': 5, '1': 5, '2': 5, '3': 5, '4': 5, '5': 5, '6': 5, - '7': 5, '8': 5, '9': 5, 'E': 25, - 'J': 13, 'L': 13, 'e': 25, 'j': 13, + '7': 5, '8': 5, '9': 5, 'E': 23, + 'J': 13, 'L': 13, 'e': 23, 'j': 13, 'l': 13}, - - {'0': 26, '1': 26, '2': 26, '3': 26, - '4': 26, '5': 26, '6': 26, '7': 26, - '8': 26, '9': 26}, - + # 6 + {'0': 24, '1': 24, '2': 24, '3': 24, + '4': 24, '5': 24, '6': 24, '7': 24, + '8': 24, '9': 24}, + # 7 {'*': 12, '=': 13}, - + # 8 {'=': 13, '>': 12}, - - {'=': 13, '<': 12, '>': 13}, - + # 9 + {'<': 12, '=': 13, '>': 13}, + # 10 {'=': 13}, - - {'=': 13, '/': 12}, - + # 11 + {'/': 12, '=': 13}, + # 12 {'=': 13}, - + # 13 {}, - + # 14 {'\n': 13}, - - {automata.DEFAULT: 19, '\n': 27, '\\': 29, "'": 28}, - - {automata.DEFAULT: 20, '"': 30, '\n': 27, '\\': 31}, - + # 15 + {automata.DEFAULT: 28, '\n': 25, + '\r': 25, "'": 26, '\\': 27}, + # 16 + {automata.DEFAULT: 31, '\n': 25, + '\r': 25, '"': 29, '\\': 30}, + # 17 {'\n': 13, '\r': 14}, - - {automata.DEFAULT: 18, '\n': 27, '\r': 27}, - - {automata.DEFAULT: 19, '\n': 27, '\\': 29, "'": 13}, - - {automata.DEFAULT: 20, '"': 13, '\n': 27, '\\': 31}, - - {'0': 21, '1': 21, '2': 21, '3': 21, - '4': 21, '5': 21, '6': 21, '7': 21, - '8': 21, '9': 21, 'A': 21, 'B': 21, - 'C': 21, 'D': 21, 'E': 21, 'F': 21, - 'L': 13, 'a': 21, 'b': 21, 'c': 21, - 'd': 21, 'e': 21, 'f': 21, 'l': 13}, - - {'.': 24, '0': 22, '1': 22, '2': 22, - '3': 22, '4': 22, '5': 22, '6': 22, - '7': 22, '8': 23, '9': 23, 'E': 25, - 'J': 13, 'L': 13, 'e': 25, 'j': 13, + # 18 + {automata.DEFAULT: 18, '\n': 25, '\r': 25}, + # 19 + {'0': 19, '1': 19, '2': 19, '3': 19, + '4': 19, '5': 19, '6': 19, '7': 19, + '8': 19, '9': 19, 'A': 19, 'B': 19, + 'C': 19, 'D': 19, 'E': 19, 'F': 19, + 'L': 13, 'a': 19, 'b': 19, 'c': 19, + 'd': 19, 'e': 19, 'f': 19, 'l': 13}, + # 20 + {'.': 22, '0': 20, '1': 20, '2': 20, + '3': 20, '4': 20, '5': 20, '6': 20, + '7': 20, '8': 21, '9': 21, 'E': 23, + 'J': 13, 'L': 13, 'e': 23, 'j': 13, 'l': 13}, - - {'.': 24, '0': 23, '1': 23, '2': 23, - '3': 23, '4': 23, '5': 23, '6': 23, - '7': 23, '8': 23, '9': 23, 'E': 25, - 'J': 13, 'e': 25, 'j': 13}, - - {'0': 24, '1': 24, '2': 24, '3': 24, - '4': 24, '5': 24, '6': 24, '7': 24, - '8': 24, '9': 24, 'E': 32, 'J': 13, + # 21 + {'.': 22, '0': 21, '1': 21, '2': 21, + '3': 21, '4': 21, '5': 21, '6': 21, + '7': 21, '8': 21, '9': 21, 'E': 23, + 'J': 13, 'e': 23, 'j': 13}, + # 22 + {'0': 22, '1': 22, '2': 22, '3': 22, + '4': 22, '5': 22, '6': 22, '7': 22, + '8': 22, '9': 22, 'E': 32, 'J': 13, 'e': 32, 'j': 13}, - + # 23 {'+': 33, '-': 33, '0': 34, '1': 34, '2': 34, '3': 34, '4': 34, '5': 34, '6': 34, '7': 34, '8': 34, '9': 34}, - - {'0': 26, '1': 26, '2': 26, '3': 26, - '4': 26, '5': 26, '6': 26, '7': 26, - '8': 26, '9': 26, 'E': 32, 'J': 13, + # 24 + {'0': 24, '1': 24, '2': 24, '3': 24, + '4': 24, '5': 24, '6': 24, '7': 24, + '8': 24, '9': 24, 'E': 32, 'J': 13, 'e': 32, 'j': 13}, - + # 25 {}, - + # 26 {"'": 13}, - + # 27 {automata.DEFAULT: 35, '\n': 13, '\r': 14}, - + # 28 + {automata.DEFAULT: 28, '\n': 25, + '\r': 25, "'": 13, '\\': 27}, + # 29 {'"': 13}, - + # 30 {automata.DEFAULT: 36, '\n': 13, '\r': 14}, - + # 31 + {automata.DEFAULT: 31, '\n': 25, + '\r': 25, '"': 13, '\\': 30}, + # 32 {'+': 37, '-': 37, '0': 38, '1': 38, '2': 38, '3': 38, '4': 38, '5': 38, '6': 38, '7': 38, '8': 38, '9': 38}, - - + # 33 {'0': 34, '1': 34, '2': 34, '3': 34, '4': 34, '5': 34, '6': 34, '7': 34, '8': 34, '9': 34}, - + # 34 {'0': 34, '1': 34, '2': 34, '3': 34, '4': 34, '5': 34, '6': 34, '7': 34, '8': 34, '9': 34, 'J': 13, 'j': 13}, - - {automata.DEFAULT: 35, '\n': 27, '\\': 29, "'": 13}, - - {automata.DEFAULT: 36, '"': 13, '\n': 27, '\\': 31}, - + # 35 + {automata.DEFAULT: 35, '\n': 25, + '\r': 25, "'": 13, '\\': 27}, + # 36 + {automata.DEFAULT: 36, '\n': 25, + '\r': 25, '"': 13, '\\': 30}, + # 37 {'0': 38, '1': 38, '2': 38, '3': 38, '4': 38, '5': 38, '6': 38, '7': 38, '8': 38, '9': 38}, - + # 38 {'0': 38, '1': 38, '2': 38, '3': 38, '4': 38, '5': 38, '6': 38, '7': 38, '8': 38, '9': 38, 'J': 13, 'j': 13}, ] +pseudoDFA = automata.DFA(states, accepts) -pseudoDFA = automata.DFA(pseudoStates, pseudoStatesAccepts) - -double3StatesAccepts = [False, False, False, False, False, True] -double3States = [ +accepts = [False, False, False, False, False, True] +states = [ + # 0 {automata.DEFAULT: 0, '"': 1, '\\': 2}, + # 1 {automata.DEFAULT: 4, '"': 3, '\\': 2}, + # 2 {automata.DEFAULT: 4}, + # 3 {automata.DEFAULT: 4, '"': 5, '\\': 2}, + # 4 {automata.DEFAULT: 4, '"': 1, '\\': 2}, + # 5 {automata.DEFAULT: 4, '"': 5, '\\': 2}, ] -double3DFA = automata.NonGreedyDFA(double3States, double3StatesAccepts) +double3DFA = automata.NonGreedyDFA(states, accepts) -single3StatesAccepts = [False, False, False, False, False, True] -single3States = [ - {automata.DEFAULT: 0, '\\': 2, "'": 1}, - {automata.DEFAULT: 4, '\\': 2, "'": 3}, +accepts = [False, False, False, False, False, True] +states = [ + # 0 + {automata.DEFAULT: 0, "'": 1, '\\': 2}, + # 1 + {automata.DEFAULT: 4, "'": 3, '\\': 2}, + # 2 {automata.DEFAULT: 4}, - {automata.DEFAULT: 4, '\\': 2, "'": 5}, - {automata.DEFAULT: 4, '\\': 2, "'": 1}, - {automata.DEFAULT: 4, '\\': 2, "'": 5}, + # 3 + {automata.DEFAULT: 4, "'": 5, '\\': 2}, + # 4 + {automata.DEFAULT: 4, "'": 1, '\\': 2}, + # 5 + {automata.DEFAULT: 4, "'": 5, '\\': 2}, ] -single3DFA = automata.NonGreedyDFA(single3States, single3StatesAccepts) +single3DFA = automata.NonGreedyDFA(states, accepts) -singleStatesAccepts = [False, True, False] -singleStates = [ - {automata.DEFAULT: 0, '\\': 2, "'": 1}, +accepts = [False, True, False, False] +states = [ + # 0 + {automata.DEFAULT: 0, "'": 1, '\\': 2}, + # 1 {}, - {automata.DEFAULT: 0}, + # 2 + {automata.DEFAULT: 3}, + # 3 + {automata.DEFAULT: 3, "'": 1, '\\': 2}, ] -singleDFA = automata.DFA(singleStates, singleStatesAccepts) +singleDFA = automata.DFA(states, accepts) -doubleStatesAccepts = [False, True, False] -doubleStates = [ +accepts = [False, True, False, False] +states = [ + # 0 {automata.DEFAULT: 0, '"': 1, '\\': 2}, + # 1 {}, - {automata.DEFAULT: 0}, + # 2 + {automata.DEFAULT: 3}, + # 3 + {automata.DEFAULT: 3, '"': 1, '\\': 2}, ] -doubleDFA = automata.DFA(doubleStates, doubleStatesAccepts) +doubleDFA = automata.DFA(states, accepts) + +#_______________________________________________________________________ +# End of automatically generated DFA's endDFAs = {"'" : singleDFA, '"' : doubleDFA, Modified: pypy/branch/arm-backend/pypy/interpreter/pyparser/test/test_pyparse.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/pyparser/test/test_pyparse.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/pyparser/test/test_pyparse.py Tue Nov 23 10:12:47 2010 @@ -92,6 +92,9 @@ exc = py.test.raises(IndentationError, parse, input).value assert exc.msg == "unindent does not match any outer indentation level" + def test_mac_newline(self): + self.parse("this_is\ra_mac\rfile") + def test_mode(self): assert self.parse("x = 43*54").type == syms.file_input tree = self.parse("43**54", "eval") Modified: pypy/branch/arm-backend/pypy/interpreter/test/test_argument.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/test/test_argument.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/test/test_argument.py Tue Nov 23 10:12:47 2010 @@ -52,12 +52,17 @@ assert y == "d" assert z == "e" +class dummy_wrapped_dict(dict): + def __nonzero__(self): + raise NotImplementedError class DummySpace(object): def newtuple(self, items): return tuple(items) def is_true(self, obj): + if isinstance(obj, dummy_wrapped_dict): + return bool(dict(obj)) return bool(obj) def fixedview(self, it): @@ -229,7 +234,7 @@ kwds_w = dict(kwds[:i]) keywords = kwds_w.keys() keywords_w = kwds_w.values() - w_kwds = dict(kwds[i:]) + w_kwds = dummy_wrapped_dict(kwds[i:]) if i == 2: w_kwds = None assert len(keywords) == len(keywords_w) @@ -265,7 +270,7 @@ kwds_w = dict(kwds[:i]) keywords = kwds_w.keys() keywords_w = kwds_w.values() - w_kwds = dict(kwds[i:]) + w_kwds = dummy_wrapped_dict(kwds[i:]) if i == 3: w_kwds = None args = Arguments(space, [1, 2], keywords, keywords_w, w_starstararg=w_kwds) @@ -448,7 +453,11 @@ assert set(args.keywords) == set(['a', 'b']) assert args.keywords_w[args.keywords.index('a')] == 2 assert args.keywords_w[args.keywords.index('b')] == 3 - + + args = Arguments(space, [1]) + w_args, w_kwds = args.topacked() + assert w_args == (1, ) + assert not w_kwds class TestErrorHandling(object): def test_missing_args(self): Modified: pypy/branch/arm-backend/pypy/interpreter/test/test_executioncontext.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/test/test_executioncontext.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/test/test_executioncontext.py Tue Nov 23 10:12:47 2010 @@ -19,7 +19,6 @@ space = self.space a1 = DemoAction(space) - space.actionflag.register_action(a1) for i in range(20): # assert does not raise: space.appexec([], """(): @@ -50,7 +49,7 @@ space = self.space a2 = DemoAction(space) - space.actionflag.register_action(a2) + space.actionflag.register_periodic_action(a2, True) try: for i in range(500): space.appexec([], """(): @@ -59,7 +58,8 @@ """) except Finished: pass - assert space.sys.checkinterval / 10 < i < space.sys.checkinterval * 3 + checkinterval = space.actionflag.getcheckinterval() + assert checkinterval / 10 < i < checkinterval * 1.1 def test_llprofile(self): l = [] Modified: pypy/branch/arm-backend/pypy/interpreter/test/test_function.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/test/test_function.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/test/test_function.py Tue Nov 23 10:12:47 2010 @@ -482,6 +482,11 @@ raises(TypeError, m, MyInst(None)) raises(TypeError, m, MyInst(42)) + def test_invalid_creation(self): + import new + def f(): pass + raises(TypeError, new.instancemethod, f, None) + class TestMethod: def setup_method(self, method): Modified: pypy/branch/arm-backend/pypy/interpreter/typedef.py ============================================================================== --- pypy/branch/arm-backend/pypy/interpreter/typedef.py (original) +++ pypy/branch/arm-backend/pypy/interpreter/typedef.py Tue Nov 23 10:12:47 2010 @@ -133,6 +133,13 @@ typedef = cls.typedef if wants_dict and typedef.hasdict: wants_dict = False + if config.objspace.std.withmapdict and not typedef.hasdict: + # mapdict only works if the type does not already have a dict + if wants_del: + parentcls = get_unique_interplevel_subclass(config, cls, True, True, + False, True) + return _usersubclswithfeature(config, parentcls, "del") + return _usersubclswithfeature(config, cls, "user", "dict", "weakref", "slots") # Forest of if's - see the comment above. if wants_del: if wants_dict: @@ -186,10 +193,21 @@ def add(Proto): for key, value in Proto.__dict__.items(): - if not key.startswith('__') or key == '__del__': + if (not key.startswith('__') and not key.startswith('_mixin_') + or key == '__del__'): + if hasattr(value, "func_name"): + value = func_with_new_name(value, value.func_name) body[key] = value + if (config.objspace.std.withmapdict and "dict" in features): + from pypy.objspace.std.mapdict import BaseMapdictObject, ObjectMixin + add(BaseMapdictObject) + add(ObjectMixin) + body["user_overridden_class"] = True + features = () + if "user" in features: # generic feature needed by all subcls + class Proto(object): user_overridden_class = True @@ -245,16 +263,10 @@ return self.slots_w[index] add(Proto) - wantdict = "dict" in features - if wantdict and config.objspace.std.withinlineddict: - from pypy.objspace.std.objectobject import W_ObjectObject - from pypy.objspace.std.inlinedict import make_mixin - if supercls is W_ObjectObject: - Mixin = make_mixin(config) - add(Mixin) - wantdict = False - - if wantdict: + if "dict" in features: + base_user_setup = supercls.user_setup.im_func + if "user_setup" in body: + base_user_setup = body["user_setup"] class Proto(object): def getdict(self): return self.w__dict__ @@ -263,24 +275,13 @@ self.w__dict__ = check_new_dictionary(space, w_dict) def user_setup(self, space, w_subtype): - self.space = space - self.w__class__ = w_subtype self.w__dict__ = space.newdict( instance=True, classofinstance=w_subtype) - self.user_setup_slots(w_subtype.nslots) + base_user_setup(self, space, w_subtype) def setclass(self, space, w_subtype): # only used by descr_set___class__ self.w__class__ = w_subtype - if space.config.objspace.std.withshadowtracking: - self.w__dict__.set_shadows_anything() - - def getdictvalue_attr_is_in_class(self, space, name): - w_dict = self.w__dict__ - if space.config.objspace.std.withshadowtracking: - if not w_dict.shadows_anything(): - return None - return space.finditem_str(w_dict, name) add(Proto) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Tue Nov 23 10:12:47 2010 @@ -173,7 +173,8 @@ reg = r.lr # XXX free this memory # XXX allocate correct amount of memory - mem = lltype.malloc(rffi.CArray(lltype.Char), len(args)*6+9, flavor='raw') + mem = lltype.malloc(rffi.CArray(lltype.Char), len(args)*6+9, + flavor='raw', track_allocation=False) # Note, the actual frame depth is one less than the value stored in # regalloc.frame_manager.frame_depth self.encode32(mem, 0, regalloc.frame_manager.frame_depth - 1) Modified: pypy/branch/arm-backend/pypy/jit/backend/detect_cpu.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/detect_cpu.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/detect_cpu.py Tue Nov 23 10:12:47 2010 @@ -23,11 +23,6 @@ mach = os.popen('uname -m', 'r').read().strip() if not mach: raise ProcessorAutodetectError, "cannot run 'uname -m'" - if mach == 'x86_64': - if sys.maxint == 2147483647: - mach = 'x86' # it's a 64-bit processor but in 32-bits mode, maybe - else: - assert sys.maxint == 2 ** 63 - 1 try: return {'i386': 'x86', 'i486': 'x86', @@ -40,14 +35,28 @@ 'armv7l': 'arm', }[mach] except KeyError: - raise ProcessorAutodetectError, "unsupported processor '%s'" % mach + return mach + +def autodetect_main_model_and_size(): + model = autodetect_main_model() + if sys.maxint == 2**31-1: + model += '_32' + elif sys.maxint == 2**63-1: + model += '_64' + else: + raise AssertionError, "bad value for sys.maxint" + return model def autodetect(): model = autodetect_main_model() - if model == 'x86': - from pypy.jit.backend.x86.detect_sse2 import detect_sse2 - if not detect_sse2(): - model = 'x86-without-sse2' + if sys.maxint == 2**63-1: + model += '_64' + else: + assert sys.maxint == 2**31-1 + if model == 'x86': + from pypy.jit.backend.x86.detect_sse2 import detect_sse2 + if not detect_sse2(): + model = 'x86-without-sse2' return model def getcpuclassname(backend_name="auto"): Modified: pypy/branch/arm-backend/pypy/jit/backend/llgraph/llimpl.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/llgraph/llimpl.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/llgraph/llimpl.py Tue Nov 23 10:12:47 2010 @@ -10,7 +10,7 @@ BoxInt, BoxPtr, BoxObj, BoxFloat, REF, INT, FLOAT) from pypy.jit.codewriter import heaptracker -from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rstr +from pypy.rpython.lltypesystem import lltype, llmemory, rclass, rstr, rffi from pypy.rpython.ootypesystem import ootype from pypy.rpython.module.support import LLSupport, OOSupport from pypy.rpython.llinterp import LLException @@ -129,7 +129,7 @@ 'arraylen_gc' : (('ref',), 'int'), 'call' : (('ref', 'varargs'), 'intorptr'), 'call_assembler' : (('varargs',), 'intorptr'), - 'cond_call_gc_wb' : (('ptr',), None), + 'cond_call_gc_wb' : (('ptr', 'ptr'), None), 'oosend' : (('varargs',), 'intorptr'), 'oosend_pure' : (('varargs',), 'intorptr'), 'guard_true' : (('bool',), None), @@ -152,7 +152,7 @@ 'unicodegetitem' : (('ref', 'int'), 'int'), 'unicodesetitem' : (('ref', 'int', 'int'), 'int'), 'cast_ptr_to_int' : (('ref',), 'int'), - 'debug_merge_point': (('ref',), None), + 'debug_merge_point': (('ref', 'int'), None), 'force_token' : ((), 'int'), 'call_may_force' : (('int', 'varargs'), 'intorptr'), 'guard_not_forced': ((), None), @@ -305,12 +305,12 @@ loop = _from_opaque(loop) loop.operations.append(Operation(opnum)) -def compile_add_descr(loop, ofs, type): +def compile_add_descr(loop, ofs, type, arg_types): from pypy.jit.backend.llgraph.runner import Descr loop = _from_opaque(loop) op = loop.operations[-1] assert isinstance(type, str) and len(type) == 1 - op.descr = Descr(ofs, type) + op.descr = Descr(ofs, type, arg_types=arg_types) def compile_add_loop_token(loop, descr): if we_are_translated(): @@ -568,7 +568,7 @@ # return _op_default_implementation - def op_debug_merge_point(self, _, value): + def op_debug_merge_point(self, _, value, recdepth): from pypy.jit.metainterp.warmspot import get_stats loc = ConstPtr(value)._get_str() get_stats().add_merge_point_location(loc) @@ -801,7 +801,7 @@ else: raise TypeError(x) try: - return _do_call_common(func, args_in_order) + return _do_call_common(func, args_in_order, calldescr) except LLException, lle: _last_exception = lle d = {'v': None, @@ -810,7 +810,7 @@ FLOAT: 0.0} return d[calldescr.typeinfo] - def op_cond_call_gc_wb(self, descr, a): + def op_cond_call_gc_wb(self, descr, a, b): py.test.skip("cond_call_gc_wb not supported") def op_oosend(self, descr, obj, *args): @@ -1018,6 +1018,9 @@ if isinstance(TYPE, lltype.Ptr): if isinstance(x, (int, long, llmemory.AddressAsInt)): x = llmemory.cast_int_to_adr(x) + if TYPE is rffi.VOIDP: + # assume that we want a "C-style" cast, without typechecking the value + return rffi.cast(TYPE, x) return llmemory.cast_adr_to_ptr(x, TYPE) elif TYPE == llmemory.Address: if isinstance(x, (int, long, llmemory.AddressAsInt)): @@ -1411,10 +1414,26 @@ def do_call_pushfloat(x): _call_args_f.append(x) -def _do_call_common(f, args_in_order=None): +kind2TYPE = { + 'i': lltype.Signed, + 'f': lltype.Float, + 'v': lltype.Void, + } + +def _do_call_common(f, args_in_order=None, calldescr=None): ptr = llmemory.cast_int_to_adr(f).ptr - FUNC = lltype.typeOf(ptr).TO - ARGS = FUNC.ARGS + PTR = lltype.typeOf(ptr) + if PTR == rffi.VOIDP: + # it's a pointer to a C function, so we don't have a precise + # signature: create one from the descr + ARGS = map(kind2TYPE.get, calldescr.arg_types) + RESULT = kind2TYPE[calldescr.typeinfo] + FUNC = lltype.FuncType(ARGS, RESULT) + func_to_call = rffi.cast(lltype.Ptr(FUNC), ptr) + else: + FUNC = PTR.TO + ARGS = FUNC.ARGS + func_to_call = ptr._obj._callable args = cast_call_args(ARGS, _call_args_i, _call_args_r, _call_args_f, args_in_order) del _call_args_i[:] @@ -1426,7 +1445,7 @@ result = llinterp.eval_graph(ptr._obj.graph, args) # ^^^ may raise, in which case we get an LLException else: - result = ptr._obj._callable(*args) + result = func_to_call(*args) return result def do_call_void(f): Modified: pypy/branch/arm-backend/pypy/jit/backend/llgraph/runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/llgraph/runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/llgraph/runner.py Tue Nov 23 10:12:47 2010 @@ -118,13 +118,13 @@ self._descrs[key] = descr return descr - def compile_bridge(self, faildescr, inputargs, operations): + def compile_bridge(self, faildescr, inputargs, operations, log=True): c = llimpl.compile_start() self._compile_loop_or_bridge(c, inputargs, operations) old, oldindex = faildescr._compiled_fail llimpl.compile_redirect_fail(old, oldindex, c) - def compile_loop(self, inputargs, operations, loopdescr): + def compile_loop(self, inputargs, operations, loopdescr, log=True): """In a real assembler backend, this should assemble the given list of operations. Here we just generate a similar CompiledLoop instance. The code here is RPython, whereas the code in llimpl @@ -154,7 +154,7 @@ llimpl.compile_add(c, op.getopnum()) descr = op.getdescr() if isinstance(descr, Descr): - llimpl.compile_add_descr(c, descr.ofs, descr.typeinfo) + llimpl.compile_add_descr(c, descr.ofs, descr.typeinfo, descr.arg_types) if isinstance(descr, history.LoopToken) and op.getopnum() != rop.JUMP: llimpl.compile_add_loop_token(c, descr) if self.is_oo and isinstance(descr, (OODescr, MethDescr)): @@ -297,6 +297,18 @@ return self.getdescr(0, token[0], extrainfo=extrainfo, arg_types=''.join(arg_types)) + def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None): + from pypy.jit.backend.llsupport.ffisupport import get_ffi_type_kind + arg_types = [] + for arg in ffi_args: + kind = get_ffi_type_kind(arg) + if kind != history.VOID: + arg_types.append(kind) + reskind = get_ffi_type_kind(ffi_result) + return self.getdescr(0, reskind, extrainfo=extrainfo, + arg_types=''.join(arg_types)) + + def grab_exc_value(self): return llimpl.grab_exc_value() Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/descr.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/llsupport/descr.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/descr.py Tue Nov 23 10:12:47 2010 @@ -82,6 +82,7 @@ _is_pointer_field = False # unless overridden by GcPtrFieldDescr _is_float_field = False # unless overridden by FloatFieldDescr + _is_field_signed = False # unless overridden by XxxFieldDescr def is_pointer_field(self): return self._is_pointer_field @@ -89,6 +90,9 @@ def is_float_field(self): return self._is_float_field + def is_field_signed(self): + return self._is_field_signed + def repr_of_descr(self): return '<%s %s %s>' % (self._clsname, self.name, self.offset) @@ -105,7 +109,7 @@ def getFieldDescrClass(TYPE): return getDescrClass(TYPE, BaseFieldDescr, GcPtrFieldDescr, NonGcPtrFieldDescr, 'Field', 'get_field_size', - '_is_float_field') + '_is_float_field', '_is_field_signed') def get_field_descr(gccache, STRUCT, fieldname): cache = gccache._cache_field @@ -126,6 +130,7 @@ # ArrayDescrs _A = lltype.GcArray(lltype.Signed) # a random gcarray +_AF = lltype.GcArray(lltype.Float) # an array of C doubles class BaseArrayDescr(AbstractDescr): @@ -144,6 +149,7 @@ _is_array_of_pointers = False # unless overridden by GcPtrArrayDescr _is_array_of_floats = False # unless overridden by FloatArrayDescr + _is_item_signed = False # unless overridden by XxxArrayDescr def is_array_of_pointers(self): return self._is_array_of_pointers @@ -151,6 +157,9 @@ def is_array_of_floats(self): return self._is_array_of_floats + def is_item_signed(self): + return self._is_item_signed + def repr_of_descr(self): return '<%s>' % self._clsname @@ -163,16 +172,21 @@ _clsname = 'GcPtrArrayDescr' _is_array_of_pointers = True -_CA = rffi.CArray(lltype.Signed) +class FloatArrayDescr(BaseArrayDescr): + _clsname = 'FloatArrayDescr' + _is_array_of_floats = True + def get_base_size(self, translate_support_code): + basesize, _, _ = symbolic.get_array_token(_AF, translate_support_code) + return basesize + def get_item_size(self, translate_support_code): + return symbolic.get_size(lltype.Float, translate_support_code) class BaseArrayNoLengthDescr(BaseArrayDescr): def get_base_size(self, translate_support_code): - basesize, _, _ = symbolic.get_array_token(_CA, translate_support_code) - return basesize + return 0 def get_ofs_length(self, translate_support_code): - _, _, ofslength = symbolic.get_array_token(_CA, translate_support_code) - return ofslength + return -1 class NonGcPtrArrayNoLengthDescr(BaseArrayNoLengthDescr): _clsname = 'NonGcPtrArrayNoLengthDescr' @@ -184,14 +198,16 @@ _is_array_of_pointers = True def getArrayDescrClass(ARRAY): + if ARRAY.OF is lltype.Float: + return FloatArrayDescr return getDescrClass(ARRAY.OF, BaseArrayDescr, GcPtrArrayDescr, NonGcPtrArrayDescr, 'Array', 'get_item_size', - '_is_array_of_floats') + '_is_array_of_floats', '_is_item_signed') def getArrayNoLengthDescrClass(ARRAY): return getDescrClass(ARRAY.OF, BaseArrayNoLengthDescr, GcPtrArrayNoLengthDescr, NonGcPtrArrayNoLengthDescr, 'ArrayNoLength', 'get_item_size', - '_is_array_of_floats') + '_is_array_of_floats', '_is_item_signed') def get_array_descr(gccache, ARRAY): cache = gccache._cache_array @@ -211,7 +227,8 @@ basesize, itemsize, ofslength = symbolic.get_array_token(ARRAY, False) assert basesize == arraydescr.get_base_size(False) assert itemsize == arraydescr.get_item_size(False) - assert ofslength == arraydescr.get_ofs_length(False) + if not ARRAY._hints.get('nolength', False): + assert ofslength == arraydescr.get_ofs_length(False) if isinstance(ARRAY, lltype.GcArray): gccache.init_array_descr(ARRAY, arraydescr) cache[ARRAY] = arraydescr @@ -242,6 +259,9 @@ def get_result_size(self, translate_support_code): raise NotImplementedError + def is_result_signed(self): + return False # unless overridden + def create_call_stub(self, rtyper, RESULT): def process(c): arg = 'args_%s[%d]' % (c, seen[c]) @@ -307,6 +327,30 @@ _return_type = history.INT call_stub = staticmethod(lambda func, args_i, args_r, args_f: 0) + _is_result_signed = False # can be overridden in XxxCallDescr + def is_result_signed(self): + return self._is_result_signed + +class DynamicIntCallDescr(BaseIntCallDescr): + """ + calldescr that works for every integer type, by explicitly passing it the + size of the result. Used only by get_call_descr_dynamic + """ + _clsname = 'DynamicIntCallDescr' + + def __init__(self, arg_classes, result_size, result_sign, extrainfo=None): + BaseIntCallDescr.__init__(self, arg_classes, extrainfo) + assert isinstance(result_sign, bool) + self._result_size = chr(result_size) + self._result_sign = result_sign + + def get_result_size(self, translate_support_code): + return ord(self._result_size) + + def is_result_signed(self): + return self._result_sign + + class NonGcPtrCallDescr(BaseIntCallDescr): _clsname = 'NonGcPtrCallDescr' def get_result_size(self, translate_support_code): @@ -341,7 +385,8 @@ return FloatCallDescr return getDescrClass(RESULT, BaseIntCallDescr, GcPtrCallDescr, NonGcPtrCallDescr, 'Call', 'get_result_size', - Ellipsis) # <= floatattrname should not be used here + Ellipsis, # <= floatattrname should not be used here + '_is_result_signed') def get_call_descr(gccache, ARGS, RESULT, extrainfo=None): arg_classes = [] @@ -368,7 +413,8 @@ # ____________________________________________________________ def getDescrClass(TYPE, BaseDescr, GcPtrDescr, NonGcPtrDescr, - nameprefix, methodname, floatattrname, _cache={}): + nameprefix, methodname, floatattrname, signedattrname, + _cache={}): if isinstance(TYPE, lltype.Ptr): if TYPE.TO._gckind == 'gc': return GcPtrDescr @@ -388,6 +434,8 @@ # if TYPE is lltype.Float: setattr(Descr, floatattrname, True) + elif TYPE is not lltype.Bool and rffi.cast(TYPE, -1) == -1: + setattr(Descr, signedattrname, True) # _cache[nameprefix, TYPE] = Descr return Descr Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/gc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/llsupport/gc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/gc.py Tue Nov 23 10:12:47 2010 @@ -19,6 +19,7 @@ # ____________________________________________________________ class GcLLDescription(GcCache): + minimal_size_in_nursery = 0 def __init__(self, gcdescr, translator=None, rtyper=None): GcCache.__init__(self, translator is not None, rtyper) self.gcdescr = gcdescr @@ -158,7 +159,7 @@ # used to avoid too many duplications in the GCREF_LISTs. self.hashtable = lltype.malloc(self.HASHTABLE, self.HASHTABLE_SIZE+1, - flavor='raw') + flavor='raw', track_allocation=False) dummy = lltype.direct_ptradd(lltype.direct_arrayitems(self.hashtable), self.HASHTABLE_SIZE) dummy = llmemory.cast_ptr_to_adr(dummy) @@ -252,14 +253,15 @@ def _enlarge_gcmap(self): newlength = 250 + self._gcmap_maxlength * 2 - newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw') + newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw', + track_allocation=False) oldgcmap = self._gcmap for i in range(self._gcmap_curlength): newgcmap[i] = oldgcmap[i] self._gcmap = newgcmap self._gcmap_maxlength = newlength if oldgcmap: - lltype.free(oldgcmap, flavor='raw') + lltype.free(oldgcmap, flavor='raw', track_allocation=False) def get_basic_shape(self, is_64_bit=False): # XXX: Should this code even really know about stack frame layout of @@ -308,7 +310,8 @@ # them inside bigger arrays) and we never try to share them. length = len(shape) compressed = lltype.malloc(self.CALLSHAPE_ARRAY, length, - flavor='raw') + flavor='raw', + track_allocation=False) # memory leak for i in range(length): compressed[length-1-i] = rffi.cast(rffi.UCHAR, shape[i]) return llmemory.cast_ptr_to_adr(compressed) @@ -384,6 +387,7 @@ (self.array_basesize, _, self.array_length_ofs) = \ symbolic.get_array_token(lltype.GcArray(lltype.Signed), True) self.max_size_of_young_obj = self.GCClass.JIT_max_size_of_young_obj() + self.minimal_size_in_nursery=self.GCClass.JIT_minimal_size_in_nursery() # make a malloc function, with three arguments def malloc_basic(size, tid): @@ -404,7 +408,7 @@ self.GC_MALLOC_BASIC = lltype.Ptr(lltype.FuncType( [lltype.Signed, lltype.Signed], llmemory.GCREF)) self.WB_FUNCPTR = lltype.Ptr(lltype.FuncType( - [llmemory.Address], lltype.Void)) + [llmemory.Address, llmemory.Address], lltype.Void)) self.write_barrier_descr = WriteBarrierDescr(self) # def malloc_array(itemsize, tid, num_elem): @@ -466,6 +470,7 @@ def malloc_fixedsize_slowpath(size): if self.DEBUG: random_usage_of_xmm_registers() + assert size >= self.minimal_size_in_nursery try: gcref = llop1.do_malloc_fixedsize_clear(llmemory.GCREF, 0, size, True, False, False) @@ -550,7 +555,8 @@ # the GC, and call it immediately llop1 = self.llop1 funcptr = llop1.get_write_barrier_failing_case(self.WB_FUNCPTR) - funcptr(llmemory.cast_ptr_to_adr(gcref_struct)) + funcptr(llmemory.cast_ptr_to_adr(gcref_struct), + llmemory.cast_ptr_to_adr(gcref_newptr)) def rewrite_assembler(self, cpu, operations): # Perform two kinds of rewrites in parallel: @@ -567,6 +573,9 @@ # GETFIELD_RAW from the array 'gcrefs.list'. # newops = [] + # we can only remember one malloc since the next malloc can possibly + # collect + last_malloc = None for op in operations: if op.getopnum() == rop.DEBUG_MERGE_POINT: continue @@ -584,29 +593,39 @@ [ConstInt(addr)], box, self.single_gcref_descr)) op.setarg(i, box) + if op.is_malloc(): + last_malloc = op.result + elif op.can_malloc(): + last_malloc = None # ---------- write barrier for SETFIELD_GC ---------- if op.getopnum() == rop.SETFIELD_GC: - v = op.getarg(1) - if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and - bool(v.value)): # store a non-NULL - self._gen_write_barrier(newops, op.getarg(0)) - op = op.copy_and_change(rop.SETFIELD_RAW) + val = op.getarg(0) + # no need for a write barrier in the case of previous malloc + if val is not last_malloc: + v = op.getarg(1) + if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and + bool(v.value)): # store a non-NULL + self._gen_write_barrier(newops, op.getarg(0), v) + op = op.copy_and_change(rop.SETFIELD_RAW) # ---------- write barrier for SETARRAYITEM_GC ---------- if op.getopnum() == rop.SETARRAYITEM_GC: - v = op.getarg(2) - if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and - bool(v.value)): # store a non-NULL - # XXX detect when we should produce a - # write_barrier_from_array - self._gen_write_barrier(newops, op.getarg(0)) - op = op.copy_and_change(rop.SETARRAYITEM_RAW) + val = op.getarg(0) + # no need for a write barrier in the case of previous malloc + if val is not last_malloc: + v = op.getarg(2) + if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and + bool(v.value)): # store a non-NULL + # XXX detect when we should produce a + # write_barrier_from_array + self._gen_write_barrier(newops, op.getarg(0), v) + op = op.copy_and_change(rop.SETARRAYITEM_RAW) # ---------- newops.append(op) del operations[:] operations.extend(newops) - def _gen_write_barrier(self, newops, v_base): - args = [v_base] + def _gen_write_barrier(self, newops, v_base, v_value): + args = [v_base, v_value] newops.append(ResOperation(rop.COND_CALL_GC_WB, args, None, descr=self.write_barrier_descr)) Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/llmodel.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/llsupport/llmodel.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/llmodel.py Tue Nov 23 10:12:47 2010 @@ -17,6 +17,7 @@ from pypy.jit.backend.llsupport.descr import get_call_descr from pypy.jit.backend.llsupport.descr import BaseIntCallDescr, GcPtrCallDescr from pypy.jit.backend.llsupport.descr import FloatCallDescr, VoidCallDescr +from pypy.jit.backend.llsupport.ffisupport import get_call_descr_dynamic from pypy.rpython.annlowlevel import cast_instance_to_base_ptr @@ -82,7 +83,8 @@ # read back by the machine code reading at the address given by # pos_exception() and pos_exc_value(). _exception_emulator = lltype.malloc(rffi.CArray(lltype.Signed), 2, - zero=True, flavor='raw') + zero=True, flavor='raw', + immortal=True) self._exception_emulator = _exception_emulator def _store_exception(lle): @@ -210,7 +212,8 @@ assert isinstance(fielddescr, BaseFieldDescr) ofs = fielddescr.offset size = fielddescr.get_field_size(self.translate_support_code) - return ofs, size + sign = fielddescr.is_field_signed() + return ofs, size, sign unpack_fielddescr_size._always_inline_ = True def arraydescrof(self, A): @@ -225,12 +228,16 @@ assert isinstance(arraydescr, BaseArrayDescr) ofs = arraydescr.get_base_size(self.translate_support_code) size = arraydescr.get_item_size(self.translate_support_code) - return ofs, size + sign = arraydescr.is_item_signed() + return ofs, size, sign unpack_arraydescr_size._always_inline_ = True def calldescrof(self, FUNC, ARGS, RESULT, extrainfo=None): return get_call_descr(self.gc_ll_descr, ARGS, RESULT, extrainfo) + def calldescrof_dynamic(self, ffi_args, ffi_result, extrainfo=None): + return get_call_descr_dynamic(ffi_args, ffi_result, extrainfo) + def get_overflow_error(self): ovf_vtable = self.cast_adr_to_int(self._ovf_error_vtable) ovf_inst = lltype.cast_opaque_ptr(llmemory.GCREF, @@ -252,15 +259,21 @@ @specialize.argtype(2) def bh_getarrayitem_gc_i(self, arraydescr, gcref, itemindex): - ofs, size = self.unpack_arraydescr_size(arraydescr) + ofs, size, sign = self.unpack_arraydescr_size(arraydescr) # --- start of GC unsafe code (no GC operation!) --- items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs) - for TYPE, itemsize in unroll_basic_sizes: + for STYPE, UTYPE, itemsize in unroll_basic_sizes: if size == itemsize: - items = rffi.cast(rffi.CArrayPtr(TYPE), items) - val = items[itemindex] + if sign: + items = rffi.cast(rffi.CArrayPtr(STYPE), items) + val = items[itemindex] + val = rffi.cast(lltype.Signed, val) + else: + items = rffi.cast(rffi.CArrayPtr(UTYPE), items) + val = items[itemindex] + val = rffi.cast(lltype.Signed, val) # --- end of GC unsafe code --- - return rffi.cast(lltype.Signed, val) + return val else: raise NotImplementedError("size = %d" % size) @@ -285,10 +298,10 @@ @specialize.argtype(2) def bh_setarrayitem_gc_i(self, arraydescr, gcref, itemindex, newvalue): - ofs, size = self.unpack_arraydescr_size(arraydescr) + ofs, size, sign = self.unpack_arraydescr_size(arraydescr) # --- start of GC unsafe code (no GC operation!) --- items = rffi.ptradd(rffi.cast(rffi.CCHARP, gcref), ofs) - for TYPE, itemsize in unroll_basic_sizes: + for TYPE, _, itemsize in unroll_basic_sizes: if size == itemsize: items = rffi.cast(rffi.CArrayPtr(TYPE), items) items[itemindex] = rffi.cast(TYPE, newvalue) @@ -339,14 +352,22 @@ @specialize.argtype(1) def _base_do_getfield_i(self, struct, fielddescr): - ofs, size = self.unpack_fielddescr_size(fielddescr) + ofs, size, sign = self.unpack_fielddescr_size(fielddescr) # --- start of GC unsafe code (no GC operation!) --- fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs) - for TYPE, itemsize in unroll_basic_sizes: + for STYPE, UTYPE, itemsize in unroll_basic_sizes: if size == itemsize: - val = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr)[0] + # Note that in the common case where size==sizeof(Signed), + # both cases of what follows are doing the same thing. + # But gcc is clever enough to figure this out :-) + if sign: + val = rffi.cast(rffi.CArrayPtr(STYPE), fieldptr)[0] + val = rffi.cast(lltype.Signed, val) + else: + val = rffi.cast(rffi.CArrayPtr(UTYPE), fieldptr)[0] + val = rffi.cast(lltype.Signed, val) # --- end of GC unsafe code --- - return rffi.cast(lltype.Signed, val) + return val else: raise NotImplementedError("size = %d" % size) @@ -378,10 +399,10 @@ @specialize.argtype(1) def _base_do_setfield_i(self, struct, fielddescr, newvalue): - ofs, size = self.unpack_fielddescr_size(fielddescr) + ofs, size, sign = self.unpack_fielddescr_size(fielddescr) # --- start of GC unsafe code (no GC operation!) --- fieldptr = rffi.ptradd(rffi.cast(rffi.CCHARP, struct), ofs) - for TYPE, itemsize in unroll_basic_sizes: + for TYPE, _, itemsize in unroll_basic_sizes: if size == itemsize: fieldptr = rffi.cast(rffi.CArrayPtr(TYPE), fieldptr) fieldptr[0] = rffi.cast(TYPE, newvalue) Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py Tue Nov 23 10:12:47 2010 @@ -365,8 +365,9 @@ arg = op.getarg(j) if isinstance(arg, Box): if arg not in start_live: - print "Bogus arg in operation %d at %d" % (op.getopnum(), i) - raise AssertionError + not_implemented("Bogus arg in operation %d at %d" % + (op.getopnum(), i)) + longevity[arg] = (start_live[arg], i) if op.is_guard(): for arg in op.getfailargs(): @@ -374,8 +375,8 @@ continue assert isinstance(arg, Box) if arg not in start_live: - print "Bogus arg in guard %d at %d" % (op.getopnum(), i) - raise AssertionError + not_implemented("Bogus arg in guard %d at %d" % + (op.getopnum(), i)) longevity[arg] = (start_live[arg], i) for arg in inputargs: if arg not in longevity: @@ -394,3 +395,6 @@ loop_consts[inputargs[i]] = i return loop_consts +def not_implemented(msg): + os.write(2, '[llsupport/regalloc] %s\n' % msg) + raise NotImplementedError(msg) Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/symbolic.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/llsupport/symbolic.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/symbolic.py Tue Nov 23 10:12:47 2010 @@ -69,8 +69,9 @@ SIZEOF_INT = get_size(rffi.INT, False) SIZEOF_FLOAT = get_size(lltype.Float, False) -unroll_basic_sizes = unrolling_iterable([(lltype.Signed, WORD), - (lltype.Char, SIZEOF_CHAR), - (rffi.SHORT, SIZEOF_SHORT), - (rffi.INT, SIZEOF_INT)]) +unroll_basic_sizes = unrolling_iterable([ + (lltype.Signed, lltype.Unsigned, WORD), + (rffi.SIGNEDCHAR, lltype.Char, SIZEOF_CHAR), + (rffi.SHORT, rffi.USHORT, SIZEOF_SHORT), + (rffi.INT, rffi.UINT, SIZEOF_INT)]) # does not contain Float ^^^ which must be special-cased Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_descr.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_descr.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_descr.py Tue Nov 23 10:12:47 2010 @@ -5,6 +5,7 @@ from pypy.rpython.annlowlevel import llhelper from pypy.jit.metainterp.history import BoxInt, BoxFloat, BoxPtr from pypy.jit.metainterp import history +import struct def test_get_size_descr(): c0 = GcCache(False) @@ -83,6 +84,18 @@ assert descr_f.is_float_field() +def test_get_field_descr_sign(): + for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False), + (rffi.SHORT, True), (rffi.USHORT, False), + (rffi.INT, True), (rffi.UINT, False), + (rffi.LONG, True), (rffi.ULONG, False)]: + S = lltype.GcStruct('S', ('x', RESTYPE)) + for tsc in [False, True]: + c2 = GcCache(tsc) + descr_x = get_field_descr(c2, S, 'x') + assert descr_x.is_field_signed() == signed + + def test_get_array_descr(): U = lltype.Struct('U') T = lltype.GcStruct('T') @@ -118,11 +131,13 @@ assert not descr3.is_array_of_floats() assert descr4.is_array_of_floats() # - WORD = rffi.sizeof(lltype.Signed) - assert descr1.get_base_size(False) == WORD - assert descr2.get_base_size(False) == WORD - assert descr3.get_base_size(False) == WORD - assert descr4.get_base_size(False) == WORD + def get_alignment(code): + # Retrieve default alignment for the compiler/platform + return struct.calcsize('l' + code) - struct.calcsize(code) + assert descr1.get_base_size(False) == get_alignment('c') + assert descr2.get_base_size(False) == get_alignment('p') + assert descr3.get_base_size(False) == get_alignment('p') + assert descr4.get_base_size(False) == get_alignment('d') assert descr1.get_ofs_length(False) == 0 assert descr2.get_ofs_length(False) == 0 assert descr3.get_ofs_length(False) == 0 @@ -164,6 +179,25 @@ assert descr.get_base_size(False) == 0 assert descr.get_ofs_length(False) == -1 + +def test_get_array_descr_sign(): + for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False), + (rffi.SHORT, True), (rffi.USHORT, False), + (rffi.INT, True), (rffi.UINT, False), + (rffi.LONG, True), (rffi.ULONG, False)]: + A = lltype.GcArray(RESTYPE) + for tsc in [False, True]: + c2 = GcCache(tsc) + arraydescr = get_array_descr(c2, A) + assert arraydescr.is_item_signed() == signed + # + RA = rffi.CArray(RESTYPE) + for tsc in [False, True]: + c2 = GcCache(tsc) + arraydescr = get_array_descr(c2, RA) + assert arraydescr.is_item_signed() == signed + + def test_get_call_descr_not_translated(): c0 = GcCache(False) descr1 = get_call_descr(c0, [lltype.Char, lltype.Signed], lltype.Char) @@ -219,6 +253,17 @@ extrainfo = descr3.get_extra_info() assert extrainfo is None +def test_get_call_descr_sign(): + for RESTYPE, signed in [(rffi.SIGNEDCHAR, True), (rffi.UCHAR, False), + (rffi.SHORT, True), (rffi.USHORT, False), + (rffi.INT, True), (rffi.UINT, False), + (rffi.LONG, True), (rffi.ULONG, False)]: + A = lltype.GcArray(RESTYPE) + for tsc in [False, True]: + c2 = GcCache(tsc) + descr1 = get_call_descr(c2, [], RESTYPE) + assert descr1.is_result_signed() == signed + def test_repr_of_descr(): c0 = GcCache(False) Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_gc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_gc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/test/test_gc.py Tue Nov 23 10:12:47 2010 @@ -6,7 +6,9 @@ from pypy.jit.backend.llsupport.gc import * from pypy.jit.backend.llsupport import symbolic from pypy.jit.metainterp.gc import get_description - +from pypy.jit.tool.oparser import parse +from pypy.rpython.lltypesystem.rclass import OBJECT, OBJECT_VTABLE +from pypy.jit.metainterp.test.test_optimizeopt import equaloplists def test_boehm(): gc_ll_descr = GcLLDescr_boehm(None, None, None) @@ -114,7 +116,7 @@ assert gcrootmap._gcmap[i*2+1] == expected_shapeaddr[i] -class FakeLLOp: +class FakeLLOp(object): def __init__(self): self.record = [] @@ -141,26 +143,26 @@ repr(offset_to_length), p)) return p - def _write_barrier_failing_case(self, adr_struct): - self.record.append(('barrier', adr_struct)) + def _write_barrier_failing_case(self, adr_struct, adr_newptr): + self.record.append(('barrier', adr_struct, adr_newptr)) def get_write_barrier_failing_case(self, FPTRTYPE): return llhelper(FPTRTYPE, self._write_barrier_failing_case) -class TestFramework: +class TestFramework(object): gc = 'hybrid' def setup_method(self, meth): - class config_: - class translation: + class config_(object): + class translation(object): gc = self.gc gcrootfinder = 'asmgcc' gctransformer = 'framework' gcremovetypeptr = False - class FakeTranslator: + class FakeTranslator(object): config = config_ - class FakeCPU: + class FakeCPU(object): def cast_adr_to_int(self, adr): ptr = llmemory.cast_adr_to_ptr(adr, gc_ll_descr.WB_FUNCPTR) assert ptr._obj._callable == llop1._write_barrier_failing_case @@ -239,6 +241,7 @@ s_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, s) r_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, r) s_adr = llmemory.cast_ptr_to_adr(s) + r_adr = llmemory.cast_ptr_to_adr(r) # s_hdr.tid &= ~gc_ll_descr.GCClass.JIT_WB_IF_FLAG gc_ll_descr.do_write_barrier(s_gcref, r_gcref) @@ -246,7 +249,7 @@ # s_hdr.tid |= gc_ll_descr.GCClass.JIT_WB_IF_FLAG gc_ll_descr.do_write_barrier(s_gcref, r_gcref) - assert self.llop1.record == [('barrier', s_adr)] + assert self.llop1.record == [('barrier', s_adr, r_adr)] def test_gen_write_barrier(self): gc_ll_descr = self.gc_ll_descr @@ -254,11 +257,13 @@ # newops = [] v_base = BoxPtr() - gc_ll_descr._gen_write_barrier(newops, v_base) + v_value = BoxPtr() + gc_ll_descr._gen_write_barrier(newops, v_base, v_value) assert llop1.record == [] assert len(newops) == 1 assert newops[0].getopnum() == rop.COND_CALL_GC_WB assert newops[0].getarg(0) == v_base + assert newops[0].getarg(1) == v_value assert newops[0].result is None wbdescr = newops[0].getdescr() assert isinstance(wbdescr.jit_wb_if_flag, int) @@ -267,7 +272,7 @@ def test_get_rid_of_debug_merge_point(self): operations = [ - ResOperation(rop.DEBUG_MERGE_POINT, ['dummy'], None), + ResOperation(rop.DEBUG_MERGE_POINT, ['dummy', 2], None), ] gc_ll_descr = self.gc_ll_descr gc_ll_descr.rewrite_assembler(None, operations) @@ -275,11 +280,11 @@ def test_rewrite_assembler_1(self): # check rewriting of ConstPtrs - class MyFakeCPU: + class MyFakeCPU(object): def cast_adr_to_int(self, adr): assert adr == "some fake address" return 43 - class MyFakeGCRefList: + class MyFakeGCRefList(object): def get_address_of_gcref(self, s_gcref1): assert s_gcref1 == s_gcref return "some fake address" @@ -308,10 +313,10 @@ def test_rewrite_assembler_1_cannot_move(self): # check rewriting of ConstPtrs - class MyFakeCPU: + class MyFakeCPU(object): def cast_adr_to_int(self, adr): xxx # should not be called - class MyFakeGCRefList: + class MyFakeGCRefList(object): def get_address_of_gcref(self, s_gcref1): seen.append(s_gcref1) assert s_gcref1 == s_gcref @@ -358,6 +363,7 @@ # assert operations[0].getopnum() == rop.COND_CALL_GC_WB assert operations[0].getarg(0) == v_base + assert operations[0].getarg(1) == v_value assert operations[0].result is None # assert operations[1].getopnum() == rop.SETFIELD_RAW @@ -381,6 +387,7 @@ # assert operations[0].getopnum() == rop.COND_CALL_GC_WB assert operations[0].getarg(0) == v_base + assert operations[0].getarg(1) == v_value assert operations[0].result is None # assert operations[1].getopnum() == rop.SETARRAYITEM_RAW @@ -389,6 +396,68 @@ assert operations[1].getarg(2) == v_value assert operations[1].getdescr() == array_descr + def test_rewrite_assembler_initialization_store(self): + S = lltype.GcStruct('S', ('parent', OBJECT), + ('x', lltype.Signed)) + s_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True) + xdescr = get_field_descr(self.gc_ll_descr, S, 'x') + ops = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + # no write barrier + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) + + def test_rewrite_assembler_initialization_store_2(self): + S = lltype.GcStruct('S', ('parent', OBJECT), + ('x', lltype.Signed)) + s_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True) + wbdescr = self.gc_ll_descr.write_barrier_descr + xdescr = get_field_descr(self.gc_ll_descr, S, 'x') + ops = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + p3 = new_with_vtable(ConstClass(s_vtable)) + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + p3 = new_with_vtable(ConstClass(s_vtable)) + cond_call_gc_wb(p0, p1, descr=wbdescr) + setfield_raw(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) + + def test_rewrite_assembler_initialization_store_3(self): + A = lltype.GcArray(lltype.Ptr(lltype.GcStruct('S'))) + arraydescr = get_array_descr(self.gc_ll_descr, A) + ops = parse(""" + [p1] + p0 = new_array(3, descr=arraydescr) + setarrayitem_gc(p0, 0, p1, descr=arraydescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_array(3, descr=arraydescr) + setarrayitem_gc(p0, 0, p1, descr=arraydescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) class TestFrameworkMiniMark(TestFramework): gc = 'minimark' Modified: pypy/branch/arm-backend/pypy/jit/backend/model.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/model.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/model.py Tue Nov 23 10:12:47 2010 @@ -33,14 +33,14 @@ pass - def compile_loop(self, inputargs, operations, looptoken): + def compile_loop(self, inputargs, operations, looptoken, log=True): """Assemble the given loop. Extra attributes should be put in the LoopToken to point to the compiled loop in assembler. """ raise NotImplementedError - def compile_bridge(self, faildescr, inputargs, operations): + def compile_bridge(self, faildescr, inputargs, operations, log=True): """Assemble the bridge. The FailDescr is the descr of the original guard that failed. """ Modified: pypy/branch/arm-backend/pypy/jit/backend/test/runner_test.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/test/runner_test.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/test/runner_test.py Tue Nov 23 10:12:47 2010 @@ -9,12 +9,13 @@ ConstObj, BoxFloat, ConstFloat) from pypy.jit.metainterp.resoperation import ResOperation, rop from pypy.jit.metainterp.typesystem import deref -from pypy.jit.metainterp.test.oparser import parse +from pypy.jit.tool.oparser import parse from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi, rclass from pypy.rpython.ootypesystem import ootype from pypy.rpython.annlowlevel import llhelper from pypy.rpython.llinterp import LLException from pypy.jit.codewriter import heaptracker +from pypy.rlib.rarithmetic import intmask class Runner(object): @@ -421,6 +422,7 @@ assert x == 3.5 - 42 def test_call(self): + from pypy.rlib.libffi import types def func_int(a, b): return a + b @@ -428,23 +430,31 @@ return chr(ord(c) + ord(c1)) functions = [ - (func_int, lltype.Signed, 655360), - (func_int, rffi.SHORT, 1213), - (func_char, lltype.Char, 12) + (func_int, lltype.Signed, types.sint, 655360), + (func_int, rffi.SHORT, types.sint16, 1213), + (func_char, lltype.Char, types.uchar, 12) ] - for func, TP, num in functions: + for func, TP, ffi_type, num in functions: cpu = self.cpu # FPTR = self.Ptr(self.FuncType([TP, TP], TP)) func_ptr = llhelper(FPTR, func) FUNC = deref(FPTR) - calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT) funcbox = self.get_funcbox(cpu, func_ptr) + # first, try it with the "normal" calldescr + calldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT) res = self.execute_operation(rop.CALL, [funcbox, BoxInt(num), BoxInt(num)], 'int', descr=calldescr) assert res.value == 2 * num + # then, try it with the dynamic calldescr + dyn_calldescr = cpu.calldescrof_dynamic([ffi_type, ffi_type], ffi_type) + res = self.execute_operation(rop.CALL, + [funcbox, BoxInt(num), BoxInt(num)], + 'int', descr=dyn_calldescr) + assert res.value == 2 * num + if cpu.supports_floats: def func(f0, f1, f2, f3, f4, f5, f6, i0, i1, f7, f8, f9): @@ -507,6 +517,24 @@ 'int', descr=calldescr) assert res.value == func_ints(*args) + def test_call_to_c_function(self): + from pypy.rlib.libffi import CDLL, types, ArgChain + from pypy.rpython.lltypesystem.ll2ctypes import libc_name + libc = CDLL(libc_name) + c_tolower = libc.getpointer('tolower', [types.uchar], types.sint) + argchain = ArgChain().arg(ord('A')) + assert c_tolower.call(argchain, rffi.INT) == ord('a') + + func_adr = llmemory.cast_ptr_to_adr(c_tolower.funcsym) + funcbox = ConstInt(heaptracker.adr2int(func_adr)) + calldescr = self.cpu.calldescrof_dynamic([types.uchar], types.sint) + res = self.execute_operation(rop.CALL, + [funcbox, BoxInt(ord('A'))], + 'int', + descr=calldescr) + assert res.value == ord('a') + + def test_field_basic(self): t_box, T_box = self.alloc_instance(self.T) fielddescr = self.cpu.fielddescrof(self.S, 'value') @@ -833,6 +861,23 @@ length_box], 'void') assert self.look_string(r_box) == "!??cdef?!" + def test_copyunicodecontent(self): + s_box = self.alloc_unicode(u"abcdef") + for s_box in [s_box, s_box.constbox()]: + for srcstart_box in [BoxInt(2), ConstInt(2)]: + for dststart_box in [BoxInt(3), ConstInt(3)]: + for length_box in [BoxInt(4), ConstInt(4)]: + for r_box_is_const in [False, True]: + r_box = self.alloc_unicode(u"!???????!") + if r_box_is_const: + r_box = r_box.constbox() + self.execute_operation(rop.COPYUNICODECONTENT, + [s_box, r_box, + srcstart_box, + dststart_box, + length_box], 'void') + assert self.look_unicode(r_box) == u"!??cdef?!" + def test_do_unicode_basic(self): u = self.cpu.bh_newunicode(5) self.cpu.bh_unicodesetitem(u, 4, 123) @@ -1227,6 +1272,10 @@ u_box = BoxPtr(lltype.cast_opaque_ptr(llmemory.GCREF, u)) return u_box + def look_unicode(self, unicode_box): + u = unicode_box.getref(lltype.Ptr(rstr.UNICODE)) + return u''.join(u.chars) + def test_casts(self): py.test.skip("xxx fix or kill") @@ -1283,6 +1332,7 @@ descr=fd) res = self.execute_operation(get_op, [s_box], 'int', descr=fd) assert res.getint() == 32 + lltype.free(s, flavor='raw') def test_new_with_vtable(self): cpu = self.cpu @@ -1427,12 +1477,12 @@ assert not excvalue def test_cond_call_gc_wb(self): - def func_void(a): - record.append(a) + def func_void(a, b): + record.append((a, b)) record = [] # S = lltype.GcStruct('S', ('tid', lltype.Signed)) - FUNC = self.FuncType([lltype.Ptr(S)], lltype.Void) + FUNC = self.FuncType([lltype.Ptr(S), lltype.Signed], lltype.Void) func_ptr = llhelper(lltype.Ptr(FUNC), func_void) funcbox = self.get_funcbox(self.cpu, func_ptr) class WriteBarrierDescr(AbstractDescr): @@ -1453,10 +1503,10 @@ sgcref = lltype.cast_opaque_ptr(llmemory.GCREF, s) del record[:] self.execute_operation(rop.COND_CALL_GC_WB, - [BoxPtr(sgcref)], + [BoxPtr(sgcref), ConstInt(-2121)], 'void', descr=WriteBarrierDescr()) if cond: - assert record == [s] + assert record == [(s, -2121)] else: assert record == [] @@ -2001,6 +2051,200 @@ assert self.cpu.get_latest_value_float(0) == 13.5 assert called + def test_short_result_of_getfield_direct(self): + # Test that a getfield that returns a CHAR, SHORT or INT, signed + # or unsigned, properly gets zero-extended or sign-extended. + # Direct bh_xxx test. + cpu = self.cpu + for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR, + rffi.SHORT, rffi.USHORT, + rffi.INT, rffi.UINT, + rffi.LONG, rffi.ULONG]: + S = lltype.GcStruct('S', ('x', RESTYPE)) + descrfld_x = cpu.fielddescrof(S, 'x') + s = lltype.malloc(S) + value = intmask(0xFFEEDDCCBBAA9988) + expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value)) + s.x = rffi.cast(RESTYPE, value) + x = cpu.bh_getfield_gc_i(lltype.cast_opaque_ptr(llmemory.GCREF, s), + descrfld_x) + assert x == expected, ( + "%r: got %r, expected %r" % (RESTYPE, x, expected)) + + def test_short_result_of_getfield_compiled(self): + # Test that a getfield that returns a CHAR, SHORT or INT, signed + # or unsigned, properly gets zero-extended or sign-extended. + # Machine code compilation test. + cpu = self.cpu + for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR, + rffi.SHORT, rffi.USHORT, + rffi.INT, rffi.UINT, + rffi.LONG, rffi.ULONG]: + S = lltype.GcStruct('S', ('x', RESTYPE)) + descrfld_x = cpu.fielddescrof(S, 'x') + s = lltype.malloc(S) + value = intmask(0xFFEEDDCCBBAA9988) + expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value)) + s.x = rffi.cast(RESTYPE, value) + s_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, s) + res = self.execute_operation(rop.GETFIELD_GC, [BoxPtr(s_gcref)], + 'int', descr=descrfld_x) + assert res.value == expected, ( + "%r: got %r, expected %r" % (RESTYPE, res.value, expected)) + + def test_short_result_of_getarrayitem_direct(self): + # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed + # or unsigned, properly gets zero-extended or sign-extended. + # Direct bh_xxx test. + cpu = self.cpu + for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR, + rffi.SHORT, rffi.USHORT, + rffi.INT, rffi.UINT, + rffi.LONG, rffi.ULONG]: + A = lltype.GcArray(RESTYPE) + descrarray = cpu.arraydescrof(A) + a = lltype.malloc(A, 5) + value = intmask(0xFFEEDDCCBBAA9988) + expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value)) + a[3] = rffi.cast(RESTYPE, value) + x = cpu.bh_getarrayitem_gc_i( + descrarray, lltype.cast_opaque_ptr(llmemory.GCREF, a), 3) + assert x == expected, ( + "%r: got %r, expected %r" % (RESTYPE, x, expected)) + + def test_short_result_of_getarrayitem_compiled(self): + # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed + # or unsigned, properly gets zero-extended or sign-extended. + # Machine code compilation test. + cpu = self.cpu + for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR, + rffi.SHORT, rffi.USHORT, + rffi.INT, rffi.UINT, + rffi.LONG, rffi.ULONG]: + A = lltype.GcArray(RESTYPE) + descrarray = cpu.arraydescrof(A) + a = lltype.malloc(A, 5) + value = intmask(0xFFEEDDCCBBAA9988) + expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value)) + a[3] = rffi.cast(RESTYPE, value) + a_gcref = lltype.cast_opaque_ptr(llmemory.GCREF, a) + res = self.execute_operation(rop.GETARRAYITEM_GC, + [BoxPtr(a_gcref), BoxInt(3)], + 'int', descr=descrarray) + assert res.value == expected, ( + "%r: got %r, expected %r" % (RESTYPE, res.value, expected)) + + def test_short_result_of_getarrayitem_raw_direct(self): + # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed + # or unsigned, properly gets zero-extended or sign-extended. + # Direct bh_xxx test. + cpu = self.cpu + for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR, + rffi.SHORT, rffi.USHORT, + rffi.INT, rffi.UINT, + rffi.LONG, rffi.ULONG]: + A = rffi.CArray(RESTYPE) + descrarray = cpu.arraydescrof(A) + a = lltype.malloc(A, 5, flavor='raw') + value = intmask(0xFFEEDDCCBBAA9988) + expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value)) + a[3] = rffi.cast(RESTYPE, value) + a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a)) + x = cpu.bh_getarrayitem_raw_i(descrarray, a_rawint, 3) + assert x == expected, ( + "%r: got %r, expected %r" % (RESTYPE, x, expected)) + lltype.free(a, flavor='raw') + + def test_short_result_of_getarrayitem_raw_compiled(self): + # Test that a getarrayitem that returns a CHAR, SHORT or INT, signed + # or unsigned, properly gets zero-extended or sign-extended. + # Machine code compilation test. + cpu = self.cpu + for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR, + rffi.SHORT, rffi.USHORT, + rffi.INT, rffi.UINT, + rffi.LONG, rffi.ULONG]: + A = rffi.CArray(RESTYPE) + descrarray = cpu.arraydescrof(A) + a = lltype.malloc(A, 5, flavor='raw') + value = intmask(0xFFEEDDCCBBAA9988) + expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value)) + a[3] = rffi.cast(RESTYPE, value) + a_rawint = heaptracker.adr2int(llmemory.cast_ptr_to_adr(a)) + res = self.execute_operation(rop.GETARRAYITEM_RAW, + [BoxInt(a_rawint), BoxInt(3)], + 'int', descr=descrarray) + assert res.value == expected, ( + "%r: got %r, expected %r" % (RESTYPE, res.value, expected)) + lltype.free(a, flavor='raw') + + def test_short_result_of_call_direct(self): + # Test that calling a function that returns a CHAR, SHORT or INT, + # signed or unsigned, properly gets zero-extended or sign-extended. + from pypy.translator.tool.cbuild import ExternalCompilationInfo + for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR, + rffi.SHORT, rffi.USHORT, + rffi.INT, rffi.UINT, + rffi.LONG, rffi.ULONG]: + # Tested with a function that intentionally does not cast the + # result to RESTYPE, but makes sure that we return the whole + # value in eax or rax. + eci = ExternalCompilationInfo( + separate_module_sources=[""" + long fn_test_result_of_call(long x) + { + return x + 1; + } + """], + export_symbols=['fn_test_result_of_call']) + f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed], + RESTYPE, compilation_info=eci, _nowrapper=True) + value = intmask(0xFFEEDDCCBBAA9988) + expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1)) + assert intmask(f(value)) == expected + # + FUNC = self.FuncType([lltype.Signed], RESTYPE) + FPTR = self.Ptr(FUNC) + calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT) + x = self.cpu.bh_call_i(self.get_funcbox(self.cpu, f).value, + calldescr, [value], None, None) + assert x == expected, ( + "%r: got %r, expected %r" % (RESTYPE, x, expected)) + + def test_short_result_of_call_compiled(self): + # Test that calling a function that returns a CHAR, SHORT or INT, + # signed or unsigned, properly gets zero-extended or sign-extended. + from pypy.translator.tool.cbuild import ExternalCompilationInfo + for RESTYPE in [rffi.SIGNEDCHAR, rffi.UCHAR, + rffi.SHORT, rffi.USHORT, + rffi.INT, rffi.UINT, + rffi.LONG, rffi.ULONG]: + # Tested with a function that intentionally does not cast the + # result to RESTYPE, but makes sure that we return the whole + # value in eax or rax. + eci = ExternalCompilationInfo( + separate_module_sources=[""" + long fn_test_result_of_call(long x) + { + return x + 1; + } + """], + export_symbols=['fn_test_result_of_call']) + f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed], + RESTYPE, compilation_info=eci, _nowrapper=True) + value = intmask(0xFFEEDDCCBBAA9988) + expected = rffi.cast(lltype.Signed, rffi.cast(RESTYPE, value + 1)) + assert intmask(f(value)) == expected + # + FUNC = self.FuncType([lltype.Signed], RESTYPE) + FPTR = self.Ptr(FUNC) + calldescr = self.cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT) + funcbox = self.get_funcbox(self.cpu, f) + res = self.execute_operation(rop.CALL, [funcbox, BoxInt(value)], + 'int', descr=calldescr) + assert res.value == expected, ( + "%r: got %r, expected %r" % (RESTYPE, res.value, expected)) + class OOtypeBackendTest(BaseBackendTest): Modified: pypy/branch/arm-backend/pypy/jit/backend/test/test_ll_random.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/test/test_ll_random.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/test/test_ll_random.py Tue Nov 23 10:12:47 2010 @@ -386,6 +386,20 @@ v_string = self.get_string(builder, r) builder.do(self.opnum, [v_string]) +class AbstractCopyContentOperation(AbstractStringOperation): + def produce_into(self, builder, r): + v_srcstring = self.get_string(builder, r) + v_dststring = self.get_string(builder, r) + if v_srcstring.value == v_dststring.value: # because it's not a + raise test_random.CannotProduceOperation # memmove(), but memcpy() + srclen = len(v_srcstring.getref(self.ptr).chars) + dstlen = len(v_dststring.getref(self.ptr).chars) + v_length = builder.get_index(min(srclen, dstlen), r) + v_srcstart = builder.get_index(srclen - v_length.value + 1, r) + v_dststart = builder.get_index(dstlen - v_length.value + 1, r) + builder.do(self.opnum, [v_srcstring, v_dststring, + v_srcstart, v_dststart, v_length]) + class StrGetItemOperation(AbstractGetItemOperation, _StrOperation): pass @@ -404,6 +418,13 @@ class UnicodeLenOperation(AbstractStringLenOperation, _UnicodeOperation): pass +class CopyStrContentOperation(AbstractCopyContentOperation, _StrOperation): + pass + +class CopyUnicodeContentOperation(AbstractCopyContentOperation, + _UnicodeOperation): + pass + # there are five options in total: # 1. non raising call and guard_no_exception @@ -577,6 +598,8 @@ OPERATIONS.append(UnicodeSetItemOperation(rop.UNICODESETITEM)) OPERATIONS.append(StrLenOperation(rop.STRLEN)) OPERATIONS.append(UnicodeLenOperation(rop.UNICODELEN)) + OPERATIONS.append(CopyStrContentOperation(rop.COPYSTRCONTENT)) + OPERATIONS.append(CopyUnicodeContentOperation(rop.COPYUNICODECONTENT)) for i in range(2): OPERATIONS.append(GuardClassOperation(rop.GUARD_CLASS)) Modified: pypy/branch/arm-backend/pypy/jit/backend/test/test_random.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/test/test_random.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/test/test_random.py Tue Nov 23 10:12:47 2010 @@ -1,7 +1,7 @@ import py, sys from pypy.rlib.rarithmetic import intmask, LONG_BIT from pypy.rpython.lltypesystem import llmemory -from pypy.jit.backend.test import conftest as demo_conftest +from pypy.jit.backend import conftest as demo_conftest from pypy.jit.metainterp.history import BasicFailDescr, TreeLoop from pypy.jit.metainterp.history import BoxInt, ConstInt, LoopToken from pypy.jit.metainterp.history import BoxPtr, ConstPtr @@ -102,7 +102,7 @@ elif isinstance(v, ConstFloat): args.append('ConstFloat(%r)' % v.value) elif isinstance(v, ConstInt): - args.append('ConstInt(%d)' % v.value) + args.append('ConstInt(%s)' % v.value) else: raise NotImplementedError(v) if op.getdescr() is None: @@ -113,7 +113,7 @@ except AttributeError: descrstr = ', descr=...' print >>s, ' ResOperation(rop.%s, [%s], %s%s),' % ( - opname[op.opnum], ', '.join(args), names[op.result], descrstr) + opname[op.getopnum()], ', '.join(args), names[op.result], descrstr) #if getattr(op, 'suboperations', None) is not None: # subops.append(op) @@ -189,7 +189,7 @@ v.value) print >>s, ' op = cpu.execute_token(looptoken)' if self.should_fail_by is None: - fail_args = self.loop.operations[-1].args + fail_args = self.loop.operations[-1].getarglist() else: fail_args = self.should_fail_by.getfailargs() for i, v in enumerate(fail_args): Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/x86/assembler.py Tue Nov 23 10:12:47 2010 @@ -8,7 +8,8 @@ from pypy.rpython.annlowlevel import llhelper from pypy.tool.uid import fixid from pypy.jit.backend.x86.regalloc import (RegAlloc, X86RegisterManager, - X86XMMRegisterManager, get_ebp_ofs) + X86XMMRegisterManager, get_ebp_ofs, + _get_scale) from pypy.jit.backend.x86.arch import (FRAME_FIXED_SIZE, FORCE_INDEX_OFS, WORD, IS_X86_32, IS_X86_64) @@ -22,7 +23,8 @@ X86_64_SCRATCH_REG, X86_64_XMM_SCRATCH_REG, RegLoc, StackLoc, ConstFloatLoc, - ImmedLoc, AddressLoc, imm) + ImmedLoc, AddressLoc, imm, + imm0, imm1) from pypy.rlib.objectmodel import we_are_translated, specialize from pypy.jit.backend.x86 import rx86, regloc, codebuf @@ -244,12 +246,13 @@ f = open_file_as_stream(output_log, "w") for i in range(len(self.loop_run_counters)): name, struct = self.loop_run_counters[i] - f.write(str(struct.i) + " " * (8 - len(str(struct.i))) + name + "\n") + f.write(str(name) + ":" + str(struct.i) + "\n") f.close() def _build_float_constants(self): # 44 bytes: 32 bytes for the data, and up to 12 bytes for alignment - addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw') + addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw', + track_allocation=False) if not we_are_translated(): self._keepalive_malloced_float_consts = addr float_constants = rffi.cast(lltype.Signed, addr) @@ -292,7 +295,7 @@ self.mc.RET() self.mc.done() - def assemble_loop(self, inputargs, operations, looptoken): + def assemble_loop(self, inputargs, operations, looptoken, log): """adds the following attributes to looptoken: _x86_loop_code (an integer giving an address) _x86_bootstrap_code (an integer giving an address) @@ -300,6 +303,7 @@ _x86_frame_depth _x86_param_depth _x86_arglocs + _x86_debug_checksum """ if not we_are_translated(): # Arguments should be unique @@ -307,10 +311,11 @@ self.setup() funcname = self._find_debug_merge_point(operations) - + if log: + self._register_counter() + operations = self._inject_debugging_code(looptoken, operations) regalloc = RegAlloc(self, self.cpu.translate_support_code) - operations = self._inject_debugging_code(operations) arglocs = regalloc.prepare_loop(inputargs, operations, looptoken) looptoken._x86_arglocs = arglocs @@ -332,18 +337,22 @@ self._assemble_bootstrap_direct_call(arglocs, curadr, frame_depth+param_depth) # - debug_print("Loop #", looptoken.number, "has address", - looptoken._x86_loop_code, "to", self.mc.tell()) + debug_print("Loop #%d has address %x to %x" % (looptoken.number, + looptoken._x86_loop_code, + self.mc.tell())) self.mc.end_function() self.write_pending_failure_recoveries() - def assemble_bridge(self, faildescr, inputargs, operations): + def assemble_bridge(self, faildescr, inputargs, operations, log): if not we_are_translated(): # Arguments should be unique assert len(set(inputargs)) == len(inputargs) self.setup() funcname = self._find_debug_merge_point(operations) + if log: + self._register_counter() + operations = self._inject_debugging_code(faildescr, operations) arglocs = self.rebuild_faillocs_from_descr( faildescr._x86_failure_recovery_bytecode) @@ -351,7 +360,6 @@ assert ([loc.assembler() for loc in arglocs] == [loc.assembler() for loc in faildescr._x86_debug_faillocs]) regalloc = RegAlloc(self, self.cpu.translate_support_code) - operations = self._inject_debugging_code(operations) fail_depths = faildescr._x86_current_depths regalloc.prepare_bridge(fail_depths, inputargs, arglocs, operations) @@ -371,9 +379,8 @@ faildescr._x86_bridge_param_depth = param_depth # patch the jump from original guard self.patch_jump_for_descr(faildescr, adr_bridge) - debug_print("Bridge out of guard", - descr_number, - "has address", adr_bridge, "to", self.mc.tell()) + debug_print("Bridge out of guard %d has address %x to %x" % + (descr_number, adr_bridge, self.mc.tell())) self.mc.end_function() self.write_pending_failure_recoveries() @@ -398,11 +405,14 @@ else: funcname = "" % len(self.loop_run_counters) # invent the counter, so we don't get too confused + return funcname + + def _register_counter(self): if self._debug: - struct = lltype.malloc(DEBUG_COUNTER, flavor='raw') + struct = lltype.malloc(DEBUG_COUNTER, flavor='raw', + track_allocation=False) # known to leak struct.i = 0 - self.loop_run_counters.append((funcname, struct)) - return funcname + self.loop_run_counters.append((len(self.loop_run_counters), struct)) def patch_jump_for_descr(self, faildescr, adr_new_target): adr_jump_offset = faildescr._x86_adr_jump_offset @@ -423,9 +433,14 @@ mc.done() - def _inject_debugging_code(self, operations): + @specialize.argtype(1) + def _inject_debugging_code(self, looptoken, operations): if self._debug: # before doing anything, let's increase a counter + s = 0 + for op in operations: + s += op.getopnum() + looptoken._x86_debug_checksum = s c_adr = ConstInt(rffi.cast(lltype.Signed, self.loop_run_counters[-1][1])) box = BoxInt() @@ -440,7 +455,7 @@ # self.mc.PUSH(eax) # adr = rffi.cast(lltype.Signed, self.loop_run_counters[-1][1]) # self.mc.MOV(eax, heap(adr)) - # self.mc.ADD(eax, imm(1)) + # self.mc.ADD(eax, imm1) # self.mc.MOV(heap(adr), eax) # self.mc.POP(eax) return operations @@ -702,16 +717,18 @@ dispatch_opnum = guard_opnum else: dispatch_opnum = op.getopnum() - res = genop_guard_list[dispatch_opnum](self, op, guard_op, guard_token, - arglocs, resloc) - faildescr._x86_adr_jump_offset = res + genop_guard_list[dispatch_opnum](self, op, guard_op, guard_token, + arglocs, resloc) + if not we_are_translated(): + # must be added by the genop_guard_list[]() + assert hasattr(faildescr, '_x86_adr_jump_offset') def regalloc_perform_guard(self, guard_op, faillocs, arglocs, resloc, current_depths): self.regalloc_perform_with_guard(None, guard_op, faillocs, arglocs, resloc, current_depths) - def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm(0)): + def load_effective_addr(self, sizereg, baseofs, scale, result, frm=imm0): self.mc.LEA(result, addr_add(frm, sizereg, baseofs, scale)) def _unaryop(asmop): @@ -761,15 +778,15 @@ if isinstance(op.getarg(0), Const): self.mc.CMP(arglocs[1], arglocs[0]) if guard_opnum == rop.GUARD_FALSE: - return self.implement_guard(guard_token, rev_cond) + self.implement_guard(guard_token, rev_cond) else: - return self.implement_guard(guard_token, false_rev_cond) + self.implement_guard(guard_token, false_rev_cond) else: self.mc.CMP(arglocs[0], arglocs[1]) if guard_opnum == rop.GUARD_FALSE: - return self.implement_guard(guard_token, cond) + self.implement_guard(guard_token, cond) else: - return self.implement_guard(guard_token, false_cond) + self.implement_guard(guard_token, false_cond) return genop_cmp_guard def _cmpop_guard_float(cond, false_cond, need_jp): @@ -783,13 +800,14 @@ if guard_opnum == rop.GUARD_FALSE: if need_jp: self.mc.J_il8(rx86.Conditions['P'], 6) - return self.implement_guard(guard_token, cond) + self.implement_guard(guard_token, cond) else: if need_jp: self.mc.J_il8(rx86.Conditions['P'], 2) self.mc.J_il8(rx86.Conditions[cond], 5) - return self.implement_guard(guard_token) - return self.implement_guard(guard_token, false_cond) + self.implement_guard(guard_token) + else: + self.implement_guard(guard_token, false_cond) return genop_cmp_guard_float def _emit_call(self, x, arglocs, start=0, tmp=eax): @@ -951,11 +969,11 @@ self.mc.ensure_bytes_available(16 + guard_token.recovery_stub_size()) if guard_opnum == rop.GUARD_TRUE: self.mc.J_il8(rx86.Conditions['P'], 6) - return self.implement_guard(guard_token, 'E') + self.implement_guard(guard_token, 'E') else: self.mc.J_il8(rx86.Conditions['P'], 2) self.mc.J_il8(rx86.Conditions['E'], 5) - return self.implement_guard(guard_token) + self.implement_guard(guard_token) def genop_float_neg(self, op, arglocs, resloc): # Following what gcc does: res = x ^ 0x8000000000000000 @@ -973,28 +991,28 @@ def genop_guard_int_is_true(self, op, guard_op, guard_token, arglocs, resloc): guard_opnum = guard_op.getopnum() - self.mc.CMP(arglocs[0], imm(0)) + self.mc.CMP(arglocs[0], imm0) if guard_opnum == rop.GUARD_TRUE: - return self.implement_guard(guard_token, 'Z') + self.implement_guard(guard_token, 'Z') else: - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') def genop_int_is_true(self, op, arglocs, resloc): - self.mc.CMP(arglocs[0], imm(0)) + self.mc.CMP(arglocs[0], imm0) rl = resloc.lowest8bits() self.mc.SET_ir(rx86.Conditions['NE'], rl.value) self.mc.MOVZX8(resloc, rl) def genop_guard_int_is_zero(self, op, guard_op, guard_token, arglocs, resloc): guard_opnum = guard_op.getopnum() - self.mc.CMP(arglocs[0], imm(0)) + self.mc.CMP(arglocs[0], imm0) if guard_opnum == rop.GUARD_TRUE: - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') else: - return self.implement_guard(guard_token, 'Z') + self.implement_guard(guard_token, 'Z') def genop_int_is_zero(self, op, arglocs, resloc): - self.mc.CMP(arglocs[0], imm(0)) + self.mc.CMP(arglocs[0], imm0) rl = resloc.lowest8bits() self.mc.SET_ir(rx86.Conditions['E'], rl.value) self.mc.MOVZX8(resloc, rl) @@ -1050,50 +1068,66 @@ assert result_loc is eax self.call(self.malloc_unicode_func_addr, arglocs, eax) - def genop_getfield_gc(self, op, arglocs, resloc): - base_loc, ofs_loc, size_loc = arglocs - assert isinstance(size_loc, ImmedLoc) + # ---------- + + def load_from_mem(self, resloc, source_addr, size_loc, sign_loc): assert isinstance(resloc, RegLoc) size = size_loc.value - - source_addr = AddressLoc(base_loc, ofs_loc) + sign = sign_loc.value if resloc.is_xmm: self.mc.MOVSD(resloc, source_addr) + elif size == WORD: + self.mc.MOV(resloc, source_addr) elif size == 1: - self.mc.MOVZX8(resloc, source_addr) + if sign: + self.mc.MOVSX8(resloc, source_addr) + else: + self.mc.MOVZX8(resloc, source_addr) elif size == 2: - self.mc.MOVZX16(resloc, source_addr) + if sign: + self.mc.MOVSX16(resloc, source_addr) + else: + self.mc.MOVZX16(resloc, source_addr) + elif IS_X86_64 and size == 4: + if sign: + self.mc.MOVSX32(resloc, source_addr) + else: + self.mc.MOV32(resloc, source_addr) # zero-extending + else: + not_implemented("load_from_mem size = %d" % size) + + def save_into_mem(self, dest_addr, value_loc, size_loc): + size = size_loc.value + if isinstance(value_loc, RegLoc) and value_loc.is_xmm: + self.mc.MOVSD(dest_addr, value_loc) + elif size == 1: + self.mc.MOV8(dest_addr, value_loc.lowest8bits()) + elif size == 2: + self.mc.MOV16(dest_addr, value_loc) elif size == 4: - # MOV32 is zero-extending on 64-bit, so this is okay - self.mc.MOV32(resloc, source_addr) + self.mc.MOV32(dest_addr, value_loc) elif IS_X86_64 and size == 8: - self.mc.MOV(resloc, source_addr) + self.mc.MOV(dest_addr, value_loc) else: - raise NotImplementedError("getfield size = %d" % size) + not_implemented("save_into_mem size = %d" % size) + + def genop_getfield_gc(self, op, arglocs, resloc): + base_loc, ofs_loc, size_loc, sign_loc = arglocs + assert isinstance(size_loc, ImmedLoc) + source_addr = AddressLoc(base_loc, ofs_loc) + self.load_from_mem(resloc, source_addr, size_loc, sign_loc) genop_getfield_raw = genop_getfield_gc genop_getfield_raw_pure = genop_getfield_gc genop_getfield_gc_pure = genop_getfield_gc def genop_getarrayitem_gc(self, op, arglocs, resloc): - base_loc, ofs_loc, scale, ofs = arglocs + base_loc, ofs_loc, size_loc, ofs, sign_loc = arglocs assert isinstance(ofs, ImmedLoc) - assert isinstance(scale, ImmedLoc) - src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale.value) - if op.result.type == FLOAT: - self.mc.MOVSD(resloc, src_addr) - else: - if scale.value == 0: - self.mc.MOVZX8(resloc, src_addr) - elif scale.value == 1: - self.mc.MOVZX16(resloc, src_addr) - elif scale.value == 2: - self.mc.MOV32(resloc, src_addr) - elif IS_X86_64 and scale.value == 3: - self.mc.MOV(resloc, src_addr) - else: - print "[asmgen]getarrayitem unsupported size: %d" % scale.value - raise NotImplementedError() + assert isinstance(size_loc, ImmedLoc) + scale = _get_scale(size_loc.value) + src_addr = addr_add(base_loc, ofs_loc, ofs.value, scale) + self.load_from_mem(resloc, src_addr, size_loc, sign_loc) genop_getarrayitem_gc_pure = genop_getarrayitem_gc genop_getarrayitem_raw = genop_getarrayitem_gc @@ -1101,40 +1135,16 @@ def genop_discard_setfield_gc(self, op, arglocs): base_loc, ofs_loc, size_loc, value_loc = arglocs assert isinstance(size_loc, ImmedLoc) - size = size_loc.value dest_addr = AddressLoc(base_loc, ofs_loc) - if isinstance(value_loc, RegLoc) and value_loc.is_xmm: - self.mc.MOVSD(dest_addr, value_loc) - elif IS_X86_64 and size == 8: - self.mc.MOV(dest_addr, value_loc) - elif size == 4: - self.mc.MOV32(dest_addr, value_loc) - elif size == 2: - self.mc.MOV16(dest_addr, value_loc) - elif size == 1: - self.mc.MOV8(dest_addr, value_loc.lowest8bits()) - else: - print "[asmgen]setfield addr size %d" % size - raise NotImplementedError("Addr size %d" % size) + self.save_into_mem(dest_addr, value_loc, size_loc) def genop_discard_setarrayitem_gc(self, op, arglocs): - base_loc, ofs_loc, value_loc, scale_loc, baseofs = arglocs + base_loc, ofs_loc, value_loc, size_loc, baseofs = arglocs assert isinstance(baseofs, ImmedLoc) - assert isinstance(scale_loc, ImmedLoc) - dest_addr = AddressLoc(base_loc, ofs_loc, scale_loc.value, baseofs.value) - if op.getarg(2).type == FLOAT: - self.mc.MOVSD(dest_addr, value_loc) - else: - if IS_X86_64 and scale_loc.value == 3: - self.mc.MOV(dest_addr, value_loc) - elif scale_loc.value == 2: - self.mc.MOV32(dest_addr, value_loc) - elif scale_loc.value == 1: - self.mc.MOV16(dest_addr, value_loc) - elif scale_loc.value == 0: - self.mc.MOV8(dest_addr, value_loc.lowest8bits()) - else: - raise NotImplementedError("scale = %d" % scale_loc.value) + assert isinstance(size_loc, ImmedLoc) + scale = _get_scale(size_loc.value) + dest_addr = AddressLoc(base_loc, ofs_loc, scale, baseofs.value) + self.save_into_mem(dest_addr, value_loc, size_loc) def genop_discard_strsetitem(self, op, arglocs): base_loc, ofs_loc, val_loc = arglocs @@ -1196,13 +1206,13 @@ def genop_guard_guard_true(self, ign_1, guard_op, guard_token, locs, ign_2): loc = locs[0] self.mc.TEST(loc, loc) - return self.implement_guard(guard_token, 'Z') + self.implement_guard(guard_token, 'Z') genop_guard_guard_nonnull = genop_guard_guard_true def genop_guard_guard_no_exception(self, ign_1, guard_op, guard_token, locs, ign_2): - self.mc.CMP(heap(self.cpu.pos_exception()), imm(0)) - return self.implement_guard(guard_token, 'NZ') + self.mc.CMP(heap(self.cpu.pos_exception()), imm0) + self.implement_guard(guard_token, 'NZ') def genop_guard_guard_exception(self, ign_1, guard_op, guard_token, locs, resloc): @@ -1210,22 +1220,21 @@ loc1 = locs[1] self.mc.MOV(loc1, heap(self.cpu.pos_exception())) self.mc.CMP(loc1, loc) - addr = self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') if resloc is not None: self.mc.MOV(resloc, heap(self.cpu.pos_exc_value())) - self.mc.MOV(heap(self.cpu.pos_exception()), imm(0)) - self.mc.MOV(heap(self.cpu.pos_exc_value()), imm(0)) - return addr + self.mc.MOV(heap(self.cpu.pos_exception()), imm0) + self.mc.MOV(heap(self.cpu.pos_exc_value()), imm0) def _gen_guard_overflow(self, guard_op, guard_token): guard_opnum = guard_op.getopnum() if guard_opnum == rop.GUARD_NO_OVERFLOW: - return self.implement_guard(guard_token, 'O') + self.implement_guard(guard_token, 'O') elif guard_opnum == rop.GUARD_OVERFLOW: - return self.implement_guard(guard_token, 'NO') + self.implement_guard(guard_token, 'NO') else: - print "int_xxx_ovf followed by", guard_op.getopname() - raise AssertionError + not_implemented("int_xxx_ovf followed by %s" % + guard_op.getopname()) def genop_guard_int_add_ovf(self, op, guard_op, guard_token, arglocs, result_loc): self.genop_int_add(op, arglocs, result_loc) @@ -1242,7 +1251,7 @@ def genop_guard_guard_false(self, ign_1, guard_op, guard_token, locs, ign_2): loc = locs[0] self.mc.TEST(loc, loc) - return self.implement_guard(guard_token, 'NZ') + self.implement_guard(guard_token, 'NZ') genop_guard_guard_isnull = genop_guard_guard_false def genop_guard_guard_value(self, ign_1, guard_op, guard_token, locs, ign_2): @@ -1251,7 +1260,7 @@ self.mc.UCOMISD(locs[0], locs[1]) else: self.mc.CMP(locs[0], locs[1]) - return self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') def _cmp_guard_class(self, locs): offset = self.cpu.vtable_offset @@ -1283,12 +1292,12 @@ def genop_guard_guard_class(self, ign_1, guard_op, guard_token, locs, ign_2): self.mc.ensure_bytes_available(256) self._cmp_guard_class(locs) - return self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') def genop_guard_guard_nonnull_class(self, ign_1, guard_op, guard_token, locs, ign_2): self.mc.ensure_bytes_available(256) - self.mc.CMP(locs[0], imm(1)) + self.mc.CMP(locs[0], imm1) # Patched below self.mc.J_il8(rx86.Conditions['B'], 0) jb_location = self.mc.get_relative_pos() @@ -1298,7 +1307,7 @@ assert 0 < offset <= 127 self.mc.overwrite(jb_location-1, [chr(offset)]) # - return self.implement_guard(guard_token, 'NE') + self.implement_guard(guard_token, 'NE') def implement_guard_recovery(self, guard_opnum, faildescr, failargs, fail_locs): @@ -1625,37 +1634,48 @@ def implement_guard(self, guard_token, condition=None): self.mc.reserve_bytes(guard_token.recovery_stub_size()) self.pending_guard_tokens.append(guard_token) - # XXX: These jumps are patched later, the self.mc.tell() are just - # dummy values + # These jumps are patched later, the mc.tell() are just + # dummy values. Also, use self.mc._mc to avoid triggering a + # "buffer full" exactly here. + mc = self.mc._mc if condition: - self.mc.J_il(rx86.Conditions[condition], self.mc.tell()) + mc.J_il(rx86.Conditions[condition], mc.tell()) else: - self.mc.JMP_l(self.mc.tell()) - return self.mc.tell() - 4 + mc.JMP_l(mc.tell()) + guard_token.faildescr._x86_adr_jump_offset = mc.tell() - 4 def genop_call(self, op, arglocs, resloc): sizeloc = arglocs[0] assert isinstance(sizeloc, ImmedLoc) size = sizeloc.value + signloc = arglocs[1] if isinstance(op.getarg(0), Const): x = imm(op.getarg(0).getint()) else: - x = arglocs[1] + x = arglocs[2] if x is eax: tmp = ecx else: tmp = eax - self._emit_call(x, arglocs, 2, tmp=tmp) + self._emit_call(x, arglocs, 3, tmp=tmp) + + if IS_X86_32 and isinstance(resloc, StackLoc) and resloc.width == 8: + self.mc.FSTP_b(resloc.value) # float return + elif size == WORD: + assert resloc is eax or resloc is xmm0 # a full word + elif size == 0: + pass # void return + else: + # use the code in load_from_mem to do the zero- or sign-extension + assert resloc is eax + if size == 1: + srcloc = eax.lowest8bits() + else: + srcloc = eax + self.load_from_mem(eax, srcloc, sizeloc, signloc) - if isinstance(resloc, StackLoc) and resloc.width == 8 and IS_X86_32: - self.mc.FSTP_b(resloc.value) - elif size == 1: - self.mc.AND_ri(eax.value, 0xff) - elif size == 2: - self.mc.AND_ri(eax.value, 0xffff) - def genop_guard_call_may_force(self, op, guard_op, guard_token, arglocs, result_loc): faildescr = guard_op.getdescr() @@ -1663,7 +1683,7 @@ self.mc.MOV_bi(FORCE_INDEX_OFS, fail_index) self.genop_call(op, arglocs, result_loc) self.mc.CMP_bi(FORCE_INDEX_OFS, 0) - return self.implement_guard(guard_token, 'L') + self.implement_guard(guard_token, 'L') def genop_guard_call_assembler(self, op, guard_op, guard_token, arglocs, result_loc): @@ -1750,16 +1770,21 @@ assert 0 <= offset <= 127 self.mc.overwrite(jmp_location - 1, [chr(offset)]) self.mc.CMP_bi(FORCE_INDEX_OFS, 0) - return self.implement_guard(guard_token, 'L') + self.implement_guard(guard_token, 'L') def genop_discard_cond_call_gc_wb(self, op, arglocs): - # use 'mc._mc' directly instead of 'mc', to avoid - # bad surprizes if the code buffer is mostly full + # Write code equivalent to write_barrier() in the GC: it checks + # a flag in the object at arglocs[0], and if set, it calls the + # function remember_young_pointer() from the GC. The two arguments + # to the call are in arglocs[:2]. The rest, arglocs[2:], contains + # registers that need to be saved and restored across the call. descr = op.getdescr() if we_are_translated(): cls = self.cpu.gc_ll_descr.has_write_barrier_class() assert cls is not None and isinstance(descr, cls) loc_base = arglocs[0] + # ensure that enough bytes are available to write the whole + # following piece of code atomically (for the JZ) self.mc.ensure_bytes_available(256) self.mc.TEST8_mi((loc_base.value, descr.jit_wb_if_flag_byteofs), descr.jit_wb_if_flag_singlebyte) @@ -1767,35 +1792,39 @@ jz_location = self.mc.get_relative_pos() # the following is supposed to be the slow path, so whenever possible # we choose the most compact encoding over the most efficient one. - # XXX improve a bit, particularly for IS_X86_64. - for i in range(len(arglocs)-1, -1, -1): + if IS_X86_32: + limit = -1 # push all arglocs on the stack + elif IS_X86_64: + limit = 1 # push only arglocs[2:] on the stack + for i in range(len(arglocs)-1, limit, -1): loc = arglocs[i] if isinstance(loc, RegLoc): self.mc.PUSH_r(loc.value) else: - if IS_X86_64: - self.mc.MOV_ri(X86_64_SCRATCH_REG.value, loc.getint()) - self.mc.PUSH_r(X86_64_SCRATCH_REG.value) - else: - self.mc.PUSH_i32(loc.getint()) - + assert not IS_X86_64 # there should only be regs in arglocs[2:] + self.mc.PUSH_i32(loc.getint()) if IS_X86_64: - # We clobber this register to pass the arguments, but that's + # We clobber these registers to pass the arguments, but that's # okay, because consider_cond_call_gc_wb makes sure that any - # caller-save registers with values in them are present in arglocs, - # so they are saved on the stack above and restored below - self.mc.MOV_rs(edi.value, 0) + # caller-save registers with values in them are present in + # arglocs[2:] too, so they are saved on the stack above and + # restored below. + remap_frame_layout(self, arglocs[:2], [edi, esi], + X86_64_SCRATCH_REG) # misaligned stack in the call, but it's ok because the write barrier # is not going to call anything more. Also, this assumes that the - # write barrier does not touch the xmm registers. + # write barrier does not touch the xmm registers. (Slightly delicate + # assumption, given that the write barrier can end up calling the + # platform's malloc() from AddressStack.append(). XXX may need to + # be done properly) self.mc.CALL(imm(descr.get_write_barrier_fn(self.cpu))) - for i in range(len(arglocs)): + if IS_X86_32: + self.mc.ADD_ri(esp.value, 2*WORD) + for i in range(2, len(arglocs)): loc = arglocs[i] - if isinstance(loc, RegLoc): - self.mc.POP_r(loc.value) - else: - self.mc.ADD_ri(esp.value, WORD) # ignore the pushed constant + assert isinstance(loc, RegLoc) + self.mc.POP_r(loc.value) # patch the JZ above offset = self.mc.get_relative_pos() - jz_location assert 0 < offset <= 127 @@ -1807,20 +1836,16 @@ self.mc.LEA_rb(resloc.value, FORCE_INDEX_OFS) def not_implemented_op_discard(self, op, arglocs): - msg = "not implemented operation: %s" % op.getopname() - print msg - raise NotImplementedError(msg) + not_implemented("not implemented operation: %s" % op.getopname()) def not_implemented_op(self, op, arglocs, resloc): - msg = "not implemented operation with res: %s" % op.getopname() - print msg - raise NotImplementedError(msg) + not_implemented("not implemented operation with res: %s" % + op.getopname()) def not_implemented_op_guard(self, op, guard_op, failaddr, arglocs, resloc): - msg = "not implemented operation (guard): %s" % op.getopname() - print msg - raise NotImplementedError(msg) + not_implemented("not implemented operation (guard): %s" % + op.getopname()) def mark_gc_roots(self): gcrootmap = self.cpu.gc_ll_descr.gcrootmap @@ -1836,6 +1861,7 @@ def malloc_cond_fixedsize(self, nursery_free_adr, nursery_top_adr, size, tid): + size = max(size, self.cpu.gc_ll_descr.minimal_size_in_nursery) self.mc.ensure_bytes_available(256) self.mc.MOV(eax, heap(nursery_free_adr)) self.mc.LEA_rm(edx.value, (eax.value, size)) @@ -1904,3 +1930,7 @@ def heap(addr): return AddressLoc(ImmedLoc(addr), ImmedLoc(0), 0, 0) + +def not_implemented(msg): + os.write(2, '[x86/asm] %s\n' % msg) + raise NotImplementedError(msg) Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/x86/regalloc.py Tue Nov 23 10:12:47 2010 @@ -2,6 +2,7 @@ """ Register allocation scheme. """ +import os from pypy.jit.metainterp.history import (Box, Const, ConstInt, ConstPtr, ResOperation, BoxPtr, LoopToken, INT, REF, FLOAT) @@ -40,12 +41,10 @@ return imm(c.value) elif isinstance(c, ConstPtr): if we_are_translated() and c.value and rgc.can_move(c.value): - print "convert_to_imm: ConstPtr needs special care" - raise AssertionError + not_implemented("convert_to_imm: ConstPtr needs special care") return imm(rffi.cast(lltype.Signed, c.value)) else: - print "convert_to_imm: got a %s" % c - raise AssertionError + not_implemented("convert_to_imm: got a %s" % c) class X86_64_RegisterManager(X86RegisterManager): # r11 omitted because it's used as scratch @@ -70,8 +69,9 @@ def _get_new_array(self): n = self.BASE_CONSTANT_SIZE + # known to leak self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n, - flavor='raw') + flavor='raw', track_allocation=False) self.cur_array_free = n _get_new_array._dont_inline_ = True @@ -349,8 +349,8 @@ if op.is_ovf(): if (operations[i + 1].getopnum() != rop.GUARD_NO_OVERFLOW and operations[i + 1].getopnum() != rop.GUARD_OVERFLOW): - print "int_xxx_ovf not followed by guard_(no)_overflow" - raise AssertionError + not_implemented("int_xxx_ovf not followed by " + "guard_(no)_overflow") return True return False if (operations[i + 1].getopnum() != rop.GUARD_TRUE and @@ -624,7 +624,13 @@ assert isinstance(calldescr, BaseCallDescr) assert len(calldescr.arg_classes) == op.numargs() - 1 size = calldescr.get_result_size(self.translate_support_code) - self._call(op, [imm(size)] + [self.loc(op.getarg(i)) for i in range(op.numargs())], + sign = calldescr.is_result_signed() + if sign: + sign_loc = imm1 + else: + sign_loc = imm0 + self._call(op, [imm(size), sign_loc] + + [self.loc(op.getarg(i)) for i in range(op.numargs())], guard_not_forced_op=guard_not_forced_op) def consider_call(self, op): @@ -645,7 +651,7 @@ self.rm._sync_var(op.getarg(vable_index)) vable = self.fm.loc(op.getarg(vable_index)) else: - vable = imm(0) + vable = imm0 self._call(op, [imm(size), vable] + [self.loc(op.getarg(i)) for i in range(op.numargs())], guard_not_forced_op=guard_op) @@ -653,9 +659,13 @@ def consider_cond_call_gc_wb(self, op): assert op.result is None args = op.getarglist() + loc_newvalue = self.rm.make_sure_var_in_reg(op.getarg(1), args) + # ^^^ we force loc_newvalue in a reg (unless it's a Const), + # because it will be needed anyway by the following setfield_gc. + # It avoids loading it twice from the memory. loc_base = self.rm.make_sure_var_in_reg(op.getarg(0), args, imm_fine=False) - arglocs = [loc_base] + arglocs = [loc_base, loc_newvalue] # add eax, ecx and edx as extra "arguments" to ensure they are # saved and restored. Fish in self.rm to know which of these # registers really need to be saved (a bit of a hack). Moreover, @@ -731,15 +741,11 @@ loc = self.loc(op.getarg(0)) return self._call(op, [loc]) # boehm GC (XXX kill the following code at some point) - ofs_items, itemsize, ofs = symbolic.get_array_token(rstr.UNICODE, self.translate_support_code) - if itemsize == 4: - return self._malloc_varsize(ofs_items, ofs, 2, op.getarg(0), - op.result) - elif itemsize == 2: - return self._malloc_varsize(ofs_items, ofs, 1, op.getarg(0), - op.result) - else: - assert False, itemsize + ofs_items, _, ofs = symbolic.get_array_token(rstr.UNICODE, + self.translate_support_code) + scale = self._get_unicode_item_scale() + return self._malloc_varsize(ofs_items, ofs, scale, op.getarg(0), + op.result) def _malloc_varsize(self, ofs_items, ofs_length, scale, v, res_v): # XXX kill this function at some point @@ -771,8 +777,9 @@ arglocs.append(self.loc(op.getarg(0))) return self._call(op, arglocs) # boehm GC (XXX kill the following code at some point) - scale_of_field, basesize, ofs_length, _ = ( + itemsize, basesize, ofs_length, _, _ = ( self._unpack_arraydescr(op.getdescr())) + scale_of_field = _get_scale(itemsize) return self._malloc_varsize(basesize, ofs_length, scale_of_field, op.getarg(0), op.result) @@ -782,21 +789,19 @@ ofs = arraydescr.get_base_size(self.translate_support_code) size = arraydescr.get_item_size(self.translate_support_code) ptr = arraydescr.is_array_of_pointers() - scale = 0 - while (1 << scale) < size: - scale += 1 - assert (1 << scale) == size - return scale, ofs, ofs_length, ptr + sign = arraydescr.is_item_signed() + return size, ofs, ofs_length, ptr, sign def _unpack_fielddescr(self, fielddescr): assert isinstance(fielddescr, BaseFieldDescr) ofs = fielddescr.offset size = fielddescr.get_field_size(self.translate_support_code) ptr = fielddescr.is_pointer_field() - return imm(ofs), imm(size), ptr + sign = fielddescr.is_field_signed() + return imm(ofs), imm(size), ptr, sign def consider_setfield_gc(self, op): - ofs_loc, size_loc, ptr = self._unpack_fielddescr(op.getdescr()) + ofs_loc, size_loc, _, _ = self._unpack_fielddescr(op.getdescr()) assert isinstance(size_loc, ImmedLoc) if size_loc.value == 1: need_lower_byte = True @@ -823,10 +828,10 @@ consider_unicodesetitem = consider_strsetitem def consider_setarrayitem_gc(self, op): - scale, ofs, _, ptr = self._unpack_arraydescr(op.getdescr()) + itemsize, ofs, _, _, _ = self._unpack_arraydescr(op.getdescr()) args = op.getarglist() base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args) - if scale == 0: + if itemsize == 1: need_lower_byte = True else: need_lower_byte = False @@ -835,30 +840,39 @@ ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args) self.possibly_free_vars(args) self.PerformDiscard(op, [base_loc, ofs_loc, value_loc, - imm(scale), imm(ofs)]) + imm(itemsize), imm(ofs)]) consider_setarrayitem_raw = consider_setarrayitem_gc def consider_getfield_gc(self, op): - ofs_loc, size_loc, _ = self._unpack_fielddescr(op.getdescr()) + ofs_loc, size_loc, _, sign = self._unpack_fielddescr(op.getdescr()) args = op.getarglist() base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args) self.rm.possibly_free_vars(args) result_loc = self.force_allocate_reg(op.result) - self.Perform(op, [base_loc, ofs_loc, size_loc], result_loc) + if sign: + sign_loc = imm1 + else: + sign_loc = imm0 + self.Perform(op, [base_loc, ofs_loc, size_loc, sign_loc], result_loc) consider_getfield_raw = consider_getfield_gc consider_getfield_raw_pure = consider_getfield_gc consider_getfield_gc_pure = consider_getfield_gc def consider_getarrayitem_gc(self, op): - scale, ofs, _, _ = self._unpack_arraydescr(op.getdescr()) + itemsize, ofs, _, _, sign = self._unpack_arraydescr(op.getdescr()) args = op.getarglist() base_loc = self.rm.make_sure_var_in_reg(op.getarg(0), args) ofs_loc = self.rm.make_sure_var_in_reg(op.getarg(1), args) self.rm.possibly_free_vars_for_op(op) result_loc = self.force_allocate_reg(op.result) - self.Perform(op, [base_loc, ofs_loc, imm(scale), imm(ofs)], result_loc) + if sign: + sign_loc = imm1 + else: + sign_loc = imm0 + self.Perform(op, [base_loc, ofs_loc, imm(itemsize), imm(ofs), + sign_loc], result_loc) consider_getarrayitem_raw = consider_getarrayitem_gc consider_getarrayitem_gc_pure = consider_getarrayitem_gc @@ -912,41 +926,85 @@ consider_unicodegetitem = consider_strgetitem def consider_copystrcontent(self, op): + self._consider_copystrcontent(op, is_unicode=False) + + def consider_copyunicodecontent(self, op): + self._consider_copystrcontent(op, is_unicode=True) + + def _consider_copystrcontent(self, op, is_unicode): # compute the source address args = op.getarglist() base_loc = self.rm.make_sure_var_in_reg(args[0], args) ofs_loc = self.rm.make_sure_var_in_reg(args[2], args) + assert args[0] is not args[1] # forbidden case of aliasing self.rm.possibly_free_var(args[0]) - self.rm.possibly_free_var(args[2]) + if args[3] is not args[2] is not args[4]: # MESS MESS MESS: don't free + self.rm.possibly_free_var(args[2]) # it if ==args[3] or args[4] srcaddr_box = TempBox() - srcaddr_loc = self.rm.force_allocate_reg(srcaddr_box) - self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc) + forbidden_vars = [args[1], args[3], args[4], srcaddr_box] + srcaddr_loc = self.rm.force_allocate_reg(srcaddr_box, forbidden_vars) + self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc, + is_unicode=is_unicode) # compute the destination address - base_loc = self.rm.make_sure_var_in_reg(args[1], args) - ofs_loc = self.rm.make_sure_var_in_reg(args[3], args) + base_loc = self.rm.make_sure_var_in_reg(args[1], forbidden_vars) + ofs_loc = self.rm.make_sure_var_in_reg(args[3], forbidden_vars) self.rm.possibly_free_var(args[1]) - self.rm.possibly_free_var(args[3]) + if args[3] is not args[4]: # more of the MESS described above + self.rm.possibly_free_var(args[3]) + forbidden_vars = [args[4], srcaddr_box] dstaddr_box = TempBox() - dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box) - self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc) + dstaddr_loc = self.rm.force_allocate_reg(dstaddr_box, forbidden_vars) + self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc, + is_unicode=is_unicode) + # compute the length in bytes + length_box = args[4] + length_loc = self.loc(length_box) + if is_unicode: + self.rm.possibly_free_var(length_box) + forbidden_vars = [srcaddr_box, dstaddr_box] + bytes_box = TempBox() + bytes_loc = self.rm.force_allocate_reg(bytes_box, forbidden_vars) + scale = self._get_unicode_item_scale() + if not (isinstance(length_loc, ImmedLoc) or + isinstance(length_loc, RegLoc)): + self.assembler.mov(length_loc, bytes_loc) + length_loc = bytes_loc + self.assembler.load_effective_addr(length_loc, 0, scale, bytes_loc) + length_box = bytes_box + length_loc = bytes_loc # call memcpy() - length_loc = self.loc(args[4]) self.rm.before_call() self.xrm.before_call() self.assembler._emit_call(imm(self.assembler.memcpy_addr), [dstaddr_loc, srcaddr_loc, length_loc]) - self.rm.possibly_free_var(args[4]) + self.rm.possibly_free_var(length_box) self.rm.possibly_free_var(dstaddr_box) self.rm.possibly_free_var(srcaddr_box) - def _gen_address_inside_string(self, baseloc, ofsloc, resloc): + def _gen_address_inside_string(self, baseloc, ofsloc, resloc, is_unicode): cpu = self.assembler.cpu - ofs_items, itemsize, _ = symbolic.get_array_token(rstr.STR, + if is_unicode: + ofs_items, _, _ = symbolic.get_array_token(rstr.UNICODE, self.translate_support_code) - assert itemsize == 1 - self.assembler.load_effective_addr(ofsloc, ofs_items, 0, + scale = self._get_unicode_item_scale() + else: + ofs_items, itemsize, _ = symbolic.get_array_token(rstr.STR, + self.translate_support_code) + assert itemsize == 1 + scale = 0 + self.assembler.load_effective_addr(ofsloc, ofs_items, scale, resloc, baseloc) + def _get_unicode_item_scale(self): + _, itemsize, _ = symbolic.get_array_token(rstr.UNICODE, + self.translate_support_code) + if itemsize == 4: + return 2 + elif itemsize == 2: + return 1 + else: + raise AssertionError("bad unicode item size") + def consider_jump(self, op): assembler = self.assembler assert self.jump_target_descr is None @@ -981,6 +1039,9 @@ def consider_debug_merge_point(self, op): pass + def consider_jit_debug(self, op): + pass + def get_mark_gc_roots(self, gcrootmap): shape = gcrootmap.get_basic_shape(IS_X86_64) for v, val in self.fm.frame_bindings.items(): @@ -1000,15 +1061,11 @@ self.Perform(op, [], loc) def not_implemented_op(self, op): - msg = "[regalloc] Not implemented operation: %s" % op.getopname() - print msg - raise NotImplementedError(msg) + not_implemented("not implemented operation: %s" % op.getopname()) def not_implemented_op_with_guard(self, op, guard_op): - msg = "[regalloc] Not implemented operation with guard: %s" % ( - op.getopname(),) - print msg - raise NotImplementedError(msg) + not_implemented("not implemented operation with guard: %s" % ( + op.getopname(),)) oplist = [RegAlloc.not_implemented_op] * rop._LAST oplist_with_guard = [RegAlloc.not_implemented_op_with_guard] * rop._LAST @@ -1042,3 +1099,14 @@ # Returns (ebp-20), (ebp-24), (ebp-28)... # i.e. the n'th word beyond the fixed frame size. return -WORD * (FRAME_FIXED_SIZE + position) + +def _get_scale(size): + assert size == 1 or size == 2 or size == 4 or size == 8 + if size < 4: + return size - 1 # 1, 2 => 0, 1 + else: + return (size >> 2) + 1 # 4, 8 => 2, 3 + +def not_implemented(msg): + os.write(2, '[x86/regalloc] %s\n' % msg) + raise NotImplementedError(msg) Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/regloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/x86/regloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/x86/regloc.py Tue Nov 23 10:12:47 2010 @@ -442,8 +442,11 @@ MOV8 = _binaryop('MOV8') MOV16 = _16_bit_binaryop('MOV') MOVZX8 = _binaryop('MOVZX8') + MOVSX8 = _binaryop('MOVSX8') MOVZX16 = _binaryop('MOVZX16') + MOVSX16 = _binaryop('MOVSX16') MOV32 = _binaryop('MOV32') + MOVSX32 = _binaryop('MOVSX32') XCHG = _binaryop('XCHG') PUSH = _unaryop('PUSH') @@ -473,6 +476,9 @@ else: return ImmedLoc(x) +imm0 = imm(0) +imm1 = imm(1) + all_extra_instructions = [name for name in LocationCodeBuilder.__dict__ if name[0].isupper()] all_extra_instructions.sort() Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/x86/runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/x86/runner.py Tue Nov 23 10:12:47 2010 @@ -49,11 +49,13 @@ self.assembler.finish_once() self.profile_agent.shutdown() - def compile_loop(self, inputargs, operations, looptoken): - self.assembler.assemble_loop(inputargs, operations, looptoken) - - def compile_bridge(self, faildescr, inputargs, operations): - self.assembler.assemble_bridge(faildescr, inputargs, operations) + def compile_loop(self, inputargs, operations, looptoken, log=True): + self.assembler.assemble_loop(inputargs, operations, looptoken, + log=log) + + def compile_bridge(self, faildescr, inputargs, operations, log=True): + self.assembler.assemble_bridge(faildescr, inputargs, operations, + log=log) def set_future_value_int(self, index, intvalue): self.assembler.fail_boxes_int.setitem(index, intvalue) @@ -87,7 +89,9 @@ def execute_token(self, executable_token): addr = executable_token._x86_bootstrap_code + #llop.debug_print(lltype.Void, ">>>> Entering", addr) func = rffi.cast(lltype.Ptr(self.BOOTSTRAP_TP), addr) + #llop.debug_print(lltype.Void, "<<<< Back") fail_index = self._execute_call(func) return self.get_fail_descr_from_number(fail_index) @@ -99,10 +103,7 @@ LLInterpreter.current_interpreter = self.debug_ll_interpreter res = 0 try: - #llop.debug_print(lltype.Void, ">>>> Entering", - # rffi.cast(lltype.Signed, func)) res = func() - #llop.debug_print(lltype.Void, "<<<< Back") finally: if not self.translate_support_code: LLInterpreter.current_interpreter = prev_interpreter @@ -114,7 +115,8 @@ return CPU386.cast_adr_to_int(adr) all_null_registers = lltype.malloc(rffi.LONGP.TO, 24, - flavor='raw', zero=True) + flavor='raw', zero=True, + immortal=True) def force(self, addr_of_force_index): TP = rffi.CArrayPtr(lltype.Signed) Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/rx86.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/x86/rx86.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/x86/rx86.py Tue Nov 23 10:12:47 2010 @@ -642,7 +642,10 @@ define_modrm_modes('MOV8_*i', [rex_w, '\xC6', orbyte(0<<3)], [immediate(2, 'b')], regtype='BYTE') define_modrm_modes('MOVZX8_r*', [rex_w, '\x0F\xB6', register(1, 8)], regtype='BYTE') +define_modrm_modes('MOVSX8_r*', [rex_w, '\x0F\xBE', register(1, 8)], regtype='BYTE') define_modrm_modes('MOVZX16_r*', [rex_w, '\x0F\xB7', register(1, 8)]) +define_modrm_modes('MOVSX16_r*', [rex_w, '\x0F\xBF', register(1, 8)]) +define_modrm_modes('MOVSX32_r*', [rex_w, '\x63', register(1, 8)]) define_modrm_modes('MOVSD_x*', ['\xF2', rex_nw, '\x0F\x10', register(1,8)], regtype='XMM') define_modrm_modes('MOVSD_*x', ['\xF2', rex_nw, '\x0F\x11', register(2,8)], regtype='XMM') Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_assembler.py Tue Nov 23 10:12:47 2010 @@ -81,7 +81,8 @@ # also test rebuild_faillocs_from_descr(), which should not # reproduce the holes at all - bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw') + bytecode = lltype.malloc(rffi.UCHARP.TO, len(mc.content), flavor='raw', + immortal=True) for i in range(len(mc.content)): assert 0 <= mc.content[i] <= 255 bytecode[i] = rffi.cast(rffi.UCHAR, mc.content[i]) @@ -115,7 +116,8 @@ assert withfloats value = random.random() - 0.5 # make sure it fits into 64 bits - tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw') + tmp = lltype.malloc(rffi.LONGP.TO, 2, flavor='raw', + track_allocation=False) rffi.cast(rffi.DOUBLEP, tmp)[0] = value return rffi.cast(rffi.DOUBLEP, tmp)[0], tmp[0], tmp[1] @@ -152,10 +154,12 @@ # prepare the expected target arrays, the descr_bytecode, # the 'registers' and the 'stack' arrays according to 'content' - xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1, flavor='raw') + xmmregisters = lltype.malloc(rffi.LONGP.TO, 16+ACTUAL_CPU.NUM_REGS+1, + flavor='raw', immortal=True) registers = rffi.ptradd(xmmregisters, 16) stacklen = baseloc + 10 - stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw') + stack = lltype.malloc(rffi.LONGP.TO, stacklen, flavor='raw', + immortal=True) expected_ints = [0] * len(content) expected_ptrs = [lltype.nullptr(llmemory.GCREF.TO)] * len(content) expected_floats = [0.0] * len(content) @@ -221,7 +225,7 @@ descr_bytecode.append(0x00) descr_bytecode.append(0xCC) # end marker descr_bytes = lltype.malloc(rffi.UCHARP.TO, len(descr_bytecode), - flavor='raw') + flavor='raw', immortal=True) for i in range(len(descr_bytecode)): assert 0 <= descr_bytecode[i] <= 255 descr_bytes[i] = rffi.cast(rffi.UCHAR, descr_bytecode[i]) Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_gc_integration.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_gc_integration.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_gc_integration.py Tue Nov 23 10:12:47 2010 @@ -12,7 +12,7 @@ from pypy.jit.backend.detect_cpu import getcpuclass from pypy.jit.backend.x86.regalloc import RegAlloc from pypy.jit.backend.x86.arch import WORD, FRAME_FIXED_SIZE -from pypy.jit.metainterp.test.oparser import parse +from pypy.jit.tool.oparser import parse from pypy.rpython.lltypesystem import lltype, llmemory, rffi from pypy.rpython.annlowlevel import llhelper from pypy.rpython.lltypesystem import rclass, rstr Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_regalloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_regalloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_regalloc.py Tue Nov 23 10:12:47 2010 @@ -11,7 +11,7 @@ from pypy.jit.backend.x86.regalloc import RegAlloc, X86RegisterManager,\ FloatConstants, is_comparison_or_ovf_op from pypy.jit.backend.x86.arch import IS_X86_32, IS_X86_64 -from pypy.jit.metainterp.test.oparser import parse +from pypy.jit.tool.oparser import parse from pypy.rpython.lltypesystem import lltype, llmemory, rffi from pypy.rpython.annlowlevel import llhelper from pypy.rpython.lltypesystem import rclass, rstr Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_runner.py Tue Nov 23 10:12:47 2010 @@ -10,7 +10,7 @@ from pypy.jit.metainterp.resoperation import rop from pypy.jit.metainterp.executor import execute from pypy.jit.backend.test.runner_test import LLtypeBackendTest -from pypy.jit.metainterp.test.oparser import parse +from pypy.jit.tool.oparser import parse from pypy.tool.udir import udir import ctypes import sys @@ -346,7 +346,7 @@ return self.val operations = [ - ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("hello")], None), + ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("hello"), 0], None), ResOperation(rop.INT_ADD, [i0, ConstInt(1)], i1), ResOperation(rop.INT_LE, [i1, ConstInt(9)], i2), ResOperation(rop.GUARD_TRUE, [i2], None, descr=faildescr1), @@ -365,7 +365,7 @@ bridge = [ ResOperation(rop.INT_LE, [i1b, ConstInt(19)], i3), ResOperation(rop.GUARD_TRUE, [i3], None, descr=faildescr2), - ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("bye")], None), + ResOperation(rop.DEBUG_MERGE_POINT, [FakeString("bye"), 0], None), ResOperation(rop.JUMP, [i1b], None, descr=looptoken), ] bridge[1].setfailargs([i1b]) @@ -478,6 +478,10 @@ # whether the test segfaults. assert self.cpu.get_latest_value_int(0) == finished.value + def test_overflow_guard_exception(self): + for i in range(50): + self.test_exceptions() + class TestDebuggingAssembler(object): def setup_method(self, meth): @@ -493,7 +497,7 @@ def test_debugger_on(self): loop = """ [i0] - debug_merge_point('xyz') + debug_merge_point('xyz', 0) i1 = int_add(i0, 1) i2 = int_ge(i1, 10) guard_false(i2) [] @@ -506,8 +510,25 @@ self.cpu.execute_token(ops.token) # check debugging info name, struct = self.cpu.assembler.loop_run_counters[0] - assert name == 'xyz' + assert name == 0 # 'xyz' assert struct.i == 10 self.cpu.finish_once() lines = py.path.local(self.logfile + ".count").readlines() - assert lines[0] == '10 xyz\n' + assert lines[0] == '0:10\n' # '10 xyz\n' + + def test_debugger_checksum(self): + loop = """ + [i0] + debug_merge_point('xyz', 0) + i1 = int_add(i0, 1) + i2 = int_ge(i1, 10) + guard_false(i2) [] + jump(i1) + """ + ops = parse(loop) + self.cpu.assembler.set_debug(True) + self.cpu.compile_loop(ops.inputargs, ops.operations, ops.token) + self.cpu.set_future_value_int(0, 0) + self.cpu.execute_token(ops.token) + assert ops.token._x86_debug_checksum == sum([op.getopnum() + for op in ops.operations]) Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_string.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_string.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_string.py Tue Nov 23 10:12:47 2010 @@ -2,8 +2,12 @@ from pypy.jit.metainterp.test import test_string from pypy.jit.backend.x86.test.test_basic import Jit386Mixin -class TestString(Jit386Mixin, test_string.StringTests): +class TestString(Jit386Mixin, test_string.TestLLtype): # for the individual tests see # ====> ../../../metainterp/test/test_string.py - CALL = 'call' - CALL_PURE = 'call_pure' + pass + +class TestUnicode(Jit386Mixin, test_string.TestLLtypeUnicode): + # for the individual tests see + # ====> ../../../metainterp/test/test_string.py + pass Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_zrpy_gc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_zrpy_gc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_zrpy_gc.py Tue Nov 23 10:12:47 2010 @@ -191,6 +191,33 @@ def run_orig(self, name, n, x): self.main_allfuncs(name, n, x) + def define_libffi_workaround(cls): + # XXX: this is a workaround for a bug in database.py. It seems that + # the problem is triggered by optimizeopt/fficall.py, and in + # particular by the ``cast_base_ptr_to_instance(Func, llfunc)``: in + # these tests, that line is the only place where libffi.Func is + # referenced. + # + # The problem occurs because the gctransformer tries to annotate a + # low-level helper to call the __del__ of libffi.Func when it's too + # late. + # + # This workaround works by forcing the annotator (and all the rest of + # the toolchain) to see libffi.Func in a "proper" context, not just as + # the target of cast_base_ptr_to_instance. Note that the function + # below is *never* called by any actual test, it's just annotated. + # + from pypy.rlib.libffi import get_libc_name, CDLL, types, ArgChain + libc_name = get_libc_name() + def f(n, x, *args): + libc = CDLL(libc_name) + ptr = libc.getpointer('labs', [types.slong], types.slong) + chain = ArgChain() + chain.arg(n) + n = ptr.call(chain, lltype.Signed) + return (n, x) + args + return None, f, None + def define_compile_framework_1(cls): # a moving GC. Supports malloc_varsize_nonmovable. Simple test, works # without write_barriers and root stack enumeration. @@ -523,3 +550,29 @@ def test_compile_framework_float(self): self.run('compile_framework_float') + + def define_compile_framework_minimal_size_in_nursery(self): + S = lltype.GcStruct('S') # no fields! + T = lltype.GcStruct('T', ('i', lltype.Signed)) + @unroll_safe + def f42(n, x, x0, x1, x2, x3, x4, x5, x6, x7, l, s): + lst1 = [] + lst2 = [] + i = 0 + while i < 42: + s1 = lltype.malloc(S) + t1 = lltype.malloc(T) + t1.i = 10000 + i + n + lst1.append(s1) + lst2.append(t1) + i += 1 + i = 0 + while i < 42: + check(lst2[i].i == 10000 + i + n) + i += 1 + n -= 1 + return n, x, x0, x1, x2, x3, x4, x5, x6, x7, l, s + return None, f42, None + + def test_compile_framework_minimal_size_in_nursery(self): + self.run('compile_framework_minimal_size_in_nursery') Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_ztranslation.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_ztranslation.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/x86/test/test_ztranslation.py Tue Nov 23 10:12:47 2010 @@ -1,7 +1,8 @@ -import py, os +import py, os, sys from pypy.tool.udir import udir from pypy.rlib.jit import JitDriver, OPTIMIZER_FULL, unroll_parameters from pypy.rlib.jit import PARAMETERS, dont_look_inside +from pypy.rlib.jit import hint from pypy.jit.metainterp.jitprof import Profiler from pypy.jit.backend.detect_cpu import getcpuclass from pypy.jit.backend.test.support import CCompiledMixin @@ -9,6 +10,7 @@ from pypy.translator.translator import TranslationContext from pypy.jit.backend.x86.arch import IS_X86_32, IS_X86_64 from pypy.config.translationoption import DEFL_GC +from pypy.rlib import rgc class TestTranslationX86(CCompiledMixin): CPUClass = getcpuclass() @@ -63,8 +65,32 @@ if k - abs(j): raise ValueError if k - abs(-j): raise ValueError return total * 10 - res = self.meta_interp(f, [40, -49]) - assert res == f(40, -49) + # + from pypy.rpython.lltypesystem import lltype, rffi + from pypy.rlib.libffi import types, CDLL, ArgChain + from pypy.rlib.test.test_libffi import get_libm_name + libm_name = get_libm_name(sys.platform) + jitdriver2 = JitDriver(greens=[], reds = ['i', 'func', 'res', 'x']) + def libffi_stuff(i, j): + lib = CDLL(libm_name) + func = lib.getpointer('fabs', [types.double], types.double) + res = 0.0 + x = float(j) + while i > 0: + jitdriver2.jit_merge_point(i=i, res=res, func=func, x=x) + jitdriver2.can_enter_jit(i=i, res=res, func=func, x=x) + func = hint(func, promote=True) + argchain = ArgChain() + argchain.arg(x) + res = func.call(argchain, rffi.DOUBLE) + i -= 1 + return int(res) + # + def main(i, j): + return f(i, j) + libffi_stuff(i, j) + expected = main(40, -49) + res = self.meta_interp(main, [40, -49]) + assert res == expected def test_direct_assembler_call_translates(self): class Thing(object): Modified: pypy/branch/arm-backend/pypy/jit/backend/x86/tool/viewcode.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/x86/tool/viewcode.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/x86/tool/viewcode.py Tue Nov 23 10:12:47 2010 @@ -37,7 +37,7 @@ 'x86_64': 'x86-64', 'i386': 'i386', } - objdump = ('objdump -M intel,%(backend)s -b binary -m i386 ' + objdump = ('objdump -M %(backend)s -b binary -m i386 ' '--adjust-vma=%(origin)d -D %(file)s') # f = open(tmpfile, 'wb') Modified: pypy/branch/arm-backend/pypy/jit/codewriter/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/codewriter/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/codewriter/assembler.py Tue Nov 23 10:12:47 2010 @@ -233,10 +233,9 @@ addr = llmemory.cast_ptr_to_adr(value) self.list_of_addr2name.append((addr, name)) - def finished(self): + def finished(self, callinfocollection): # Helper called at the end of assembling. Registers the extra # functions shown in _callinfo_for_oopspec. - from pypy.jit.codewriter.effectinfo import _callinfo_for_oopspec - for _, func in _callinfo_for_oopspec.values(): + for func in callinfocollection.all_function_addresses_as_int(): func = heaptracker.int2adr(func) self.see_raw_object(func.ptr) Modified: pypy/branch/arm-backend/pypy/jit/codewriter/call.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/codewriter/call.py (original) +++ pypy/branch/arm-backend/pypy/jit/codewriter/call.py Tue Nov 23 10:12:47 2010 @@ -7,7 +7,7 @@ from pypy.jit.codewriter.jitcode import JitCode from pypy.jit.codewriter.effectinfo import VirtualizableAnalyzer from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze -from pypy.jit.codewriter.effectinfo import EffectInfo +from pypy.jit.codewriter.effectinfo import EffectInfo, CallInfoCollection from pypy.translator.simplify import get_funcobj, get_functype from pypy.rpython.lltypesystem import lltype, llmemory from pypy.translator.backendopt.canraise import RaiseAnalyzer @@ -23,6 +23,7 @@ self.jitdrivers_sd = jitdrivers_sd self.jitcodes = {} # map {graph: jitcode} self.unfinished_graphs = [] # list of graphs with pending jitcodes + self.callinfocollection = CallInfoCollection() if hasattr(cpu, 'rtyper'): # for tests self.rtyper = cpu.rtyper translator = self.rtyper.annotator.translator @@ -237,6 +238,8 @@ effectinfo) def _canraise(self, op): + if op.opname == 'pseudo_call_cannot_raise': + return False try: return self.raise_analyzer.can_raise(op) except lltype.DelayedPointer: @@ -277,3 +280,11 @@ return seen.pop() else: return None + + def could_be_green_field(self, GTYPE, fieldname): + GTYPE_fieldname = (GTYPE, fieldname) + for jd in self.jitdrivers_sd: + if jd.greenfield_info is not None: + if GTYPE_fieldname in jd.greenfield_info.green_fields: + return True + return False Modified: pypy/branch/arm-backend/pypy/jit/codewriter/codewriter.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/codewriter/codewriter.py (original) +++ pypy/branch/arm-backend/pypy/jit/codewriter/codewriter.py Tue Nov 23 10:12:47 2010 @@ -73,7 +73,7 @@ count += 1 if not count % 500: log.info("Produced %d jitcodes" % count) - self.assembler.finished() + self.assembler.finished(self.callcontrol.callinfocollection) heaptracker.finish_registering(self.cpu) log.info("there are %d JitCode instances." % count) @@ -95,18 +95,18 @@ print '%s:' % (ssarepr.name,) print format_assembler(ssarepr) else: - dir = udir.ensure("jitcodes", dir=1) - if portal_jitdriver: - name = "%02d_portal_runner" % (portal_jitdriver.index,) - elif ssarepr.name and ssarepr.name != '?': - name = ssarepr.name - else: - name = 'unnamed' % id(ssarepr) - i = 1 - extra = '' - while name+extra in self._seen_files: - i += 1 - extra = '.%d' % i - self._seen_files.add(name+extra) - dir.join(name+extra).write(format_assembler(ssarepr)) log.dot() + dir = udir.ensure("jitcodes", dir=1) + if portal_jitdriver: + name = "%02d_portal_runner" % (portal_jitdriver.index,) + elif ssarepr.name and ssarepr.name != '?': + name = ssarepr.name + else: + name = 'unnamed' % id(ssarepr) + i = 1 + extra = '' + while name+extra in self._seen_files: + i += 1 + extra = '.%d' % i + self._seen_files.add(name+extra) + dir.join(name+extra).write(format_assembler(ssarepr)) Modified: pypy/branch/arm-backend/pypy/jit/codewriter/effectinfo.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/codewriter/effectinfo.py (original) +++ pypy/branch/arm-backend/pypy/jit/codewriter/effectinfo.py Tue Nov 23 10:12:47 2010 @@ -18,19 +18,34 @@ # the 'oopspecindex' field is one of the following values: OS_NONE = 0 # normal case, no oopspec OS_ARRAYCOPY = 1 # "list.ll_arraycopy" - OS_STR_CONCAT = 2 # "stroruni.concat" - OS_UNI_CONCAT = 3 # "stroruni.concat" - OS_STR_SLICE = 4 # "stroruni.slice" - OS_UNI_SLICE = 5 # "stroruni.slice" - OS_STR_EQUAL = 6 # "stroruni.equal" - OS_UNI_EQUAL = 7 # "stroruni.equal" - OS_STREQ_SLICE_CHECKNULL = 8 # s2!=NULL and s1[x:x+length]==s2 - OS_STREQ_SLICE_NONNULL = 9 # s1[x:x+length]==s2 (assert s2!=NULL) - OS_STREQ_SLICE_CHAR = 10 # s1[x:x+length]==char - OS_STREQ_NONNULL = 11 # s1 == s2 (assert s1!=NULL,s2!=NULL) - OS_STREQ_NONNULL_CHAR = 12 # s1 == char (assert s1!=NULL) - OS_STREQ_CHECKNULL_CHAR = 13 # s1!=NULL and s1==char - OS_STREQ_LENGTHOK = 14 # s1 == s2 (assert len(s1)==len(s2)) + OS_STR2UNICODE = 2 # "str.str2unicode" + # + OS_STR_CONCAT = 22 # "stroruni.concat" + OS_STR_SLICE = 23 # "stroruni.slice" + OS_STR_EQUAL = 24 # "stroruni.equal" + OS_STREQ_SLICE_CHECKNULL = 25 # s2!=NULL and s1[x:x+length]==s2 + OS_STREQ_SLICE_NONNULL = 26 # s1[x:x+length]==s2 (assert s2!=NULL) + OS_STREQ_SLICE_CHAR = 27 # s1[x:x+length]==char + OS_STREQ_NONNULL = 28 # s1 == s2 (assert s1!=NULL,s2!=NULL) + OS_STREQ_NONNULL_CHAR = 29 # s1 == char (assert s1!=NULL) + OS_STREQ_CHECKNULL_CHAR = 30 # s1!=NULL and s1==char + OS_STREQ_LENGTHOK = 31 # s1 == s2 (assert len(s1)==len(s2)) + # + OS_UNI_CONCAT = 42 # + OS_UNI_SLICE = 43 # + OS_UNI_EQUAL = 44 # + OS_UNIEQ_SLICE_CHECKNULL = 45 # + OS_UNIEQ_SLICE_NONNULL = 46 # + OS_UNIEQ_SLICE_CHAR = 47 # + OS_UNIEQ_NONNULL = 48 # the same for unicode + OS_UNIEQ_NONNULL_CHAR = 49 # (must be the same amount as for + OS_UNIEQ_CHECKNULL_CHAR = 50 # STR, in the same order) + OS_UNIEQ_LENGTHOK = 51 # + _OS_offset_uni = OS_UNI_CONCAT - OS_STR_CONCAT + # + OS_LIBFFI_PREPARE = 60 + OS_LIBFFI_PUSH_ARG = 61 + OS_LIBFFI_CALL = 62 def __new__(cls, readonly_descrs_fields, write_descrs_fields, write_descrs_arrays, @@ -129,30 +144,44 @@ # ____________________________________________________________ -_callinfo_for_oopspec = {} # {oopspecindex: (calldescr, func_as_int)} +class CallInfoCollection(object): + def __init__(self): + # {oopspecindex: (calldescr, func_as_int)} + self._callinfo_for_oopspec = {} -def callinfo_for_oopspec(oopspecindex): - """A function that returns the calldescr and the function - address (as an int) of one of the OS_XYZ functions defined above. - Don't use this if there might be several implementations of the same - OS_XYZ specialized by type, e.g. OS_ARRAYCOPY.""" - try: - return _callinfo_for_oopspec[oopspecindex] - except KeyError: - return (None, 0) - - -def _funcptr_for_oopspec_memo(oopspecindex): - from pypy.jit.codewriter import heaptracker - _, func_as_int = callinfo_for_oopspec(oopspecindex) - funcadr = heaptracker.int2adr(func_as_int) - return funcadr.ptr -_funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo' - -def funcptr_for_oopspec(oopspecindex): - """A memo function that returns a pointer to the function described - by OS_XYZ (as a real low-level function pointer).""" - funcptr = _funcptr_for_oopspec_memo(oopspecindex) - assert funcptr - return funcptr -funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(0)' + def _freeze_(self): + return True + + def add(self, oopspecindex, calldescr, func_as_int): + self._callinfo_for_oopspec[oopspecindex] = calldescr, func_as_int + + def has_oopspec(self, oopspecindex): + return oopspecindex in self._callinfo_for_oopspec + + def all_function_addresses_as_int(self): + return [func for (_, func) in self._callinfo_for_oopspec.values()] + + def callinfo_for_oopspec(self, oopspecindex): + """A function that returns the calldescr and the function + address (as an int) of one of the OS_XYZ functions defined above. + Don't use this if there might be several implementations of the same + OS_XYZ specialized by type, e.g. OS_ARRAYCOPY.""" + try: + return self._callinfo_for_oopspec[oopspecindex] + except KeyError: + return (None, 0) + + def _funcptr_for_oopspec_memo(self, oopspecindex): + from pypy.jit.codewriter import heaptracker + _, func_as_int = self.callinfo_for_oopspec(oopspecindex) + funcadr = heaptracker.int2adr(func_as_int) + return funcadr.ptr + _funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo' + + def funcptr_for_oopspec(self, oopspecindex): + """A memo function that returns a pointer to the function described + by OS_XYZ (as a real low-level function pointer).""" + funcptr = self._funcptr_for_oopspec_memo(oopspecindex) + assert funcptr + return funcptr + funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(1)' Modified: pypy/branch/arm-backend/pypy/jit/codewriter/format.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/codewriter/format.py (original) +++ pypy/branch/arm-backend/pypy/jit/codewriter/format.py Tue Nov 23 10:12:47 2010 @@ -80,7 +80,8 @@ def assert_format(ssarepr, expected): asm = format_assembler(ssarepr) - expected = str(py.code.Source(expected)).strip() + '\n' + if expected != '': + expected = str(py.code.Source(expected)).strip() + '\n' asmlines = asm.split("\n") explines = expected.split("\n") for asm, exp in zip(asmlines, explines): Modified: pypy/branch/arm-backend/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/codewriter/jtransform.py (original) +++ pypy/branch/arm-backend/pypy/jit/codewriter/jtransform.py Tue Nov 23 10:12:47 2010 @@ -6,7 +6,7 @@ from pypy.objspace.flow.model import Block, Link, c_last_exception from pypy.jit.codewriter.flatten import ListOfKind, IndirectCallTargets from pypy.jit.codewriter import support, heaptracker -from pypy.jit.codewriter.effectinfo import EffectInfo, _callinfo_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.jit.codewriter.policy import log from pypy.jit.metainterp.typesystem import deref, arrayItem from pypy.rlib import objectmodel @@ -172,6 +172,7 @@ def rewrite_op_same_as(self, op): pass def rewrite_op_cast_pointer(self, op): pass + def rewrite_op_cast_opaque_ptr(self, op): pass # rlib.rerased def rewrite_op_cast_primitive(self, op): pass def rewrite_op_cast_bool_to_int(self, op): pass def rewrite_op_cast_bool_to_uint(self, op): pass @@ -316,8 +317,14 @@ prepare = self._handle_list_call elif oopspec_name.startswith('stroruni.'): prepare = self._handle_stroruni_call + elif oopspec_name == 'str.str2unicode': + prepare = self._handle_str2unicode_call elif oopspec_name.startswith('virtual_ref'): prepare = self._handle_virtual_ref_call + elif oopspec_name.startswith('jit.'): + prepare = self._handle_jit_call + elif oopspec_name.startswith('libffi_'): + prepare = self._handle_libffi_call else: prepare = self.prepare_builtin_call try: @@ -427,7 +434,8 @@ op.result) def rewrite_op_free(self, op): - assert op.args[1].value == 'raw' + flags = op.args[1].value + assert flags['flavor'] == 'raw' ARRAY = op.args[0].concretetype.TO return self._do_builtin_call(op, 'raw_free', [op.args[0]], extra = (ARRAY,), extrakey = ARRAY) @@ -519,7 +527,12 @@ # check for deepfrozen structures that force constant-folding immut = v_inst.concretetype.TO._immutable_field(c_fieldname.value) if immut: - pure = '_pure' + if (self.callcontrol is not None and + self.callcontrol.could_be_green_field(v_inst.concretetype.TO, + c_fieldname.value)): + pure = '_greenfield' + else: + pure = '_pure' if immut == "[*]": self.immutable_arrays[op.result] = True else: @@ -819,6 +832,8 @@ def rewrite_op_jit_marker(self, op): key = op.args[0].value jitdriver = op.args[1].value + if not jitdriver.active: + return [] return getattr(self, 'handle_jit_marker__%s' % key)(op, jitdriver) def handle_jit_marker__jit_merge_point(self, op, jitdriver): @@ -828,9 +843,16 @@ "general mix-up of jitdrivers?") ops = self.promote_greens(op.args[2:], jitdriver) num_green_args = len(jitdriver.greens) + redlists = self.make_three_lists(op.args[2+num_green_args:]) + for redlist in redlists: + for v in redlist: + assert isinstance(v, Variable), ( + "Constant specified red in jit_merge_point()") + assert len(dict.fromkeys(redlist)) == len(list(redlist)), ( + "duplicate red variable on jit_merge_point()") args = ([Constant(self.portal_jd.index, lltype.Signed)] + self.make_three_lists(op.args[2:2+num_green_args]) + - self.make_three_lists(op.args[2+num_green_args:])) + redlists) op1 = SpaceOperation('jit_merge_point', args, None) op2 = SpaceOperation('-live-', [], None) # ^^^ we need a -live- for the case of do_recursive_call() @@ -852,6 +874,17 @@ (self.graph,)) return [] + def _handle_jit_call(self, op, oopspec_name, args): + if oopspec_name == 'jit.debug': + return SpaceOperation('jit_debug', args, None) + elif oopspec_name == 'jit.assert_green': + kind = getkind(args[0].concretetype) + return SpaceOperation('%s_assert_green' % kind, args, None) + elif oopspec_name == 'jit.current_trace_length': + return SpaceOperation('current_trace_length', [], op.result) + else: + raise AssertionError("missing support for %r" % oopspec_name) + # ---------- # Lists. @@ -870,17 +903,21 @@ prefix = 'do_resizable_' ARRAY = LIST.items.TO if self._array_of_voids(ARRAY): - raise NotSupported("resizable lists of voids") - descrs = (self.cpu.arraydescrof(ARRAY), - self.cpu.fielddescrof(LIST, 'length'), - self.cpu.fielddescrof(LIST, 'items'), - self.cpu.sizeof(LIST)) + prefix += 'void_' + descrs = () + else: + descrs = (self.cpu.arraydescrof(ARRAY), + self.cpu.fielddescrof(LIST, 'length'), + self.cpu.fielddescrof(LIST, 'items'), + self.cpu.sizeof(LIST)) else: prefix = 'do_fixed_' if self._array_of_voids(LIST): - raise NotSupported("fixed lists of voids") - arraydescr = self.cpu.arraydescrof(LIST) - descrs = (arraydescr,) + prefix += 'void_' + descrs = () + else: + arraydescr = self.cpu.arraydescrof(LIST) + descrs = (arraydescr,) # try: meth = getattr(self, prefix + oopspec_name.replace('.', '_')) @@ -919,6 +956,11 @@ descr, args[1]], v_posindex) return v_posindex, [op0, op1] + def _prepare_void_list_getset(self, op): + non_negative, can_raise = self._get_list_nonneg_canraise_flags(op) + if can_raise: + raise NotSupported("list operation can raise") + def _get_initial_newlist_length(self, op, args): # normalize number of arguments to the 'newlist' function if len(args) > 1: @@ -990,6 +1032,12 @@ def do_fixed_list_ll_arraycopy(self, op, args, arraydescr): return self._handle_oopspec_call(op, args, EffectInfo.OS_ARRAYCOPY) + def do_fixed_void_list_getitem(self, op, args): + self._prepare_void_list_getset(op) + return [] + do_fixed_void_list_getitem_foldable = do_fixed_void_list_getitem + do_fixed_void_list_setitem = do_fixed_void_list_getitem + # ---------- resizable lists ---------- def do_resizable_newlist(self, op, args, arraydescr, lengthdescr, @@ -1025,17 +1073,26 @@ return SpaceOperation('getfield_gc_i', [args[0], lengthdescr], op.result) + def do_resizable_void_list_getitem(self, op, args): + self._prepare_void_list_getset(op) + return [] + do_resizable_void_list_getitem_foldable = do_resizable_void_list_getitem + do_resizable_void_list_setitem = do_resizable_void_list_getitem + # ---------- # Strings and Unicodes. - def _handle_oopspec_call(self, op, args, oopspecindex): + def _handle_oopspec_call(self, op, args, oopspecindex, extraeffect=None): calldescr = self.callcontrol.getcalldescr(op, oopspecindex) + if extraeffect: + calldescr.get_extra_info().extraeffect = extraeffect if isinstance(op.args[0].value, str): pass # for tests only else: func = heaptracker.adr2int( llmemory.cast_ptr_to_adr(op.args[0].value)) - _callinfo_for_oopspec[oopspecindex] = calldescr, func + self.callcontrol.callinfocollection.add(oopspecindex, + calldescr, func) op1 = self.rewrite_call(op, 'residual_call', [op.args[0], calldescr], args=args) @@ -1046,37 +1103,41 @@ def _register_extra_helper(self, oopspecindex, oopspec_name, argtypes, resulttype): # a bit hackish - if oopspecindex in _callinfo_for_oopspec: + if self.callcontrol.callinfocollection.has_oopspec(oopspecindex): return c_func, TP = support.builtin_func_for_spec(self.cpu.rtyper, oopspec_name, argtypes, resulttype) - op = SpaceOperation('pseudo_call', + op = SpaceOperation('pseudo_call_cannot_raise', [c_func] + [varoftype(T) for T in argtypes], varoftype(resulttype)) calldescr = self.callcontrol.getcalldescr(op, oopspecindex) - func = heaptracker.adr2int( - llmemory.cast_ptr_to_adr(c_func.value)) - _callinfo_for_oopspec[oopspecindex] = calldescr, func + if isinstance(c_func.value, str): # in tests only + func = c_func.value + else: + func = heaptracker.adr2int( + llmemory.cast_ptr_to_adr(c_func.value)) + self.callcontrol.callinfocollection.add(oopspecindex, calldescr, func) def _handle_stroruni_call(self, op, oopspec_name, args): - if args[0].concretetype.TO == rstr.STR: + SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE) + if SoU.TO == rstr.STR: dict = {"stroruni.concat": EffectInfo.OS_STR_CONCAT, "stroruni.slice": EffectInfo.OS_STR_SLICE, "stroruni.equal": EffectInfo.OS_STR_EQUAL, } - elif args[0].concretetype.TO == rstr.UNICODE: + CHR = lltype.Char + elif SoU.TO == rstr.UNICODE: dict = {"stroruni.concat": EffectInfo.OS_UNI_CONCAT, "stroruni.slice": EffectInfo.OS_UNI_SLICE, "stroruni.equal": EffectInfo.OS_UNI_EQUAL, } + CHR = lltype.UniChar else: assert 0, "args[0].concretetype must be STR or UNICODE" # if oopspec_name == "stroruni.equal": - SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE) for otherindex, othername, argtypes, resulttype in [ - (EffectInfo.OS_STREQ_SLICE_CHECKNULL, "str.eq_slice_checknull", [SoU, lltype.Signed, lltype.Signed, SoU], @@ -1087,7 +1148,7 @@ lltype.Signed), (EffectInfo.OS_STREQ_SLICE_CHAR, "str.eq_slice_char", - [SoU, lltype.Signed, lltype.Signed, lltype.Char], + [SoU, lltype.Signed, lltype.Signed, CHR], lltype.Signed), (EffectInfo.OS_STREQ_NONNULL, "str.eq_nonnull", @@ -1095,22 +1156,27 @@ lltype.Signed), (EffectInfo.OS_STREQ_NONNULL_CHAR, "str.eq_nonnull_char", - [SoU, lltype.Char], + [SoU, CHR], lltype.Signed), (EffectInfo.OS_STREQ_CHECKNULL_CHAR, "str.eq_checknull_char", - [SoU, lltype.Char], + [SoU, CHR], lltype.Signed), (EffectInfo.OS_STREQ_LENGTHOK, "str.eq_lengthok", [SoU, SoU], lltype.Signed), ]: + if args[0].concretetype.TO == rstr.UNICODE: + otherindex += EffectInfo._OS_offset_uni self._register_extra_helper(otherindex, othername, argtypes, resulttype) # return self._handle_oopspec_call(op, args, dict[oopspec_name]) + def _handle_str2unicode_call(self, op, oopspec_name, args): + return self._handle_oopspec_call(op, args, EffectInfo.OS_STR2UNICODE) + # ---------- # VirtualRefs. @@ -1121,6 +1187,23 @@ vrefinfo.JIT_VIRTUAL_REF) return SpaceOperation(oopspec_name, list(args), op.result) + # ----------- + # rlib.libffi + + def _handle_libffi_call(self, op, oopspec_name, args): + if oopspec_name == 'libffi_prepare_call': + oopspecindex = EffectInfo.OS_LIBFFI_PREPARE + extraeffect = EffectInfo.EF_CANNOT_RAISE + elif oopspec_name.startswith('libffi_push_'): + oopspecindex = EffectInfo.OS_LIBFFI_PUSH_ARG + extraeffect = EffectInfo.EF_CANNOT_RAISE + elif oopspec_name.startswith('libffi_call_'): + oopspecindex = EffectInfo.OS_LIBFFI_CALL + extraeffect = EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE + else: + assert False, 'unsupported oopspec: %s' % oopspec_name + return self._handle_oopspec_call(op, args, oopspecindex, extraeffect) + def rewrite_op_jit_force_virtual(self, op): return self._do_builtin_call(op) Modified: pypy/branch/arm-backend/pypy/jit/codewriter/support.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/codewriter/support.py (original) +++ pypy/branch/arm-backend/pypy/jit/codewriter/support.py Tue Nov 23 10:12:47 2010 @@ -1,5 +1,5 @@ import sys -from pypy.rpython.lltypesystem import lltype, rclass +from pypy.rpython.lltypesystem import lltype, rclass, rffi from pypy.rpython.ootypesystem import ootype from pypy.rpython import rlist from pypy.rpython.lltypesystem import rstr as ll_rstr, rdict as ll_rdict @@ -8,6 +8,7 @@ from pypy.rpython.ootypesystem import rdict as oo_rdict from pypy.rpython.llinterp import LLInterpreter from pypy.rpython.extregistry import ExtRegistryEntry +from pypy.rpython.annlowlevel import cast_base_ptr_to_instance from pypy.translator.simplify import get_funcobj from pypy.translator.unsimplify import split_block from pypy.objspace.flow.model import Constant @@ -60,7 +61,7 @@ return rtyper.annotator.translator.graphs[0] def split_before_jit_merge_point(graph, portalblock, portalopindex): - """Find the block with 'jit_merge_point' and split just before, + """Split the block just before the 'jit_merge_point', making sure the input args are in the canonical order. """ # split the block just before the jit_merge_point() @@ -217,6 +218,33 @@ else: return x + +# libffi support +# -------------- + +def func(llfunc): + from pypy.rlib.libffi import Func + return cast_base_ptr_to_instance(Func, llfunc) + +def _ll_1_libffi_prepare_call(llfunc): + return func(llfunc)._prepare() + +def _ll_4_libffi_push_int(llfunc, value, ll_args, i): + return func(llfunc)._push_int(value, ll_args, i) + +def _ll_4_libffi_push_float(llfunc, value, ll_args, i): + return func(llfunc)._push_float(value, ll_args, i) + +def _ll_3_libffi_call_int(llfunc, funcsym, ll_args): + return func(llfunc)._do_call(funcsym, ll_args, rffi.LONG) + +def _ll_3_libffi_call_float(llfunc, funcsym, ll_args): + return func(llfunc)._do_call(funcsym, ll_args, rffi.DOUBLE) + +def _ll_3_libffi_call_void(llfunc, funcsym, ll_args): + return func(llfunc)._do_call(funcsym, ll_args, lltype.Void) + + # in the following calls to builtins, the JIT is allowed to look inside: inline_calls_to = [ ('int_floordiv_ovf_zer', [lltype.Signed, lltype.Signed], lltype.Signed), Modified: pypy/branch/arm-backend/pypy/jit/codewriter/test/test_codewriter.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/codewriter/test/test_codewriter.py (original) +++ pypy/branch/arm-backend/pypy/jit/codewriter/test/test_codewriter.py Tue Nov 23 10:12:47 2010 @@ -45,6 +45,7 @@ self.portal_graph = portal_graph self.portal_runner_ptr = "???" self.virtualizable_info = None + self.greenfield_info = None def test_loop(): Modified: pypy/branch/arm-backend/pypy/jit/codewriter/test/test_effectinfo.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/codewriter/test/test_effectinfo.py (original) +++ pypy/branch/arm-backend/pypy/jit/codewriter/test/test_effectinfo.py Tue Nov 23 10:12:47 2010 @@ -1,7 +1,8 @@ from pypy.rpython.lltypesystem.rclass import OBJECT from pypy.rpython.lltypesystem import lltype from pypy.rpython.ootypesystem import ootype -from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze +from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze,\ + EffectInfo class FakeCPU: def fielddescrof(self, T, fieldname): @@ -9,6 +10,14 @@ def arraydescrof(self, A): return ('arraydescr', A) +def test_no_oopspec_duplicate(): + # check that all the various EffectInfo.OS_* have unique values + oopspecs = set() + for name, value in EffectInfo.__dict__.iteritems(): + if name.startswith('OS_'): + assert value not in oopspecs + oopspecs.add(value) + def test_include_read_field(): S = lltype.GcStruct("S", ("a", lltype.Signed)) effects = frozenset([("readstruct", lltype.Ptr(S), "a")]) Modified: pypy/branch/arm-backend/pypy/jit/codewriter/test/test_jtransform.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/codewriter/test/test_jtransform.py (original) +++ pypy/branch/arm-backend/pypy/jit/codewriter/test/test_jtransform.py Tue Nov 23 10:12:47 2010 @@ -74,10 +74,48 @@ def calldescr_canraise(self, calldescr): return False +class FakeCallInfoCollection: + def __init__(self): + self.seen = [] + def add(self, oopspecindex, calldescr, func): + self.seen.append((oopspecindex, calldescr, func)) + def has_oopspec(self, oopspecindex): + for i, c, f in self.seen: + if i == oopspecindex: + return True + return False + class FakeBuiltinCallControl: + def __init__(self): + self.callinfocollection = FakeCallInfoCollection() def guess_call_kind(self, op): return 'builtin' - def getcalldescr(self, op, oopspecindex): + def getcalldescr(self, op, oopspecindex=None): + assert oopspecindex is not None # in this test + EI = effectinfo.EffectInfo + if oopspecindex != EI.OS_ARRAYCOPY: + PSTR = lltype.Ptr(rstr.STR) + PUNICODE = lltype.Ptr(rstr.UNICODE) + INT = lltype.Signed + UNICHAR = lltype.UniChar + argtypes = { + EI.OS_STR2UNICODE:([PSTR], PUNICODE), + EI.OS_STR_CONCAT: ([PSTR, PSTR], PSTR), + EI.OS_STR_SLICE: ([PSTR, INT, INT], PSTR), + EI.OS_UNI_CONCAT: ([PUNICODE, PUNICODE], PUNICODE), + EI.OS_UNI_SLICE: ([PUNICODE, INT, INT], PUNICODE), + EI.OS_UNI_EQUAL: ([PUNICODE, PUNICODE], lltype.Bool), + EI.OS_UNIEQ_SLICE_CHECKNULL:([PUNICODE, INT, INT, PUNICODE], INT), + EI.OS_UNIEQ_SLICE_NONNULL: ([PUNICODE, INT, INT, PUNICODE], INT), + EI.OS_UNIEQ_SLICE_CHAR: ([PUNICODE, INT, INT, UNICHAR], INT), + EI.OS_UNIEQ_NONNULL: ([PUNICODE, PUNICODE], INT), + EI.OS_UNIEQ_NONNULL_CHAR: ([PUNICODE, UNICHAR], INT), + EI.OS_UNIEQ_CHECKNULL_CHAR: ([PUNICODE, UNICHAR], INT), + EI.OS_UNIEQ_LENGTHOK: ([PUNICODE, PUNICODE], INT), + } + argtypes = argtypes[oopspecindex] + assert argtypes[0] == [v.concretetype for v in op.args[1:]] + assert argtypes[1] == op.result.concretetype return 'calldescr-%d' % oopspecindex def calldescr_canraise(self, calldescr): return False @@ -662,6 +700,80 @@ assert block.operations[1].result is None assert block.exits[0].args == [v1] +def test_jit_merge_point_1(): + class FakeJitDriverSD: + index = 42 + class jitdriver: + active = True + greens = ['green1', 'green2', 'voidgreen3'] + reds = ['red1', 'red2', 'voidred3'] + jd = FakeJitDriverSD() + v1 = varoftype(lltype.Signed) + v2 = varoftype(lltype.Signed) + vvoid1 = varoftype(lltype.Void) + v3 = varoftype(lltype.Signed) + v4 = varoftype(lltype.Signed) + vvoid2 = varoftype(lltype.Void) + v5 = varoftype(lltype.Void) + op = SpaceOperation('jit_marker', + [Constant('jit_merge_point', lltype.Void), + Constant(jd.jitdriver, lltype.Void), + v1, v2, vvoid1, v3, v4, vvoid2], v5) + tr = Transformer() + tr.portal_jd = jd + oplist = tr.rewrite_operation(op) + assert len(oplist) == 6 + assert oplist[0].opname == '-live-' + assert oplist[1].opname == 'int_guard_value' + assert oplist[1].args == [v1] + assert oplist[2].opname == '-live-' + assert oplist[3].opname == 'int_guard_value' + assert oplist[3].args == [v2] + assert oplist[4].opname == 'jit_merge_point' + assert oplist[4].args[0].value == 42 + assert list(oplist[4].args[1]) == [v1, v2] + assert list(oplist[4].args[4]) == [v3, v4] + assert oplist[5].opname == '-live-' + +def test_getfield_gc(): + S = lltype.GcStruct('S', ('x', lltype.Char)) + v1 = varoftype(lltype.Ptr(S)) + v2 = varoftype(lltype.Char) + op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2) + op1 = Transformer(FakeCPU()).rewrite_operation(op) + assert op1.opname == 'getfield_gc_i' + assert op1.args == [v1, ('fielddescr', S, 'x')] + assert op1.result == v2 + +def test_getfield_gc_pure(): + S = lltype.GcStruct('S', ('x', lltype.Char), + hints={'immutable': True}) + v1 = varoftype(lltype.Ptr(S)) + v2 = varoftype(lltype.Char) + op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2) + op1 = Transformer(FakeCPU()).rewrite_operation(op) + assert op1.opname == 'getfield_gc_i_pure' + assert op1.args == [v1, ('fielddescr', S, 'x')] + assert op1.result == v2 + +def test_getfield_gc_greenfield(): + class FakeCC: + def get_vinfo(self, v): + return None + def could_be_green_field(self, S1, name1): + assert S1 is S + assert name1 == 'x' + return True + S = lltype.GcStruct('S', ('x', lltype.Char), + hints={'immutable': True}) + v1 = varoftype(lltype.Ptr(S)) + v2 = varoftype(lltype.Char) + op = SpaceOperation('getfield', [v1, Constant('x', lltype.Void)], v2) + op1 = Transformer(FakeCPU(), FakeCC()).rewrite_operation(op) + assert op1.opname == 'getfield_gc_i_greenfield' + assert op1.args == [v1, ('fielddescr', S, 'x')] + assert op1.result == v2 + def test_int_abs(): v1 = varoftype(lltype.Signed) v2 = varoftype(lltype.Signed) @@ -711,7 +823,8 @@ v2 = varoftype(PSTR) v3 = varoftype(PSTR) op = SpaceOperation('direct_call', [const(func), v1, v2], v3) - tr = Transformer(FakeCPU(), FakeBuiltinCallControl()) + cc = FakeBuiltinCallControl() + tr = Transformer(FakeCPU(), cc) op1 = tr.rewrite_operation(op) assert op1.opname == 'residual_call_r_r' assert op1.args[0].value == func @@ -720,9 +833,10 @@ assert op1.result == v3 # # check the callinfo_for_oopspec - got = effectinfo.callinfo_for_oopspec(effectinfo.EffectInfo.OS_UNI_CONCAT) - assert got[0] == op1.args[1] # the calldescr - assert heaptracker.int2adr(got[1]) == llmemory.cast_ptr_to_adr(func) + got = cc.callinfocollection.seen[0] + assert got[0] == effectinfo.EffectInfo.OS_UNI_CONCAT + assert got[1] == op1.args[1] # the calldescr + assert heaptracker.int2adr(got[2]) == llmemory.cast_ptr_to_adr(func) def test_str_slice(): # test that the oopspec is present and correctly transformed @@ -766,6 +880,47 @@ assert op1.args[3] == ListOfKind('ref', [v1]) assert op1.result == v4 +def test_str2unicode(): + # test that the oopspec is present and correctly transformed + PSTR = lltype.Ptr(rstr.STR) + PUNICODE = lltype.Ptr(rstr.UNICODE) + FUNC = lltype.FuncType([PSTR], PUNICODE) + func = lltype.functionptr(FUNC, 'll_str2unicode', + _callable=rstr.LLHelpers.ll_str2unicode) + v1 = varoftype(PSTR) + v2 = varoftype(PUNICODE) + op = SpaceOperation('direct_call', [const(func), v1], v2) + tr = Transformer(FakeCPU(), FakeBuiltinCallControl()) + op1 = tr.rewrite_operation(op) + assert op1.opname == 'residual_call_r_r' + assert op1.args[0].value == func + assert op1.args[1] == 'calldescr-%d' % effectinfo.EffectInfo.OS_STR2UNICODE + assert op1.args[2] == ListOfKind('ref', [v1]) + assert op1.result == v2 + +def test_unicode_eq_checknull_char(): + # test that the oopspec is present and correctly transformed + PUNICODE = lltype.Ptr(rstr.UNICODE) + FUNC = lltype.FuncType([PUNICODE, PUNICODE], lltype.Bool) + func = lltype.functionptr(FUNC, 'll_streq', + _callable=rstr.LLHelpers.ll_streq) + v1 = varoftype(PUNICODE) + v2 = varoftype(PUNICODE) + v3 = varoftype(lltype.Bool) + op = SpaceOperation('direct_call', [const(func), v1, v2], v3) + cc = FakeBuiltinCallControl() + tr = Transformer(FakeCPU(), cc) + op1 = tr.rewrite_operation(op) + assert op1.opname == 'residual_call_r_i' + assert op1.args[0].value == func + assert op1.args[1] == 'calldescr-%d' % effectinfo.EffectInfo.OS_UNI_EQUAL + assert op1.args[2] == ListOfKind('ref', [v1, v2]) + assert op1.result == v3 + # test that the OS_UNIEQ_* functions are registered + cic = cc.callinfocollection + assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL) + assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR) + def test_list_ll_arraycopy(): from pypy.rlib.rgc import ll_arraycopy LIST = lltype.GcArray(lltype.Signed) Modified: pypy/branch/arm-backend/pypy/jit/codewriter/test/test_list.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/codewriter/test/test_list.py (original) +++ pypy/branch/arm-backend/pypy/jit/codewriter/test/test_list.py Tue Nov 23 10:12:47 2010 @@ -19,6 +19,7 @@ class FakeCPU: class arraydescrof(AbstractDescr): def __init__(self, ARRAY): + assert ARRAY.OF != lltype.Void self.ARRAY = ARRAY def __repr__(self): return '' Modified: pypy/branch/arm-backend/pypy/jit/metainterp/blackhole.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/blackhole.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/blackhole.py Tue Nov 23 10:12:47 2010 @@ -760,6 +760,24 @@ def bhimpl_debug_fatalerror(msg): llop.debug_fatalerror(lltype.Void, msg) + @arguments("r", "i", "i", "i", "i") + def bhimpl_jit_debug(string, arg1=0, arg2=0, arg3=0, arg4=0): + pass + + @arguments("i") + def bhimpl_int_assert_green(x): + pass + @arguments("r") + def bhimpl_ref_assert_green(x): + pass + @arguments("f") + def bhimpl_float_assert_green(x): + pass + + @arguments(returns="i") + def bhimpl_current_trace_length(): + return -1 + # ---------- # the main hints and recursive calls @@ -1073,6 +1091,10 @@ bhimpl_getfield_vable_r = bhimpl_getfield_gc_r bhimpl_getfield_vable_f = bhimpl_getfield_gc_f + bhimpl_getfield_gc_i_greenfield = bhimpl_getfield_gc_i + bhimpl_getfield_gc_r_greenfield = bhimpl_getfield_gc_r + bhimpl_getfield_gc_f_greenfield = bhimpl_getfield_gc_f + @arguments("cpu", "i", "d", returns="i") def bhimpl_getfield_raw_i(cpu, struct, fielddescr): return cpu.bh_getfield_raw_i(struct, fielddescr) Modified: pypy/branch/arm-backend/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/compile.py Tue Nov 23 10:12:47 2010 @@ -1,4 +1,5 @@ +from pypy.rpython.lltypesystem import lltype from pypy.rpython.ootypesystem import ootype from pypy.objspace.flow.model import Constant, Variable from pypy.rlib.objectmodel import we_are_translated @@ -14,6 +15,7 @@ from pypy.jit.metainterp.specnode import NotSpecNode, more_general_specnodes from pypy.jit.metainterp.typesystem import llhelper, oohelper from pypy.jit.metainterp.optimizeutil import InvalidLoop +from pypy.jit.metainterp.resume import NUMBERING from pypy.jit.codewriter import heaptracker def giveup(): @@ -48,13 +50,12 @@ # ____________________________________________________________ -def compile_new_loop(metainterp, old_loop_tokens, greenkey, start): +def compile_new_loop(metainterp, old_loop_tokens, start): """Try to compile a new loop by closing the current history back to the first operation. """ history = metainterp.history loop = create_empty_loop(metainterp) - loop.greenkey = greenkey loop.inputargs = history.inputargs for box in loop.inputargs: assert isinstance(box, Box) @@ -123,7 +124,6 @@ if not we_are_translated(): show_loop(metainterp_sd) TreeLoop.check_consistency_of(inputargs, operations) - pass metainterp_sd.profiler.start_backend() debug_start("jit-backend") try: @@ -208,8 +208,7 @@ } class ResumeDescr(AbstractFailDescr): - def __init__(self, original_greenkey): - self.original_greenkey = original_greenkey + pass class ResumeGuardDescr(ResumeDescr): _counter = 0 # if < 0, there is one counter per value; @@ -218,7 +217,7 @@ # this class also gets the following attributes stored by resume.py code rd_snapshot = None rd_frame_info_list = None - rd_numb = None + rd_numb = lltype.nullptr(NUMBERING) rd_consts = None rd_virtuals = None rd_pendingfields = None @@ -228,8 +227,7 @@ CNT_FLOAT = -0x60000000 CNT_MASK = 0x1FFFFFFF - def __init__(self, metainterp_sd, original_greenkey): - ResumeDescr.__init__(self, original_greenkey) + def __init__(self, metainterp_sd): self.metainterp_sd = metainterp_sd def store_final_boxes(self, guard_op, boxes): @@ -333,14 +331,14 @@ res.rd_pendingfields = self.rd_pendingfields def _clone_if_mutable(self): - res = ResumeGuardDescr(self.metainterp_sd, self.original_greenkey) + res = ResumeGuardDescr(self.metainterp_sd) self.copy_all_attrbutes_into(res) return res class ResumeGuardForcedDescr(ResumeGuardDescr): - def __init__(self, metainterp_sd, original_greenkey, jitdriver_sd): - ResumeGuardDescr.__init__(self, metainterp_sd, original_greenkey) + def __init__(self, metainterp_sd, jitdriver_sd): + ResumeGuardDescr.__init__(self, metainterp_sd) self.jitdriver_sd = jitdriver_sd def handle_fail(self, metainterp_sd, jitdriver_sd): @@ -370,7 +368,8 @@ from pypy.jit.metainterp.resume import force_from_resumedata metainterp_sd = self.metainterp_sd vinfo = self.jitdriver_sd.virtualizable_info - all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo) + ginfo = self.jitdriver_sd.greenfield_info + all_virtuals = force_from_resumedata(metainterp_sd, self, vinfo, ginfo) # The virtualizable data was stored on the real virtualizable above. # Handle all_virtuals: keep them for later blackholing from the # future failure of the GUARD_NOT_FORCED @@ -406,7 +405,6 @@ def _clone_if_mutable(self): res = ResumeGuardForcedDescr(self.metainterp_sd, - self.original_greenkey, self.jitdriver_sd) self.copy_all_attrbutes_into(res) return res @@ -473,9 +471,8 @@ class ResumeFromInterpDescr(ResumeDescr): - def __init__(self, original_greenkey, redkey): - ResumeDescr.__init__(self, original_greenkey) - self.redkey = redkey + def __init__(self, original_greenkey): + self.original_greenkey = original_greenkey def compile_and_attach(self, metainterp, new_loop): # We managed to create a bridge going from the interpreter @@ -484,10 +481,8 @@ # with completely unoptimized arguments, as in the interpreter. metainterp_sd = metainterp.staticdata jitdriver_sd = metainterp.jitdriver_sd - metainterp.history.inputargs = self.redkey - new_loop_token = make_loop_token(len(self.redkey), jitdriver_sd) - new_loop.greenkey = self.original_greenkey - new_loop.inputargs = self.redkey + redargs = new_loop.inputargs + new_loop_token = make_loop_token(len(redargs), jitdriver_sd) new_loop.token = new_loop_token send_loop_to_backend(metainterp_sd, new_loop, "entry bridge") # send the new_loop to warmspot.py, to be called directly the next time @@ -599,5 +594,5 @@ ResOperation(rop.FINISH, finishargs, None, descr=jd.portal_finishtoken) ] operations[1].setfailargs([]) - cpu.compile_loop(inputargs, operations, loop_token) + cpu.compile_loop(inputargs, operations, loop_token, log=False) return loop_token Modified: pypy/branch/arm-backend/pypy/jit/metainterp/executor.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/executor.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/executor.py Tue Nov 23 10:12:47 2010 @@ -80,6 +80,9 @@ do_call_loopinvariant = do_call do_call_may_force = do_call +def do_call_c(cpu, metainterp, argboxes, descr): + raise NotImplementedError("Should never be called directly") + def do_getarrayitem_gc(cpu, _, arraybox, indexbox, arraydescr): array = arraybox.getref_base() index = indexbox.getint() @@ -205,8 +208,8 @@ def do_copystrcontent(cpu, _, srcbox, dstbox, srcstartbox, dststartbox, lengthbox): - src = srcbox.getptr(lltype.Ptr(rstr.STR)) - dst = dstbox.getptr(lltype.Ptr(rstr.STR)) + src = srcbox.getref(lltype.Ptr(rstr.STR)) + dst = dstbox.getref(lltype.Ptr(rstr.STR)) srcstart = srcstartbox.getint() dststart = dststartbox.getint() length = lengthbox.getint() @@ -214,8 +217,8 @@ def do_copyunicodecontent(cpu, _, srcbox, dstbox, srcstartbox, dststartbox, lengthbox): - src = srcbox.getptr(lltype.Ptr(rstr.UNICODE)) - dst = dstbox.getptr(lltype.Ptr(rstr.UNICODE)) + src = srcbox.getref(lltype.Ptr(rstr.UNICODE)) + dst = dstbox.getref(lltype.Ptr(rstr.UNICODE)) srcstart = srcstartbox.getint() dststart = dststartbox.getint() length = lengthbox.getint() @@ -304,6 +307,7 @@ rop.CALL_ASSEMBLER, rop.COND_CALL_GC_WB, rop.DEBUG_MERGE_POINT, + rop.JIT_DEBUG, rop.SETARRAYITEM_RAW, ): # list of opcodes never executed by pyjitpl continue @@ -428,6 +432,10 @@ if arity == 3: func = get_execute_funclist(3, False)[opnum] return func(cpu, metainterp, argboxes[0], argboxes[1], argboxes[2]) + if arity == 5: # copystrcontent, copyunicodecontent + func = get_execute_funclist(5, False)[opnum] + return func(cpu, metainterp, argboxes[0], argboxes[1], + argboxes[2], argboxes[3], argboxes[4]) raise NotImplementedError Modified: pypy/branch/arm-backend/pypy/jit/metainterp/graphpage.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/graphpage.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/graphpage.py Tue Nov 23 10:12:47 2010 @@ -153,7 +153,7 @@ opindex = opstartindex while True: op = operations[opindex] - lines.append(repr(op)) + lines.append(op.repr(graytext=True)) if is_interesting_guard(op): tgt = op.getdescr()._debug_suboperations[0] tgt_g, tgt_i = self.all_operations[tgt] Modified: pypy/branch/arm-backend/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/history.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/history.py Tue Nov 23 10:12:47 2010 @@ -698,6 +698,21 @@ return result _const_ptr_for_string = {} +def get_const_ptr_for_unicode(s): + from pypy.rpython.annlowlevel import llunicode + if not we_are_translated(): + try: + return _const_ptr_for_unicode[s] + except KeyError: + pass + if isinstance(s, str): + s = unicode(s) + result = ConstPtr(lltype.cast_opaque_ptr(llmemory.GCREF, llunicode(s))) + if not we_are_translated(): + _const_ptr_for_unicode[s] = result + return result +_const_ptr_for_unicode = {} + # ____________________________________________________________ # The TreeLoop class contains a loop or a generalized loop, i.e. a tree Modified: pypy/branch/arm-backend/pypy/jit/metainterp/jitdriver.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/jitdriver.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/jitdriver.py Tue Nov 23 10:12:47 2010 @@ -13,8 +13,10 @@ # self.num_red_args ... pypy.jit.metainterp.warmspot # self.result_type ... pypy.jit.metainterp.warmspot # self.virtualizable_info... pypy.jit.metainterp.warmspot + # self.greenfield_info ... pypy.jit.metainterp.warmspot # self.warmstate ... pypy.jit.metainterp.warmspot # self.handle_jitexc_from_bh pypy.jit.metainterp.warmspot + # self.no_loop_header ... pypy.jit.metainterp.warmspot # self.portal_finishtoken... pypy.jit.metainterp.pyjitpl # self.index ... pypy.jit.codewriter.call # self.mainjitcode ... pypy.jit.codewriter.call Modified: pypy/branch/arm-backend/pypy/jit/metainterp/logger.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/logger.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/logger.py Tue Nov 23 10:12:47 2010 @@ -81,7 +81,8 @@ op = operations[i] if op.getopnum() == rop.DEBUG_MERGE_POINT: loc = op.getarg(0)._get_str() - debug_print("debug_merge_point('%s')" % (loc,)) + reclev = op.getarg(1).getint() + debug_print("debug_merge_point('%s', %s)" % (loc, reclev)) continue args = ", ".join([self.repr_of_arg(memo, op.getarg(i)) for i in range(op.numargs())]) if op.result is not None: Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimize_nopspec.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/optimize_nopspec.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/optimize_nopspec.py Tue Nov 23 10:12:47 2010 @@ -14,6 +14,9 @@ def _optimize_loop(metainterp_sd, old_loop_tokens, loop): cpu = metainterp_sd.cpu metainterp_sd.logger_noopt.log_loop(loop.inputargs, loop.operations) + # XXX the following lines are probably still needed, to discard invalid + # loops. bit silly to run a full perfect specialization and throw the + # result away. finder = PerfectSpecializationFinder(cpu) finder.find_nodes_loop(loop, False) if old_loop_tokens: @@ -31,6 +34,7 @@ def _optimize_bridge(metainterp_sd, old_loop_tokens, bridge): cpu = metainterp_sd.cpu metainterp_sd.logger_noopt.log_loop(bridge.inputargs, bridge.operations) + # XXX same comment as above applies finder = BridgeSpecializationFinder(cpu) finder.find_nodes_bridge(bridge) if old_loop_tokens: Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/__init__.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/__init__.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/__init__.py Tue Nov 23 10:12:47 2010 @@ -3,6 +3,7 @@ from pypy.jit.metainterp.optimizeopt.intbounds import OptIntBounds from pypy.jit.metainterp.optimizeopt.virtualize import OptVirtualize from pypy.jit.metainterp.optimizeopt.heap import OptHeap +from pypy.jit.metainterp.optimizeopt.fficall import OptFfiCall from pypy.jit.metainterp.optimizeopt.string import OptString def optimize_loop_1(metainterp_sd, loop, virtuals=True): @@ -14,8 +15,9 @@ optimizations = [OptIntBounds(), OptRewrite(), OptVirtualize(), -# OptString(), + OptString(), OptHeap(), + OptFfiCall(), ] optimizer = Optimizer(metainterp_sd, loop, optimizations, virtuals) optimizer.propagate_all_forward() Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/intbounds.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/intbounds.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/intbounds.py Tue Nov 23 10:12:47 2010 @@ -191,6 +191,7 @@ v1.intbound.make_ge(IntLowerBound(0)) optimize_STRLEN = optimize_ARRAYLEN_GC + optimize_UNICODELEN = optimize_ARRAYLEN_GC def make_int_lt(self, box1, box2): v1 = self.getvalue(box1) Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/optimizer.py Tue Nov 23 10:12:47 2010 @@ -66,10 +66,10 @@ assert isinstance(constbox, Const) self.box = constbox self.level = LEVEL_CONSTANT - try: - val = self.box.getint() + if isinstance(constbox, ConstInt): + val = constbox.getint() self.intbound = IntBound(val, val) - except NotImplementedError: + else: self.intbound = IntUnbounded() def get_constant_class(self, cpu): @@ -299,7 +299,9 @@ return CVAL_ZERO def propagate_all_forward(self): - self.exception_might_have_happened = False + self.exception_might_have_happened = True + # ^^^ at least at the start of bridges. For loops, we could set + # it to False, but we probably don't care self.newoperations = [] self.i = 0 while self.i < len(self.loop.operations): Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/rewrite.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/rewrite.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/rewrite.py Tue Nov 23 10:12:47 2010 @@ -3,6 +3,7 @@ from pypy.jit.metainterp.history import ConstInt from pypy.jit.metainterp.optimizeutil import _findall from pypy.jit.metainterp.resoperation import rop, ResOperation +from pypy.jit.codewriter.effectinfo import EffectInfo class OptRewrite(Optimization): """Rewrite operations into equivalent, cheaper operations. @@ -243,11 +244,11 @@ self.optimizer.exception_might_have_happened = False def optimize_CALL_LOOPINVARIANT(self, op): - funcvalue = self.getvalue(op.getarg(0)) - if not funcvalue.is_constant(): - self.emit_operation(op) - return - key = make_hashable_int(op.getarg(0).getint()) + arg = op.getarg(0) + # 'arg' must be a Const, because residual_call in codewriter + # expects a compile-time constant + assert isinstance(arg, Const) + key = make_hashable_int(arg.getint()) resvalue = self.optimizer.loop_invariant_results.get(key, None) if resvalue is not None: self.make_equal_to(op.result, resvalue) @@ -323,8 +324,37 @@ ## return ## self.emit_operation(op) -optimize_ops = _findall(OptRewrite, 'optimize_') - + def optimize_CALL(self, op): + # dispatch based on 'oopspecindex' to a method that handles + # specifically the given oopspec call. For non-oopspec calls, + # oopspecindex is just zero. + effectinfo = op.getdescr().get_extra_info() + if effectinfo is not None: + oopspecindex = effectinfo.oopspecindex + if oopspecindex == EffectInfo.OS_ARRAYCOPY: + if self._optimize_CALL_ARRAYCOPY(op): + return + self.emit_operation(op) + def _optimize_CALL_ARRAYCOPY(self, op): + source_value = self.getvalue(op.getarg(1)) + dest_value = self.getvalue(op.getarg(2)) + source_start_box = self.get_constant_box(op.getarg(3)) + dest_start_box = self.get_constant_box(op.getarg(4)) + length = self.get_constant_box(op.getarg(5)) + if (source_value.is_virtual() and source_start_box and dest_start_box + and length and dest_value.is_virtual()): + # XXX optimize the case where dest value is not virtual, + # but we still can avoid a mess + source_start = source_start_box.getint() + dest_start = dest_start_box.getint() + for index in range(length.getint()): + val = source_value.getitem(index + source_start) + dest_value.setitem(index + dest_start, val) + return True + if length and length.getint() == 0: + return True # 0-length arraycopy + return False + +optimize_ops = _findall(OptRewrite, 'optimize_') - Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/string.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/string.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/string.py Tue Nov 23 10:12:47 2010 @@ -3,64 +3,115 @@ from pypy.jit.metainterp.history import Box, BoxInt, BoxPtr from pypy.jit.metainterp.history import Const, ConstInt, ConstPtr from pypy.jit.metainterp.history import get_const_ptr_for_string +from pypy.jit.metainterp.history import get_const_ptr_for_unicode from pypy.jit.metainterp.resoperation import rop, ResOperation from pypy.jit.metainterp.optimizeopt import optimizer, virtualize from pypy.jit.metainterp.optimizeopt.optimizer import CONST_0, CONST_1 from pypy.jit.metainterp.optimizeopt.optimizer import llhelper from pypy.jit.metainterp.optimizeutil import _findall -from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.jit.codewriter import heaptracker from pypy.rlib.unroll import unrolling_iterable +from pypy.rlib.objectmodel import specialize, we_are_translated + + +class StrOrUnicode(object): + def __init__(self, LLTYPE, hlstr, emptystr, chr, + NEWSTR, STRLEN, STRGETITEM, STRSETITEM, COPYSTRCONTENT, + OS_offset): + self.LLTYPE = LLTYPE + self.hlstr = hlstr + self.emptystr = emptystr + self.chr = chr + self.NEWSTR = NEWSTR + self.STRLEN = STRLEN + self.STRGETITEM = STRGETITEM + self.STRSETITEM = STRSETITEM + self.COPYSTRCONTENT = COPYSTRCONTENT + self.OS_offset = OS_offset + + def _freeze_(self): + return True + +mode_string = StrOrUnicode(rstr.STR, annlowlevel.hlstr, '', chr, + rop.NEWSTR, rop.STRLEN, rop.STRGETITEM, + rop.STRSETITEM, rop.COPYSTRCONTENT, 0) +mode_unicode = StrOrUnicode(rstr.UNICODE, annlowlevel.hlunicode, u'', unichr, + rop.NEWUNICODE, rop.UNICODELEN, rop.UNICODEGETITEM, + rop.UNICODESETITEM, rop.COPYUNICODECONTENT, + EffectInfo._OS_offset_uni) + +# ____________________________________________________________ class __extend__(optimizer.OptValue): """New methods added to the base class OptValue for this file.""" - def getstrlen(self, newoperations): - s = self.get_constant_string() - if s is not None: - return ConstInt(len(s)) + def getstrlen(self, newoperations, mode): + if mode is mode_string: + s = self.get_constant_string_spec(mode_string) + if s is not None: + return ConstInt(len(s)) else: - if newoperations is None: - return None - self.ensure_nonnull() - box = self.force_box() - lengthbox = BoxInt() - newoperations.append(ResOperation(rop.STRLEN, [box], lengthbox)) - return lengthbox + s = self.get_constant_string_spec(mode_unicode) + if s is not None: + return ConstInt(len(s)) + if newoperations is None: + return None + self.ensure_nonnull() + box = self.force_box() + lengthbox = BoxInt() + newoperations.append(ResOperation(mode.STRLEN, [box], lengthbox)) + return lengthbox - def get_constant_string(self): + @specialize.arg(1) + def get_constant_string_spec(self, mode): if self.is_constant(): - s = self.box.getref(lltype.Ptr(rstr.STR)) - return annlowlevel.hlstr(s) + s = self.box.getref(lltype.Ptr(mode.LLTYPE)) + return mode.hlstr(s) else: return None - def string_copy_parts(self, newoperations, targetbox, offsetbox): + def string_copy_parts(self, newoperations, targetbox, offsetbox, mode): # Copies the pointer-to-string 'self' into the target string # given by 'targetbox', at the specified offset. Returns the offset # at the end of the copy. - lengthbox = self.getstrlen(newoperations) + lengthbox = self.getstrlen(newoperations, mode) srcbox = self.force_box() return copy_str_content(newoperations, srcbox, targetbox, - CONST_0, offsetbox, lengthbox) + CONST_0, offsetbox, lengthbox, mode) class VAbstractStringValue(virtualize.AbstractVirtualValue): - _attrs_ = () + _attrs_ = ('mode',) + + def __init__(self, optimizer, keybox, source_op, mode): + virtualize.AbstractVirtualValue.__init__(self, optimizer, keybox, + source_op) + self.mode = mode def _really_force(self): - s = self.get_constant_string() - if s is not None: - c_s = get_const_ptr_for_string(s) - self.make_constant(c_s) - return + if self.mode is mode_string: + s = self.get_constant_string_spec(mode_string) + if s is not None: + c_s = get_const_ptr_for_string(s) + self.make_constant(c_s) + return + else: + s = self.get_constant_string_spec(mode_unicode) + if s is not None: + c_s = get_const_ptr_for_unicode(s) + self.make_constant(c_s) + return assert self.source_op is not None self.box = box = self.source_op.result newoperations = self.optimizer.newoperations - lengthbox = self.getstrlen(newoperations) - newoperations.append(ResOperation(rop.NEWSTR, [lengthbox], box)) - self.string_copy_parts(newoperations, box, CONST_0) + lengthbox = self.getstrlen(newoperations, self.mode) + op = ResOperation(self.mode.NEWSTR, [lengthbox], box) + if not we_are_translated(): + op.name = 'FORCE' + newoperations.append(op) + self.string_copy_parts(newoperations, box, CONST_0, self.mode) class VStringPlainValue(VAbstractStringValue): @@ -74,7 +125,7 @@ assert 0 <= start <= stop <= len(longerlist) self._chars = longerlist[start:stop] - def getstrlen(self, _): + def getstrlen(self, _, mode): if self._lengthbox is None: self._lengthbox = ConstInt(len(self._chars)) return self._lengthbox @@ -86,18 +137,21 @@ assert isinstance(charvalue, optimizer.OptValue) self._chars[index] = charvalue - def get_constant_string(self): + @specialize.arg(1) + def get_constant_string_spec(self, mode): for c in self._chars: if c is optimizer.CVAL_UNINITIALIZED_ZERO or not c.is_constant(): return None - return ''.join([chr(c.box.getint()) for c in self._chars]) + return mode.emptystr.join([mode.chr(c.box.getint()) + for c in self._chars]) - def string_copy_parts(self, newoperations, targetbox, offsetbox): + def string_copy_parts(self, newoperations, targetbox, offsetbox, mode): for i in range(len(self._chars)): charbox = self._chars[i].force_box() - newoperations.append(ResOperation(rop.STRSETITEM, [targetbox, - offsetbox, - charbox], None)) + newoperations.append(ResOperation(mode.STRSETITEM, [targetbox, + offsetbox, + charbox], + None)) offsetbox = _int_add(newoperations, offsetbox, CONST_1) return offsetbox @@ -109,7 +163,7 @@ value.get_args_for_fail(modifier) def _make_virtual(self, modifier): - return modifier.make_vstrplain() + return modifier.make_vstrplain(self.mode is mode_unicode) class VStringConcatValue(VAbstractStringValue): @@ -120,23 +174,24 @@ self.right = right self.lengthbox = lengthbox - def getstrlen(self, _): + def getstrlen(self, _, mode): return self.lengthbox - def get_constant_string(self): - s1 = self.left.get_constant_string() + @specialize.arg(1) + def get_constant_string_spec(self, mode): + s1 = self.left.get_constant_string_spec(mode) if s1 is None: return None - s2 = self.right.get_constant_string() + s2 = self.right.get_constant_string_spec(mode) if s2 is None: return None return s1 + s2 - def string_copy_parts(self, newoperations, targetbox, offsetbox): + def string_copy_parts(self, newoperations, targetbox, offsetbox, mode): offsetbox = self.left.string_copy_parts(newoperations, targetbox, - offsetbox) + offsetbox, mode) offsetbox = self.right.string_copy_parts(newoperations, targetbox, - offsetbox) + offsetbox, mode) return offsetbox def get_args_for_fail(self, modifier): @@ -150,7 +205,7 @@ self.right.get_args_for_fail(modifier) def _make_virtual(self, modifier): - return modifier.make_vstrconcat() + return modifier.make_vstrconcat(self.mode is mode_unicode) class VStringSliceValue(VAbstractStringValue): @@ -162,12 +217,13 @@ self.vstart = vstart self.vlength = vlength - def getstrlen(self, _): + def getstrlen(self, _, mode): return self.vlength.force_box() - def get_constant_string(self): + @specialize.arg(1) + def get_constant_string_spec(self, mode): if self.vstart.is_constant() and self.vlength.is_constant(): - s1 = self.vstr.get_constant_string() + s1 = self.vstr.get_constant_string_spec(mode) if s1 is None: return None start = self.vstart.box.getint() @@ -177,12 +233,12 @@ return s1[start : start + length] return None - def string_copy_parts(self, newoperations, targetbox, offsetbox): - lengthbox = self.getstrlen(newoperations) + def string_copy_parts(self, newoperations, targetbox, offsetbox, mode): + lengthbox = self.getstrlen(newoperations, mode) return copy_str_content(newoperations, self.vstr.force_box(), targetbox, self.vstart.force_box(), offsetbox, - lengthbox) + lengthbox, mode) def get_args_for_fail(self, modifier): if self.box is None and not modifier.already_seen_virtual(self.keybox): @@ -195,11 +251,11 @@ self.vlength.get_args_for_fail(modifier) def _make_virtual(self, modifier): - return modifier.make_vstrslice() + return modifier.make_vstrslice(self.mode is mode_unicode) def copy_str_content(newoperations, srcbox, targetbox, - srcoffsetbox, offsetbox, lengthbox): + srcoffsetbox, offsetbox, lengthbox, mode): if isinstance(srcbox, ConstPtr) and isinstance(srcoffsetbox, Const): M = 5 else: @@ -208,17 +264,18 @@ # up to M characters are done "inline", i.e. with STRGETITEM/STRSETITEM # instead of just a COPYSTRCONTENT. for i in range(lengthbox.value): - charbox = _strgetitem(newoperations, srcbox, srcoffsetbox) + charbox = _strgetitem(newoperations, srcbox, srcoffsetbox, mode) srcoffsetbox = _int_add(newoperations, srcoffsetbox, CONST_1) - newoperations.append(ResOperation(rop.STRSETITEM, [targetbox, - offsetbox, - charbox], None)) + newoperations.append(ResOperation(mode.STRSETITEM, [targetbox, + offsetbox, + charbox], + None)) offsetbox = _int_add(newoperations, offsetbox, CONST_1) else: nextoffsetbox = _int_add(newoperations, offsetbox, lengthbox) - op = ResOperation(rop.COPYSTRCONTENT, [srcbox, targetbox, - srcoffsetbox, offsetbox, - lengthbox], None) + op = ResOperation(mode.COPYSTRCONTENT, [srcbox, targetbox, + srcoffsetbox, offsetbox, + lengthbox], None) newoperations.append(op) offsetbox = nextoffsetbox return offsetbox @@ -245,12 +302,16 @@ newoperations.append(ResOperation(rop.INT_SUB, [box1, box2], resbox)) return resbox -def _strgetitem(newoperations, strbox, indexbox): +def _strgetitem(newoperations, strbox, indexbox, mode): if isinstance(strbox, ConstPtr) and isinstance(indexbox, ConstInt): - s = strbox.getref(lltype.Ptr(rstr.STR)) - return ConstInt(ord(s.chars[indexbox.getint()])) + if mode is mode_string: + s = strbox.getref(lltype.Ptr(rstr.STR)) + return ConstInt(ord(s.chars[indexbox.getint()])) + else: + s = strbox.getref(lltype.Ptr(rstr.UNICODE)) + return ConstInt(ord(s.chars[indexbox.getint()])) resbox = BoxInt() - newoperations.append(ResOperation(rop.STRGETITEM, [strbox, indexbox], + newoperations.append(ResOperation(mode.STRGETITEM, [strbox, indexbox], resbox)) return resbox @@ -258,62 +319,34 @@ class OptString(optimizer.Optimization): "Handling of strings and unicodes." - def make_vstring_plain(self, box, source_op=None): - vvalue = VStringPlainValue(self.optimizer, box, source_op) + def make_vstring_plain(self, box, source_op, mode): + vvalue = VStringPlainValue(self.optimizer, box, source_op, mode) self.make_equal_to(box, vvalue) return vvalue - def make_vstring_concat(self, box, source_op=None): - vvalue = VStringConcatValue(self.optimizer, box, source_op) + def make_vstring_concat(self, box, source_op, mode): + vvalue = VStringConcatValue(self.optimizer, box, source_op, mode) self.make_equal_to(box, vvalue) return vvalue - def make_vstring_slice(self, box, source_op=None): - vvalue = VStringSliceValue(self.optimizer, box, source_op) + def make_vstring_slice(self, box, source_op, mode): + vvalue = VStringSliceValue(self.optimizer, box, source_op, mode) self.make_equal_to(box, vvalue) return vvalue - def optimize_CALL(self, op): - # dispatch based on 'oopspecindex' to a method that handles - # specifically the given oopspec call. For non-oopspec calls, - # oopspecindex is just zero. - effectinfo = op.getdescr().get_extra_info() - if effectinfo is not None: - oopspecindex = effectinfo.oopspecindex - for value, meth in opt_call_oopspec_ops: - if oopspecindex == value: - if meth(self, op): - return - self.emit_operation(op) - - def opt_call_oopspec_ARRAYCOPY(self, op): - source_value = self.getvalue(op.getarg(1)) - dest_value = self.getvalue(op.getarg(2)) - source_start_box = self.get_constant_box(op.getarg(3)) - dest_start_box = self.get_constant_box(op.getarg(4)) - length = self.get_constant_box(op.getarg(5)) - if (source_value.is_virtual() and source_start_box and dest_start_box - and length and dest_value.is_virtual()): - # XXX optimize the case where dest value is not virtual, - # but we still can avoid a mess - source_start = source_start_box.getint() - dest_start = dest_start_box.getint() - for index in range(length.getint()): - val = source_value.getitem(index + source_start) - dest_value.setitem(index + dest_start, val) - return True - if length and length.getint() == 0: - return True # 0-length arraycopy - return False - def optimize_NEWSTR(self, op): + self._optimize_NEWSTR(op, mode_string) + def optimize_NEWUNICODE(self, op): + self._optimize_NEWSTR(op, mode_unicode) + + def _optimize_NEWSTR(self, op, mode): length_box = self.get_constant_box(op.getarg(0)) if length_box: # if the original 'op' did not have a ConstInt as argument, # build a new one with the ConstInt argument if not isinstance(op.getarg(0), ConstInt): - op = ResOperation(rop.NEWSTR, [length_box], op.result) - vvalue = self.make_vstring_plain(op.result, op) + op = ResOperation(mode.NEWSTR, [length_box], op.result) + vvalue = self.make_vstring_plain(op.result, op, mode) vvalue.setup(length_box.getint()) else: self.getvalue(op.result).ensure_nonnull() @@ -329,13 +362,20 @@ value.ensure_nonnull() self.emit_operation(op) + optimize_UNICODESETITEM = optimize_STRSETITEM + def optimize_STRGETITEM(self, op): + self._optimize_STRGETITEM(op, mode_string) + def optimize_UNICODEGETITEM(self, op): + self._optimize_STRGETITEM(op, mode_unicode) + + def _optimize_STRGETITEM(self, op, mode): value = self.getvalue(op.getarg(0)) vindex = self.getvalue(op.getarg(1)) - vresult = self.strgetitem(value, vindex) + vresult = self.strgetitem(value, vindex, mode) self.make_equal_to(op.result, vresult) - def strgetitem(self, value, vindex): + def strgetitem(self, value, vindex, mode): value.ensure_nonnull() # if value.is_virtual() and isinstance(value, VStringSliceValue): @@ -350,28 +390,71 @@ return value.getitem(vindex.box.getint()) # resbox = _strgetitem(self.optimizer.newoperations, - value.force_box(),vindex.force_box()) + value.force_box(),vindex.force_box(), mode) return self.getvalue(resbox) def optimize_STRLEN(self, op): + self._optimize_STRLEN(op, mode_string) + def optimize_UNICODELEN(self, op): + self._optimize_STRLEN(op, mode_unicode) + + def _optimize_STRLEN(self, op, mode): value = self.getvalue(op.getarg(0)) - lengthbox = value.getstrlen(self.optimizer.newoperations) + lengthbox = value.getstrlen(self.optimizer.newoperations, mode) self.make_equal_to(op.result, self.getvalue(lengthbox)) - def opt_call_oopspec_STR_CONCAT(self, op): + def optimize_CALL(self, op): + # dispatch based on 'oopspecindex' to a method that handles + # specifically the given oopspec call. For non-oopspec calls, + # oopspecindex is just zero. + effectinfo = op.getdescr().get_extra_info() + if effectinfo is not None: + oopspecindex = effectinfo.oopspecindex + for value, meth in opt_call_oopspec_ops: + if oopspecindex == value: # a match with the OS_STR_xxx + if meth(self, op, mode_string): + return + break + if oopspecindex == value + EffectInfo._OS_offset_uni: + # a match with the OS_UNI_xxx + if meth(self, op, mode_unicode): + return + break + if oopspecindex == EffectInfo.OS_STR2UNICODE: + if self.opt_call_str_STR2UNICODE(op): + return + self.emit_operation(op) + + def opt_call_str_STR2UNICODE(self, op): + # Constant-fold unicode("constant string"). + # More generally, supporting non-constant but virtual cases is + # not obvious, because of the exception UnicodeDecodeError that + # can be raised by ll_str2unicode() + varg = self.getvalue(op.getarg(1)) + s = varg.get_constant_string_spec(mode_string) + if s is None: + return False + try: + u = unicode(s) + except UnicodeDecodeError: + return False + self.make_constant(op.result, get_const_ptr_for_unicode(u)) + return True + + def opt_call_stroruni_STR_CONCAT(self, op, mode): vleft = self.getvalue(op.getarg(1)) vright = self.getvalue(op.getarg(2)) vleft.ensure_nonnull() vright.ensure_nonnull() newoperations = self.optimizer.newoperations - len1box = vleft.getstrlen(newoperations) - len2box = vright.getstrlen(newoperations) + len1box = vleft.getstrlen(newoperations, mode) + len2box = vright.getstrlen(newoperations, mode) lengthbox = _int_add(newoperations, len1box, len2box) - value = self.make_vstring_concat(op.result, op) + value = self.make_vstring_concat(op.result, op, mode) value.setup(vleft, vright, lengthbox) return True - def opt_call_oopspec_STR_SLICE(self, op): + def opt_call_stroruni_STR_SLICE(self, op, mode): newoperations = self.optimizer.newoperations vstr = self.getvalue(op.getarg(1)) vstart = self.getvalue(op.getarg(2)) @@ -380,7 +463,7 @@ if (isinstance(vstr, VStringPlainValue) and vstart.is_constant() and vstop.is_constant()): # slicing with constant bounds of a VStringPlainValue - value = self.make_vstring_plain(op.result, op) + value = self.make_vstring_plain(op.result, op, mode) value.setup_slice(vstr._chars, vstart.box.getint(), vstop.box.getint()) return True @@ -398,16 +481,16 @@ vstart.force_box()) vstart = self.getvalue(startbox) # - value = self.make_vstring_slice(op.result, op) + value = self.make_vstring_slice(op.result, op, mode) value.setup(vstr, vstart, self.getvalue(lengthbox)) return True - def opt_call_oopspec_STR_EQUAL(self, op): + def opt_call_stroruni_STR_EQUAL(self, op, mode): v1 = self.getvalue(op.getarg(1)) v2 = self.getvalue(op.getarg(2)) # - l1box = v1.getstrlen(None) - l2box = v2.getstrlen(None) + l1box = v1.getstrlen(None, mode) + l2box = v2.getstrlen(None, mode) if (l1box is not None and l2box is not None and isinstance(l1box, ConstInt) and isinstance(l2box, ConstInt) and @@ -416,13 +499,13 @@ self.make_constant(op.result, CONST_0) return True # - if self.handle_str_equal_level1(v1, v2, op.result): + if self.handle_str_equal_level1(v1, v2, op.result, mode): return True - if self.handle_str_equal_level1(v2, v1, op.result): + if self.handle_str_equal_level1(v2, v1, op.result, mode): return True - if self.handle_str_equal_level2(v1, v2, op.result): + if self.handle_str_equal_level2(v1, v2, op.result, mode): return True - if self.handle_str_equal_level2(v2, v1, op.result): + if self.handle_str_equal_level2(v2, v1, op.result, mode): return True # if v1.is_nonnull() and v2.is_nonnull(): @@ -434,37 +517,37 @@ else: do = EffectInfo.OS_STREQ_NONNULL self.generate_modified_call(do, [v1.force_box(), - v2.force_box()], op.result) + v2.force_box()], op.result, mode) return True return False - def handle_str_equal_level1(self, v1, v2, resultbox): - l2box = v2.getstrlen(None) + def handle_str_equal_level1(self, v1, v2, resultbox, mode): + l2box = v2.getstrlen(None, mode) if isinstance(l2box, ConstInt): if l2box.value == 0: - lengthbox = v1.getstrlen(self.optimizer.newoperations) + lengthbox = v1.getstrlen(self.optimizer.newoperations, mode) seo = self.optimizer.send_extra_operation seo(ResOperation(rop.INT_EQ, [lengthbox, CONST_0], resultbox)) return True if l2box.value == 1: - l1box = v1.getstrlen(None) + l1box = v1.getstrlen(None, mode) if isinstance(l1box, ConstInt) and l1box.value == 1: # comparing two single chars - vchar1 = self.strgetitem(v1, optimizer.CVAL_ZERO) - vchar2 = self.strgetitem(v2, optimizer.CVAL_ZERO) + vchar1 = self.strgetitem(v1, optimizer.CVAL_ZERO, mode) + vchar2 = self.strgetitem(v2, optimizer.CVAL_ZERO, mode) seo = self.optimizer.send_extra_operation seo(ResOperation(rop.INT_EQ, [vchar1.force_box(), vchar2.force_box()], resultbox)) return True if isinstance(v1, VStringSliceValue): - vchar = self.strgetitem(v2, optimizer.CVAL_ZERO) + vchar = self.strgetitem(v2, optimizer.CVAL_ZERO, mode) do = EffectInfo.OS_STREQ_SLICE_CHAR self.generate_modified_call(do, [v1.vstr.force_box(), v1.vstart.force_box(), v1.vlength.force_box(), vchar.force_box()], - resultbox) + resultbox, mode) return True # if v2.is_null(): @@ -482,17 +565,18 @@ # return False - def handle_str_equal_level2(self, v1, v2, resultbox): - l2box = v2.getstrlen(None) + def handle_str_equal_level2(self, v1, v2, resultbox, mode): + l2box = v2.getstrlen(None, mode) if isinstance(l2box, ConstInt): if l2box.value == 1: - vchar = self.strgetitem(v2, optimizer.CVAL_ZERO) + vchar = self.strgetitem(v2, optimizer.CVAL_ZERO, mode) if v1.is_nonnull(): do = EffectInfo.OS_STREQ_NONNULL_CHAR else: do = EffectInfo.OS_STREQ_CHECKNULL_CHAR self.generate_modified_call(do, [v1.force_box(), - vchar.force_box()], resultbox) + vchar.force_box()], resultbox, + mode) return True # if v1.is_virtual() and isinstance(v1, VStringSliceValue): @@ -503,12 +587,14 @@ self.generate_modified_call(do, [v1.vstr.force_box(), v1.vstart.force_box(), v1.vlength.force_box(), - v2.force_box()], resultbox) + v2.force_box()], resultbox, mode) return True return False - def generate_modified_call(self, oopspecindex, args, result): - calldescr, func = callinfo_for_oopspec(oopspecindex) + def generate_modified_call(self, oopspecindex, args, result, mode): + oopspecindex += mode.OS_offset + cic = self.optimizer.metainterp_sd.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(oopspecindex) op = ResOperation(rop.CALL, [ConstInt(func)] + args, result, descr=calldescr) self.optimizer.newoperations.append(op) @@ -525,7 +611,7 @@ optimize_ops = _findall(OptString, 'optimize_') def _findall_call_oopspec(): - prefix = 'opt_call_oopspec_' + prefix = 'opt_call_stroruni_' result = [] for name in dir(OptString): if name.startswith(prefix): Modified: pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/virtualize.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/virtualize.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/optimizeopt/virtualize.py Tue Nov 23 10:12:47 2010 @@ -74,6 +74,8 @@ assert self.source_op is not None # ^^^ This case should not occur any more (see test_bug_3). # + if not we_are_translated(): + self.source_op.name = 'FORCE ' + self.source_op.name newoperations = self.optimizer.newoperations newoperations.append(self.source_op) self.box = box = self.source_op.result @@ -134,6 +136,11 @@ fielddescrs = self._get_field_descr_list() return modifier.make_virtual(self.known_class, fielddescrs) + def __repr__(self): + cls_name = self.known_class.value.adr.ptr._obj._TYPE._name + field_names = [field.name for field in self._fields] + return "" % (cls_name, field_names) + class VStructValue(AbstractVirtualStructValue): def __init__(self, optimizer, structdescr, keybox, source_op=None): @@ -165,6 +172,8 @@ def _really_force(self): assert self.source_op is not None + if not we_are_translated(): + self.source_op.name = 'FORCE ' + self.source_op.name newoperations = self.optimizer.newoperations newoperations.append(self.source_op) self.box = box = self.source_op.result Modified: pypy/branch/arm-backend/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/pyjitpl.py Tue Nov 23 10:12:47 2010 @@ -1,4 +1,4 @@ -import py, os +import py, os, sys from pypy.rpython.lltypesystem import lltype, llmemory, rclass from pypy.rlib.objectmodel import we_are_translated from pypy.rlib.unroll import unrolling_iterable @@ -14,7 +14,7 @@ from pypy.jit.metainterp.logger import Logger from pypy.jit.metainterp.jitprof import EmptyProfiler from pypy.jit.metainterp.jitprof import GUARDS, RECORDED_OPS, ABORT_ESCAPE -from pypy.jit.metainterp.jitprof import ABORT_TOO_LONG, ABORT_BRIDGE +from pypy.jit.metainterp.jitprof import ABORT_TOO_LONG from pypy.jit.metainterp.jitexc import JitException, get_llexception from pypy.rlib.rarithmetic import intmask from pypy.rlib.objectmodel import specialize @@ -94,6 +94,18 @@ else: raise AssertionError(argcode) outvalue[startindex+i] = reg + def _put_back_list_of_boxes(self, outvalue, startindex, position): + code = self.bytecode + length = ord(code[position]) + position += 1 + for i in range(length): + index = ord(code[position+i]) + box = outvalue[startindex+i] + if box.type == history.INT: self.registers_i[index] = box + elif box.type == history.REF: self.registers_r[index] = box + elif box.type == history.FLOAT: self.registers_f[index] = box + else: raise AssertionError(box.type) + def get_current_position_info(self): return self.jitcode.get_live_vars_info(self.pc) @@ -498,6 +510,22 @@ opimpl_getfield_gc_r_pure = _opimpl_getfield_gc_pure_any opimpl_getfield_gc_f_pure = _opimpl_getfield_gc_pure_any + @arguments("orgpc", "box", "descr") + def _opimpl_getfield_gc_greenfield_any(self, pc, box, fielddescr): + ginfo = self.metainterp.jitdriver_sd.greenfield_info + if (ginfo is not None and fielddescr in ginfo.green_field_descrs + and not self._nonstandard_virtualizable(pc, box)): + # fetch the result, but consider it as a Const box and don't + # record any operation + resbox = executor.execute(self.metainterp.cpu, self.metainterp, + rop.GETFIELD_GC_PURE, fielddescr, box) + return resbox.constbox() + # fall-back + return self.execute_with_descr(rop.GETFIELD_GC_PURE, fielddescr, box) + opimpl_getfield_gc_i_greenfield = _opimpl_getfield_gc_greenfield_any + opimpl_getfield_gc_r_greenfield = _opimpl_getfield_gc_greenfield_any + opimpl_getfield_gc_f_greenfield = _opimpl_getfield_gc_greenfield_any + @arguments("box", "descr", "box") def _opimpl_setfield_gc_any(self, box, fielddescr, valuebox): self.execute_with_descr(rop.SETFIELD_GC, fielddescr, box, valuebox) @@ -529,7 +557,8 @@ def _nonstandard_virtualizable(self, pc, box): # returns True if 'box' is actually not the "standard" virtualizable # that is stored in metainterp.virtualizable_boxes[-1] - if self.metainterp.jitdriver_sd.virtualizable_info is None: + if (self.metainterp.jitdriver_sd.virtualizable_info is None and + self.metainterp.jitdriver_sd.greenfield_info is None): return True # can occur in case of multiple JITs standard_box = self.metainterp.virtualizable_boxes[-1] if standard_box is box: @@ -797,14 +826,20 @@ for i in range(num_green_args): assert isinstance(varargs[i], Const) - @arguments("orgpc", "int", "boxes3", "boxes3") - def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, redboxes): + @arguments("orgpc", "int", "boxes3", "jitcode_position", "boxes3") + def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, + jcposition, redboxes): + any_operation = len(self.metainterp.history.operations) > 0 jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex] self.verify_green_args(jitdriver_sd, greenboxes) # xxx we may disable the following line in some context later - self.debug_merge_point(jitdriver_sd, greenboxes) + self.debug_merge_point(jitdriver_sd, self.metainterp.in_recursion, + greenboxes) if self.metainterp.seen_loop_header_for_jdindex < 0: - return + if not jitdriver_sd.no_loop_header or not any_operation: + return + # automatically add a loop_header if there is none + self.metainterp.seen_loop_header_for_jdindex = jdindex # assert self.metainterp.seen_loop_header_for_jdindex == jdindex, ( "found a loop_header for a JitDriver that does not match " @@ -821,6 +856,10 @@ self.pc = orgpc self.metainterp.reached_loop_header(greenboxes, redboxes) self.pc = saved_pc + # no exception, which means that the jit_merge_point did not + # close the loop. We have to put the possibly-modified list + # 'redboxes' back into the registers where it comes from. + put_back_list_of_boxes3(self, jcposition, redboxes) else: # warning! careful here. We have to return from the current # frame containing the jit_merge_point, and then use @@ -839,13 +878,13 @@ assembler_call=True) raise ChangeFrame - def debug_merge_point(self, jitdriver_sd, greenkey): + def debug_merge_point(self, jitdriver_sd, in_recursion, greenkey): # debugging: produce a DEBUG_MERGE_POINT operation loc = jitdriver_sd.warmstate.get_location_str(greenkey) debug_print(loc) constloc = self.metainterp.cpu.ts.conststr(loc) self.metainterp.history.record(rop.DEBUG_MERGE_POINT, - [constloc], None) + [constloc, ConstInt(in_recursion)], None) @arguments("box", "label") def opimpl_goto_if_exception_mismatch(self, vtablebox, next_exc_target): @@ -893,6 +932,45 @@ msg = box.getref(lltype.Ptr(rstr.STR)) lloperation.llop.debug_fatalerror(msg) + @arguments("box", "box", "box", "box", "box") + def opimpl_jit_debug(self, stringbox, arg1box, arg2box, arg3box, arg4box): + from pypy.rpython.lltypesystem import rstr + from pypy.rpython.annlowlevel import hlstr + msg = stringbox.getref(lltype.Ptr(rstr.STR)) + debug_print('jit_debug:', hlstr(msg), + arg1box.getint(), arg2box.getint(), + arg3box.getint(), arg4box.getint()) + args = [stringbox, arg1box, arg2box, arg3box, arg4box] + i = 4 + while i > 0 and args[i].getint() == -sys.maxint-1: + i -= 1 + assert i >= 0 + op = self.metainterp.history.record(rop.JIT_DEBUG, args[:i+1], None) + self.metainterp.attach_debug_info(op) + + @arguments("box") + def _opimpl_assert_green(self, box): + if not isinstance(box, Const): + msg = "assert_green failed at %s:%d" % ( + self.jitcode.name, + self.pc) + if we_are_translated(): + from pypy.rpython.annlowlevel import llstr + from pypy.rpython.lltypesystem import lloperation + lloperation.llop.debug_fatalerror(lltype.Void, llstr(msg)) + else: + from pypy.rlib.jit import AssertGreenFailed + raise AssertGreenFailed(msg) + + opimpl_int_assert_green = _opimpl_assert_green + opimpl_ref_assert_green = _opimpl_assert_green + opimpl_float_assert_green = _opimpl_assert_green + + @arguments() + def opimpl_current_trace_length(self): + trace_length = len(self.metainterp.history.operations) + return ConstInt(trace_length) + @arguments("box") def opimpl_virtual_ref(self, box): # Details on the content of metainterp.virtualref_boxes: @@ -987,18 +1065,16 @@ else: moreargs = list(extraargs) metainterp_sd = metainterp.staticdata - original_greenkey = metainterp.resumekey.original_greenkey if opnum == rop.GUARD_NOT_FORCED: resumedescr = compile.ResumeGuardForcedDescr(metainterp_sd, - original_greenkey, metainterp.jitdriver_sd) else: - resumedescr = compile.ResumeGuardDescr(metainterp_sd, - original_greenkey) + resumedescr = compile.ResumeGuardDescr(metainterp_sd) guard_op = metainterp.history.record(opnum, moreargs, None, descr=resumedescr) virtualizable_boxes = None - if metainterp.jitdriver_sd.virtualizable_info is not None: + if (metainterp.jitdriver_sd.virtualizable_info is not None or + metainterp.jitdriver_sd.greenfield_info is not None): virtualizable_boxes = metainterp.virtualizable_boxes saved_pc = self.pc if resumepc >= 0: @@ -1199,6 +1275,7 @@ # self.jitdrivers_sd = codewriter.callcontrol.jitdrivers_sd self.virtualref_info = codewriter.callcontrol.virtualref_info + self.callinfocollection = codewriter.callcontrol.callinfocollection self.setup_jitdrivers_sd(optimizer) # # store this information for fastpath of call_assembler @@ -1562,7 +1639,7 @@ assert jitdriver_sd is self.jitdriver_sd self.create_empty_history() try: - original_boxes = self.initialize_original_boxes(jitdriver_sd,*args) + original_boxes = self.initialize_original_boxes(jitdriver_sd, *args) return self._compile_and_run_once(original_boxes) finally: self.staticdata.profiler.end_tracing() @@ -1573,9 +1650,8 @@ self.current_merge_points = [(original_boxes, 0)] num_green_args = self.jitdriver_sd.num_green_args original_greenkey = original_boxes[:num_green_args] - redkey = original_boxes[num_green_args:] - self.resumekey = compile.ResumeFromInterpDescr(original_greenkey, - redkey) + self.resumekey = compile.ResumeFromInterpDescr(original_greenkey) + self.history.inputargs = original_boxes[num_green_args:] self.seen_loop_header_for_jdindex = -1 try: self.interpret() @@ -1597,11 +1673,7 @@ debug_stop('jit-tracing') def _handle_guard_failure(self, key): - original_greenkey = key.original_greenkey - # notice that here we just put the greenkey - # use -1 to mark that we will have to give up - # because we cannot reconstruct the beginning of the proper loop - self.current_merge_points = [(original_greenkey, -1)] + self.current_merge_points = [] self.resumekey = key self.seen_loop_header_for_jdindex = -1 try: @@ -1646,6 +1718,7 @@ duplicates) live_arg_boxes += self.virtualizable_boxes live_arg_boxes.pop() + # assert len(self.virtualref_boxes) == 0, "missing virtual_ref_finish()?" # Called whenever we reach the 'loop_header' hint. # First, attempt to make a bridge: @@ -1664,7 +1737,7 @@ num_green_args = self.jitdriver_sd.num_green_args for j in range(len(self.current_merge_points)-1, -1, -1): original_boxes, start = self.current_merge_points[j] - assert len(original_boxes) == len(live_arg_boxes) or start < 0 + assert len(original_boxes) == len(live_arg_boxes) for i in range(num_green_args): box1 = original_boxes[i] box2 = live_arg_boxes[i] @@ -1673,10 +1746,6 @@ break else: # Found! Compile it as a loop. - if start < 0: - # we cannot reconstruct the beginning of the proper loop - raise SwitchToBlackhole(ABORT_BRIDGE) - # raises in case it works -- which is the common case self.compile(original_boxes, live_arg_boxes, start) # creation of the loop was cancelled! @@ -1741,8 +1810,7 @@ greenkey = original_boxes[:num_green_args] old_loop_tokens = self.get_compiled_merge_points(greenkey) self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None) - loop_token = compile.compile_new_loop(self, old_loop_tokens, - greenkey, start) + loop_token = compile.compile_new_loop(self, old_loop_tokens, start) if loop_token is not None: # raise if it *worked* correctly raise GenerateMergePoint(live_arg_boxes, loop_token) self.history.operations.pop() # remove the JUMP @@ -1832,6 +1900,7 @@ f.setup_call(original_boxes) assert self.in_recursion == 0 self.virtualref_boxes = [] + self.initialize_withgreenfields(original_boxes) self.initialize_virtualizable(original_boxes) def initialize_state_from_guard_failure(self, resumedescr): @@ -1856,6 +1925,14 @@ self.virtualizable_boxes.append(virtualizable_box) self.initialize_virtualizable_enter() + def initialize_withgreenfields(self, original_boxes): + ginfo = self.jitdriver_sd.greenfield_info + if ginfo is not None: + assert self.jitdriver_sd.virtualizable_info is None + index = (self.jitdriver_sd.num_green_args + + ginfo.red_index) + self.virtualizable_boxes = [original_boxes[index]] + def initialize_virtualizable_enter(self): vinfo = self.jitdriver_sd.virtualizable_info virtualizable_box = self.virtualizable_boxes[-1] @@ -1949,8 +2026,10 @@ def rebuild_state_after_failure(self, resumedescr): vinfo = self.jitdriver_sd.virtualizable_info + ginfo = self.jitdriver_sd.greenfield_info self.framestack = [] - boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo) + boxlists = resume.rebuild_from_resumedata(self, resumedescr, vinfo, + ginfo) inputargs_and_holes, virtualizable_boxes, virtualref_boxes = boxlists # # virtual refs: make the vrefs point to the freshly allocated virtuals @@ -1975,6 +2054,12 @@ assert not virtualizable.vable_token # fill the virtualizable with the local boxes self.synchronize_virtualizable() + # + elif self.jitdriver_sd.greenfield_info: + self.virtualizable_boxes = virtualizable_boxes + else: + assert not virtualizable_boxes + # return inputargs_and_holes def check_synchronized_virtualizable(self): @@ -2048,7 +2133,8 @@ for i in range(len(boxes)): if boxes[i] is oldbox: boxes[i] = newbox - if self.jitdriver_sd.virtualizable_info is not None: + if (self.jitdriver_sd.virtualizable_info is not None or + self.jitdriver_sd.greenfield_info is not None): boxes = self.virtualizable_boxes for i in range(len(boxes)): if boxes[i] is oldbox: @@ -2224,6 +2310,8 @@ else: raise AssertionError("bad argcode") position += 1 + elif argtype == "jitcode_position": + value = position else: raise AssertionError("bad argtype: %r" % (argtype,)) args += (value,) @@ -2268,3 +2356,15 @@ argtypes = unrolling_iterable(unboundmethod.argtypes) handler.func_name = 'handler_' + name return handler + +def put_back_list_of_boxes3(frame, position, newvalue): + code = frame.bytecode + length1 = ord(code[position]) + position2 = position + 1 + length1 + length2 = ord(code[position2]) + position3 = position2 + 1 + length2 + length3 = ord(code[position3]) + assert len(newvalue) == length1 + length2 + length3 + frame._put_back_list_of_boxes(newvalue, 0, position) + frame._put_back_list_of_boxes(newvalue, length1, position2) + frame._put_back_list_of_boxes(newvalue, length1 + length2, position3) Modified: pypy/branch/arm-backend/pypy/jit/metainterp/resoperation.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/resoperation.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/resoperation.py Tue Nov 23 10:12:47 2010 @@ -93,7 +93,7 @@ def __repr__(self): return self.repr() - def repr(self): + def repr(self, graytext=False): # RPython-friendly version if self.result is not None: sres = '%s = ' % (self.result,) @@ -101,6 +101,8 @@ sres = '' if self.name: prefix = "%s:%s " % (self.name, self.pc) + if graytext: + prefix = "\f%s\f" % prefix else: prefix = "" args = self.getarglist() @@ -141,6 +143,16 @@ def can_raise(self): return rop._CANRAISE_FIRST <= self.getopnum() <= rop._CANRAISE_LAST + def is_malloc(self): + # a slightly different meaning from can_malloc + return rop._MALLOC_FIRST <= self.getopnum() <= rop._MALLOC_LAST + + def can_malloc(self): + return self.is_call() or self.is_malloc() + + def is_call(self): + return rop._CALL_FIRST <= self.getopnum() <= rop._CALL_LAST + def is_ovf(self): return rop._OVF_FIRST <= self.getopnum() <= rop._OVF_LAST @@ -439,9 +451,13 @@ 'GETARRAYITEM_RAW/2d', 'GETFIELD_GC/1d', 'GETFIELD_RAW/1d', + '_MALLOC_FIRST', 'NEW/0d', 'NEW_WITH_VTABLE/1', 'NEW_ARRAY/1d', + 'NEWSTR/1', + 'NEWUNICODE/1', + '_MALLOC_LAST', 'FORCE_TOKEN/0', 'VIRTUAL_REF/2', # removed before it's passed to the backend '_NOSIDEEFFECT_LAST', # ----- end of no_side_effect operations ----- @@ -450,26 +466,27 @@ 'SETARRAYITEM_RAW/3d', 'SETFIELD_GC/2d', 'SETFIELD_RAW/2d', - 'NEWSTR/1', 'STRSETITEM/3', 'UNICODESETITEM/3', - 'NEWUNICODE/1', #'RUNTIMENEW/1', # ootype operation - 'COND_CALL_GC_WB/1d', # [objptr] (for the write barrier) - 'DEBUG_MERGE_POINT/1', # debugging only + 'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier) + 'DEBUG_MERGE_POINT/2', # debugging only + 'JIT_DEBUG/*', # debugging only 'VIRTUAL_REF_FINISH/2', # removed before it's passed to the backend 'COPYSTRCONTENT/5', # src, dst, srcstart, dststart, length 'COPYUNICODECONTENT/5', '_CANRAISE_FIRST', # ----- start of can_raise operations ----- + '_CALL_FIRST', 'CALL/*d', - 'CALL_ASSEMBLER/*d', + 'CALL_ASSEMBLER/*d', # call already compiled assembler 'CALL_MAY_FORCE/*d', 'CALL_LOOPINVARIANT/*d', #'OOSEND', # ootype operation #'OOSEND_PURE', # ootype operation 'CALL_PURE/*d', # removed before it's passed to the backend # CALL_PURE(result, func, arg_1,..,arg_n) + '_CALL_LAST', '_CANRAISE_LAST', # ----- end of can_raise operations ----- '_OVF_FIRST', # ----- start of is_ovf operations ----- Modified: pypy/branch/arm-backend/pypy/jit/metainterp/resume.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/resume.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/resume.py Tue Nov 23 10:12:47 2010 @@ -4,8 +4,7 @@ from pypy.jit.metainterp.history import INT, REF, FLOAT, HOLE from pypy.jit.metainterp.resoperation import rop from pypy.jit.metainterp import jitprof -from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec -from pypy.jit.codewriter.effectinfo import funcptr_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.rpython.lltypesystem import lltype, llmemory, rffi, rstr from pypy.rlib import rarithmetic from pypy.rlib.objectmodel import we_are_translated, specialize @@ -66,12 +65,21 @@ snapshot = Snapshot(snapshot, boxes) storage.rd_snapshot = snapshot -class Numbering(object): - __slots__ = ('prev', 'nums') - - def __init__(self, prev, nums): - self.prev = prev - self.nums = nums +# +# The following is equivalent to the RPython-level declaration: +# +# class Numbering: __slots__ = ['prev', 'nums'] +# +# except that it is more compact in translated programs, because the +# array 'nums' is inlined in the single NUMBERING object. This is +# important because this is often the biggest single consumer of memory +# in a pypy-c-jit. +# +NUMBERINGP = lltype.Ptr(lltype.GcForwardReference()) +NUMBERING = lltype.GcStruct('Numbering', + ('prev', NUMBERINGP), + ('nums', lltype.Array(rffi.SHORT))) +NUMBERINGP.TO.become(NUMBERING) TAGMASK = 3 @@ -163,7 +171,7 @@ def number(self, values, snapshot): if snapshot is None: - return None, {}, 0 + return lltype.nullptr(NUMBERING), {}, 0 if snapshot in self.numberings: numb, liveboxes, v = self.numberings[snapshot] return numb, liveboxes.copy(), v @@ -172,7 +180,7 @@ n = len(liveboxes)-v boxes = snapshot.boxes length = len(boxes) - nums = [UNASSIGNED] * length + numb = lltype.malloc(NUMBERING, length) for i in range(length): box = boxes[i] value = values.get(box, None) @@ -191,9 +199,9 @@ tagged = tag(n, TAGBOX) n += 1 liveboxes[box] = tagged - nums[i] = tagged + numb.nums[i] = tagged # - numb = Numbering(numb1, nums) + numb.prev = numb1 self.numberings[snapshot] = numb, liveboxes, v return numb, liveboxes.copy(), v @@ -255,13 +263,19 @@ def make_varray(self, arraydescr): return VArrayInfo(arraydescr) - def make_vstrplain(self): + def make_vstrplain(self, is_unicode=False): + if is_unicode: + return VUniPlainInfo() return VStrPlainInfo() - def make_vstrconcat(self): + def make_vstrconcat(self, is_unicode=False): + if is_unicode: + return VUniConcatInfo() return VStrConcatInfo() - def make_vstrslice(self): + def make_vstrslice(self, is_unicode=False): + if is_unicode: + return VUniSliceInfo() return VStrSliceInfo() def register_virtual_fields(self, virtualbox, fieldboxes): @@ -292,7 +306,7 @@ # compute the numbering storage = self.storage # make sure that nobody attached resume data to this guard yet - assert storage.rd_numb is None + assert not storage.rd_numb numb, liveboxes_from_env, v = self.memo.number(values, storage.rd_snapshot) self.liveboxes_from_env = liveboxes_from_env @@ -550,6 +564,60 @@ for i in self.fieldnums: debug_print("\t\t", str(untag(i))) + +class VUniPlainInfo(AbstractVirtualInfo): + """Stands for the unicode string made out of the characters of all + fieldnums.""" + + @specialize.argtype(1) + def allocate(self, decoder, index): + length = len(self.fieldnums) + string = decoder.allocate_unicode(length) + decoder.virtuals_cache[index] = string + for i in range(length): + decoder.unicode_setitem(string, i, self.fieldnums[i]) + return string + + def debug_prints(self): + debug_print("\tvuniplaininfo length", len(self.fieldnums)) + + +class VUniConcatInfo(AbstractVirtualInfo): + """Stands for the unicode string made out of the concatenation of two + other unicode strings.""" + + @specialize.argtype(1) + def allocate(self, decoder, index): + # xxx for blackhole resuming, this will build all intermediate + # strings and throw them away immediately, which is a bit sub- + # efficient. Not sure we care. + left, right = self.fieldnums + string = decoder.concat_unicodes(left, right) + decoder.virtuals_cache[index] = string + return string + + def debug_prints(self): + debug_print("\tvuniconcatinfo") + for i in self.fieldnums: + debug_print("\t\t", str(untag(i))) + + +class VUniSliceInfo(AbstractVirtualInfo): + """Stands for the unicode string made out of slicing another + unicode string.""" + + @specialize.argtype(1) + def allocate(self, decoder, index): + largerstr, start, length = self.fieldnums + string = decoder.slice_unicode(largerstr, start, length) + decoder.virtuals_cache[index] = string + return string + + def debug_prints(self): + debug_print("\tvunisliceinfo") + for i in self.fieldnums: + debug_print("\t\t", str(untag(i))) + # ____________________________________________________________ class AbstractResumeDataReader(object): @@ -629,9 +697,11 @@ # ---------- when resuming for pyjitpl.py, make boxes ---------- -def rebuild_from_resumedata(metainterp, storage, virtualizable_info): +def rebuild_from_resumedata(metainterp, storage, virtualizable_info, + greenfield_info): resumereader = ResumeDataBoxReader(storage, metainterp) - boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info) + boxes = resumereader.consume_vref_and_vable_boxes(virtualizable_info, + greenfield_info) virtualizable_boxes, virtualref_boxes = boxes frameinfo = storage.rd_frame_info_list while True: @@ -661,31 +731,36 @@ self.boxes_f = boxes_f self._prepare_next_section(info) - def consume_virtualizable_boxes(self, vinfo, nums): + def consume_virtualizable_boxes(self, vinfo, numb): # we have to ignore the initial part of 'nums' (containing vrefs), # find the virtualizable from nums[-1], and use it to know how many # boxes of which type we have to return. This does not write # anything into the virtualizable. - virtualizablebox = self.decode_ref(nums[-1]) + index = len(numb.nums) - 1 + virtualizablebox = self.decode_ref(numb.nums[index]) virtualizable = vinfo.unwrap_virtualizable_box(virtualizablebox) - return vinfo.load_list_of_boxes(virtualizable, self, nums) + return vinfo.load_list_of_boxes(virtualizable, self, numb) - def consume_virtualref_boxes(self, nums, end): + def consume_virtualref_boxes(self, numb, end): # Returns a list of boxes, assumed to be all BoxPtrs. # We leave up to the caller to call vrefinfo.continue_tracing(). assert (end & 1) == 0 - return [self.decode_ref(nums[i]) for i in range(end)] + return [self.decode_ref(numb.nums[i]) for i in range(end)] - def consume_vref_and_vable_boxes(self, vinfo): - nums = self.cur_numb.nums - self.cur_numb = self.cur_numb.prev - if vinfo is None: - virtualizable_boxes = None - end = len(nums) + def consume_vref_and_vable_boxes(self, vinfo, ginfo): + numb = self.cur_numb + self.cur_numb = numb.prev + if vinfo is not None: + virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, numb) + end = len(numb.nums) - len(virtualizable_boxes) + elif ginfo is not None: + index = len(numb.nums) - 1 + virtualizable_boxes = [self.decode_ref(numb.nums[index])] + end = len(numb.nums) - 1 else: - virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, nums) - end = len(nums) - len(virtualizable_boxes) - virtualref_boxes = self.consume_virtualref_boxes(nums, end) + virtualizable_boxes = None + end = len(numb.nums) + virtualref_boxes = self.consume_virtualref_boxes(numb, end) return virtualizable_boxes, virtualref_boxes def allocate_with_vtable(self, known_class): @@ -709,14 +784,44 @@ strbox, ConstInt(index), charbox) def concat_strings(self, str1num, str2num): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT) str1box = self.decode_box(str1num, REF) str2box = self.decode_box(str2num, REF) return self.metainterp.execute_and_record_varargs( rop.CALL, [ConstInt(func), str1box, str2box], calldescr) def slice_string(self, strnum, startnum, lengthnum): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_SLICE) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_SLICE) + strbox = self.decode_box(strnum, REF) + startbox = self.decode_box(startnum, INT) + lengthbox = self.decode_box(lengthnum, INT) + stopbox = self.metainterp.execute_and_record(rop.INT_ADD, None, + startbox, lengthbox) + return self.metainterp.execute_and_record_varargs( + rop.CALL, [ConstInt(func), strbox, startbox, stopbox], calldescr) + + def allocate_unicode(self, length): + return self.metainterp.execute_and_record(rop.NEWUNICODE, + None, ConstInt(length)) + + def unicode_setitem(self, strbox, index, charnum): + charbox = self.decode_box(charnum, INT) + self.metainterp.execute_and_record(rop.UNICODESETITEM, None, + strbox, ConstInt(index), charbox) + + def concat_unicodes(self, str1num, str2num): + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT) + str1box = self.decode_box(str1num, REF) + str2box = self.decode_box(str2num, REF) + return self.metainterp.execute_and_record_varargs( + rop.CALL, [ConstInt(func), str1box, str2box], calldescr) + + def slice_unicode(self, strnum, startnum, lengthnum): + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE) strbox = self.decode_box(strnum, REF) startbox = self.decode_box(startnum, INT) lengthbox = self.decode_box(lengthnum, INT) @@ -812,11 +917,12 @@ def blackhole_from_resumedata(blackholeinterpbuilder, jitdriver_sd, storage, all_virtuals=None): - resumereader = ResumeDataDirectReader(blackholeinterpbuilder.cpu, storage, - all_virtuals) + resumereader = ResumeDataDirectReader(blackholeinterpbuilder.metainterp_sd, + storage, all_virtuals) vinfo = jitdriver_sd.virtualizable_info + ginfo = jitdriver_sd.greenfield_info vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info - resumereader.consume_vref_and_vable(vrefinfo, vinfo) + resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo) # # First get a chain of blackhole interpreters whose length is given # by the depth of rd_frame_info_list. The first one we get must be @@ -846,11 +952,11 @@ resumereader.done() return firstbh -def force_from_resumedata(metainterp_sd, storage, vinfo=None): - resumereader = ResumeDataDirectReader(metainterp_sd.cpu, storage) +def force_from_resumedata(metainterp_sd, storage, vinfo, ginfo): + resumereader = ResumeDataDirectReader(metainterp_sd, storage) resumereader.handling_async_forcing() vrefinfo = metainterp_sd.virtualref_info - resumereader.consume_vref_and_vable(vrefinfo, vinfo) + resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo) return resumereader.force_all_virtuals() class ResumeDataDirectReader(AbstractResumeDataReader): @@ -861,8 +967,9 @@ # 1: in handle_async_forcing # 2: resuming from the GUARD_NOT_FORCED - def __init__(self, cpu, storage, all_virtuals=None): - self._init(cpu, storage) + def __init__(self, metainterp_sd, storage, all_virtuals=None): + self._init(metainterp_sd.cpu, storage) + self.callinfocollection = metainterp_sd.callinfocollection if all_virtuals is None: # common case self._prepare(storage) else: @@ -881,23 +988,24 @@ info = blackholeinterp.get_current_position_info() self._prepare_next_section(info) - def consume_virtualref_info(self, vrefinfo, nums, end): + def consume_virtualref_info(self, vrefinfo, numb, end): # we have to decode a list of references containing pairs # [..., virtual, vref, ...] stopping at 'end' assert (end & 1) == 0 for i in range(0, end, 2): - virtual = self.decode_ref(nums[i]) - vref = self.decode_ref(nums[i+1]) + virtual = self.decode_ref(numb.nums[i]) + vref = self.decode_ref(numb.nums[i+1]) # For each pair, we store the virtual inside the vref. vrefinfo.continue_tracing(vref, virtual) - def consume_vable_info(self, vinfo, nums): + def consume_vable_info(self, vinfo, numb): # we have to ignore the initial part of 'nums' (containing vrefs), # find the virtualizable from nums[-1], load all other values # from the CPU stack, and copy them into the virtualizable if vinfo is None: - return len(nums) - virtualizable = self.decode_ref(nums[-1]) + return len(numb.nums) + index = len(numb.nums) - 1 + virtualizable = self.decode_ref(numb.nums[index]) virtualizable = vinfo.cast_gcref_to_vtype(virtualizable) if self.resume_after_guard_not_forced == 1: # in the middle of handle_async_forcing() @@ -909,7 +1017,7 @@ # is and stays 0. Note the call to reset_vable_token() in # warmstate.py. assert not virtualizable.vable_token - return vinfo.write_from_resume_data_partial(virtualizable, self, nums) + return vinfo.write_from_resume_data_partial(virtualizable, self, numb) def load_value_of_type(self, TYPE, tagged): from pypy.jit.metainterp.warmstate import specialize_value @@ -925,12 +1033,13 @@ return specialize_value(TYPE, x) load_value_of_type._annspecialcase_ = 'specialize:arg(1)' - def consume_vref_and_vable(self, vrefinfo, vinfo): - nums = self.cur_numb.nums - self.cur_numb = self.cur_numb.prev + def consume_vref_and_vable(self, vrefinfo, vinfo, ginfo): + numb = self.cur_numb + self.cur_numb = numb.prev if self.resume_after_guard_not_forced != 2: - end_vref = self.consume_vable_info(vinfo, nums) - self.consume_virtualref_info(vrefinfo, nums, end_vref) + end_vref = self.consume_vable_info(vinfo, numb) + if ginfo is not None: end_vref -= 1 + self.consume_virtualref_info(vrefinfo, numb, end_vref) def allocate_with_vtable(self, known_class): from pypy.jit.metainterp.executor import exec_new_with_vtable @@ -954,7 +1063,8 @@ str2 = self.decode_ref(str2num) str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str1) str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str2) - funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT) result = funcptr(str1, str2) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -963,7 +1073,35 @@ start = self.decode_int(startnum) length = self.decode_int(lengthnum) str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str) - funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_SLICE) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_SLICE) + result = funcptr(str, start, start + length) + return lltype.cast_opaque_ptr(llmemory.GCREF, result) + + def allocate_unicode(self, length): + return self.cpu.bh_newunicode(length) + + def unicode_setitem(self, str, index, charnum): + char = self.decode_int(charnum) + self.cpu.bh_unicodesetitem(str, index, char) + + def concat_unicodes(self, str1num, str2num): + str1 = self.decode_ref(str1num) + str2 = self.decode_ref(str2num) + str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str1) + str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str2) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT) + result = funcptr(str1, str2) + return lltype.cast_opaque_ptr(llmemory.GCREF, result) + + def slice_unicode(self, strnum, startnum, lengthnum): + str = self.decode_ref(strnum) + start = self.decode_int(startnum) + length = self.decode_int(lengthnum) + str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE) result = funcptr(str, start, start + length) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1054,8 +1192,9 @@ 'at', compute_unique_id(frameinfo)) frameinfo = frameinfo.prev numb = storage.rd_numb - while numb is not None: - debug_print('\tnumb', str([untag(i) for i in numb.nums]), + while numb: + debug_print('\tnumb', str([untag(numb.nums[i]) + for i in range(len(numb.nums))]), 'at', compute_unique_id(numb)) numb = numb.prev for const in storage.rd_consts: Modified: pypy/branch/arm-backend/pypy/jit/metainterp/simple_optimize.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/simple_optimize.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/simple_optimize.py Tue Nov 23 10:12:47 2010 @@ -9,11 +9,13 @@ def transform(op): from pypy.jit.metainterp.history import AbstractDescr - # Rename CALL_PURE to CALL. + # Rename CALL_PURE and CALL_LOOPINVARIANT to CALL. # Simplify the VIRTUAL_REF_* so that they don't show up in the backend. if op.getopnum() == rop.CALL_PURE: op = ResOperation(rop.CALL, op.getarglist()[1:], op.result, op.getdescr()) + elif op.getopnum() == rop.CALL_LOOPINVARIANT: + op = op.copy_and_change(rop.CALL) elif op.getopnum() == rop.VIRTUAL_REF: op = ResOperation(rop.SAME_AS, [op.getarg(0)], op.result) elif op.getopnum() == rop.VIRTUAL_REF_FINISH: Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_basic.py Tue Nov 23 10:12:47 2010 @@ -2,6 +2,8 @@ import sys from pypy.rlib.jit import JitDriver, we_are_jitted, hint, dont_look_inside from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE, loop_invariant +from pypy.rlib.jit import jit_debug, assert_green, AssertGreenFailed +from pypy.rlib.jit import unroll_safe, current_trace_length from pypy.jit.metainterp.warmspot import ll_meta_interp, get_stats from pypy.jit.backend.llgraph import runner from pypy.jit.metainterp import pyjitpl, history @@ -44,6 +46,7 @@ num_green_args = 0 portal_graph = graphs[0] virtualizable_info = None + greenfield_info = None result_type = result_kind portal_runner_ptr = "???" @@ -1644,6 +1647,58 @@ res = self.interp_operations(f, [10, 3.5]) assert res == 3.5 + def test_jit_debug(self): + myjitdriver = JitDriver(greens = [], reds = ['x']) + class A: + pass + def f(x): + while x > 0: + myjitdriver.can_enter_jit(x=x) + myjitdriver.jit_merge_point(x=x) + jit_debug("hi there:", x) + jit_debug("foobar") + x -= 1 + return x + res = self.meta_interp(f, [8]) + assert res == 0 + self.check_loops(jit_debug=2) + + def test_assert_green(self): + def f(x, promote): + if promote: + x = hint(x, promote=True) + assert_green(x) + return x + res = self.interp_operations(f, [8, 1]) + assert res == 8 + py.test.raises(AssertGreenFailed, self.interp_operations, f, [8, 0]) + + def test_current_trace_length(self): + myjitdriver = JitDriver(greens = ['g'], reds = ['x']) + @dont_look_inside + def residual(): + print "hi there" + @unroll_safe + def loop(g): + y = 0 + while y < g: + residual() + y += 1 + def f(x, g): + n = 0 + while x > 0: + myjitdriver.can_enter_jit(x=x, g=g) + myjitdriver.jit_merge_point(x=x, g=g) + loop(g) + x -= 1 + n = current_trace_length() + return n + res = self.meta_interp(f, [5, 8]) + assert 14 < res < 42 + res = self.meta_interp(f, [5, 2]) + assert 4 < res < 14 + + class TestOOtype(BasicTests, OOJitMixin): def test_oohash(self): @@ -1751,7 +1806,7 @@ c = bool(p1) d = not bool(p2) return 1000*a + 100*b + 10*c + d - prebuilt = [lltype.malloc(TP, flavor='raw')] * 2 + prebuilt = [lltype.malloc(TP, flavor='raw', immortal=True)] * 2 expected = f(0, 1) assert self.interp_operations(f, [0, 1]) == expected Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_compile.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_compile.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_compile.py Tue Nov 23 10:12:47 2010 @@ -6,8 +6,8 @@ from pypy.jit.metainterp.compile import ResumeGuardCountersInt from pypy.jit.metainterp.compile import compile_tmp_callback from pypy.jit.metainterp import optimize, jitprof, typesystem, compile -from pypy.jit.metainterp.test.oparser import parse from pypy.jit.metainterp.test.test_optimizefindnode import LLtypeMixin +from pypy.jit.tool.oparser import parse def test_insert_loop_token(): @@ -85,7 +85,7 @@ metainterp.history.inputargs = loop.inputargs[:] # loop_tokens = [] - loop_token = compile_new_loop(metainterp, loop_tokens, [], 0) + loop_token = compile_new_loop(metainterp, loop_tokens, 0) assert loop_tokens == [loop_token] assert loop_token.number == 1 assert staticdata.globaldata.loopnumbering == 2 @@ -101,7 +101,7 @@ metainterp.history.operations = loop.operations[:] metainterp.history.inputargs = loop.inputargs[:] # - loop_token_2 = compile_new_loop(metainterp, loop_tokens, [], 0) + loop_token_2 = compile_new_loop(metainterp, loop_tokens, 0) assert loop_token_2 is loop_token assert loop_tokens == [loop_token] assert len(cpu.seen) == 0 Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_del.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_del.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_del.py Tue Nov 23 10:12:47 2010 @@ -85,6 +85,7 @@ def test_signal_action(self): from pypy.module.signal.interp_signal import SignalActionFlag action = SignalActionFlag() + action.has_bytecode_counter = True # myjitdriver = JitDriver(greens = [], reds = ['n', 'x']) class X: @@ -92,17 +93,17 @@ # def f(n): x = X() - while n > 0: + action.reset_ticker(n) + while True: myjitdriver.can_enter_jit(n=n, x=x) myjitdriver.jit_merge_point(n=n, x=x) x.foo = n n -= 1 - if action.get() != 0: + if action.decrement_ticker(1) < 0: break - action.set(0) return 42 self.meta_interp(f, [20]) - self.check_loops(getfield_raw=1, call=0, call_pure=0) + self.check_loops(getfield_raw=1, setfield_raw=1, call=0, call_pure=0) class TestOOtype(DelTests, OOJitMixin): def setup_class(cls): Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_exception.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_exception.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_exception.py Tue Nov 23 10:12:47 2010 @@ -1,6 +1,6 @@ import py, sys from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin -from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE +from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE, dont_look_inside from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask from pypy.jit.codewriter.policy import StopAtXPolicy @@ -588,6 +588,33 @@ res = self.interp_operations(f, [99]) assert res == 21 + def test_bug_exc1_noexc_exc2(self): + myjitdriver = JitDriver(greens=[], reds=['i']) + @dont_look_inside + def rescall(i): + if i < 10: + raise KeyError + if i < 20: + return None + raise ValueError + def f(i): + while i < 30: + myjitdriver.can_enter_jit(i=i) + myjitdriver.jit_merge_point(i=i) + try: + rescall(i) + except KeyError: + assert i < 10 + except ValueError: + assert i >= 20 + else: + assert 10 <= i < 20 + i += 1 + return i + res = self.meta_interp(f, [0], inline=True) + assert res == 30 + + class MyError(Exception): def __init__(self, n): self.n = n Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_jitdriver.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_jitdriver.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_jitdriver.py Tue Nov 23 10:12:47 2010 @@ -72,6 +72,33 @@ # we expect no loop at all for 'loop1': it should always be inlined self.check_tree_loop_count(2) # 1 x loop, 1 x enter bridge + def test_inactive_jitdriver(self): + myjitdriver1 = JitDriver(greens=[], reds=['n', 'm'], + get_printable_location = getloc1) + myjitdriver2 = JitDriver(greens=['g'], reds=['r'], + get_printable_location = getloc2) + # + myjitdriver1.active = False # <=== + # + def loop1(n, m): + while n > 0: + myjitdriver1.can_enter_jit(n=n, m=m) + myjitdriver1.jit_merge_point(n=n, m=m) + n -= m + return n + # + def loop2(g, r): + while r > 0: + myjitdriver2.can_enter_jit(g=g, r=r) + myjitdriver2.jit_merge_point(g=g, r=r) + r += loop1(r, g) + (-1) + return r + # + res = self.meta_interp(loop2, [4, 40], repeat=7, inline=True) + assert res == loop2(4, 40) + # we expect no int_sub, but a residual call + self.check_loops(int_sub=0, call=1) + class TestLLtype(MultipleJitDriversTests, LLJitMixin): pass Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_logger.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_logger.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_logger.py Tue Nov 23 10:12:47 2010 @@ -1,6 +1,6 @@ import sys from pypy.rlib import debug -from pypy.jit.metainterp.test.oparser import pure_parse +from pypy.jit.tool.oparser import pure_parse from pypy.jit.metainterp import logger from pypy.jit.metainterp.typesystem import llhelper from StringIO import StringIO @@ -97,7 +97,7 @@ def test_debug_merge_point(self): inp = ''' [] - debug_merge_point("info") + debug_merge_point("info", 0) ''' loop, oloop = self.reparse(inp, check_equal=False) assert loop.operations[0].getarg(0)._get_str() == 'info' Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizefindnode.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizefindnode.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizefindnode.py Tue Nov 23 10:12:47 2010 @@ -18,7 +18,7 @@ from pypy.jit.metainterp.specnode import ConstantSpecNode from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.jit.codewriter.heaptracker import register_known_gctype, adr2int -from pypy.jit.metainterp.test.oparser import parse +from pypy.jit.tool.oparser import parse def test_sort_descrs(): class PseudoDescr(AbstractDescr): @@ -117,33 +117,32 @@ EffectInfo.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE)) arraycopydescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, EffectInfo([], [], [], oopspecindex=EffectInfo.OS_ARRAYCOPY)) - strconcatdescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, - EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_CONCAT)) - slicedescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, - EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_SLICE)) - strequaldescr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, - EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR_EQUAL)) - streq_slice_checknull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, - EffectInfo([], [], [], - oopspecindex=EffectInfo.OS_STREQ_SLICE_CHECKNULL)) - streq_slice_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, - EffectInfo([], [], [], - oopspecindex=EffectInfo.OS_STREQ_SLICE_NONNULL)) - streq_slice_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, - EffectInfo([], [], [], - oopspecindex=EffectInfo.OS_STREQ_SLICE_CHAR)) - streq_nonnull_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, - EffectInfo([], [], [], - oopspecindex=EffectInfo.OS_STREQ_NONNULL)) - streq_nonnull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, - EffectInfo([], [], [], - oopspecindex=EffectInfo.OS_STREQ_NONNULL_CHAR)) - streq_checknull_char_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, - EffectInfo([], [], [], - oopspecindex=EffectInfo.OS_STREQ_CHECKNULL_CHAR)) - streq_lengthok_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, - EffectInfo([], [], [], - oopspecindex=EffectInfo.OS_STREQ_LENGTHOK)) + + for _name, _os in [ + ('strconcatdescr', 'OS_STR_CONCAT'), + ('strslicedescr', 'OS_STR_SLICE'), + ('strequaldescr', 'OS_STR_EQUAL'), + ('streq_slice_checknull_descr', 'OS_STREQ_SLICE_CHECKNULL'), + ('streq_slice_nonnull_descr', 'OS_STREQ_SLICE_NONNULL'), + ('streq_slice_char_descr', 'OS_STREQ_SLICE_CHAR'), + ('streq_nonnull_descr', 'OS_STREQ_NONNULL'), + ('streq_nonnull_char_descr', 'OS_STREQ_NONNULL_CHAR'), + ('streq_checknull_char_descr', 'OS_STREQ_CHECKNULL_CHAR'), + ('streq_lengthok_descr', 'OS_STREQ_LENGTHOK'), + ]: + _oopspecindex = getattr(EffectInfo, _os) + locals()[_name] = \ + cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, + EffectInfo([], [], [], oopspecindex=_oopspecindex)) + # + _oopspecindex = getattr(EffectInfo, _os.replace('STR', 'UNI')) + locals()[_name.replace('str', 'unicode')] = \ + cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, + EffectInfo([], [], [], oopspecindex=_oopspecindex)) + + s2u_descr = cpu.calldescrof(FUNC, FUNC.ARGS, FUNC.RESULT, + EffectInfo([], [], [], oopspecindex=EffectInfo.OS_STR2UNICODE)) + # class LoopToken(AbstractDescr): pass Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_optimizeopt.py Tue Nov 23 10:12:47 2010 @@ -12,7 +12,7 @@ from pypy.jit.metainterp.jitprof import EmptyProfiler from pypy.jit.metainterp import executor, compile, resume, history from pypy.jit.metainterp.resoperation import rop, opname, ResOperation -from pypy.jit.metainterp.test.oparser import pure_parse +from pypy.jit.tool.oparser import pure_parse ##class FakeFrame(object): ## parent_resumedata_snapshot = None @@ -41,7 +41,7 @@ b1 = BoxInt() opt = optimizeopt.Optimizer(FakeMetaInterpStaticData(LLtypeMixin.cpu), None) - fdescr = ResumeGuardDescr(None, None) + fdescr = ResumeGuardDescr(None) op = ResOperation(rop.GUARD_TRUE, ['dummy'], None, descr=fdescr) # setup rd data fi0 = resume.FrameInfo(None, "code0", 11) @@ -51,12 +51,12 @@ # opt.store_final_boxes_in_guard(op) if op.getfailargs() == [b0, b1]: - assert fdescr.rd_numb.nums == [tag(1, TAGBOX)] - assert fdescr.rd_numb.prev.nums == [tag(0, TAGBOX)] + assert list(fdescr.rd_numb.nums) == [tag(1, TAGBOX)] + assert list(fdescr.rd_numb.prev.nums) == [tag(0, TAGBOX)] else: assert op.getfailargs() == [b1, b0] - assert fdescr.rd_numb.nums == [tag(0, TAGBOX)] - assert fdescr.rd_numb.prev.nums == [tag(1, TAGBOX)] + assert list(fdescr.rd_numb.nums) == [tag(0, TAGBOX)] + assert list(fdescr.rd_numb.prev.nums) == [tag(1, TAGBOX)] assert fdescr.rd_virtuals is None assert fdescr.rd_consts == [] @@ -132,14 +132,21 @@ # ____________________________________________________________ def equaloplists(oplist1, oplist2, strict_fail_args=True, remap={}): - print '-'*20, 'Comparing lists', '-'*20 + # try to use the full width of the terminal to display the list + # unfortunately, does not work with the default capture method of py.test + # (which is fd), you you need to use either -s or --capture=sys, else you + # get the standard 80 columns width + totwidth = py.io.get_terminal_width() + width = totwidth / 2 - 1 + print ' Comparing lists '.center(totwidth, '-') + print '%s| %s' % ('optimized'.center(width), 'expected'.center(width)) for op1, op2 in zip(oplist1, oplist2): txt1 = str(op1) txt2 = str(op2) while txt1 or txt2: - print '%-39s| %s' % (txt1[:39], txt2[:39]) - txt1 = txt1[39:] - txt2 = txt2[39:] + print '%s| %s' % (txt1[:width].ljust(width), txt2[:width]) + txt1 = txt1[width:] + txt2 = txt2[width:] assert op1.getopnum() == op2.getopnum() assert op1.numargs() == op2.numargs() for i in range(op1.numargs()): @@ -257,11 +264,17 @@ metainterp_sd = FakeMetaInterpStaticData(self.cpu) if hasattr(self, 'vrefinfo'): metainterp_sd.virtualref_info = self.vrefinfo + if hasattr(self, 'callinfocollection'): + metainterp_sd.callinfocollection = self.callinfocollection optimize_loop_1(metainterp_sd, loop) # expected = self.parse(optops) print '\n'.join([str(o) for o in loop.operations]) self.assert_equal(loop, expected) + return loop + + +class OptimizeOptTest(BaseTestOptimizeOpt): def test_simple(self): ops = """ @@ -776,8 +789,12 @@ i3 = call(i2, descr=nonwritedescr) jump(i1) # the exception is considered lost when we loop back """ + # note that 'guard_no_exception' at the very start must be kept + # around: bridges may start with one. (In case of loops we could + # remove it, but we probably don't care.) expected = """ [i] + guard_no_exception() [] i1 = int_add(i, 3) i2 = call(i1, descr=nonwritedescr) guard_no_exception() [i1, i2] @@ -796,10 +813,10 @@ guard_value(i2, 1) [] i3 = call_loopinvariant(1, i1, descr=nonwritedescr) guard_no_exception() [] - guard_value(i2, 1) [] + guard_value(i3, 1) [] i4 = call_loopinvariant(1, i1, descr=nonwritedescr) guard_no_exception() [] - guard_value(i2, 1) [] + guard_value(i4, 1) [] jump(i1) """ expected = """ @@ -1379,7 +1396,7 @@ ops = """ [p1] i1 = getfield_gc(p1, descr=valuedescr) - debug_merge_point(15) + debug_merge_point(15, 0) i2 = getfield_gc(p1, descr=valuedescr) escape(i1) escape(i2) @@ -1388,7 +1405,7 @@ expected = """ [p1] i1 = getfield_gc(p1, descr=valuedescr) - debug_merge_point(15) + debug_merge_point(15, 0) escape(i1) escape(i1) jump(p1) @@ -2643,7 +2660,7 @@ ''', rop.GUARD_TRUE) -class TestLLtype(BaseTestOptimizeOpt, LLtypeMixin): +class TestLLtype(OptimizeOptTest, LLtypeMixin): def test_residual_call_does_not_invalidate_caches(self): ops = """ @@ -3487,7 +3504,7 @@ i0 = strlen(p0) jump(p0) """ - self.optimize_loop(ops, 'Not', expected) + self.optimize_strunicode_loop(ops, 'Not', expected) def test_addsub_const(self): ops = """ @@ -3893,6 +3910,15 @@ """ self.optimize_loop(ops, 'Not, Not', expected) + # ---------- + def optimize_strunicode_loop(self, ops, spectext, optops): + # check with the arguments passed in + self.optimize_loop(ops, spectext, optops) + # check with replacing 'str' with 'unicode' everywhere + self.optimize_loop(ops.replace('str','unicode').replace('s"', 'u"'), + spectext, + optops.replace('str','unicode').replace('s"', 'u"')) + def test_newstr_1(self): ops = """ [i0] @@ -3905,7 +3931,7 @@ [i0] jump(i0) """ - self.optimize_loop(ops, 'Not', expected) + self.optimize_strunicode_loop(ops, 'Not', expected) def test_newstr_2(self): ops = """ @@ -3921,7 +3947,7 @@ [i0, i1] jump(i1, i0) """ - self.optimize_loop(ops, 'Not, Not', expected) + self.optimize_strunicode_loop(ops, 'Not, Not', expected) def test_str_concat_1(self): ops = """ @@ -3942,7 +3968,7 @@ copystrcontent(p2, p3, 0, i4, i5) jump(p2, p3) """ - self.optimize_loop(ops, 'Not, Not', expected) + self.optimize_strunicode_loop(ops, 'Not, Not', expected) def test_str_concat_vstr2_str(self): ops = """ @@ -3965,7 +3991,7 @@ copystrcontent(p2, p3, 0, 2, i4) jump(i1, i0, p3) """ - self.optimize_loop(ops, 'Not, Not, Not', expected) + self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected) def test_str_concat_str_vstr2(self): ops = """ @@ -3989,7 +4015,7 @@ i6 = int_add(i5, 1) # will be killed by the backend jump(i1, i0, p3) """ - self.optimize_loop(ops, 'Not, Not, Not', expected) + self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected) def test_str_concat_str_str_str(self): ops = """ @@ -4016,12 +4042,12 @@ copystrcontent(p3, p5, 0, i12b, i3b) jump(p2, p3, p5) """ - self.optimize_loop(ops, 'Not, Not, Not', expected) + self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected) def test_str_concat_str_cstr1(self): ops = """ [p2] - p3 = call(0, p2, "x", descr=strconcatdescr) + p3 = call(0, p2, s"x", descr=strconcatdescr) jump(p3) """ expected = """ @@ -4035,28 +4061,28 @@ i5 = int_add(i4, 1) # will be killed by the backend jump(p3) """ - self.optimize_loop(ops, 'Not', expected) + self.optimize_strunicode_loop(ops, 'Not', expected) def test_str_concat_consts(self): ops = """ [] - p1 = same_as("ab") - p2 = same_as("cde") + p1 = same_as(s"ab") + p2 = same_as(s"cde") p3 = call(0, p1, p2, descr=strconcatdescr) escape(p3) jump() """ expected = """ [] - escape("abcde") + escape(s"abcde") jump() """ - self.optimize_loop(ops, '', expected) + self.optimize_strunicode_loop(ops, '', expected) def test_str_slice_1(self): ops = """ [p1, i1, i2] - p2 = call(0, p1, i1, i2, descr=slicedescr) + p2 = call(0, p1, i1, i2, descr=strslicedescr) jump(p2, i1, i2) """ expected = """ @@ -4066,12 +4092,12 @@ copystrcontent(p1, p2, i1, 0, i3) jump(p2, i1, i2) """ - self.optimize_loop(ops, 'Not, Not, Not', expected) + self.optimize_strunicode_loop(ops, 'Not, Not, Not', expected) def test_str_slice_2(self): ops = """ [p1, i2] - p2 = call(0, p1, 0, i2, descr=slicedescr) + p2 = call(0, p1, 0, i2, descr=strslicedescr) jump(p2, i2) """ expected = """ @@ -4080,13 +4106,13 @@ copystrcontent(p1, p2, 0, 0, i2) jump(p2, i2) """ - self.optimize_loop(ops, 'Not, Not', expected) + self.optimize_strunicode_loop(ops, 'Not, Not', expected) def test_str_slice_3(self): ops = """ [p1, i1, i2, i3, i4] - p2 = call(0, p1, i1, i2, descr=slicedescr) - p3 = call(0, p2, i3, i4, descr=slicedescr) + p2 = call(0, p1, i1, i2, descr=strslicedescr) + p3 = call(0, p2, i3, i4, descr=strslicedescr) jump(p3, i1, i2, i3, i4) """ expected = """ @@ -4098,12 +4124,12 @@ copystrcontent(p1, p3, i6, 0, i5) jump(p3, i1, i2, i3, i4) """ - self.optimize_loop(ops, 'Not, Not, Not, Not, Not', expected) + self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not, Not', expected) def test_str_slice_getitem1(self): ops = """ [p1, i1, i2, i3] - p2 = call(0, p1, i1, i2, descr=slicedescr) + p2 = call(0, p1, i1, i2, descr=strslicedescr) i4 = strgetitem(p2, i3) escape(i4) jump(p1, i1, i2, i3) @@ -4116,7 +4142,7 @@ escape(i4) jump(p1, i1, i2, i3) """ - self.optimize_loop(ops, 'Not, Not, Not, Not', expected) + self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected) def test_str_slice_plain(self): ops = """ @@ -4124,7 +4150,7 @@ p1 = newstr(2) strsetitem(p1, 0, i3) strsetitem(p1, 1, i4) - p2 = call(0, p1, 1, 2, descr=slicedescr) + p2 = call(0, p1, 1, 2, descr=strslicedescr) i5 = strgetitem(p2, 0) escape(i5) jump(i3, i4) @@ -4134,12 +4160,12 @@ escape(i4) jump(i3, i4) """ - self.optimize_loop(ops, 'Not, Not', expected) + self.optimize_strunicode_loop(ops, 'Not, Not', expected) def test_str_slice_concat(self): ops = """ [p1, i1, i2, p2] - p3 = call(0, p1, i1, i2, descr=slicedescr) + p3 = call(0, p1, i1, i2, descr=strslicedescr) p4 = call(0, p3, p2, descr=strconcatdescr) jump(p4, i1, i2, p2) """ @@ -4155,27 +4181,25 @@ copystrcontent(p2, p4, 0, i3, i4b) jump(p4, i1, i2, p2) """ - self.optimize_loop(ops, 'Not, Not, Not, Not', expected) + self.optimize_strunicode_loop(ops, 'Not, Not, Not, Not', expected) # ---------- - def optimize_loop_extradescrs(self, ops, spectext, optops): + def optimize_strunicode_loop_extradescrs(self, ops, spectext, optops): from pypy.jit.metainterp.optimizeopt import string - def my_callinfo_for_oopspec(oopspecindex): - calldescrtype = type(LLtypeMixin.strequaldescr) - for value in LLtypeMixin.__dict__.values(): - if isinstance(value, calldescrtype): - if (value.get_extra_info() and - value.get_extra_info().oopspecindex == oopspecindex): - # returns 0 for 'func' in this test - return value, 0 - raise AssertionError("not found: oopspecindex=%d" % oopspecindex) + class FakeCallInfoCollection: + def callinfo_for_oopspec(self, oopspecindex): + calldescrtype = type(LLtypeMixin.strequaldescr) + for value in LLtypeMixin.__dict__.values(): + if isinstance(value, calldescrtype): + extra = value.get_extra_info() + if extra and extra.oopspecindex == oopspecindex: + # returns 0 for 'func' in this test + return value, 0 + raise AssertionError("not found: oopspecindex=%d" % + oopspecindex) # - saved = string.callinfo_for_oopspec - try: - string.callinfo_for_oopspec = my_callinfo_for_oopspec - self.optimize_loop(ops, spectext, optops) - finally: - string.callinfo_for_oopspec = saved + self.callinfocollection = FakeCallInfoCollection() + self.optimize_strunicode_loop(ops, spectext, optops) def test_str_equal_noop1(self): ops = """ @@ -4184,7 +4208,7 @@ escape(i0) jump(p1, p2) """ - self.optimize_loop_extradescrs(ops, 'Not, Not', ops) + self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', ops) def test_str_equal_noop2(self): ops = """ @@ -4209,12 +4233,13 @@ escape(i0) jump(p1, p2, p3) """ - self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not', + expected) def test_str_equal_slice1(self): ops = """ [p1, i1, i2, p3] - p4 = call(0, p1, i1, i2, descr=slicedescr) + p4 = call(0, p1, i1, i2, descr=strslicedescr) i0 = call(0, p4, p3, descr=strequaldescr) escape(i0) jump(p1, i1, i2, p3) @@ -4226,12 +4251,13 @@ escape(i0) jump(p1, i1, i2, p3) """ - self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not', + expected) def test_str_equal_slice2(self): ops = """ [p1, i1, i2, p3] - p4 = call(0, p1, i1, i2, descr=slicedescr) + p4 = call(0, p1, i1, i2, descr=strslicedescr) i0 = call(0, p3, p4, descr=strequaldescr) escape(i0) jump(p1, i1, i2, p3) @@ -4243,13 +4269,14 @@ escape(i0) jump(p1, i1, i2, p3) """ - self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not', + expected) def test_str_equal_slice3(self): ops = """ [p1, i1, i2, p3] guard_nonnull(p3) [] - p4 = call(0, p1, i1, i2, descr=slicedescr) + p4 = call(0, p1, i1, i2, descr=strslicedescr) i0 = call(0, p3, p4, descr=strequaldescr) escape(i0) jump(p1, i1, i2, p3) @@ -4262,13 +4289,14 @@ escape(i0) jump(p1, i1, i2, p3) """ - self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not', + expected) def test_str_equal_slice4(self): ops = """ [p1, i1, i2] - p3 = call(0, p1, i1, i2, descr=slicedescr) - i0 = call(0, p3, "x", descr=strequaldescr) + p3 = call(0, p1, i1, i2, descr=strslicedescr) + i0 = call(0, p3, s"x", descr=strequaldescr) escape(i0) jump(p1, i1, i2) """ @@ -4279,12 +4307,13 @@ escape(i0) jump(p1, i1, i2) """ - self.optimize_loop_extradescrs(ops, 'Not, Not, Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not', + expected) def test_str_equal_slice5(self): ops = """ [p1, i1, i2, i3] - p4 = call(0, p1, i1, i2, descr=slicedescr) + p4 = call(0, p1, i1, i2, descr=strslicedescr) p5 = newstr(1) strsetitem(p5, 0, i3) i0 = call(0, p5, p4, descr=strequaldescr) @@ -4298,7 +4327,8 @@ escape(i0) jump(p1, i1, i2, i3) """ - self.optimize_loop_extradescrs(ops, 'Not, Not, Not, Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not, Not, Not', + expected) def test_str_equal_none1(self): ops = """ @@ -4313,7 +4343,7 @@ escape(i0) jump(p1) """ - self.optimize_loop_extradescrs(ops, 'Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected) def test_str_equal_none2(self): ops = """ @@ -4328,30 +4358,30 @@ escape(i0) jump(p1) """ - self.optimize_loop_extradescrs(ops, 'Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected) def test_str_equal_nonnull1(self): ops = """ [p1] guard_nonnull(p1) [] - i0 = call(0, p1, "hello world", descr=strequaldescr) + i0 = call(0, p1, s"hello world", descr=strequaldescr) escape(i0) jump(p1) """ expected = """ [p1] guard_nonnull(p1) [] - i0 = call(0, p1, "hello world", descr=streq_nonnull_descr) + i0 = call(0, p1, s"hello world", descr=streq_nonnull_descr) escape(i0) jump(p1) """ - self.optimize_loop_extradescrs(ops, 'Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected) def test_str_equal_nonnull2(self): ops = """ [p1] guard_nonnull(p1) [] - i0 = call(0, p1, "", descr=strequaldescr) + i0 = call(0, p1, s"", descr=strequaldescr) escape(i0) jump(p1) """ @@ -4363,13 +4393,13 @@ escape(i0) jump(p1) """ - self.optimize_loop_extradescrs(ops, 'Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected) def test_str_equal_nonnull3(self): ops = """ [p1] guard_nonnull(p1) [] - i0 = call(0, p1, "x", descr=strequaldescr) + i0 = call(0, p1, s"x", descr=strequaldescr) escape(i0) jump(p1) """ @@ -4380,13 +4410,13 @@ escape(i0) jump(p1) """ - self.optimize_loop_extradescrs(ops, 'Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected) def test_str_equal_nonnull4(self): ops = """ [p1, p2] p4 = call(0, p1, p2, descr=strconcatdescr) - i0 = call(0, "hello world", p4, descr=strequaldescr) + i0 = call(0, s"hello world", p4, descr=strequaldescr) escape(i0) jump(p1, p2) """ @@ -4401,17 +4431,17 @@ i5 = strlen(p2) i6 = int_add(i4, i5) # will be killed by the backend copystrcontent(p2, p4, 0, i4, i5) - i0 = call(0, "hello world", p4, descr=streq_nonnull_descr) + i0 = call(0, s"hello world", p4, descr=streq_nonnull_descr) escape(i0) jump(p1, p2) """ - self.optimize_loop_extradescrs(ops, 'Not, Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected) def test_str_equal_chars0(self): ops = """ [i1] p1 = newstr(0) - i0 = call(0, p1, "", descr=strequaldescr) + i0 = call(0, p1, s"", descr=strequaldescr) escape(i0) jump(i1) """ @@ -4420,14 +4450,14 @@ escape(1) jump(i1) """ - self.optimize_loop_extradescrs(ops, 'Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected) def test_str_equal_chars1(self): ops = """ [i1] p1 = newstr(1) strsetitem(p1, 0, i1) - i0 = call(0, p1, "x", descr=strequaldescr) + i0 = call(0, p1, s"x", descr=strequaldescr) escape(i0) jump(i1) """ @@ -4437,7 +4467,7 @@ escape(i0) jump(i1) """ - self.optimize_loop_extradescrs(ops, 'Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected) def test_str_equal_chars2(self): ops = """ @@ -4445,7 +4475,7 @@ p1 = newstr(2) strsetitem(p1, 0, i1) strsetitem(p1, 1, i2) - i0 = call(0, p1, "xy", descr=strequaldescr) + i0 = call(0, p1, s"xy", descr=strequaldescr) escape(i0) jump(i1, i2) """ @@ -4454,16 +4484,16 @@ p1 = newstr(2) strsetitem(p1, 0, i1) strsetitem(p1, 1, i2) - i0 = call(0, p1, "xy", descr=streq_lengthok_descr) + i0 = call(0, p1, s"xy", descr=streq_lengthok_descr) escape(i0) jump(i1, i2) """ - self.optimize_loop_extradescrs(ops, 'Not, Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not, Not', expected) def test_str_equal_chars3(self): ops = """ [p1] - i0 = call(0, "x", p1, descr=strequaldescr) + i0 = call(0, s"x", p1, descr=strequaldescr) escape(i0) jump(p1) """ @@ -4473,14 +4503,14 @@ escape(i0) jump(p1) """ - self.optimize_loop_extradescrs(ops, 'Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected) def test_str_equal_lengthmismatch1(self): ops = """ [i1] p1 = newstr(1) strsetitem(p1, 0, i1) - i0 = call(0, "xy", p1, descr=strequaldescr) + i0 = call(0, s"xy", p1, descr=strequaldescr) escape(i0) jump(i1) """ @@ -4489,13 +4519,36 @@ escape(0) jump(i1) """ - self.optimize_loop_extradescrs(ops, 'Not', expected) + self.optimize_strunicode_loop_extradescrs(ops, 'Not', expected) + + def test_str2unicode_constant(self): + ops = """ + [] + p0 = call(0, "xy", descr=s2u_descr) # string -> unicode + escape(p0) + jump() + """ + expected = """ + [] + escape(u"xy") + jump() + """ + self.optimize_strunicode_loop_extradescrs(ops, '', expected) - # XXX unicode operations - # XXX str2unicode + def test_str2unicode_nonconstant(self): + ops = """ + [p0] + p1 = call(0, p0, descr=s2u_descr) # string -> unicode + escape(p1) + jump(p1) + """ + self.optimize_strunicode_loop_extradescrs(ops, 'Not', ops) + # more generally, supporting non-constant but virtual cases is + # not obvious, because of the exception UnicodeDecodeError that + # can be raised by ll_str2unicode() -##class TestOOtype(BaseTestOptimizeOpt, OOtypeMixin): +##class TestOOtype(OptimizeOptTest, OOtypeMixin): ## def test_instanceof(self): ## ops = """ Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_recursive.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_recursive.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_recursive.py Tue Nov 23 10:12:47 2010 @@ -1142,6 +1142,19 @@ res = self.meta_interp(main, [], inline=True, trace_limit=tlimit) assert ''.join(res.chars) == 'ABCDEFGHIabcdefghijJ' * 5 + def test_no_duplicates_bug(self): + driver = JitDriver(greens = ['codeno'], reds = ['i'], + get_printable_location = lambda codeno: str(codeno)) + def portal(codeno, i): + while i > 0: + driver.can_enter_jit(codeno=codeno, i=i) + driver.jit_merge_point(codeno=codeno, i=i) + if codeno > 0: + break + portal(i, i) + i -= 1 + self.meta_interp(portal, [0, 10], inline=True) + class TestLLtype(RecursiveTests, LLJitMixin): pass Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resoperation.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resoperation.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resoperation.py Tue Nov 23 10:12:47 2010 @@ -61,3 +61,10 @@ assert op.getarglist() == ['a', 'b'] assert op.result == 'c' assert op.getdescr() is mydescr + +def test_can_malloc(): + mydescr = AbstractDescr() + assert rop.ResOperation(rop.rop.NEW, [], 'b').can_malloc() + call = rop.ResOperation(rop.rop.CALL, ['a', 'b'], 'c', descr=mydescr) + assert call.can_malloc() + assert not rop.ResOperation(rop.rop.INT_ADD, ['a', 'b'], 'c').can_malloc() Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resume.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resume.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_resume.py Tue Nov 23 10:12:47 2010 @@ -51,6 +51,7 @@ class MyMetaInterp: _already_allocated_resume_virtuals = None + callinfocollection = None def __init__(self, cpu=None): if cpu is None: @@ -141,6 +142,13 @@ assert bh.written_f == expected_f +def Numbering(prev, nums): + numb = lltype.malloc(NUMBERING, len(nums)) + numb.prev = prev or lltype.nullptr(NUMBERING) + for i in range(len(nums)): + numb.nums[i] = nums[i] + return numb + def test_simple_read(): #b1, b2, b3 = [BoxInt(), BoxPtr(), BoxInt()] c1, c2, c3 = [ConstInt(111), ConstInt(222), ConstInt(333)] @@ -156,12 +164,12 @@ storage.rd_numb = numb # cpu = MyCPU([42, gcref1, -66]) - reader = ResumeDataDirectReader(cpu, storage) + metainterp = MyMetaInterp(cpu) + reader = ResumeDataDirectReader(metainterp, storage) _next_section(reader, 42, 111, gcrefnull, 42, gcref1) _next_section(reader, 222, 333) _next_section(reader, 42, gcref1, -66) # - metainterp = MyMetaInterp(cpu) reader = ResumeDataBoxReader(storage, metainterp) bi, br, bf = [None]*3, [None]*2, [None]*0 info = MyBlackholeInterp([lltype.Signed, lltype.Signed, @@ -193,7 +201,7 @@ storage.rd_numb = numb # cpu = MyCPU([]) - reader = ResumeDataDirectReader(cpu, storage) + reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage) _next_section(reader, 100) @@ -211,7 +219,7 @@ class FakeMetainterp(object): _already_allocated_resume_virtuals = None cpu = None - reader = ResumeDataDirectReader(None, FakeStorage()) + reader = ResumeDataDirectReader(MyMetaInterp(None), FakeStorage()) assert reader.force_all_virtuals() == ["allocated", reader.virtual_default] # ____________________________________________________________ @@ -240,6 +248,17 @@ return FakeBuiltObject(strconcat=[left, right]) def slice_string(self, str, start, length): return FakeBuiltObject(strslice=[str, start, length]) + def allocate_unicode(self, length): + return FakeBuiltObject(unistring=[None]*length) + def unicode_setitem(self, unistring, i, fieldnum): + value, tag = untag(fieldnum) + assert tag == TAGINT + assert 0 <= i < len(unistring.unistring) + unistring.unistring[i] = value + def concat_unicodes(self, left, right): + return FakeBuiltObject(uniconcat=[left, right]) + def slice_unicode(self, str, start, length): + return FakeBuiltObject(unislice=[str, start, length]) class FakeBuiltObject(object): def __init__(self, **kwds): @@ -304,6 +323,30 @@ assert reader.force_all_virtuals() == [ FakeBuiltObject(strslice=info.fieldnums)] +def test_vuniplaininfo(): + info = VUniPlainInfo() + info.fieldnums = [tag(60, TAGINT)] + reader = FakeResumeDataReader() + reader._prepare_virtuals([info]) + assert reader.force_all_virtuals() == [ + FakeBuiltObject(unistring=[60])] + +def test_vuniconcatinfo(): + info = VUniConcatInfo() + info.fieldnums = [tag(10, TAGBOX), tag(20, TAGBOX)] + reader = FakeResumeDataReader() + reader._prepare_virtuals([info]) + assert reader.force_all_virtuals() == [ + FakeBuiltObject(uniconcat=info.fieldnums)] + +def test_vunisliceinfo(): + info = VUniSliceInfo() + info.fieldnums = [tag(10, TAGBOX), tag(20, TAGBOX), tag(30, TAGBOX)] + reader = FakeResumeDataReader() + reader._prepare_virtuals([info]) + assert reader.force_all_virtuals() == [ + FakeBuiltObject(unislice=info.fieldnums)] + # ____________________________________________________________ @@ -355,15 +398,15 @@ assert fi1.pc == 3 def test_Numbering_create(): - l = [1, 2] + l = [rffi.r_short(1), rffi.r_short(2)] numb = Numbering(None, l) - assert numb.prev is None - assert numb.nums is l + assert not numb.prev + assert list(numb.nums) == l - l1 = ['b3'] + l1 = [rffi.r_short(3)] numb1 = Numbering(numb, l1) - assert numb1.prev is numb - assert numb1.nums is l1 + assert numb1.prev == numb + assert list(numb1.nums) == l1 def test_capture_resumedata(): b1, b2, b3 = [BoxInt(), BoxPtr(), BoxInt()] @@ -729,11 +772,12 @@ assert liveboxes == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b3: tag(2, TAGBOX)} - assert numb.nums == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), - tag(1, TAGINT)] - assert numb.prev.nums == [tag(0, TAGBOX), tag(1, TAGINT), tag(1, TAGBOX), - tag(0, TAGBOX), tag(2, TAGINT)] - assert numb.prev.prev is None + assert list(numb.nums) == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), + tag(1, TAGINT)] + assert list(numb.prev.nums) == [tag(0, TAGBOX), tag(1, TAGINT), + tag(1, TAGBOX), + tag(0, TAGBOX), tag(2, TAGINT)] + assert not numb.prev.prev numb2, liveboxes2, v = memo.number({}, snap2) assert v == 0 @@ -741,9 +785,9 @@ assert liveboxes2 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b3: tag(2, TAGBOX)} assert liveboxes2 is not liveboxes - assert numb2.nums == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb2.prev is numb.prev + assert list(numb2.nums) == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), + tag(3, TAGINT)] + assert numb2.prev == numb.prev env3 = [c3, b3, b1, c3] snap3 = Snapshot(snap, env3) @@ -764,9 +808,9 @@ assert v == 0 assert liveboxes3 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX)} - assert numb3.nums == [tag(3, TAGINT), tag(4, TAGINT), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb3.prev is numb.prev + assert list(numb3.nums) == [tag(3, TAGINT), tag(4, TAGINT), tag(0, TAGBOX), + tag(3, TAGINT)] + assert numb3.prev == numb.prev # virtual env4 = [c3, b4, b1, c3] @@ -777,9 +821,9 @@ assert liveboxes4 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b4: tag(0, TAGVIRTUAL)} - assert numb4.nums == [tag(3, TAGINT), tag(0, TAGVIRTUAL), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb4.prev is numb.prev + assert list(numb4.nums) == [tag(3, TAGINT), tag(0, TAGVIRTUAL), + tag(0, TAGBOX), tag(3, TAGINT)] + assert numb4.prev == numb.prev env5 = [b1, b4, b5] snap5 = Snapshot(snap4, env5) @@ -790,9 +834,9 @@ assert liveboxes5 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b4: tag(0, TAGVIRTUAL), b5: tag(1, TAGVIRTUAL)} - assert numb5.nums == [tag(0, TAGBOX), tag(0, TAGVIRTUAL), - tag(1, TAGVIRTUAL)] - assert numb5.prev is numb4 + assert list(numb5.nums) == [tag(0, TAGBOX), tag(0, TAGVIRTUAL), + tag(1, TAGVIRTUAL)] + assert numb5.prev == numb4 def test_ResumeDataLoopMemo_number_boxes(): memo = ResumeDataLoopMemo(FakeMetaInterpStaticData()) @@ -890,7 +934,7 @@ liveboxes = modifier.finish({}) assert storage.rd_snapshot is None cpu = MyCPU([]) - reader = ResumeDataDirectReader(cpu, storage) + reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage) _next_section(reader, sys.maxint, 2**16, -65) _next_section(reader, 2, 3) _next_section(reader, sys.maxint, 1, sys.maxint, 2**16) Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_string.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_string.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_string.py Tue Nov 23 10:12:47 2010 @@ -6,14 +6,17 @@ class StringTests: + _str, _chr = str, chr + def test_eq_residual(self): + _str = self._str jitdriver = JitDriver(greens = [], reds = ['n', 'i', 's']) - global_s = "hello" + global_s = _str("hello") def f(n, b, s): if b: - s += "ello" + s += _str("ello") else: - s += "allo" + s += _str("allo") i = 0 while n > 0: jitdriver.can_enter_jit(s=s, n=n, i=i) @@ -21,18 +24,19 @@ n -= 1 + (s == global_s) i += 1 return i - res = self.meta_interp(f, [10, True, 'h'], listops=True) + res = self.meta_interp(f, [10, True, _str('h')], listops=True) assert res == 5 self.check_loops(**{self.CALL: 1, self.CALL_PURE: 0}) def test_eq_folded(self): + _str = self._str jitdriver = JitDriver(greens = ['s'], reds = ['n', 'i']) - global_s = "hello" + global_s = _str("hello") def f(n, b, s): if b: - s += "ello" + s += _str("ello") else: - s += "allo" + s += _str("allo") i = 0 while n > 0: jitdriver.can_enter_jit(s=s, n=n, i=i) @@ -40,31 +44,18 @@ n -= 1 + (s == global_s) i += 1 return i - res = self.meta_interp(f, [10, True, 'h'], listops=True) + res = self.meta_interp(f, [10, True, _str('h')], listops=True) assert res == 5 self.check_loops(**{self.CALL: 0, self.CALL_PURE: 0}) def test_newstr(self): + _str, _chr = self._str, self._chr jitdriver = JitDriver(greens = [], reds = ['n', 'm']) def f(n, m): while True: jitdriver.can_enter_jit(m=m, n=n) jitdriver.jit_merge_point(m=m, n=n) - bytecode = 'adlfkj' + chr(n) - res = bytecode[n] - m -= 1 - if m < 0: - return ord(res) - res = self.meta_interp(f, [6, 10]) - assert res == 6 - - def test_newunicode(self): - jitdriver = JitDriver(greens = [], reds = ['n', 'm']) - def f(n, m): - while True: - jitdriver.can_enter_jit(m=m, n=n) - jitdriver.jit_merge_point(m=m, n=n) - bytecode = u'adlfkj' + unichr(n) + bytecode = _str('adlfkj') + _chr(n) res = bytecode[n] m -= 1 if m < 0: @@ -73,95 +64,96 @@ assert res == 6 def test_char2string_pure(self): - for dochr in [chr, ]: #unichr]: - jitdriver = JitDriver(greens = [], reds = ['n']) - @dont_look_inside - def escape(x): - pass - def f(n): - while n > 0: - jitdriver.can_enter_jit(n=n) - jitdriver.jit_merge_point(n=n) - s = dochr(n) - if not we_are_jitted(): - s += s # forces to be a string - if n > 100: - escape(s) - n -= 1 - return 42 - self.meta_interp(f, [6]) - self.check_loops(newstr=0, strsetitem=0, strlen=0, - newunicode=0, unicodesetitem=0, unicodelen=0) + _str, _chr = self._str, self._chr + jitdriver = JitDriver(greens = [], reds = ['n']) + @dont_look_inside + def escape(x): + pass + def f(n): + while n > 0: + jitdriver.can_enter_jit(n=n) + jitdriver.jit_merge_point(n=n) + s = _chr(n) + if not we_are_jitted(): + s += s # forces to be a string + if n > 100: + escape(s) + n -= 1 + return 42 + self.meta_interp(f, [6]) + self.check_loops(newstr=0, strsetitem=0, strlen=0, + newunicode=0, unicodesetitem=0, unicodelen=0) def test_char2string_escape(self): - for dochr in [chr, ]: #unichr]: - jitdriver = JitDriver(greens = [], reds = ['n', 'total']) - @dont_look_inside - def escape(x): - return ord(x[0]) - def f(n): - total = 0 - while n > 0: - jitdriver.can_enter_jit(n=n, total=total) - jitdriver.jit_merge_point(n=n, total=total) - s = dochr(n) - if not we_are_jitted(): - s += s # forces to be a string - total += escape(s) - n -= 1 - return total - res = self.meta_interp(f, [6]) - assert res == 21 + _str, _chr = self._str, self._chr + jitdriver = JitDriver(greens = [], reds = ['n', 'total']) + @dont_look_inside + def escape(x): + return ord(x[0]) + def f(n): + total = 0 + while n > 0: + jitdriver.can_enter_jit(n=n, total=total) + jitdriver.jit_merge_point(n=n, total=total) + s = _chr(n) + if not we_are_jitted(): + s += s # forces to be a string + total += escape(s) + n -= 1 + return total + res = self.meta_interp(f, [6]) + assert res == 21 def test_char2string2char(self): - for dochr in [chr, ]: #unichr]: - jitdriver = JitDriver(greens = [], reds = ['m', 'total']) - def f(m): - total = 0 - while m > 0: - jitdriver.can_enter_jit(m=m, total=total) - jitdriver.jit_merge_point(m=m, total=total) - string = dochr(m) - if m > 100: - string += string # forces to be a string - # read back the character - c = string[0] - total += ord(c) - m -= 1 - return total - res = self.meta_interp(f, [6]) - assert res == 21 - self.check_loops(newstr=0, strgetitem=0, strsetitem=0, strlen=0, - newunicode=0, unicodegetitem=0, unicodesetitem=0, - unicodelen=0) + _str, _chr = self._str, self._chr + jitdriver = JitDriver(greens = [], reds = ['m', 'total']) + def f(m): + total = 0 + while m > 0: + jitdriver.can_enter_jit(m=m, total=total) + jitdriver.jit_merge_point(m=m, total=total) + string = _chr(m) + if m > 100: + string += string # forces to be a string + # read back the character + c = string[0] + total += ord(c) + m -= 1 + return total + res = self.meta_interp(f, [6]) + assert res == 21 + self.check_loops(newstr=0, strgetitem=0, strsetitem=0, strlen=0, + newunicode=0, unicodegetitem=0, unicodesetitem=0, + unicodelen=0) def test_strconcat_pure(self): - for somestr in ["abc", ]: #u"def"]: - jitdriver = JitDriver(greens = [], reds = ['m', 'n']) - @dont_look_inside - def escape(x): - pass - mylist = [somestr+str(i) for i in range(10)] - def f(n, m): - while m >= 0: - jitdriver.can_enter_jit(m=m, n=n) - jitdriver.jit_merge_point(m=m, n=n) - s = mylist[n] + mylist[m] - if m > 100: - escape(s) - m -= 1 - return 42 - self.meta_interp(f, [6, 7]) - self.check_loops(newstr=0, strsetitem=0, - newunicode=0, unicodesetitem=0, - call=0, call_pure=0) + _str = self._str + jitdriver = JitDriver(greens = [], reds = ['m', 'n']) + @dont_look_inside + def escape(x): + pass + mylist = [_str("abc") + _str(i) for i in range(10)] + def f(n, m): + while m >= 0: + jitdriver.can_enter_jit(m=m, n=n) + jitdriver.jit_merge_point(m=m, n=n) + s = mylist[n] + mylist[m] + if m > 100: + escape(s) + m -= 1 + return 42 + self.meta_interp(f, [6, 7]) + self.check_loops(newstr=0, strsetitem=0, + newunicode=0, unicodesetitem=0, + call=0, call_pure=0) def test_strconcat_escape_str_str(self): + _str = self._str jitdriver = JitDriver(greens = [], reds = ['m', 'n']) @dont_look_inside def escape(x): pass - mylist = ["somestr"+str(i) for i in range(10)] + mylist = [_str("somestr") + _str(i) for i in range(10)] def f(n, m): while m >= 0: jitdriver.can_enter_jit(m=m, n=n) @@ -171,46 +163,64 @@ m -= 1 return 42 self.meta_interp(f, [6, 7]) - self.check_loops(newstr=1, strsetitem=0, copystrcontent=2, - call=1, call_pure=0) # escape + if _str is str: + self.check_loops(newstr=1, strsetitem=0, copystrcontent=2, + call=1, call_pure=0) # escape + else: + self.check_loops(newunicode=1, unicodesetitem=0, + copyunicodecontent=2, + call=1, call_pure=0) # escape def test_strconcat_escape_str_char(self): + _str, _chr = self._str, self._chr jitdriver = JitDriver(greens = [], reds = ['m', 'n']) @dont_look_inside def escape(x): pass - mylist = ["somestr"+str(i) for i in range(10)] + mylist = [_str("somestr") + _str(i) for i in range(10)] def f(n, m): while m >= 0: jitdriver.can_enter_jit(m=m, n=n) jitdriver.jit_merge_point(m=m, n=n) - s = mylist[n] + chr(m) + s = mylist[n] + _chr(m) escape(s) m -= 1 return 42 self.meta_interp(f, [6, 7]) - self.check_loops(newstr=1, strsetitem=1, copystrcontent=1, - call=1, call_pure=0) # escape + if _str is str: + self.check_loops(newstr=1, strsetitem=1, copystrcontent=1, + call=1, call_pure=0) # escape + else: + self.check_loops(newunicode=1, unicodesetitem=1, + copyunicodecontent=1, + call=1, call_pure=0) # escape def test_strconcat_escape_char_str(self): + _str, _chr = self._str, self._chr jitdriver = JitDriver(greens = [], reds = ['m', 'n']) @dont_look_inside def escape(x): pass - mylist = ["somestr"+str(i) for i in range(10)] + mylist = [_str("somestr") + _str(i) for i in range(10)] def f(n, m): while m >= 0: jitdriver.can_enter_jit(m=m, n=n) jitdriver.jit_merge_point(m=m, n=n) - s = chr(n) + mylist[m] + s = _chr(n) + mylist[m] escape(s) m -= 1 return 42 self.meta_interp(f, [6, 7]) - self.check_loops(newstr=1, strsetitem=1, copystrcontent=1, - call=1, call_pure=0) # escape + if _str is str: + self.check_loops(newstr=1, strsetitem=1, copystrcontent=1, + call=1, call_pure=0) # escape + else: + self.check_loops(newunicode=1, unicodesetitem=1, + copyunicodecontent=1, + call=1, call_pure=0) # escape def test_strconcat_escape_char_char(self): + _str, _chr = self._str, self._chr jitdriver = JitDriver(greens = [], reds = ['m', 'n']) @dont_look_inside def escape(x): @@ -219,91 +229,132 @@ while m >= 0: jitdriver.can_enter_jit(m=m, n=n) jitdriver.jit_merge_point(m=m, n=n) - s = chr(n) + chr(m) + s = _chr(n) + _chr(m) escape(s) m -= 1 return 42 self.meta_interp(f, [6, 7]) - self.check_loops(newstr=1, strsetitem=2, copystrcontent=0, - call=1, call_pure=0) # escape + if _str is str: + self.check_loops(newstr=1, strsetitem=2, copystrcontent=0, + call=1, call_pure=0) # escape + else: + self.check_loops(newunicode=1, unicodesetitem=2, + copyunicodecontent=0, + call=1, call_pure=0) # escape def test_strconcat_escape_str_char_str(self): + _str, _chr = self._str, self._chr jitdriver = JitDriver(greens = [], reds = ['m', 'n']) @dont_look_inside def escape(x): pass - mylist = ["somestr"+str(i) for i in range(10)] + mylist = [_str("somestr") + _str(i) for i in range(10)] def f(n, m): while m >= 0: jitdriver.can_enter_jit(m=m, n=n) jitdriver.jit_merge_point(m=m, n=n) - s = mylist[n] + chr(n) + mylist[m] + s = mylist[n] + _chr(n) + mylist[m] escape(s) m -= 1 return 42 self.meta_interp(f, [6, 7]) - self.check_loops(newstr=1, strsetitem=1, copystrcontent=2, - call=1, call_pure=0) # escape + if _str is str: + self.check_loops(newstr=1, strsetitem=1, copystrcontent=2, + call=1, call_pure=0) # escape + else: + self.check_loops(newunicode=1, unicodesetitem=1, + copyunicodecontent=2, + call=1, call_pure=0) # escape def test_strconcat_guard_fail(self): - for somestr in ["abc", ]: #u"def"]: - jitdriver = JitDriver(greens = [], reds = ['m', 'n']) - @dont_look_inside - def escape(x): - pass - mylist = [somestr+str(i) for i in range(12)] - def f(n, m): - while m >= 0: - jitdriver.can_enter_jit(m=m, n=n) - jitdriver.jit_merge_point(m=m, n=n) - s = mylist[n] + mylist[m] - if m & 1: - escape(s) - m -= 1 - return 42 - self.meta_interp(f, [6, 10]) + _str = self._str + jitdriver = JitDriver(greens = [], reds = ['m', 'n']) + @dont_look_inside + def escape(x): + pass + mylist = [_str("abc") + _str(i) for i in range(12)] + def f(n, m): + while m >= 0: + jitdriver.can_enter_jit(m=m, n=n) + jitdriver.jit_merge_point(m=m, n=n) + s = mylist[n] + mylist[m] + if m & 1: + escape(s) + m -= 1 + return 42 + self.meta_interp(f, [6, 10]) def test_strslice(self): - for somestr in ["abc", ]: #u"def"]: - jitdriver = JitDriver(greens = [], reds = ['m', 'n']) - @dont_look_inside - def escape(x): - pass - def f(n, m): - assert n >= 0 - while m >= 0: - jitdriver.can_enter_jit(m=m, n=n) - jitdriver.jit_merge_point(m=m, n=n) - s = "foobarbazetc"[m:n] - if m <= 5: - escape(s) - m -= 1 - return 42 - self.meta_interp(f, [10, 10]) + _str = self._str + longstring = _str("foobarbazetc") + jitdriver = JitDriver(greens = [], reds = ['m', 'n']) + @dont_look_inside + def escape(x): + pass + def f(n, m): + assert n >= 0 + while m >= 0: + jitdriver.can_enter_jit(m=m, n=n) + jitdriver.jit_merge_point(m=m, n=n) + s = longstring[m:n] + if m <= 5: + escape(s) + m -= 1 + return 42 + self.meta_interp(f, [10, 10]) def test_streq_char(self): - for somestr in ["?abcdefg", ]: #u"def"]: - jitdriver = JitDriver(greens = [], reds = ['m', 'n']) - @dont_look_inside - def escape(x): - pass - def f(n, m): - assert n >= 0 - while m >= 0: - jitdriver.can_enter_jit(m=m, n=n) - jitdriver.jit_merge_point(m=m, n=n) - s = somestr[:m] - escape(s == "?") - m -= 1 - return 42 - self.meta_interp(f, [6, 7]) - self.check_loops(newstr=0, newunicode=0) - - -class TestOOtype(StringTests, OOJitMixin): - CALL = "oosend" - CALL_PURE = "oosend_pure" + _str = self._str + longstring = _str("?abcdefg") + somechar = _str("?") + jitdriver = JitDriver(greens = [], reds = ['m', 'n']) + @dont_look_inside + def escape(x): + pass + def f(n, m): + assert n >= 0 + while m >= 0: + jitdriver.can_enter_jit(m=m, n=n) + jitdriver.jit_merge_point(m=m, n=n) + s = longstring[:m] + escape(s == somechar) + m -= 1 + return 42 + self.meta_interp(f, [6, 7]) + self.check_loops(newstr=0, newunicode=0) + + +#class TestOOtype(StringTests, OOJitMixin): +# CALL = "oosend" +# CALL_PURE = "oosend_pure" class TestLLtype(StringTests, LLJitMixin): CALL = "call" CALL_PURE = "call_pure" + +class TestLLtypeUnicode(TestLLtype): + _str, _chr = unicode, unichr + + def test_str2unicode(self): + _str = self._str + jitdriver = JitDriver(greens = [], reds = ['m', 'n']) + class Foo: + pass + @dont_look_inside + def escape(x): + assert x == _str("6y") + def f(n, m): + while m >= 0: + jitdriver.can_enter_jit(m=m, n=n) + jitdriver.jit_merge_point(m=m, n=n) + foo = Foo() + foo.y = chr(m) + foo.y = "y" + s = _str(str(n)) + _str(foo.y) + escape(s) + m -= 1 + return 42 + self.meta_interp(f, [6, 7]) + self.check_loops(call=3, # str(), _str(), escape() + newunicode=1, unicodegetitem=0, + unicodesetitem=1, copyunicodecontent=1) Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_virtualref.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_virtualref.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_virtualref.py Tue Nov 23 10:12:47 2010 @@ -88,12 +88,16 @@ cpu.get_latest_value_int = lambda i:guard_op.getfailargs()[i].getint() cpu.get_latest_value_ref = lambda i:guard_op.getfailargs()[i].getref_base() cpu.clear_latest_values = lambda count: None - resumereader = ResumeDataDirectReader(cpu, guard_op.getdescr()) + class FakeMetaInterpSd: + callinfocollection = None + FakeMetaInterpSd.cpu = cpu + resumereader = ResumeDataDirectReader(FakeMetaInterpSd(), + guard_op.getdescr()) vrefinfo = self.metainterp.staticdata.virtualref_info lst = [] vrefinfo.continue_tracing = lambda vref, virtual: \ lst.append((vref, virtual)) - resumereader.consume_vref_and_vable(vrefinfo, None) + resumereader.consume_vref_and_vable(vrefinfo, None, None) del vrefinfo.continue_tracing assert len(lst) == 1 lltype.cast_opaque_ptr(lltype.Ptr(JIT_VIRTUAL_REF), Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_warmspot.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_warmspot.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_warmspot.py Tue Nov 23 10:12:47 2010 @@ -296,6 +296,69 @@ assert res == 1 self.check_loops(int_add=1) # I get 13 without the loop_header() + def test_omit_can_enter_jit(self): + # Simple test comparing the effects of always giving a can_enter_jit(), + # or not giving any. Mostly equivalent, except that if given, it is + # ignored the first time, and so it ends up taking one extra loop to + # start JITting. + mydriver = JitDriver(greens=[], reds=['m']) + # + for i2 in range(10): + def f2(m): + while m > 0: + mydriver.jit_merge_point(m=m) + m -= 1 + self.meta_interp(f2, [i2]) + try: + self.check_tree_loop_count(1) + break + except AssertionError: + print "f2: no loop generated for i2==%d" % i2 + else: + raise # re-raise the AssertionError: check_loop_count never 1 + # + for i1 in range(10): + def f1(m): + while m > 0: + mydriver.can_enter_jit(m=m) + mydriver.jit_merge_point(m=m) + m -= 1 + self.meta_interp(f1, [i1]) + try: + self.check_tree_loop_count(1) + break + except AssertionError: + print "f1: no loop generated for i1==%d" % i1 + else: + raise # re-raise the AssertionError: check_loop_count never 1 + # + assert i1 - 1 == i2 + + def test_no_loop_at_all(self): + mydriver = JitDriver(greens=[], reds=['m']) + def f2(m): + mydriver.jit_merge_point(m=m) + return m - 1 + def f1(m): + while m > 0: + m = f2(m) + self.meta_interp(f1, [8]) + # it should generate one "loop" only, which ends in a FINISH + # corresponding to the return from f2. + self.check_tree_loop_count(1) + self.check_loop_count(0) + + def test_simple_loop(self): + mydriver = JitDriver(greens=[], reds=['m']) + def f1(m): + while m > 0: + mydriver.jit_merge_point(m=m) + m = m - 1 + self.meta_interp(f1, [8]) + self.check_loop_count(1) + self.check_loops({'int_sub': 1, 'int_gt': 1, 'guard_true': 1, + 'jump': 1}) + class TestLLWarmspot(WarmspotTests, LLJitMixin): CPUClass = runner.LLtypeCPU Modified: pypy/branch/arm-backend/pypy/jit/metainterp/test/test_ztranslation.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/test/test_ztranslation.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/test/test_ztranslation.py Tue Nov 23 10:12:47 2010 @@ -79,7 +79,7 @@ res = ll_meta_interp(main, [40, 5], CPUClass=self.CPUClass, type_system=self.type_system) assert res == main(40, 5) - res = rpython_ll_meta_interp(main, [40, 5], loops=2, + res = rpython_ll_meta_interp(main, [40, 5], CPUClass=self.CPUClass, type_system=self.type_system, optimizer=OPTIMIZER_FULL, @@ -120,7 +120,7 @@ res = ll_meta_interp(main, [40], CPUClass=self.CPUClass, type_system=self.type_system) assert res == main(40) - res = rpython_ll_meta_interp(main, [40], loops=2, CPUClass=self.CPUClass, + res = rpython_ll_meta_interp(main, [40], CPUClass=self.CPUClass, type_system=self.type_system, optimizer=OPTIMIZER_FULL, ProfilerClass=Profiler) Modified: pypy/branch/arm-backend/pypy/jit/metainterp/virtualizable.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/virtualizable.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/virtualizable.py Tue Nov 23 10:12:47 2010 @@ -100,48 +100,48 @@ i = i + 1 assert len(boxes) == i + 1 # - def write_from_resume_data_partial(virtualizable, reader, nums): + def write_from_resume_data_partial(virtualizable, reader, numb): # Load values from the reader (see resume.py) described by # the list of numbers 'nums', and write them in their proper # place in the 'virtualizable'. This works from the end of # the list and returns the index in 'nums' of the start of # the virtualizable data found, allowing the caller to do # further processing with the start of the list. - i = len(nums) - 1 + i = len(numb.nums) - 1 assert i >= 0 for ARRAYITEMTYPE, fieldname in unroll_array_fields_rev: lst = getattr(virtualizable, fieldname) for j in range(getlength(lst)-1, -1, -1): i -= 1 assert i >= 0 - x = reader.load_value_of_type(ARRAYITEMTYPE, nums[i]) + x = reader.load_value_of_type(ARRAYITEMTYPE, numb.nums[i]) setarrayitem(lst, j, x) for FIELDTYPE, fieldname in unroll_static_fields_rev: i -= 1 assert i >= 0 - x = reader.load_value_of_type(FIELDTYPE, nums[i]) + x = reader.load_value_of_type(FIELDTYPE, numb.nums[i]) setattr(virtualizable, fieldname, x) return i # - def load_list_of_boxes(virtualizable, reader, nums): + def load_list_of_boxes(virtualizable, reader, numb): # Uses 'virtualizable' only to know the length of the arrays; # does not write anything into it. The returned list is in # the format expected of virtualizable_boxes, so it ends in # the virtualizable itself. - i = len(nums) - 1 + i = len(numb.nums) - 1 assert i >= 0 - boxes = [reader.decode_box_of_type(self.VTYPEPTR, nums[i])] + boxes = [reader.decode_box_of_type(self.VTYPEPTR, numb.nums[i])] for ARRAYITEMTYPE, fieldname in unroll_array_fields_rev: lst = getattr(virtualizable, fieldname) for j in range(getlength(lst)-1, -1, -1): i -= 1 assert i >= 0 - box = reader.decode_box_of_type(ARRAYITEMTYPE, nums[i]) + box = reader.decode_box_of_type(ARRAYITEMTYPE,numb.nums[i]) boxes.append(box) for FIELDTYPE, fieldname in unroll_static_fields_rev: i -= 1 assert i >= 0 - box = reader.decode_box_of_type(FIELDTYPE, nums[i]) + box = reader.decode_box_of_type(FIELDTYPE, numb.nums[i]) boxes.append(box) boxes.reverse() return boxes Modified: pypy/branch/arm-backend/pypy/jit/metainterp/virtualref.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/virtualref.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/virtualref.py Tue Nov 23 10:12:47 2010 @@ -16,7 +16,8 @@ ('virtualref_index', lltype.Signed), ('forced', rclass.OBJECTPTR)) self.jit_virtual_ref_vtable = lltype.malloc(rclass.OBJECT_VTABLE, - zero=True, flavor='raw') + zero=True, flavor='raw', + immortal=True) self.jit_virtual_ref_vtable.name = rclass.alloc_array_name( 'jit_virtual_ref') # build some constants Modified: pypy/branch/arm-backend/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/warmspot.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/warmspot.py Tue Nov 23 10:12:47 2010 @@ -67,9 +67,16 @@ def jittify_and_run(interp, graph, args, repeat=1, backendopt=False, trace_limit=sys.maxint, debug_level=DEBUG_STEPS, inline=False, **kwds): + from pypy.config.config import ConfigError translator = interp.typer.annotator.translator - translator.config.translation.gc = "boehm" - translator.config.translation.list_comprehension_operations = True + try: + translator.config.translation.gc = "boehm" + except ConfigError: + pass + try: + translator.config.translation.list_comprehension_operations = True + except ConfigError: + pass warmrunnerdesc = WarmRunnerDesc(translator, backendopt=backendopt, **kwds) for jd in warmrunnerdesc.jitdrivers_sd: jd.warmstate.set_param_threshold(3) # for tests @@ -91,8 +98,7 @@ repeat -= 1 return res -def rpython_ll_meta_interp(function, args, backendopt=True, - loops='not used right now', **kwds): +def rpython_ll_meta_interp(function, args, backendopt=True, **kwds): return ll_meta_interp(function, args, backendopt=backendopt, translate_support_code=True, **kwds) @@ -103,15 +109,16 @@ for i in range(len(block.operations)): op = block.operations[i] if (op.opname == 'jit_marker' and - op.args[0].value == marker_name): + op.args[0].value == marker_name and + op.args[1].value.active): # the jitdriver results.append((graph, block, i)) return results def find_can_enter_jit(graphs): - results = _find_jit_marker(graphs, 'can_enter_jit') - if not results: - raise Exception("no can_enter_jit found!") - return results + return _find_jit_marker(graphs, 'can_enter_jit') + +def find_loop_headers(graphs): + return _find_jit_marker(graphs, 'loop_header') def find_jit_merge_points(graphs): results = _find_jit_marker(graphs, 'jit_merge_point') @@ -204,9 +211,9 @@ "there are multiple jit_merge_points with the same jitdriver" def split_graph_and_record_jitdriver(self, graph, block, pos): - jd = JitDriverStaticData() - jd._jit_merge_point_pos = (graph, block, pos) op = block.operations[pos] + jd = JitDriverStaticData() + jd._jit_merge_point_pos = (graph, op) args = op.args[2:] s_binding = self.translator.annotator.binding jd._portal_args_s = [s_binding(v) for v in args] @@ -279,10 +286,20 @@ def make_virtualizable_infos(self): vinfos = {} for jd in self.jitdrivers_sd: + # + jd.greenfield_info = None + for name in jd.jitdriver.greens: + if '.' in name: + from pypy.jit.metainterp.greenfield import GreenFieldInfo + jd.greenfield_info = GreenFieldInfo(self.cpu, jd) + break + # if not jd.jitdriver.virtualizables: jd.virtualizable_info = None jd.index_of_virtualizable = -1 continue + else: + assert jd.greenfield_info is None, "XXX not supported yet" # jitdriver = jd.jitdriver assert len(jitdriver.virtualizables) == 1 # for now @@ -450,8 +467,7 @@ self.make_args_specification(jd) def make_args_specification(self, jd): - graph, block, index = jd._jit_merge_point_pos - op = block.operations[index] + graph, op = jd._jit_merge_point_pos greens_v, reds_v = support.decode_hp_hint_args(op) ALLARGS = [v.concretetype for v in (greens_v + reds_v)] jd._green_args_spec = [v.concretetype for v in greens_v] @@ -467,26 +483,37 @@ [lltype.Signed, llmemory.GCREF], RESTYPE) def rewrite_can_enter_jits(self): - can_enter_jits = find_can_enter_jit(self.translator.graphs) sublists = {} for jd in self.jitdrivers_sd: - sublists[jd.jitdriver] = [] + sublists[jd.jitdriver] = jd, [] + jd.no_loop_header = True + # + loop_headers = find_loop_headers(self.translator.graphs) + for graph, block, index in loop_headers: + op = block.operations[index] + jitdriver = op.args[1].value + assert jitdriver in sublists, \ + "loop_header with no matching jit_merge_point" + jd, sublist = sublists[jitdriver] + jd.no_loop_header = False + # + can_enter_jits = find_can_enter_jit(self.translator.graphs) for graph, block, index in can_enter_jits: op = block.operations[index] jitdriver = op.args[1].value assert jitdriver in sublists, \ "can_enter_jit with no matching jit_merge_point" + jd, sublist = sublists[jitdriver] origportalgraph = jd._jit_merge_point_pos[0] if graph is not origportalgraph: - sublists[jitdriver].append((graph, block, index)) + sublist.append((graph, block, index)) + jd.no_loop_header = False else: pass # a 'can_enter_jit' before the 'jit-merge_point', but # originally in the same function: we ignore it here # see e.g. test_jitdriver.test_simple for jd in self.jitdrivers_sd: - sublist = sublists[jd.jitdriver] - assert len(sublist) > 0, \ - "found no can_enter_jit for %r" % (jd.jitdriver,) + _, sublist = sublists[jd.jitdriver] self.rewrite_can_enter_jit(jd, sublist) def rewrite_can_enter_jit(self, jd, can_enter_jits): @@ -494,6 +521,19 @@ FUNCPTR = jd._PTR_JIT_ENTER_FUNCTYPE jit_enter_fnptr = self.helper_func(FUNCPTR, jd._maybe_enter_jit_fn) + if len(can_enter_jits) == 0: + # see test_warmspot.test_no_loop_at_all + operations = jd.portal_graph.startblock.operations + op1 = operations[0] + assert (op1.opname == 'jit_marker' and + op1.args[0].value == 'jit_merge_point') + op0 = SpaceOperation( + 'jit_marker', + [Constant('can_enter_jit', lltype.Void)] + op1.args[1:], + None) + operations.insert(0, op0) + can_enter_jits = [(jd.portal_graph, jd.portal_graph.startblock, 0)] + for graph, block, index in can_enter_jits: if graph is jd._jit_merge_point_pos[0]: continue @@ -702,8 +742,14 @@ # ____________________________________________________________ # Now mutate origportalgraph to end with a call to portal_runner_ptr # - _, origblock, origindex = jd._jit_merge_point_pos - op = origblock.operations[origindex] + _, op = jd._jit_merge_point_pos + for origblock in origportalgraph.iterblocks(): + if op in origblock.operations: + break + else: + assert False, "lost the operation %r in the graph %r" % ( + op, origportalgraph) + origindex = origblock.operations.index(op) assert op.opname == 'jit_marker' assert op.args[0].value == 'jit_merge_point' greens_v, reds_v = support.decode_hp_hint_args(op) Modified: pypy/branch/arm-backend/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/metainterp/warmstate.py (original) +++ pypy/branch/arm-backend/pypy/jit/metainterp/warmstate.py Tue Nov 23 10:12:47 2010 @@ -1,5 +1,5 @@ import sys -from pypy.rpython.lltypesystem import lltype, llmemory, rstr +from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi from pypy.rpython.ootypesystem import ootype from pypy.rpython.annlowlevel import hlstr, llstr, cast_base_ptr_to_instance from pypy.rpython.annlowlevel import cast_object_to_ptr @@ -24,7 +24,11 @@ """ INPUT = lltype.typeOf(x) if INPUT is lltype.Signed: - return lltype.cast_primitive(TYPE, x) # XXX missing: Ptr(non-gc) + if isinstance(TYPE, lltype.Ptr) and TYPE.TO._gckind == 'raw': + # non-gc pointer + return rffi.cast(TYPE, x) + else: + return lltype.cast_primitive(TYPE, x) elif INPUT is lltype.Float: assert TYPE is lltype.Float return x @@ -172,6 +176,9 @@ meth(default_value) def set_param_threshold(self, threshold): + if threshold < 0: + self.increment_threshold = 0 # never reach the THRESHOLD_LIMIT + return if threshold < 2: threshold = 2 self.increment_threshold = (self.THRESHOLD_LIMIT // threshold) + 1 Modified: pypy/branch/arm-backend/pypy/jit/tl/pypyjit.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/tl/pypyjit.py (original) +++ pypy/branch/arm-backend/pypy/jit/tl/pypyjit.py Tue Nov 23 10:12:47 2010 @@ -38,10 +38,12 @@ config.objspace.allworkingmodules = False config.objspace.usemodules.pypyjit = True config.objspace.usemodules.array = True -config.objspace.usemodules._weakref = False +config.objspace.usemodules._weakref = True config.objspace.usemodules._sre = False +# +config.objspace.usemodules._ffi = True +# set_pypy_opt_level(config, level='jit') -config.objspace.std.withinlineddict = True if BACKEND == 'c': config.objspace.std.multimethods = 'mrd' Modified: pypy/branch/arm-backend/pypy/jit/tl/pypyjit_child.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/tl/pypyjit_child.py (original) +++ pypy/branch/arm-backend/pypy/jit/tl/pypyjit_child.py Tue Nov 23 10:12:47 2010 @@ -16,7 +16,7 @@ interp.heap.malloc_nonmovable = returns_null # XXX from pypy.jit.backend.llgraph.runner import LLtypeCPU - LLtypeCPU.supports_floats = False # for now + #LLtypeCPU.supports_floats = False # for now apply_jit(interp, graph, LLtypeCPU) Modified: pypy/branch/arm-backend/pypy/jit/tl/spli/interpreter.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/tl/spli/interpreter.py (original) +++ pypy/branch/arm-backend/pypy/jit/tl/spli/interpreter.py Tue Nov 23 10:12:47 2010 @@ -1,12 +1,14 @@ import os -from pypy.tool import stdlib_opcode as opcode +from pypy.tool import stdlib_opcode from pypy.jit.tl.spli import objects, pycode -from pypy.tool.stdlib_opcode import unrolling_opcode_descs -from pypy.tool.stdlib_opcode import opcode_method_names from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.jit import JitDriver, hint, dont_look_inside from pypy.rlib.objectmodel import we_are_translated +opcode_method_names = stdlib_opcode.host_bytecode_spec.method_names +unrolling_opcode_descs = unrolling_iterable( + stdlib_opcode.host_bytecode_spec.ordered_opdescs) +HAVE_ARGUMENT = stdlib_opcode.host_HAVE_ARGUMENT compare_ops = [ "cmp_lt", # "<" @@ -79,7 +81,7 @@ self.stack_depth = hint(self.stack_depth, promote=True) op = ord(code[instr_index]) instr_index += 1 - if op >= opcode.HAVE_ARGUMENT: + if op >= HAVE_ARGUMENT: low = ord(code[instr_index]) hi = ord(code[instr_index + 1]) oparg = (hi << 8) | low @@ -183,6 +185,12 @@ next_instr += arg return next_instr + def POP_JUMP_IF_FALSE(self, arg, next_instr, code): + w_cond = self.pop() + if not w_cond.is_true(): + next_instr = arg + return next_instr + def JUMP_FORWARD(self, arg, next_instr, code): return next_instr + arg Modified: pypy/branch/arm-backend/pypy/jit/tool/loopviewer.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/tool/loopviewer.py (original) +++ pypy/branch/arm-backend/pypy/jit/tool/loopviewer.py Tue Nov 23 10:12:47 2010 @@ -9,7 +9,7 @@ import optparse from pprint import pprint from pypy.tool import logparser -from pypy.jit.metainterp.test.oparser import parse +from pypy.jit.tool.oparser import parse from pypy.jit.metainterp.history import ConstInt from pypy.rpython.lltypesystem import llmemory, lltype Modified: pypy/branch/arm-backend/pypy/jit/tool/pypytrace-mode.el ============================================================================== --- pypy/branch/arm-backend/pypy/jit/tool/pypytrace-mode.el (original) +++ pypy/branch/arm-backend/pypy/jit/tool/pypytrace-mode.el Tue Nov 23 10:12:47 2010 @@ -26,7 +26,7 @@ ("<.*FieldDescr \\([^ ]*\\)" (1 'font-lock-variable-name-face)) ;; comment out debug_merge_point, but then highlight specific part of it ("^debug_merge_point.*" . font-lock-comment-face) - ("^\\(debug_merge_point\\).*code object\\(.*\\), file \\('.*'\\), \\(line .*\\)> \\(.*\\)')" + ("^\\(debug_merge_point\\).*code object\\(.*\\), file \\('.*'\\), \\(line .*\\)> \\(.*\\)" (1 'compilation-warning t) (2 'escape-glyph t) (3 'font-lock-string-face t) Modified: pypy/branch/arm-backend/pypy/jit/tool/showstats.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/tool/showstats.py (original) +++ pypy/branch/arm-backend/pypy/jit/tool/showstats.py Tue Nov 23 10:12:47 2010 @@ -4,15 +4,16 @@ import autopath import sys, py from pypy.tool import logparser -from pypy.jit.metainterp.test.oparser import parse +from pypy.jit.tool.oparser import parse from pypy.jit.metainterp.resoperation import rop from pypy.rpython.lltypesystem import lltype, llmemory def main(argv): log = logparser.parse_log_file(argv[0]) + log_count_lines = open(argv[0] + '.count').readlines() parts = logparser.extract_category(log, "jit-log-opt-") for i, oplist in enumerate(parts): - loop = parse(oplist, no_namespace=True) + loop = parse(oplist, no_namespace=True, nonstrict=True) num_ops = 0 num_dmp = 0 num_guards = 0 @@ -23,7 +24,11 @@ num_ops += 1 if op.is_guard(): num_guards += 1 - print "Loop #%d, length: %d, opcodes: %d, guards: %d, %f" % (i, num_ops, num_dmp, num_guards, num_ops/num_dmp) - + if num_dmp == 0: + print "Loop #%d, length: %d, opcodes: %d, guards: %d" % (i, num_ops, num_dmp, num_guards) + else: + print "Loop #%d, length: %d, opcodes: %d, guards: %d, %f" % (i, num_ops, num_dmp, num_guards, num_ops/num_dmp) + print loop.comment, "run", log_count_lines[i].split(":")[1].strip(), "times" + if __name__ == '__main__': main(sys.argv[1:]) Modified: pypy/branch/arm-backend/pypy/jit/tool/test/test_traceviewer.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/tool/test/test_traceviewer.py (original) +++ pypy/branch/arm-backend/pypy/jit/tool/test/test_traceviewer.py Tue Nov 23 10:12:47 2010 @@ -19,7 +19,7 @@ def test_no_of_loops(self): data = [preparse(""" # Loop 0 : loop with 39 ops - debug_merge_point('') + debug_merge_point('', 0) guard_class(p4, 141310752, descr=) [p0, p1] p60 = getfield_gc(p4, descr=) guard_nonnull(p60, descr=) [p0, p1] @@ -51,7 +51,7 @@ assert loop.right.content == 'extra' def test_postparse(self): - real_loops = [FinalBlock("debug_merge_point(' #40 POP_TOP')", None)] + real_loops = [FinalBlock("debug_merge_point(' #40 POP_TOP', 0)", None)] postprocess(real_loops, real_loops[:], {}) assert real_loops[0].header.startswith("_runCallbacks, file '/tmp/x/twisted-trunk/twisted/internet/defer.py', line 357") Modified: pypy/branch/arm-backend/pypy/jit/tool/traceviewer.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/tool/traceviewer.py (original) +++ pypy/branch/arm-backend/pypy/jit/tool/traceviewer.py Tue Nov 23 10:12:47 2010 @@ -253,10 +253,10 @@ def main(loopfile, use_threshold, view=True): countname = py.path.local(loopfile + '.count') if countname.check(): - counts = [re.split('( 20 and use_threshold: counts.threshold = l[-20] Modified: pypy/branch/arm-backend/pypy/module/__builtin__/descriptor.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/__builtin__/descriptor.py (original) +++ pypy/branch/arm-backend/pypy/module/__builtin__/descriptor.py Tue Nov 23 10:12:47 2010 @@ -96,6 +96,8 @@ ) class W_Property(Wrappable): + _immutable_fields_ = ["w_fget", "w_fset", "w_fdel"] + def init(self, space, w_fget=None, w_fset=None, w_fdel=None, w_doc=None): self.w_fget = w_fget self.w_fset = w_fset @@ -183,4 +185,3 @@ fget = interp_attrproperty_w('w_fget', W_Property), fset = interp_attrproperty_w('w_fset', W_Property), ) - Modified: pypy/branch/arm-backend/pypy/module/__builtin__/interp_classobj.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/__builtin__/interp_classobj.py (original) +++ pypy/branch/arm-backend/pypy/module/__builtin__/interp_classobj.py Tue Nov 23 10:12:47 2010 @@ -2,9 +2,11 @@ from pypy.interpreter.error import OperationError, operationerrfmt from pypy.interpreter.gateway import ObjSpace, W_Root, NoneNotWrapped, applevel from pypy.interpreter.gateway import interp2app, ObjSpace -from pypy.interpreter.typedef import TypeDef, make_weakref_descr +from pypy.interpreter.typedef import TypeDef from pypy.interpreter.argument import Arguments from pypy.interpreter.baseobjspace import Wrappable +from pypy.interpreter.typedef import GetSetProperty, descr_get_dict +from pypy.interpreter.typedef import descr_set_dict from pypy.rlib.rarithmetic import r_uint, intmask from pypy.rlib.objectmodel import compute_identity_hash from pypy.rlib.debug import make_sure_not_resized @@ -57,6 +59,14 @@ self.bases_w = bases self.w_dict = w_dict + def instantiate(self, space): + cache = space.fromcache(Cache) + if self.lookup(space, '__del__') is not None: + w_inst = cache.cls_with_del(space, self) + else: + w_inst = cache.cls_without_del(space, self) + return w_inst + def getdict(self): return self.w_dict @@ -100,15 +110,15 @@ return False @jit.unroll_safe - def lookup(self, space, w_attr): + def lookup(self, space, attr): # returns w_value or interplevel None - w_result = space.finditem(self.w_dict, w_attr) + w_result = space.finditem_str(self.w_dict, attr) if w_result is not None: return w_result for base in self.bases_w: # XXX fix annotation of bases_w to be a list of W_ClassObjects assert isinstance(base, W_ClassObject) - w_result = base.lookup(space, w_attr) + w_result = base.lookup(space, attr) if w_result is not None: return w_result return None @@ -122,7 +132,7 @@ return space.wrap(self.name) elif name == "__bases__": return space.newtuple(self.bases_w) - w_value = self.lookup(space, w_attr) + w_value = self.lookup(space, name) if w_value is None: raise operationerrfmt( space.w_AttributeError, @@ -147,7 +157,7 @@ self.setbases(space, w_value) return elif name == "__del__": - if self.lookup(space, w_attr) is None: + if self.lookup(space, name) is None: msg = ("a __del__ method added to an existing class " "will not be called") space.warn(msg, space.w_RuntimeWarning) @@ -195,13 +205,20 @@ # NOT_RPYTHON return '' % self.name +class Cache: + def __init__(self, space): + from pypy.interpreter.typedef import _usersubclswithfeature + # evil + self.cls_without_del = _usersubclswithfeature( + space.config, W_InstanceObject, "dict", "weakref") + self.cls_with_del = _usersubclswithfeature( + space.config, self.cls_without_del, "del") + + def class_descr_call(space, w_self, __args__): self = space.interp_w(W_ClassObject, w_self) - if self.lookup(space, space.wrap('__del__')) is not None: - w_inst = W_InstanceObjectWithDel(space, self) - else: - w_inst = W_InstanceObject(space, self) - w_init = w_inst.getattr_from_class(space, space.wrap('__init__')) + w_inst = self.instantiate(space) + w_init = w_inst.getattr_from_class(space, '__init__') if w_init is not None: w_result = space.call_args(w_init, __args__) if not space.is_w(w_result, space.w_None): @@ -234,7 +251,7 @@ def make_unary_instance_method(name): def unaryop(self, space): - w_meth = self.getattr(space, space.wrap(name), True) + w_meth = self.getattr(space, name, True) return space.call_function(w_meth) unaryop.func_name = name return unaryop @@ -242,7 +259,7 @@ def make_binary_returning_notimplemented_instance_method(name): def binaryop(self, space, w_other): try: - w_meth = self.getattr(space, space.wrap(name), False) + w_meth = self.getattr(space, name, False) except OperationError, e: if e.match(space, space.w_AttributeError): return space.w_NotImplemented @@ -267,7 +284,7 @@ w_a = self w_b = w_other if w_a is self: - w_meth = self.getattr(space, space.wrap(specialname), False) + w_meth = self.getattr(space, specialname, False) if w_meth is None: return space.w_NotImplemented return space.call_function(w_meth, w_b) @@ -278,7 +295,7 @@ def rbinaryop(self, space, w_other): w_a, w_b = _coerce_helper(space, self, w_other) if w_a is None or w_a is self: - w_meth = self.getattr(space, space.wrap(rspecialname), False) + w_meth = self.getattr(space, rspecialname, False) if w_meth is None: return space.w_NotImplemented return space.call_function(w_meth, w_other) @@ -302,46 +319,34 @@ raise OperationError( space.w_TypeError, space.wrap("instance() first arg must be class")) - if space.is_w(w_dict, space.w_None): - w_dict = None - elif not space.is_true(space.isinstance(w_dict, space.w_dict)): - raise OperationError( - space.w_TypeError, - space.wrap("instance() second arg must be dictionary or None")) - return W_InstanceObject(space, w_class, w_dict) + w_result = w_class.instantiate(space) + if not space.is_w(w_dict, space.w_None): + w_result.setdict(space, w_dict) + return w_result class W_InstanceObject(Wrappable): - def __init__(self, space, w_class, w_dict=None): - if w_dict is None: - w_dict = space.newdict(instance=True) + def __init__(self, space, w_class): + # note that user_setup is overridden by the typedef.py machinery + self.user_setup(space, space.gettypeobject(self.typedef)) assert isinstance(w_class, W_ClassObject) self.w_class = w_class - self.w_dict = w_dict - self.space = space - - def getdict(self): - return self.w_dict - def setdict(self, space, w_dict): - if (w_dict is None or - not space.is_true(space.isinstance(w_dict, space.w_dict))): - raise OperationError( - space.w_TypeError, - space.wrap("__dict__ must be a dictionary object")) - self.w_dict = w_dict + def user_setup(self, space, w_subtype): + self.space = space - def setclass(self, space, w_class): + def set_oldstyle_class(self, space, w_class): if w_class is None or not isinstance(w_class, W_ClassObject): raise OperationError( space.w_TypeError, space.wrap("__class__ must be set to a class")) self.w_class = w_class - def getattr_from_class(self, space, w_name): + def getattr_from_class(self, space, name): # Look up w_name in the class dict, and call its __get__. # This method ignores the instance dict and the __getattr__. # Returns None if not found. - w_value = self.w_class.lookup(space, w_name) + assert isinstance(name, str) + w_value = self.w_class.lookup(space, name) if w_value is None: return None w_descr_get = space.lookup(w_value, '__get__') @@ -349,19 +354,20 @@ return w_value return space.call_function(w_descr_get, w_value, self, self.w_class) - def getattr(self, space, w_name, exc=True): + def getattr(self, space, name, exc=True): # Normal getattr rules: look up w_name in the instance dict, # in the class dict, and then via a call to __getatttr__. - w_result = space.finditem(self.w_dict, w_name) + assert isinstance(name, str) + w_result = self.getdictvalue(space, name) if w_result is not None: return w_result - w_result = self.getattr_from_class(space, w_name) + w_result = self.getattr_from_class(space, name) if w_result is not None: return w_result - w_meth = self.getattr_from_class(space, space.wrap('__getattr__')) + w_meth = self.getattr_from_class(space, '__getattr__') if w_meth is not None: try: - return space.call_function(w_meth, w_name) + return space.call_function(w_meth, space.wrap(name)) except OperationError, e: if not exc and e.match(space, space.w_AttributeError): return None # eat the AttributeError @@ -371,7 +377,7 @@ raise operationerrfmt( space.w_AttributeError, "%s instance has no attribute '%s'", - self.w_class.name, space.str_w(w_name)) + self.w_class.name, name) else: return None @@ -379,24 +385,25 @@ name = space.str_w(w_attr) if len(name) >= 8 and name[0] == '_': if name == "__dict__": - return self.w_dict + return self.getdict() elif name == "__class__": return self.w_class - return self.getattr(space, w_attr) + return self.getattr(space, name) def descr_setattr(self, space, w_name, w_value): name = unwrap_attr(space, w_name) - w_meth = self.getattr_from_class(space, space.wrap('__setattr__')) + w_meth = self.getattr_from_class(space, '__setattr__') if name and name[0] == "_": if name == '__dict__': self.setdict(space, w_value) return if name == '__class__': - self.setclass(space, w_value) + self.set_oldstyle_class(space, w_value) return if name == '__del__' and w_meth is None: - if (not isinstance(self, W_InstanceObjectWithDel) - and space.finditem(self.w_dict, w_name) is None): + cache = space.fromcache(Cache) + if (not isinstance(self, cache.cls_with_del) + and self.getdictvalue(space, '__del__') is None): msg = ("a __del__ method added to an instance " "with no __del__ in the class will not be called") space.warn(msg, space.w_RuntimeWarning) @@ -410,13 +417,13 @@ if name and name[0] == "_": if name == '__dict__': # use setdict to raise the error - self.setdict(space, None) + self.setdict(space, space.w_None) return elif name == '__class__': - # use setclass to raise the error - self.setclass(space, None) + # use set_oldstyle_class to raise the error + self.set_oldstyle_class(space, None) return - w_meth = self.getattr_from_class(space, space.wrap('__delattr__')) + w_meth = self.getattr_from_class(space, '__delattr__') if w_meth is not None: space.call_function(w_meth, w_name) else: @@ -427,7 +434,7 @@ self.w_class.name, name) def descr_repr(self, space): - w_meth = self.getattr(space, space.wrap('__repr__'), False) + w_meth = self.getattr(space, '__repr__', False) if w_meth is None: w_class = self.w_class mod = w_class.get_module_string(space) @@ -435,19 +442,19 @@ return space.call_function(w_meth) def descr_str(self, space): - w_meth = self.getattr(space, space.wrap('__str__'), False) + w_meth = self.getattr(space, '__str__', False) if w_meth is None: return self.descr_repr(space) return space.call_function(w_meth) def descr_unicode(self, space): - w_meth = self.getattr(space, space.wrap('__unicode__'), False) + w_meth = self.getattr(space, '__unicode__', False) if w_meth is None: return self.descr_str(space) return space.call_function(w_meth) def descr_len(self, space): - w_meth = self.getattr(space, space.wrap('__len__')) + w_meth = self.getattr(space, '__len__') w_result = space.call_function(w_meth) if space.is_true(space.isinstance(w_result, space.w_int)): if space.is_true(space.lt(w_result, space.wrap(0))): @@ -460,22 +467,22 @@ space.wrap("__len__() should return an int")) def descr_getitem(self, space, w_key): - w_meth = self.getattr(space, space.wrap('__getitem__')) + w_meth = self.getattr(space, '__getitem__') return space.call_function(w_meth, w_key) def descr_setitem(self, space, w_key, w_value): - w_meth = self.getattr(space, space.wrap('__setitem__')) + w_meth = self.getattr(space, '__setitem__') space.call_function(w_meth, w_key, w_value) def descr_delitem(self, space, w_key): - w_meth = self.getattr(space, space.wrap('__delitem__')) + w_meth = self.getattr(space, '__delitem__') space.call_function(w_meth, w_key) def descr_iter(self, space): - w_meth = self.getattr(space, space.wrap('__iter__'), False) + w_meth = self.getattr(space, '__iter__', False) if w_meth is not None: return space.call_function(w_meth) - w_meth = self.getattr(space, space.wrap('__getitem__'), False) + w_meth = self.getattr(space, '__getitem__', False) if w_meth is None: raise OperationError( space.w_TypeError, @@ -485,14 +492,14 @@ # don't see the point def descr_getslice(self, space, w_i, w_j): - w_meth = self.getattr(space, space.wrap('__getslice__'), False) + w_meth = self.getattr(space, '__getslice__', False) if w_meth is not None: return space.call_function(w_meth, w_i, w_j) else: return space.getitem(self, space.newslice(w_i, w_j, space.w_None)) def descr_setslice(self, space, w_i, w_j, w_sequence): - w_meth = self.getattr(space, space.wrap('__setslice__'), False) + w_meth = self.getattr(space, '__setslice__', False) if w_meth is not None: space.call_function(w_meth, w_i, w_j, w_sequence) else: @@ -500,20 +507,20 @@ w_sequence) def descr_delslice(self, space, w_i, w_j): - w_meth = self.getattr(space, space.wrap('__delslice__'), False) + w_meth = self.getattr(space, '__delslice__', False) if w_meth is not None: space.call_function(w_meth, w_i, w_j) else: return space.delitem(self, space.newslice(w_i, w_j, space.w_None)) def descr_call(self, space, __args__): - w_meth = self.getattr(space, space.wrap('__call__')) + w_meth = self.getattr(space, '__call__') return space.call_args(w_meth, __args__) def descr_nonzero(self, space): - w_func = self.getattr(space, space.wrap('__nonzero__'), False) + w_func = self.getattr(space, '__nonzero__', False) if w_func is None: - w_func = self.getattr(space, space.wrap('__len__'), False) + w_func = self.getattr(space, '__len__', False) if w_func is None: return space.w_True w_result = space.call_function(w_func) @@ -537,7 +544,7 @@ not isinstance(w_b, W_InstanceObject)): return space.cmp(w_a, w_b) if isinstance(w_a, W_InstanceObject): - w_func = w_a.getattr(space, space.wrap('__cmp__'), False) + w_func = w_a.getattr(space, '__cmp__', False) if w_func is not None: w_res = space.call_function(w_func, w_b) if space.is_w(w_res, space.w_NotImplemented): @@ -556,7 +563,7 @@ return space.wrap(-1) return space.wrap(0) if isinstance(w_b, W_InstanceObject): - w_func = w_b.getattr(space, space.wrap('__cmp__'), False) + w_func = w_b.getattr(space, '__cmp__', False) if w_func is not None: w_res = space.call_function(w_func, w_a) if space.is_w(w_res, space.w_NotImplemented): @@ -577,10 +584,10 @@ return space.w_NotImplemented def descr_hash(self, space): - w_func = self.getattr(space, space.wrap('__hash__'), False) + w_func = self.getattr(space, '__hash__', False) if w_func is None: - w_eq = self.getattr(space, space.wrap('__eq__'), False) - w_cmp = self.getattr(space, space.wrap('__cmp__'), False) + w_eq = self.getattr(space, '__eq__', False) + w_cmp = self.getattr(space, '__cmp__', False) if w_eq is not None or w_cmp is not None: raise OperationError(space.w_TypeError, space.wrap("unhashable instance")) @@ -595,7 +602,7 @@ return w_ret def descr_index(self, space): - w_func = self.getattr(space, space.wrap('__index__'), False) + w_func = self.getattr(space, '__index__', False) if w_func is not None: return space.call_function(w_func) raise OperationError( @@ -603,7 +610,7 @@ space.wrap("object cannot be interpreted as an index")) def descr_contains(self, space, w_obj): - w_func = self.getattr(space, space.wrap('__contains__'), False) + w_func = self.getattr(space, '__contains__', False) if w_func is not None: return space.wrap(space.is_true(space.call_function(w_func, w_obj))) # now do it ourselves @@ -626,7 +633,7 @@ w_a = self w_b = w_other if w_a is self: - w_func = self.getattr(space, space.wrap('__pow__'), False) + w_func = self.getattr(space, '__pow__', False) if w_func is not None: return space.call_function(w_func, w_other) return space.w_NotImplemented @@ -634,7 +641,7 @@ return space.pow(w_a, w_b, space.w_None) else: # CPython also doesn't try coercion in this case - w_func = self.getattr(space, space.wrap('__pow__'), False) + w_func = self.getattr(space, '__pow__', False) if w_func is not None: return space.call_function(w_func, w_other, w_modulo) return space.w_NotImplemented @@ -646,7 +653,7 @@ w_a = self w_b = w_other if w_a is self: - w_func = self.getattr(space, space.wrap('__rpow__'), False) + w_func = self.getattr(space, '__rpow__', False) if w_func is not None: return space.call_function(w_func, w_other) return space.w_NotImplemented @@ -654,13 +661,13 @@ return space.pow(w_b, w_a, space.w_None) else: # CPython also doesn't try coercion in this case - w_func = self.getattr(space, space.wrap('__rpow__'), False) + w_func = self.getattr(space, '__rpow__', False) if w_func is not None: return space.call_function(w_func, w_other, w_modulo) return space.w_NotImplemented def descr_next(self, space): - w_func = self.getattr(space, space.wrap('next'), False) + w_func = self.getattr(space, 'next', False) if w_func is None: raise OperationError(space.w_TypeError, space.wrap("instance has no next() method")) @@ -669,10 +676,9 @@ def descr_del(self, space): # Note that this is called from executioncontext.UserDelAction # via the space.userdel() method. - w_name = space.wrap('__del__') - w_func = space.finditem(self.w_dict, w_name) + w_func = self.getdictvalue(space, '__del__') if w_func is None: - w_func = self.getattr_from_class(space, w_name) + w_func = self.getattr_from_class(space, '__del__') if w_func is not None: space.call_function(w_func) @@ -717,6 +723,14 @@ rmeth, unwrap_spec=["self", ObjSpace, W_Root]) + +def descr_del_dict(space, w_inst): + # use setdict to raise the error + w_inst.setdict(space, space.w_None) + +dict_descr = GetSetProperty(descr_get_dict, descr_set_dict, descr_del_dict) +dict_descr.name = '__dict__' + W_InstanceObject.typedef = TypeDef("instance", __new__ = interp2app(descr_instance_new), __getattribute__ = interp2app(W_InstanceObject.descr_getattribute, @@ -766,12 +780,9 @@ unwrap_spec=['self', ObjSpace, W_Root, W_Root]), next = interp2app(W_InstanceObject.descr_next, unwrap_spec=['self', ObjSpace]), - __weakref__ = make_weakref_descr(W_InstanceObject), __del__ = interp2app(W_InstanceObject.descr_del, unwrap_spec=['self', ObjSpace]), + __dict__ = dict_descr, **rawdict ) - -class W_InstanceObjectWithDel(W_InstanceObject): - def __del__(self): - self._enqueue_for_destruction(self.space) +W_InstanceObject.typedef.acceptable_as_base_class = False Modified: pypy/branch/arm-backend/pypy/module/__builtin__/test/test_classobj.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/__builtin__/test/test_classobj.py (original) +++ pypy/branch/arm-backend/pypy/module/__builtin__/test/test_classobj.py Tue Nov 23 10:12:47 2010 @@ -928,29 +928,33 @@ assert x is b assert y == 5 - -class AppTestOldStyleSharing(AppTestOldstyle): - def setup_class(cls): - cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True}) - if option.runappdirect: - py.test.skip("can only be run on py.py") - def is_sharing(space, w_inst): - from pypy.objspace.std.sharingdict import SharedDictImplementation - w_d = w_inst.getdict() - return space.wrap(isinstance(w_d, SharedDictImplementation) and w_d.r_dict_content is None) - cls.w_is_sharing = cls.space.wrap(gateway.interp2app(is_sharing)) - - - def test_real_sharing(self): + def test_cant_subclass_instance(self): class A: - def __init__(self): - self.x = 42 - A1, A2, A3 = A(), A(), A() - assert self.is_sharing(A3) - assert self.is_sharing(A2) - assert self.is_sharing(A1) + pass + try: + class B(type(A())): + pass + except TypeError: + pass + else: + assert 0, "should have raised" + + def test_dict_descriptor(self): + import sys + if not hasattr(sys, 'pypy_objspaceclass'): + skip("on CPython old-style instances don't have a __dict__ descriptor") + class A: + pass + a = A() + a.x = 1 + descr = type(a).__dict__['__dict__'] + assert descr.__get__(a) == {'x': 1} + descr.__set__(a, {'x': 2}) + assert a.x == 2 + raises(TypeError, descr.__delete__, a) + -class AppTestOldStyleModDict(object): +class AppTestOldStyleClassStrDict(object): def setup_class(cls): if option.runappdirect: py.test.skip("can only be run on py.py") @@ -966,3 +970,22 @@ a = 1 b = 2 assert self.is_strdict(A) + +class AppTestOldStyleMapDict(AppTestOldstyle): + def setup_class(cls): + cls.space = gettestobjspace(**{"objspace.std.withmapdict": True}) + if option.runappdirect: + py.test.skip("can only be run on py.py") + def has_mapdict(space, w_inst): + return space.wrap(w_inst._get_mapdict_map() is not None) + cls.w_has_mapdict = cls.space.wrap(gateway.interp2app(has_mapdict)) + + + def test_has_mapdict(self): + class A: + def __init__(self): + self.x = 42 + a = A() + assert a.x == 42 + assert self.has_mapdict(a) + Modified: pypy/branch/arm-backend/pypy/module/__pypy__/__init__.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/__pypy__/__init__.py (original) +++ pypy/branch/arm-backend/pypy/module/__pypy__/__init__.py Tue Nov 23 10:12:47 2010 @@ -23,5 +23,12 @@ 'interp_magic.method_cache_counter') self.extra_interpdef('reset_method_cache_counter', 'interp_magic.reset_method_cache_counter') + if self.space.config.objspace.std.withmapdict: + self.extra_interpdef('mapdict_cache_counter', + 'interp_magic.mapdict_cache_counter') PYC_MAGIC = get_pyc_magic(self.space) self.extra_interpdef('PYC_MAGIC', 'space.wrap(%d)' % PYC_MAGIC) + # + from pypy.jit.backend import detect_cpu + model = detect_cpu.autodetect_main_model_and_size() + self.extra_interpdef('cpumodel', 'space.wrap(%r)' % model) Modified: pypy/branch/arm-backend/pypy/module/__pypy__/interp_magic.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/__pypy__/interp_magic.py (original) +++ pypy/branch/arm-backend/pypy/module/__pypy__/interp_magic.py Tue Nov 23 10:12:47 2010 @@ -2,6 +2,7 @@ from pypy.interpreter.gateway import ObjSpace from pypy.rlib.objectmodel import we_are_translated from pypy.objspace.std.typeobject import MethodCache +from pypy.objspace.std.mapdict import IndexCache def internal_repr(space, w_object): return space.wrap('%r' % (w_object,)) @@ -36,4 +37,17 @@ cache = space.fromcache(MethodCache) cache.misses = {} cache.hits = {} - + if space.config.objspace.std.withmapdict: + cache = space.fromcache(IndexCache) + cache.misses = {} + cache.hits = {} + +def mapdict_cache_counter(space, name): + """Return a tuple (index_cache_hits, index_cache_misses) for lookups + in the mapdict cache with the given attribute name.""" + assert space.config.objspace.std.withmethodcachecounter + assert space.config.objspace.std.withmapdict + cache = space.fromcache(IndexCache) + return space.newtuple([space.newint(cache.hits.get(name, 0)), + space.newint(cache.misses.get(name, 0))]) +mapdict_cache_counter.unwrap_spec = [ObjSpace, str] Modified: pypy/branch/arm-backend/pypy/module/__pypy__/test/test_special.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/__pypy__/test/test_special.py (original) +++ pypy/branch/arm-backend/pypy/module/__pypy__/test/test_special.py Tue Nov 23 10:12:47 2010 @@ -17,3 +17,7 @@ from __pypy__ import isfake import select assert isfake(select) + + def test_cpumodel(self): + import __pypy__ + assert hasattr(__pypy__, 'cpumodel') Modified: pypy/branch/arm-backend/pypy/module/_rawffi/__init__.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/_rawffi/__init__.py (original) +++ pypy/branch/arm-backend/pypy/module/_rawffi/__init__.py Tue Nov 23 10:12:47 2010 @@ -1,5 +1,5 @@ -""" Low-level interface to libffi +""" Low-level interface to clibffi """ from pypy.interpreter.mixedmodule import MixedModule @@ -39,11 +39,11 @@ if hasattr(interp_rawffi, 'check_HRESULT'): Module.interpleveldefs['check_HRESULT'] = 'interp_rawffi.check_HRESULT' - from pypy.rlib import libffi + from pypy.rlib import clibffi for name in ['FUNCFLAG_STDCALL', 'FUNCFLAG_CDECL', 'FUNCFLAG_PYTHONAPI', ]: - if hasattr(libffi, name): - Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(libffi, name) + if hasattr(clibffi, name): + Module.interpleveldefs[name] = "space.wrap(%r)" % getattr(clibffi, name) super(Module, cls).buildloaders() buildloaders = classmethod(buildloaders) Modified: pypy/branch/arm-backend/pypy/module/_rawffi/array.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/_rawffi/array.py (original) +++ pypy/branch/arm-backend/pypy/module/_rawffi/array.py Tue Nov 23 10:12:47 2010 @@ -97,7 +97,15 @@ class W_ArrayInstance(W_DataInstance): def __init__(self, space, shape, length, address=r_uint(0)): - W_DataInstance.__init__(self, space, shape.size * length, address) + # Workaround for a strange behavior of libffi: make sure that + # we always have at least 8 bytes. For W_ArrayInstances that are + # used as the result value of a function call, ffi_call() writes + # 8 bytes into it even if the function's result type asks for less. + # This strange behavior is documented. + memsize = shape.size * length + if memsize < 8: + memsize = 8 + W_DataInstance.__init__(self, space, memsize, address) self.length = length self.shape = shape Modified: pypy/branch/arm-backend/pypy/module/_rawffi/callback.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/_rawffi/callback.py (original) +++ pypy/branch/arm-backend/pypy/module/_rawffi/callback.py Tue Nov 23 10:12:47 2010 @@ -8,8 +8,8 @@ from pypy.module._rawffi.array import get_elem, push_elem from pypy.module._rawffi.interp_rawffi import W_DataInstance, letter2tp, \ wrap_value, unwrap_value, unwrap_truncate_int -from pypy.rlib.libffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL -from pypy.rlib.libffi import ffi_type_void +from pypy.rlib.clibffi import USERDATA_P, CallbackFuncPtr, FUNCFLAG_CDECL +from pypy.rlib.clibffi import ffi_type_void from pypy.module._rawffi.tracker import tracker from pypy.interpreter.error import OperationError from pypy.interpreter import gateway Modified: pypy/branch/arm-backend/pypy/module/_rawffi/interp_rawffi.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/_rawffi/interp_rawffi.py (original) +++ pypy/branch/arm-backend/pypy/module/_rawffi/interp_rawffi.py Tue Nov 23 10:12:47 2010 @@ -5,7 +5,7 @@ from pypy.interpreter.gateway import interp2app, NoneNotWrapped from pypy.interpreter.typedef import TypeDef, GetSetProperty -from pypy.rlib.libffi import * +from pypy.rlib.clibffi import * from pypy.rpython.lltypesystem import lltype, rffi from pypy.rlib.unroll import unrolling_iterable Modified: pypy/branch/arm-backend/pypy/module/_rawffi/structure.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/_rawffi/structure.py (original) +++ pypy/branch/arm-backend/pypy/module/_rawffi/structure.py Tue Nov 23 10:12:47 2010 @@ -15,7 +15,7 @@ from pypy.module._rawffi.interp_rawffi import wrap_value, unwrap_value from pypy.module._rawffi.interp_rawffi import unpack_shape_with_length from pypy.module._rawffi.interp_rawffi import size_alignment -from pypy.rlib import libffi +from pypy.rlib import clibffi from pypy.rlib.rarithmetic import intmask, r_uint def unpack_fields(space, w_fields): @@ -34,7 +34,7 @@ def round_up(size, alignment): return (size + alignment - 1) & -alignment -def size_alignment_pos(fields): +def size_alignment_pos(fields, is_union=False): size = 0 alignment = 1 pos = [] @@ -42,16 +42,20 @@ # fieldtype is a W_Array fieldsize = fieldtype.size fieldalignment = fieldtype.alignment - size = round_up(size, fieldalignment) alignment = max(alignment, fieldalignment) - pos.append(size) - size += intmask(fieldsize) + if is_union: + pos.append(0) + size = max(size, fieldsize) + else: + size = round_up(size, fieldalignment) + pos.append(size) + size += intmask(fieldsize) size = round_up(size, alignment) return size, alignment, pos class W_Structure(W_DataShape): - def __init__(self, space, fields, size, alignment): + def __init__(self, space, fields, size, alignment, is_union=False): name_to_index = {} if fields is not None: for i in range(len(fields)): @@ -60,7 +64,7 @@ raise operationerrfmt(space.w_ValueError, "duplicate field name %s", name) name_to_index[name] = i - size, alignment, pos = size_alignment_pos(fields) + size, alignment, pos = size_alignment_pos(fields, is_union) else: # opaque case fields = [] pos = [] @@ -104,14 +108,14 @@ descr_fieldoffset.unwrap_spec = ['self', ObjSpace, str] # get the corresponding ffi_type - ffi_struct = lltype.nullptr(libffi.FFI_STRUCT_P.TO) + ffi_struct = lltype.nullptr(clibffi.FFI_STRUCT_P.TO) def get_basic_ffi_type(self): if not self.ffi_struct: # Repeated fields are delicate. Consider for example # struct { int a[5]; } # or struct { struct {int x;} a[5]; } - # Seeing no corresponding doc in libffi, let's just repeat + # Seeing no corresponding doc in clibffi, let's just repeat # the field 5 times... fieldtypes = [] for name, tp in self.fields: @@ -122,7 +126,7 @@ while count + basic_size <= total_size: fieldtypes.append(basic_ffi_type) count += basic_size - self.ffi_struct = libffi.make_struct_ffitype_e(self.size, + self.ffi_struct = clibffi.make_struct_ffitype_e(self.size, self.alignment, fieldtypes) return self.ffi_struct.ffistruct @@ -133,15 +137,17 @@ -def descr_new_structure(space, w_type, w_shapeinfo): +def descr_new_structure(space, w_type, w_shapeinfo, union=0): + is_union = bool(union) if space.is_true(space.isinstance(w_shapeinfo, space.w_tuple)): w_size, w_alignment = space.fixedview(w_shapeinfo, expected_length=2) S = W_Structure(space, None, space.int_w(w_size), - space.int_w(w_alignment)) + space.int_w(w_alignment), is_union) else: fields = unpack_fields(space, w_shapeinfo) - S = W_Structure(space, fields, 0, 0) + S = W_Structure(space, fields, 0, 0, is_union) return space.wrap(S) +descr_new_structure.unwrap_spec = [ObjSpace, W_Root, W_Root, int] W_Structure.typedef = TypeDef( 'Structure', Modified: pypy/branch/arm-backend/pypy/module/_rawffi/test/test__rawffi.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/_rawffi/test/test__rawffi.py (original) +++ pypy/branch/arm-backend/pypy/module/_rawffi/test/test__rawffi.py Tue Nov 23 10:12:47 2010 @@ -70,7 +70,7 @@ return s[num]; } - char *char_check(char x, char y) + const char *char_check(char x, char y) { if (y == static_str[0]) return static_str; @@ -185,13 +185,14 @@ sum_x_y give perturb get_s2a check_s2a AAA_first_ordinal_function + ret_un_func """.split() eci = ExternalCompilationInfo(export_symbols=symbols) return str(platform.compile([c_file], eci, 'x', standalone=False)) prepare_c_example = staticmethod(prepare_c_example) def setup_class(cls): - from pypy.rlib.libffi import get_libc_name + from pypy.rlib.clibffi import get_libc_name space = gettestobjspace(usemodules=('_rawffi', 'struct')) cls.space = space cls.w_lib_name = space.wrap(cls.prepare_c_example()) @@ -295,6 +296,7 @@ assert _rawffi.charp2string(res[0]) is None arg1.free() arg2.free() + a.free() def test_raw_callable(self): import _rawffi @@ -945,14 +947,15 @@ assert a[4] == 't' def test_union(self): - skip("segfaulting") import _rawffi longsize = _rawffi.sizeof('l') - S = _rawffi.Structure((longsize, longsize)) + S = _rawffi.Structure([('x', 'h'), ('y', 'l')], union=True) s = S(autofree=False) + s.x = 12345 lib = _rawffi.CDLL(self.lib_name) f = lib.ptr('ret_un_func', [(S, 1)], (S, 1)) ret = f(s) + assert ret.y == 1234500, "ret.y == %d" % (ret.y,) s.free() class AppTestAutoFree: Modified: pypy/branch/arm-backend/pypy/module/_rawffi/test/test_nested.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/_rawffi/test/test_nested.py (original) +++ pypy/branch/arm-backend/pypy/module/_rawffi/test/test_nested.py Tue Nov 23 10:12:47 2010 @@ -107,7 +107,6 @@ assert S.fieldoffset('x') == 0 assert S.fieldoffset('ar') == A5alignment s = S() - s = S() s.x = 'G' raises(TypeError, 's.ar') assert s.fieldaddress('ar') == s.buffer + S.fieldoffset('ar') Modified: pypy/branch/arm-backend/pypy/module/_socket/test/test_sock_app.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/_socket/test/test_sock_app.py (original) +++ pypy/branch/arm-backend/pypy/module/_socket/test/test_sock_app.py Tue Nov 23 10:12:47 2010 @@ -254,6 +254,7 @@ s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) fd = s.fileno() w_obj = rsocket.make_address(c_addr, addrlen).as_object(fd, space) + lltype.free(c_addr_ll, flavor='raw') assert space.is_true(space.eq(w_obj, space.newtuple([ space.wrap('lo'), space.wrap(socket.ntohs(8)), Modified: pypy/branch/arm-backend/pypy/module/_sre/interp_sre.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/_sre/interp_sre.py (original) +++ pypy/branch/arm-backend/pypy/module/_sre/interp_sre.py Tue Nov 23 10:12:47 2010 @@ -290,10 +290,8 @@ def SRE_Pattern__new__(space, w_subtype, w_pattern, flags, w_code, groups=0, w_groupindex=None, w_indexgroup=None): n = space.int_w(space.len(w_code)) - code = [0] * n - for i in range(n): - x = space.uint_w(space.getitem(w_code, space.wrap(i))) - code[i] = intmask(x) + code = [intmask(space.uint_w(space.getitem(w_code, space.wrap(i)))) + for i in range(n)] # w_srepat = space.allocate_instance(W_SRE_Pattern, w_subtype) srepat = space.interp_w(W_SRE_Pattern, w_srepat) Modified: pypy/branch/arm-backend/pypy/module/_weakref/interp__weakref.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/_weakref/interp__weakref.py (original) +++ pypy/branch/arm-backend/pypy/module/_weakref/interp__weakref.py Tue Nov 23 10:12:47 2010 @@ -7,7 +7,7 @@ import weakref -class WeakrefLifeline(object): +class WeakrefLifeline(W_Root): def __init__(self, space): self.space = space # this is here for W_Root.clear_all_weakrefs() self.refs_weak = [] Modified: pypy/branch/arm-backend/pypy/module/_winreg/interp_winreg.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/_winreg/interp_winreg.py (original) +++ pypy/branch/arm-backend/pypy/module/_winreg/interp_winreg.py Tue Nov 23 10:12:47 2010 @@ -1,3 +1,4 @@ +from __future__ import with_statement from pypy.interpreter.baseobjspace import Wrappable from pypy.interpreter.baseobjspace import ObjSpace, W_Root from pypy.interpreter.gateway import interp2app @@ -212,13 +213,10 @@ subkey = None else: subkey = space.str_w(w_subkey) - dataptr = rffi.str2charp(value) - try: + with rffi.scoped_str2charp(value) as dataptr: ret = rwinreg.RegSetValue(hkey, subkey, rwinreg.REG_SZ, dataptr, len(value)) - finally: - rffi.free_charp(dataptr) - if ret != 0: - raiseWindowsError(space, ret, 'RegSetValue') + if ret != 0: + raiseWindowsError(space, ret, 'RegSetValue') SetValue.unwrap_spec = [ObjSpace, W_Root, W_Root, int, str] def QueryValue(space, w_hkey, w_subkey): @@ -237,23 +235,15 @@ subkey = None else: subkey = space.str_w(w_subkey) - bufsize_p = lltype.malloc(rwin32.PLONG.TO, 1, flavor='raw') - try: + with lltype.scoped_alloc(rwin32.PLONG.TO, 1) as bufsize_p: ret = rwinreg.RegQueryValue(hkey, subkey, None, bufsize_p) if ret != 0: raiseWindowsError(space, ret, 'RegQueryValue') - buf = lltype.malloc(rffi.CCHARP.TO, bufsize_p[0], flavor='raw') - try: + with lltype.scoped_alloc(rffi.CCHARP.TO, bufsize_p[0]) as buf: ret = rwinreg.RegQueryValue(hkey, subkey, buf, bufsize_p) if ret != 0: raiseWindowsError(space, ret, 'RegQueryValue') return space.wrap(rffi.charp2strn(buf, bufsize_p[0] - 1)) - finally: - lltype.free(buf, flavor='raw') - finally: - lltype.free(bufsize_p, flavor='raw') - if ret != 0: - raiseWindowsError(space, ret, 'RegQueryValue') QueryValue.unwrap_spec = [ObjSpace, W_Root, W_Root] def convert_to_regdata(space, w_value, typ): @@ -412,16 +402,14 @@ value_name is a string indicating the value to query""" hkey = hkey_w(w_hkey, space) null_dword = lltype.nullptr(rwin32.LPDWORD.TO) - retDataSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw') - try: + with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retDataSize: ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword, null_dword, None, retDataSize) if ret != 0: raiseWindowsError(space, ret, 'RegQueryValueEx') - databuf = lltype.malloc(rffi.CCHARP.TO, retDataSize[0], flavor='raw') - try: - retType = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw') - try: + + with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf: + with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType: ret = rwinreg.RegQueryValueEx(hkey, subkey, null_dword, retType, databuf, retDataSize) @@ -432,12 +420,6 @@ retDataSize[0], retType[0]), space.wrap(retType[0]), ]) - finally: - lltype.free(retType, flavor='raw') - finally: - lltype.free(databuf, flavor='raw') - finally: - lltype.free(retDataSize, flavor='raw') QueryValueEx.unwrap_spec = [ObjSpace, W_Root, str] @@ -454,14 +436,11 @@ The return value is the handle of the opened key. If the function fails, an exception is raised.""" hkey = hkey_w(w_hkey, space) - rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw') - try: + with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey: ret = rwinreg.RegCreateKey(hkey, subkey, rethkey) if ret != 0: raiseWindowsError(space, ret, 'CreateKey') return space.wrap(W_HKEY(rethkey[0])) - finally: - lltype.free(rethkey, flavor='raw') CreateKey.unwrap_spec = [ObjSpace, W_Root, str] def DeleteKey(space, w_hkey, subkey): @@ -504,14 +483,11 @@ The result is a new handle to the specified key If the function fails, an EnvironmentError exception is raised.""" hkey = hkey_w(w_hkey, space) - rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw') - try: + with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey: ret = rwinreg.RegOpenKeyEx(hkey, subkey, res, sam, rethkey) if ret != 0: raiseWindowsError(space, ret, 'RegOpenKeyEx') return space.wrap(W_HKEY(rethkey[0])) - finally: - lltype.free(rethkey, flavor='raw') OpenKey.unwrap_spec = [ObjSpace, W_Root, str, int, rffi.r_uint] def EnumValue(space, w_hkey, index): @@ -531,10 +507,8 @@ hkey = hkey_w(w_hkey, space) null_dword = lltype.nullptr(rwin32.LPDWORD.TO) - retValueSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw') - try: - retDataSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw') - try: + with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retValueSize: + with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retDataSize: ret = rwinreg.RegQueryInfoKey( hkey, None, null_dword, null_dword, null_dword, null_dword, null_dword, @@ -546,14 +520,9 @@ retValueSize[0] += 1 retDataSize[0] += 1 - valuebuf = lltype.malloc(rffi.CCHARP.TO, retValueSize[0], - flavor='raw') - try: - databuf = lltype.malloc(rffi.CCHARP.TO, retDataSize[0], - flavor='raw') - try: - retType = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw') - try: + with lltype.scoped_alloc(rffi.CCHARP.TO, retValueSize[0]) as valuebuf: + with lltype.scoped_alloc(rffi.CCHARP.TO, retDataSize[0]) as databuf: + with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retType: ret = rwinreg.RegEnumValue( hkey, index, valuebuf, retValueSize, null_dword, retType, databuf, retDataSize) @@ -566,16 +535,6 @@ retDataSize[0], retType[0]), space.wrap(retType[0]), ]) - finally: - lltype.free(retType, flavor='raw') - finally: - lltype.free(databuf, flavor='raw') - finally: - lltype.free(valuebuf, flavor='raw') - finally: - lltype.free(retDataSize, flavor='raw') - finally: - lltype.free(retValueSize, flavor='raw') EnumValue.unwrap_spec = [ObjSpace, W_Root, int] @@ -592,10 +551,8 @@ null_dword = lltype.nullptr(rwin32.LPDWORD.TO) # max key name length is 255 - buf = lltype.malloc(rffi.CCHARP.TO, 256, flavor='raw') - try: - retValueSize = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw') - try: + with lltype.scoped_alloc(rffi.CCHARP.TO, 256) as buf: + with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as retValueSize: retValueSize[0] = 256 # includes NULL terminator ret = rwinreg.RegEnumKeyEx(hkey, index, buf, retValueSize, null_dword, None, null_dword, @@ -603,10 +560,6 @@ if ret != 0: raiseWindowsError(space, ret, 'RegEnumKeyEx') return space.wrap(rffi.charp2str(buf)) - finally: - lltype.free(retValueSize, flavor='raw') - finally: - lltype.free(buf, flavor='raw') EnumKey.unwrap_spec = [ObjSpace, W_Root, int] @@ -621,12 +574,9 @@ A long integer that identifies when the key was last modified (if available) as 100's of nanoseconds since Jan 1, 1600.""" hkey = hkey_w(w_hkey, space) - nSubKeys = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw') - try: - nValues = lltype.malloc(rwin32.LPDWORD.TO, 1, flavor='raw') - try: - ft = lltype.malloc(rwin32.PFILETIME.TO, 1, flavor='raw') - try: + with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as nSubKeys: + with lltype.scoped_alloc(rwin32.LPDWORD.TO, 1) as nValues: + with lltype.scoped_alloc(rwin32.PFILETIME.TO, 1) as ft: null_dword = lltype.nullptr(rwin32.LPDWORD.TO) ret = rwinreg.RegQueryInfoKey( hkey, None, null_dword, null_dword, @@ -640,12 +590,6 @@ return space.newtuple([space.wrap(nSubKeys[0]), space.wrap(nValues[0]), space.wrap(l)]) - finally: - lltype.free(ft, flavor='raw') - finally: - lltype.free(nValues, flavor='raw') - finally: - lltype.free(nSubKeys, flavor='raw') QueryInfoKey.unwrap_spec = [ObjSpace, W_Root] def str_or_None_w(space, w_obj): @@ -666,12 +610,9 @@ If the function fails, an EnvironmentError exception is raised.""" machine = str_or_None_w(space, w_machine) hkey = hkey_w(w_hkey, space) - rethkey = lltype.malloc(rwinreg.PHKEY.TO, 1, flavor='raw') - try: + with lltype.scoped_alloc(rwinreg.PHKEY.TO, 1) as rethkey: ret = rwinreg.RegConnectRegistry(machine, hkey, rethkey) if ret != 0: raiseWindowsError(space, ret, 'RegConnectRegistry') return space.wrap(W_HKEY(rethkey[0])) - finally: - lltype.free(rethkey, flavor='raw') ConnectRegistry.unwrap_spec = [ObjSpace, W_Root, W_Root] Modified: pypy/branch/arm-backend/pypy/module/array/interp_array.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/array/interp_array.py (original) +++ pypy/branch/arm-backend/pypy/module/array/interp_array.py Tue Nov 23 10:12:47 2010 @@ -27,7 +27,7 @@ typecode = typecode[0] if space.is_w(w_cls, space.gettypeobject(W_ArrayBase.typedef)): - if len(w_args.keywords_w) > 0: + if w_args.keywords: # XXX this might be forbidden fishing msg = 'array.array() does not take keyword arguments' raise OperationError(space.w_TypeError, space.wrap(msg)) @@ -192,32 +192,30 @@ mytype.bytes raise OperationError(space.w_OverflowError, space.wrap(msg)) - elif mytype.unwrap == 'str_w' or mytype.unwrap == 'unicode_w': + return rffi.cast(mytype.itemtype, item) + if mytype.unwrap == 'str_w' or mytype.unwrap == 'unicode_w': if len(item) != 1: msg = 'array item must be char' raise OperationError(space.w_TypeError, space.wrap(msg)) item = item[0] - + return rffi.cast(mytype.itemtype, item) + # + # "regular" case: it fits in an rpython integer (lltype.Signed) + result = rffi.cast(mytype.itemtype, item) if mytype.canoverflow: - msg = None - if mytype.signed: - if item < -1 << (mytype.bytes * 8 - 1): + if rffi.cast(lltype.Signed, result) != item: + # overflow. build the correct message + if item < 0: msg = ('signed %d-byte integer is less than minimum' % mytype.bytes) - elif item > (1 << (mytype.bytes * 8 - 1)) - 1: + else: msg = ('signed %d-byte integer is greater than maximum' % mytype.bytes) - else: - if item < 0: - msg = ('unsigned %d-byte integer is less than minimum' - % mytype.bytes) - elif item > (1 << (mytype.bytes * 8)) - 1: - msg = ('unsigned %d-byte integer is greater' - ' than maximum' % mytype.bytes) - if msg is not None: + if not mytype.signed: + msg = 'un' + msg # 'signed' => 'unsigned' raise OperationError(space.w_OverflowError, space.wrap(msg)) - return rffi.cast(mytype.itemtype, item) + return result def __del__(self): self.setlen(0) Modified: pypy/branch/arm-backend/pypy/module/bz2/interp_bz2.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/bz2/interp_bz2.py (original) +++ pypy/branch/arm-backend/pypy/module/bz2/interp_bz2.py Tue Nov 23 10:12:47 2010 @@ -1,3 +1,4 @@ +from __future__ import with_statement from pypy.rpython.tool import rffi_platform as platform from pypy.rpython.lltypesystem import rffi from pypy.rpython.lltypesystem import lltype @@ -225,6 +226,11 @@ if self.current_size > 0: rffi.keep_buffer_alive_until_here(self.raw_buf, self.gc_buf) + def __enter__(self): + return self + def __exit__(self, *args): + self.free() + # ____________________________________________________________ # # Make the BZ2File type by internally inheriting from W_File. @@ -531,33 +537,30 @@ if not self.running: raise OperationError(self.space.w_ValueError, self.space.wrap("this object was already flushed")) - - out = OutBuffer(self.bzs) + in_bufsize = datasize - in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw') - for i in range(datasize): - in_buf[i] = data[i] - try: - - self.bzs.c_next_in = in_buf - rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize) + with OutBuffer(self.bzs) as out: + with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf: - while True: - bzerror = BZ2_bzCompress(self.bzs, BZ_RUN) - if bzerror != BZ_RUN_OK: - _catch_bz2_error(self.space, bzerror) + for i in range(datasize): + in_buf[i] = data[i] - if rffi.getintfield(self.bzs, 'c_avail_in') == 0: - break - elif rffi.getintfield(self.bzs, 'c_avail_out') == 0: - out.prepare_next_chunk() + self.bzs.c_next_in = in_buf + rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize) - res = out.make_result_string() - return self.space.wrap(res) - finally: - lltype.free(in_buf, flavor='raw') - out.free() + while True: + bzerror = BZ2_bzCompress(self.bzs, BZ_RUN) + if bzerror != BZ_RUN_OK: + _catch_bz2_error(self.space, bzerror) + + if rffi.getintfield(self.bzs, 'c_avail_in') == 0: + break + elif rffi.getintfield(self.bzs, 'c_avail_out') == 0: + out.prepare_next_chunk() + + res = out.make_result_string() + return self.space.wrap(res) compress.unwrap_spec = ['self', 'bufferstr'] @@ -566,9 +569,8 @@ raise OperationError(self.space.w_ValueError, self.space.wrap("this object was already flushed")) self.running = False - - out = OutBuffer(self.bzs) - try: + + with OutBuffer(self.bzs) as out: while True: bzerror = BZ2_bzCompress(self.bzs, BZ_FINISH) if bzerror == BZ_STREAM_END: @@ -581,8 +583,6 @@ res = out.make_result_string() return self.space.wrap(res) - finally: - out.free() flush.unwrap_spec = ['self'] W_BZ2Compressor.typedef = TypeDef("BZ2Compressor", @@ -641,38 +641,37 @@ if not self.running: raise OperationError(self.space.w_EOFError, self.space.wrap("end of stream was already found")) - + in_bufsize = len(data) - in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw') - for i in range(in_bufsize): - in_buf[i] = data[i] - out = OutBuffer(self.bzs) - try: + with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf: + for i in range(in_bufsize): + in_buf[i] = data[i] self.bzs.c_next_in = in_buf rffi.setintfield(self.bzs, 'c_avail_in', in_bufsize) - while True: - bzerror = BZ2_bzDecompress(self.bzs) - if bzerror == BZ_STREAM_END: - if rffi.getintfield(self.bzs, 'c_avail_in') != 0: - unused = [self.bzs.c_next_in[i] for i in range(rffi.getintfield(self.bzs, 'c_avail_in'))] - self.unused_data = "".join(unused) - self.running = False - break - if bzerror != BZ_OK: - _catch_bz2_error(self.space, bzerror) - - if rffi.getintfield(self.bzs, 'c_avail_in') == 0: - break - elif rffi.getintfield(self.bzs, 'c_avail_out') == 0: - out.prepare_next_chunk() + with OutBuffer(self.bzs) as out: + while True: + bzerror = BZ2_bzDecompress(self.bzs) + if bzerror == BZ_STREAM_END: + if rffi.getintfield(self.bzs, 'c_avail_in') != 0: + unused = [self.bzs.c_next_in[i] + for i in range( + rffi.getintfield(self.bzs, + 'c_avail_in'))] + self.unused_data = "".join(unused) + self.running = False + break + if bzerror != BZ_OK: + _catch_bz2_error(self.space, bzerror) + + if rffi.getintfield(self.bzs, 'c_avail_in') == 0: + break + elif rffi.getintfield(self.bzs, 'c_avail_out') == 0: + out.prepare_next_chunk() - res = out.make_result_string() - return self.space.wrap(res) - finally: - lltype.free(in_buf, flavor='raw') - out.free() + res = out.make_result_string() + return self.space.wrap(res) decompress.unwrap_spec = ['self', 'bufferstr'] @@ -695,43 +694,39 @@ if compresslevel < 1 or compresslevel > 9: raise OperationError(space.w_ValueError, space.wrap("compresslevel must be between 1 and 9")) - - bzs = lltype.malloc(bz_stream.TO, flavor='raw', zero=True) - in_bufsize = len(data) - # conforming to bz2 manual, this is large enough to fit compressed - # data in one shot. We will check it later anyway. - out = OutBuffer(bzs, in_bufsize + (in_bufsize / 100 + 1) + 600) - - in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw') - for i in range(in_bufsize): - in_buf[i] = data[i] - - try: - bzs.c_next_in = in_buf - rffi.setintfield(bzs, 'c_avail_in', in_bufsize) - bzerror = BZ2_bzCompressInit(bzs, compresslevel, 0, 0) - if bzerror != BZ_OK: - _catch_bz2_error(space, bzerror) + with lltype.scoped_alloc(bz_stream.TO, zero=True) as bzs: + in_bufsize = len(data) + + with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf: + for i in range(in_bufsize): + in_buf[i] = data[i] + bzs.c_next_in = in_buf + rffi.setintfield(bzs, 'c_avail_in', in_bufsize) + + # conforming to bz2 manual, this is large enough to fit compressed + # data in one shot. We will check it later anyway. + with OutBuffer(bzs, + in_bufsize + (in_bufsize / 100 + 1) + 600) as out: + + bzerror = BZ2_bzCompressInit(bzs, compresslevel, 0, 0) + if bzerror != BZ_OK: + _catch_bz2_error(space, bzerror) - while True: - bzerror = BZ2_bzCompress(bzs, BZ_FINISH) - if bzerror == BZ_STREAM_END: - break - elif bzerror != BZ_FINISH_OK: + while True: + bzerror = BZ2_bzCompress(bzs, BZ_FINISH) + if bzerror == BZ_STREAM_END: + break + elif bzerror != BZ_FINISH_OK: + BZ2_bzCompressEnd(bzs) + _catch_bz2_error(space, bzerror) + + if rffi.getintfield(bzs, 'c_avail_out') == 0: + out.prepare_next_chunk() + + res = out.make_result_string() BZ2_bzCompressEnd(bzs) - _catch_bz2_error(space, bzerror) - - if rffi.getintfield(bzs, 'c_avail_out') == 0: - out.prepare_next_chunk() - - res = out.make_result_string() - BZ2_bzCompressEnd(bzs) - return space.wrap(res) - finally: - lltype.free(bzs, flavor='raw') - lltype.free(in_buf, flavor='raw') - out.free() + return space.wrap(res) compress.unwrap_spec = [ObjSpace, 'bufferstr', int] def decompress(space, data): @@ -744,40 +739,34 @@ if in_bufsize == 0: return space.wrap("") - bzs = lltype.malloc(bz_stream.TO, flavor='raw', zero=True) - in_buf = lltype.malloc(rffi.CCHARP.TO, in_bufsize, flavor='raw') - for i in range(in_bufsize): - in_buf[i] = data[i] - - out = OutBuffer(bzs) - try: - bzs.c_next_in = in_buf - rffi.setintfield(bzs, 'c_avail_in', in_bufsize) - - bzerror = BZ2_bzDecompressInit(bzs, 0, 0) - if bzerror != BZ_OK: - _catch_bz2_error(space, bzerror) - - while True: - bzerror = BZ2_bzDecompress(bzs) - if bzerror == BZ_STREAM_END: - break - if bzerror != BZ_OK: - BZ2_bzDecompressEnd(bzs) - _catch_bz2_error(space, bzerror) - - if rffi.getintfield(bzs, 'c_avail_in') == 0: + with lltype.scoped_alloc(bz_stream.TO, zero=True) as bzs: + with lltype.scoped_alloc(rffi.CCHARP.TO, in_bufsize) as in_buf: + for i in range(in_bufsize): + in_buf[i] = data[i] + bzs.c_next_in = in_buf + rffi.setintfield(bzs, 'c_avail_in', in_bufsize) + + with OutBuffer(bzs) as out: + bzerror = BZ2_bzDecompressInit(bzs, 0, 0) + if bzerror != BZ_OK: + _catch_bz2_error(space, bzerror) + + while True: + bzerror = BZ2_bzDecompress(bzs) + if bzerror == BZ_STREAM_END: + break + if bzerror != BZ_OK: + BZ2_bzDecompressEnd(bzs) + _catch_bz2_error(space, bzerror) + + if rffi.getintfield(bzs, 'c_avail_in') == 0: + BZ2_bzDecompressEnd(bzs) + raise OperationError(space.w_ValueError, space.wrap( + "couldn't find end of stream")) + elif rffi.getintfield(bzs, 'c_avail_out') == 0: + out.prepare_next_chunk() + + res = out.make_result_string() BZ2_bzDecompressEnd(bzs) - raise OperationError(space.w_ValueError, - space.wrap("couldn't find end of stream")) - elif rffi.getintfield(bzs, 'c_avail_out') == 0: - out.prepare_next_chunk() - - res = out.make_result_string() - BZ2_bzDecompressEnd(bzs) - return space.wrap(res) - finally: - lltype.free(bzs, flavor='raw') - lltype.free(in_buf, flavor='raw') - out.free() + return space.wrap(res) decompress.unwrap_spec = [ObjSpace, 'bufferstr'] Modified: pypy/branch/arm-backend/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/cpyext/api.py (original) +++ pypy/branch/arm-backend/pypy/module/cpyext/api.py Tue Nov 23 10:12:47 2010 @@ -226,7 +226,7 @@ def unwrapper(space, *args): from pypy.module.cpyext.pyobject import Py_DecRef from pypy.module.cpyext.pyobject import make_ref, from_ref - from pypy.module.cpyext.pyobject import BorrowPair + from pypy.module.cpyext.pyobject import Reference newargs = () to_decref = [] assert len(args) == len(api_function.argtypes) @@ -270,8 +270,8 @@ return api_function.error_value if res is None: return None - elif isinstance(res, BorrowPair): - return res.w_borrowed + elif isinstance(res, Reference): + return res.get_wrapped(space) else: return res finally: @@ -473,7 +473,7 @@ @specialize.ll() def wrapper(*args): from pypy.module.cpyext.pyobject import make_ref, from_ref - from pypy.module.cpyext.pyobject import BorrowPair + from pypy.module.cpyext.pyobject import Reference # we hope that malloc removal removes the newtuple() that is # inserted exactly here by the varargs specializer llop.gc_stack_bottom(lltype.Void) # marker for trackgcroot.py @@ -525,7 +525,7 @@ elif is_PyObject(callable.api_func.restype): if result is None: retval = make_ref(space, None) - elif isinstance(result, BorrowPair): + elif isinstance(result, Reference): retval = result.get_ref(space) elif not rffi._isllptr(result): retval = rffi.cast(callable.api_func.restype, @@ -908,8 +908,10 @@ from pypy.rlib import rdynload try: ll_libname = rffi.str2charp(path) - dll = rdynload.dlopen(ll_libname) - lltype.free(ll_libname, flavor='raw') + try: + dll = rdynload.dlopen(ll_libname) + finally: + lltype.free(ll_libname, flavor='raw') except rdynload.DLOpenError, e: raise operationerrfmt( space.w_ImportError, Modified: pypy/branch/arm-backend/pypy/module/cpyext/cdatetime.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/cpyext/cdatetime.py (original) +++ pypy/branch/arm-backend/pypy/module/cpyext/cdatetime.py Tue Nov 23 10:12:47 2010 @@ -4,7 +4,7 @@ from pypy.module.cpyext.api import ( cpython_api, CANNOT_FAIL, cpython_struct, PyObjectFields) from pypy.module.cpyext.import_ import PyImport_Import -from pypy.module.cpyext.typeobject import PyTypeObjectPtr +from pypy.module.cpyext.typeobject import PyTypeObjectPtr, render_immortal from pypy.module.cpyext.state import State from pypy.interpreter.error import OperationError from pypy.tool.sourcetools import func_renamer @@ -22,25 +22,34 @@ @cpython_api([], lltype.Ptr(PyDateTime_CAPI), error=lltype.nullptr(PyDateTime_CAPI)) def _PyDateTime_Import(space): - datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw') + datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw', + track_allocation=False) if not we_are_translated(): datetimeAPI_dealloc(space) space.fromcache(State).datetimeAPI = datetimeAPI w_datetime = PyImport_Import(space, space.wrap("datetime")) + w_type = space.getattr(w_datetime, space.wrap("date")) datetimeAPI.c_DateType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DateType, w_type) + w_type = space.getattr(w_datetime, space.wrap("datetime")) datetimeAPI.c_DateTimeType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DateTimeType, w_type) + w_type = space.getattr(w_datetime, space.wrap("time")) datetimeAPI.c_TimeType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_TimeType, w_type) + w_type = space.getattr(w_datetime, space.wrap("timedelta")) datetimeAPI.c_DeltaType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DeltaType, w_type) return datetimeAPI Modified: pypy/branch/arm-backend/pypy/module/cpyext/classobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/cpyext/classobject.py (original) +++ pypy/branch/arm-backend/pypy/module/cpyext/classobject.py Tue Nov 23 10:12:47 2010 @@ -15,16 +15,20 @@ class is the class of new object. The dict parameter will be used as the object's __dict__; if NULL, a new dictionary will be created for the instance.""" - if not PyClass_Check(space, w_class): + if not isinstance(w_class, W_ClassObject): return PyErr_BadInternalCall(space) - return W_InstanceObject(space, w_class, w_dict) + w_result = w_class.instantiate(space) + if w_dict is not None: + w_result.setdict(space, w_dict) + return w_result @cpython_api([PyObject, PyObject], PyObject, error=CANNOT_FAIL) def _PyInstance_Lookup(space, w_instance, w_name): + name = space.str_w(w_name) assert isinstance(w_instance, W_InstanceObject) - w_result = space.finditem(w_instance.w_dict, w_name) + w_result = w_instance.getdictvalue(space, name) if w_result is not None: return w_result - return w_instance.w_class.lookup(space, w_name) + return w_instance.w_class.lookup(space, name) Modified: pypy/branch/arm-backend/pypy/module/cpyext/presetup.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/cpyext/presetup.py (original) +++ pypy/branch/arm-backend/pypy/module/cpyext/presetup.py Tue Nov 23 10:12:47 2010 @@ -21,6 +21,8 @@ from pypy.conftest import gettestobjspace from pypy.module.cpyext.api import build_bridge +from pypy.module.imp.importing import get_so_extension + usemodules = ['cpyext', 'thread'] if sys.platform == 'win32': usemodules.append('_winreg') # necessary in distutils @@ -35,6 +37,7 @@ def patch_distutils(): sysconfig.get_python_inc = get_python_inc + sysconfig.get_config_vars()['SO'] = get_so_extension(space) patch_distutils() Modified: pypy/branch/arm-backend/pypy/module/cpyext/pyobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/cpyext/pyobject.py (original) +++ pypy/branch/arm-backend/pypy/module/cpyext/pyobject.py Tue Nov 23 10:12:47 2010 @@ -424,7 +424,18 @@ state = space.fromcache(RefcountState) return state.make_borrowed(w_container, w_borrowed) -class BorrowPair: +class Reference: + def __init__(self, pyobj): + assert not isinstance(pyobj, W_Root) + self.pyobj = pyobj + + def get_ref(self, space): + return self.pyobj + + def get_wrapped(self, space): + return from_ref(space, self.pyobj) + +class BorrowPair(Reference): """ Delays the creation of a borrowed reference. """ @@ -435,6 +446,9 @@ def get_ref(self, space): return make_borrowed_ref(space, self.w_container, self.w_borrowed) + def get_wrapped(self, space): + return self.w_borrowed + def borrow_from(container, borrowed): return BorrowPair(container, borrowed) Modified: pypy/branch/arm-backend/pypy/module/cpyext/test/test_borrow.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/cpyext/test/test_borrow.py (original) +++ pypy/branch/arm-backend/pypy/module/cpyext/test/test_borrow.py Tue Nov 23 10:12:47 2010 @@ -31,6 +31,7 @@ g = PyTuple_GetItem(t, 0); // borrows reference again printf("Refcnt4: %i\\n", f->ob_refcnt); printf("COMPARE: %i\\n", f == g); + fflush(stdout); Py_DECREF(t); Py_RETURN_TRUE; """), Modified: pypy/branch/arm-backend/pypy/module/cpyext/test/test_cpyext.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/cpyext/test/test_cpyext.py (original) +++ pypy/branch/arm-backend/pypy/module/cpyext/test/test_cpyext.py Tue Nov 23 10:12:47 2010 @@ -16,6 +16,7 @@ from pypy.module.cpyext.pyobject import Py_DecRef, InvalidPointerException from pypy.translator.goal import autopath from pypy.tool.identity_dict import identity_dict +from pypy.tool import leakfinder @api.cpython_api([], api.PyObject) def PyPy_Crash1(space): @@ -41,7 +42,7 @@ raises(ImportError, cpyext.load_module, "missing.file", "foo") raises(ImportError, cpyext.load_module, self.libc, "invalid.function") -def compile_module(modname, **kwds): +def compile_module(space, modname, **kwds): """ Build an extension module and return the filename of the resulting native code file. @@ -64,10 +65,8 @@ [], eci, outputfilename=str(dirname/modname), standalone=False) - if sys.platform == 'win32': - pydname = soname.new(purebasename=modname, ext='.pyd') - else: - pydname = soname.new(purebasename=modname, ext='.so') + from pypy.module.imp.importing import get_so_extension + pydname = soname.new(purebasename=modname, ext=get_so_extension(space)) soname.rename(pydname) return str(pydname) @@ -78,7 +77,6 @@ self.frozen_refcounts[w_obj] = obj.c_ob_refcnt #state.print_refcounts() self.frozen_ll2callocations = set(ll2ctypes.ALLOCATED.values()) - lltype.start_tracking_allocations() class LeakCheckingTest(object): def check_and_print_leaks(self): @@ -126,17 +124,8 @@ for w_obj in lost_objects_w: print >>sys.stderr, "Lost object %r" % (w_obj, ) leaking = True - for llvalue in set(ll2ctypes.ALLOCATED.values()) - self.frozen_ll2callocations: - if getattr(llvalue, "_traceback", None): # this means that the allocation should be tracked - leaking = True - print >>sys.stderr, "Did not deallocate %r (ll2ctypes)" % (llvalue, ) - print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines()) - for llvalue in lltype.ALLOCATED.keys(): - leaking = True - print >>sys.stderr, "Did not deallocate %r (llvalue)" % (llvalue, ) - print >>sys.stderr, "\t" + "\n\t".join(llvalue._traceback.splitlines()) - - lltype.stop_tracking_allocations() + # the actual low-level leak checking is done by pypy.tool.leakfinder, + # enabled automatically by pypy.conftest. return leaking class AppTestCpythonExtensionBase(LeakCheckingTest): @@ -162,7 +151,7 @@ kwds["link_files"] = [str(api_library + '.so')] if sys.platform == 'linux2': kwds["compile_extra"]=["-Werror=implicit-function-declaration"] - return compile_module(name, **kwds) + return compile_module(self.space, name, **kwds) def import_module(self, name, init=None, body='', load_it=True, filename=None): Modified: pypy/branch/arm-backend/pypy/module/cpyext/test/test_unicodeobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/cpyext/test/test_unicodeobject.py (original) +++ pypy/branch/arm-backend/pypy/module/cpyext/test/test_unicodeobject.py Tue Nov 23 10:12:47 2010 @@ -177,13 +177,14 @@ encoded_charp = rffi.str2charp(encoded) strict_charp = rffi.str2charp("strict") if endian is not None: - pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw') if endian < 0: - pendian[0] = -1 + value = -1 elif endian > 0: - pendian[0] = 1 + value = 1 else: - pendian[0] = 0 + value = 0 + pendian = lltype.malloc(rffi.INTP.TO, 1, flavor='raw') + pendian[0] = rffi.cast(rffi.INT, value) else: pendian = None Modified: pypy/branch/arm-backend/pypy/module/cpyext/typeobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/cpyext/typeobject.py (original) +++ pypy/branch/arm-backend/pypy/module/cpyext/typeobject.py Tue Nov 23 10:12:47 2010 @@ -182,10 +182,10 @@ subtype = rffi.cast(PyTypeObjectPtr, make_ref(space, w_subtype)) try: - obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds) + w_obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds) finally: Py_DecRef(space, w_subtype) - return obj + return w_obj @specialize.memo() def get_new_method_def(space): @@ -193,10 +193,14 @@ if state.new_method_def: return state.new_method_def from pypy.module.cpyext.modsupport import PyMethodDef - ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True) + ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True, + immortal=True) ptr.c_ml_name = rffi.str2charp("__new__") + lltype.render_immortal(ptr.c_ml_name) rffi.setintfield(ptr, 'c_ml_flags', METH_VARARGS | METH_KEYWORDS) - ptr.c_ml_doc = rffi.str2charp("T.__new__(S, ...) -> a new object with type S, a subtype of T") + ptr.c_ml_doc = rffi.str2charp( + "T.__new__(S, ...) -> a new object with type S, a subtype of T") + lltype.render_immortal(ptr.c_ml_doc) state.new_method_def = ptr return ptr @@ -429,6 +433,12 @@ finish_type_1(space, pto) finish_type_2(space, pto, w_type) + if space.type(w_type).is_cpytype(): + # XXX Types with a C metatype are never freed, try to see why... + render_immortal(pto, w_type) + lltype.render_immortal(pto) + lltype.render_immortal(pto.c_tp_name) + pto.c_tp_basicsize = rffi.sizeof(typedescr.basestruct) if pto.c_tp_base: if pto.c_tp_base.c_tp_basicsize > pto.c_tp_basicsize: @@ -534,12 +544,25 @@ w_obj.ready() finish_type_2(space, py_type, w_obj) + render_immortal(py_type, w_obj) state = space.fromcache(RefcountState) state.non_heaptypes_w.append(w_obj) return w_obj +def render_immortal(py_type, w_obj): + lltype.render_immortal(py_type.c_tp_bases) + lltype.render_immortal(py_type.c_tp_mro) + + assert isinstance(w_obj, W_TypeObject) + if w_obj.is_cpytype(): + lltype.render_immortal(py_type.c_tp_dict) + else: + lltype.render_immortal(py_type.c_tp_name) + if not w_obj.is_cpytype() and w_obj.is_heaptype(): + lltype.render_immortal(py_type) + def finish_type_1(space, pto): """ Sets up tp_bases, necessary before creating the interpreter type. Modified: pypy/branch/arm-backend/pypy/module/gc/__init__.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/gc/__init__.py (original) +++ pypy/branch/arm-backend/pypy/module/gc/__init__.py Tue Nov 23 10:12:47 2010 @@ -29,6 +29,7 @@ 'get_referents': 'referents.get_referents', 'get_referrers': 'referents.get_referrers', '_dump_rpy_heap': 'referents._dump_rpy_heap', + 'get_typeids_z': 'referents.get_typeids_z', 'GcRef': 'referents.W_GcRef', }) MixedModule.__init__(self, space, w_name) Modified: pypy/branch/arm-backend/pypy/module/gc/app_referents.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/gc/app_referents.py (original) +++ pypy/branch/arm-backend/pypy/module/gc/app_referents.py Tue Nov 23 10:12:47 2010 @@ -14,11 +14,25 @@ and [addr1]..[addrn] are addresses of other objects that this object points to. The full dump is a list of such objects, with a marker [0][0][0][-1] inserted after all GC roots, before all non-roots. + + If the argument is a filename and the 'zlib' module is available, + we also write a 'typeids.txt' in the same directory, if none exists. """ if isinstance(file, str): f = open(file, 'wb') gc._dump_rpy_heap(f.fileno()) f.close() + try: + import zlib, os + except ImportError: + pass + else: + filename2 = os.path.join(os.path.dirname(file), 'typeids.txt') + if not os.path.exists(filename2): + data = zlib.decompress(gc.get_typeids_z()) + f = open(filename2, 'wb') + f.write(data) + f.close() else: if isinstance(file, int): fd = file Modified: pypy/branch/arm-backend/pypy/module/gc/interp_gc.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/gc/interp_gc.py (original) +++ pypy/branch/arm-backend/pypy/module/gc/interp_gc.py Tue Nov 23 10:12:47 2010 @@ -5,6 +5,15 @@ def collect(space): "Run a full collection." + # First clear the method cache. See test_gc for an example of why. + if space.config.objspace.std.withmethodcache: + from pypy.objspace.std.typeobject import MethodCache + cache = space.fromcache(MethodCache) + cache.clear() + if space.config.objspace.std.withmapdict: + from pypy.objspace.std.mapdict import IndexCache + cache = space.fromcache(IndexCache) + cache.clear() rgc.collect() return space.wrap(0) Modified: pypy/branch/arm-backend/pypy/module/gc/referents.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/gc/referents.py (original) +++ pypy/branch/arm-backend/pypy/module/gc/referents.py Tue Nov 23 10:12:47 2010 @@ -15,8 +15,16 @@ def try_cast_gcref_to_w_root(gcref): w_obj = rgc.try_cast_gcref_to_instance(W_Root, gcref) - if not we_are_translated() and not hasattr(w_obj, 'typedef'): - w_obj = None + # Ignore the instances of W_Root that are not really valid as Python + # objects. There is e.g. WeakrefLifeline in module/_weakref that + # inherits from W_Root for internal reasons. Such instances don't + # have a typedef at all (or have a null typedef after translation). + if not we_are_translated(): + if not hasattr(w_obj, 'typedef'): + return None + else: + if w_obj is None or not w_obj.typedef: + return None return w_obj def wrap(space, gcref): @@ -169,3 +177,9 @@ if not ok: raise missing_operation(space) _dump_rpy_heap.unwrap_spec = [ObjSpace, int] + +def get_typeids_z(space): + a = rgc.get_typeids_z() + s = ''.join([a[i] for i in range(len(a))]) + return space.wrap(s) +get_typeids_z.unwrap_spec = [ObjSpace] Modified: pypy/branch/arm-backend/pypy/module/gc/test/test_gc.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/gc/test/test_gc.py (original) +++ pypy/branch/arm-backend/pypy/module/gc/test/test_gc.py Tue Nov 23 10:12:47 2010 @@ -103,3 +103,49 @@ import gc gc.dump_heap_stats(self.fname) + +class AppTestGcMethodCache(object): + def setup_class(cls): + cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True}) + + def test_clear_method_cache(self): + import gc, weakref + rlist = [] + def f(): + class C(object): + def f(self): + pass + C().f() # Fill the method cache + rlist.append(weakref.ref(C)) + for i in range(10): + f() + gc.collect() # the classes C should all go away here + # the last class won't go in mapdict, as long as the code object of f + # is around + rlist.pop() + for r in rlist: + assert r() is None + +class AppTestGcMapDictIndexCache(AppTestGcMethodCache): + def setup_class(cls): + cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True, + "objspace.std.withmapdict": True}) + + + def test_clear_index_cache(self): + import gc, weakref + rlist = [] + def f(): + class C(object): + def f(self): + pass + c = C() + c.x = 1 + getattr(c, "x") # fill the index cache without using the local cache + getattr(c, "x") + rlist.append(weakref.ref(C)) + for i in range(5): + f() + gc.collect() # the classes C should all go away here + for r in rlist: + assert r() is None Modified: pypy/branch/arm-backend/pypy/module/imp/importing.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/imp/importing.py (original) +++ pypy/branch/arm-backend/pypy/module/imp/importing.py Tue Nov 23 10:12:47 2010 @@ -12,7 +12,7 @@ from pypy.rlib import streamio, jit from pypy.rlib.streamio import StreamErrors from pypy.rlib.rarithmetic import intmask -from pypy.rlib.objectmodel import we_are_translated +from pypy.rlib.objectmodel import we_are_translated, specialize SEARCH_ERROR = 0 PY_SOURCE = 1 @@ -25,10 +25,26 @@ # PY_CODERESOURCE = 8 IMP_HOOK = 9 -if sys.platform.startswith('win'): - so_extension = ".pyd" +if sys.platform == 'win32': + SO = ".pyd" else: - so_extension = ".so" + SO = ".so" +DEFAULT_SOABI = 'pypy-14' + + at specialize.memo() +def get_so_extension(space): + if space.config.objspace.soabi is not None: + soabi = space.config.objspace.soabi + else: + soabi = DEFAULT_SOABI + + if not soabi: + return SO + + if not space.config.translating: + soabi += 'i' + + return '.' + soabi + SO def find_modtype(space, filepart): """Check which kind of module to import for the given filepart, @@ -53,6 +69,7 @@ return PY_COMPILED, ".pyc", "rb" if space.config.objspace.usemodules.cpyext: + so_extension = get_so_extension(space) pydfile = filepart + so_extension if os.path.exists(pydfile) and case_ok(pydfile): return C_EXTENSION, so_extension, "rb" @@ -122,7 +139,7 @@ n = len(ctxt_name_prefix_parts)-level+1 assert n>=0 ctxt_name_prefix_parts = ctxt_name_prefix_parts[:n] - if ctxt_w_path is None: # plain module + if ctxt_name_prefix_parts and ctxt_w_path is None: # plain module ctxt_name_prefix_parts.pop() if ctxt_name_prefix_parts: rel_modulename = '.'.join(ctxt_name_prefix_parts) Modified: pypy/branch/arm-backend/pypy/module/imp/interp_imp.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/imp/interp_imp.py (original) +++ pypy/branch/arm-backend/pypy/module/imp/interp_imp.py Tue Nov 23 10:12:47 2010 @@ -8,10 +8,16 @@ def get_suffixes(space): w = space.wrap - return space.newlist([ + suffixes_w = [] + if space.config.objspace.usemodules.cpyext: + suffixes_w.append( + space.newtuple([w(importing.get_so_extension(space)), + w('rb'), w(importing.C_EXTENSION)])) + suffixes_w.extend([ space.newtuple([w('.py'), w('U'), w(importing.PY_SOURCE)]), space.newtuple([w('.pyc'), w('rb'), w(importing.PY_COMPILED)]), ]) + return space.newlist(suffixes_w) def get_magic(space): x = importing.get_pyc_magic(space) Modified: pypy/branch/arm-backend/pypy/module/imp/test/test_app.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/imp/test/test_app.py (original) +++ pypy/branch/arm-backend/pypy/module/imp/test/test_app.py Tue Nov 23 10:12:47 2010 @@ -47,6 +47,9 @@ elif mode == self.imp.PY_COMPILED: assert suffix in ('.pyc', '.pyo') assert type == 'rb' + elif mode == self.imp.C_EXTENSION: + assert suffix.endswith(('.pyd', '.so')) + assert type == 'rb' def test_obscure_functions(self): Modified: pypy/branch/arm-backend/pypy/module/imp/test/test_import.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/imp/test/test_import.py (original) +++ pypy/branch/arm-backend/pypy/module/imp/test/test_import.py Tue Nov 23 10:12:47 2010 @@ -360,6 +360,12 @@ """.rstrip() raises(ValueError, imp) + def test_future_relative_import_error_when_in_non_package2(self): + exec """def imp(): + from .. import inpackage + """.rstrip() + raises(ValueError, imp) + def test_relative_import_with___name__(self): import sys mydict = {'__name__': 'sys.foo'} @@ -467,6 +473,17 @@ except ImportError: pass +class TestAbi: + def test_abi_tag(self): + space1 = gettestobjspace(soabi='TEST') + space2 = gettestobjspace(soabi='') + if sys.platform == 'win32': + assert importing.get_so_extension(space1) == '.TESTi.pyd' + assert importing.get_so_extension(space2) == '.pyd' + else: + assert importing.get_so_extension(space1) == '.TESTi.so' + assert importing.get_so_extension(space2) == '.so' + def _getlong(data): x = marshal.dumps(data) return x[-4:] @@ -784,6 +801,7 @@ extrapath = udir.ensure("pythonpath", dir=1) extrapath.join("urllib.py").write("print 42\n") old = os.environ.get('PYTHONPATH', None) + oldlang = os.environ.pop('LANG', None) try: os.environ['PYTHONPATH'] = str(extrapath) output = py.process.cmdexec('''"%s" "%s" -c "import urllib"''' % @@ -792,6 +810,8 @@ finally: if old: os.environ['PYTHONPATH'] = old + if oldlang: + os.environ['LANG'] = oldlang class AppTestImportHooks(object): def test_meta_path(self): Modified: pypy/branch/arm-backend/pypy/module/parser/test/test_parser.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/parser/test/test_parser.py (original) +++ pypy/branch/arm-backend/pypy/module/parser/test/test_parser.py Tue Nov 23 10:12:47 2010 @@ -10,6 +10,9 @@ cls.w_m = space.appexec([], """(): import parser return parser""") + cls.w_symbol = space.appexec([], """(): + import symbol + return symbol""") class AppTestParser(ParserModuleTest): @@ -36,7 +39,7 @@ seq = getattr(s, meth)() assert isinstance(seq, tp) assert len(seq) == 4 - assert seq[0] == 286 + assert seq[0] == self.symbol.file_input assert len(seq[2]) == 2 assert len(seq[3]) == 2 assert seq[2][0] == 4 Modified: pypy/branch/arm-backend/pypy/module/posix/interp_posix.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/posix/interp_posix.py (original) +++ pypy/branch/arm-backend/pypy/module/posix/interp_posix.py Tue Nov 23 10:12:47 2010 @@ -454,7 +454,8 @@ self.w_environ = space.newdict() if _WIN: self.cryptProviderPtr = lltype.malloc( - rffi.CArray(HCRYPTPROV), 1, zero=True, flavor='raw') + rffi.CArray(HCRYPTPROV), 1, zero=True, + flavor='raw', immortal=True) def startup(self, space): _convertenviron(space, self.w_environ) def _freeze_(self): Modified: pypy/branch/arm-backend/pypy/module/pypyjit/__init__.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/pypyjit/__init__.py (original) +++ pypy/branch/arm-backend/pypy/module/pypyjit/__init__.py Tue Nov 23 10:12:47 2010 @@ -6,6 +6,7 @@ interpleveldefs = { 'set_param': 'interp_jit.set_param', + 'residual_call': 'interp_jit.residual_call', } def setup_after_space_initialization(self): @@ -14,6 +15,5 @@ # add the 'defaults' attribute from pypy.rlib.jit import PARAMETERS space = self.space - # XXX this is not really the default compiled into a pypy-c-jit XXX w_obj = space.wrap(PARAMETERS) space.setattr(space.wrap(self), space.wrap('defaults'), w_obj) Modified: pypy/branch/arm-backend/pypy/module/pypyjit/interp_jit.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/pypyjit/interp_jit.py (original) +++ pypy/branch/arm-backend/pypy/module/pypyjit/interp_jit.py Tue Nov 23 10:12:47 2010 @@ -5,10 +5,11 @@ from pypy.tool.pairtype import extendabletype from pypy.rlib.rarithmetic import r_uint, intmask -from pypy.rlib.jit import JitDriver, hint, we_are_jitted +from pypy.rlib.jit import JitDriver, hint, we_are_jitted, dont_look_inside +from pypy.rlib.jit import current_trace_length import pypy.interpreter.pyopcode # for side-effects from pypy.interpreter.error import OperationError, operationerrfmt -from pypy.interpreter.gateway import ObjSpace, Arguments +from pypy.interpreter.gateway import ObjSpace, Arguments, W_Root from pypy.interpreter.pycode import PyCode, CO_GENERATOR from pypy.interpreter.pyframe import PyFrame from pypy.interpreter.pyopcode import ExitFrame @@ -80,9 +81,22 @@ def jump_absolute(self, jumpto, _, ec=None): if we_are_jitted(): + # Normally, the tick counter is decremented by 100 for every + # Python opcode. Here, to better support JIT compilation of + # small loops, we decrement it by a possibly smaller constant. + # We get the maximum 100 when the (unoptimized) trace length + # is at least 3200 (a bit randomly). + trace_length = r_uint(current_trace_length()) + decr_by = trace_length // 32 + if decr_by < 1: + decr_by = 1 + elif decr_by > 100: # also if current_trace_length() returned -1 + decr_by = 100 + # self.last_instr = intmask(jumpto) - ec.bytecode_trace(self) + ec.bytecode_trace(self, intmask(decr_by)) jumpto = r_uint(self.last_instr) + # pypyjitdriver.can_enter_jit(frame=self, ec=ec, next_instr=jumpto, pycode=self.getcode()) return jumpto @@ -131,3 +145,10 @@ "no JIT parameter '%s'", key) set_param.unwrap_spec = [ObjSpace, Arguments] + + at dont_look_inside +def residual_call(space, w_callable, args): + '''For testing. Invokes callable(...), but without letting + the JIT follow the call.''' + return space.call_args(w_callable, args) +residual_call.unwrap_spec = [ObjSpace, W_Root, Arguments] Modified: pypy/branch/arm-backend/pypy/module/pypyjit/policy.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/pypyjit/policy.py (original) +++ pypy/branch/arm-backend/pypy/module/pypyjit/policy.py Tue Nov 23 10:12:47 2010 @@ -6,12 +6,13 @@ if (modname == '__builtin__.operation' or modname == '__builtin__.abstractinst' or modname == '__builtin__.interp_classobj' or - modname == '__builtin__.functional'): + modname == '__builtin__.functional' or + modname == '__builtin__.descriptor'): return True if '.' in modname: modname, _ = modname.split('.', 1) if modname in ['pypyjit', 'signal', 'micronumpy', 'math', 'exceptions', - 'imp', 'sys', 'array']: + 'imp', 'sys', 'array', '_ffi']: return True return False @@ -19,7 +20,7 @@ # this function should never actually return True directly # but instead call the base implementation mod = func.__module__ or '?' - + if mod.startswith('pypy.objspace.'): # gc_id operation if func.__name__ == 'id__ANY': @@ -36,5 +37,5 @@ modname = mod[len('pypy.module.'):] if not self.look_inside_pypy_module(modname): return False - + return True Modified: pypy/branch/arm-backend/pypy/module/pypyjit/test/test_policy.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/pypyjit/test/test_policy.py (original) +++ pypy/branch/arm-backend/pypy/module/pypyjit/test/test_policy.py Tue Nov 23 10:12:47 2010 @@ -12,7 +12,7 @@ def test_rlocale(): from pypy.rlib.rlocale import setlocale - assert not pypypolicy.look_inside_function(setlocale) + assert not pypypolicy.look_inside_function(setlocale) def test_geninterp(): d = {'_geninterp_': True} @@ -28,6 +28,10 @@ from pypy.interpreter.pyparser import parser assert not pypypolicy.look_inside_function(parser.Grammar.__init__.im_func) +def test_property(): + from pypy.module.__builtin__.descriptor import W_Property + assert pypypolicy.look_inside_function(W_Property.get.im_func) + def test_pypy_module(): from pypy.module._random.interp_random import W_Random assert not pypypolicy.look_inside_function(W_Random.random) @@ -35,6 +39,7 @@ assert pypypolicy.look_inside_pypy_module('__builtin__.operation') assert pypypolicy.look_inside_pypy_module('__builtin__.abstractinst') assert pypypolicy.look_inside_pypy_module('__builtin__.functional') + assert pypypolicy.look_inside_pypy_module('__builtin__.descriptor') assert pypypolicy.look_inside_pypy_module('exceptions.interp_exceptions') for modname in 'pypyjit', 'signal', 'micronumpy', 'math', 'imp': assert pypypolicy.look_inside_pypy_module(modname) @@ -42,4 +47,3 @@ def test_see_jit_module(): assert pypypolicy.look_inside_pypy_module('pypyjit.interp_jit') - Modified: pypy/branch/arm-backend/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/arm-backend/pypy/module/pypyjit/test/test_pypy_c.py Tue Nov 23 10:12:47 2010 @@ -79,8 +79,11 @@ class PyPyCJITTests(object): - def run_source(self, source, expected_max_ops, *testcases): + def run_source(self, source, expected_max_ops, *testcases, **kwds): assert isinstance(expected_max_ops, int) + threshold = kwds.pop('threshold', 3) + if kwds: + raise TypeError, 'Unsupported keyword arguments: %s' % kwds.keys() source = py.code.Source(source) filepath = self.tmpdir.join('case%d.py' % self.counter) logfilepath = filepath.new(ext='.log') @@ -90,9 +93,12 @@ # some support code... print >> f, py.code.Source(""" import sys + # we don't want to see the small bridges created + # by the checkinterval reaching the limit + sys.setcheckinterval(10000000) try: # make the file runnable by CPython import pypyjit - pypyjit.set_param(threshold=3) + pypyjit.set_param(threshold=%d) except ImportError: pass @@ -102,7 +108,7 @@ print >> sys.stderr, 'got:', repr(result) assert result == expected assert type(result) is type(expected) - """) + """ % threshold) for testcase in testcases * 2: print >> f, "check(%r, %r)" % testcase print >> f, "print 'OK :-)'" @@ -116,6 +122,8 @@ result = child_stdout.read() child_stdout.close() assert result + if result.strip().startswith('SKIP:'): + py.test.skip(result.strip()) assert result.splitlines()[-1].strip() == 'OK :-)' self.parse_loops(logfilepath) self.print_loops() @@ -123,16 +131,18 @@ if self.total_ops > expected_max_ops: assert 0, "too many operations: got %d, expected maximum %d" % ( self.total_ops, expected_max_ops) + return result def parse_loops(self, opslogfile): - from pypy.jit.metainterp.test.oparser import parse + from pypy.jit.tool.oparser import parse from pypy.tool import logparser assert opslogfile.check() log = logparser.parse_log_file(str(opslogfile)) parts = logparser.extract_category(log, 'jit-log-opt-') + self.rawloops = [part for part in parts + if not from_entry_bridge(part, parts)] # skip entry bridges, they can contain random things - self.loops = [parse(part, no_namespace=True) for part in parts - if not from_entry_bridge(part, parts)] + self.loops = [parse(part, no_namespace=True) for part in self.rawloops] self.sliced_loops = [] # contains all bytecodes of all loops self.total_ops = 0 for loop in self.loops: @@ -156,12 +166,11 @@ return [ops for ops in self.sliced_loops if ops.bytecode == name] def print_loops(self): - for loop in self.loops: + for rawloop in self.rawloops: print print '@' * 79 print - for op in loop.operations: - print op + print rawloop.rstrip() print print '@' * 79 @@ -272,7 +281,7 @@ assert len(ops) == 2 assert not ops[0].get_opnames("call") assert not ops[0].get_opnames("new") - assert len(ops[0].get_opnames("guard")) <= 7 + assert len(ops[0].get_opnames("guard")) <= 2 assert not ops[1] # second LOOKUP_METHOD folded away ops = self.get_by_bytecode("CALL_METHOD") @@ -283,12 +292,16 @@ else: assert not bytecode.get_opnames("call") assert not bytecode.get_opnames("new") - assert len(bytecode.get_opnames("guard")) <= 9 + assert len(bytecode.get_opnames("guard")) <= 6 assert len(ops[1]) < len(ops[0]) ops = self.get_by_bytecode("LOAD_ATTR") assert len(ops) == 2 - assert ops[0].get_opnames() == ["getfield_gc", "getarrayitem_gc", + # With mapdict, we get fast access to (so far) the 5 first + # attributes, which means it is done with only the following + # operations. (For the other attributes there is additionally + # a getarrayitem_gc.) + assert ops[0].get_opnames() == ["getfield_gc", "guard_nonnull_class"] assert not ops[1] # second LOAD_ATTR folded away @@ -317,8 +330,8 @@ assert len(ops) == 2 assert not ops[0].get_opnames("call") assert not ops[0].get_opnames("new") - assert len(ops[0].get_opnames("guard")) <= 7 - assert len(ops[0].get_opnames("getfield")) < 6 + assert len(ops[0].get_opnames("guard")) <= 2 + assert len(ops[0].get_opnames("getfield")) <= 4 assert not ops[1] # second LOOKUP_METHOD folded away def test_default_and_kw(self): @@ -364,10 +377,75 @@ ([1000], 49500), ([10000], 495000), ([100000], 4950000)) - assert len(self.loops) == 2 + assert len(self.loops) == 3 op, = self.get_by_bytecode("CALL_FUNCTION_KW") # XXX a bit too many guards, but better than before - assert len(op.get_opnames("guard")) <= 10 + assert len(op.get_opnames("guard")) <= 12 + + def test_stararg_virtual(self): + self.run_source(''' + d = {} + + def g(*args): + return len(args) + def h(a, b, c): + return c + + def main(x): + s = 0 + for i in range(x): + l = [i, x, 2] + s += g(*l) + s += h(*l) + s += g(i, x, 2) + for i in range(x): + l = [x, 2] + s += g(i, *l) + s += h(i, *l) + return s + ''', 100000, ([100], 1300), + ([1000], 13000), + ([10000], 130000), + ([100000], 1300000)) + assert len(self.loops) == 2 + ops = self.get_by_bytecode("CALL_FUNCTION_VAR") + assert len(ops) == 4 + for op in ops: + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 + + ops = self.get_by_bytecode("CALL_FUNCTION") + for op in ops: + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 + + def test_stararg(self): + self.run_source(''' + d = {} + + def g(*args): + return args[-1] + def h(*args): + return len(args) + + def main(x): + s = 0 + l = [] + i = 0 + while i < x: + l.append(1) + s += g(*l) + i = h(*l) + return s + ''', 100000, ([100], 100), + ([1000], 1000), + ([2000], 2000), + ([4000], 4000)) + assert len(self.loops) == 1 + ops = self.get_by_bytecode("CALL_FUNCTION_VAR") + for op in ops: + assert len(op.get_opnames("new_with_vtable")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 def test_virtual_instance(self): self.run_source(''' @@ -382,7 +460,7 @@ a.x = 2 i = i + a.x return i - ''', 67, + ''', 69, ([20], 20), ([31], 32)) @@ -390,7 +468,7 @@ self.get_by_bytecode("CALL_FUNCTION")) assert not callA.get_opnames("call") assert not callA.get_opnames("new") - assert len(callA.get_opnames("guard")) <= 8 + assert len(callA.get_opnames("guard")) <= 2 assert not callisinstance1.get_opnames("call") assert not callisinstance1.get_opnames("new") assert len(callisinstance1.get_opnames("guard")) <= 2 @@ -554,17 +632,13 @@ def test_blockstack_virtualizable(self): self.run_source(''' - def g(k): - s = 0 - for i in range(k, k+2): - s += 1 - return s + from pypyjit import residual_call def main(): i = 0 while i < 100: try: - g(i) + residual_call(len, []) except: pass i += 1 @@ -606,16 +680,17 @@ # call that can raise is not exchanged into getarrayitem_gc def test_overflow_checking(self): + startvalue = sys.maxint - 2147483647 self.run_source(''' def main(): def f(a,b): if a < 0: return -1 return a-b - total = 0 + total = %d for i in range(100000): total += f(i, 5) return total - ''', 170, ([], 4999450000L)) + ''' % startvalue, 170, ([], startvalue + 4999450000L)) def test_boolrewrite_invers(self): for a, b, res, ops in (('2000', '2000', 20001000, 51), @@ -742,6 +817,8 @@ '''%(op1, float(a)/4.0, float(b)/4.0, op2), 109, ([], res)) def test_boolrewrite_ptr(self): + # XXX this test is way too imprecise in what it is actually testing + # it should count the number of guards instead compares = ('a == b', 'b == a', 'a != b', 'b != a', 'a == c', 'c != b') for e1 in compares: for e2 in compares: @@ -765,7 +842,7 @@ print print 'Test:', e1, e2, n, res self.run_source(''' - class tst: + class tst(object): pass def main(): a = tst() @@ -781,24 +858,6 @@ return sa '''%(e1, e2), n, ([], res)) - def test_boolrewrite_ptr_single(self): - self.run_source(''' - class tst: - pass - def main(): - a = tst() - b = tst() - c = tst() - sa = 0 - for i in range(1000): - if a == b: sa += 1 - else: sa += 2 - if a != b: sa += 10000 - else: sa += 20000 - if i > 750: a = b - return sa - ''', 215, ([], 12481752)) - def test_array_sum(self): for tc, maxops in zip('bhilBHILfd', (38,) * 6 + (40, 40, 41, 38)): res = 19352859 @@ -847,7 +906,12 @@ ''', 65, ([], 122880)) def test_array_intimg(self): - for tc, maxops in zip('ilILd', (67, 67, 69, 69, 61)): + # XXX this test is way too imprecise in what it is actually testing + # it should count the number of guards instead + for tc, maxops in zip('ilILd', (67, 67, 70, 70, 61)): + print + print '='*65 + print '='*20, 'running test for tc=%r' % (tc,), '='*20 res = 73574560 if tc in 'IL': res = long(res) @@ -1130,6 +1194,44 @@ return sa ''', 88, ([], 1997001)) + def test__ffi_call(self): + from pypy.rlib.test.test_libffi import get_libm_name + libm_name = get_libm_name(sys.platform) + out = self.run_source(''' + def main(): + try: + from _ffi import CDLL, types + except ImportError: + sys.stdout.write('SKIP: cannot import _ffi') + return 0 + + libm = CDLL('%(libm_name)s') + pow = libm.getfunc('pow', [types.double, types.double], + types.double) + print pow.getaddr() + i = 0 + res = 0 + while i < 2000: + res += pow(2, 3) + i += 1 + return res + ''' % locals(), + 76, ([], 8.0*2000), threshold=1000) + pow_addr = int(out.splitlines()[0]) + ops = self.get_by_bytecode('CALL_FUNCTION') + assert len(ops) == 2 # we get two loops, because of specialization + call_function = ops[0] + last_ops = [op.getopname() for op in call_function[-5:]] + assert last_ops == ['force_token', + 'setfield_gc', + 'call_may_force', + 'guard_not_forced', + 'guard_no_exception'] + call = call_function[-3] + assert call.getarg(0).value == pow_addr + assert call.getarg(1).value == 2.0 + assert call.getarg(2).value == 3.0 + # test_circular class AppTestJIT(PyPyCJITTests): @@ -1155,6 +1257,17 @@ cls.pypy_c = option.pypy_c +def test_interface_residual_call(): + space = gettestobjspace(usemodules=['pypyjit']) + space.appexec([], """(): + import pypyjit + def f(*args, **kwds): + return (args, kwds) + res = pypyjit.residual_call(f, 4, x=6) + assert res == ((4,), {'x': 6}) + """) + + def has_info(pypy_c, option): g = os.popen('"%s" --info' % pypy_c, 'r') lines = g.readlines() Modified: pypy/branch/arm-backend/pypy/module/rctime/interp_time.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/rctime/interp_time.py (original) +++ pypy/branch/arm-backend/pypy/module/rctime/interp_time.py Tue Nov 23 10:12:47 2010 @@ -69,7 +69,7 @@ CLOCKS_PER_SEC = cConfig.CLOCKS_PER_SEC clock_t = cConfig.clock_t tm = cConfig.tm -glob_buf = lltype.malloc(tm, flavor='raw', zero=True) +glob_buf = lltype.malloc(tm, flavor='raw', zero=True, immortal=True) if cConfig.has_gettimeofday: c_gettimeofday = external('gettimeofday', [rffi.VOIDP, rffi.VOIDP], rffi.INT) Modified: pypy/branch/arm-backend/pypy/module/signal/__init__.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/signal/__init__.py (original) +++ pypy/branch/arm-backend/pypy/module/signal/__init__.py Tue Nov 23 10:12:47 2010 @@ -34,9 +34,7 @@ MixedModule.__init__(self, space, *args) # add the signal-checking callback as an action on the space space.check_signal_action = interp_signal.CheckSignalAction(space) - space.actionflag.register_action(space.check_signal_action) - # use the C-level pypysig_occurred variable as the action flag - # (the result is that the C-level signal handler will directly - # set the flag for the CheckSignalAction) + space.actionflag.register_periodic_action(space.check_signal_action, + use_bytecode_counter=False) space.actionflag.__class__ = interp_signal.SignalActionFlag # xxx yes I know the previous line is a hack Modified: pypy/branch/arm-backend/pypy/module/signal/interp_signal.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/signal/interp_signal.py (original) +++ pypy/branch/arm-backend/pypy/module/signal/interp_signal.py Tue Nov 23 10:12:47 2010 @@ -1,7 +1,7 @@ from pypy.interpreter.error import OperationError from pypy.interpreter.baseobjspace import W_Root, ObjSpace from pypy.interpreter.executioncontext import AsyncAction, AbstractActionFlag -from pypy.rlib.rarithmetic import LONG_BIT, intmask +from pypy.interpreter.executioncontext import PeriodicAsyncAction import signal as cpy_signal from pypy.rpython.lltypesystem import lltype, rffi from pypy.translator.tool.cbuild import ExternalCompilationInfo @@ -53,19 +53,29 @@ class SignalActionFlag(AbstractActionFlag): - def get(self): + # This class uses the C-level pypysig_counter variable as the tick + # counter. The C-level signal handler will reset it to -1 whenever + # a signal is received. + + def get_ticker(self): p = pypysig_getaddr_occurred() return p.c_value - def set(self, value): + + def reset_ticker(self, value): p = pypysig_getaddr_occurred() p.c_value = value + def decrement_ticker(self, by): + p = pypysig_getaddr_occurred() + value = p.c_value + if self.has_bytecode_counter: # this 'if' is constant-folded + value -= by + p.c_value = value + return value -class CheckSignalAction(AsyncAction): - """An action that is automatically invoked when a signal is received.""" - # The C-level signal handler sets the highest bit of pypysig_occurred: - bitmask = intmask(1 << (LONG_BIT-1)) +class CheckSignalAction(PeriodicAsyncAction): + """An action that is automatically invoked when a signal is received.""" def __init__(self, space): AsyncAction.__init__(self, space) @@ -74,7 +84,6 @@ # need a helper action in case signals arrive in a non-main thread self.pending_signals = {} self.reissue_signal_action = ReissueSignalAction(space) - space.actionflag.register_action(self.reissue_signal_action) else: self.reissue_signal_action = None Modified: pypy/branch/arm-backend/pypy/module/signal/test/test_signal.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/signal/test/test_signal.py (original) +++ pypy/branch/arm-backend/pypy/module/signal/test/test_signal.py Tue Nov 23 10:12:47 2010 @@ -1,6 +1,38 @@ import os, py +import signal as cpy_signal from pypy.conftest import gettestobjspace + +class TestCheckSignals: + + def setup_class(cls): + if not hasattr(os, 'kill') or not hasattr(os, 'getpid'): + py.test.skip("requires os.kill() and os.getpid()") + cls.space = gettestobjspace(usemodules=['signal']) + + def test_checksignals(self): + space = self.space + w_received = space.appexec([], """(): + import signal + received = [] + def myhandler(signum, frame): + received.append(signum) + signal.signal(signal.SIGUSR1, myhandler) + return received""") + # + assert not space.is_true(w_received) + # + # send the signal now + os.kill(os.getpid(), cpy_signal.SIGUSR1) + # + # myhandler() should not be immediately called + assert not space.is_true(w_received) + # + # calling ec.checksignals() should call it + space.getexecutioncontext().checksignals() + assert space.is_true(w_received) + + class AppTestSignal: def setup_class(cls): @@ -24,18 +56,12 @@ signal.signal(signal.SIGUSR1, myhandler) posix.kill(posix.getpid(), signal.SIGUSR1) - for i in range(10000): - # wait a bit for the signal to be delivered to the handler - if received: - break + # the signal should be delivered to the handler immediately assert received == [signal.SIGUSR1] del received[:] posix.kill(posix.getpid(), signal.SIGUSR1) - for i in range(10000): - # wait a bit for the signal to be delivered to the handler - if received: - break + # the signal should be delivered to the handler immediately assert received == [signal.SIGUSR1] del received[:] Modified: pypy/branch/arm-backend/pypy/module/sys/__init__.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/sys/__init__.py (original) +++ pypy/branch/arm-backend/pypy/module/sys/__init__.py Tue Nov 23 10:12:47 2010 @@ -7,13 +7,14 @@ """Sys Builtin Module. """ def __init__(self, space, w_name): """NOT_RPYTHON""" # because parent __init__ isn't + if space.config.translating: + del self.__class__.interpleveldefs['pypy_getudir'] super(Module, self).__init__(space, w_name) - self.checkinterval = 100 self.recursionlimit = 100 self.w_default_encoder = None self.defaultencoding = "ascii" self.filesystemencoding = None - + interpleveldefs = { '__name__' : '(space.wrap("sys"))', '__doc__' : '(space.wrap("PyPy sys module"))', @@ -37,7 +38,7 @@ 'argv' : 'state.get(space).w_argv', 'warnoptions' : 'state.get(space).w_warnoptions', 'builtin_module_names' : 'state.w_None', - 'pypy_getudir' : 'state.pypy_getudir', + 'pypy_getudir' : 'state.pypy_getudir', # not translated 'pypy_initial_path' : 'state.pypy_initial_path', '_getframe' : 'vm._getframe', Modified: pypy/branch/arm-backend/pypy/module/sys/state.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/sys/state.py (original) +++ pypy/branch/arm-backend/pypy/module/sys/state.py Tue Nov 23 10:12:47 2010 @@ -33,6 +33,8 @@ raise OSError(errno.ENOTDIR, path) +platform = sys.platform + def getinitialpath(prefix): from pypy.module.sys.version import CPYTHON_VERSION dirname = '%d.%d.%d' % (CPYTHON_VERSION[0], @@ -51,6 +53,15 @@ importlist.append(lib_pypy) importlist.append(python_std_lib_modified) importlist.append(python_std_lib) + # + # List here the extra platform-specific paths. + if platform != 'win32': + importlist.append(os.path.join(python_std_lib, 'plat-'+platform)) + if platform == 'darwin': + platmac = os.path.join(python_std_lib, 'plat-mac') + importlist.append(platmac) + importlist.append(os.path.join(platmac, 'lib-scriptpackages')) + # return importlist def pypy_initial_path(space, srcdir): @@ -95,15 +106,8 @@ def getio(space): return space.fromcache(IOState) -def _pypy_getudir(space): - """NOT_RPYTHON""" +def pypy_getudir(space): + """NOT_RPYTHON + (should be removed from interpleveldefs before translation)""" from pypy.tool.udir import udir return space.wrap(str(udir)) -_pypy_getudir._annspecialcase_ = "override:ignore" - -# we need the indirection because this function will live in a dictionary with other -# RPYTHON functions and share call sites with them. Better it not be a special-case -# directly. -def pypy_getudir(space): - return _pypy_getudir(space) - Modified: pypy/branch/arm-backend/pypy/module/sys/test/test_initialpath.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/sys/test/test_initialpath.py (original) +++ pypy/branch/arm-backend/pypy/module/sys/test/test_initialpath.py Tue Nov 23 10:12:47 2010 @@ -15,4 +15,5 @@ def test_stdlib_in_prefix(tmpdir): dirs = build_hierarchy(tmpdir) path = getinitialpath(str(tmpdir)) - assert path == map(str, dirs) + # we get at least 'dirs', and maybe more (e.g. plat-linux2) + assert path[:len(dirs)] == map(str, dirs) Modified: pypy/branch/arm-backend/pypy/module/sys/version.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/sys/version.py (original) +++ pypy/branch/arm-backend/pypy/module/sys/version.py Tue Nov 23 10:12:47 2010 @@ -8,7 +8,7 @@ CPYTHON_VERSION = (2, 5, 2, "beta", 42) #XXX # sync patchlevel.h CPYTHON_API_VERSION = 1012 #XXX # sync with include/modsupport.h -PYPY_VERSION = (1, 3, 0, "beta", '?') #XXX # sync patchlevel.h +PYPY_VERSION = (1, 4, 0, "beta", '?') #XXX # sync patchlevel.h # the last item is replaced by the svn revision ^^^ TRIM_URL_UP_TO = 'svn/pypy/' Modified: pypy/branch/arm-backend/pypy/module/sys/vm.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/sys/vm.py (original) +++ pypy/branch/arm-backend/pypy/module/sys/vm.py Tue Nov 23 10:12:47 2010 @@ -67,7 +67,7 @@ def setcheckinterval(space, interval): """Tell the Python interpreter to check for asynchronous events every n instructions. This also affects how often thread switches occur.""" - space.actionflag.setcheckinterval(space, interval) + space.actionflag.setcheckinterval(interval) setcheckinterval.unwrap_spec = [ObjSpace, int] def getcheckinterval(space): @@ -77,7 +77,7 @@ # return 0. The idea is that according to the CPython docs, <= 0 # means "check every virtual instruction, maximizing responsiveness # as well as overhead". - result = space.sys.checkinterval + result = space.actionflag.getcheckinterval() if result <= 1: result = 0 return space.wrap(result) Modified: pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c ============================================================================== --- pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c (original) +++ pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/_ctypes_test.c Tue Nov 23 10:12:47 2010 @@ -533,3 +533,7 @@ return inp; } +int my_unused_function(void) +{ + return 42; +} Modified: pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py (original) +++ pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_funcptr.py Tue Nov 23 10:12:47 2010 @@ -148,3 +148,7 @@ # but it segfaults for some reason. if sys.platform == 'win32': assert f() == 0x12345678 + + def test_restype(self): + foo = lib.my_unused_function + assert foo.restype is c_int # by default Modified: pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py (original) +++ pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_functions.py Tue Nov 23 10:12:47 2010 @@ -29,6 +29,7 @@ class RECT(Structure): _fields_ = [("left", c_int), ("top", c_int), ("right", c_int), ("bottom", c_int)] + class TestFunctions(BaseCTypesTestChecker): def test_mro(self): @@ -392,6 +393,18 @@ result = f("abcd", ord("b")) assert result == "bcd" + def test_caching_bug_1(self): + # the same test as test_call_some_args, with two extra lines + # in the middle that trigger caching in f._ptr, which then + # makes the last two lines fail + f = dll.my_strchr + f.argtypes = [c_char_p, c_int] + f.restype = c_char_p + result = f("abcd", ord("b")) + assert result == "bcd" + result = f("abcd", ord("b"), 42) + assert result == "bcd" + def test_sf1651235(self): py.test.skip("we are less strict in checking callback parameters") # see http://www.python.org/sf/1651235 Modified: pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py (original) +++ pypy/branch/arm-backend/pypy/module/test_lib_pypy/ctypes_tests/test_keepalive.py Tue Nov 23 10:12:47 2010 @@ -10,7 +10,7 @@ """ def test_array_of_pointers(self): # tests array item assignements & pointer.contents = ... - A = POINTER(c_int) * 24 + A = POINTER(c_long) * 24 a = A() l = c_long(2) p = pointer(l) Modified: pypy/branch/arm-backend/pypy/module/thread/gil.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/thread/gil.py (original) +++ pypy/branch/arm-backend/pypy/module/thread/gil.py Tue Nov 23 10:12:47 2010 @@ -20,7 +20,8 @@ def initialize(self, space): # add the GIL-releasing callback as an action on the space - space.actionflag.register_action(GILReleaseAction(space)) + space.actionflag.register_periodic_action(GILReleaseAction(space), + use_bytecode_counter=True) def setup_threads(self, space): """Enable threads in the object space, if they haven't already been.""" @@ -44,7 +45,6 @@ # test_compile_lock. As a workaround, we repatch these global # fields systematically. spacestate.ll_GIL = self.ll_GIL - spacestate.actionflag = space.actionflag invoke_around_extcall(before_external_call, after_external_call) return result @@ -68,18 +68,17 @@ def _freeze_(self): self.ll_GIL = thread.null_ll_lock - self.actionflag = None - self.set_actionflag_bit_after_thread_switch = 0 + self.action_after_thread_switch = None + # ^^^ set by AsyncAction.fire_after_thread_switch() return False def after_thread_switch(self): # this is support logic for the signal module, to help it deliver # signals to the main thread. - actionflag = self.actionflag - if actionflag is not None: - flag = actionflag.get() - flag |= self.set_actionflag_bit_after_thread_switch - actionflag.set(flag) + action = self.action_after_thread_switch + if action is not None: + self.action_after_thread_switch = None + action.fire() spacestate = SpaceState() spacestate._freeze_() Modified: pypy/branch/arm-backend/pypy/module/thread/ll_thread.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/thread/ll_thread.py (original) +++ pypy/branch/arm-backend/pypy/module/thread/ll_thread.py Tue Nov 23 10:12:47 2010 @@ -111,7 +111,7 @@ c_thread_releaselock(self._lock) def __del__(self): - lltype.free(self._lock, flavor='raw') + lltype.free(self._lock, flavor='raw', track_allocation=False) # ____________________________________________________________ # @@ -128,10 +128,13 @@ null_ll_lock = lltype.nullptr(TLOCKP.TO) def allocate_ll_lock(): - ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw') + # track_allocation=False here; be careful to lltype.free() it. The + # reason it is set to False is that we get it from all app-level + # lock objects, as well as from the GIL, which exists at shutdown. + ll_lock = lltype.malloc(TLOCKP.TO, flavor='raw', track_allocation=False) res = c_thread_lock_init(ll_lock) if res == -1: - lltype.free(ll_lock, flavor='raw') + lltype.free(ll_lock, flavor='raw', track_allocation=False) raise error("out of resources") return ll_lock Modified: pypy/branch/arm-backend/pypy/module/thread/test/test_gil.py ============================================================================== --- pypy/branch/arm-backend/pypy/module/thread/test/test_gil.py (original) +++ pypy/branch/arm-backend/pypy/module/thread/test/test_gil.py Tue Nov 23 10:12:47 2010 @@ -8,7 +8,7 @@ pass class FakeActionFlag(object): - def register_action(self, action): + def register_periodic_action(self, action, use_bytecode_counter): pass def get(self): return 0 Modified: pypy/branch/arm-backend/pypy/objspace/descroperation.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/descroperation.py (original) +++ pypy/branch/arm-backend/pypy/objspace/descroperation.py Tue Nov 23 10:12:47 2010 @@ -51,7 +51,7 @@ return space.is_w(w_typ1, w_typ2) -class Object: +class Object(object): def descr__getattribute__(space, w_obj, w_name): name = space.str_w(w_name) w_descr = space.lookup(w_obj, name) @@ -64,9 +64,7 @@ w_type = space.type(w_obj) return space.get_and_call_function(w_get, w_descr, w_obj, w_type) - w_value = w_obj.getdictvalue_attr_is_in_class(space, name) - else: - w_value = w_obj.getdictvalue(space, name) + w_value = w_obj.getdictvalue(space, name) if w_value is not None: return w_value if w_descr is not None: @@ -76,13 +74,11 @@ def descr__setattr__(space, w_obj, w_name, w_value): name = space.str_w(w_name) w_descr = space.lookup(w_obj, name) - shadows_type = False if w_descr is not None: if space.is_data_descr(w_descr): space.set(w_descr, w_obj, w_value) return - shadows_type = True - if w_obj.setdictvalue(space, name, w_value, shadows_type): + if w_obj.setdictvalue(space, name, w_value): return raiseattrerror(space, w_obj, name, w_descr) @@ -100,7 +96,7 @@ def descr__init__(space, w_obj, __args__): pass -class DescrOperation: +class DescrOperation(object): _mixin_ = True def is_data_descr(space, w_obj): @@ -545,7 +541,7 @@ # what is the maximum value slices can get on CPython? # we need to stick to that value, because fake.py etc. -class Temp: +class Temp(object): def __getslice__(self, i, j): return j slice_max = Temp()[:] Modified: pypy/branch/arm-backend/pypy/objspace/flow/flowcontext.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/flow/flowcontext.py (original) +++ pypy/branch/arm-backend/pypy/objspace/flow/flowcontext.py Tue Nov 23 10:12:47 2010 @@ -460,7 +460,22 @@ else: v = self.peekvalue(oparg - 1) self.space.call_method(v, 'append', w) - + + # `with` statement + + def SETUP_WITH(self, offsettoend, next_instr): + # A simpler version than the 'real' 2.7 one: + # directly call manager.__enter__(), don't use special lookup functions + # which don't make sense on the RPython type system. + from pypy.interpreter.pyopcode import WithBlock + w_manager = self.peekvalue() + w_exit = self.space.getattr(w_manager, self.space.wrap("__exit__")) + self.settopvalue(w_exit) + w_result = self.space.call_method(w_manager, "__enter__") + block = WithBlock(self, next_instr + offsettoend) + self.append_block(block) + self.pushvalue(w_result) + # XXX Unimplemented 2.7 opcodes ---------------- # Set literals, set comprehensions @@ -476,12 +491,6 @@ def MAP_ADD(self, oparg, next_instr): raise NotImplementedError("MAP_ADD") - # `with` statement - - def SETUP_WITH(self, oparg, next_instr): - raise NotImplementedError("SETUP_WITH") - - def make_arguments(self, nargs): return ArgumentsForTranslation(self.space, self.peekvalues(nargs)) def argument_factory(self, *args): @@ -493,3 +502,12 @@ raise operr return pyframe.PyFrame.handle_operation_error(self, ec, operr, *args, **kwds) + + def call_contextmanager_exit_function(self, w_func, w_typ, w_val, w_tb): + if w_typ is not self.space.w_None: + # The annotator won't allow to merge exception types with None. + # Replace it with an object which will break translation when used + # (except maybe with 'exc_typ is None') + w_typ = self.space.wrap(self.space) + return self.space.call_function(w_func, w_typ, w_val, w_tb) + Modified: pypy/branch/arm-backend/pypy/objspace/flow/model.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/flow/model.py (original) +++ pypy/branch/arm-backend/pypy/objspace/flow/model.py Tue Nov 23 10:12:47 2010 @@ -355,7 +355,7 @@ return "%r = %s(%s)" % (self.result, self.opname, ", ".join(map(repr, self.args))) -class Atom: +class Atom(object): def __init__(self, name): self.__name__ = name # make save_global happy def __repr__(self): Modified: pypy/branch/arm-backend/pypy/objspace/flow/objspace.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/flow/objspace.py (original) +++ pypy/branch/arm-backend/pypy/objspace/flow/objspace.py Tue Nov 23 10:12:47 2010 @@ -210,6 +210,11 @@ check_class = self.unwrap(w_check_class) except UnwrapException: raise Exception, "non-constant except guard" + if check_class in (NotImplementedError, AssertionError): + # if we are in geninterp, we cannot catch these exceptions + if not self.config.translation.builtins_can_raise_exceptions: + raise error.FlowingError("Catching %s is not valid in RPython" % + check_class.__name__) if not isinstance(check_class, tuple): # the simple case return ObjSpace.exception_match(self, w_exc_type, w_check_class) Modified: pypy/branch/arm-backend/pypy/objspace/flow/test/test_objspace.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/flow/test/test_objspace.py (original) +++ pypy/branch/arm-backend/pypy/objspace/flow/test/test_objspace.py Tue Nov 23 10:12:47 2010 @@ -1,10 +1,11 @@ +from __future__ import with_statement import new import py from pypy.objspace.flow.model import Constant, Block, Link, Variable, traverse from pypy.objspace.flow.model import flatten, mkentrymap, c_last_exception from pypy.interpreter.argument import Arguments from pypy.translator.simplify import simplify_graph -from pypy.objspace.flow.objspace import FlowObjSpace +from pypy.objspace.flow.objspace import FlowObjSpace, error from pypy.objspace.flow import objspace, flowcontext from pypy import conftest from pypy.tool.stdlib_opcode import bytecode_spec @@ -828,6 +829,25 @@ simplify_graph(graph) assert self.all_operations(graph) == {'getitem': 1} + def test_context_manager(self): + def f(c, x): + with x: + pass + graph = self.codetest(f) + # 2 method calls: x.__enter__() and x.__exit__(None, None, None) + assert self.all_operations(graph) == {'getattr': 2, + 'simple_call': 2} + # + def g(): pass + def f(c, x): + with x: + g() + graph = self.codetest(f) + assert self.all_operations(graph) == { + 'getattr': 2, # __enter__ and __exit__ + 'simple_call': 4, # __enter__, g and 2 possible calls to __exit__ + 'is_true': 1} # check the result of __exit__() + def monkey_patch_code(self, code, stacksize, flags, codestring, names, varnames): c = code return new.code(c.co_argcount, c.co_nlocals, stacksize, flags, @@ -933,6 +953,22 @@ assert op.args[0] == Constant(g) + def test_cannot_catch_special_exceptions(self): + def f(): + try: + f() + except NotImplementedError: + pass + py.test.raises(error.FlowingError, "self.codetest(f)") + # + def f(): + try: + f() + except AssertionError: + pass + py.test.raises(error.FlowingError, "self.codetest(f)") + + class TestFlowObjSpaceDelay(Base): def setup_class(cls): cls.space = FlowObjSpace() @@ -993,6 +1029,15 @@ expected.sort() assert excfound == expected + def test_can_catch_special_exceptions(self): + def f(): + try: + f() + except NotImplementedError: + pass + graph = self.codetest(f) + # assert did not crash + DATA = {'x': 5, 'y': 6} Modified: pypy/branch/arm-backend/pypy/objspace/std/callmethod.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/callmethod.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/callmethod.py Tue Nov 23 10:12:47 2010 @@ -13,6 +13,9 @@ from pypy.interpreter import function from pypy.objspace.descroperation import object_getattribute from pypy.rlib import jit, rstack # for resume points +from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict, \ + LOOKUP_METHOD_mapdict_fill_cache_method + # This module exports two extra methods for StdObjSpaceFrame implementing # the LOOKUP_METHOD and CALL_METHOD opcodes in an efficient way, as well @@ -30,6 +33,13 @@ # space = f.space w_obj = f.popvalue() + + if space.config.objspace.std.withmapdict and not jit.we_are_jitted(): + # mapdict has an extra-fast version of this function + from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict + if LOOKUP_METHOD_mapdict(f, nameindex, w_obj): + return + w_name = f.getname_w(nameindex) w_value = None @@ -44,12 +54,17 @@ else: typ = type(w_descr) if typ is function.Function or typ is function.FunctionWithFixedCode: - w_value = w_obj.getdictvalue_attr_is_in_class(space, name) + w_value = w_obj.getdictvalue(space, name) if w_value is None: # fast method path: a function object in the class, # nothing in the instance f.pushvalue(w_descr) f.pushvalue(w_obj) + if (space.config.objspace.std.withmapdict and + not jit.we_are_jitted()): + # let mapdict cache stuff + LOOKUP_METHOD_mapdict_fill_cache_method( + f.getcode(), nameindex, w_obj, w_type, w_descr) return if w_value is None: w_value = space.getattr(w_obj, w_name) @@ -103,7 +118,7 @@ w_descr = space.lookup(w_obj, methname) typ = type(w_descr) if typ is function.Function or typ is function.FunctionWithFixedCode: - w_value = w_obj.getdictvalue_attr_is_in_class(space, methname) + w_value = w_obj.getdictvalue(space, methname) if w_value is None: # fast method path: a function object in the class, # nothing in the instance Modified: pypy/branch/arm-backend/pypy/objspace/std/celldict.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/celldict.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/celldict.py Tue Nov 23 10:12:47 2010 @@ -45,9 +45,9 @@ if space.is_w(space.type(w_key), space.w_str): self.impl_setitem_str(self.space.str_w(w_key), w_value) else: - self._as_rdict().setitem(w_key, w_value) + self._as_rdict().impl_fallback_setitem(w_key, w_value) - def impl_setitem_str(self, name, w_value, shadows_type=True): + def impl_setitem_str(self, name, w_value): self.getcell(name, True).w_value = w_value def impl_delitem(self, w_key): @@ -66,7 +66,7 @@ elif _is_sane_hash(space, w_key_type): raise KeyError else: - self._as_rdict().delitem(w_key) + self._as_rdict().impl_fallback_delitem(w_key) def impl_length(self): # inefficient, but do we care? @@ -85,7 +85,7 @@ elif _is_sane_hash(space, w_lookup_type): return None else: - return self._as_rdict().getitem(w_lookup) + return self._as_rdict().impl_fallback_getitem(w_lookup) def impl_getitem_str(self, lookup): res = self.getcell(lookup, False) Modified: pypy/branch/arm-backend/pypy/objspace/std/dictmultiobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/dictmultiobject.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/dictmultiobject.py Tue Nov 23 10:12:47 2010 @@ -49,14 +49,6 @@ elif space.config.objspace.std.withdictmeasurement: assert w_type is None return MeasuringDictImplementation(space) - elif space.config.objspace.std.withsharingdict and instance: - from pypy.objspace.std.sharingdict import SharedDictImplementation - assert w_type is None - return SharedDictImplementation(space) - elif (space.config.objspace.std.withshadowtracking and instance and - classofinstance is not None): - assert w_type is None - return ShadowDetectingDictImplementation(space, classofinstance) elif instance or strdict or module: assert w_type is None return StrDictImplementation(space) @@ -102,17 +94,17 @@ else: return None - # _________________________________________________________________ + # _________________________________________________________________ # implementation methods def impl_getitem(self, w_key): #return w_value or None raise NotImplementedError("abstract base class") - def impl_getitem_str(self, w_key): + def impl_getitem_str(self, key): #return w_value or None raise NotImplementedError("abstract base class") - def impl_setitem_str(self, key, w_value, shadows_type=True): + def impl_setitem_str(self, key, w_value): raise NotImplementedError("abstract base class") def impl_setitem(self, w_key, w_value): @@ -120,7 +112,7 @@ def impl_delitem(self, w_key): raise NotImplementedError("abstract base class") - + def impl_length(self): raise NotImplementedError("abstract base class") @@ -165,12 +157,15 @@ key = OPTIMIZED_BUILTINS[i] return self.impl_getitem_str(key) - # this method will only be seen whan a certain config option is used - def impl_shadows_anything(self): - return True - - def impl_set_shadows_anything(self): - pass + def impl_popitem(self): + # default implementation + space = self.space + iterator = self.impl_iter() + w_key, w_value = iterator.next() + if w_key is None: + raise KeyError + self.impl_delitem(w_key) + return w_key, w_value # _________________________________________________________________ # fallback implementation methods @@ -178,7 +173,7 @@ def impl_fallback_setitem(self, w_key, w_value): self.r_dict_content[w_key] = w_value - def impl_fallback_setitem_str(self, key, w_value, shadows_type=True): + def impl_fallback_setitem_str(self, key, w_value): return self.impl_fallback_setitem(self.space.wrap(key), w_value) def impl_fallback_delitem(self, w_key): @@ -211,18 +206,15 @@ key = OPTIMIZED_BUILTINS[i] return self.impl_fallback_getitem_str(key) - def impl_fallback_shadows_anything(self): - return True - - def impl_fallback_set_shadows_anything(self): - pass + def impl_fallback_popitem(self): + return self.r_dict_content.popitem() implementation_methods = [ ("getitem", 1), ("getitem_str", 1), ("length", 0), - ("setitem_str", 3), + ("setitem_str", 2), ("setitem", 2), ("delitem", 1), ("iter", 0), @@ -231,8 +223,7 @@ ("keys", 0), ("clear", 0), ("get_builtin_indexed", 1), - ("shadows_anything", 0), - ("set_shadows_anything", 0), + ("popitem", 0), ] @@ -310,9 +301,9 @@ if space.is_w(space.type(w_key), space.w_str): self.impl_setitem_str(self.space.str_w(w_key), w_value) else: - self._as_rdict().setitem(w_key, w_value) + self._as_rdict().impl_fallback_setitem(w_key, w_value) - def impl_setitem_str(self, key, w_value, shadows_type=True): + def impl_setitem_str(self, key, w_value): self.content[key] = w_value def impl_delitem(self, w_key): @@ -324,7 +315,7 @@ elif _is_sane_hash(space, w_key_type): raise KeyError else: - self._as_rdict().delitem(w_key) + self._as_rdict().impl_fallback_delitem(w_key) def impl_length(self): return len(self.content) @@ -344,7 +335,7 @@ elif _is_sane_hash(space, w_lookup_type): return None else: - return self._as_rdict().getitem(w_key) + return self._as_rdict().impl_fallback_getitem(w_key) def impl_iter(self): return StrIteratorImplementation(self.space, self) @@ -388,47 +379,12 @@ return None, None -class ShadowDetectingDictImplementation(StrDictImplementation): - def __init__(self, space, w_type): - StrDictImplementation.__init__(self, space) - self.w_type = w_type - self.original_version_tag = w_type.version_tag() - if self.original_version_tag is None: - self._shadows_anything = True - else: - self._shadows_anything = False - - def impl_setitem_str(self, key, w_value, shadows_type=True): - if shadows_type: - self._shadows_anything = True - StrDictImplementation.impl_setitem_str( - self, key, w_value, shadows_type) - - def impl_setitem(self, w_key, w_value): - space = self.space - if space.is_w(space.type(w_key), space.w_str): - if not self._shadows_anything: - w_obj = self.w_type.lookup(space.str_w(w_key)) - if w_obj is not None: - self._shadows_anything = True - StrDictImplementation.impl_setitem_str( - self, self.space.str_w(w_key), w_value, False) - else: - self._as_rdict().setitem(w_key, w_value) - - def impl_shadows_anything(self): - return (self._shadows_anything or - self.w_type.version_tag() is not self.original_version_tag) - - def impl_set_shadows_anything(self): - self._shadows_anything = True - class WaryDictImplementation(StrDictImplementation): def __init__(self, space): StrDictImplementation.__init__(self, space) self.shadowed = [None] * len(BUILTIN_TO_INDEX) - def impl_setitem_str(self, key, w_value, shadows_type=True): + def impl_setitem_str(self, key, w_value): i = BUILTIN_TO_INDEX.get(key, -1) if i != -1: self.shadowed[i] = w_value @@ -446,7 +402,7 @@ elif _is_sane_hash(space, w_key_type): raise KeyError else: - self._as_rdict().delitem(w_key) + self._as_rdict().impl_fallback_delitem(w_key) def impl_get_builtin_indexed(self, i): return self.shadowed[i] @@ -558,7 +514,7 @@ self.info.writes += 1 self.content[w_key] = w_value self.info.maxcontents = max(self.info.maxcontents, len(self.content)) - def impl_setitem_str(self, key, w_value, shadows_type=True): + def impl_setitem_str(self, key, w_value): self.info.setitem_strs += 1 self.impl_setitem(self.space.wrap(key), w_value) def impl_delitem(self, w_key): @@ -641,30 +597,50 @@ init_signature = Signature(['seq_or_map'], None, 'kwargs') init_defaults = [None] -def init__DictMulti(space, w_dict, __args__): - w_src, w_kwds = __args__.parse_obj( - None, 'dict', - init_signature, # signature - init_defaults) # default argument - if w_src is None: - pass - elif space.findattr(w_src, space.wrap("keys")) is None: - list_of_w_pairs = space.listview(w_src) - for w_pair in list_of_w_pairs: +def update1(space, w_dict, w_data): + if space.findattr(w_data, space.wrap("keys")) is None: + # no 'keys' method, so we assume it is a sequence of pairs + for w_pair in space.listview(w_data): pair = space.fixedview(w_pair) - if len(pair)!=2: + if len(pair) != 2: raise OperationError(space.w_ValueError, - space.wrap("dict() takes a sequence of pairs")) - w_k, w_v = pair - w_dict.setitem(w_k, w_v) + space.wrap("sequence of pairs expected")) + w_key, w_value = pair + w_dict.setitem(w_key, w_value) else: - if space.is_true(w_src): - from pypy.objspace.std.dicttype import update1 - update1(space, w_dict, w_src) + if isinstance(w_data, W_DictMultiObject): # optimization case only + update1_dict_dict(space, w_dict, w_data) + else: + # general case -- "for k in o.keys(): dict.__setitem__(d, k, o[k])" + w_keys = space.call_method(w_data, "keys") + for w_key in space.listview(w_keys): + w_value = space.getitem(w_data, w_key) + w_dict.setitem(w_key, w_value) + +def update1_dict_dict(space, w_dict, w_data): + iterator = w_data.iter() + while 1: + w_key, w_value = iterator.next() + if w_key is None: + break + w_dict.setitem(w_key, w_value) + +def init_or_update(space, w_dict, __args__, funcname): + w_src, w_kwds = __args__.parse_obj( + None, funcname, + init_signature, # signature + init_defaults) # default argument + if w_src is not None: + update1(space, w_dict, w_src) if space.is_true(w_kwds): - from pypy.objspace.std.dicttype import update1 update1(space, w_dict, w_kwds) +def init__DictMulti(space, w_dict, __args__): + init_or_update(space, w_dict, __args__, 'dict') + +def dict_update__DictMulti(space, w_dict, __args__): + init_or_update(space, w_dict, __args__, 'dict.update') + def getitem__DictMulti_ANY(space, w_dict, w_key): w_value = w_dict.getitem(w_key) if w_value is not None: @@ -758,9 +734,8 @@ return w_res def dict_copy__DictMulti(space, w_self): - from pypy.objspace.std.dicttype import update1 w_new = W_DictMultiObject.allocate_and_init_instance(space) - update1(space, w_new, w_self) + update1_dict_dict(space, w_new, w_self) return w_new def dict_items__DictMulti(space, w_self): @@ -791,6 +766,15 @@ else: return w_default +def dict_setdefault__DictMulti_ANY_ANY(space, w_dict, w_key, w_default): + # XXX should be more efficient, with only one dict lookup + w_value = w_dict.getitem(w_key) + if w_value is not None: + return w_value + else: + w_dict.setitem(w_key, w_default) + return w_default + def dict_pop__DictMulti_ANY(space, w_dict, w_key, w_defaults): defaults = space.listview(w_defaults) len_defaults = len(defaults) @@ -808,6 +792,14 @@ w_dict.delitem(w_key) return w_item +def dict_popitem__DictMulti(space, w_dict): + try: + w_key, w_value = w_dict.popitem() + except KeyError: + raise OperationError(space.w_KeyError, + space.wrap("popitem(): dictionary is empty")) + return space.newtuple([w_key, w_value]) + app = gateway.applevel(''' def dictrepr(currently_in_repr, d): # Now we only handle one implementation of dicts, this one. Modified: pypy/branch/arm-backend/pypy/objspace/std/dicttype.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/dicttype.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/dicttype.py Tue Nov 23 10:12:47 2010 @@ -53,106 +53,21 @@ # This can return when multimethods have been fixed #dict_str = StdObjSpace.str -# default application-level implementations for some operations -# most of these (notably not popitem and update*) are overwritten -# in dictmultiobject -# gateway is imported in the stdtypedef module -app = gateway.applevel(''' - - # in the following functions we use dict.__setitem__ instead of - # d[k]=... because when a subclass of dict override __setitem__, - # CPython does not call it when doing builtin operations. The - # same for other operations. - - def update1(d, o): - if hasattr(o, 'keys'): - for k in o.keys(): - dict.__setitem__(d, k, o[k]) - else: - for k,v in o: - dict.__setitem__(d, k, v) - - def update(d, *args, **kwargs): - len_args = len(args) - if len_args == 1: - update1(d, args[0]) - elif len_args > 1: - raise TypeError("update takes at most 1 (non-keyword) argument") - if kwargs: - update1(d, kwargs) - - def popitem(d): - for k in dict.iterkeys(d): - break - else: - raise KeyError("popitem(): dictionary is empty") - v = dict.__getitem__(d, k) - dict.__delitem__(d, k) - return k, v - - def get(d, k, v=None): - if k in d: - return dict.__getitem__(d, k) - else: - return v - - def setdefault(d, k, v=None): - if k in d: - return dict.__getitem__(d, k) - else: - dict.__setitem__(d, k, v) - return v - - def pop(d, k, defaults): # XXX defaults is actually *defaults - if len(defaults) > 1: - raise TypeError, "pop expected at most 2 arguments, got %d" % ( - 1 + len(defaults)) - try: - v = dict.__getitem__(d, k) - dict.__delitem__(d, k) - except KeyError, e: - if defaults: - return defaults[0] - else: - raise e - return v - - def iteritems(d): - return iter(dict.items(d)) - - def iterkeys(d): - return iter(dict.keys(d)) - - def itervalues(d): - return iter(dict.values(d)) -''', filename=__file__) - -dict_update__ANY = app.interphook("update") -dict_popitem__ANY = app.interphook("popitem") -dict_get__ANY_ANY_ANY = app.interphook("get") -dict_setdefault__ANY_ANY_ANY = app.interphook("setdefault") -dict_pop__ANY_ANY = app.interphook("pop") -dict_iteritems__ANY = app.interphook("iteritems") -dict_iterkeys__ANY = app.interphook("iterkeys") -dict_itervalues__ANY = app.interphook("itervalues") -update1 = app.interphook("update1") - register_all(vars(), globals()) @gateway.unwrap_spec(ObjSpace, W_Root, W_Root, W_Root) def descr_fromkeys(space, w_type, w_keys, w_fill=None): + from pypy.objspace.std.dictmultiobject import W_DictMultiObject if w_fill is None: w_fill = space.w_None - w_dict = space.call_function(w_type) - w_iter = space.iter(w_keys) - while True: - try: - w_key = space.next(w_iter) - except OperationError, e: - if not e.match(space, space.w_StopIteration): - raise - break - space.setitem(w_dict, w_key, w_fill) + if w_type is space.w_dict: + w_dict = W_DictMultiObject.allocate_and_init_instance(space, w_type) + for w_key in space.listview(w_keys): + w_dict.setitem(w_key, w_fill) + else: + w_dict = space.call_function(w_type) + for w_key in space.listview(w_keys): + space.setitem(w_dict, w_key, w_fill) return w_dict Modified: pypy/branch/arm-backend/pypy/objspace/std/fake.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/fake.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/fake.py Tue Nov 23 10:12:47 2010 @@ -21,7 +21,6 @@ #debug_print("faking obj %s" % x) ft = fake_type(type(x)) return ft(space, x) -fake_object._annspecialcase_ = "override:fake_object" import sys @@ -47,7 +46,6 @@ w_exc = space.wrap(exc) w_value = space.wrap(value) raise OperationError, OperationError(w_exc, w_value), tb -wrap_exception._annspecialcase_ = "override:ignore" def fake_type(cpy_type): assert type(cpy_type) is type Modified: pypy/branch/arm-backend/pypy/objspace/std/frozensettype.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/frozensettype.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/frozensettype.py Tue Nov 23 10:12:47 2010 @@ -39,13 +39,13 @@ def descr__frozenset__new__(space, w_frozensettype, w_iterable=gateway.NoneNotWrapped): from pypy.objspace.std.setobject import W_FrozensetObject - from pypy.objspace.std.setobject import _is_frozenset_exact + from pypy.objspace.std.setobject import make_setdata_from_w_iterable if (space.is_w(w_frozensettype, space.w_frozenset) and - _is_frozenset_exact(w_iterable)): + w_iterable is not None and type(w_iterable) is W_FrozensetObject): return w_iterable w_obj = space.allocate_instance(W_FrozensetObject, w_frozensettype) - W_FrozensetObject.__init__(w_obj, space, None) - + data = make_setdata_from_w_iterable(space, w_iterable) + W_FrozensetObject.__init__(w_obj, space, data) return w_obj frozenset_typedef = StdTypeDef("frozenset", Modified: pypy/branch/arm-backend/pypy/objspace/std/inttype.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/inttype.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/inttype.py Tue Nov 23 10:12:47 2010 @@ -1,7 +1,7 @@ from pypy.interpreter import gateway from pypy.interpreter.error import OperationError from pypy.objspace.std.stdtypedef import StdTypeDef -from pypy.objspace.std.strutil import (string_to_int, string_to_w_long, +from pypy.objspace.std.strutil import (string_to_int, string_to_bigint, ParseStringError, ParseStringOverflowError) from pypy.rlib.rarithmetic import r_uint @@ -45,10 +45,12 @@ def retry_to_w_long(space, parser, base=0): parser.rewind() try: - return string_to_w_long(space, None, base=base, parser=parser) + bigint = string_to_bigint(None, base=base, parser=parser) except ParseStringError, e: raise OperationError(space.w_ValueError, space.wrap(e.msg)) + from pypy.objspace.std.longobject import W_LongObject + return W_LongObject(bigint) def descr__new__(space, w_inttype, w_x=0, w_base=gateway.NoneNotWrapped): from pypy.objspace.std.intobject import W_IntObject Modified: pypy/branch/arm-backend/pypy/objspace/std/listobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/listobject.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/listobject.py Tue Nov 23 10:12:47 2010 @@ -283,7 +283,7 @@ elif start >= 0: del items[start:start+delta] else: - assert delta==0 + assert delta==0 # start<0 is only possible with slicelength==0 elif len2 != slicelength: # No resize for extended slices raise operationerrfmt(space.w_ValueError, "attempt to " "assign sequence of size %d to extended slice of size %d", Modified: pypy/branch/arm-backend/pypy/objspace/std/longtype.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/longtype.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/longtype.py Tue Nov 23 10:12:47 2010 @@ -1,7 +1,7 @@ from pypy.interpreter.error import OperationError from pypy.interpreter import gateway from pypy.objspace.std.stdtypedef import StdTypeDef -from pypy.objspace.std.strutil import string_to_w_long, ParseStringError +from pypy.objspace.std.strutil import string_to_bigint, ParseStringError def descr__new__(space, w_longtype, w_x=0, w_base=gateway.NoneNotWrapped): from pypy.objspace.std.longobject import W_LongObject @@ -9,10 +9,10 @@ if w_base is None: # check for easy cases if type(w_value) is W_LongObject: - pass + bigint = w_value.num elif space.is_true(space.isinstance(w_value, space.w_str)): try: - w_value = string_to_w_long(space, space.str_w(w_value)) + bigint = string_to_bigint(space.str_w(w_value)) except ParseStringError, e: raise OperationError(space.w_ValueError, space.wrap(e.msg)) @@ -22,7 +22,7 @@ from pypy.objspace.std.ropeunicodeobject import unicode_to_decimal_w else: from pypy.objspace.std.unicodeobject import unicode_to_decimal_w - w_value = string_to_w_long(space, unicode_to_decimal_w(space, w_value)) + bigint = string_to_bigint(unicode_to_decimal_w(space, w_value)) except ParseStringError, e: raise OperationError(space.w_ValueError, space.wrap(e.msg)) @@ -35,10 +35,11 @@ if space.is_true(space.isinstance(w_obj, space.w_long)): assert isinstance(w_obj, W_LongObject) # XXX this could fail! # XXX find a way to do that even if w_obj is not a W_LongObject - w_value = w_obj + bigint = w_obj.num elif space.is_true(space.isinstance(w_obj, space.w_int)): + from pypy.rlib.rbigint import rbigint intval = space.int_w(w_obj) - w_value = W_LongObject.fromint(space, intval) + bigint = rbigint.fromint(intval) else: raise OperationError(space.w_ValueError, space.wrap("value can't be converted to long")) @@ -56,13 +57,13 @@ space.wrap("long() can't convert non-string " "with explicit base")) try: - w_value = string_to_w_long(space, s, base) + bigint = string_to_bigint(s, base) except ParseStringError, e: raise OperationError(space.w_ValueError, space.wrap(e.msg)) w_obj = space.allocate_instance(W_LongObject, w_longtype) - W_LongObject.__init__(w_obj, w_value.num) + W_LongObject.__init__(w_obj, bigint) return w_obj # ____________________________________________________________ Modified: pypy/branch/arm-backend/pypy/objspace/std/model.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/model.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/model.py Tue Nov 23 10:12:47 2010 @@ -18,6 +18,7 @@ "withsmallint" : ["smallintobject.W_SmallIntObject"], "withstrslice" : ["strsliceobject.W_StringSliceObject"], "withstrjoin" : ["strjoinobject.W_StringJoinObject"], + "withstrbuf" : ["strbufobject.W_StringBufferObject"], "withrope" : ["ropeobject.W_RopeObject", "ropeobject.W_RopeIterObject"], "withropeunicode": ["ropeunicodeobject.W_RopeUnicodeObject", @@ -75,6 +76,7 @@ from pypy.objspace.std import ropeunicodeobject from pypy.objspace.std import strsliceobject from pypy.objspace.std import strjoinobject + from pypy.objspace.std import strbufobject from pypy.objspace.std import typeobject from pypy.objspace.std import sliceobject from pypy.objspace.std import longobject @@ -222,6 +224,13 @@ (unicodeobject.W_UnicodeObject, strjoinobject.delegate_join2unicode) ] + elif config.objspace.std.withstrbuf: + self.typeorder[strbufobject.W_StringBufferObject] += [ + (stringobject.W_StringObject, + strbufobject.delegate_buf2str), + (unicodeobject.W_UnicodeObject, + strbufobject.delegate_buf2unicode) + ] if config.objspace.std.withrangelist: self.typeorder[rangeobject.W_RangeListObject] += [ (listobject.W_ListObject, Modified: pypy/branch/arm-backend/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/objspace.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/objspace.py Tue Nov 23 10:12:47 2010 @@ -23,6 +23,7 @@ from pypy.objspace.std.listobject import W_ListObject from pypy.objspace.std.longobject import W_LongObject from pypy.objspace.std.noneobject import W_NoneObject +from pypy.objspace.std.objectobject import W_ObjectObject from pypy.objspace.std.ropeobject import W_RopeObject from pypy.objspace.std.iterobject import W_SeqIterObject from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject @@ -242,7 +243,6 @@ w_result = getattr(self, 'w_' + x.__name__) return w_result return None - wrap_exception_cls._annspecialcase_ = "override:wrap_exception_cls" def unwrap(self, w_obj): if isinstance(w_obj, Wrappable): @@ -318,9 +318,14 @@ w_subtype = w_type.check_user_subclass(w_subtype) if cls.typedef.applevel_subclasses_base is not None: cls = cls.typedef.applevel_subclasses_base - subcls = get_unique_interplevel_subclass( - self.config, cls, w_subtype.hasdict, w_subtype.nslots != 0, - w_subtype.needsdel, w_subtype.weakrefable) + if (self.config.objspace.std.withmapdict and cls is W_ObjectObject + and not w_subtype.needsdel): + from pypy.objspace.std.mapdict import get_subclass_of_correct_size + subcls = get_subclass_of_correct_size(self, cls, w_subtype) + else: + subcls = get_unique_interplevel_subclass( + self.config, cls, w_subtype.hasdict, w_subtype.nslots != 0, + w_subtype.needsdel, w_subtype.weakrefable) instance = instantiate(subcls) assert isinstance(instance, cls) instance.user_setup(self, w_subtype) @@ -432,7 +437,7 @@ if is_data: w_get = self.lookup(w_descr, "__get__") if w_get is None: - w_value = w_obj.getdictvalue_attr_is_in_class(self, name) + w_value = w_obj.getdictvalue(self, name) if w_value is not None: return w_value if not is_data: @@ -484,14 +489,12 @@ return w_obj.getitem(w_key) return ObjSpace.finditem(self, w_obj, w_key) - def setitem_str(self, w_obj, key, w_value, shadows_type=True): + def setitem_str(self, w_obj, key, w_value): """ Same as setitem, but takes string instead of any wrapped object - - XXX what shadows_type means??? """ if (isinstance(w_obj, W_DictMultiObject) and not w_obj.user_overridden_class): - w_obj.setitem_str(key, w_value, shadows_type) + w_obj.setitem_str(key, w_value) else: self.setitem(w_obj, self.wrap(key), w_value) Modified: pypy/branch/arm-backend/pypy/objspace/std/proxyobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/proxyobject.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/proxyobject.py Tue Nov 23 10:12:47 2010 @@ -43,7 +43,7 @@ raise return None - def setdictvalue(self, space, attr, w_value, shadows_type=True): + def setdictvalue(self, space, attr, w_value): try: space.call_function(self.w_controller, space.wrap('__setattr__'), space.wrap(attr), w_value) Modified: pypy/branch/arm-backend/pypy/objspace/std/setobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/setobject.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/setobject.py Tue Nov 23 10:12:47 2010 @@ -21,11 +21,10 @@ return False - def __init__(w_self, space, setdata=None): - if setdata is None: - w_self.setdata = r_dict(space.eq_w, space.hash_w) - else: - w_self.setdata = setdata.copy() + def __init__(w_self, space, setdata): + """Initialize the set by taking ownership of 'setdata'.""" + assert setdata is not None + w_self.setdata = setdata def __repr__(w_self): """representation for debugging purposes""" @@ -33,6 +32,7 @@ return "<%s(%s)>" % (w_self.__class__.__name__, ', '.join(reprlist)) def _newobj(w_self, space, rdict_w=None): + """Make a new set or frozenset by taking ownership of 'rdict_w'.""" #return space.call(space.type(w_self),W_SetIterObject(rdict_w)) objtype = type(w_self) if objtype is W_SetObject: @@ -55,10 +55,7 @@ class W_FrozensetObject(W_BaseSetObject): from pypy.objspace.std.frozensettype import frozenset_typedef as typedef - - def __init__(w_self, space, setdata): - W_BaseSetObject.__init__(w_self, space, setdata) - w_self.hash = -1 + hash = 0 registerimplementation(W_BaseSetObject) registerimplementation(W_SetObject) @@ -109,8 +106,14 @@ # some helper functions +def newset(space): + return r_dict(space.eq_w, space.hash_w) + def make_setdata_from_w_iterable(space, w_iterable=None): - data = r_dict(space.eq_w, space.hash_w) + """Return a new r_dict with the content of w_iterable.""" + if isinstance(w_iterable, W_BaseSetObject): + return w_iterable.setdata.copy() + data = newset(space) if w_iterable is not None: for w_item in space.listview(w_iterable): data[w_item] = None @@ -119,7 +122,7 @@ def _initialize_set(space, w_obj, w_iterable=None): w_obj.setdata.clear() if w_iterable is not None: - w_obj.setdata.update(make_setdata_from_w_iterable(space, w_iterable)) + w_obj.setdata = make_setdata_from_w_iterable(space, w_iterable) def _convert_set_to_frozenset(space, w_obj): if space.is_true(space.isinstance(w_obj, space.w_set)): @@ -130,12 +133,6 @@ # helper functions for set operation on dicts -def _is_frozenset_exact(w_obj): - if (w_obj is not None) and (type(w_obj) is W_FrozensetObject): - return True - else: - return False - def _is_eq(ld, rd): if len(ld) != len(rd): return False @@ -144,66 +141,41 @@ return False return True -def _union_dict(ldict, rdict, isupdate): - if isupdate: - ld = ldict - else: - ld = ldict.copy() - ld.update(rdict) - return ld, rdict - -def _difference_dict(ldict, rdict, isupdate): - if isupdate: - ld = ldict - else: - ld = ldict.copy() - del_list_w = [] +def _difference_dict(space, ld, rd): + result = newset(space) for w_key in ld: - if w_key in rdict: - del_list_w.append(w_key) - for w_key in del_list_w: - del ld[w_key] - - return ld, rdict - -def _intersection_dict(ldict, rdict, isupdate): - if isupdate: - ld = ldict - else: - ld = ldict.copy() - del_list_w = [] - for w_key in ld: - if w_key not in rdict: - del_list_w.append(w_key) - - for w_key in del_list_w: - del ld[w_key] - - return ld, rdict - + if w_key not in rd: + result[w_key] = None + return result -def _symmetric_difference_dict(ldict, rdict, isupdate): - if isupdate: - ld = ldict +def _difference_dict_update(space, ld, rd): + if ld is rd: + ld.clear() # for the case 'a.difference_update(a)' else: - ld = ldict.copy() - del_list_w = [] - add_list_w = [] + for w_key in rd: + try: + del ld[w_key] + except KeyError: + pass + +def _intersection_dict(space, ld, rd): + result = newset(space) + if len(ld) > len(rd): + ld, rd = rd, ld # loop over the smaller dict for w_key in ld: - if w_key in rdict: - del_list_w.append(w_key) + if w_key in rd: + result[w_key] = None + return result - for w_key in rdict: +def _symmetric_difference_dict(space, ld, rd): + result = newset(space) + for w_key in ld: + if w_key not in rd: + result[w_key] = None + for w_key in rd: if w_key not in ld: - add_list_w.append(w_key) - - for w_key in del_list_w: - del ld[w_key] - - for w_key in add_list_w: - ld[w_key] = None - - return ld, rdict + result[w_key] = None + return result def _issubset_dict(ldict, rdict): if len(ldict) > len(rdict): @@ -220,14 +192,13 @@ def set_update__Set_BaseSet(space, w_left, w_other): # optimization only (the general case works too) ld, rd = w_left.setdata, w_other.setdata - new_ld, rd = _union_dict(ld, rd, True) - return space.w_None + ld.update(rd) def set_update__Set_ANY(space, w_left, w_other): """Update a set with the union of itself and another.""" - ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) - new_ld, rd = _union_dict(ld, rd, True) - return space.w_None + ld = w_left.setdata + for w_item in space.listview(w_other): + ld[w_item] = None def inplace_or__Set_Set(space, w_left, w_other): set_update__Set_BaseSet(space, w_left, w_other) @@ -241,25 +212,23 @@ This has no effect if the element is already present. """ w_left.setdata[w_other] = None - return space.w_None def set_copy__Set(space, w_set): - return w_set._newobj(space,w_set.setdata) + return w_set._newobj(space, w_set.setdata.copy()) def frozenset_copy__Frozenset(space, w_left): - if _is_frozenset_exact(w_left): + if type(w_left) is W_FrozensetObject: return w_left else: - return set_copy__Set(space,w_left) + return set_copy__Set(space, w_left) def set_clear__Set(space, w_left): w_left.setdata.clear() - return space.w_None def set_difference__Set_Set(space, w_left, w_other): # optimization only (the general case works too) ld, rd = w_left.setdata, w_other.setdata - new_ld, rd = _difference_dict(ld, rd, False) + new_ld = _difference_dict(space, ld, rd) return w_left._newobj(space, new_ld) set_difference__Set_Frozenset = set_difference__Set_Set @@ -272,7 +241,7 @@ def set_difference__Set_ANY(space, w_left, w_other): ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) - new_ld, rd = _difference_dict(ld, rd, False) + new_ld = _difference_dict(space, ld, rd) return w_left._newobj(space, new_ld) frozenset_difference__Frozenset_ANY = set_difference__Set_ANY @@ -281,15 +250,17 @@ def set_difference_update__Set_Set(space, w_left, w_other): # optimization only (the general case works too) ld, rd = w_left.setdata, w_other.setdata - new_ld, rd = _difference_dict(ld, rd, True) - return space.w_None + _difference_dict_update(space, ld, rd) set_difference_update__Set_Frozenset = set_difference_update__Set_Set def set_difference_update__Set_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) - new_ld, rd = _difference_dict(ld, rd, True) - return space.w_None + ld = w_left.setdata + for w_key in space.listview(w_other): + try: + del ld[w_key] + except KeyError: + pass def inplace_sub__Set_Set(space, w_left, w_other): set_difference_update__Set_Set(space, w_left, w_other) @@ -467,7 +438,7 @@ def hash__Frozenset(space, w_set): multi = r_uint(1822399083) + r_uint(1822399083) + 1 - if w_set.hash != -1: + if w_set.hash != 0: return space.wrap(w_set.hash) hash = 1927868237 hash *= (len(w_set.setdata) + 1) @@ -476,7 +447,7 @@ value = ((h ^ (h << 16) ^ 89869747) * multi) hash = intmask(hash ^ value) hash = hash * 69069 + 907133923 - if hash == -1: + if hash == 0: hash = 590923713 hash = intmask(hash) w_set.hash = hash @@ -484,29 +455,31 @@ return space.wrap(hash) def set_pop__Set(space, w_left): - if len(w_left.setdata) == 0: + for w_key in w_left.setdata: + break + else: raise OperationError(space.w_KeyError, space.wrap('pop from an empty set')) - w_keys = w_left.setdata.keys() - w_value = w_keys[0] - del w_left.setdata[w_value] - - return w_value + del w_left.setdata[w_key] + return w_key def set_intersection__Set_Set(space, w_left, w_other): # optimization only (the general case works too) ld, rd = w_left.setdata, w_other.setdata - new_ld, rd = _intersection_dict(ld, rd, False) - return w_left._newobj(space,new_ld) + new_ld = _intersection_dict(space, ld, rd) + return w_left._newobj(space, new_ld) set_intersection__Set_Frozenset = set_intersection__Set_Set set_intersection__Frozenset_Frozenset = set_intersection__Set_Set set_intersection__Frozenset_Set = set_intersection__Set_Set def set_intersection__Set_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) - new_ld, rd = _intersection_dict(ld, rd, False) - return w_left._newobj(space,new_ld) + result = newset(space) + ld = w_left.setdata + for w_key in space.listview(w_other): + if w_key in ld: + result[w_key] = None + return w_left._newobj(space, result) frozenset_intersection__Frozenset_ANY = set_intersection__Set_ANY @@ -518,15 +491,18 @@ def set_intersection_update__Set_Set(space, w_left, w_other): # optimization only (the general case works too) ld, rd = w_left.setdata, w_other.setdata - new_ld, rd = _intersection_dict(ld, rd, True) - return space.w_None + new_ld = _intersection_dict(space, ld, rd) + w_left.setdata = new_ld set_intersection_update__Set_Frozenset = set_intersection_update__Set_Set def set_intersection_update__Set_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) - new_ld, rd = _intersection_dict(ld, rd, True) - return space.w_None + result = newset(space) + ld = w_left.setdata + for w_key in space.listview(w_other): + if w_key in ld: + result[w_key] = None + w_left.setdata = result def inplace_and__Set_Set(space, w_left, w_other): set_intersection_update__Set_Set(space, w_left, w_other) @@ -537,7 +513,7 @@ def set_symmetric_difference__Set_Set(space, w_left, w_other): # optimization only (the general case works too) ld, rd = w_left.setdata, w_other.setdata - new_ld, rd = _symmetric_difference_dict(ld, rd, False) + new_ld = _symmetric_difference_dict(space, ld, rd) return w_left._newobj(space, new_ld) set_symmetric_difference__Set_Frozenset = set_symmetric_difference__Set_Set @@ -553,7 +529,7 @@ def set_symmetric_difference__Set_ANY(space, w_left, w_other): ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) - new_ld, rd = _symmetric_difference_dict(ld, rd, False) + new_ld = _symmetric_difference_dict(space, ld, rd) return w_left._newobj(space, new_ld) frozenset_symmetric_difference__Frozenset_ANY = \ @@ -562,16 +538,16 @@ def set_symmetric_difference_update__Set_Set(space, w_left, w_other): # optimization only (the general case works too) ld, rd = w_left.setdata, w_other.setdata - new_ld, rd = _symmetric_difference_dict(ld, rd, True) - return space.w_None + new_ld = _symmetric_difference_dict(space, ld, rd) + w_left.setdata = new_ld set_symmetric_difference_update__Set_Frozenset = \ set_symmetric_difference_update__Set_Set def set_symmetric_difference_update__Set_ANY(space, w_left, w_other): ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) - new_ld, rd = _symmetric_difference_dict(ld, rd, True) - return space.w_None + new_ld = _symmetric_difference_dict(space, ld, rd) + w_left.setdata = new_ld def inplace_xor__Set_Set(space, w_left, w_other): set_symmetric_difference_update__Set_Set(space, w_left, w_other) @@ -582,8 +558,9 @@ def set_union__Set_Set(space, w_left, w_other): # optimization only (the general case works too) ld, rd = w_left.setdata, w_other.setdata - new_ld, rd = _union_dict(ld, rd, False) - return w_left._newobj(space, new_ld) + result = ld.copy() + result.update(rd) + return w_left._newobj(space, result) set_union__Set_Frozenset = set_union__Set_Set set_union__Frozenset_Set = set_union__Set_Set @@ -595,9 +572,11 @@ def set_union__Set_ANY(space, w_left, w_other): - ld, rd = w_left.setdata, make_setdata_from_w_iterable(space, w_other) - new_ld, rd = _union_dict(ld, rd, False) - return w_left._newobj(space, new_ld) + ld = w_left.setdata + result = ld.copy() + for w_key in space.listview(w_other): + result[w_key] = None + return w_left._newobj(space, result) frozenset_union__Frozenset_ANY = set_union__Set_ANY @@ -629,15 +608,6 @@ init_defaults) _initialize_set(space, w_set, w_iterable) -def init__Frozenset(space, w_set, __args__): - w_iterable, = __args__.parse_obj( - None, 'set', - init_signature, - init_defaults) - if w_set.hash == -1: - _initialize_set(space, w_set, w_iterable) - hash__Frozenset(space, w_set) - app = gateway.applevel(""" def setrepr(currently_in_repr, s): 'The app-level part of repr().' Modified: pypy/branch/arm-backend/pypy/objspace/std/settype.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/settype.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/settype.py Tue Nov 23 10:12:47 2010 @@ -66,9 +66,9 @@ register_all(vars(), globals()) def descr__new__(space, w_settype, __args__): - from pypy.objspace.std.setobject import W_SetObject + from pypy.objspace.std.setobject import W_SetObject, newset w_obj = space.allocate_instance(W_SetObject, w_settype) - W_SetObject.__init__(w_obj, space, None) + W_SetObject.__init__(w_obj, space, newset(space)) return w_obj set_typedef = StdTypeDef("set", Modified: pypy/branch/arm-backend/pypy/objspace/std/smallintobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/smallintobject.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/smallintobject.py Tue Nov 23 10:12:47 2010 @@ -2,18 +2,13 @@ Implementation of small ints, stored as odd-valued pointers in the translated PyPy. To enable them, see inttype.py. """ -import types -from pypy.interpreter.error import OperationError from pypy.objspace.std import intobject from pypy.objspace.std.model import registerimplementation, W_Object from pypy.objspace.std.register_all import register_all -from pypy.objspace.std.multimethod import FailedToImplementArgs from pypy.objspace.std.noneobject import W_NoneObject -from pypy.rlib.rarithmetic import ovfcheck, ovfcheck_lshift, LONG_BIT, r_uint -from pypy.objspace.std.inttype import wrapint -from pypy.objspace.std.intobject import W_IntObject, _impl_int_int_pow +from pypy.objspace.std.intobject import W_IntObject from pypy.rlib.objectmodel import UnboxedValue -from pypy.rlib.rbigint import rbigint +from pypy.tool.sourcetools import func_with_new_name class W_SmallIntObject(W_Object, UnboxedValue): @@ -46,9 +41,7 @@ new_name = name.replace("Int", "SmallInt") # Copy the function, so the annotator specializes it for # W_SmallIntObject. - ns[new_name] = types.FunctionType(func.func_code, ns, new_name, - func.func_defaults, - func.func_closure) + ns[new_name] = func_with_new_name(func, new_name) ns["get_integer"] = ns["pos__SmallInt"] = ns["int__SmallInt"] ns["get_negint"] = ns["neg__SmallInt"] Modified: pypy/branch/arm-backend/pypy/objspace/std/stringobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/stringobject.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/stringobject.py Tue Nov 23 10:12:47 2010 @@ -14,7 +14,7 @@ from pypy.rlib.rstring import StringBuilder, string_repeat from pypy.interpreter.buffer import StringBuffer -from pypy.objspace.std.stringtype import sliced, joined, wrapstr, wrapchar, \ +from pypy.objspace.std.stringtype import sliced, wrapstr, wrapchar, \ stringendswith, stringstartswith, joined2 from pypy.objspace.std.formatting import mod_format @@ -937,10 +937,17 @@ string = w_string._value chars = [] - for char in string: - w_char = W_StringObject.PREBUILT[ord(char)] - if not space.is_true(space.contains(w_deletechars, w_char)): - chars.append(table[ord(char)]) + deletechars = space.str_w(w_deletechars) + if len(deletechars) == 0: + for char in string: + chars.append(table[ord(char)]) + else: + deletion_table = [False] * 256 + for c in deletechars: + deletion_table[ord(c)] = True + for char in string: + if not deletion_table[ord(char)]: + chars.append(table[ord(char)]) return W_StringObject(''.join(chars)) def str_decode__String_ANY_ANY(space, w_string, w_encoding=None, w_errors=None): Modified: pypy/branch/arm-backend/pypy/objspace/std/stringtype.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/stringtype.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/stringtype.py Tue Nov 23 10:12:47 2010 @@ -55,19 +55,14 @@ return W_StringSliceObject(s, start, stop) return wrapstr(space, s[start:stop]) -def joined(space, strlist): - assert not space.config.objspace.std.withrope - if space.config.objspace.std.withstrjoin: - from pypy.objspace.std.strjoinobject import W_StringJoinObject - return W_StringJoinObject(strlist) - else: - return wrapstr(space, "".join(strlist)) - def joined2(space, str1, str2): assert not space.config.objspace.std.withrope if space.config.objspace.std.withstrjoin: from pypy.objspace.std.strjoinobject import W_StringJoinObject return W_StringJoinObject([str1, str2]) + elif space.config.objspace.std.withstrbuf: + from pypy.objspace.std.strbufobject import joined2 + return joined2(str1, str2) else: return wrapstr(space, str1 + str2) Modified: pypy/branch/arm-backend/pypy/objspace/std/strutil.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/strutil.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/strutil.py Tue Nov 23 10:12:47 2010 @@ -4,6 +4,7 @@ from pypy.rlib.rarithmetic import ovfcheck, break_up_float, parts_to_float,\ INFINITY, NAN +from pypy.rlib.rbigint import rbigint, parse_digit_string from pypy.interpreter.error import OperationError import math @@ -91,9 +92,10 @@ return -1 def string_to_int(s, base=10): - """Utility to converts a string to an integer (or possibly a long). + """Utility to converts a string to an integer. If base is 0, the proper base is guessed based on the leading characters of 's'. Raises ParseStringError in case of error. + Raises ParseStringOverflowError in case the result does not fit. """ s = literal = strip_spaces(s) p = NumberStringParser(s, literal, base, 'int') @@ -113,11 +115,9 @@ except OverflowError: raise ParseStringOverflowError(p) -def string_to_long(space, s, base=10, parser=None): - return string_to_w_long(space, s, base, parser).longval() - -def string_to_w_long(space, s, base=10, parser=None): - """As string_to_int(), but ignores an optional 'l' or 'L' suffix.""" +def string_to_bigint(s, base=10, parser=None): + """As string_to_int(), but ignores an optional 'l' or 'L' suffix + and returns an rbigint.""" if parser is None: s = literal = strip_spaces(s) if (s.endswith('l') or s.endswith('L')) and base < 22: @@ -126,18 +126,7 @@ p = NumberStringParser(s, literal, base, 'long') else: p = parser - w_base = space.newlong(p.base) - w_result = space.newlong(0) - while True: - digit = p.next_digit() - if digit == -1: - if p.sign == -1: - w_result = space.neg(w_result) - # XXX grumble - from pypy.objspace.std.longobject import W_LongObject - assert isinstance(w_result, W_LongObject) - return w_result - w_result = space.add(space.mul(w_result,w_base), space.newlong(digit)) + return parse_digit_string(p) # Tim's comment: # 57 bits are more than needed in any case. Modified: pypy/branch/arm-backend/pypy/objspace/std/test/test_dictmultiobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/test/test_dictmultiobject.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/test/test_dictmultiobject.py Tue Nov 23 10:12:47 2010 @@ -4,7 +4,6 @@ StrDictImplementation from pypy.objspace.std.celldict import ModuleDictImplementation -from pypy.objspace.std.sharingdict import SharedDictImplementation from pypy.conftest import gettestobjspace @@ -238,7 +237,17 @@ it1 = d.popitem() assert len(d) == 0 assert (it!=it1) and (it1==(1,2) or it1==(3,4)) - + raises(KeyError, d.popitem) + + def test_popitem_2(self): + class A(object): + pass + d = A().__dict__ + d['x'] = 5 + it1 = d.popitem() + assert it1 == ('x', 5) + raises(KeyError, d.popitem) + def test_setdefault(self): d = {1:2, 3:4} dd = d.copy() @@ -446,7 +455,9 @@ d1 = {} d2 = D(a='foo') d1.update(d2) - assert d1['a'] == 42 # fails on CPython, d1['a'] == 'foo' + assert d1['a'] == 'foo' + # a bit of an obscure case: now (from r78295) we get the same result + # as CPython does def test_index_keyerror_unpacking(self): d = {} @@ -510,32 +521,6 @@ assert getattr(a, s) == 42 -class TestW_DictSharing(TestW_DictObject): - def setup_class(cls): - cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True}) - -class AppTest_DictSharing(AppTest_DictObject): - def setup_class(cls): - cls.space = gettestobjspace(**{"objspace.std.withsharingdict": True}) - - def test_values_does_not_share(self): - class A(object): - pass - a = A() - a.abc = 12 - l = a.__dict__.values() - assert l == [12] - l[0] = 24 - assert a.abc == 12 - - def test_items(self): - class A(object): - pass - a = A() - a.abc = 12 - a.__dict__.items() == [("abc", 12)] - - class AppTestModuleDict(object): def setup_class(cls): cls.space = gettestobjspace(**{"objspace.std.withcelldict": True}) @@ -602,6 +587,15 @@ classofinstance=classofinstance, from_strdict_shared=from_strdict_shared) + def finditem_str(self, w_dict, s): + return w_dict.getitem_str(s) # assume it's a multidict + + def setitem_str(self, w_dict, s, w_value): + return w_dict.setitem_str(s, w_value) # assume it's a multidict + + def delitem(self, w_dict, w_s): + return w_dict.delitem(w_s) # assume it's a multidict + def allocate_instance(self, cls, type): return object.__new__(cls) @@ -611,7 +605,7 @@ w_StopIteration = StopIteration w_None = None StringObjectCls = FakeString - w_dict = None + w_dict = W_DictMultiObject iter = iter fixedview = list listview = list @@ -620,10 +614,9 @@ class objspace: class std: withdictmeasurement = False - withsharingdict = False withsmalldicts = False withcelldict = False - withshadowtracking = False + withmethodcache = False class opcodes: CALL_LIKELY_BUILTIN = False @@ -687,6 +680,14 @@ assert self.impl.length() == 0 self.check_not_devolved() + def test_clear(self): + self.fill_impl() + assert self.impl.length() == 2 + self.impl.clear() + assert self.impl.length() == 0 + self.check_not_devolved() + + def test_keys(self): self.fill_impl() keys = self.impl.keys() @@ -750,9 +751,6 @@ string = "int" string2 = "isinstance" -class TestSharedDictImplementation(BaseTestRDictImplementation): - ImplementionClass = SharedDictImplementation - class BaseTestDevolvedDictImplementation(BaseTestRDictImplementation): def fill_impl(self): @@ -774,8 +772,6 @@ string = "int" string2 = "isinstance" -class TestDevolvedSharedDictImplementation(BaseTestDevolvedDictImplementation): - ImplementionClass = SharedDictImplementation def test_module_uses_strdict(): fakespace = FakeSpace() Modified: pypy/branch/arm-backend/pypy/objspace/std/test/test_longobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/test/test_longobject.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/test/test_longobject.py Tue Nov 23 10:12:47 2010 @@ -41,56 +41,64 @@ class AppTestLong: def test_add(self): - assert int(123L + 12443L) == 123 + 12443 - assert -20 + 2 + 3L + True == -14L + x = 123L + assert int(x + 12443L) == 123 + 12443 + x = -20 + assert x + 2 + 3L + True == -14L def test_sub(self): - assert int(58543L - 12332L) == 58543 - 12332 - assert 237123838281233L * 12 == 237123838281233L * 12L + x = 58543L + assert int(x - 12332L) == 58543 - 12332 + x = 237123838281233L + assert x * 12 == x * 12L def test_mul(self): - assert 363L * 2 ** 40 == 363L << 40 + x = 363L + assert x * 2 ** 40 == x << 40 def test_truediv(self): exec "from __future__ import division; a = 31415926L / 10000000L" assert a == 3.1415926 def test_floordiv(self): - a = 31415926L // 10000000L + x = 31415926L + a = x // 10000000L assert a == 3L def test_compare(self): + Z = 0 + ZL = 0L for BIG in (1L, 1L << 62, 1L << 9999): - assert 0 == 0L - assert not (0 != 0L) - assert 0L == 0 - assert not (0L != 0) - assert not (0 == BIG) - assert 0 != BIG - assert not (BIG == 0) - assert BIG != 0 - assert not (0L == BIG) - assert 0L != BIG - assert 0 <= 0L - assert not (0 < 0L) - assert 0 <= BIG - assert 0 < BIG - assert not (BIG <= 0) - assert not (BIG < 0) - assert 0L <= 0L - assert not (0L < 0L) - assert 0L <= BIG - assert 0L < BIG - assert not (BIG <= 0L) - assert not (BIG < 0L) - assert not (0 <= -BIG) - assert not (0 < -BIG) - assert -BIG <= 0 - assert -BIG < 0 - assert not (0L <= -BIG) - assert not (0L < -BIG) - assert -BIG <= 0L - assert -BIG < 0L + assert Z == ZL + assert not (Z != ZL) + assert ZL == Z + assert not (ZL != Z) + assert not (Z == BIG) + assert Z != BIG + assert not (BIG == Z) + assert BIG != Z + assert not (ZL == BIG) + assert ZL != BIG + assert Z <= ZL + assert not (Z < ZL) + assert Z <= BIG + assert Z < BIG + assert not (BIG <= Z) + assert not (BIG < Z) + assert ZL <= ZL + assert not (ZL < ZL) + assert ZL <= BIG + assert ZL < BIG + assert not (BIG <= ZL) + assert not (BIG < ZL) + assert not (Z <= -BIG) + assert not (Z < -BIG) + assert -BIG <= Z + assert -BIG < Z + assert not (ZL <= -BIG) + assert not (ZL < -BIG) + assert -BIG <= ZL + assert -BIG < ZL # assert not (BIG < int(BIG)) assert (BIG <= int(BIG)) @@ -137,7 +145,8 @@ def test_conversion(self): class long2(long): pass - x = long2(1L<<100) + x = 1L + x = long2(x<<100) y = int(x) assert type(y) == long assert type(+long2(5)) is long @@ -152,7 +161,8 @@ assert type(long2(5) // 1) is long def test_pow(self): - assert pow(0L, 0L, 1L) == 0L + x = 0L + assert pow(x, 0L, 1L) == 0L def test_getnewargs(self): assert 0L .__getnewargs__() == (0L,) @@ -193,13 +203,14 @@ assert oct(01234567012345670L) == '01234567012345670L' def test_bits(self): - assert 0xAAAAAAAAL | 0x55555555L == 0xFFFFFFFFL - assert 0xAAAAAAAAL & 0x55555555L == 0x00000000L - assert 0xAAAAAAAAL ^ 0x55555555L == 0xFFFFFFFFL - assert -0xAAAAAAAAL | 0x55555555L == -0xAAAAAAA9L - assert 0xAAAAAAAAL | 0x555555555L == 0x5FFFFFFFFL - assert 0xAAAAAAAAL & 0x555555555L == 0x000000000L - assert 0xAAAAAAAAL ^ 0x555555555L == 0x5FFFFFFFFL + x = 0xAAAAAAAAL + assert x | 0x55555555L == 0xFFFFFFFFL + assert x & 0x55555555L == 0x00000000L + assert x ^ 0x55555555L == 0xFFFFFFFFL + assert -x | 0x55555555L == -0xAAAAAAA9L + assert x | 0x555555555L == 0x5FFFFFFFFL + assert x & 0x555555555L == 0x000000000L + assert x ^ 0x555555555L == 0x5FFFFFFFFL def test_hash(self): # ints have the same hash as equal longs @@ -229,7 +240,8 @@ def test_huge_longs(self): import operator - huge = 1L << 40000L + x = 1L + huge = x << 40000L raises(OverflowError, float, huge) raises(OverflowError, operator.truediv, huge, 3) raises(OverflowError, operator.truediv, huge, 3L) @@ -242,3 +254,7 @@ class myotherlong(long): pass assert long(myotherlong(21)) == 21L + + def test_negative_zero(self): + x = eval("-0L") + assert x == 0L Modified: pypy/branch/arm-backend/pypy/objspace/std/test/test_setobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/test/test_setobject.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/test/test_setobject.py Tue Nov 23 10:12:47 2010 @@ -10,6 +10,7 @@ import py.test from pypy.objspace.std.setobject import W_SetObject, W_FrozensetObject from pypy.objspace.std.setobject import _initialize_set +from pypy.objspace.std.setobject import newset, make_setdata_from_w_iterable from pypy.objspace.std.setobject import set_intersection__Set_Set from pypy.objspace.std.setobject import set_intersection__Set_ANY from pypy.objspace.std.setobject import eq__Set_Set @@ -28,12 +29,12 @@ self.false = self.space.w_False def test_and(self): - s = W_SetObject(self.space, None) + s = W_SetObject(self.space, newset(self.space)) _initialize_set(self.space, s, self.word) - t0 = W_SetObject(self.space, None) + t0 = W_SetObject(self.space, newset(self.space)) _initialize_set(self.space, t0, self.otherword) - t1 = W_FrozensetObject(self.space, None) - _initialize_set(self.space, t1, self.otherword) + t1 = W_FrozensetObject(self.space, + make_setdata_from_w_iterable(self.space, self.otherword)) r0 = set_intersection__Set_Set(self.space, s, t0) r1 = set_intersection__Set_Set(self.space, s, t1) assert eq__Set_Set(self.space, r0, r1) == self.true @@ -41,9 +42,9 @@ assert eq__Set_Set(self.space, r0, sr) == self.true def test_compare(self): - s = W_SetObject(self.space, None) + s = W_SetObject(self.space, newset(self.space)) _initialize_set(self.space, s, self.word) - t = W_SetObject(self.space, None) + t = W_SetObject(self.space, newset(self.space)) _initialize_set(self.space, t, self.word) assert self.space.eq_w(s,t) u = self.space.wrap(set('simsalabim')) @@ -56,6 +57,11 @@ b = a | set('abc') assert type(b) is subset + def test_union(self): + a = set([4, 5]) + b = a.union([5, 7]) + assert sorted(b) == [4, 5, 7] + def test_compare(self): raises(TypeError, cmp, set('abc'), set('abd')) assert set('abc') != 'abc' Modified: pypy/branch/arm-backend/pypy/objspace/std/test/test_strutil.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/test/test_strutil.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/test/test_strutil.py Tue Nov 23 10:12:47 2010 @@ -25,7 +25,7 @@ ] for s, expected in cases: assert string_to_int(s) == expected - assert string_to_w_long(space, s).longval() == expected + assert string_to_bigint(s).tolong() == expected def test_string_to_int_base(self): space = self.space @@ -119,17 +119,16 @@ raises(ParseStringError, string_to_int, '+'+s, base) raises(ParseStringError, string_to_int, '-'+s, base) - def test_string_to_w_long(self): - space = self.space - assert string_to_w_long(space, '123L').longval() == 123 - assert string_to_w_long(space, '123L ').longval() == 123 - raises(ParseStringError, string_to_w_long, space, 'L') - raises(ParseStringError, string_to_w_long, space, 'L ') - assert string_to_w_long(space, '123L', 4).longval() == 27 - assert string_to_w_long(space, '123L', 30).longval() == 27000 + 1800 + 90 + 21 - assert string_to_w_long(space, '123L', 22).longval() == 10648 + 968 + 66 + 21 - assert string_to_w_long(space, '123L', 21).longval() == 441 + 42 + 3 - assert string_to_w_long(space, '1891234174197319').longval() == 1891234174197319 + def test_string_to_bigint(self): + assert string_to_bigint('123L').tolong() == 123 + assert string_to_bigint('123L ').tolong() == 123 + raises(ParseStringError, string_to_bigint, 'L') + raises(ParseStringError, string_to_bigint, 'L ') + assert string_to_bigint('123L', 4).tolong() == 27 + assert string_to_bigint('123L', 30).tolong() == 27000 + 1800 + 90 + 21 + assert string_to_bigint('123L', 22).tolong() == 10648 + 968 + 66 + 21 + assert string_to_bigint('123L', 21).tolong() == 441 + 42 + 3 + assert string_to_bigint('1891234174197319').tolong() == 1891234174197319 def test_string_to_float(self): def string_to_float(x): Modified: pypy/branch/arm-backend/pypy/objspace/std/test/test_typeobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/test/test_typeobject.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/test/test_typeobject.py Tue Nov 23 10:12:47 2010 @@ -1085,4 +1085,12 @@ assert b == 1 - + def test_slots_with_method_in_class(self): + # this works in cpython... + class A(object): + __slots__ = ["f"] + def f(self, x): + return x + 1 + a = A() + assert a.f(1) == 2 + Modified: pypy/branch/arm-backend/pypy/objspace/std/test/test_versionedtype.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/test/test_versionedtype.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/test/test_versionedtype.py Tue Nov 23 10:12:47 2010 @@ -12,10 +12,12 @@ def f(self): pass class B(A): pass - class metatype(type): + class X: + pass + class Y(object): + pass + class C(Y, X): pass - class C(object): - __metaclass__ = metatype return A, B, C """) return space.unpackiterable(w_types) Modified: pypy/branch/arm-backend/pypy/objspace/std/typeobject.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/std/typeobject.py (original) +++ pypy/branch/arm-backend/pypy/objspace/std/typeobject.py Tue Nov 23 10:12:47 2010 @@ -60,6 +60,15 @@ self.hits = {} self.misses = {} + def clear(self): + None_None = (None, None) + for i in range(len(self.versions)): + self.versions[i] = None + for i in range(len(self.names)): + self.names[i] = None + for i in range(len(self.lookup_where)): + self.lookup_where[i] = None_None + class W_TypeObject(W_Object): from pypy.objspace.std.typetype import type_typedef as typedef @@ -75,7 +84,9 @@ 'weakrefable', 'hasdict', 'nslots', - 'instancetypedef'] + 'instancetypedef', + 'terminator', + ] # for config.objspace.std.getattributeshortcut # (False is a conservative default, fixed during real usage) @@ -102,20 +113,24 @@ if overridetypedef is not None: setup_builtin_type(w_self) - custom_metaclass = False else: setup_user_defined_type(w_self) - custom_metaclass = not space.is_w(space.type(w_self), space.w_type) w_self.w_same_layout_as = get_parent_layout(w_self) if space.config.objspace.std.withtypeversion: - if custom_metaclass or not is_mro_purely_of_types(w_self.mro_w): + if not is_mro_purely_of_types(w_self.mro_w): pass else: # the _version_tag should change, whenever the content of # dict_w of any of the types in the mro changes, or if the mro # itself changes w_self._version_tag = VersionTag() + if space.config.objspace.std.withmapdict: + from pypy.objspace.std.mapdict import DictTerminator, NoDictTerminator + if w_self.hasdict: + w_self.terminator = DictTerminator(space, w_self) + else: + w_self.terminator = NoDictTerminator(space, w_self) def mutated(w_self): space = w_self.space @@ -551,11 +566,13 @@ space.wrap('__slots__ must be identifiers')) # create member slot_name = _mangle(slot_name, w_self.name) - # Force interning of slot names. - slot_name = space.str_w(space.new_interned_str(slot_name)) - member = Member(w_self.nslots, slot_name, w_self) - w_self.dict_w[slot_name] = space.wrap(member) - w_self.nslots += 1 + if slot_name not in w_self.dict_w: + # Force interning of slot names. + slot_name = space.str_w(space.new_interned_str(slot_name)) + # in cpython it is ignored less, but we probably don't care + member = Member(w_self.nslots, slot_name, w_self) + w_self.dict_w[slot_name] = space.wrap(member) + w_self.nslots += 1 def create_dict_slot(w_self): if not w_self.hasdict: Modified: pypy/branch/arm-backend/pypy/objspace/test/test_descroperation.py ============================================================================== --- pypy/branch/arm-backend/pypy/objspace/test/test_descroperation.py (original) +++ pypy/branch/arm-backend/pypy/objspace/test/test_descroperation.py Tue Nov 23 10:12:47 2010 @@ -400,7 +400,7 @@ class D(object): def __init__(self, a): self.a = a - + assert A(1) == B(1) assert B(1) == A(1) assert A(1) == C(1) @@ -457,6 +457,20 @@ else: assert False, "did not raise" + def test_attribute_error(self): + class classmethodonly(classmethod): + def __get__(self, instance, type): + if instance is not None: + raise AttributeError("Must be called on a class, not an instance.") + return super(classmethodonly, self).__get__(instance, type) + + class A(object): + @classmethodonly + def a(cls): + return 3 + + raises(AttributeError, lambda: A().a) + class AppTestWithBuiltinShortcut(AppTest_Descroperation): OPTIONS = {'objspace.std.builtinshortcut': True} Modified: pypy/branch/arm-backend/pypy/rlib/_rsocket_rffi.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/_rsocket_rffi.py (original) +++ pypy/branch/arm-backend/pypy/rlib/_rsocket_rffi.py Tue Nov 23 10:12:47 2010 @@ -324,10 +324,11 @@ ('sll_hatype', rffi.INT), ('sll_addr', rffi.CFixedArray(rffi.CHAR, 8)), ('sll_halen', rffi.INT)], - ) + ifdef='AF_PACKET') CConfig.ifreq = platform.Struct('struct ifreq', [('ifr_ifindex', rffi.INT), - ('ifr_name', rffi.CFixedArray(rffi.CHAR, 8))]) + ('ifr_name', rffi.CFixedArray(rffi.CHAR, 8))], + ifdef='AF_PACKET') if _WIN32: CConfig.WSAEVENT = platform.SimpleType('WSAEVENT', rffi.VOIDP) @@ -532,8 +533,9 @@ socketpair_t = rffi.CArray(socketfd_type) socketpair = external('socketpair', [rffi.INT, rffi.INT, rffi.INT, lltype.Ptr(socketpair_t)], rffi.INT) - ioctl = external('ioctl', [socketfd_type, rffi.INT, lltype.Ptr(ifreq)], - rffi.INT) + if ifreq is not None: + ioctl = external('ioctl', [socketfd_type, rffi.INT, lltype.Ptr(ifreq)], + rffi.INT) if _WIN32: ioctlsocket = external('ioctlsocket', Modified: pypy/branch/arm-backend/pypy/rlib/jit.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/jit.py (original) +++ pypy/branch/arm-backend/pypy/rlib/jit.py Tue Nov 23 10:12:47 2010 @@ -4,6 +4,7 @@ from pypy.rlib.objectmodel import CDefinedIntSymbolic from pypy.rlib.objectmodel import keepalive_until_here from pypy.rlib.unroll import unrolling_iterable +from pypy.rlib.nonconst import NonConstant def purefunction(func): """ Decorate a function as pure. Pure means precisely that: @@ -77,6 +78,12 @@ return result return decorator +def oopspec(spec): + def decorator(func): + func.oopspec = spec + return func + return decorator + class Entry(ExtRegistryEntry): _about_ = hint @@ -139,6 +146,32 @@ return hop.inputconst(lltype.Signed, _we_are_jitted) +def current_trace_length(): + """During JIT tracing, returns the current trace length (as a constant). + If not tracing, returns -1.""" + if NonConstant(False): + return 73 + return -1 +current_trace_length.oopspec = 'jit.current_trace_length()' + +def jit_debug(string, arg1=-sys.maxint-1, arg2=-sys.maxint-1, + arg3=-sys.maxint-1, arg4=-sys.maxint-1): + """When JITted, cause an extra operation DEBUG_MERGE_POINT to appear in + the graphs. Should not be left after debugging.""" + keepalive_until_here(string) # otherwise the whole function call is removed +jit_debug.oopspec = 'jit.debug(string, arg1, arg2, arg3, arg4)' + +def assert_green(value): + """Very strong assert: checks that 'value' is a green + (a JIT compile-time constant).""" + keepalive_until_here(value) +assert_green._annspecialcase_ = 'specialize:argtype(0)' +assert_green.oopspec = 'jit.assert_green(value)' + +class AssertGreenFailed(Exception): + pass + + ##def force_virtualizable(virtualizable): ## pass @@ -250,8 +283,9 @@ several independent JITting interpreters in it. """ + active = True # if set to False, this JitDriver is ignored virtualizables = [] - + def __init__(self, greens=None, reds=None, virtualizables=None, get_jitcell_at=None, set_jitcell_at=None, get_printable_location=None, confirm_enter_jit=None, @@ -266,7 +300,8 @@ self.virtualizables = virtualizables for v in self.virtualizables: assert v in self.reds - self._alllivevars = dict.fromkeys(self.greens + self.reds) + self._alllivevars = dict.fromkeys( + [name for name in self.greens + self.reds if '.' not in name]) self._make_extregistryentries() self.get_jitcell_at = get_jitcell_at self.set_jitcell_at = set_jitcell_at @@ -355,10 +390,16 @@ def compute_result_annotation(self, **kwds_s): from pypy.annotation import model as annmodel + + if self.instance.__name__ == 'jit_merge_point': + if not self.annotate_hooks(**kwds_s): + return None # wrong order, try again later + driver = self.instance.im_self keys = kwds_s.keys() keys.sort() - expected = ['s_' + name for name in driver.greens + driver.reds] + expected = ['s_' + name for name in driver.greens + driver.reds + if '.' not in name] expected.sort() if keys != expected: raise JitHintError("%s expects the following keyword " @@ -382,30 +423,35 @@ key[2:]) cache[key] = s_value - if self.instance.__name__ == 'jit_merge_point': - self.annotate_hooks(**kwds_s) - return annmodel.s_None def annotate_hooks(self, **kwds_s): driver = self.instance.im_self s_jitcell = self.bookkeeper.valueoftype(BaseJitCell) - self.annotate_hook(driver.get_jitcell_at, driver.greens, **kwds_s) - self.annotate_hook(driver.set_jitcell_at, driver.greens, [s_jitcell], - **kwds_s) - self.annotate_hook(driver.get_printable_location, driver.greens, **kwds_s) + h = self.annotate_hook + return (h(driver.get_jitcell_at, driver.greens, **kwds_s) + and h(driver.set_jitcell_at, driver.greens, [s_jitcell], **kwds_s) + and h(driver.get_printable_location, driver.greens, **kwds_s)) def annotate_hook(self, func, variables, args_s=[], **kwds_s): if func is None: - return + return True bk = self.bookkeeper s_func = bk.immutablevalue(func) uniquekey = 'jitdriver.%s' % func.func_name args_s = args_s[:] for name in variables: - s_arg = kwds_s['s_' + name] + if '.' not in name: + s_arg = kwds_s['s_' + name] + else: + objname, fieldname = name.split('.') + s_instance = kwds_s['s_' + objname] + s_arg = s_instance.classdef.about_attribute(fieldname) + if s_arg is None: + return False # wrong order, try again later args_s.append(s_arg) bk.emulate_pbc_call(uniquekey, s_func, args_s) + return True def specialize_call(self, hop, **kwds_i): # XXX to be complete, this could also check that the concretetype @@ -416,9 +462,42 @@ greens_v = [] reds_v = [] for name in driver.greens: - i = kwds_i['i_' + name] - r_green = hop.args_r[i] - v_green = hop.inputarg(r_green, arg=i) + if '.' not in name: + i = kwds_i['i_' + name] + r_green = hop.args_r[i] + v_green = hop.inputarg(r_green, arg=i) + else: + if hop.rtyper.type_system.name == 'ootypesystem': + py.test.skip("lltype only") + objname, fieldname = name.split('.') # see test_green_field + assert objname in driver.reds + i = kwds_i['i_' + objname] + s_red = hop.args_s[i] + r_red = hop.args_r[i] + while True: + try: + mangled_name, r_field = r_red._get_field(fieldname) + break + except KeyError: + pass + assert r_red.rbase is not None, ( + "field %r not found in %r" % (name, + r_red.lowleveltype.TO)) + r_red = r_red.rbase + GTYPE = r_red.lowleveltype.TO + assert GTYPE._immutable_field(mangled_name), ( + "field %r must be declared as immutable" % name) + if not hasattr(driver, 'll_greenfields'): + driver.ll_greenfields = {} + driver.ll_greenfields[name] = GTYPE, mangled_name + # + v_red = hop.inputarg(r_red, arg=i) + c_llname = hop.inputconst(lltype.Void, mangled_name) + v_green = hop.genop('getfield', [v_red, c_llname], + resulttype = r_field) + s_green = s_red.classdef.about_attribute(fieldname) + assert s_green is not None + hop.rtyper.annotator.setbinding(v_green, s_green) greens_v.append(v_green) for name in driver.reds: i = kwds_i['i_' + name] Modified: pypy/branch/arm-backend/pypy/rlib/objectmodel.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/objectmodel.py (original) +++ pypy/branch/arm-backend/pypy/rlib/objectmodel.py Tue Nov 23 10:12:47 2010 @@ -475,6 +475,10 @@ def setdefault(self, key, default): return self._dict.setdefault(_r_dictkey(self, key), default) + def popitem(self): + dk, value = self._dict.popitem() + return dk.key, value + def copy(self): result = r_dict(self.key_eq, self.key_hash) result.update(self) Modified: pypy/branch/arm-backend/pypy/rlib/rarithmetic.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/rarithmetic.py (original) +++ pypy/branch/arm-backend/pypy/rlib/rarithmetic.py Tue Nov 23 10:12:47 2010 @@ -501,6 +501,11 @@ def __cmp__(self, other): raise TypeError("not supported on r_singlefloat instances") + def __eq__(self, other): + return self.__class__ is other.__class__ and self._bytes == other._bytes + + def __ne__(self, other): + return not self.__eq__(other) class For_r_singlefloat_values_Entry(extregistry.ExtRegistryEntry): _type_ = r_singlefloat Modified: pypy/branch/arm-backend/pypy/rlib/rbigint.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/rbigint.py (original) +++ pypy/branch/arm-backend/pypy/rlib/rbigint.py Tue Nov 23 10:12:47 2010 @@ -1596,11 +1596,16 @@ # a few internal helpers -DEC_PER_DIGIT = 1 -while int('9' * DEC_PER_DIGIT) < MASK: - DEC_PER_DIGIT += 1 -DEC_PER_DIGIT -= 1 -DEC_MAX = 10 ** DEC_PER_DIGIT +def digits_max_for_base(base): + dec_per_digit = 1 + while base ** dec_per_digit < MASK: + dec_per_digit += 1 + dec_per_digit -= 1 + return base ** dec_per_digit + +BASE_MAX = [0, 0] + [digits_max_for_base(_base) for _base in range(2, 37)] +DEC_MAX = digits_max_for_base(10) +assert DEC_MAX == BASE_MAX[10] def _decimalstr_to_bigint(s): # a string that has been already parsed to be decimal and valid, @@ -1615,7 +1620,6 @@ p += 1 a = rbigint.fromint(0) - cnt = DEC_PER_DIGIT tens = 1 dig = 0 ord0 = ord('0') @@ -1627,8 +1631,26 @@ a = _muladd1(a, tens, dig) tens = 1 dig = 0 - if sign: + if sign and a.sign == 1: a.sign = -1 return a - +def parse_digit_string(parser): + # helper for objspace.std.strutil + a = rbigint.fromint(0) + base = parser.base + digitmax = BASE_MAX[base] + tens, dig = 1, 0 + while True: + digit = parser.next_digit() + if tens == digitmax or digit < 0: + a = _muladd1(a, tens, dig) + if digit < 0: + break + dig = digit + tens = base + else: + dig = dig * base + digit + tens *= base + a.sign *= parser.sign + return a Modified: pypy/branch/arm-backend/pypy/rlib/rgc.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/rgc.py (original) +++ pypy/branch/arm-backend/pypy/rlib/rgc.py Tue Nov 23 10:12:47 2010 @@ -170,7 +170,14 @@ return hop.genop('gc_set_max_heap_size', [v_nbytes], resulttype=lltype.Void) -def can_move(p): # NB. must not be called with NULL pointers +def can_move(p): + """Check if the GC object 'p' is at an address that can move. + Must not be called with None. With non-moving GCs, it is always False. + With some moving GCs like the SemiSpace GC, it is always True. + With other moving GCs like the MiniMark GC, it can be True for some + time, then False for the same object, when we are sure that it won't + move any more. + """ return True class CanMoveEntry(ExtRegistryEntry): @@ -372,6 +379,11 @@ "NOT_RPYTHON" raise NotImplementedError +def get_typeids_z(): + "NOT_RPYTHON" + raise NotImplementedError + +ARRAY_OF_CHAR = lltype.Array(lltype.Char) NULL_GCREF = lltype.nullptr(llmemory.GCREF.TO) class _GcRef(object): @@ -523,3 +535,12 @@ vlist = hop.inputargs(lltype.Signed) hop.exception_is_here() return hop.genop('gc_dump_rpy_heap', vlist, resulttype = hop.r_result) + +class Entry(ExtRegistryEntry): + _about_ = get_typeids_z + def compute_result_annotation(self): + from pypy.annotation.model import SomePtr + return SomePtr(lltype.Ptr(ARRAY_OF_CHAR)) + def specialize_call(self, hop): + hop.exception_is_here() + return hop.genop('gc_typeids_z', [], resulttype = hop.r_result) Modified: pypy/branch/arm-backend/pypy/rlib/rmmap.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/rmmap.py (original) +++ pypy/branch/arm-backend/pypy/rlib/rmmap.py Tue Nov 23 10:12:47 2010 @@ -50,7 +50,7 @@ constant_names = ['MAP_SHARED', 'MAP_PRIVATE', 'PROT_READ', 'PROT_WRITE', 'MS_SYNC'] - opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS', + opt_constant_names = ['MAP_ANON', 'MAP_ANONYMOUS', 'MAP_NORESERVE', 'PROT_EXEC', 'MAP_DENYWRITE', 'MAP_EXECUTABLE'] for name in constant_names: Modified: pypy/branch/arm-backend/pypy/rlib/rsre/rsre_char.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/rsre/rsre_char.py (original) +++ pypy/branch/arm-backend/pypy/rlib/rsre/rsre_char.py Tue Nov 23 10:12:47 2010 @@ -4,6 +4,7 @@ import sys from pypy.rlib.rlocale import tolower, isalnum from pypy.rlib.unroll import unrolling_iterable +from pypy.rlib import jit # Note: the unicode parts of this module require you to call # rsre_char.set_unicode_db() first, to select one of the modules @@ -43,6 +44,7 @@ # XXX can we import those safely from sre_constants? SRE_INFO_PREFIX = 1 SRE_INFO_LITERAL = 2 +SRE_INFO_CHARSET = 4 SRE_FLAG_LOCALE = 4 # honour system locale SRE_FLAG_UNICODE = 32 # use unicode locale OPCODE_INFO = 17 @@ -64,33 +66,27 @@ #### Category helpers -ascii_char_info = [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 6, 2, -2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, -0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 25, 25, 25, 25, 25, 25, 25, 25, -25, 25, 0, 0, 0, 0, 0, 0, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, -24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0, -0, 0, 16, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, -24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0, 0, 0, 0 ] - +is_a_word = [(chr(i).isalnum() or chr(i) == '_') for i in range(256)] linebreak = ord("\n") underline = ord("_") def is_digit(code): - return code < 128 and (ascii_char_info[code] & 1 != 0) + return code <= 57 and code >= 48 def is_uni_digit(code): assert unicodedb is not None return unicodedb.isdigit(code) def is_space(code): - return code < 128 and (ascii_char_info[code] & 2 != 0) + return code == 32 or (code <= 13 and code >= 9) def is_uni_space(code): assert unicodedb is not None return unicodedb.isspace(code) def is_word(code): - return code < 128 and (ascii_char_info[code] & 16 != 0) + assert code >= 0 + return code < 256 and is_a_word[code] def is_uni_word(code): assert unicodedb is not None @@ -142,6 +138,7 @@ SET_OK = -1 SET_NOT_OK = -2 + at jit.unroll_safe def check_charset(pattern, ppos, char_code): """Checks whether a character matches set of arbitrary length. The set starts at pattern[ppos].""" Modified: pypy/branch/arm-backend/pypy/rlib/rsre/rsre_core.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/rsre/rsre_core.py (original) +++ pypy/branch/arm-backend/pypy/rlib/rsre/rsre_core.py Tue Nov 23 10:12:47 2010 @@ -1,9 +1,11 @@ import sys -from pypy.rlib.debug import check_nonneg +from pypy.rlib.debug import check_nonneg, make_sure_not_modified from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.rsre import rsre_char from pypy.tool.sourcetools import func_with_new_name from pypy.rlib.objectmodel import we_are_translated +from pypy.rlib import jit +from pypy.rlib.rsre.rsre_jit import install_jitdriver, install_jitdriver_spec OPCODE_FAILURE = 0 @@ -56,16 +58,19 @@ _seen_specname[specname] = True # Install a copy of the function under the name '_spec_funcname' in each # concrete subclass + specialized_methods = [] for prefix, concreteclass in [('str', StrMatchContext), ('uni', UnicodeMatchContext)]: newfunc = func_with_new_name(func, prefix + specname) assert not hasattr(concreteclass, specname) setattr(concreteclass, specname, newfunc) + specialized_methods.append(newfunc) # Return a dispatcher function, specialized on the exact type of 'ctx' def dispatch(ctx, *args): return getattr(ctx, specname)(*args) dispatch._annspecialcase_ = 'specialize:argtype(0)' - return dispatch + dispatch._specialized_methods_ = specialized_methods + return func_with_new_name(dispatch, specname) # ____________________________________________________________ @@ -75,6 +80,7 @@ class AbstractMatchContext(object): """Abstract base class""" + _immutable_fields_ = ['pattern[*]', 'flags', 'end'] match_start = 0 match_end = 0 match_marks = None @@ -164,6 +170,8 @@ def __init__(self, pattern, string, match_start, end, flags): AbstractMatchContext.__init__(self, pattern, match_start, end, flags) self._string = string + if not we_are_translated() and isinstance(string, unicode): + self.flags |= rsre_char.SRE_FLAG_UNICODE # for rsre_re.py def str(self, index): check_nonneg(index) @@ -238,8 +246,9 @@ self.start_ptr = ptr self.start_marks = marks + @jit.unroll_safe def find_first_result(self, ctx): - ppos = self.ppos + ppos = jit.hint(self.ppos, promote=True) while ctx.pat(ppos): result = sre_match(ctx, ppos + 1, self.start_ptr, self.start_marks) ppos += ctx.pat(ppos) @@ -250,6 +259,10 @@ find_next_result = find_first_result class RepeatOneMatchResult(MatchResult): + install_jitdriver('RepeatOne', + greens=['nextppos', 'ctx.pattern'], + reds=['ptr', 'self', 'ctx'], + debugprint=(1, 0)) # indices in 'greens' def __init__(self, nextppos, minptr, ptr, marks): self.nextppos = nextppos @@ -259,8 +272,11 @@ def find_first_result(self, ctx): ptr = self.start_ptr + nextppos = self.nextppos while ptr >= self.minptr: - result = sre_match(ctx, self.nextppos, ptr, self.start_marks) + ctx.jitdriver_RepeatOne.jit_merge_point( + self=self, ptr=ptr, ctx=ctx, nextppos=nextppos) + result = sre_match(ctx, nextppos, ptr, self.start_marks) ptr -= 1 if result is not None: self.subresult = result @@ -270,6 +286,10 @@ class MinRepeatOneMatchResult(MatchResult): + install_jitdriver('MinRepeatOne', + greens=['nextppos', 'ppos3', 'ctx.pattern'], + reds=['ptr', 'self', 'ctx'], + debugprint=(2, 0)) # indices in 'greens' def __init__(self, nextppos, ppos3, maxptr, ptr, marks): self.nextppos = nextppos @@ -280,29 +300,32 @@ def find_first_result(self, ctx): ptr = self.start_ptr + nextppos = self.nextppos + ppos3 = self.ppos3 while ptr <= self.maxptr: - result = sre_match(ctx, self.nextppos, ptr, self.start_marks) + ctx.jitdriver_MinRepeatOne.jit_merge_point( + self=self, ptr=ptr, ctx=ctx, nextppos=nextppos, ppos3=ppos3) + result = sre_match(ctx, nextppos, ptr, self.start_marks) if result is not None: self.subresult = result self.start_ptr = ptr return self - if not self.next_char_ok(ctx, ptr): + if not self.next_char_ok(ctx, ptr, ppos3): break ptr += 1 def find_next_result(self, ctx): ptr = self.start_ptr - if not self.next_char_ok(ctx, ptr): + if not self.next_char_ok(ctx, ptr, self.ppos3): return self.start_ptr = ptr + 1 return self.find_first_result(ctx) - def next_char_ok(self, ctx, ptr): + def next_char_ok(self, ctx, ptr, ppos): if ptr == ctx.end: return False - ppos = self.ppos3 op = ctx.pat(ppos) - for op1, (checkerfn, _) in unroll_char_checker: + for op1, checkerfn in unroll_char_checker: if op1 == op: return checkerfn(ctx, ptr, ppos) raise Error("next_char_ok[%d]" % op) @@ -325,41 +348,34 @@ self.next = next # chained list class MaxUntilMatchResult(AbstractUntilMatchResult): + install_jitdriver('MaxUntil', + greens=['ppos', 'tailppos', 'match_more', 'ctx.pattern'], + reds=['ptr', 'marks', 'self', 'ctx'], + debugprint=(3, 0, 2)) def find_first_result(self, ctx): - enum = sre_match(ctx, self.ppos + 3, self.cur_ptr, self.cur_marks) - return self.search_next(ctx, enum, resume=False) + return self.search_next(ctx, match_more=True) def find_next_result(self, ctx): - return self.search_next(ctx, None, resume=True) + return self.search_next(ctx, match_more=False) - def search_next(self, ctx, enum, resume): + def search_next(self, ctx, match_more): ppos = self.ppos - min = ctx.pat(ppos+1) - max = ctx.pat(ppos+2) + tailppos = self.tailppos ptr = self.cur_ptr marks = self.cur_marks while True: - while True: - if (enum is not None and - (ptr != ctx.match_end or self.num_pending < min)): - # ^^^^^^^^^^ zero-width match protection - # matched one more 'item'. record it and continue. - self.pending = Pending(ptr, marks, enum, self.pending) - self.num_pending += 1 - ptr = ctx.match_end - marks = ctx.match_marks - break - # 'item' no longer matches. - if not resume and self.num_pending >= min: - # try to match 'tail' if we have enough 'item' - result = sre_match(ctx, self.tailppos, ptr, marks) - if result is not None: - self.subresult = result - self.cur_ptr = ptr - self.cur_marks = marks - return self - resume = False + ctx.jitdriver_MaxUntil.jit_merge_point( + ppos=ppos, tailppos=tailppos, match_more=match_more, + ptr=ptr, marks=marks, self=self, ctx=ctx) + if match_more: + max = ctx.pat(ppos+2) + if max == 65535 or self.num_pending < max: + # try to match one more 'item' + enum = sre_match(ctx, ppos + 3, ptr, marks) + else: + enum = None # 'max' reached, no more matches + else: p = self.pending if p is None: return @@ -369,11 +385,27 @@ marks = p.marks enum = p.enum.move_to_next_result(ctx) # - if max == 65535 or self.num_pending < max: - # try to match one more 'item' - enum = sre_match(ctx, ppos + 3, ptr, marks) + min = ctx.pat(ppos+1) + if (enum is not None and + (ptr != ctx.match_end or self.num_pending < min)): + # ^^^^^^^^^^ zero-width match protection + # matched one more 'item'. record it and continue. + self.pending = Pending(ptr, marks, enum, self.pending) + self.num_pending += 1 + ptr = ctx.match_end + marks = ctx.match_marks + match_more = True else: - enum = None # 'max' reached, no more matches + # 'item' no longer matches. + if self.num_pending >= min: + # try to match 'tail' if we have enough 'item' + result = sre_match(ctx, tailppos, ptr, marks) + if result is not None: + self.subresult = result + self.cur_ptr = ptr + self.cur_marks = marks + return self + match_more = False class MinUntilMatchResult(AbstractUntilMatchResult): @@ -384,6 +416,7 @@ return self.search_next(ctx, resume=True) def search_next(self, ctx, resume): + # XXX missing jit support here ppos = self.ppos min = ctx.pat(ppos+1) max = ctx.pat(ppos+2) @@ -429,6 +462,7 @@ # ____________________________________________________________ @specializectx + at jit.unroll_safe def sre_match(ctx, ppos, ptr, marks): """Returns either None or a MatchResult object. Usually we only need the first result, but there is the case of REPEAT...UNTIL where we @@ -437,6 +471,13 @@ while True: op = ctx.pat(ppos) ppos += 1 + make_sure_not_modified(ctx.pattern) + + #jit.jit_debug("sre_match", op, ppos, ptr) + # + # When using the JIT, calls to sre_match() must always have a constant + # (green) argument for 'ppos'. If not, the following assert fails. + jit.assert_green(op) if op == OPCODE_FAILURE: return @@ -712,13 +753,23 @@ @specializectx def find_repetition_end(ctx, ppos, ptr, maxcount): end = ctx.end - # adjust end - if maxcount != 65535: + if maxcount <= 1: + if maxcount == 1 and ptr < end: + # Relatively common case: maxcount == 1. If we are not at the + # end of the string, it's done by a single direct check. + op = ctx.pat(ppos) + for op1, checkerfn in unroll_char_checker: + if op1 == op: + if checkerfn(ctx, ptr, ppos): + return ptr + 1 + return ptr + elif maxcount != 65535: + # adjust end end1 = ptr + maxcount if end1 <= end: end = end1 op = ctx.pat(ppos) - for op1, (_, fre) in unroll_char_checker: + for op1, fre in unroll_fre_checker: if op1 == op: return fre(ctx, ptr, end, ppos) raise Error("rsre.find_repetition_end[%d]" % op) @@ -751,23 +802,60 @@ if checkerfn == match_ANY_ALL: def fre(ctx, ptr, end, ppos): return end + elif checkerfn == match_IN: + install_jitdriver_spec('MatchIn', + greens=['ppos', 'ctx.pattern'], + reds=['ptr', 'end', 'ctx'], + debugprint=(1, 0)) + @specializectx + def fre(ctx, ptr, end, ppos): + while True: + ctx.jitdriver_MatchIn.jit_merge_point(ctx=ctx, ptr=ptr, + end=end, ppos=ppos) + if ptr < end and checkerfn(ctx, ptr, ppos): + ptr += 1 + else: + return ptr + elif checkerfn == match_IN_IGNORE: + install_jitdriver_spec('MatchInIgnore', + greens=['ppos', 'ctx.pattern'], + reds=['ptr', 'end', 'ctx'], + debugprint=(1, 0)) + @specializectx + def fre(ctx, ptr, end, ppos): + while True: + ctx.jitdriver_MatchInIgnore.jit_merge_point(ctx=ctx, ptr=ptr, + end=end, ppos=ppos) + if ptr < end and checkerfn(ctx, ptr, ppos): + ptr += 1 + else: + return ptr else: + # in the other cases, the fre() function is not JITted at all + # and is present as a residual call. + @specializectx def fre(ctx, ptr, end, ppos): while ptr < end and checkerfn(ctx, ptr, ppos): ptr += 1 return ptr - return checkerfn, fre + fre = func_with_new_name(fre, 'fre_' + checkerfn.__name__) + return fre + +unroll_char_checker = [ + (OPCODE_ANY, match_ANY), + (OPCODE_ANY_ALL, match_ANY_ALL), + (OPCODE_IN, match_IN), + (OPCODE_IN_IGNORE, match_IN_IGNORE), + (OPCODE_LITERAL, match_LITERAL), + (OPCODE_LITERAL_IGNORE, match_LITERAL_IGNORE), + (OPCODE_NOT_LITERAL, match_NOT_LITERAL), + (OPCODE_NOT_LITERAL_IGNORE, match_NOT_LITERAL_IGNORE), + ] +unroll_fre_checker = [(_op, _make_fre(_fn)) + for (_op, _fn) in unroll_char_checker] -unroll_char_checker = unrolling_iterable([ - (OPCODE_ANY, _make_fre(match_ANY)), - (OPCODE_ANY_ALL, _make_fre(match_ANY_ALL)), - (OPCODE_IN, _make_fre(match_IN)), - (OPCODE_IN_IGNORE, _make_fre(match_IN_IGNORE)), - (OPCODE_LITERAL, _make_fre(match_LITERAL)), - (OPCODE_LITERAL_IGNORE, _make_fre(match_LITERAL_IGNORE)), - (OPCODE_NOT_LITERAL, _make_fre(match_NOT_LITERAL)), - (OPCODE_NOT_LITERAL_IGNORE, _make_fre(match_NOT_LITERAL_IGNORE)), - ]) +unroll_char_checker = unrolling_iterable(unroll_char_checker) +unroll_fre_checker = unrolling_iterable(unroll_fre_checker) ##### At dispatch @@ -873,74 +961,139 @@ else: return None +install_jitdriver('Match', + greens=['ctx.pattern'], reds=['ctx'], + debugprint=(0,)) + def match_context(ctx): ctx.original_pos = ctx.match_start if ctx.end < ctx.match_start: return False + ctx.jitdriver_Match.jit_merge_point(ctx=ctx) return sre_match(ctx, 0, ctx.match_start, None) is not None def search_context(ctx): ctx.original_pos = ctx.match_start if ctx.end < ctx.match_start: return False - if ctx.pat(0) == OPCODE_INFO: - if ctx.pat(2) & rsre_char.SRE_INFO_PREFIX and ctx.pat(5) > 1: - return fast_search(ctx) - return regular_search(ctx) + base = 0 + charset = False + if ctx.pat(base) == OPCODE_INFO: + flags = ctx.pat(2) + if flags & rsre_char.SRE_INFO_PREFIX: + if ctx.pat(5) > 1: + return fast_search(ctx) + else: + charset = (flags & rsre_char.SRE_INFO_CHARSET) + base += 1 + ctx.pat(1) + if ctx.pat(base) == OPCODE_LITERAL: + return literal_search(ctx, base) + if charset: + return charset_search(ctx, base) + return regular_search(ctx, base) + +install_jitdriver('RegularSearch', + greens=['base', 'ctx.pattern'], + reds=['start', 'ctx'], + debugprint=(1, 0)) -def regular_search(ctx): +def regular_search(ctx, base): start = ctx.match_start while start <= ctx.end: - if sre_match(ctx, 0, start, None) is not None: + ctx.jitdriver_RegularSearch.jit_merge_point(ctx=ctx, start=start, + base=base) + if sre_match(ctx, base, start, None) is not None: ctx.match_start = start return True start += 1 return False +install_jitdriver_spec("LiteralSearch", + greens=['base', 'character', 'ctx.pattern'], + reds=['start', 'ctx'], + debugprint=(2, 0, 1)) + at specializectx +def literal_search(ctx, base): + # pattern starts with a literal character. this is used + # for short prefixes, and if fast search is disabled + character = ctx.pat(base + 1) + base += 2 + start = ctx.match_start + while start < ctx.end: + ctx.jitdriver_LiteralSearch.jit_merge_point(ctx=ctx, start=start, + base=base, character=character) + if ctx.str(start) == character: + if sre_match(ctx, base, start + 1, None) is not None: + ctx.match_start = start + return True + start += 1 + return False + +install_jitdriver_spec("CharsetSearch", + greens=['base', 'ctx.pattern'], + reds=['start', 'ctx'], + debugprint=(1, 0)) + at specializectx +def charset_search(ctx, base): + # pattern starts with a character from a known set + start = ctx.match_start + while start < ctx.end: + ctx.jitdriver_CharsetSearch.jit_merge_point(ctx=ctx, start=start, + base=base) + if rsre_char.check_charset(ctx.pattern, 5, ctx.str(start)): + if sre_match(ctx, base, start, None) is not None: + ctx.match_start = start + return True + start += 1 + return False + +install_jitdriver_spec('FastSearch', + greens=['i', 'prefix_len', 'ctx.pattern'], + reds=['string_position', 'ctx'], + debugprint=(2, 0)) @specializectx def fast_search(ctx): # skips forward in a string as fast as possible using information from # an optimization info block # <1=skip> <2=flags> <3=min> <4=...> # <5=length> <6=skip> <7=prefix data> - flags = ctx.pat(2) + string_position = ctx.match_start + if string_position >= ctx.end: + return False prefix_len = ctx.pat(5) assert prefix_len >= 0 - prefix_skip = ctx.pat(6) - assert prefix_skip >= 0 - overlap_offset = 7 + prefix_len - 1 - assert overlap_offset >= 0 - pattern_offset = ctx.pat(1) + 1 - ppos_start = pattern_offset + 2 * prefix_skip - assert ppos_start >= 0 i = 0 - string_position = ctx.match_start - end = ctx.end - while string_position < end: - while True: - char_ord = ctx.str(string_position) - if char_ord != ctx.pat(7 + i): - if i == 0: - break - else: - i = ctx.pat(overlap_offset + i) - else: - i += 1 - if i == prefix_len: - # found a potential match - start = string_position + 1 - prefix_len - assert start >= 0 - ptr = start + prefix_skip - if flags & rsre_char.SRE_INFO_LITERAL: - # matched all of pure literal pattern - ctx.match_start = start - ctx.match_end = ptr - ctx.match_marks = None - return True - if sre_match(ctx, ppos_start, ptr, None) is not None: - ctx.match_start = start - return True - i = ctx.pat(overlap_offset + i) - break + while True: + ctx.jitdriver_FastSearch.jit_merge_point(ctx=ctx, + string_position=string_position, i=i, prefix_len=prefix_len) + char_ord = ctx.str(string_position) + if char_ord != ctx.pat(7 + i): + if i > 0: + overlap_offset = prefix_len + (7 - 1) + i = ctx.pat(overlap_offset + i) + continue + else: + i += 1 + if i == prefix_len: + # found a potential match + start = string_position + 1 - prefix_len + assert start >= 0 + prefix_skip = ctx.pat(6) + ptr = start + prefix_skip + #flags = ctx.pat(2) + #if flags & rsre_char.SRE_INFO_LITERAL: + # # matched all of pure literal pattern + # ctx.match_start = start + # ctx.match_end = ptr + # ctx.match_marks = None + # return True + pattern_offset = ctx.pat(1) + 1 + ppos_start = pattern_offset + 2 * prefix_skip + if sre_match(ctx, ppos_start, ptr, None) is not None: + ctx.match_start = start + return True + overlap_offset = prefix_len + (7 - 1) + i = ctx.pat(overlap_offset + i) string_position += 1 - return False + if string_position >= ctx.end: + return False Modified: pypy/branch/arm-backend/pypy/rlib/rsre/test/test_match.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/rsre/test/test_match.py (original) +++ pypy/branch/arm-backend/pypy/rlib/rsre/test/test_match.py Tue Nov 23 10:12:47 2010 @@ -1,24 +1,49 @@ -import _sre, re, sre_compile -from pypy.rlib.rsre import rsre_core +import re +from pypy.rlib.rsre import rsre_core, rsre_char -def get_code(regexp, flags=0, allargs=False): - class GotIt(Exception): - pass - def my_compile(pattern, flags, code, *args): - print code - raise GotIt(code, flags, args) - saved = _sre.compile - try: - _sre.compile = my_compile - try: - sre_compile.compile(regexp, flags) - except GotIt, e: - pass +def get_hacked_sre_compile(my_compile): + """Return a copy of the sre_compile module for which the _sre + module is a custom module that has _sre.compile == my_compile + and CODESIZE == rsre_char.CODESIZE. + """ + import sre_compile, __builtin__, new + sre_hacked = new.module("_sre_hacked") + sre_hacked.compile = my_compile + sre_hacked.MAGIC = sre_compile.MAGIC + sre_hacked.CODESIZE = rsre_char.CODESIZE + sre_hacked.getlower = rsre_char.getlower + def my_import(name, *args): + if name == '_sre': + return sre_hacked else: - raise ValueError("did not reach _sre.compile()!") + return default_import(name, *args) + src = sre_compile.__file__ + if src.lower().endswith('.pyc') or src.lower().endswith('.pyo'): + src = src[:-1] + mod = new.module("sre_compile_hacked") + default_import = __import__ + try: + __builtin__.__import__ = my_import + execfile(src, mod.__dict__) finally: - _sre.compile = saved + __builtin__.__import__ = default_import + return mod + +class GotIt(Exception): + pass +def my_compile(pattern, flags, code, *args): + print code + raise GotIt(code, flags, args) +sre_compile_hacked = get_hacked_sre_compile(my_compile) + +def get_code(regexp, flags=0, allargs=False): + try: + sre_compile_hacked.compile(regexp, flags) + except GotIt, e: + pass + else: + raise ValueError("did not reach _sre.compile()!") if allargs: return e.args else: Modified: pypy/branch/arm-backend/pypy/rlib/rstring.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/rstring.py (original) +++ pypy/branch/arm-backend/pypy/rlib/rstring.py Tue Nov 23 10:12:47 2010 @@ -54,6 +54,7 @@ self.l = [] def append(self, s): + assert isinstance(s, self._type) self.l.append(s) def append_slice(self, s, start, end): @@ -63,11 +64,16 @@ def append_multiple_char(self, c, times): self.l.append(c * times) + def getlength(self): + return len(self.build()) + class StringBuilder(AbstractStringBuilder): + _type = str def build(self): return "".join(self.l) class UnicodeBuilder(AbstractStringBuilder): + _type = unicode def build(self): return u''.join(self.l) @@ -121,9 +127,12 @@ assert s_times.nonneg return s_None + def method_getlength(self): + return SomeInteger(nonneg=True) + def method_build(self): return SomeString() - + def rtyper_makerepr(self, rtyper): return rtyper.type_system.rbuilder.stringbuilder_repr @@ -146,6 +155,9 @@ assert s_times.nonneg return s_None + def method_getlength(self): + return SomeInteger(nonneg=True) + def method_build(self): return SomeUnicodeString() Modified: pypy/branch/arm-backend/pypy/rlib/streamio.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/streamio.py (original) +++ pypy/branch/arm-backend/pypy/rlib/streamio.py Tue Nov 23 10:12:47 2010 @@ -12,7 +12,7 @@ * some other methods also have no default parameters. * close() should be called exactly once and no further operations performed; there is no __del__() closing the stream for you. - * some methods may raise NotImplementedError. + * some methods may raise MyNotImplementedError. * peek() returns some (or no) characters that have already been read ahead. * flushable() returns True/False if flushing that stream is useful/pointless. @@ -54,6 +54,12 @@ ('a', True): O_RDWR | O_CREAT, } +class MyNotImplementedError(Exception): + """ + Catching NotImplementedError is not RPython, so we use this custom class + instead of it + """ + # ____________________________________________________________ @@ -209,16 +215,16 @@ some methods.""" def read(self, n): - raise NotImplementedError + raise MyNotImplementedError def write(self, data): - raise NotImplementedError + raise MyNotImplementedError def tell(self): - raise NotImplementedError + raise MyNotImplementedError def seek(self, offset, whence): - raise NotImplementedError + raise MyNotImplementedError def readall(self): bufsize = 8192 @@ -251,7 +257,7 @@ return ''.join(result) def truncate(self, size): - raise NotImplementedError + raise MyNotImplementedError def flush_buffers(self): pass @@ -487,7 +493,7 @@ if self.lines or self.buf: try: self.do_seek(self.tell(), 0) - except NotImplementedError: + except MyNotImplementedError: pass else: self.lines = [] @@ -535,14 +541,14 @@ self.buf = "" try: self.do_seek(offset, 1) - except NotImplementedError: + except MyNotImplementedError: intoffset = offset2int(offset) self.read(intoffset) return if whence == 2: try: self.do_seek(offset, 2) - except NotImplementedError: + except MyNotImplementedError: pass else: self.lines = [] @@ -1019,7 +1025,7 @@ if self.buf: try: self.base.seek(-len(self.buf), 1) - except NotImplementedError: + except MyNotImplementedError: pass else: self.buf = "" Modified: pypy/branch/arm-backend/pypy/rlib/test/test_jit.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/test/test_jit.py (original) +++ pypy/branch/arm-backend/pypy/rlib/test/test_jit.py Tue Nov 23 10:12:47 2010 @@ -1,10 +1,17 @@ import py +from pypy.conftest import option from pypy.rlib.jit import hint, we_are_jitted, JitDriver, purefunction_promote -from pypy.rlib.jit import JitHintError +from pypy.rlib.jit import JitHintError, oopspec from pypy.translator.translator import TranslationContext, graphof from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin from pypy.rpython.lltypesystem import lltype +def test_oopspec(): + @oopspec('foobar') + def fn(): + pass + assert fn.oopspec == 'foobar' + class BaseTestJIT(BaseRtypingTest): def test_hint(self): def f(): @@ -104,6 +111,26 @@ return n py.test.raises(JitHintError, self.gengraph, fn, [int]) + def test_green_field(self): + def get_printable_location(xfoo): + return str(ord(xfoo)) # xfoo must be annotated as a character + myjitdriver = JitDriver(greens=['x.foo'], reds=['n', 'x'], + get_printable_location=get_printable_location) + class A(object): + _immutable_fields_ = ['foo'] + def fn(n): + x = A() + x.foo = chr(n) + while n > 0: + myjitdriver.can_enter_jit(x=x, n=n) + myjitdriver.jit_merge_point(x=x, n=n) + n -= 1 + return n + t = self.gengraph(fn, [int])[0] + if option.view: + t.view() + # assert did not raise + class TestJITLLtype(BaseTestJIT, LLRtypeMixin): pass Modified: pypy/branch/arm-backend/pypy/rlib/test/test_rarithmetic.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/test/test_rarithmetic.py (original) +++ pypy/branch/arm-backend/pypy/rlib/test/test_rarithmetic.py Tue Nov 23 10:12:47 2010 @@ -325,6 +325,15 @@ assert float(x) != 2.1 assert abs(float(x) - 2.1) < 1E-6 +def test_r_singlefloat_eq(): + x = r_singlefloat(2.5) # exact number + y = r_singlefloat(2.5) + assert x == y + assert not x != y + assert not x == 2.5 + assert x != 2.5 + py.test.raises(TypeError, "x>y") + class BaseTestRarithmetic(BaseRtypingTest): def test_formatd(self): from pypy.rlib.rarithmetic import formatd @@ -358,7 +367,17 @@ assert res == 1.0 res = self.interpret(f, [1]) - assert res == 1e-100 + assert res == 1e-100 + + def test_compare_singlefloat_crashes(self): + from pypy.rlib.rarithmetic import r_singlefloat + from pypy.rpython.error import MissingRTypeOperation + def f(x): + a = r_singlefloat(x) + b = r_singlefloat(x+1) + return a == b + py.test.raises(MissingRTypeOperation, "self.interpret(f, [42.0])") + class TestLLtype(BaseTestRarithmetic, LLRtypeMixin): pass Modified: pypy/branch/arm-backend/pypy/rlib/test/test_rbigint.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/test/test_rbigint.py (original) +++ pypy/branch/arm-backend/pypy/rlib/test/test_rbigint.py Tue Nov 23 10:12:47 2010 @@ -112,6 +112,23 @@ assert rbigint.fromrarith_int(r_uint(2*sys.maxint+1)).eq( rbigint.fromlong(2*sys.maxint+1)) + def test_fromdecimalstr(self): + x = rbigint.fromdecimalstr("12345678901234567890523897987") + assert x.tolong() == 12345678901234567890523897987L + assert x.tobool() is True + x = rbigint.fromdecimalstr("+12345678901234567890523897987") + assert x.tolong() == 12345678901234567890523897987L + assert x.tobool() is True + x = rbigint.fromdecimalstr("-12345678901234567890523897987") + assert x.tolong() == -12345678901234567890523897987L + assert x.tobool() is True + x = rbigint.fromdecimalstr("+0") + assert x.tolong() == 0 + assert x.tobool() is False + x = rbigint.fromdecimalstr("-0") + assert x.tolong() == 0 + assert x.tobool() is False + def test_add(self): x = 123456789123456789000000L y = 123858582373821923936744221L @@ -448,6 +465,35 @@ assert (rbigint.fromlong(-9**50).ulonglongmask() == r_ulonglong(-9**50)) + def test_parse_digit_string(self): + from pypy.rlib.rbigint import parse_digit_string + class Parser: + def __init__(self, base, sign, digits): + self.base = base + self.sign = sign + self.next_digit = iter(digits + [-1]).next + x = parse_digit_string(Parser(10, 1, [6])) + assert x.eq(rbigint.fromint(6)) + x = parse_digit_string(Parser(10, 1, [6, 2, 3])) + assert x.eq(rbigint.fromint(623)) + x = parse_digit_string(Parser(10, -1, [6, 2, 3])) + assert x.eq(rbigint.fromint(-623)) + x = parse_digit_string(Parser(16, 1, [0xA, 0x4, 0xF])) + assert x.eq(rbigint.fromint(0xA4F)) + num = 0 + for i in range(36): + x = parse_digit_string(Parser(36, 1, range(i))) + assert x.eq(rbigint.fromlong(num)) + num = num * 36 + i + x = parse_digit_string(Parser(16, -1, range(15,-1,-1)*99)) + assert x.eq(rbigint.fromlong(long('-0x' + 'FEDCBA9876543210'*99, 16))) + assert x.tobool() is True + x = parse_digit_string(Parser(7, 1, [0, 0, 0])) + assert x.tobool() is False + x = parse_digit_string(Parser(7, -1, [0, 0, 0])) + assert x.tobool() is False + + BASE = 2 ** SHIFT class TestTranslatable(object): Modified: pypy/branch/arm-backend/pypy/rlib/test/test_rdynload.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/test/test_rdynload.py (original) +++ pypy/branch/arm-backend/pypy/rlib/test/test_rdynload.py Tue Nov 23 10:12:47 2010 @@ -1,15 +1,22 @@ from pypy.rlib.rdynload import * -from pypy.rlib.libffi import get_libc_name +from pypy.rlib.clibffi import get_libc_name from pypy.rpython.lltypesystem import rffi, lltype import py class TestDLOperations: def test_dlopen(self): - py.test.raises(DLOpenError, "dlopen(rffi.str2charp('xxxxxxxxxxxx'))") - assert dlopen(rffi.str2charp(get_libc_name())) + s = rffi.str2charp('xxxxxxxxxxxx') + py.test.raises(DLOpenError, "dlopen(s)") + rffi.free_charp(s) + # + s = rffi.str2charp(get_libc_name()) + assert dlopen(s) + rffi.free_charp(s) def test_dlsym(self): - lib = dlopen(rffi.str2charp(get_libc_name())) + s = rffi.str2charp(get_libc_name()) + lib = dlopen(s) + rffi.free_charp(s) handle = rffi.cast(lltype.Ptr(lltype.FuncType([lltype.Signed], lltype.Signed)), dlsym(lib, 'abs')) assert 1 == handle(1) Modified: pypy/branch/arm-backend/pypy/rlib/test/test_rsocket.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/test/test_rsocket.py (original) +++ pypy/branch/arm-backend/pypy/rlib/test/test_rsocket.py Tue Nov 23 10:12:47 2010 @@ -437,3 +437,31 @@ foo = self.serv.accept() py.test.raises(SocketError, raise_error) +def _test_cond_include(cond): + # Test that _rsocket_rffi is importable even on platforms where + # AF_PACKET or AF_NETLINK is not defined. + import re + from pypy.rlib import _rsocket_rffi + srcfile = _rsocket_rffi.__file__ + if srcfile.lower().endswith('c') or srcfile.lower().endswith('o'): + srcfile = srcfile[:-1] # .pyc => .py + assert srcfile.lower().endswith('.py') + sourcelines = open(srcfile, 'rb').read().splitlines() + found = False + for i, line in enumerate(sourcelines): + line2 = re.sub(r"(\s*COND_HEADER\s*=)", + r"\1'#undef %s\\n'+" % cond, + line) + if line2 != line: + found = True + sourcelines[i] = line2 + assert found + d = {} + sourcelines.append('') + exec '\n'.join(sourcelines) in d + +def test_no_AF_PACKET(): + _test_cond_include('AF_PACKET') + +def test_no_AF_NETLINK(): + _test_cond_include('AF_NETLINK') Modified: pypy/branch/arm-backend/pypy/rlib/test/test_rstring.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/test/test_rstring.py (original) +++ pypy/branch/arm-backend/pypy/rlib/test/test_rstring.py Tue Nov 23 10:12:47 2010 @@ -29,6 +29,7 @@ s = StringBuilder() s.append("a") s.append("abc") + assert s.getlength() == len('aabc') s.append("a") s.append_slice("abc", 1, 2) s.append_multiple_char('d', 4) @@ -39,6 +40,7 @@ s.append(u'a') s.append(u'abc') s.append_slice(u'abcdef', 1, 2) + assert s.getlength() == len('aabcb') s.append_multiple_char('d', 4) assert s.build() == 'aabcbdddd' assert isinstance(s.build(), unicode) Modified: pypy/branch/arm-backend/pypy/rlib/test/test_runicode.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/test/test_runicode.py (original) +++ pypy/branch/arm-backend/pypy/rlib/test/test_runicode.py Tue Nov 23 10:12:47 2010 @@ -76,7 +76,7 @@ assert start == startingpos assert stop == endingpos return u"42424242", stop - return "", endingpos + return u"", endingpos decoder = self.getdecoder(encoding) if addstuff: s += "some rest in ascii" Modified: pypy/branch/arm-backend/pypy/rlib/test/test_rzlib.py ============================================================================== --- pypy/branch/arm-backend/pypy/rlib/test/test_rzlib.py (original) +++ pypy/branch/arm-backend/pypy/rlib/test/test_rzlib.py Tue Nov 23 10:12:47 2010 @@ -189,6 +189,8 @@ assert unused3 == len('more_garbage') assert data3 == '' + rzlib.deflateEnd(stream) + def test_decompress_max_length(): """ @@ -205,6 +207,8 @@ assert finished2 is True assert unused2 == 0 + rzlib.deflateEnd(stream) + def test_cornercases(): """ @@ -215,6 +219,7 @@ bytes += rzlib.compress(stream, "") bytes += rzlib.compress(stream, "", rzlib.Z_FINISH) assert zlib.decompress(bytes) == "" + rzlib.deflateEnd(stream) stream = rzlib.inflateInit() data, finished, unused = rzlib.decompress(stream, "") @@ -228,3 +233,4 @@ assert finished is False assert unused > 0 buf = buf[-unused:] + rzlib.deflateEnd(stream) Modified: pypy/branch/arm-backend/pypy/rpython/annlowlevel.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/annlowlevel.py (original) +++ pypy/branch/arm-backend/pypy/rpython/annlowlevel.py Tue Nov 23 10:12:47 2010 @@ -136,7 +136,7 @@ return funcdesc.cachedgraph(TYPE, alt_name=valid_identifier(alt_name)) -class MixLevelHelperAnnotator: +class MixLevelHelperAnnotator(object): def __init__(self, rtyper): self.rtyper = rtyper Modified: pypy/branch/arm-backend/pypy/rpython/llinterp.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/llinterp.py (original) +++ pypy/branch/arm-backend/pypy/rpython/llinterp.py Tue Nov 23 10:12:47 2010 @@ -48,8 +48,7 @@ current_interpreter = None - def __init__(self, typer, tracing=True, exc_data_ptr=None, - malloc_check=True): + def __init__(self, typer, tracing=True, exc_data_ptr=None): self.bindings = {} self.typer = typer # 'heap' is module or object that provides malloc, etc for lltype ops @@ -57,9 +56,7 @@ self.exc_data_ptr = exc_data_ptr self.frame_stack = [] self.tracer = None - self.malloc_check = malloc_check self.frame_class = LLFrame - self.mallocs = {} if tracing: self.tracer = Tracer() @@ -163,24 +160,6 @@ return self.exc_data_ptr return None - def remember_malloc(self, ptr, llframe): - # err.... - self.mallocs[ptr._obj] = llframe - - def remember_free(self, ptr): - try: - del self.mallocs[ptr._obj] - except KeyError: - self._rehash_mallocs() - del self.mallocs[ptr._obj] - - def _rehash_mallocs(self): - # rehashing is needed because some objects' hash may change - # when being turned to - items = self.mallocs.items() - self.mallocs = {} - self.mallocs.update(items) - def _store_exception(self, exc): raise PleaseOverwriteStoreException("You just invoked ll2ctypes callback without overwriting _store_exception on llinterpreter") @@ -726,23 +705,23 @@ def op_malloc(self, obj, flags): flavor = flags['flavor'] zero = flags.get('zero', False) + track_allocation = flags.get('track_allocation', True) if flavor == "stack": result = self.heap.malloc(obj, zero=zero, flavor='raw') self.alloca_objects.append(result) return result - ptr = self.heap.malloc(obj, zero=zero, flavor=flavor) - if flavor == 'raw' and self.llinterpreter.malloc_check: - self.llinterpreter.remember_malloc(ptr, self) + ptr = self.heap.malloc(obj, zero=zero, flavor=flavor, + track_allocation=track_allocation) return ptr def op_malloc_varsize(self, obj, flags, size): flavor = flags['flavor'] zero = flags.get('zero', False) + track_allocation = flags.get('track_allocation', True) assert flavor in ('gc', 'raw') try: - ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor) - if flavor == 'raw' and self.llinterpreter.malloc_check: - self.llinterpreter.remember_malloc(ptr, self) + ptr = self.heap.malloc(obj, size, zero=zero, flavor=flavor, + track_allocation=track_allocation) return ptr except MemoryError: self.make_llexception() @@ -759,11 +738,10 @@ zero = flags.get('zero', False) return self.heap.malloc_nonmovable(TYPE, size, zero=zero) - def op_free(self, obj, flavor): - assert isinstance(flavor, str) - if flavor == 'raw' and self.llinterpreter.malloc_check: - self.llinterpreter.remember_free(obj) - self.heap.free(obj, flavor=flavor) + def op_free(self, obj, flags): + assert flags['flavor'] == 'raw' + track_allocation = flags.get('track_allocation', True) + self.heap.free(obj, flavor='raw', track_allocation=track_allocation) def op_shrink_array(self, obj, smallersize): return self.heap.shrink_array(obj, smallersize) @@ -934,6 +912,9 @@ def op_gc_dump_rpy_heap(self): raise NotImplementedError("gc_dump_rpy_heap") + def op_gc_typeids_z(self): + raise NotImplementedError("gc_typeids_z") + def op_do_malloc_fixedsize_clear(self): raise NotImplementedError("do_malloc_fixedsize_clear") @@ -1037,6 +1018,13 @@ def op_stack_malloc(self, size): # mmh raise NotImplementedError("backend only") + def op_track_alloc_start(self, addr): + # we don't do tracking at this level + checkadr(addr) + + def op_track_alloc_stop(self, addr): + checkadr(addr) + # ____________________________________________________________ # Overflow-detecting variants Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/ll2ctypes.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/lltypesystem/ll2ctypes.py (original) +++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/ll2ctypes.py Tue Nov 23 10:12:47 2010 @@ -69,7 +69,7 @@ PIECESIZE = 0x08000000 PIECES = 10 m = rmmap.mmap(-1, PIECES * PIECESIZE, - rmmap.MAP_PRIVATE|rmmap.MAP_ANONYMOUS, + rmmap.MAP_PRIVATE|rmmap.MAP_ANONYMOUS|rmmap.MAP_NORESERVE, rmmap.PROT_READ|rmmap.PROT_WRITE) m.close = lambda : None # leak instead of giving a spurious # error at CPython's shutdown @@ -823,6 +823,8 @@ except (ValueError, OverflowError): for tc in 'HIL': if array(tc).itemsize == array('u').itemsize: + import struct + cobj &= 256 ** struct.calcsize(tc) - 1 llobj = array('u', array(tc, (cobj,)).tostring())[0] break else: Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/llarena.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/lltypesystem/llarena.py (original) +++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/llarena.py Tue Nov 23 10:12:47 2010 @@ -441,6 +441,7 @@ rffi.INT, sandboxsafe=True, _nowrapper=True) _dev_zero = rffi.str2charp('/dev/zero') # prebuilt + lltype.render_immortal(_dev_zero) def clear_large_memory_chunk(baseaddr, size): # on some Unixy platforms, reading from /dev/zero is the fastest way Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/llmemory.py Tue Nov 23 10:12:47 2010 @@ -105,11 +105,13 @@ if (isinstance(self.TYPE, lltype.ContainerType) and self.TYPE._gckind == 'gc'): assert self.repeat == 1 - p = lltype.malloc(self.TYPE, flavor='raw', zero=zero) + p = lltype.malloc(self.TYPE, flavor='raw', zero=zero, + track_allocation=False) return cast_ptr_to_adr(p) else: T = lltype.FixedSizeArray(self.TYPE, self.repeat) - p = lltype.malloc(T, flavor='raw', zero=zero) + p = lltype.malloc(T, flavor='raw', zero=zero, + track_allocation=False) array_adr = cast_ptr_to_adr(p) return array_adr + ArrayItemsOffset(T) @@ -288,7 +290,8 @@ count = 0 p = lltype.malloc(parenttype or self.TYPE, count, immortal = self.TYPE._gckind == 'raw', - zero = zero) + zero = zero, + track_allocation = False) return cast_ptr_to_adr(p) def raw_memcopy(self, srcadr, dstadr): @@ -763,8 +766,10 @@ return '<%s>' % (self,) def __str__(self): return 'gctransformed_wref(%s)' % (self._ptr,) - def _normalizedcontainer(self): - return self._ptr._obj + def _normalizedcontainer(self, check=True): + return self._ptr._getobj(check=check)._normalizedcontainer(check=check) + def _was_freed(self): + return self._ptr._was_freed() # ____________________________________________________________ Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/lloperation.py Tue Nov 23 10:12:47 2010 @@ -403,6 +403,8 @@ 'raw_load': LLOp(sideeffects=False), 'raw_store': LLOp(), 'stack_malloc': LLOp(), # mmh + 'track_alloc_start': LLOp(), + 'track_alloc_stop': LLOp(), 'adr_add': LLOp(canfold=True), 'adr_sub': LLOp(canfold=True), 'adr_delta': LLOp(canfold=True), @@ -474,6 +476,7 @@ 'gc_get_rpy_type_index': LLOp(), 'gc_is_rpy_instance' : LLOp(), 'gc_dump_rpy_heap' : LLOp(), + 'gc_typeids_z' : LLOp(), # ------- JIT & GC interaction, only for some GCs ---------- Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/lltype.py Tue Nov 23 10:12:47 2010 @@ -1,7 +1,3 @@ -import StringIO -import traceback -import sys - import py from pypy.rlib.rarithmetic import (r_int, r_uint, intmask, r_singlefloat, r_ulonglong, r_longlong, base_int, @@ -10,25 +6,13 @@ from pypy.tool.uid import Hashable from pypy.tool.tls import tlsobject from pypy.tool.identity_dict import identity_dict +from pypy.tool import leakfinder from types import NoneType from sys import maxint import weakref TLS = tlsobject() -# Track allocations to detect memory leaks -# Don't track 'gc' and immortal mallocs -TRACK_ALLOCATIONS = False -ALLOCATED = identity_dict() - -def start_tracking_allocations(): - global TRACK_ALLOCATIONS - TRACK_ALLOCATIONS = True - ALLOCATED.clear() - -def stop_tracking_allocations(): - global TRACK_ALLOCATIONS - TRACK_ALLOCATIONS = False class _uninitialized(object): def __init__(self, TYPE): @@ -794,6 +778,8 @@ return llmemory.cast_adr_to_ptr(value, TGT) elif TGT == llmemory.Address and isinstance(ORIG, Ptr): return llmemory.cast_ptr_to_adr(value) + elif TGT == Signed and isinstance(ORIG, Ptr) and ORIG.TO._gckind == 'raw': + return llmemory.cast_adr_to_int(llmemory.cast_ptr_to_adr(value), 'symbolic') raise TypeError("don't know how to cast from %r to %r" % (ORIG, TGT)) @@ -1380,41 +1366,21 @@ __slots__ = ('_TYPE', '_parent_type', '_parent_index', '_keepparent', '_wrparent', - '__weakref__', '_traceback', - '__storage') + '__weakref__', + '_storage') - def __init__(self, TYPE, track_allocation=None): + def __init__(self, TYPE): self._wrparent = None self._TYPE = TYPE self._storage = True # means "use default storage", as opposed to: # None - container was freed # - using ctypes # (see ll2ctypes.py) - if track_allocation is not False and TRACK_ALLOCATIONS: - self._traceback = self._get_traceback() - ALLOCATED[self] = None - else: - self._traceback = None - - def _get_traceback(self): - frame = sys._getframe().f_back.f_back.f_back.f_back - sio = StringIO.StringIO() - traceback.print_stack(frame, file=sio) - return sio.getvalue() def _free(self): self._check() # no double-frees self._storage = None - def _storage_get(self): - return self.__storage - - def _storage_set(self, value): - self.__storage = value - if value is not True and self in ALLOCATED: - del ALLOCATED[self] - _storage = property(_storage_get, _storage_set) - def _was_freed(self): if self._storage is None: return True @@ -1493,12 +1459,12 @@ __slots__ = ('_hash_cache_', '_compilation_info') - def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None): + def __new__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None): my_variety = _struct_variety(TYPE._names) return object.__new__(my_variety) - def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None, track_allocation=None): - _parentable.__init__(self, TYPE, track_allocation) + def __init__(self, TYPE, n=None, initialization=None, parent=None, parentindex=None): + _parentable.__init__(self, TYPE) if n is not None and TYPE._arrayfld is None: raise TypeError("%r is not variable-sized" % (TYPE,)) if n is None and TYPE._arrayfld is not None: @@ -1506,8 +1472,7 @@ first, FIRSTTYPE = TYPE._first_struct() for fld, typ in TYPE._flds.items(): if fld == TYPE._arrayfld: - value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld, - track_allocation=track_allocation) + value = _array(typ, n, initialization=initialization, parent=self, parentindex=fld) else: value = typ._allocate(initialization=initialization, parent=self, parentindex=fld) setattr(self, fld, value) @@ -1568,12 +1533,12 @@ __slots__ = ('items',) - def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None, track_allocation=None): + def __init__(self, TYPE, n, initialization=None, parent=None, parentindex=None): if not isinstance(n, int): raise TypeError, "array length must be an int" if n < 0: raise ValueError, "negative array length" - _parentable.__init__(self, TYPE, track_allocation) + _parentable.__init__(self, TYPE) try: myrange = range(n) except OverflowError: @@ -1640,7 +1605,7 @@ _cache = weakref.WeakKeyDictionary() # parentarray -> {subarrays} def __init__(self, TYPE, parent, baseoffset_or_fieldname): - _parentable.__init__(self, TYPE, track_allocation=False) + _parentable.__init__(self, TYPE) self._setparentstructure(parent, baseoffset_or_fieldname) # Keep the parent array alive, we share the same allocation. # Don't do it if we are inside a GC object, though -- it's someone @@ -1648,6 +1613,13 @@ if typeOf(top_container(parent))._gckind == 'raw': self._keepparent = parent + def __str__(self): + parent = self._wrparent() + if parent is None: + return '_subarray at %s in already freed' % (self._parent_index,) + return '_subarray at %r in %s' % (self._parent_index, + parent._TYPE) + def __repr__(self): parent = self._wrparent() if parent is None: @@ -1861,8 +1833,9 @@ return id(self.value) -def malloc(T, n=None, flavor='gc', immortal=False, zero=False): - assert flavor != 'cpy' +def malloc(T, n=None, flavor='gc', immortal=False, zero=False, + track_allocation=True): + assert flavor in ('gc', 'raw') if zero or immortal: initialization = 'example' elif flavor == 'raw': @@ -1870,9 +1843,9 @@ else: initialization = 'malloc' if isinstance(T, Struct): - o = _struct(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal) + o = _struct(T, n, initialization=initialization) elif isinstance(T, Array): - o = _array(T, n, initialization=initialization, track_allocation=flavor == "raw" and not immortal) + o = _array(T, n, initialization=initialization) elif isinstance(T, OpaqueType): assert n is None o = _opaque(T, initialization=initialization) @@ -1880,17 +1853,57 @@ raise TypeError, "malloc for Structs and Arrays only" if T._gckind != 'gc' and not immortal and flavor.startswith('gc'): raise TypeError, "gc flavor malloc of a non-GC non-immortal structure" + if flavor == "raw" and not immortal and track_allocation: + leakfinder.remember_malloc(o, framedepth=2) solid = immortal or not flavor.startswith('gc') # immortal or non-gc case return _ptr(Ptr(T), o, solid) -def free(p, flavor): +def free(p, flavor, track_allocation=True): if flavor.startswith('gc'): raise TypeError, "gc flavor free" T = typeOf(p) if not isinstance(T, Ptr) or p._togckind() != 'raw': raise TypeError, "free(): only for pointers to non-gc containers" + if track_allocation: + leakfinder.remember_free(p._obj0) p._obj0._free() +def render_immortal(p, track_allocation=True): + T = typeOf(p) + if not isinstance(T, Ptr) or p._togckind() != 'raw': + raise TypeError, "free(): only for pointers to non-gc containers" + if track_allocation: + leakfinder.remember_free(p._obj0) + +def _make_scoped_allocator(T): + class ScopedAlloc: + def __init__(self, n=None, zero=False): + if n is None: + self.buf = malloc(T, flavor='raw', zero=zero) + else: + self.buf = malloc(T, n, flavor='raw', zero=zero) + + def __enter__(self): + return self.buf + + def __exit__(self, *args): + free(self.buf, flavor='raw') + + ScopedAlloc.__name__ = 'ScopedAlloc_%s' % (T,) + return ScopedAlloc +_make_scoped_allocator._annspecialcase_ = 'specialize:memo' + +def scoped_alloc(T, n=None, zero=False): + """Returns a context manager which handles allocation and + deallocation of temporary memory. Use it in a with statement:: + + with scoped_alloc(Array(Signed), 1) as array: + ...use array... + ...it's freed now. + """ + return _make_scoped_allocator(T)(n=n, zero=zero) +scoped_alloc._annspecialcase_ = 'specialize:arg(0)' + def functionptr(TYPE, name, **attrs): if not isinstance(TYPE, FuncType): raise TypeError, "functionptr() for FuncTypes only" Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/rbuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/lltypesystem/rbuilder.py (original) +++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/rbuilder.py Tue Nov 23 10:12:47 2010 @@ -100,6 +100,10 @@ ll_builder.used = used @staticmethod + def ll_getlength(ll_builder): + return ll_builder.used + + @staticmethod def ll_build(ll_builder): final_size = ll_builder.used assert final_size >= 0 Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/rclass.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/lltypesystem/rclass.py (original) +++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/rclass.py Tue Nov 23 10:12:47 2010 @@ -329,16 +329,33 @@ fields['__class__'] = 'typeptr', get_type_repr(self.rtyper) else: # instance attributes - if llfields is None: - llfields = [] attrs = self.classdef.attrs.items() attrs.sort() + myllfields = [] for name, attrdef in attrs: if not attrdef.readonly: r = self.rtyper.getrepr(attrdef.s_value) mangled_name = 'inst_' + name fields[name] = mangled_name, r - llfields.append((mangled_name, r.lowleveltype)) + myllfields.append((mangled_name, r.lowleveltype)) + + # Sort the instance attributes by decreasing "likely size", + # as reported by rffi.sizeof(), to minimize padding holes in C. + # Fields of the same size are sorted by name (by attrs.sort() + # above) just to minimize randomness. + def keysize((_, T)): + if T is lltype.Void: + return None + from pypy.rpython.lltypesystem.rffi import sizeof + try: + return -sizeof(T) + except StandardError: + return None + myllfields.sort(key = keysize) + if llfields is None: + llfields = myllfields + else: + llfields = llfields + myllfields self.rbase = getinstancerepr(self.rtyper, self.classdef.basedef, self.gcflavor) @@ -403,7 +420,7 @@ return cast_pointer(self.lowleveltype, result) def create_instance(self): - return malloc(self.object_type, flavor=self.gcflavor) + return malloc(self.object_type, flavor=self.gcflavor, immortal=True) def initialize_prebuilt_data(self, value, classdef, result): if self.classdef is not None: Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/rdict.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/lltypesystem/rdict.py (original) +++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/rdict.py Tue Nov 23 10:12:47 2010 @@ -306,6 +306,13 @@ hop.exception_cannot_occur() return hop.gendirectcall(ll_clear, v_dict) + def rtype_method_popitem(self, hop): + v_dict, = hop.inputargs(self) + r_tuple = hop.r_result + cTUPLE = hop.inputconst(lltype.Void, r_tuple.lowleveltype) + hop.exception_is_here() + return hop.gendirectcall(ll_popitem, cTUPLE, v_dict) + class __extend__(pairtype(DictRepr, rmodel.Repr)): def rtype_getitem((r_dict, r_key), hop): @@ -465,6 +472,10 @@ i = ll_dict_lookup(d, key, d.keyhash(key)) if not d.entries.valid(i): raise KeyError + _ll_dict_del(d, i) +ll_dict_delitem.oopspec = 'dict.delitem(d, key)' + +def _ll_dict_del(d, i): d.entries.mark_deleted(i) d.num_items -= 1 # clear the key and the value if they are GC pointers @@ -481,7 +492,6 @@ num_entries = len(d.entries) if num_entries > DICT_INITSIZE and d.num_items < num_entries / 4: ll_dict_resize(d) -ll_dict_delitem.oopspec = 'dict.delitem(d, key)' def ll_dict_resize(d): old_entries = d.entries @@ -810,3 +820,26 @@ i = ll_dict_lookup(d, key, d.keyhash(key)) return d.entries.valid(i) ll_contains.oopspec = 'dict.contains(d, key)' + +POPITEMINDEX = lltype.Struct('PopItemIndex', ('nextindex', lltype.Signed)) +global_popitem_index = lltype.malloc(POPITEMINDEX, zero=True, immortal=True) + +def ll_popitem(ELEM, dic): + entries = dic.entries + dmask = len(entries) - 1 + base = global_popitem_index.nextindex + counter = 0 + while counter <= dmask: + i = (base + counter) & dmask + counter += 1 + if entries.valid(i): + break + else: + raise KeyError + global_popitem_index.nextindex += counter + entry = entries[i] + r = lltype.malloc(ELEM.TO) + r.item0 = recast(ELEM.TO.item0, entry.key) + r.item1 = recast(ELEM.TO.item1, entry.value) + _ll_dict_del(dic, i) + return r Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/rffi.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/lltypesystem/rffi.py (original) +++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/rffi.py Tue Nov 23 10:12:47 2010 @@ -644,10 +644,18 @@ """ Either free a non-moving buffer or keep the original storage alive. """ - if rgc.can_move(data): + # We cannot rely on rgc.can_move(data) here, because its result + # might have changed since get_nonmovingbuffer(). Instead we check + # if 'buf' points inside 'data'. This is only possible if we + # followed the 2nd case in get_nonmovingbuffer(); in the first case, + # 'buf' points to its own raw-malloced memory. + data = llstrtype(data) + data_start = cast_ptr_to_adr(data) + \ + offsetof(STRTYPE, 'chars') + itemoffsetof(STRTYPE.chars, 0) + followed_2nd_path = (buf == cast(TYPEP, data_start)) + keepalive_until_here(data) + if not followed_2nd_path: lltype.free(buf, flavor='raw') - else: - keepalive_until_here(data) # int -> (char*, str) def alloc_buffer(count): @@ -916,3 +924,11 @@ """ return cast(lltype.Signed, getattr(pdst, fieldname)) getintfield._annspecialcase_ = 'specialize:ll_and_arg(1)' + +class scoped_str2charp: + def __init__(self, value): + self.buf = str2charp(value) + def __enter__(self): + return self.buf + def __exit__(self, *args): + free_charp(self.buf) Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_ll2ctypes.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_ll2ctypes.py (original) +++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_ll2ctypes.py Tue Nov 23 10:12:47 2010 @@ -765,6 +765,7 @@ assert abs(float(b[1]) - 1.1) < 1E-6 assert isinstance(b[2], rffi.r_singlefloat) assert abs(float(b[2]) - 2.2) < 1E-6 + lltype.free(a, flavor='raw') def test_different_signatures(self): if sys.platform=='win32': @@ -879,6 +880,7 @@ qsort(rffi.cast(rffi.VOIDP, a), 5, rffi.sizeof(rffi.INT), compare) for i in range(5): assert a[i] == i + 1 + lltype.free(a, flavor='raw') def test_array_type_bug(self): A = lltype.Array(lltype.Signed) Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_llmemory.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_llmemory.py (original) +++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_llmemory.py Tue Nov 23 10:12:47 2010 @@ -324,12 +324,14 @@ p_t = lltype.malloc(T) assert p_t.s == lltype.nullptr(S) # raw malloc does not - p_raw_t = lltype.malloc(T, flavor="raw") - py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s") + U = lltype.Struct("U", ('x', lltype.Signed)) + p_raw_t = lltype.malloc(U, flavor="raw") + py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x") + lltype.free(p_raw_t, flavor="raw") # this sort of raw_malloc too - p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(T)), lltype.Ptr(T)) - py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.s") - + p_raw_t = cast_adr_to_ptr(raw_malloc(sizeof(U)), lltype.Ptr(U)) + py.test.raises(lltype.UninitializedMemoryAccess, "p_raw_t.x") + def test_raw_malloc_signed_bunch(): adr = raw_malloc(sizeof(lltype.Signed) * 50) @@ -601,7 +603,8 @@ a = lltype.malloc(A, flavor='raw') src = cast_ptr_to_adr(a) + itemoffsetof(A, 0) raw_memclear(src, sizeof(lltype.Signed) * 0) - + lltype.free(a, flavor="raw") + def test_nonneg(): S1 = lltype.GcStruct('S1', ('x', lltype.Float)) A1 = lltype.GcArray(lltype.Float) Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_lltype.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_lltype.py (original) +++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_lltype.py Tue Nov 23 10:12:47 2010 @@ -1,7 +1,9 @@ +from __future__ import with_statement import py from pypy.rpython.lltypesystem.lltype import * from pypy.rpython.lltypesystem import lltype, rffi from pypy.tool.identity_dict import identity_dict +from pypy.tool import leakfinder def isweak(p, T): try: @@ -804,22 +806,20 @@ class TestTrackAllocation: - def setup_method(self, func): - start_tracking_allocations() - - def teardown_method(self, func): - assert not lltype.ALLOCATED, "Memory was not correctly freed" - stop_tracking_allocations() + def test_automatic_tracking(self): + # calls to start_tracking_allocations/stop_tracking_allocations + # should occur automatically from pypy/conftest.py. Check that. + assert leakfinder.TRACK_ALLOCATIONS def test_track_allocation(self): """A malloc'd buffer fills the ALLOCATED dictionary""" - assert lltype.TRACK_ALLOCATIONS - assert not lltype.ALLOCATED + assert leakfinder.TRACK_ALLOCATIONS + assert not leakfinder.ALLOCATED buf = malloc(Array(Signed), 1, flavor="raw") - assert len(lltype.ALLOCATED) == 1 - assert lltype.ALLOCATED.keys() == [buf._obj] + assert len(leakfinder.ALLOCATED) == 1 + assert leakfinder.ALLOCATED.keys() == [buf._obj] free(buf, flavor="raw") - assert not lltype.ALLOCATED + assert not leakfinder.ALLOCATED def test_str_from_buffer(self): """gc-managed memory does not need to be freed""" @@ -828,16 +828,28 @@ for i in range(size): raw_buf[i] = 'a' rstr = rffi.str_from_buffer(raw_buf, gc_buf, size, size) rffi.keep_buffer_alive_until_here(raw_buf, gc_buf) - assert not lltype.ALLOCATED + assert not leakfinder.ALLOCATED def test_leak_traceback(self): """Test info stored for allocated items""" buf = malloc(Array(Signed), 1, flavor="raw") - traceback = lltype.ALLOCATED.keys()[0]._traceback + traceback = leakfinder.ALLOCATED.values()[0] lines = traceback.splitlines() assert 'malloc(' in lines[-1] and 'flavor="raw")' in lines[-1] - # XXX The traceback should not be too long + # The traceback should not be too long print traceback free(buf, flavor="raw") + + def test_no_tracking(self): + p1 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False) + p2 = malloc(Array(Signed), 1, flavor='raw', track_allocation=False) + free(p2, flavor='raw', track_allocation=False) + # p1 is not freed + + def test_scoped_allocator(self): + with scoped_alloc(Array(Signed), 1) as array: + array[0] = -42 + x = array[0] + assert x == -42 Modified: pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_rffi.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_rffi.py (original) +++ pypy/branch/arm-backend/pypy/rpython/lltypesystem/test/test_rffi.py Tue Nov 23 10:12:47 2010 @@ -9,7 +9,7 @@ from pypy.rpython.lltypesystem.rstr import STR from pypy.rpython.lltypesystem import lltype from pypy.tool.udir import udir -from pypy.rpython.test.test_llinterp import interpret, MallocMismatch +from pypy.rpython.test.test_llinterp import interpret from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin from pypy.annotation.annrpython import RPythonAnnotator from pypy.rpython.rtyper import RPythonTyper @@ -787,3 +787,10 @@ mixann.getgraph(f2, [], s_None) mixann.finish() +def test_force_cast_unichar(): + x = cast(lltype.UniChar, -1) + assert isinstance(x, unicode) + if sys.maxunicode == 65535: + assert cast(LONG, x) == 65535 + else: + assert cast(LONG, cast(INT, x)) == -1 Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/base.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gc/base.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gc/base.py Tue Nov 23 10:12:47 2010 @@ -20,12 +20,15 @@ prebuilt_gc_objects_are_static_roots = True object_minimal_size = 0 - def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE): + def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, + translated_to_c=True): self.gcheaderbuilder = GCHeaderBuilder(self.HDR) self.AddressStack = get_address_stack(chunk_size) self.AddressDeque = get_address_deque(chunk_size) self.AddressDict = AddressDict self.config = config + assert isinstance(translated_to_c, bool) + self.translated_to_c = translated_to_c def setup(self): # all runtime mutable values' setup should happen here @@ -79,7 +82,7 @@ def set_root_walker(self, root_walker): self.root_walker = root_walker - def write_barrier(self, addr_struct): + def write_barrier(self, newvalue, addr_struct): pass def statistics(self, index): @@ -244,6 +247,21 @@ (not self.config.taggedpointers or llmemory.cast_adr_to_int(addr) & 1 == 0)) + def enumerate_all_roots(self, callback, arg): + """For each root object, invoke callback(obj, arg). + 'callback' should not be a bound method. + Note that this method is not suitable for actually doing the + collection in a moving GC, because you cannot write back a + modified address. It is there only for inspection. + """ + # overridden in some subclasses, for GCs which have an additional + # list of last generation roots + callback2, attrname = _convert_callback_formats(callback) # :-/ + setattr(self, attrname, arg) + self.root_walker.walk_roots(callback2, callback2, callback2) + self.run_finalizers.foreach(callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' + def debug_check_consistency(self): """To use after a collection. If self.DEBUG is set, this enumerates all roots and traces all objects to check if we didn't @@ -257,8 +275,7 @@ self._debug_pending = self.AddressStack() if not we_are_translated(): self.root_walker._walk_prebuilt_gc(self._debug_record) - callback = GCBase._debug_callback - self.root_walker.walk_roots(callback, callback, callback) + self.enumerate_all_roots(GCBase._debug_callback, self) pending = self._debug_pending while pending.non_empty(): obj = pending.pop() @@ -272,9 +289,8 @@ seen.add(obj) self.debug_check_object(obj) self._debug_pending.append(obj) - def _debug_callback(self, root): - obj = root.address[0] - ll_assert(bool(obj), "NULL address from walk_roots()") + @staticmethod + def _debug_callback(obj, self): self._debug_record(obj) def _debug_callback2(self, pointer, ignored): obj = pointer.address[0] @@ -429,3 +445,17 @@ if factor != 1: return 0.0 return value + +def _convert_callback_formats(callback): + callback = getattr(callback, 'im_func', callback) + if callback not in _converted_callback_formats: + def callback2(gc, root): + obj = root.address[0] + ll_assert(bool(obj), "NULL address from walk_roots()") + callback(obj, getattr(gc, attrname)) + attrname = '_callback2_arg%d' % len(_converted_callback_formats) + _converted_callback_formats[callback] = callback2, attrname + return _converted_callback_formats[callback] + +_convert_callback_formats._annspecialcase_ = 'specialize:memo' +_converted_callback_formats = {} Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/generation.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gc/generation.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gc/generation.py Tue Nov 23 10:12:47 2010 @@ -5,7 +5,6 @@ from pypy.rpython.memory.gc.base import read_from_env from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage from pypy.rpython.lltypesystem import lltype, llmemory, llarena -from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE from pypy.rlib.objectmodel import free_non_gc_object from pypy.rlib.debug import ll_assert from pypy.rlib.debug import debug_print, debug_start, debug_stop @@ -49,15 +48,17 @@ nursery_hash_base = -1 - def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, + def __init__(self, config, nursery_size=32*WORD, min_nursery_size=32*WORD, auto_nursery_size=False, space_size=1024*WORD, - max_space_size=sys.maxint//2+1): - SemiSpaceGC.__init__(self, config, chunk_size = chunk_size, + max_space_size=sys.maxint//2+1, + **kwds): + SemiSpaceGC.__init__(self, config, space_size = space_size, - max_space_size = max_space_size) + max_space_size = max_space_size, + **kwds) assert min_nursery_size <= nursery_size <= space_size // 2 self.initial_nursery_size = nursery_size self.auto_nursery_size = auto_nursery_size @@ -157,6 +158,14 @@ "odd-valued (i.e. tagged) pointer unexpected here") return self.nursery <= addr < self.nursery_top + def appears_to_be_in_nursery(self, addr): + # same as is_in_nursery(), but may return True accidentally if + # 'addr' is a tagged pointer with just the wrong value. + if not self.translated_to_c: + if not self.is_valid_gc_object(addr): + return False + return self.nursery <= addr < self.nursery_top + def malloc_fixedsize_clear(self, typeid, size, can_collect, has_finalizer=False, contains_weakptr=False): if (has_finalizer or not can_collect or @@ -326,7 +335,7 @@ addr = pointer.address[0] newaddr = self.copy(addr) pointer.address[0] = newaddr - self.write_into_last_generation_obj(obj) + self.write_into_last_generation_obj(obj, newaddr) # ____________________________________________________________ # Implementation of nursery-only collections @@ -457,9 +466,9 @@ # "if addr_struct.int0 & JIT_WB_IF_FLAG: remember_young_pointer()") JIT_WB_IF_FLAG = GCFLAG_NO_YOUNG_PTRS - def write_barrier(self, addr_struct): - if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS: - self.remember_young_pointer(addr_struct) + def write_barrier(self, newvalue, addr_struct): + if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS: + self.remember_young_pointer(addr_struct, newvalue) def _setup_wb(self): DEBUG = self.DEBUG @@ -470,23 +479,33 @@ # For x86, there is also an extra requirement: when the JIT calls # remember_young_pointer(), it assumes that it will not touch the SSE # registers, so it does not save and restore them (that's a *hack*!). - def remember_young_pointer(addr_struct): + def remember_young_pointer(addr_struct, addr): #llop.debug_print(lltype.Void, "\tremember_young_pointer", # addr_struct, "<-", addr) if DEBUG: ll_assert(not self.is_in_nursery(addr_struct), "nursery object with GCFLAG_NO_YOUNG_PTRS") - self.old_objects_pointing_to_young.append(addr_struct) - self.header(addr_struct).tid &= ~GCFLAG_NO_YOUNG_PTRS - self.write_into_last_generation_obj(addr_struct) + # + # What is important in this function is that it *must* + # clear the flag GCFLAG_NO_YOUNG_PTRS from 'addr_struct' + # if 'addr' is in the nursery. It is ok if, accidentally, + # it also clears the flag in some more rare cases, like + # 'addr' being a tagged pointer whose value happens to be + # a large integer that fools is_in_nursery(). + if self.appears_to_be_in_nursery(addr): + self.old_objects_pointing_to_young.append(addr_struct) + self.header(addr_struct).tid &= ~GCFLAG_NO_YOUNG_PTRS + self.write_into_last_generation_obj(addr_struct, addr) remember_young_pointer._dont_inline_ = True self.remember_young_pointer = remember_young_pointer - def write_into_last_generation_obj(self, addr_struct): + def write_into_last_generation_obj(self, addr_struct, addr): objhdr = self.header(addr_struct) if objhdr.tid & GCFLAG_NO_HEAP_PTRS: - objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS - self.last_generation_root_objects.append(addr_struct) + if (self.is_valid_gc_object(addr) and + not self.is_last_generation(addr)): + objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS + self.last_generation_root_objects.append(addr_struct) write_into_last_generation_obj._always_inline_ = True def assume_young_pointers(self, addr_struct): @@ -553,16 +572,10 @@ def _compute_current_nursery_hash(self, obj): return intmask(llmemory.cast_adr_to_int(obj) + self.nursery_hash_base) - def heap_stats_walk_roots(self): - self.last_generation_root_objects.foreach( - self._track_heap_ext, None) - self.root_walker.walk_roots( - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root) - - def _track_heap_ext(self, adr, ignored): - self.trace(adr, self.track_heap_parent, adr) + def enumerate_all_roots(self, callback, arg): + self.last_generation_root_objects.foreach(callback, arg) + SemiSpaceGC.enumerate_all_roots(self, callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' def debug_check_object(self, obj): """Check the invariants about 'obj' that should be true Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gc/inspector.py Tue Nov 23 10:12:47 2010 @@ -4,25 +4,22 @@ from pypy.rpython.lltypesystem import lltype, llmemory, rffi from pypy.rlib.objectmodel import free_non_gc_object from pypy.rpython.module.ll_os import underscore_on_windows -from pypy.rlib import rposix +from pypy.rlib import rposix, rgc from pypy.rpython.memory.support import AddressDict, get_address_stack # ---------- implementation of pypy.rlib.rgc.get_rpy_roots() ---------- -def _counting_rpy_root(gc, root): +def _counting_rpy_root(obj, gc): gc._count_rpy += 1 def _do_count_rpy_roots(gc): gc._count_rpy = 0 - gc.root_walker.walk_roots( - _counting_rpy_root, - _counting_rpy_root, - _counting_rpy_root) + gc.enumerate_all_roots(_counting_rpy_root, gc) return gc._count_rpy -def _append_rpy_root(gc, root): +def _append_rpy_root(obj, gc): # Can use the gc list, but should not allocate! # It is essential that the list is not resizable! lst = gc._list_rpy @@ -30,15 +27,12 @@ if index >= len(lst): raise ValueError gc._count_rpy = index + 1 - lst[index] = llmemory.cast_adr_to_ptr(root.address[0], llmemory.GCREF) + lst[index] = llmemory.cast_adr_to_ptr(obj, llmemory.GCREF) def _do_append_rpy_roots(gc, lst): gc._count_rpy = 0 gc._list_rpy = lst - gc.root_walker.walk_roots( - _append_rpy_root, - _append_rpy_root, - _append_rpy_root) + gc.enumerate_all_roots(_append_rpy_root, gc) gc._list_rpy = None def get_rpy_roots(gc): @@ -172,12 +166,7 @@ self.pending.append(obj) def add_roots(self): - self.gc._heap_dumper = self - self.gc.root_walker.walk_roots( - _hd_add_root, - _hd_add_root, - _hd_add_root) - self.gc._heap_dumper = None + self.gc.enumerate_all_roots(_hd_add_root, self) pendingroots = self.pending self.pending = AddressStack() self.walk(pendingroots) @@ -188,8 +177,8 @@ while pending.non_empty(): self.writeobj(pending.pop()) -def _hd_add_root(gc, root): - gc._heap_dumper.add(root.address[0]) +def _hd_add_root(obj, heap_dumper): + heap_dumper.add(obj) def dump_rpy_heap(gc, fd): heapdumper = HeapDumper(gc, fd) @@ -198,3 +187,7 @@ heapdumper.flush() heapdumper.delete() return True + +def get_typeids_z(gc): + srcaddress = gc.root_walker.gcdata.typeids_z + return llmemory.cast_adr_to_ptr(srcaddress, lltype.Ptr(rgc.ARRAY_OF_CHAR)) Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/markcompact.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gc/markcompact.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gc/markcompact.py Tue Nov 23 10:12:47 2010 @@ -2,7 +2,6 @@ from pypy.rpython.memory.gc.base import MovingGCBase, read_from_env from pypy.rlib.debug import ll_assert, have_debug_prints from pypy.rlib.debug import debug_print, debug_start, debug_stop -from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE from pypy.rpython.memory.support import get_address_stack, get_address_deque from pypy.rpython.memory.support import AddressDict from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage @@ -86,9 +85,9 @@ free = NULL next_collect_after = -1 - def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, space_size=4096, - min_next_collect_after=128): - MovingGCBase.__init__(self, config, chunk_size) + def __init__(self, config, space_size=4096, + min_next_collect_after=128, **kwds): + MovingGCBase.__init__(self, config, **kwds) self.space_size = space_size self.min_next_collect_after = min_next_collect_after Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/marksweep.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gc/marksweep.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gc/marksweep.py Tue Nov 23 10:12:47 2010 @@ -1,7 +1,6 @@ from pypy.rpython.lltypesystem.llmemory import raw_malloc, raw_free from pypy.rpython.lltypesystem.llmemory import raw_memcopy, raw_memclear from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage -from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE from pypy.rpython.memory.support import get_address_stack from pypy.rpython.memory.gcheader import GCHeaderBuilder from pypy.rpython.lltypesystem import lltype, llmemory, rffi, llgroup @@ -48,9 +47,9 @@ # translating to a real backend. TRANSLATION_PARAMS = {'start_heap_size': 8*1024*1024} # XXX adjust - def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, start_heap_size=4096): + def __init__(self, config, start_heap_size=4096, **kwds): self.param_start_heap_size = start_heap_size - GCBase.__init__(self, config, chunk_size) + GCBase.__init__(self, config, **kwds) def setup(self): GCBase.setup(self) @@ -714,8 +713,8 @@ _alloc_flavor_ = "raw" COLLECT_EVERY = 2000 - def __init__(self, chunk_size=DEFAULT_CHUNK_SIZE, start_heap_size=4096): - MarkSweepGC.__init__(self, chunk_size, start_heap_size) + def __init__(self, config, **kwds): + MarkSweepGC.__init__(self, config, **kwds) self.count_mallocs = 0 def maybe_collect(self): Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gc/minimark.py Tue Nov 23 10:12:47 2010 @@ -1,10 +1,42 @@ +""" MiniMark GC. + +Environment variables can be used to fine-tune the following parameters: + + PYPY_GC_NURSERY The nursery size. Defaults to half the size of + the L2 cache. Try values like '1.2MB'. + + PYPY_GC_MAJOR_COLLECT Major collection memory factor. Default is '1.82', + which means trigger a major collection when the + memory consumed equals 1.82 times the memory + really used at the end of the previous major + collection. + + PYPY_GC_GROWTH Major collection threshold's max growth rate. + Default is '1.3'. Useful to collect more often + than normally on sudden memory growth, e.g. when + there is a temporary peak in memory usage. + + PYPY_GC_MAX The max heap size. If coming near this limit, it + will first collect more often, then raise an + RPython MemoryError, and if that is not enough, + crash the program with a fatal error. Try values + like '1.6GB'. + + PYPY_GC_MIN Don't collect while the memory size is below this + limit. Useful to avoid spending all the time in + the GC in very small programs. Defaults to 8 + times the nursery. +""" +# XXX Should find a way to bound the major collection threshold by the +# XXX total addressable size. Maybe by keeping some minimarkpage arenas +# XXX pre-reserved, enough for a few nursery collections? What about +# XXX raw-malloced memory? import sys from pypy.rpython.lltypesystem import lltype, llmemory, llarena, llgroup from pypy.rpython.lltypesystem.lloperation import llop from pypy.rpython.lltypesystem.llmemory import raw_malloc_usage from pypy.rpython.memory.gc.base import GCBase, MovingGCBase from pypy.rpython.memory.gc import minimarkpage, base, generation -from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask, r_uint from pypy.rlib.rarithmetic import LONG_BIT_SHIFT from pypy.rlib.debug import ll_assert, debug_print, debug_start, debug_stop @@ -78,7 +110,7 @@ # During a minor collection, the objects in the nursery that are # moved outside are changed in-place: their header is replaced with - # the value -1, and the following word is set to the address of + # the value -42, and the following word is set to the address of # where the object was moved. This means that all objects in the # nursery need to be at least 2 words long, but objects outside the # nursery don't need to. @@ -88,13 +120,8 @@ TRANSLATION_PARAMS = { # Automatically adjust the size of the nursery and the - # 'major_collection_threshold' from the environment. For - # 'nursery_size' it will look it up in the env var - # PYPY_GC_NURSERY and fall back to half the size of - # the L2 cache. For 'major_collection_threshold' it will look - # it up in the env var PYPY_GC_MAJOR_COLLECT. It also sets - # 'max_heap_size' to PYPY_GC_MAX. Finally, PYPY_GC_MIN sets - # the minimal value of 'next_major_collection_threshold'. + # 'major_collection_threshold' from the environment. + # See docstring at the start of the file. "read_from_env": True, # The size of the nursery. Note that this is only used as a @@ -122,6 +149,13 @@ # we trigger the next major collection. "major_collection_threshold": 1.82, + # Threshold to avoid that the total heap size grows by a factor of + # major_collection_threshold at every collection: it can only + # grow at most by the following factor from one collection to the + # next. Used e.g. when there is a sudden, temporary peak in memory + # usage; this avoids that the upper bound grows too fast. + "growth_rate_max": 1.3, + # The number of array indices that are mapped to a single bit in # write_barrier_from_array(). Must be a power of two. The default # value of 128 means that card pages are 512 bytes (1024 on 64-bits) @@ -140,23 +174,26 @@ "large_object_gcptrs": 8250*WORD, } - def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, + def __init__(self, config, read_from_env=False, nursery_size=32*WORD, page_size=16*WORD, arena_size=64*WORD, small_request_threshold=5*WORD, major_collection_threshold=2.5, + growth_rate_max=2.5, # for tests card_page_indices=0, large_object=8*WORD, large_object_gcptrs=10*WORD, - ArenaCollectionClass=None): - MovingGCBase.__init__(self, config, chunk_size) + ArenaCollectionClass=None, + **kwds): + MovingGCBase.__init__(self, config, **kwds) assert small_request_threshold % WORD == 0 self.read_from_env = read_from_env self.nursery_size = nursery_size self.small_request_threshold = small_request_threshold self.major_collection_threshold = major_collection_threshold + self.growth_rate_max = growth_rate_max self.num_major_collects = 0 self.min_heap_size = 0.0 self.max_heap_size = 0.0 @@ -178,6 +215,7 @@ self.nursery = NULL self.nursery_free = NULL self.nursery_top = NULL + self.debug_always_do_minor_collect = False # # The ArenaCollection() handles the nonmovable objects allocation. if ArenaCollectionClass is None: @@ -245,6 +283,10 @@ # From there on, the GC is fully initialized and the code # below can use it newsize = base.read_from_env('PYPY_GC_NURSERY') + # PYPY_GC_NURSERY=1 forces a minor collect for every malloc. + # Useful to debug external factors, like trackgcroot or the + # handling of the write barrier. + self.debug_always_do_minor_collect = newsize == 1 if newsize <= 0: newsize = generation.estimate_best_nursery_size() if newsize <= 0: @@ -252,9 +294,13 @@ newsize = max(newsize, minsize) # major_coll = base.read_float_from_env('PYPY_GC_MAJOR_COLLECT') - if major_coll >= 1.0: + if major_coll > 1.0: self.major_collection_threshold = major_coll # + growth = base.read_float_from_env('PYPY_GC_GROWTH') + if growth > 1.0: + self.growth_rate_max = growth + # min_heap_size = base.read_uint_from_env('PYPY_GC_MIN') if min_heap_size > 0: self.min_heap_size = float(min_heap_size) @@ -290,11 +336,19 @@ # initialize the threshold self.min_heap_size = max(self.min_heap_size, self.nursery_size * self.major_collection_threshold) + self.next_major_collection_threshold = self.min_heap_size self.set_major_threshold_from(0.0) debug_stop("gc-set-nursery-size") - def set_major_threshold_from(self, threshold): + + def set_major_threshold_from(self, threshold, reserving_size=0): # Set the next_major_collection_threshold. + threshold_max = (self.next_major_collection_threshold * + self.growth_rate_max) + if threshold > threshold_max: + threshold = threshold_max + # + threshold += reserving_size if threshold < self.min_heap_size: threshold = self.min_heap_size # @@ -444,6 +498,10 @@ result = self.nursery_free self.nursery_free = result + totalsize ll_assert(self.nursery_free <= self.nursery_top, "nursery overflow") + # + if self.debug_always_do_minor_collect: + self.nursery_free = self.nursery_top + # return result collect_and_reserve._dont_inline_ = True @@ -636,13 +694,24 @@ "odd-valued (i.e. tagged) pointer unexpected here") return self.nursery <= addr < self.nursery_top + def appears_to_be_in_nursery(self, addr): + # same as is_in_nursery(), but may return True accidentally if + # 'addr' is a tagged pointer with just the wrong value. + if not self.translated_to_c: + if not self.is_valid_gc_object(addr): + return False + return self.nursery <= addr < self.nursery_top + def is_forwarded(self, obj): """Returns True if the nursery obj is marked as forwarded. Implemented a bit obscurely by checking an unrelated flag - that can never be set on a young object -- except if tid == -1. + that can never be set on a young object -- except if tid == -42. """ assert self.is_in_nursery(obj) - return self.header(obj).tid & GCFLAG_FINALIZATION_ORDERING + result = (self.header(obj).tid & GCFLAG_FINALIZATION_ORDERING != 0) + if result: + ll_assert(self.header(obj).tid == -42, "bogus header for young obj") + return result def get_forwarding_address(self, obj): return llmemory.cast_adr_to_ptr(obj, FORWARDSTUBPTR).forw @@ -726,16 +795,20 @@ def JIT_max_size_of_young_obj(cls): return cls.TRANSLATION_PARAMS['large_object'] - def write_barrier(self, addr_struct): + @classmethod + def JIT_minimal_size_in_nursery(cls): + return cls.minimal_size_in_nursery + + def write_barrier(self, newvalue, addr_struct): if self.header(addr_struct).tid & GCFLAG_NO_YOUNG_PTRS: - self.remember_young_pointer(addr_struct) + self.remember_young_pointer(addr_struct, newvalue) - def write_barrier_from_array(self, addr_array, index): + def write_barrier_from_array(self, newvalue, addr_array, index): if self.header(addr_array).tid & GCFLAG_NO_YOUNG_PTRS: if self.card_page_indices > 0: # <- constant-folded self.remember_young_pointer_from_array(addr_array, index) else: - self.remember_young_pointer(addr_array) + self.remember_young_pointer(addr_array, newvalue) def _init_writebarrier_logic(self): DEBUG = self.DEBUG @@ -746,27 +819,28 @@ # For x86, there is also an extra requirement: when the JIT calls # remember_young_pointer(), it assumes that it will not touch the SSE # registers, so it does not save and restore them (that's a *hack*!). - def remember_young_pointer(addr_struct): + def remember_young_pointer(addr_struct, newvalue): # 'addr_struct' is the address of the object in which we write. + # 'newvalue' is the address that we are going to write in there. if DEBUG: ll_assert(not self.is_in_nursery(addr_struct), "nursery object with GCFLAG_NO_YOUNG_PTRS") # - # We assume that what we are writing is a pointer to the nursery - # (and don't care for the fact that this new pointer may not - # actually point to the nursery, which seems ok). What we need is + # If it seems that what we are writing is a pointer to the nursery + # (as checked with appears_to_be_in_nursery()), then we need # to remove the flag GCFLAG_NO_YOUNG_PTRS and add the old object # to the list 'old_objects_pointing_to_young'. We know that # 'addr_struct' cannot be in the nursery, because nursery objects # never have the flag GCFLAG_NO_YOUNG_PTRS to start with. - self.old_objects_pointing_to_young.append(addr_struct) objhdr = self.header(addr_struct) - objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS + if self.appears_to_be_in_nursery(newvalue): + self.old_objects_pointing_to_young.append(addr_struct) + objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS # # Second part: if 'addr_struct' is actually a prebuilt GC # object and it's the first time we see a write to it, we # add it to the list 'prebuilt_root_objects'. Note that we - # do it even in the (rare?) case of 'addr' being another + # do it even in the (rare?) case of 'addr' being NULL or another # prebuilt object, to simplify code. if objhdr.tid & GCFLAG_NO_HEAP_PTRS: objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS @@ -780,16 +854,24 @@ def _init_writebarrier_with_card_marker(self): + DEBUG = self.DEBUG def remember_young_pointer_from_array(addr_array, index): # 'addr_array' is the address of the object in which we write, # which must have an array part; 'index' is the index of the # item that is (or contains) the pointer that we write. + if DEBUG: + ll_assert(not self.is_in_nursery(addr_array), + "nursery array with GCFLAG_NO_YOUNG_PTRS") objhdr = self.header(addr_array) if objhdr.tid & GCFLAG_HAS_CARDS == 0: # - # no cards, use default logic. The 'nocard_logic()' is just - # 'remember_young_pointer()', but forced to be inlined here. - nocard_logic(addr_array) + # no cards, use default logic. Mostly copied from above. + self.old_objects_pointing_to_young.append(addr_array) + objhdr = self.header(addr_array) + objhdr.tid &= ~GCFLAG_NO_YOUNG_PTRS + if objhdr.tid & GCFLAG_NO_HEAP_PTRS: + objhdr.tid &= ~GCFLAG_NO_HEAP_PTRS + self.prebuilt_root_objects.append(addr_array) return # # 'addr_array' is a raw_malloc'ed array with card markers @@ -807,17 +889,15 @@ return # # We set the flag (even if the newly written address does not - # actually point to the nursery -- like remember_young_pointer()). + # actually point to the nursery, which seems to be ok -- actually + # it seems more important that remember_young_pointer_from_array() + # does not take 3 arguments). addr_byte.char[0] = chr(byte | bitmask) # if objhdr.tid & GCFLAG_CARDS_SET == 0: self.old_objects_with_cards_set.append(addr_array) objhdr.tid |= GCFLAG_CARDS_SET - nocard_logic = func_with_new_name(self.remember_young_pointer, - 'remember_young_pointer_nocard') - del nocard_logic._dont_inline_ - nocard_logic._always_inline_ = True remember_young_pointer_from_array._dont_inline_ = True self.remember_young_pointer_from_array = ( remember_young_pointer_from_array) @@ -891,7 +971,7 @@ # # Now all live nursery objects should be out. Update the # young weakrefs' targets. - if self.young_objects_with_weakrefs.length() > 0: + if self.young_objects_with_weakrefs.non_empty(): self.invalidate_young_weakrefs() # # Clear this mapping. @@ -985,7 +1065,7 @@ obj = oldlist.pop() # # Add the flag GCFLAG_NO_YOUNG_PTRS. All live objects should have - # this flag after a nursery collection. + # this flag set after a nursery collection. self.header(obj).tid |= GCFLAG_NO_YOUNG_PTRS # # Trace the 'obj' to replace pointers to nursery with pointers @@ -1048,7 +1128,7 @@ # nursery are kept unchanged in this step. llmemory.raw_memcopy(obj - size_gc_header, newhdr, totalsize) # - # Set the old object's tid to -1 (containing all flags) and + # Set the old object's tid to -42 (containing all flags) and # replace the old object's content with the target address. # A bit of no-ops to convince llarena that we are changing # the layout, in non-translated versions. @@ -1056,7 +1136,7 @@ llarena.arena_reset(obj - size_gc_header, totalsize, 0) llarena.arena_reserve(obj - size_gc_header, size_gc_header + llmemory.sizeof(FORWARDSTUB)) - self.header(obj).tid = -1 + self.header(obj).tid = -42 newobj = newhdr + size_gc_header llmemory.cast_adr_to_ptr(obj, FORWARDSTUBPTR).forw = newobj # @@ -1167,8 +1247,8 @@ # have allocated 'major_collection_threshold' times more than # we currently have. bounded = self.set_major_threshold_from( - (self.get_total_memory_used() * self.major_collection_threshold) - + reserving_size) + self.get_total_memory_used() * self.major_collection_threshold, + reserving_size) # # Max heap size: gives an upper bound on the threshold. If we # already have at least this much allocated, raise MemoryError. @@ -1257,6 +1337,11 @@ self.run_finalizers.foreach(self._collect_obj, self.objects_to_trace) + def enumerate_all_roots(self, callback, arg): + self.prebuilt_root_objects.foreach(callback, arg) + MovingGCBase.enumerate_all_roots(self, callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' + @staticmethod def _collect_obj(obj, objects_to_trace): objects_to_trace.append(obj) @@ -1308,7 +1393,7 @@ if self.is_valid_gc_object(obj): if self.is_in_nursery(obj): # - # The object not a tagged pointer, and is it still in the + # The object is not a tagged pointer, and it is still in the # nursery. Find or allocate a "shadow" object, which is # where the object will be moved by the next minor # collection Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/minimarkpage.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gc/minimarkpage.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gc/minimarkpage.py Tue Nov 23 10:12:47 2010 @@ -100,11 +100,14 @@ # allocation of the given size. length = small_request_threshold / WORD + 1 self.page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length, - flavor='raw', zero=True) + flavor='raw', zero=True, + immortal=True) self.full_page_for_size = lltype.malloc(rffi.CArray(PAGE_PTR), length, - flavor='raw', zero=True) + flavor='raw', zero=True, + immortal=True) self.nblocks_for_size = lltype.malloc(rffi.CArray(lltype.Signed), - length, flavor='raw') + length, flavor='raw', + immortal=True) self.hdrsize = llmemory.raw_malloc_usage(llmemory.sizeof(PAGE_HEADER)) assert page_size > self.hdrsize self.nblocks_for_size[0] = 0 # unused @@ -114,11 +117,13 @@ self.max_pages_per_arena = arena_size // page_size self.arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR), self.max_pages_per_arena, - flavor='raw', zero=True) + flavor='raw', zero=True, + immortal=True) # this is used in mass_free() only self.old_arenas_lists = lltype.malloc(rffi.CArray(ARENA_PTR), self.max_pages_per_arena, - flavor='raw', zero=True) + flavor='raw', zero=True, + immortal=True) # # the arena currently consumed; it must have at least one page # available, or be NULL. The arena object that we point to is @@ -281,7 +286,7 @@ npages = (arena_end - firstpage) // self.page_size # # Allocate an ARENA object and initialize it - arena = lltype.malloc(ARENA, flavor='raw') + arena = lltype.malloc(ARENA, flavor='raw', track_allocation=False) arena.base = arena_base arena.nfreepages = 0 # they are all uninitialized pages arena.totalpages = npages @@ -332,7 +337,7 @@ # # The whole arena is empty. Free it. llarena.arena_free(arena.base) - lltype.free(arena, flavor='raw') + lltype.free(arena, flavor='raw', track_allocation=False) # else: # Insert 'arena' in the correct arenas_lists[n] Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gc/semispace.py Tue Nov 23 10:12:47 2010 @@ -1,7 +1,6 @@ from pypy.rpython.lltypesystem.llmemory import raw_malloc, raw_free from pypy.rpython.lltypesystem.llmemory import raw_memcopy, raw_memclear from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage -from pypy.rpython.memory.support import DEFAULT_CHUNK_SIZE from pypy.rpython.memory.support import get_address_stack, get_address_deque from pypy.rpython.memory.support import AddressDict from pypy.rpython.lltypesystem import lltype, llmemory, llarena, rffi, llgroup @@ -59,11 +58,11 @@ # translating to a real backend. TRANSLATION_PARAMS = {'space_size': 8*1024*1024} # XXX adjust - def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, space_size=4096, - max_space_size=sys.maxint//2+1): + def __init__(self, config, space_size=4096, max_space_size=sys.maxint//2+1, + **kwds): self.param_space_size = space_size self.param_max_space_size = max_space_size - MovingGCBase.__init__(self, config, chunk_size) + MovingGCBase.__init__(self, config, **kwds) def setup(self): #self.total_collection_time = 0.0 @@ -231,6 +230,10 @@ while self.max_space_size > size: self.max_space_size >>= 1 + @classmethod + def JIT_minimal_size_in_nursery(cls): + return cls.object_minimal_size + def collect(self, gen=0): self.debug_check_consistency() self.semispace_collect() @@ -690,15 +693,10 @@ self._ll_typeid_map[idx].size += llmemory.raw_malloc_usage(totsize) self.trace(adr, self.track_heap_parent, adr) - def _track_heap_root(self, root): - self.track_heap(root.address[0]) + @staticmethod + def _track_heap_root(obj, self): + self.track_heap(obj) - def heap_stats_walk_roots(self): - self.root_walker.walk_roots( - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root) - def heap_stats(self): self._tracked_dict = self.AddressDict() max_tid = self.root_walker.gcdata.max_type_id @@ -711,7 +709,7 @@ while i < max_tid: self._tracked_dict.add(llmemory.cast_ptr_to_adr(ll_typeid_map[i])) i += 1 - self.heap_stats_walk_roots() + self.enumerate_all_roots(SemiSpaceGC._track_heap_root, self) self._ll_typeid_map = lltype.nullptr(ARRAY_TYPEID_MAP) self._tracked_dict.delete() return ll_typeid_map Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_direct.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_direct.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_direct.py Tue Nov 23 10:12:47 2010 @@ -70,6 +70,7 @@ GC_PARAMS = self.GC_PARAMS.copy() if hasattr(meth, 'GC_PARAMS'): GC_PARAMS.update(meth.GC_PARAMS) + GC_PARAMS['translated_to_c'] = False self.gc = self.GCClass(config, **GC_PARAMS) self.gc.DEBUG = True self.rootwalker = DirectRootWalker(self) @@ -86,17 +87,19 @@ def write(self, p, fieldname, newvalue): if self.gc.needs_write_barrier: + newaddr = llmemory.cast_ptr_to_adr(newvalue) addr_struct = llmemory.cast_ptr_to_adr(p) - self.gc.write_barrier(addr_struct) + self.gc.write_barrier(newaddr, addr_struct) setattr(p, fieldname, newvalue) def writearray(self, p, index, newvalue): if self.gc.needs_write_barrier: + newaddr = llmemory.cast_ptr_to_adr(newvalue) addr_struct = llmemory.cast_ptr_to_adr(p) if hasattr(self.gc, 'write_barrier_from_array'): - self.gc.write_barrier_from_array(addr_struct, index) + self.gc.write_barrier_from_array(newaddr, addr_struct, index) else: - self.gc.write_barrier(addr_struct) + self.gc.write_barrier(newaddr, addr_struct) p[index] = newvalue def malloc(self, TYPE, n=None): Modified: pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_minimark.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_minimark.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gc/test/test_minimark.py Tue Nov 23 10:12:47 2010 @@ -28,3 +28,42 @@ assert gc.card_marking_bytes_for_length(P+P+1) == 3 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P) == 8 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P+1) == 9 + +def test_set_major_threshold(): + gc = MiniMarkGC(None, major_collection_threshold=2.0, + growth_rate_max=1.5) + gc.min_heap_size = 100.0 + gc.max_heap_size = 300.0 + gc.next_major_collection_threshold = 0.0 + # first, we don't grow past min_heap_size + for i in range(5): + gc.set_major_threshold_from(100.0) + assert gc.next_major_collection_threshold == 100.0 + # then we grow a lot + b = gc.set_major_threshold_from(100 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 150.0 + b = gc.set_major_threshold_from(150 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 225.0 + b = gc.set_major_threshold_from(225 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 # max reached + b = gc.set_major_threshold_from(300 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 + # then we shrink instantly + b = gc.set_major_threshold_from(100.0) + assert b is False + assert gc.next_major_collection_threshold == 100.0 + # then we grow a bit + b = gc.set_major_threshold_from(100 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 125.0 + b = gc.set_major_threshold_from(125 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 156.25 + # check that we cannot shrink below min_heap_size + b = gc.set_major_threshold_from(42.7) + assert b is False + assert gc.next_major_collection_threshold == 100.0 Modified: pypy/branch/arm-backend/pypy/rpython/memory/gctransform/asmgcroot.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gctransform/asmgcroot.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gctransform/asmgcroot.py Tue Nov 23 10:12:47 2010 @@ -75,7 +75,8 @@ key = (TYPE, num) if key not in sradict: CONTAINER = lltype.FixedSizeArray(TYPE, 1) - p = lltype.malloc(CONTAINER, flavor='raw', zero=True) + p = lltype.malloc(CONTAINER, flavor='raw', zero=True, + immortal=True) sradict[key] = Constant(p, lltype.Ptr(CONTAINER)) sra.append(sradict[key]) # Modified: pypy/branch/arm-backend/pypy/rpython/memory/gctransform/framework.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gctransform/framework.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gctransform/framework.py Tue Nov 23 10:12:47 2010 @@ -172,6 +172,7 @@ gcdata.static_root_nongcend = a_random_address # patched in finish() gcdata.static_root_end = a_random_address # patched in finish() gcdata.max_type_id = 13 # patched in finish() + gcdata.typeids_z = a_random_address # patched in finish() self.gcdata = gcdata self.malloc_fnptr_cache = {} @@ -212,6 +213,9 @@ data_classdef.generalize_attr( 'max_type_id', annmodel.SomeInteger()) + data_classdef.generalize_attr( + 'typeids_z', + annmodel.SomeAddress()) annhelper = annlowlevel.MixLevelHelperAnnotator(self.translator.rtyper) @@ -415,6 +419,11 @@ [s_gc, annmodel.SomeInteger()], annmodel.s_Bool, minimal_transform=False) + self.get_typeids_z_ptr = getfn(inspector.get_typeids_z, + [s_gc], + annmodel.SomePtr( + lltype.Ptr(rgc.ARRAY_OF_CHAR)), + minimal_transform=False) self.set_max_heap_size_ptr = getfn(GCClass.set_max_heap_size.im_func, [s_gc, @@ -426,6 +435,7 @@ if GCClass.needs_write_barrier: self.write_barrier_ptr = getfn(GCClass.write_barrier.im_func, [s_gc, + annmodel.SomeAddress(), annmodel.SomeAddress()], annmodel.s_None, inline=True) @@ -434,13 +444,15 @@ # func should not be a bound method, but a real function assert isinstance(func, types.FunctionType) self.write_barrier_failing_case_ptr = getfn(func, - [annmodel.SomeAddress()], + [annmodel.SomeAddress(), + annmodel.SomeAddress()], annmodel.s_None) func = getattr(GCClass, 'write_barrier_from_array', None) if func is not None: self.write_barrier_from_array_ptr = getfn(func.im_func, [s_gc, annmodel.SomeAddress(), + annmodel.SomeAddress(), annmodel.SomeInteger()], annmodel.s_None, inline=True) @@ -569,7 +581,14 @@ newgcdependencies = [] newgcdependencies.append(ll_static_roots_inside) ll_instance.inst_max_type_id = len(group.members) - self.write_typeid_list() + typeids_z = self.write_typeid_list() + ll_typeids_z = lltype.malloc(rgc.ARRAY_OF_CHAR, + len(typeids_z), + immortal=True) + for i in range(len(typeids_z)): + ll_typeids_z[i] = typeids_z[i] + ll_instance.inst_typeids_z = llmemory.cast_ptr_to_adr(ll_typeids_z) + newgcdependencies.append(ll_typeids_z) return newgcdependencies def get_finish_tables(self): @@ -596,6 +615,11 @@ for index in range(len(self.layoutbuilder.type_info_group.members)): f.write("member%-4d %s\n" % (index, all_ids.get(index, '?'))) f.close() + try: + import zlib + return zlib.compress(udir.join("typeids.txt").read(), 9) + except ImportError: + return '' def transform_graph(self, graph): func = getattr(graph, 'func', None) @@ -985,6 +1009,13 @@ resultvar=hop.spaceop.result) self.pop_roots(hop, livevars) + def gct_gc_typeids_z(self, hop): + livevars = self.push_roots(hop) + hop.genop("direct_call", + [self.get_typeids_z_ptr, self.c_const_gc], + resultvar=hop.spaceop.result) + self.pop_roots(hop, livevars) + def gct_malloc_nonmovable_varsize(self, hop): TYPE = hop.spaceop.result.concretetype if self.gcdata.gc.can_malloc_nonmovable(): @@ -1021,6 +1052,8 @@ and not isinstance(v_newvalue, Constant) and v_struct.concretetype.TO._gckind == "gc" and hop.spaceop not in self.clean_sets): + v_newvalue = hop.genop("cast_ptr_to_adr", [v_newvalue], + resulttype = llmemory.Address) v_structaddr = hop.genop("cast_ptr_to_adr", [v_struct], resulttype = llmemory.Address) if (self.write_barrier_from_array_ptr is not None and @@ -1030,12 +1063,14 @@ assert v_index.concretetype == lltype.Signed hop.genop("direct_call", [self.write_barrier_from_array_ptr, self.c_const_gc, + v_newvalue, v_structaddr, v_index]) else: self.write_barrier_calls += 1 hop.genop("direct_call", [self.write_barrier_ptr, self.c_const_gc, + v_newvalue, v_structaddr]) hop.rename('bare_' + opname) @@ -1203,7 +1238,7 @@ sizeofaddr = llmemory.sizeof(llmemory.Address) -class BaseRootWalker: +class BaseRootWalker(object): need_root_stack = False def __init__(self, gctransformer): Modified: pypy/branch/arm-backend/pypy/rpython/memory/gctransform/transform.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gctransform/transform.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gctransform/transform.py Tue Nov 23 10:12:47 2010 @@ -430,7 +430,8 @@ return self.parenttransformer.gct_malloc_varsize(hop) def gct_free(self, hop): - flavor = hop.spaceop.args[1].value + flags = hop.spaceop.args[1].value + flavor = flags['flavor'] assert flavor == 'raw' return self.parenttransformer.gct_free(hop) @@ -532,6 +533,8 @@ resulttype=llmemory.Address) if flags.get('zero'): hop.genop("raw_memclear", [v_raw, c_size]) + if flags.get('track_allocation', True): + hop.genop("track_alloc_start", [v_raw]) return v_raw def gct_fv_stack_malloc(self, hop, flags, TYPE, c_size): @@ -602,15 +605,20 @@ [self.raw_malloc_varsize_ptr, v_length, c_const_size, c_item_size, c_offset_to_length], resulttype=llmemory.Address) + if flags.get('track_allocation', True): + hop.genop("track_alloc_start", [v_raw]) return v_raw def gct_free(self, hop): op = hop.spaceop - flavor = op.args[1].value + flags = op.args[1].value + flavor = flags['flavor'] v = op.args[0] assert flavor != 'cpy', "cannot free CPython objects directly" if flavor == 'raw': v = hop.genop("cast_ptr_to_adr", [v], resulttype=llmemory.Address) + if flags.get('track_allocation', True): + hop.genop("track_alloc_stop", [v]) hop.genop('raw_free', [v]) else: assert False, "%s has no support for free with flavor %r" % (self, flavor) Modified: pypy/branch/arm-backend/pypy/rpython/memory/gcwrapper.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/gcwrapper.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/gcwrapper.py Tue Nov 23 10:12:47 2010 @@ -9,7 +9,10 @@ def __init__(self, llinterp, flowgraphs, gc_class, GC_PARAMS={}): translator = llinterp.typer.annotator.translator config = translator.config.translation - self.gc = gc_class(config, chunk_size = 10, **GC_PARAMS) + self.gc = gc_class(config, + chunk_size = 10, + translated_to_c = False, + **GC_PARAMS) self.gc.set_root_walker(LLInterpRootWalker(self)) self.gc.DEBUG = True self.llinterp = llinterp @@ -39,7 +42,8 @@ # # Interface for the llinterp # - def malloc(self, TYPE, n=None, flavor='gc', zero=False): + def malloc(self, TYPE, n=None, flavor='gc', zero=False, + track_allocation=True): if flavor == 'gc': typeid = self.get_type_id(TYPE) addr = self.gc.malloc(typeid, n, zero=zero) @@ -48,7 +52,8 @@ gctypelayout.zero_gc_pointers(result) return result else: - return lltype.malloc(TYPE, n, flavor=flavor, zero=zero) + return lltype.malloc(TYPE, n, flavor=flavor, zero=zero, + track_allocation=track_allocation) def malloc_nonmovable(self, TYPE, n=None, zero=False): typeid = self.get_type_id(TYPE) @@ -66,9 +71,10 @@ return self.gc.shrink_array(addr, smallersize) return False - def free(self, TYPE, flavor='gc'): + def free(self, TYPE, flavor='gc', track_allocation=True): assert flavor != 'gc' - return lltype.free(TYPE, flavor=flavor) + return lltype.free(TYPE, flavor=flavor, + track_allocation=track_allocation) def setfield(self, obj, fieldname, fieldvalue): STRUCT = lltype.typeOf(obj).TO @@ -94,6 +100,7 @@ assert (type(index) is int # <- fast path or lltype.typeOf(index) == lltype.Signed) self.gc.write_barrier_from_array( + llmemory.cast_ptr_to_adr(newvalue), llmemory.cast_ptr_to_adr(toplevelcontainer), index) wb = False @@ -101,6 +108,7 @@ # if wb: self.gc.write_barrier( + llmemory.cast_ptr_to_adr(newvalue), llmemory.cast_ptr_to_adr(toplevelcontainer)) llheap.setinterior(toplevelcontainer, inneraddr, INNERTYPE, newvalue) Modified: pypy/branch/arm-backend/pypy/rpython/memory/support.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/support.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/support.py Tue Nov 23 10:12:47 2010 @@ -30,7 +30,8 @@ # we zero-initialize the chunks to make the translation # backends happy, but we don't need to do it at run-time. zero = not we_are_translated() - return lltype.malloc(CHUNK, flavor="raw", zero=zero) + return lltype.malloc(CHUNK, flavor="raw", zero=zero, + track_allocation=False) result = self.free_list self.free_list = result.next @@ -44,7 +45,7 @@ # Don't cache the old chunks but free them immediately. # Helps debugging, and avoids that old chunks full of # addresses left behind by a test end up in genc... - lltype.free(chunk, flavor="raw") + lltype.free(chunk, flavor="raw", track_allocation=False) unused_chunks = FreeList() cache[chunk_size] = unused_chunks, null_chunk @@ -111,7 +112,7 @@ cur = next free_non_gc_object(self) - def length(self): + def _length_estimate(self): chunk = self.chunk count = self.used_in_last_chunk while chunk: @@ -134,7 +135,7 @@ foreach._annspecialcase_ = 'specialize:arg(1)' def stack2dict(self): - result = AddressDict(self.length()) + result = AddressDict(self._length_estimate()) self.foreach(_add_in_dict, result) return result Modified: pypy/branch/arm-backend/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/test/test_gc.py Tue Nov 23 10:12:47 2010 @@ -278,6 +278,105 @@ res = self.interpret(f, []) assert res + def test_bug_1(self): + import weakref + class B(object): + pass + def g(): + b = B() + llop.gc__collect(lltype.Void) # force 'b' to be old + ref = weakref.ref(B()) + b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = (ref() is None) + return result + res = self.interpret(f, []) + assert res + + def test_cycle_with_weakref_and_del(self): + import weakref, gc + class A(object): + count = 0 + a = A() + class B(object): + def __del__(self): + # when __del__ is called, the weakref to c should be dead + if self.ref() is None: + a.count += 10 # ok + else: + a.count = 666 # not ok + class C(object): + pass + def g(): + c = C() + c.b = B() + ref = weakref.ref(c) + c.b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = a.count + (ref() is None) + return result + res = self.interpret(f, []) + assert res == 11 + + def test_weakref_to_object_with_finalizer_ordering(self): + import weakref, gc + class A(object): + count = 0 + a = A() + class B(object): + def __del__(self): + # when __del__ is called, the weakref to myself is still valid + # in RPython (at least with most GCs; this test might be + # skipped for specific GCs) + if self.ref() is self: + a.count += 10 # ok + else: + a.count = 666 # not ok + def g(): + b = B() + ref = weakref.ref(b) + b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = a.count + (ref() is None) + return result + res = self.interpret(f, []) + assert res == 11 + + def test_weakref_bug_1(self): + import weakref + class A(object): + pass + class B(object): + def __del__(self): + self.wref().x += 1 + def g(a): + b = B() + b.wref = weakref.ref(a) + # the only way to reach this weakref is via B, which is an + # object with finalizer (but the weakref itself points to + # a, which does not go away but will move during the next + # gc.collect) + def f(): + a = A() + a.x = 10 + g(a) + llop.gc__collect(lltype.Void) + return a.x + res = self.interpret(f, []) + assert res == 11 + def test_id(self): class A(object): pass @@ -368,6 +467,14 @@ res = self.interpret(f, [4, 42]) assert res == 12 + def test_print_leak(self): + def f(n): + for i in range(n): + print i + return 42 + res = self.interpret(f, [10]) + assert res == 42 + def test_weakref_across_minor_collection(self): import weakref class A: @@ -627,6 +734,9 @@ class TestMarkSweepGC(GCTest): from pypy.rpython.memory.gc.marksweep import MarkSweepGC as GCClass + def test_weakref_to_object_with_finalizer_ordering(self): + py.test.skip("Does not work") + class TestSemiSpaceGC(GCTest, snippet.SemiSpaceGCTests): from pypy.rpython.memory.gc.semispace import SemiSpaceGC as GCClass GC_CAN_MOVE = True Modified: pypy/branch/arm-backend/pypy/rpython/memory/test/test_transformed_gc.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/memory/test/test_transformed_gc.py (original) +++ pypy/branch/arm-backend/pypy/rpython/memory/test/test_transformed_gc.py Tue Nov 23 10:12:47 2010 @@ -906,7 +906,8 @@ gcname = "marksweep" class gcpolicy(gc.FrameworkGcPolicy): class transformerclass(framework.FrameworkGCTransformer): - GC_PARAMS = {'start_heap_size': 1024*WORD } + GC_PARAMS = {'start_heap_size': 1024*WORD, + 'translated_to_c': False} root_stack_depth = 200 @@ -1144,7 +1145,8 @@ class gcpolicy(gc.FrameworkGcPolicy): class transformerclass(framework.FrameworkGCTransformer): from pypy.rpython.memory.gc.marksweep import PrintingMarkSweepGC as GCClass - GC_PARAMS = {'start_heap_size': 1024*WORD } + GC_PARAMS = {'start_heap_size': 1024*WORD, + 'translated_to_c': False} root_stack_depth = 200 class TestSemiSpaceGC(GenericMovingGCTests): @@ -1154,7 +1156,8 @@ class gcpolicy(gc.FrameworkGcPolicy): class transformerclass(framework.FrameworkGCTransformer): from pypy.rpython.memory.gc.semispace import SemiSpaceGC as GCClass - GC_PARAMS = {'space_size': 512*WORD} + GC_PARAMS = {'space_size': 512*WORD, + 'translated_to_c': False} root_stack_depth = 200 class TestMarkCompactGC(GenericMovingGCTests): @@ -1163,7 +1166,8 @@ class gcpolicy(gc.FrameworkGcPolicy): class transformerclass(framework.FrameworkGCTransformer): from pypy.rpython.memory.gc.markcompact import MarkCompactGC as GCClass - GC_PARAMS = {'space_size': 4096*WORD} + GC_PARAMS = {'space_size': 4096*WORD, + 'translated_to_c': False} root_stack_depth = 200 class TestGenerationGC(GenericMovingGCTests): @@ -1175,7 +1179,8 @@ from pypy.rpython.memory.gc.generation import GenerationGC as \ GCClass GC_PARAMS = {'space_size': 512*WORD, - 'nursery_size': 32*WORD} + 'nursery_size': 32*WORD, + 'translated_to_c': False} root_stack_depth = 200 def define_weakref_across_minor_collection(cls): @@ -1372,7 +1377,8 @@ GenerationGC._teardown(self) GC_PARAMS = {'space_size': 512*WORD, - 'nursery_size': 128*WORD} + 'nursery_size': 128*WORD, + 'translated_to_c': False} root_stack_depth = 200 def define_working_nursery(cls): @@ -1404,7 +1410,8 @@ from pypy.rpython.memory.gc.hybrid import HybridGC as GCClass GC_PARAMS = {'space_size': 512*WORD, 'nursery_size': 32*WORD, - 'large_object': 8*WORD} + 'large_object': 8*WORD, + 'translated_to_c': False} root_stack_depth = 200 def define_ref_from_rawmalloced_to_regular(cls): @@ -1477,6 +1484,7 @@ 'large_object': 8*WORD, 'large_object_gcptrs': 10*WORD, 'card_page_indices': 4, + 'translated_to_c': False, } root_stack_depth = 200 @@ -1585,7 +1593,8 @@ gcname = "marksweep" class gcpolicy(gc.FrameworkGcPolicy): class transformerclass(framework.FrameworkGCTransformer): - GC_PARAMS = {'start_heap_size': 1024*WORD } + GC_PARAMS = {'start_heap_size': 1024*WORD, + 'translated_to_c': False} root_stack_depth = 200 class TestHybridTaggedPointerGC(TaggedPointerGCTests): @@ -1596,7 +1605,8 @@ from pypy.rpython.memory.gc.generation import GenerationGC as \ GCClass GC_PARAMS = {'space_size': 512*WORD, - 'nursery_size': 32*WORD} + 'nursery_size': 32*WORD, + 'translated_to_c': False} root_stack_depth = 200 class TestMarkCompactTaggedpointerGC(TaggedPointerGCTests): @@ -1605,5 +1615,6 @@ class gcpolicy(gc.FrameworkGcPolicy): class transformerclass(framework.FrameworkGCTransformer): from pypy.rpython.memory.gc.markcompact import MarkCompactGC as GCClass - GC_PARAMS = {'space_size': 4096*WORD} + GC_PARAMS = {'space_size': 4096*WORD, + 'translated_to_c': False} root_stack_depth = 200 Modified: pypy/branch/arm-backend/pypy/rpython/module/ll_time.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/module/ll_time.py (original) +++ pypy/branch/arm-backend/pypy/rpython/module/ll_time.py Tue Nov 23 10:12:47 2010 @@ -108,7 +108,7 @@ errcode = -1 if self.GETTIMEOFDAY_NO_TZ: - errcode = g_gettimeofday(t) + errcode = c_gettimeofday(t) else: errcode = c_gettimeofday(t, void) Modified: pypy/branch/arm-backend/pypy/rpython/rbuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/rbuilder.py (original) +++ pypy/branch/arm-backend/pypy/rpython/rbuilder.py Tue Nov 23 10:12:47 2010 @@ -36,8 +36,12 @@ hop.exception_cannot_occur() return hop.gendirectcall(self.ll_append_multiple_char, *vlist) + def rtype_method_getlength(self, hop): + vlist = hop.inputargs(self) + hop.exception_cannot_occur() + return hop.gendirectcall(self.ll_getlength, *vlist) + def rtype_method_build(self, hop): vlist = hop.inputargs(self) hop.exception_cannot_occur() return hop.gendirectcall(self.ll_build, *vlist) - Modified: pypy/branch/arm-backend/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/rbuiltin.py (original) +++ pypy/branch/arm-backend/pypy/rpython/rbuiltin.py Tue Nov 23 10:12:47 2010 @@ -345,17 +345,22 @@ BUILTIN_TYPER[object.__init__] = rtype_object__init__ # annotation of low-level types -def rtype_malloc(hop, i_flavor=None, i_zero=None): +def rtype_malloc(hop, i_flavor=None, i_zero=None, i_track_allocation=None): assert hop.args_s[0].is_constant() vlist = [hop.inputarg(lltype.Void, arg=0)] opname = 'malloc' - v_flavor, v_zero = parse_kwds(hop, (i_flavor, lltype.Void), (i_zero, None)) + v_flavor, v_zero, v_track_allocation = parse_kwds(hop, + (i_flavor, lltype.Void), + (i_zero, None), + (i_track_allocation, None)) flags = {'flavor': 'gc'} if v_flavor is not None: flags['flavor'] = v_flavor.value if i_zero is not None: flags['zero'] = v_zero.value + if i_track_allocation is not None: + flags['track_allocation'] = v_track_allocation.value vlist.append(hop.inputconst(lltype.Void, flags)) if hop.nb_args == 2: @@ -366,12 +371,29 @@ hop.exception_is_here() return hop.genop(opname, vlist, resulttype = hop.r_result.lowleveltype) -def rtype_free(hop, i_flavor): - assert i_flavor == 1 +def rtype_free(hop, i_flavor, i_track_allocation=None): + vlist = [hop.inputarg(hop.args_r[0], arg=0)] + v_flavor, v_track_allocation = parse_kwds(hop, + (i_flavor, lltype.Void), + (i_track_allocation, None)) + # + assert v_flavor is not None and v_flavor.value == 'raw' + flags = {'flavor': 'raw'} + if i_track_allocation is not None: + flags['track_allocation'] = v_track_allocation.value + vlist.append(hop.inputconst(lltype.Void, flags)) + # hop.exception_cannot_occur() - vlist = hop.inputargs(hop.args_r[0], lltype.Void) hop.genop('free', vlist) +def rtype_render_immortal(hop, i_track_allocation=None): + vlist = [hop.inputarg(hop.args_r[0], arg=0)] + v_track_allocation = parse_kwds(hop, + (i_track_allocation, None)) + hop.exception_cannot_occur() + if i_track_allocation is None or v_track_allocation.value: + hop.genop('track_alloc_stop', vlist) + def rtype_const_result(hop): hop.exception_cannot_occur() return hop.inputconst(hop.r_result.lowleveltype, hop.s_result.const) @@ -509,6 +531,7 @@ BUILTIN_TYPER[lltype.malloc] = rtype_malloc BUILTIN_TYPER[lltype.free] = rtype_free +BUILTIN_TYPER[lltype.render_immortal] = rtype_render_immortal BUILTIN_TYPER[lltype.cast_primitive] = rtype_cast_primitive BUILTIN_TYPER[lltype.cast_pointer] = rtype_cast_pointer BUILTIN_TYPER[lltype.cast_opaque_ptr] = rtype_cast_opaque_ptr @@ -584,8 +607,9 @@ vinst, = hop.inputargs(hop.args_r[0]) flavor = hop.args_r[0].gcflavor assert flavor != 'gc' - cflavor = hop.inputconst(lltype.Void, flavor) - return hop.genop('free', [vinst, cflavor]) + flags = {'flavor': flavor} + cflags = hop.inputconst(lltype.Void, flags) + return hop.genop('free', [vinst, cflags]) BUILTIN_TYPER[objectmodel.free_non_gc_object] = rtype_free_non_gc_object Modified: pypy/branch/arm-backend/pypy/rpython/rmodel.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/rmodel.py (original) +++ pypy/branch/arm-backend/pypy/rpython/rmodel.py Tue Nov 23 10:12:47 2010 @@ -11,14 +11,14 @@ # initialization states for Repr instances -class setupstate: +class setupstate(object): NOTINITIALIZED = 0 INPROGRESS = 1 BROKEN = 2 FINISHED = 3 DELAYED = 4 -class Repr: +class Repr(object): """ An instance of Repr is associated with each instance of SomeXxx. It defines the chosen representation for the SomeXxx. The Repr subclasses generally follows the SomeXxx subclass hierarchy, but there are numerous Modified: pypy/branch/arm-backend/pypy/rpython/rpbc.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/rpbc.py (original) +++ pypy/branch/arm-backend/pypy/rpython/rpbc.py Tue Nov 23 10:12:47 2010 @@ -256,6 +256,8 @@ def convert_const(self, value): if isinstance(value, types.MethodType) and value.im_self is None: value = value.im_func # unbound method -> bare function + elif isinstance(value, staticmethod): + value = value.__get__(42) # hackish, get the function wrapped by staticmethod if self.lowleveltype is Void: return None if value is None: Modified: pypy/branch/arm-backend/pypy/rpython/rtyper.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/rtyper.py (original) +++ pypy/branch/arm-backend/pypy/rpython/rtyper.py Tue Nov 23 10:12:47 2010 @@ -421,7 +421,7 @@ assert noexclink.exitcase is None if pos == "removed": # the exception cannot actually occur at all. - # See for example rspecialcase.rtype_call_specialcase(). + # This is set by calling exception_cannot_occur(). # We just remove all exception links. block.exitswitch = None block.exits = block.exits[:1] @@ -1019,7 +1019,7 @@ from pypy.rpython import rint, rbool, rfloat from pypy.rpython import rrange from pypy.rpython import rstr, rdict, rlist -from pypy.rpython import rclass, rbuiltin, rpbc, rspecialcase +from pypy.rpython import rclass, rbuiltin, rpbc from pypy.rpython import rexternalobj from pypy.rpython import rptr from pypy.rpython import rgeneric Modified: pypy/branch/arm-backend/pypy/rpython/test/test_llinterp.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/test/test_llinterp.py (original) +++ pypy/branch/arm-backend/pypy/rpython/test/test_llinterp.py Tue Nov 23 10:12:47 2010 @@ -1,3 +1,4 @@ +from __future__ import with_statement import py import sys from pypy.rpython.lltypesystem.lltype import typeOf, pyobjectptr, Ptr,\ @@ -5,6 +6,7 @@ from pypy.rpython.lltypesystem.lloperation import llop from pypy.rpython.llinterp import LLInterpreter, LLException, log from pypy.rpython.rmodel import inputconst +from pypy.rpython.annlowlevel import hlstr from pypy.translator.translator import TranslationContext, graphof from pypy.rpython.rint import signed_repr from pypy.rpython.lltypesystem import rstr, lltype @@ -12,13 +14,11 @@ from pypy.annotation.model import lltype_to_annotation from pypy.rlib.rarithmetic import r_uint, ovfcheck from pypy.rpython.ootypesystem import ootype +from pypy.tool import leakfinder from pypy import conftest # switch on logging of interp to show more info on failing tests -class MallocMismatch(Exception): - pass - def setup_module(mod): mod.logstate = py.log._getstate() py.log.setconsumer("llinterp", py.log.STDOUT) @@ -72,7 +72,7 @@ def get_interpreter(func, values, view='auto', viewbefore='auto', policy=None, someobjects=False, type_system="lltype", backendopt=False, - config=None, malloc_check=True, **extraconfigopts): + config=None, **extraconfigopts): extra_key = [(key, value) for key, value in extraconfigopts.iteritems()] extra_key.sort() extra_key = tuple(extra_key) @@ -97,7 +97,7 @@ viewbefore, policy, type_system=type_system, backendopt=backendopt, config=config, **extraconfigopts) - interp = LLInterpreter(typer, malloc_check=malloc_check) + interp = LLInterpreter(typer) _tcache[key] = (t, interp, graph) # keep the cache small _lastinterpreted.append(key) @@ -115,10 +115,17 @@ interp, graph = get_interpreter(func, values, view, viewbefore, policy, someobjects, type_system=type_system, backendopt=backendopt, config=config, - malloc_check=malloc_check, **kwargs) - result = interp.eval_graph(graph, values) - if malloc_check and interp.mallocs: - raise MallocMismatch(interp.mallocs) + **kwargs) + if not malloc_check: + result = interp.eval_graph(graph, values) + else: + prev = leakfinder.start_tracking_allocations() + try: + result = interp.eval_graph(graph, values) + finally: + leaks = leakfinder.stop_tracking_allocations(False, prev) + if leaks: + raise leakfinder.MallocMismatch(leaks) return result def interpret_raises(exc, func, values, view='auto', viewbefore='auto', @@ -418,6 +425,7 @@ assert result def test_stack_malloc(): + py.test.skip("stack-flavored mallocs no longer supported") class A(object): pass def f(): @@ -430,6 +438,7 @@ assert result == 1 def test_invalid_stack_access(): + py.test.skip("stack-flavored mallocs no longer supported") class A(object): pass globala = A() @@ -605,7 +614,7 @@ if x: free(t, flavor='raw') interpret(f, [1]) - py.test.raises(MallocMismatch, "interpret(f, [0])") + py.test.raises(leakfinder.MallocMismatch, "interpret(f, [0])") def f(): t1 = malloc(T, flavor='raw') @@ -615,3 +624,37 @@ interpret(f, []) +def test_context_manager(): + state = [] + class C: + def __enter__(self): + state.append('acquire') + return self + def __exit__(self, *args): + if args[1] is not None: + state.append('raised') + state.append('release') + def f(): + try: + with C() as c: + state.append('use') + raise ValueError + except ValueError: + pass + return ', '.join(state) + res = interpret(f, []) + assert hlstr(res) == 'acquire, use, raised, release' + + +def test_scoped_allocator(): + from pypy.rpython.lltypesystem.lltype import scoped_alloc, Array, Signed + T = Array(Signed) + + def f(): + x = 0 + with scoped_alloc(T, 1) as array: + array[0] = -42 + x = array[0] + assert x == -42 + + res = interpret(f, []) Modified: pypy/branch/arm-backend/pypy/rpython/test/test_nongc.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/test/test_nongc.py (original) +++ pypy/branch/arm-backend/pypy/rpython/test/test_nongc.py Tue Nov 23 10:12:47 2010 @@ -79,7 +79,7 @@ py.test.raises(TypeError,rtyper.specialize) # results in an invalid cast def test_isinstance(): - class A: + class A(object): _alloc_flavor_ = "raw" class B(A): pass @@ -95,7 +95,24 @@ o = B() else: o = C() - return 100*isinstance(o, A)+10*isinstance(o, B)+1*isinstance(o ,C) + res = 100*isinstance(o, A) + 10*isinstance(o, B) + 1*isinstance(o, C) + if i == 0: + pass + elif i == 1: + assert isinstance(o, A) + free_non_gc_object(o) + elif i == 2: + assert isinstance(o, B) + free_non_gc_object(o) + else: + assert isinstance(o, C) + free_non_gc_object(o) + return res + + assert f(1) == 100 + assert f(2) == 110 + assert f(3) == 111 + assert f(0) == 0 a = RPythonAnnotator() #does not raise: @@ -131,10 +148,14 @@ d = b elif i == 2: e = c - return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) | + res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) | 0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) | 0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) | 0x0200*(d is e)) + free_non_gc_object(a) + free_non_gc_object(b) + free_non_gc_object(c) + return res a = RPythonAnnotator() #does not raise: s = a.build_types(f, [int]) @@ -169,10 +190,13 @@ d = b elif i == 2: e = c - return (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) | + res = (0x0001*(a is b) | 0x0002*(a is c) | 0x0004*(a is d) | 0x0008*(a is e) | 0x0010*(b is c) | 0x0020*(b is d) | 0x0040*(b is e) | 0x0080*(c is d) | 0x0100*(c is e) | 0x0200*(d is e)) + free_non_gc_object(a) + free_non_gc_object(b) + return res a = RPythonAnnotator() #does not raise: s = a.build_types(f, [int]) Modified: pypy/branch/arm-backend/pypy/rpython/test/test_rbuilder.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/test/test_rbuilder.py (original) +++ pypy/branch/arm-backend/pypy/rpython/test/test_rbuilder.py Tue Nov 23 10:12:47 2010 @@ -1,4 +1,4 @@ - +import py from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin from pypy.rpython.lltypesystem.rbuilder import * from pypy.rpython.annlowlevel import llstr, hlstr @@ -55,8 +55,29 @@ assert res == 'aabcabcdefbuuuu' assert isinstance(res, unicode) + def test_string_getlength(self): + def func(): + s = StringBuilder() + s.append("a") + s.append("abc") + return s.getlength() + res = self.interpret(func, []) + assert res == 4 + + def test_unicode_getlength(self): + def func(): + s = UnicodeBuilder() + s.append(u"a") + s.append(u"abc") + return s.getlength() + res = self.interpret(func, []) + assert res == 4 + class TestLLtype(BaseTestStringBuilder, LLRtypeMixin): pass class TestOOtype(BaseTestStringBuilder, OORtypeMixin): - pass + def test_string_getlength(self): + py.test.skip("getlength(): not implemented on ootype") + def test_unicode_getlength(self): + py.test.skip("getlength(): not implemented on ootype") Modified: pypy/branch/arm-backend/pypy/rpython/test/test_rclass.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/test/test_rclass.py (original) +++ pypy/branch/arm-backend/pypy/rpython/test/test_rclass.py Tue Nov 23 10:12:47 2010 @@ -3,7 +3,7 @@ from pypy.translator.translator import TranslationContext, graphof from pypy.rpython.lltypesystem.lltype import * from pypy.rpython.ootypesystem import ootype -from pypy.rlib.rarithmetic import intmask +from pypy.rlib.rarithmetic import intmask, r_longlong from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin from pypy.objspace.flow.model import summary @@ -319,6 +319,17 @@ res = self.interpret(f, []) assert res == 42 + def test_staticmethod2(self): + class A(object): + f = staticmethod(lambda x, y: x*y) + class B(A): + f = staticmethod(lambda x, y: x+y) + def f(): + b = B() + return b.f(6, 7) + res = self.interpret(f, []) + assert res == 13 + def test_is(self): class A: pass class B(A): pass @@ -1001,6 +1012,40 @@ res = self.interpret(f, [5]) assert res == 0 + def test_order_of_fields(self): + class A(object): + pass + def f(n): + a = A() + a.as_int = n + a.as_char = chr(n) + a.as_unichar = unichr(n) + a.as_double = n + 0.5 + a.as_bool = bool(n) + a.as_void = None + a.as_longlong = r_longlong(n) + a.as_reference = A() + return a + + res = self.interpret(f, [5]) + names = list(typeOf(res).TO._names) + i = names.index('inst_as_int') + c = names.index('inst_as_char') + u = names.index('inst_as_unichar') + d = names.index('inst_as_double') + b = names.index('inst_as_bool') + v = names.index('inst_as_void') + l = names.index('inst_as_longlong') + r = names.index('inst_as_reference') + assert v == 1 # void fields are first + assert sorted([c, b]) == [7, 8] + if sys.maxint == 2147483647: + assert sorted([u, i, r]) == [4, 5, 6] # 32-bit types + assert sorted([d, l]) == [2, 3] # 64-bit types + else: + assert sorted([u]) == [6] # 32-bit types + assert sorted([i, r, d, l]) == [2, 3, 4, 5] # 64-bit types + class TestOOtype(BaseTestRclass, OORtypeMixin): Modified: pypy/branch/arm-backend/pypy/rpython/test/test_rdict.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/test/test_rdict.py (original) +++ pypy/branch/arm-backend/pypy/rpython/test/test_rdict.py Tue Nov 23 10:12:47 2010 @@ -682,6 +682,26 @@ # if it does not crash, we are fine. It crashes if you forget the hash field. self.interpret(func, []) + def test_dict_popitem(self): + def func(): + d = {} + d[5] = 2 + d[6] = 3 + k1, v1 = d.popitem() + assert len(d) == 1 + k2, v2 = d.popitem() + try: + d.popitem() + except KeyError: + pass + else: + assert 0, "should have raised KeyError" + assert len(d) == 0 + return k1*1000 + v1*100 + k2*10 + v2 + + res = self.interpret(func, []) + assert res in [5263, 6352] + # ____________________________________________________________ def test_opt_nullkeymarker(self): Modified: pypy/branch/arm-backend/pypy/rpython/test/test_rpbc.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/test/test_rpbc.py (original) +++ pypy/branch/arm-backend/pypy/rpython/test/test_rpbc.py Tue Nov 23 10:12:47 2010 @@ -1547,7 +1547,7 @@ def test_always_raising_methods(self): class Base: def m(self): - raise NotImplementedError + raise KeyError class A(Base): def m(self): return 42 @@ -1560,11 +1560,11 @@ o = B() try: o.m() - except NotImplementedError: - pass + except KeyError: + assert 0 return B().m() - self.interpret_raises(NotImplementedError, f, [7]) + self.interpret_raises(KeyError, f, [7]) def test_possible_missing_attribute_access(self): py.test.skip("Should explode or give some warning") Modified: pypy/branch/arm-backend/pypy/rpython/test/test_rptr.py ============================================================================== --- pypy/branch/arm-backend/pypy/rpython/test/test_rptr.py (original) +++ pypy/branch/arm-backend/pypy/rpython/test/test_rptr.py Tue Nov 23 10:12:47 2010 @@ -212,10 +212,31 @@ S = Struct('S', ('x', Signed)) def fn(n): - p = malloc(S, flavor='whatever') + p = malloc(S, flavor='raw') p.x = n result = p.x - free(p, flavor='whatever') + free(p, flavor='raw') + return n + + res = interpret(fn, [23]) + assert res == 23 + + S = Struct('S', ('x', Signed)) + def fn(n): + p = malloc(S, flavor='raw', track_allocation=False) + p.x = n + result = p.x + return n + + res = interpret(fn, [23]) + assert res == 23 + + S = Struct('S', ('x', Signed)) + def fn(n): + p = malloc(S, flavor='raw', track_allocation=False) + p.x = n + result = p.x + free(p, flavor='raw', track_allocation=False) return n res = interpret(fn, [23]) Modified: pypy/branch/arm-backend/pypy/tool/alarm.py ============================================================================== --- pypy/branch/arm-backend/pypy/tool/alarm.py (original) +++ pypy/branch/arm-backend/pypy/tool/alarm.py Tue Nov 23 10:12:47 2010 @@ -38,8 +38,9 @@ sys.path.insert(0, os.path.dirname(sys.argv[0])) return sys.argv[0] -finished = [] -try: - execfile(_main_with_alarm(finished)) -finally: - finished.append(True) +if __name__ == '__main__': + finished = [] + try: + execfile(_main_with_alarm(finished)) + finally: + finished.append(True) Modified: pypy/branch/arm-backend/pypy/tool/readdictinfo.py ============================================================================== --- pypy/branch/arm-backend/pypy/tool/readdictinfo.py (original) +++ pypy/branch/arm-backend/pypy/tool/readdictinfo.py Tue Nov 23 10:12:47 2010 @@ -7,37 +7,38 @@ import sys -infile = open(sys.argv[1]) +if __name__ == '__main__': + infile = open(sys.argv[1]) -curr = None -slots = [] -for line in infile: - if line == '------------------\n': - if curr: - break - curr = 1 - else: - attr, val = [s.strip() for s in line.split(':')] - slots.append(attr) + curr = None + slots = [] + for line in infile: + if line == '------------------\n': + if curr: + break + curr = 1 + else: + attr, val = [s.strip() for s in line.split(':')] + slots.append(attr) -class DictInfo(object): - __slots__ = slots + class DictInfo(object): + __slots__ = slots -infile = open(sys.argv[1]) + infile = open(sys.argv[1]) -infos = [] + infos = [] -for line in infile: - if line == '------------------\n': - curr = object.__new__(DictInfo) - infos.append(curr) - else: - attr, val = [s.strip() for s in line.split(':')] - if '.' in val: - val = float(val) + for line in infile: + if line == '------------------\n': + curr = object.__new__(DictInfo) + infos.append(curr) else: - val = int(val) - setattr(curr, attr, val) + attr, val = [s.strip() for s in line.split(':')] + if '.' in val: + val = float(val) + else: + val = int(val) + setattr(curr, attr, val) def histogram(infos, keyattr, *attrs): r = {} Modified: pypy/branch/arm-backend/pypy/tool/release/force-builds.py ============================================================================== --- pypy/branch/arm-backend/pypy/tool/release/force-builds.py (original) +++ pypy/branch/arm-backend/pypy/tool/release/force-builds.py Tue Nov 23 10:12:47 2010 @@ -20,11 +20,12 @@ 'own-linux-x86-32', 'own-linux-x86-64', # 'own-macosx-x86-32', - 'pypy-c-app-level-linux-x86-32', - 'pypy-c-app-level-linux-x86-64', +# 'pypy-c-app-level-linux-x86-32', +# 'pypy-c-app-level-linux-x86-64', 'pypy-c-stackless-app-level-linux-x86-32', 'pypy-c-app-level-win-x86-32', 'pypy-c-jit-linux-x86-32', + 'pypy-c-jit-linux-x86-64', # 'pypy-c-jit-macosx-x86-32', 'pypy-c-jit-win-x86-32', ] Modified: pypy/branch/arm-backend/pypy/tool/release/make_release.py ============================================================================== --- pypy/branch/arm-backend/pypy/tool/release/make_release.py (original) +++ pypy/branch/arm-backend/pypy/tool/release/make_release.py Tue Nov 23 10:12:47 2010 @@ -4,7 +4,7 @@ into release packages. Note: you must run apropriate buildbots first and make sure there are no failures. Use force-builds.py from the same directory. -Usage: make_release.py release/ +Usage: make_release.py release/ release_version """ import autopath @@ -30,7 +30,8 @@ else: xml = override_xml dom = minidom.parseString(xml) - refs = [node.getAttribute('href') for node in dom.getElementsByTagName('a')] + refs = [node.getAttribute('href') for node in dom.getElementsByTagName('a') + if 'pypy' in node.getAttribute('href')] # all refs are of form: pypy-{type}-{revision}-{platform}.tar.bz2 r = re.compile('pypy-c-([\w\d]+)-(\d+)-([\w\d]+).tar.bz2$') d = {} @@ -76,7 +77,7 @@ t.add('pypy-%s' % release) alltars.append(name) t.close() - shutil.rmtree(str(tmpdir.join('pypy-1.3'))) + shutil.rmtree(str(tmpdir.join('pypy-' + release))) for name in alltars: print "Uploading %s" % name os.system('scp %s codespeak.net:/www/pypy.org/htdocs/download' % name) @@ -84,8 +85,8 @@ os.chdir(olddir) if __name__ == '__main__': - if len(sys.argv) != 2: + if len(sys.argv) != 3: print __doc__ sys.exit(1) - main(sys.argv[1], release='1.3') + main(sys.argv[1], release=sys.argv[2]) Modified: pypy/branch/arm-backend/pypy/tool/release/package.py ============================================================================== --- pypy/branch/arm-backend/pypy/tool/release/package.py (original) +++ pypy/branch/arm-backend/pypy/tool/release/package.py Tue Nov 23 10:12:47 2010 @@ -1,8 +1,12 @@ #!/usr/bin/env python """ A sample script that packages PyPy, provided that it's already built. -Usage: +It uses 'pypy/translator/goal/pypy-c' and parts of the rest of the working +copy. Usage: -package.py pypydir [name-of-archive] [name-of-pypy-c] + package.py root-pypy-dir [name-of-archive] [name-of-pypy-c] + +Usually you would do: package.py ../../.. pypy-VER-PLATFORM. +The output is found in the directory /tmp/usession-YOURNAME/build/. """ import autopath @@ -32,7 +36,7 @@ class PyPyCNotFound(Exception): pass -def package(basedir, name='pypy-nightly', rename_pypy_c='pypy-c', +def package(basedir, name='pypy-nightly', rename_pypy_c='pypy', copy_to_dir = None, override_pypy_c = None): basedir = py.path.local(basedir) if sys.platform == 'win32': @@ -64,6 +68,10 @@ headers = includedir.listdir('*.h') + includedir.listdir('*.inl') for n in headers: shutil.copy(str(n), str(pypydir.join('include'))) + # + spdir = pypydir.ensure('site-packages', dir=True) + shutil.copy(str(basedir.join('site-packages', 'README')), str(spdir)) + # pypydir.ensure('bin', dir=True) archive_pypy_c = pypydir.join('bin', rename_pypy_c) shutil.copy(str(pypy_c), str(archive_pypy_c)) Modified: pypy/branch/arm-backend/pypy/tool/release/test/test_package.py ============================================================================== --- pypy/branch/arm-backend/pypy/tool/release/test/test_package.py (original) +++ pypy/branch/arm-backend/pypy/tool/release/test/test_package.py Tue Nov 23 10:12:47 2010 @@ -18,7 +18,7 @@ prefix = builddir.join(test) cpyver = '%d.%d.%d' % CPYTHON_VERSION[:3] assert prefix.join('lib-python', cpyver, 'test').check() - assert prefix.join('bin', 'pypy-c').check() + assert prefix.join('bin', 'pypy').check() assert prefix.join('lib_pypy', 'syslog.py').check() assert not prefix.join('lib_pypy', 'py').check() assert not prefix.join('lib_pypy', 'ctypes_configure').check() Modified: pypy/branch/arm-backend/pypy/tool/rundictbenchmarks.py ============================================================================== --- pypy/branch/arm-backend/pypy/tool/rundictbenchmarks.py (original) +++ pypy/branch/arm-backend/pypy/tool/rundictbenchmarks.py Tue Nov 23 10:12:47 2010 @@ -7,20 +7,21 @@ # need to hack a copy of rst2html for yourself (svn docutils # required). -try: - os.unlink("dictinfo.txt") -except os.error: - pass +if __name__ == '__main__': + try: + os.unlink("dictinfo.txt") + except os.error: + pass -progs = [('pystone', ['-c', 'from test import pystone; pystone.main()']), - ('richards', ['richards.py']), - ('docutils', ['rst2html.py', '../../doc/coding-guide.txt', 'foo.html']), - ('translate', ['translate.py', '--backendopt', '--no-compile', '--batch', - 'targetrpystonedalone.py']) - ] + progs = [('pystone', ['-c', 'from test import pystone; pystone.main()']), + ('richards', ['richards.py']), + ('docutils', ['rst2html.py', '../../doc/coding-guide.txt', 'foo.html']), + ('translate', ['translate.py', '--backendopt', '--no-compile', '--batch', + 'targetrpystonedalone.py']) + ] -EXE = sys.argv[1] + EXE = sys.argv[1] -for suffix, args in progs: - os.spawnv(os.P_WAIT, EXE, [EXE] + args) - os.rename('dictinfo.txt', 'dictinfo-%s.txt'%suffix) + for suffix, args in progs: + os.spawnv(os.P_WAIT, EXE, [EXE] + args) + os.rename('dictinfo.txt', 'dictinfo-%s.txt'%suffix) Modified: pypy/branch/arm-backend/pypy/tool/statistic_irc_log.py ============================================================================== --- pypy/branch/arm-backend/pypy/tool/statistic_irc_log.py (original) +++ pypy/branch/arm-backend/pypy/tool/statistic_irc_log.py Tue Nov 23 10:12:47 2010 @@ -2,53 +2,54 @@ from os import system, chdir from urllib import urlopen -log_URL = 'http://tismerysoft.de/pypy/irc-logs/' -archive_FILENAME = 'pypy.tar.gz' - -tempdir = py.test.ensuretemp("irc-log") - -# get compressed archive -chdir( str(tempdir)) -system('wget -q %s%s' % (log_URL, archive_FILENAME)) -system('tar xzf %s' % archive_FILENAME) -chdir('pypy') - -# get more recent daily logs -pypydir = tempdir.join('pypy') -for line in urlopen(log_URL + 'pypy/').readlines(): - i = line.find('%23pypy.log.') - if i == -1: - continue - filename = line[i:].split('"')[0] - system('wget -q %spypy/%s' % (log_URL, filename)) - -# rename to YYYYMMDD -for log_filename in pypydir.listdir('#pypy.log.*'): - rename_to = None - b = log_filename.basename - if '-' in b: - rename_to = log_filename.basename.replace('-', '') - elif len(b) == 19: - months= 'Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec'.split() - day = b[10:12] - month = months.index(b[12:15]) + 1 - year = b[15:20] - rename_to = '#pypy.log.%04s%02d%02s' % (year, month, day) - - if rename_to: - log_filename.rename(rename_to) - #print 'RENAMED', log_filename, 'TO', rename_to - -# print sorted list of filenames of daily logs -print 'irc://irc.freenode.org/pypy' -print 'date, messages, visitors' -for log_filename in pypydir.listdir('#pypy.log.*'): - n_messages, visitors = 0, {} - f = str(log_filename) - for s in file(f): - if '<' in s and '>' in s: - n_messages += 1 - elif ' joined #pypy' in s: - v = s.split()[1] - visitors[v] = True - print '%04s-%02s-%02s, %d, %d' % (f[-8:-4], f[-4:-2], f[-2:], n_messages, len(visitors.keys())) +if __name__ == '__main__': + log_URL = 'http://tismerysoft.de/pypy/irc-logs/' + archive_FILENAME = 'pypy.tar.gz' + + tempdir = py.test.ensuretemp("irc-log") + + # get compressed archive + chdir( str(tempdir)) + system('wget -q %s%s' % (log_URL, archive_FILENAME)) + system('tar xzf %s' % archive_FILENAME) + chdir('pypy') + + # get more recent daily logs + pypydir = tempdir.join('pypy') + for line in urlopen(log_URL + 'pypy/').readlines(): + i = line.find('%23pypy.log.') + if i == -1: + continue + filename = line[i:].split('"')[0] + system('wget -q %spypy/%s' % (log_URL, filename)) + + # rename to YYYYMMDD + for log_filename in pypydir.listdir('#pypy.log.*'): + rename_to = None + b = log_filename.basename + if '-' in b: + rename_to = log_filename.basename.replace('-', '') + elif len(b) == 19: + months= 'Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec'.split() + day = b[10:12] + month = months.index(b[12:15]) + 1 + year = b[15:20] + rename_to = '#pypy.log.%04s%02d%02s' % (year, month, day) + + if rename_to: + log_filename.rename(rename_to) + #print 'RENAMED', log_filename, 'TO', rename_to + + # print sorted list of filenames of daily logs + print 'irc://irc.freenode.org/pypy' + print 'date, messages, visitors' + for log_filename in pypydir.listdir('#pypy.log.*'): + n_messages, visitors = 0, {} + f = str(log_filename) + for s in file(f): + if '<' in s and '>' in s: + n_messages += 1 + elif ' joined #pypy' in s: + v = s.split()[1] + visitors[v] = True + print '%04s-%02s-%02s, %d, %d' % (f[-8:-4], f[-4:-2], f[-2:], n_messages, len(visitors.keys())) Modified: pypy/branch/arm-backend/pypy/tool/terminal.py ============================================================================== --- pypy/branch/arm-backend/pypy/tool/terminal.py (original) +++ pypy/branch/arm-backend/pypy/tool/terminal.py Tue Nov 23 10:12:47 2010 @@ -62,9 +62,10 @@ for control in CONTROLS: # Set the control escape sequence setattr(MODULE, control, curses.tigetstr(CONTROLS[control]) or '') - for value in VALUES: - # Set terminal related values - setattr(MODULE, value, curses.tigetnum(VALUES[value])) + if hasattr(curses, 'tigetnum'): + for value in VALUES: + # Set terminal related values + setattr(MODULE, value, curses.tigetnum(VALUES[value])) def render(text): """Helper function to apply controls easily @@ -74,7 +75,16 @@ return text % MODULE.__dict__ try: - import curses + if '__pypy__' in sys.builtin_module_names: + # this is not really the whole curses, but our _minimal_curses it's + # better than nothing + import _minimal_curses as curses + # a bit of a hack: we have tigetstr but not tigetnum, so we call + # default() to have default values, then setup() will overwrite the + # ones it can + default() + else: + import curses setup() except Exception, e: # There is a failure; set all attributes to default Modified: pypy/branch/arm-backend/pypy/tool/watchdog.py ============================================================================== --- pypy/branch/arm-backend/pypy/tool/watchdog.py (original) +++ pypy/branch/arm-backend/pypy/tool/watchdog.py Tue Nov 23 10:12:47 2010 @@ -7,9 +7,6 @@ return name return 'signal %d' % (n,) -timeout = float(sys.argv[1]) -timedout = False - def childkill(): global timedout timedout = True @@ -20,31 +17,35 @@ except OSError: pass -pid = os.fork() -if pid == 0: - os.execvp(sys.argv[2], sys.argv[2:]) -else: # parent - t = threading.Timer(timeout, childkill) - t.start() - while True: - try: - pid, status = os.waitpid(pid, 0) - except KeyboardInterrupt: - continue +if __name__ == '__main__': + timeout = float(sys.argv[1]) + timedout = False + + pid = os.fork() + if pid == 0: + os.execvp(sys.argv[2], sys.argv[2:]) + else: # parent + t = threading.Timer(timeout, childkill) + t.start() + while True: + try: + pid, status = os.waitpid(pid, 0) + except KeyboardInterrupt: + continue + else: + t.cancel() + break + if os.WIFEXITED(status): + sys.exit(os.WEXITSTATUS(status)) else: - t.cancel() - break - if os.WIFEXITED(status): - sys.exit(os.WEXITSTATUS(status)) - else: - assert os.WIFSIGNALED(status) - sign = os.WTERMSIG(status) - if timedout and sign == signal.SIGTERM: + assert os.WIFSIGNALED(status) + sign = os.WTERMSIG(status) + if timedout and sign == signal.SIGTERM: + sys.exit(1) + signame = getsignalname(sign) + sys.stderr.write("="*26 + "timedout" + "="*26 + "\n") + sys.stderr.write("="*25 + " %-08s " % signame + "="*25 + "\n") sys.exit(1) - signame = getsignalname(sign) - sys.stderr.write("="*26 + "timedout" + "="*26 + "\n") - sys.stderr.write("="*25 + " %-08s " % signame + "="*25 + "\n") - sys.exit(1) - - + + Modified: pypy/branch/arm-backend/pypy/tool/watchdog_nt.py ============================================================================== --- pypy/branch/arm-backend/pypy/tool/watchdog_nt.py (original) +++ pypy/branch/arm-backend/pypy/tool/watchdog_nt.py Tue Nov 23 10:12:47 2010 @@ -2,11 +2,6 @@ import threading import ctypes -PROCESS_TERMINATE = 0x1 - -timeout = float(sys.argv[1]) -timedout = False - def childkill(pid): global timedout timedout = True @@ -14,19 +9,25 @@ sys.stderr.write("="*26 + "timedout" + "="*26 + "\n") ctypes.windll.kernel32.TerminateProcess(pid, 1) -pid = os.spawnv(os.P_NOWAIT, sys.argv[2], sys.argv[2:]) +if __name__ == '__main__': + PROCESS_TERMINATE = 0x1 + + timeout = float(sys.argv[1]) + timedout = False + + pid = os.spawnv(os.P_NOWAIT, sys.argv[2], sys.argv[2:]) + + t = threading.Timer(timeout, childkill, (pid,)) + t.start() + while True: + try: + pid, status = os.waitpid(pid, 0) + except KeyboardInterrupt: + continue + else: + t.cancel() + break + + #print 'status ', status >> 8 + sys.exit(status >> 8) -t = threading.Timer(timeout, childkill, (pid,)) -t.start() -while True: - try: - pid, status = os.waitpid(pid, 0) - except KeyboardInterrupt: - continue - else: - t.cancel() - break - -#print 'status ', status >> 8 -sys.exit(status >> 8) - Modified: pypy/branch/arm-backend/pypy/translator/c/funcgen.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/funcgen.py (original) +++ pypy/branch/arm-backend/pypy/translator/c/funcgen.py Tue Nov 23 10:12:47 2010 @@ -427,7 +427,7 @@ r = self.expr(op.result) return 'OP_CALL_ARGS((%s), %s);' % (', '.join(args), r) - def generic_call(self, FUNC, fnexpr, args_v, v_result): + def generic_call(self, FUNC, fnexpr, args_v, v_result, targets=None): args = [] assert len(args_v) == len(FUNC.TO.ARGS) for v, ARGTYPE in zip(args_v, FUNC.TO.ARGS): @@ -444,17 +444,26 @@ # skip assignment of 'void' return value r = self.expr(v_result) line = '%s = %s' % (r, line) + if targets: + for graph in targets: + if getattr(graph, 'inhibit_tail_call', False): + line += '\nPYPY_INHIBIT_TAIL_CALL();' + break return line def OP_DIRECT_CALL(self, op): fn = op.args[0] + try: + targets = [fn.value._obj.graph] + except AttributeError: + targets = None return self.generic_call(fn.concretetype, self.expr(fn), - op.args[1:], op.result) + op.args[1:], op.result, targets) def OP_INDIRECT_CALL(self, op): fn = op.args[0] return self.generic_call(fn.concretetype, self.expr(fn), - op.args[1:-1], op.result) + op.args[1:-1], op.result, op.args[-1].value) def OP_ADR_CALL(self, op): ARGTYPES = [v.concretetype for v in op.args[1:]] Modified: pypy/branch/arm-backend/pypy/translator/c/gcc/instruction.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/gcc/instruction.py (original) +++ pypy/branch/arm-backend/pypy/translator/c/gcc/instruction.py Tue Nov 23 10:12:47 2010 @@ -82,6 +82,11 @@ def all_sources_of(self, localvar): return [localvar] +class InsnCondJump(Insn): # only for debugging; not used internally + _args_ = ['label'] + def __init__(self, label): + self.label = label + class Label(Insn): _args_ = ['label', 'lineno'] def __init__(self, label, lineno): @@ -170,9 +175,12 @@ self.delta = -7 # use an odd value as marker class InsnStop(Insn): - pass + _args_ = ['reason'] + def __init__(self, reason='?'): + self.reason = reason class InsnRet(InsnStop): + _args_ = [] framesize = 0 def __init__(self, registers): self.registers = registers Modified: pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf/track5.s ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf/track5.s (original) +++ pypy/branch/arm-backend/pypy/translator/c/gcc/test/elf/track5.s Tue Nov 23 10:12:47 2010 @@ -44,7 +44,7 @@ addl %eax, %ebx jmp .L1221 .L1227: - call RPyAbort + ;;call RPyAbort cmpl 12(%esi), %ebx jb .L1229 addl $20, %esp Modified: pypy/branch/arm-backend/pypy/translator/c/gcc/test/test_trackgcroot.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/gcc/test/test_trackgcroot.py (original) +++ pypy/branch/arm-backend/pypy/translator/c/gcc/test/test_trackgcroot.py Tue Nov 23 10:12:47 2010 @@ -98,18 +98,17 @@ """ lines = source.splitlines(True) parts = list(DarwinAssemblerParser().find_functions(iter(lines))) - assert len(parts) == 7 + assert len(parts) == 6 assert parts[0] == (False, lines[:3]) assert parts[1] == (True, lines[3:7]) assert parts[2] == (True, lines[7:11]) - assert parts[3] == (True, lines[11:13]) - assert parts[4] == (False, lines[13:18]) - assert parts[5] == (True, lines[18:20]) - assert parts[6] == (False, lines[20:]) + assert parts[3] == (True, lines[11:18]) + assert parts[4] == (True, lines[18:20]) + assert parts[5] == (False, lines[20:]) def test_computegcmaptable(): tests = [] - for format in ('elf', 'darwin', 'msvc', 'elf64'): + for format in ('elf', 'elf64', 'darwin', 'darwin64', 'msvc'): for path in this_dir.join(format).listdir("track*.s"): n = path.purebasename[5:] try: Modified: pypy/branch/arm-backend/pypy/translator/c/gcc/trackgcroot.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/gcc/trackgcroot.py (original) +++ pypy/branch/arm-backend/pypy/translator/c/gcc/trackgcroot.py Tue Nov 23 10:12:47 2010 @@ -6,7 +6,7 @@ from pypy.translator.c.gcc.instruction import InsnFunctionStart, InsnStop from pypy.translator.c.gcc.instruction import InsnSetLocal, InsnCopyLocal from pypy.translator.c.gcc.instruction import InsnPrologue, InsnEpilogue -from pypy.translator.c.gcc.instruction import InsnGCROOT +from pypy.translator.c.gcc.instruction import InsnGCROOT, InsnCondJump from pypy.translator.c.gcc.instruction import InsnStackAdjust from pypy.translator.c.gcc.instruction import InsnCannotFollowEsp from pypy.translator.c.gcc.instruction import LocalVar, somenewvalue @@ -46,6 +46,7 @@ self.findlabels() self.parse_instructions() try: + self.trim_unreachable_instructions() self.find_noncollecting_calls() if not self.list_collecting_call_insns(): return [] @@ -122,19 +123,36 @@ assert label not in self.labels, "duplicate label: %s" % label self.labels[label] = Label(label, lineno) + def trim_unreachable_instructions(self): + reached = set([self.insns[0]]) + prevlen = 0 + while len(reached) > prevlen: + prevlen = len(reached) + for insn in self.insns: + if insn not in reached: + for previnsn in insn.previous_insns: + if previnsn in reached: + # this instruction is reachable too + reached.add(insn) + break + # now kill all unreachable instructions + i = 0 + while i < len(self.insns): + if self.insns[i] in reached: + i += 1 + else: + del self.insns[i] + def find_noncollecting_calls(self): - cannot_collect = self.CANNOT_COLLECT.copy() + cannot_collect = {} for line in self.lines: match = self.r_gcnocollect_marker.search(line) if match: name = match.group(1) cannot_collect[name] = True # - if self.format in ('darwin', 'mingw32', 'msvc'): - self.cannot_collect = dict.fromkeys( - ['_' + name for name in cannot_collect]) - else: - self.cannot_collect = cannot_collect + self.cannot_collect = dict.fromkeys( + [self.function_names_prefix + name for name in cannot_collect]) def append_instruction(self, insn): # Add the instruction to the list, and link it to the previous one. @@ -410,7 +428,8 @@ return result # ____________________________________________________________ - CANNOT_COLLECT = { # some of the most used functions that cannot collect + BASE_FUNCTIONS_NOT_RETURNING = { + 'abort': None, 'pypy_debug_catch_fatal_exception': None, 'RPyAbort': None, 'RPyAssertFailed': None, @@ -436,7 +455,7 @@ 'inc', 'dec', 'not', 'neg', 'or', 'and', 'sbb', 'adc', 'shl', 'shr', 'sal', 'sar', 'rol', 'ror', 'mul', 'imul', 'div', 'idiv', 'bswap', 'bt', 'rdtsc', - 'punpck', 'pshufd', + 'punpck', 'pshufd', 'psll', # zero-extending moves should not produce GC pointers 'movz', ]) @@ -644,7 +663,7 @@ if label != '0': self.register_jump_to(label) tablelin += 1 - return InsnStop() + return InsnStop("jump table") if self.r_unaryinsn_star.match(line): # that looks like an indirect tail-call. # tail-calls are equivalent to RET for us @@ -658,7 +677,7 @@ assert not target.startswith('.') # tail-calls are equivalent to RET for us return InsnRet(self.CALLEE_SAVE_REGISTERS) - return InsnStop() + return InsnStop("jump") def register_jump_to(self, label): if not isinstance(self.insns[-1], InsnStop): @@ -682,7 +701,7 @@ else: label = match.group(1) self.register_jump_to(label) - return [] + return [InsnCondJump(label)] visit_jmpl = visit_jmp visit_je = conditional_jump @@ -754,7 +773,7 @@ target, = sources if target in self.FUNCTIONS_NOT_RETURNING: - return [InsnStop(), InsnCannotFollowEsp()] + return [InsnStop(target)] if self.format == 'mingw32' and target == '__alloca': # in functions with large stack requirements, windows # needs a call to _alloca(), to turn reserved pages @@ -885,7 +904,7 @@ # statically known pointer to a register # %eax -> %rax - new_line = re.sub(r"%e(ax|bx|cx|dx|di|si)$", r"%r\1", line) + new_line = re.sub(r"%e(ax|bx|cx|dx|di|si|bp)$", r"%r\1", line) # %r10d -> %r10 new_line = re.sub(r"%r(\d+)d$", r"%r\1", new_line) return func(self, new_line) @@ -951,6 +970,7 @@ class ElfFunctionGcRootTracker32(FunctionGcRootTracker32): format = 'elf' + function_names_prefix = '' ESP = '%esp' EBP = '%ebp' @@ -984,13 +1004,14 @@ r_bottom_marker = re.compile(r"\t/[*] GC_STACK_BOTTOM [*]/") FUNCTIONS_NOT_RETURNING = { - 'abort': None, '_exit': None, '__assert_fail': None, '___assert_rtn': None, 'L___assert_rtn$stub': None, 'L___eprintf$stub': None, } + for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING: + FUNCTIONS_NOT_RETURNING[_name] = None def __init__(self, lines, filetag=0): match = self.r_functionstart.match(lines[0]) @@ -1010,6 +1031,8 @@ class ElfFunctionGcRootTracker64(FunctionGcRootTracker64): format = 'elf64' + function_names_prefix = '' + ESP = '%rsp' EBP = '%rbp' EAX = '%rax' @@ -1042,13 +1065,14 @@ r_bottom_marker = re.compile(r"\t/[*] GC_STACK_BOTTOM [*]/") FUNCTIONS_NOT_RETURNING = { - 'abort': None, '_exit': None, '__assert_fail': None, '___assert_rtn': None, 'L___assert_rtn$stub': None, 'L___eprintf$stub': None, } + for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING: + FUNCTIONS_NOT_RETURNING[_name] = None def __init__(self, lines, filetag=0): match = self.r_functionstart.match(lines[0]) @@ -1066,8 +1090,9 @@ ElfFunctionGcRootTracker64.init_regexp() -class DarwinFunctionGcRootTracker(ElfFunctionGcRootTracker32): +class DarwinFunctionGcRootTracker32(ElfFunctionGcRootTracker32): format = 'darwin' + function_names_prefix = '_' r_functionstart = re.compile(r"_(\w+):\s*$") OFFSET_LABELS = 0 @@ -1077,17 +1102,36 @@ funcname = '_' + match.group(1) FunctionGcRootTracker32.__init__(self, funcname, lines, filetag) -class Mingw32FunctionGcRootTracker(DarwinFunctionGcRootTracker): +class DarwinFunctionGcRootTracker64(ElfFunctionGcRootTracker64): + format = 'darwin64' + function_names_prefix = '_' + + LABEL = ElfFunctionGcRootTracker32.LABEL + r_jmptable_item = re.compile(r"\t.(?:long|quad)\t"+LABEL+"(-\"?[A-Za-z0-9$]+\"?)?\s*$") + + r_functionstart = re.compile(r"_(\w+):\s*$") + OFFSET_LABELS = 0 + + def __init__(self, lines, filetag=0): + match = self.r_functionstart.match(lines[0]) + funcname = '_' + match.group(1) + FunctionGcRootTracker64.__init__(self, funcname, lines, filetag) + +class Mingw32FunctionGcRootTracker(DarwinFunctionGcRootTracker32): format = 'mingw32' + function_names_prefix = '_' FUNCTIONS_NOT_RETURNING = { - '_abort': None, '_exit': None, '__assert': None, } + for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING: + FUNCTIONS_NOT_RETURNING['_' + _name] = None class MsvcFunctionGcRootTracker(FunctionGcRootTracker32): format = 'msvc' + function_names_prefix = '_' + ESP = 'esp' EBP = 'ebp' EAX = 'eax' @@ -1127,7 +1171,6 @@ r_bottom_marker = re.compile(r"; .+\tpypy_asm_stack_bottom\(\);") FUNCTIONS_NOT_RETURNING = { - '_abort': None, '__exit': None, '__assert': None, '__wassert': None, @@ -1136,6 +1179,8 @@ 'DWORD PTR __imp__abort': None, 'DWORD PTR __imp___wassert': None, } + for _name in FunctionGcRootTracker.BASE_FUNCTIONS_NOT_RETURNING: + FUNCTIONS_NOT_RETURNING['_' + _name] = None @classmethod def init_regexp(cls): @@ -1343,7 +1388,7 @@ class DarwinAssemblerParser(AssemblerParser): format = "darwin" - FunctionGcRootTracker = DarwinFunctionGcRootTracker + FunctionGcRootTracker = DarwinFunctionGcRootTracker32 r_textstart = re.compile(r"\t.text\s*$") @@ -1360,6 +1405,7 @@ 'const_data' ] r_sectionstart = re.compile(r"\t\.("+'|'.join(OTHERSECTIONS)+").*$") + sections_doesnt_end_function = {'cstring': True, 'const': True} def find_functions(self, iterlines): functionlines = [] @@ -1367,20 +1413,20 @@ in_function = False for n, line in enumerate(iterlines): if self.r_textstart.match(line): - assert not in_text, "unexpected repeated .text start: %d" % n in_text = True elif self.r_sectionstart.match(line): - if in_function: + sectionname = self.r_sectionstart.match(line).group(1) + if (in_function and + sectionname not in self.sections_doesnt_end_function): yield in_function, functionlines functionlines = [] + in_function = False in_text = False - in_function = False elif in_text and self.FunctionGcRootTracker.r_functionstart.match(line): yield in_function, functionlines functionlines = [] in_function = True functionlines.append(line) - if functionlines: yield in_function, functionlines @@ -1389,27 +1435,14 @@ return super(DarwinAssemblerParser, self).process_function( lines, entrypoint, filename) +class DarwinAssemblerParser64(DarwinAssemblerParser): + format = "darwin64" + FunctionGcRootTracker = DarwinFunctionGcRootTracker64 + class Mingw32AssemblerParser(DarwinAssemblerParser): format = "mingw32" FunctionGcRootTracker = Mingw32FunctionGcRootTracker - def find_functions(self, iterlines): - functionlines = [] - in_text = False - in_function = False - for n, line in enumerate(iterlines): - if self.r_textstart.match(line): - in_text = True - elif self.r_sectionstart.match(line): - in_text = False - elif in_text and self.FunctionGcRootTracker.r_functionstart.match(line): - yield in_function, functionlines - functionlines = [] - in_function = True - functionlines.append(line) - if functionlines: - yield in_function, functionlines - class MsvcAssemblerParser(AssemblerParser): format = "msvc" FunctionGcRootTracker = MsvcFunctionGcRootTracker @@ -1512,6 +1545,7 @@ 'elf': ElfAssemblerParser, 'elf64': ElfAssemblerParser64, 'darwin': DarwinAssemblerParser, + 'darwin64': DarwinAssemblerParser64, 'mingw32': Mingw32AssemblerParser, 'msvc': MsvcAssemblerParser, } @@ -1543,15 +1577,13 @@ assert self.seen_main def _globalname(name, disp=""): - if self.format in ('darwin', 'mingw32', 'msvc'): - name = '_' + name - return name + return tracker_cls.function_names_prefix + name def _variant(**kwargs): txt = kwargs[self.format] print >> output, "\t%s" % txt - if self.format == 'elf64': + if self.format in ('elf64', 'darwin64'): word_decl = '.quad' else: word_decl = '.long' @@ -1604,10 +1636,11 @@ } } """ - elif self.format == 'elf64': + elif self.format in ('elf64', 'darwin64'): print >> output, "\t.text" print >> output, "\t.globl %s" % _globalname('pypy_asm_stackwalk') - print >> output, "\t.type pypy_asm_stackwalk, @function" + _variant(elf64='.type pypy_asm_stackwalk, @function', + darwin64='') print >> output, "%s:" % _globalname('pypy_asm_stackwalk') print >> output, """\ @@ -1652,8 +1685,9 @@ /* the return value is the one of the 'call' above, */ /* because %rax (and possibly %rdx) are unmodified */ ret - .size pypy_asm_stackwalk, .-pypy_asm_stackwalk """ + _variant(elf64='.size pypy_asm_stackwalk, .-pypy_asm_stackwalk', + darwin64='') else: print >> output, "\t.text" print >> output, "\t.globl %s" % _globalname('pypy_asm_stackwalk') @@ -1780,6 +1814,7 @@ _variant(elf='.section\t.rodata', elf64='.section\t.rodata', darwin='.const', + darwin64='.const', mingw32='') print >> output, """\ @@ -1853,11 +1888,14 @@ if __name__ == '__main__': - verbose = 1 + verbose = 0 shuffle = False output_raw_table = False if sys.platform == 'darwin': - format = 'darwin' + if sys.maxint > 2147483647: + format = 'darwin64' + else: + format = 'darwin' elif sys.platform == 'win32': format = 'mingw32' else: Modified: pypy/branch/arm-backend/pypy/translator/c/genc.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/genc.py (original) +++ pypy/branch/arm-backend/pypy/translator/c/genc.py Tue Nov 23 10:12:47 2010 @@ -553,7 +553,7 @@ ('debug', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT" $(TARGET)'), ('debug_exc', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DDO_LOG_EXC" $(TARGET)'), ('debug_mem', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DTRIVIAL_MALLOC_DEBUG" $(TARGET)'), - ('no_obmalloc', '', '$(MAKE) CFLAGS="-g -O1 -DRPY_ASSERT -DNO_OBMALLOC" $(TARGET)'), + ('no_obmalloc', '', '$(MAKE) CFLAGS="-g -O2 -DRPY_ASSERT -DNO_OBMALLOC" $(TARGET)'), ('linuxmemchk', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DLINUXMEMCHK" $(TARGET)'), ('llsafer', '', '$(MAKE) CFLAGS="-O2 -DRPY_LL_ASSERT" $(TARGET)'), ('lldebug', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DRPY_LL_ASSERT" $(TARGET)'), @@ -623,9 +623,16 @@ mk.definition('OBJECTS', '$(ASMLBLFILES) gcmaptable.s') mk.rule('%.s', '%.c', '$(CC) $(CFLAGS) $(CFLAGSEXTRA) -frandom-seed=$< -o $@ -S $< $(INCLUDEDIRS)') mk.rule('%.lbl.s %.gcmap', '%.s', - python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py -m$(PYPY_MAIN_FUNCTION) -t $< > $*.gcmap') + [python + + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py ' + '-m$(PYPY_MAIN_FUNCTION) -t $< > $*.gctmp', + 'mv $*.gctmp $*.gcmap']) mk.rule('gcmaptable.s', '$(GCMAPFILES)', - python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py $(GCMAPFILES) > $@') + [python + + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py ' + '$(GCMAPFILES) > $@.tmp', + 'mv $@.tmp $@']) + mk.rule('.PRECIOUS', '%.s', "# don't remove .s files if Ctrl-C'ed") else: mk.definition('DEBUGFLAGS', '-O1 -g') Modified: pypy/branch/arm-backend/pypy/translator/c/node.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/node.py (original) +++ pypy/branch/arm-backend/pypy/translator/c/node.py Tue Nov 23 10:12:47 2010 @@ -714,7 +714,11 @@ s = ''.join([self.obj.getitem(i) for i in range(len(self.obj.items))]) else: s = ''.join(self.obj.items) - yield '\t%s%s' % (length, c_char_array_constant(s)) + array_constant = c_char_array_constant(s) + if array_constant.startswith('{') and barebonearray(T): + assert array_constant.endswith('}') + array_constant = array_constant[1:-1].strip() + yield '\t%s%s' % (length, array_constant) yield '}' else: barebone = barebonearray(T) Modified: pypy/branch/arm-backend/pypy/translator/c/src/asm_gcc_x86.h ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/src/asm_gcc_x86.h (original) +++ pypy/branch/arm-backend/pypy/translator/c/src/asm_gcc_x86.h Tue Nov 23 10:12:47 2010 @@ -2,6 +2,8 @@ * It replaces some complex macros with native assembler instructions. */ +#if 0 /* --- disabled: does not give any speed-up --- */ + #undef OP_INT_ADD_OVF #define OP_INT_ADD_OVF(x,y,r) \ asm volatile( \ @@ -50,6 +52,13 @@ : "0"(x), "g"(y) /* inputs */ \ : "cc", "memory") /* clobber */ +extern void op_int_overflowed(void) + asm ("_op_int_overflowed") + __attribute__((used)); + +#endif /* 0 */ + + /* Pentium only! */ #define READ_TIMESTAMP(val) \ asm volatile("rdtsc" : "=A" (val)) @@ -62,19 +71,15 @@ // I don't know how important it is, comment talks about time warps -/* prototypes */ - -extern void op_int_overflowed(void) - asm ("_op_int_overflowed") - __attribute__((used)); - /* implementations */ #ifndef PYPY_NOT_MAIN_FILE +# if 0 /* disabled */ void op_int_overflowed(void) { FAIL_OVF("integer operation"); } +# endif #endif Modified: pypy/branch/arm-backend/pypy/translator/c/src/debug_print.h ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/src/debug_print.h (original) +++ pypy/branch/arm-backend/pypy/translator/c/src/debug_print.h Tue Nov 23 10:12:47 2010 @@ -10,6 +10,7 @@ but not any nested debug_print :fname full logging prefix:fname conditional logging + prefix1,prefix2:fname conditional logging with multiple selections Conditional logging means that it only includes the debug_start/debug_stop sections whose name match 'prefix'. Other sections are ignored, including @@ -70,6 +71,8 @@ static void pypy_debug_open(void) { char *filename = getenv("PYPYLOG"); + if (filename) + unsetenv("PYPYLOG"); /* don't pass it to subprocesses */ if (filename && filename[0]) { char *colon = strchr(filename, ':'); @@ -139,12 +142,22 @@ #endif -static bool_t startswith(const char *str, const char *substr) +static bool_t startswithoneof(const char *str, const char *substr) { - while (*substr) - if (*str++ != *substr++) - return 0; - return 1; + const char *p = str; + for (; *substr; substr++) + { + if (*substr != ',') + { + if (p && *p++ != *substr) + p = NULL; /* mismatch */ + } + else if (p != NULL) + return 1; /* match */ + else + p = str; /* mismatched, retry with the next */ + } + return p != NULL; } #if defined(_MSC_VER) || defined(__MINGW32__) @@ -175,7 +188,7 @@ if (!debug_profile) { /* non-profiling version */ - if (!debug_prefix || !startswith(category, debug_prefix)) + if (!debug_prefix || !startswithoneof(category, debug_prefix)) { /* wrong section name, or no PYPYLOG at all, skip it */ return; Modified: pypy/branch/arm-backend/pypy/translator/c/src/g_include.h ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/src/g_include.h (original) +++ pypy/branch/arm-backend/pypy/translator/c/src/g_include.h Tue Nov 23 10:12:47 2010 @@ -43,6 +43,10 @@ # include "src/asm_gcc_x86.h" #endif +#if defined(__GNUC__) && defined(__amd64__) +# include "src/asm_gcc_x86_64.h" +#endif + #if defined(__GNUC__) && defined(__ppc__) # include "src/asm_ppc.h" #endif @@ -53,6 +57,7 @@ # include "src/rtyper.h" # include "src/debug_print.h" # include "src/debug_traceback.h" +# include "src/debug_alloc.h" #ifndef AVR # include "src/ll_os.h" # include "src/ll_strtod.h" Modified: pypy/branch/arm-backend/pypy/translator/c/src/int.h ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/src/int.h (original) +++ pypy/branch/arm-backend/pypy/translator/c/src/int.h Tue Nov 23 10:12:47 2010 @@ -2,40 +2,27 @@ /************************************************************/ /*** C header subsection: operations between ints ***/ -#ifndef LLONG_MAX -# if SIZEOF_LONG_LONG == 8 -# define LLONG_MAX 0X7FFFFFFFFFFFFFFFLL -# else -# error "fix LLONG_MAX" -# endif -#endif - -#ifndef LLONG_MIN -# define LLONG_MIN (-LLONG_MAX-1) -#endif /*** unary operations ***/ -#define OP_INT_IS_TRUE(x,r) OP_INT_NE(x,0,r) - -#define OP_INT_INVERT(x,r) r = ~((x)) - -#define OP_INT_NEG(x,r) r = -(x) +#define OP_INT_IS_TRUE(x,r) r = ((x) != 0) +#define OP_INT_INVERT(x,r) r = ~(x) +#define OP_INT_NEG(x,r) r = -(x) #define OP_INT_NEG_OVF(x,r) \ - if ((x) == LONG_MIN) FAIL_OVF("integer negate"); \ + if ((x) == LONG_MIN) FAIL_OVF("integer negate"); \ OP_INT_NEG(x,r) #define OP_LLONG_NEG_OVF(x,r) \ - if ((x) == LLONG_MIN) FAIL_OVF("integer negate"); \ + if ((x) == LLONG_MIN) FAIL_OVF("integer negate"); \ OP_LLONG_NEG(x,r) #define OP_INT_ABS(x,r) r = (x) >= 0 ? x : -(x) #define OP_INT_ABS_OVF(x,r) \ - if ((x) == LONG_MIN) FAIL_OVF("integer absolute"); \ + if ((x) == LONG_MIN) FAIL_OVF("integer absolute"); \ OP_INT_ABS(x,r) #define OP_LLONG_ABS_OVF(x,r) \ - if ((x) == LLONG_MIN) FAIL_OVF("integer absolute"); \ + if ((x) == LLONG_MIN) FAIL_OVF("integer absolute"); \ OP_LLONG_ABS(x,r) /*** binary operations ***/ @@ -59,50 +46,46 @@ #define OP_INT_ADD(x,y,r) r = (x) + (y) +/* cast to avoid undefined behaviour on overflow */ #define OP_INT_ADD_OVF(x,y,r) \ - OP_INT_ADD(x,y,r); \ - if ((r^(x)) >= 0 || (r^(y)) >= 0); \ - else FAIL_OVF("integer addition") + r = (long)((unsigned long)x + y); \ + if ((r^x) < 0 && (r^y) < 0) FAIL_OVF("integer addition") + +#define OP_LLONG_ADD_OVF(x,y,r) \ + r = (long long)((unsigned long long)x + y); \ + if ((r^x) < 0 && (r^y) < 0) FAIL_OVF("integer addition") #define OP_INT_ADD_NONNEG_OVF(x,y,r) /* y can be assumed >= 0 */ \ - r = (long)((unsigned long)x + (unsigned long)y); \ - if (r >= (x)); \ - else FAIL_OVF("integer addition") -/* Can a C compiler be too clever and think it can "prove" that - * r >= x always holds above? Yes. Hence the casting. */ + r = (long)((unsigned long)x + y); \ + if ((r&~x) < 0) FAIL_OVF("integer addition") #define OP_INT_SUB(x,y,r) r = (x) - (y) #define OP_INT_SUB_OVF(x,y,r) \ - OP_INT_SUB(x,y,r); \ - if ((r^(x)) >= 0 || (r^~(y)) >= 0); \ - else FAIL_OVF("integer subtraction") - -#define OP_INT_MUL(x,y,r) r = (x) * (y) - -#if defined(HAVE_LONG_LONG) && SIZE_OF_LONG_LONG < SIZE_OF_LONG -# define OP_INT_MUL_OVF_LL 1 -#lse -# define OP_INT_MUL_OVF_LL 0 -#endif + r = (long)((unsigned long)x - y); \ + if ((r^x) < 0 && (r^~y) < 0) FAIL_OVF("integer subtraction") -#if !OP_INT_MUL_OVF_LL +#define OP_LLONG_SUB_OVF(x,y,r) \ + r = (long long)((unsigned long long)x - y); \ + if ((r^x) < 0 && (r^~y) < 0) FAIL_OVF("integer subtraction") -#define OP_INT_MUL_OVF(x,y,r) \ - if (op_int_mul_ovf(x,y,&r)); \ - else FAIL_OVF("integer multiplication") - -#else +#define OP_INT_MUL(x,y,r) r = (x) * (y) +#if SIZEOF_LONG * 2 <= SIZEOF_LONG_LONG #define OP_INT_MUL_OVF(x,y,r) \ { \ - PY_LONG_LONG lr = (PY_LONG_LONG)(x) * (PY_LONG_LONG)(y); \ - r = (long)lr; \ - if ((PY_LONG_LONG)r == lr); \ - else FAIL_OVF("integer multiplication"); \ + long long _lr = (long long)x * y; \ + r = (long)_lr; \ + if (_lr != (long long)r) FAIL_OVF("integer multiplication"); \ } +#else +#define OP_INT_MUL_OVF(x,y,r) \ + r = op_llong_mul_ovf(x, y) /* long == long long */ #endif +#define OP_LLONG_MUL_OVF(x,y,r) \ + r = op_llong_mul_ovf(x, y) + /* shifting */ /* NB. shifting has same limitations as C: the shift count must be @@ -121,6 +104,10 @@ OP_INT_LSHIFT(x,y,r); \ if ((x) != Py_ARITHMETIC_RIGHT_SHIFT(long, r, (y))) \ FAIL_OVF("x< -#ifndef LONG_MAX -#if SIZEOF_LONG == 4 -#define LONG_MAX 0X7FFFFFFFL -#elif SIZEOF_LONG == 8 -#define LONG_MAX 0X7FFFFFFFFFFFFFFFL -#else -#error "could not set LONG_MAX in pyport.h" -#endif -#endif - -#ifndef LONG_MIN -#define LONG_MIN (-LONG_MAX-1) -#endif - #include #ifdef MS_WINDOWS @@ -28,10 +14,6 @@ #include -#ifndef SIG_ERR -#define SIG_ERR ((PyOS_sighandler_t)(-1)) -#endif - #if defined(PYOS_OS2) && !defined(PYCC_GCC) #define NSIG 12 #include @@ -65,16 +47,12 @@ /* utility to poll for signals that arrived */ int pypysig_poll(void); /* => signum or -1 */ -/* When a signal is received, the high bit of pypysig_occurred is set. - After all signals are processed by pypysig_poll(), the high bit is - cleared again. The variable is exposed and RPython code is free to - use the other bits in any way. */ -#define PENDING_SIGNAL_BIT (LONG_MIN) /* high bit */ +/* When a signal is received, pypysig_counter is set to -1. */ /* This is a struct for the JIT. See interp_signal.py. */ struct pypysig_long_struct { long value; }; -extern struct pypysig_long_struct pypysig_occurred; +extern struct pypysig_long_struct pypysig_counter; /* some C tricks to get/set the variable as efficiently as possible: use macros when compiling as a stand-alone program, but still @@ -82,18 +60,20 @@ #undef pypysig_getaddr_occurred void *pypysig_getaddr_occurred(void); #ifndef PYPY_NOT_MAIN_FILE -void *pypysig_getaddr_occurred(void) { return (void *)(&pypysig_occurred); } +void *pypysig_getaddr_occurred(void) { return (void *)(&pypysig_counter); } #endif -#define pypysig_getaddr_occurred() ((void *)(&pypysig_occurred)) +#define pypysig_getaddr_occurred() ((void *)(&pypysig_counter)) /************************************************************/ /* Implementation */ #ifndef PYPY_NOT_MAIN_FILE -struct pypysig_long_struct pypysig_occurred; -static volatile long *pypysig_occurred_v = (volatile long *)&pypysig_occurred.value; -static volatile int pypysig_flags[NSIG]; +struct pypysig_long_struct pypysig_counter = {0}; +static char volatile pypysig_flags[NSIG] = {0}; +static int volatile pypysig_occurred = 0; +/* pypysig_occurred is only an optimization: it tells if any + pypysig_flags could be set. */ void pypysig_ignore(int signum) { @@ -126,10 +106,11 @@ static void signal_setflag_handler(int signum) { if (0 <= signum && signum < NSIG) + { pypysig_flags[signum] = 1; - /* the point of "*pypysig_occurred_v" instead of just "pypysig_occurred" - is the volatile declaration */ - *pypysig_occurred_v |= PENDING_SIGNAL_BIT; + pypysig_occurred = 1; + pypysig_counter.value = -1; + } } void pypysig_setflag(int signum) @@ -148,27 +129,21 @@ int pypysig_poll(void) { - /* the two commented out lines below are useful for performance in - normal usage of pypysig_poll(); however, pypy/module/signal/ is - not normal usage. It only calls pypysig_poll() if the - PENDING_SIGNAL_BIT is set, and it clears that bit first. */ - -/* if (pypysig_occurred & PENDING_SIGNAL_BIT) */ + if (pypysig_occurred) { - int i; -/* pypysig_occurred &= ~PENDING_SIGNAL_BIT; */ - for (i=0; i Modified: pypy/branch/arm-backend/pypy/translator/c/test/test_genc.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/test/test_genc.py (original) +++ pypy/branch/arm-backend/pypy/translator/c/test/test_genc.py Tue Nov 23 10:12:47 2010 @@ -426,6 +426,7 @@ if py.test.config.option.view: t.view() assert ' BarStruct ' in t.driver.cbuilder.c_source_filename.read() + free(foo, flavor="raw") def test_recursive_llhelper(): from pypy.rpython.annlowlevel import llhelper @@ -473,7 +474,27 @@ return f(s) a_f = A(f, "f") a_g = A(g, "g") - t = lltype.malloc(STRUCT, flavor="raw") + t = lltype.malloc(STRUCT, flavor="raw", immortal=True) t.bar = llhelper(FTPTR, a_f.make_func()) fn = compile(chooser, [bool]) assert fn(True) + +def test_inhibit_tail_call(): + from pypy.rpython.lltypesystem import lltype + def foobar_fn(n): + return 42 + foobar_fn._dont_inline_ = True + def main(n): + return foobar_fn(n) + # + t = Translation(main, [int], backend="c") + t.rtype() + t.context._graphof(foobar_fn).inhibit_tail_call = True + t.source_c() + lines = t.driver.cbuilder.c_source_filename.readlines() + for i, line in enumerate(lines): + if '= pypy_g_foobar_fn' in line: + break + else: + assert 0, "the call was not found in the C source" + assert 'PYPY_INHIBIT_TAIL_CALL();' in lines[i+1] Modified: pypy/branch/arm-backend/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/branch/arm-backend/pypy/translator/c/test/test_lltyped.py Tue Nov 23 10:12:47 2010 @@ -1,6 +1,7 @@ import py from pypy.rpython.lltypesystem.lltype import * from pypy.translator.c.test import test_typed +from pypy.tool.sourcetools import func_with_new_name class TestLowLevelType(test_typed.CompilationTestCase): @@ -401,6 +402,7 @@ for i in range(n): p = malloc(S, flavor='raw', zero=True) if p.x != 0 or p.y != 0: + free(p, flavor='raw') return -1 p.x = i p.y = i @@ -418,14 +420,16 @@ def f(n): for length in range(n-1, -1, -1): p = malloc(S, length, flavor='raw', zero=True) - if p.x != 0: - return -1 - p.x = n - for j in range(length): - if p.y[j] != 0: - return -3 - p.y[j] = n^j - free(p, flavor='raw') + try: + if p.x != 0: + return -1 + p.x = n + for j in range(length): + if p.y[j] != 0: + return -3 + p.y[j] = n^j + finally: + free(p, flavor='raw') return 42 fn = self.getcompiled(f, [int]) @@ -652,10 +656,49 @@ fn = self.getcompiled(llf) fn() + def test_prebuilt_raw_arrays(self): + from pypy.rpython.lltypesystem import rffi, ll2ctypes + # + def make_test_function(cast, haslength, length): + a = malloc(A, length, flavor='raw', immortal=True) + # two cases: a zero-terminated array if length == 6 or 1030, + # a non-zero-terminated array if length == 557 or 1031 + for i in range(length): + a[i] = cast(256 - 5 + i) + def llf(): + for i in range(length): + if a[i] != cast(256 - 5 + i): + return False + if haslength and len(a) != length: + return False + return True + return func_with_new_name(llf, repr((A, haslength, length))) + # + testfns = [] + records = [] + for OF, cast in [(Void, lambda n: None), + (Char, lambda n: chr(n & 0xFF)), + (Signed, lambda n: n)]: + for A, haslength in [(rffi.CArray(OF), False), + (Array(OF), True)]: + for length in [0, 6, 557, 1030, 1031]: + testfns.append(make_test_function(cast, haslength, length)) + records.append((A, haslength, length)) + def llf(): + i = 0 + for fn in testfns: + if not fn(): + return i # returns the index of the failing function + i += 1 + return -42 + fn = self.getcompiled(llf) + res = fn() + assert res == -42, "failing function: %r" % (records[res],) + def test_prebuilt_ll2ctypes_array(self): from pypy.rpython.lltypesystem import rffi, ll2ctypes A = rffi.CArray(Char) - a = malloc(A, 6, flavor='raw') + a = malloc(A, 6, flavor='raw', immortal=True) a[0] = 'a' a[1] = 'b' a[2] = 'c' @@ -676,7 +719,7 @@ def test_ll2ctypes_array_from_c(self): from pypy.rpython.lltypesystem import rffi, ll2ctypes A = rffi.CArray(Char) - a = malloc(A, 6, flavor='raw') + a = malloc(A, 6, flavor='raw', immortal=True) a[0] = 'a' a[1] = 'b' a[2] = 'c' @@ -838,3 +881,17 @@ assert res == -98765432 res = fn(1) assert res == -9999999 + + def test_render_immortal(self): + A = FixedSizeArray(Signed, 1) + a1 = malloc(A, flavor='raw') + render_immortal(a1) + a1[0] = 42 + def llf(): + a2 = malloc(A, flavor='raw') + render_immortal(a2) + a2[0] = 3 + return a1[0] + a2[0] + fn = self.getcompiled(llf) + assert fn() == 45 + Modified: pypy/branch/arm-backend/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/test/test_newgc.py (original) +++ pypy/branch/arm-backend/pypy/translator/c/test/test_newgc.py Tue Nov 23 10:12:47 2010 @@ -624,13 +624,13 @@ os.unlink(self.filename) def define_callback_with_collect(cls): - from pypy.rlib.libffi import ffi_type_pointer, cast_type_to_ffitype,\ + from pypy.rlib.clibffi import ffi_type_pointer, cast_type_to_ffitype,\ CDLL, ffi_type_void, CallbackFuncPtr, ffi_type_sint from pypy.rpython.lltypesystem import rffi, ll2ctypes import gc ffi_size_t = cast_type_to_ffitype(rffi.SIZE_T) - from pypy.rlib.libffi import get_libc_name + from pypy.rlib.clibffi import get_libc_name def callback(ll_args, ll_res, stuff): gc.collect() @@ -1066,7 +1066,9 @@ filename_dump = str(udir.join('test_dump_rpy_heap')) def define_dump_rpy_heap(self): - U = lltype.GcStruct('U', ('x', lltype.Signed)) + U = lltype.GcForwardReference() + U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)), + ('x', lltype.Signed))) S = lltype.GcStruct('S', ('u', lltype.Ptr(U))) A = lltype.GcArray(lltype.Ptr(S)) filename = self.filename_dump @@ -1074,11 +1076,16 @@ def fn(): s = lltype.malloc(S) s.u = lltype.malloc(U) + s.u.next = lltype.malloc(U) + s.u.next.next = lltype.malloc(U) a = lltype.malloc(A, 1000) s2 = lltype.malloc(S) # fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666) rgc.dump_rpy_heap(fd) + keepalive_until_here(s2) + keepalive_until_here(s) + keepalive_until_here(a) os.close(fd) return 0 @@ -1087,8 +1094,43 @@ def test_dump_rpy_heap(self): self.run("dump_rpy_heap") assert os.path.exists(self.filename_dump) - assert os.path.getsize(self.filename_dump) > 0 # minimal test + assert os.path.getsize(self.filename_dump) > 64 + + filename_dump_typeids_z = str(udir.join('test_typeids_z')) + def define_write_typeids_z(self): + U = lltype.GcForwardReference() + U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)), + ('x', lltype.Signed))) + S = lltype.GcStruct('S', ('u', lltype.Ptr(U))) + A = lltype.GcArray(lltype.Ptr(S)) + filename = self.filename_dump_typeids_z + + def fn(): + s = lltype.malloc(S) + s.u = lltype.malloc(U) + s.u.next = lltype.malloc(U) + s.u.next.next = lltype.malloc(U) + a = lltype.malloc(A, 1000) + s2 = lltype.malloc(S) + # + p = rgc.get_typeids_z() + s = ''.join([p[i] for i in range(len(p))]) + fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666) + os.write(fd, s) + os.close(fd) + return 0 + + return fn + def test_write_typeids_z(self): + self.run("write_typeids_z") + f = open(self.filename_dump_typeids_z) + data_z = f.read() + f.close() + import zlib + data = zlib.decompress(data_z) + assert data.startswith('member0') + assert 'GcArray of * GcStruct S {' in data class TestSemiSpaceGC(TestUsingFramework, snippet.SemiSpaceGCTestDefines): gcpolicy = "semispace" @@ -1173,21 +1215,22 @@ b = 0 c = 0 for i in range(len(tb)): - if tb[i].count == 10: + if tb[i].count == 10: # the type of S a += 1 nr = i for i in range(len(tb)): - if tb[i].count == 3: + if tb[i].count == 3: # the type GcArray(Ptr(S)) b += 1 c += tb[i].links[nr] - # we don't count b here since there can be more singletons, + # b can be 1 or 2 here since _heap_stats() is free to return or + # ignore the three GcStructs that point to the GcArray(Ptr(S)). # important one is c, a is for check return c * 100 + b * 10 + a return f def test_gc_heap_stats(self): res = self.run("gc_heap_stats") - assert res == 3011 + assert res == 3011 or res == 3021 def definestr_string_builder(cls): def fn(_): Modified: pypy/branch/arm-backend/pypy/translator/c/test/test_standalone.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/test/test_standalone.py (original) +++ pypy/branch/arm-backend/pypy/translator/c/test/test_standalone.py Tue Nov 23 10:12:47 2010 @@ -16,11 +16,16 @@ class StandaloneTests(object): config = None - def compile(self, entry_point, debug=True, shared=False): + def compile(self, entry_point, debug=True, shared=False, + stackcheck=False): t = TranslationContext(self.config) t.buildannotator().build_types(entry_point, [s_list_of_strings]) t.buildrtyper().specialize() + if stackcheck: + from pypy.translator.transform import insert_ll_stackcheck + insert_ll_stackcheck(t) + t.config.translation.shared = shared cbuilder = CStandaloneBuilder(t, entry_point, t.config) @@ -363,12 +368,27 @@ assert not err assert path.check(file=1) data = path.read() - assert 'toplevel' in path.read() - assert 'mycat' not in path.read() - assert 'foo 2 bar 3' not in path.read() + assert 'toplevel' in data + assert 'mycat' not in data + assert 'foo 2 bar 3' not in data assert 'cat2' in data assert 'baz' in data assert 'bok' not in data + # check with PYPYLOG=myc,cat2:somefilename (includes mycat and cat2) + path = udir.join('test_debug_xxx_myc_cat2.log') + out, err = cbuilder.cmdexec("", err=True, + env={'PYPYLOG': 'myc,cat2:%s' % path}) + assert out.strip() == 'got:bcda.' + assert not err + assert path.check(file=1) + data = path.read() + assert 'toplevel' in data + assert '{mycat' in data + assert 'mycat}' in data + assert 'foo 2 bar 3' in data + assert 'cat2' in data + assert 'baz' in data + assert 'bok' in data # # finally, check compiling with logging disabled from pypy.config.pypyoption import get_pypy_config @@ -630,6 +650,22 @@ else: os.environ['CC'] = old_cc + def test_inhibit_tail_call(self): + # the point is to check that the f()->f() recursion stops + from pypy.rlib.rstackovf import StackOverflow + def f(n): + if n <= 0: + return 42 + return f(n+1) + def entry_point(argv): + try: + return f(1) + except StackOverflow: + print 'hi!' + return 0 + t, cbuilder = self.compile(entry_point, stackcheck=True) + out = cbuilder.cmdexec("") + assert out.strip() == "hi!" class TestMaemo(TestStandalone): def setup_class(cls): Modified: pypy/branch/arm-backend/pypy/translator/c/test/test_typed.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/c/test/test_typed.py (original) +++ pypy/branch/arm-backend/pypy/translator/c/test/test_typed.py Tue Nov 23 10:12:47 2010 @@ -1,3 +1,4 @@ +from __future__ import with_statement import autopath import sys import math @@ -823,3 +824,41 @@ while int(x + frac) >= -sys.maxint-1: x -= 1 assert f(x + frac) == -666 + + def test_context_manager(self): + state = [] + class C: + def __init__(self, name): + self.name = name + def __enter__(self): + state.append('acquire') + return self + def __exit__(self, typ, value, tb): + if typ is not None: + if value is None: + raise RuntimeError('test failed') + state.append('raised') + else: + if value is not None: + raise RuntimeError('test failed') + state.append('release') + + def func(n): + del state[:] + try: + with C('hello') as c: + state.append(c.name) + if n == 1: + raise ValueError + elif n == 2: + raise TypeError + except (ValueError, TypeError): + pass + return ', '.join(state) + f = self.getcompiled(func, [int]) + res = f(0) + assert res == 'acquire, hello, release' + res = f(1) + assert res == 'acquire, hello, raised, release' + res = f(2) + assert res == 'acquire, hello, raised, release' Modified: pypy/branch/arm-backend/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/branch/arm-backend/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/branch/arm-backend/pypy/translator/cli/src/pypylib.cs Tue Nov 23 10:12:47 2010 @@ -83,8 +83,12 @@ return Double.NegativeInfinity; else if (s == "nan") return Double.NaN; - else - return System.Convert.ToDouble(s); + else { + System.Globalization.NumberFormatInfo formatter; + formatter = new System.Globalization.NumberFormatInfo(); + formatter.NumberDecimalSeparator = "."; + return System.Convert.ToDouble(s, formatter); + } } } Modified: pypy/branch/arm-backend/pypy/translator/driver.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/driver.py (original) +++ pypy/branch/arm-backend/pypy/translator/driver.py Tue Nov 23 10:12:47 2010 @@ -426,6 +426,22 @@ [OOTYPE], "JIT compiler generation") + def task_jittest_lltype(self): + """ Run with the JIT on top of the llgraph backend + """ + # parent process loop: spawn a child, wait for the child to finish, + # print a message, and restart + from pypy.translator.goal import unixcheckpoint + unixcheckpoint.restartable_point(auto='run') + # load the module pypy/jit/tl/jittest.py, which you can hack at + # and restart without needing to restart the whole translation process + from pypy.jit.tl import jittest + jittest.jittest(self) + # + task_jittest_lltype = taskdef(task_jittest_lltype, + [RTYPE], + "test of the JIT on the llgraph backend") + def task_backendopt_lltype(self): """ Run all backend optimizations - lltype version """ @@ -433,7 +449,8 @@ backend_optimizations(self.translator) # task_backendopt_lltype = taskdef(task_backendopt_lltype, - [RTYPE, '??pyjitpl_lltype'], + [RTYPE, '??pyjitpl_lltype', + '??jittest_lltype'], "lltype back-end optimisations") BACKENDOPT = 'backendopt_lltype' Modified: pypy/branch/arm-backend/pypy/translator/goal/ann_override.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/goal/ann_override.py (original) +++ pypy/branch/arm-backend/pypy/translator/goal/ann_override.py Tue Nov 23 10:12:47 2010 @@ -27,21 +27,6 @@ pol.pypytypes = {} pol.single_space = single_space - #def override__wrap_exception_cls(pol, space, x): - # import pypy.objspace.std.typeobject as typeobject - # clsdef = getbookkeeper().getuniqueclassdef(typeobject.W_TypeObject) - # return annmodel.SomeInstance(clsdef, can_be_None=True) - # - #def override__fake_object(pol, space, x): - # from pypy.interpreter import typedef - # clsdef = getbookkeeper().getuniqueclassdef(typedef.W_Root) - # return annmodel.SomeInstance(clsdef) - # - #def override__cpy_compile(pol, self, source, filename, mode, flags): - # from pypy.interpreter import pycode - # clsdef = getbookkeeper().getuniqueclassdef(pycode.PyCode) - # return annmodel.SomeInstance(clsdef) - def specialize__wrap(pol, funcdesc, args_s): from pypy.interpreter.baseobjspace import Wrappable from pypy.annotation.classdef import ClassDef Modified: pypy/branch/arm-backend/pypy/translator/goal/app_main.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/goal/app_main.py (original) +++ pypy/branch/arm-backend/pypy/translator/goal/app_main.py Tue Nov 23 10:12:47 2010 @@ -255,7 +255,7 @@ break elif arg == '-u': unbuffered = True - elif arg == '-O': + elif arg == '-O' or arg == '-OO': pass elif arg == '--version' or arg == '-V': print "Python", sys.version @@ -326,10 +326,6 @@ except: print >> sys.stderr, "'import site' failed" - # update sys.path *after* loading site.py, in case there is a - # "site.py" file in the script's directory. - sys.path.insert(0, '') - if warnoptions: sys.warnoptions.append(warnoptions) from warnings import _processoptions @@ -366,6 +362,9 @@ try: if run_command: # handle the "-c" command + # Put '' on sys.path + sys.path.insert(0, '') + def run_it(): exec cmd in mainmodule.__dict__ success = run_toplevel(run_it) @@ -378,6 +377,13 @@ elif run_stdin: # handle the case where no command/filename/module is specified # on the command-line. + + # update sys.path *after* loading site.py, in case there is a + # "site.py" file in the script's directory. Only run this if we're + # executing the interactive prompt, if we're running a script we + # put it's directory on sys.path + sys.path.insert(0, '') + if go_interactive or sys.stdin.isatty(): # If stdin is a tty or if "-i" is specified, we print # a banner and run $PYTHONSTARTUP. @@ -385,9 +391,12 @@ python_startup = os.getenv('PYTHONSTARTUP') if python_startup: try: - startup = open(python_startup).read() - except IOError: - pass + f = open(python_startup) + startup = f.read() + f.close() + except IOError, e: + print >> sys.stderr, "Could not open PYTHONSTARTUP" + print >> sys.stderr, "IOError:", e else: def run_it(): co_python_startup = compile(startup, Modified: pypy/branch/arm-backend/pypy/translator/goal/targetpypystandalone.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/goal/targetpypystandalone.py (original) +++ pypy/branch/arm-backend/pypy/translator/goal/targetpypystandalone.py Tue Nov 23 10:12:47 2010 @@ -28,9 +28,14 @@ w_call_finish_gateway = space.wrap(gateway.interp2app(call_finish)) w_call_startup_gateway = space.wrap(gateway.interp2app(call_startup)) w_os = setup_nanos(space) + withjit = space.config.objspace.usemodules.pypyjit def entry_point(argv): space.timer.start("Entrypoint") + if withjit: + from pypy.jit.backend.hlinfo import highleveljitinfo + highleveljitinfo.sys_executable = argv[0] + #debug("entry point starting") #for arg in argv: # debug(" argv -> " + arg) @@ -150,6 +155,9 @@ if config.objspace.allworkingmodules: from pypy.config.pypyoption import enable_allworkingmodules enable_allworkingmodules(config) + if config.objspace.translationmodules: + from pypy.config.pypyoption import enable_translationmodules + enable_translationmodules(config) if config.translation.type_system == 'ootype': config.objspace.usemodules.suggest(rbench=True) Modified: pypy/branch/arm-backend/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/branch/arm-backend/pypy/translator/goal/test2/test_app_main.py Tue Nov 23 10:12:47 2010 @@ -95,6 +95,11 @@ child.expect('>>> ') child.sendline('__name__') child.expect("'__main__'") + child.expect('>>> ') + child.sendline('import sys') + child.expect('>>> ') + child.sendline("'' in sys.path") + child.expect("True") def test_run_script(self): child = self.spawn([demo_script]) @@ -463,8 +468,10 @@ yield finally: old_cwd.chdir() - os.putenv('PYTHONPATH', old_pythonpath) - + # Can't call putenv with a None argument. + if old_pythonpath is not None: + os.putenv('PYTHONPATH', old_pythonpath) + tmpdir.join('site.py').write('print "SHOULD NOT RUN"') runme_py = tmpdir.join('runme.py') runme_py.write('print "some text"') @@ -485,9 +492,12 @@ with chdir_and_unset_pythonpath(tmpdir): data = self.run(cmdline2, python_flags='-S') - assert data.startswith("some new text\n") assert repr(str(tmpdir.join('otherpath'))) in data + assert "''" not in data + + data = self.run('-c "import sys; print sys.path"') + assert data.startswith("[''") class AppTestAppMain: @@ -524,7 +534,8 @@ newpath = app_main.get_library_path('/tmp/pypy-c') # stdlib not found assert newpath == sys.path newpath = app_main.get_library_path(self.fake_exe) - assert newpath == self.expected_path + # we get at least 'expected_path', and maybe more (e.g.plat-linux2) + assert newpath[:len(self.expected_path)] == self.expected_path finally: sys.path.pop() @@ -537,7 +548,9 @@ app_main.os = os pypy_c = os.path.join(self.trunkdir, 'pypy', 'translator', 'goal', 'pypy-c') newpath = app_main.get_library_path(pypy_c) - assert len(newpath) == 3 + # we get at least lib_pypy, lib-python/modified-X.Y.Z, + # lib-python/X.Y.Z, and maybe more (e.g. plat-linux2) + assert len(newpath) >= 3 for p in newpath: assert p.startswith(self.trunkdir) finally: Modified: pypy/branch/arm-backend/pypy/translator/goal/translate.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/goal/translate.py (original) +++ pypy/branch/arm-backend/pypy/translator/goal/translate.py Tue Nov 23 10:12:47 2010 @@ -27,6 +27,7 @@ ("annotate", "do type inference", "-a --annotate", ""), ("rtype", "do rtyping", "-t --rtype", ""), ("pyjitpl", "JIT generation step", "--pyjitpl", ""), + ("jittest", "JIT test with llgraph backend", "--jittest", ""), ("backendopt", "do backend optimizations", "--backendopt", ""), ("source", "create source", "-s --source", ""), ("compile", "compile", "-c --compile", " (default goal)"), Modified: pypy/branch/arm-backend/pypy/translator/jvm/test/test_class.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/jvm/test/test_class.py (original) +++ pypy/branch/arm-backend/pypy/translator/jvm/test/test_class.py Tue Nov 23 10:12:47 2010 @@ -1,6 +1,6 @@ import py from pypy.translator.jvm.test.runtest import JvmTest -from pypy.translator.oosupport.test_template.class_ import BaseTestClass, BaseTestSpecialcase +from pypy.translator.oosupport.test_template.class_ import BaseTestClass class TestJvmClass(JvmTest, BaseTestClass): def test_overridden_classattr_as_defaults(self): @@ -26,6 +26,3 @@ def test_specialize_methods(self): py.test.skip('ABSTRACT METHOD FIX: RE-TEST AFTER MERGE') - -class TestJvmSpecialCase(JvmTest, BaseTestSpecialcase): - pass Modified: pypy/branch/arm-backend/pypy/translator/oosupport/test_template/class_.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/oosupport/test_template/class_.py (original) +++ pypy/branch/arm-backend/pypy/translator/oosupport/test_template/class_.py Tue Nov 23 10:12:47 2010 @@ -1,6 +1,5 @@ import py from pypy.rpython.test import test_rclass -from pypy.rpython.test.test_rspecialcase import BaseTestRspecialcase class BaseTestClass(test_rclass.TestOOtype): def test_abstract_method(self): @@ -66,6 +65,3 @@ def test_cast_object_mix_null(self): py.test.skip('cannot return ootype.NULL from translated functions') - -class BaseTestSpecialcase(BaseTestRspecialcase): - pass Modified: pypy/branch/arm-backend/pypy/translator/platform/darwin.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/platform/darwin.py (original) +++ pypy/branch/arm-backend/pypy/translator/platform/darwin.py Tue Nov 23 10:12:47 2010 @@ -14,7 +14,10 @@ def __init__(self, cc=None): if cc is None: - cc = 'gcc' + try: + cc = os.environ['CC'] + except KeyError: + cc = 'gcc' self.cc = cc def _args_for_shared(self, args): Modified: pypy/branch/arm-backend/pypy/translator/platform/linux.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/platform/linux.py (original) +++ pypy/branch/arm-backend/pypy/translator/platform/linux.py Tue Nov 23 10:12:47 2010 @@ -27,6 +27,8 @@ class Linux(BaseLinux): + shared_only = () # it seems that on 32-bit linux, compiling with -fPIC + # gives assembler that asmgcc is not happy about. def library_dirs_for_libffi_a(self): # places where we need to look for libffi.a return self.library_dirs_for_libffi() + ['/usr/lib'] Modified: pypy/branch/arm-backend/pypy/translator/platform/posix.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/platform/posix.py (original) +++ pypy/branch/arm-backend/pypy/translator/platform/posix.py Tue Nov 23 10:12:47 2010 @@ -104,12 +104,10 @@ else: target_name = exe_name.basename - cflags = self.cflags - if sys.maxint > 2147483647: # XXX XXX XXX sort this out - if shared: - cflags = self.cflags + self.shared_only - else: - cflags = self.cflags + self.standalone_only + if shared: + cflags = self.cflags + self.shared_only + else: + cflags = self.cflags + self.standalone_only m = GnuMakefile(path) m.exe_name = exe_name Modified: pypy/branch/arm-backend/pypy/translator/transform.py ============================================================================== --- pypy/branch/arm-backend/pypy/translator/transform.py (original) +++ pypy/branch/arm-backend/pypy/translator/transform.py Tue Nov 23 10:12:47 2010 @@ -221,15 +221,19 @@ stack_check_ptr_const = Constant(stack_check_ptr, lltype.typeOf(stack_check_ptr)) edges = set() insert_in = set() + block2graph = {} for caller in translator.graphs: for block, callee in find_calls_from(translator, caller): if getattr(getattr(callee, 'func', None), 'insert_stack_check_here', False): insert_in.add(callee.startblock) + block2graph[callee.startblock] = callee continue if block is not caller.startblock: edges.add((caller.startblock, block)) + block2graph[caller.startblock] = caller edges.add((block, callee.startblock)) + block2graph[block] = caller edgelist = [Edge(block1, block2) for (block1, block2) in edges] edgedict = make_edge_dict(edgelist) @@ -241,6 +245,10 @@ v.concretetype = lltype.Void unwind_op = SpaceOperation('direct_call', [stack_check_ptr_const], v) block.operations.insert(0, unwind_op) + # prevents cycles of tail calls from occurring -- such cycles would + # not consume any stack, so would turn into potentially infinite loops + graph = block2graph[block] + graph.inhibit_tail_call = True return len(insert_in) From david at codespeak.net Tue Nov 23 12:10:24 2010 From: david at codespeak.net (david at codespeak.net) Date: Tue, 23 Nov 2010 12:10:24 +0100 (CET) Subject: [pypy-svn] r79387 - pypy/branch/arm-backend/pypy/jit/backend/llsupport Message-ID: <20101123111024.F30F6282B90@codespeak.net> Author: david Date: Tue Nov 23 12:10:22 2010 New Revision: 79387 Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py Log: Forgot import Modified: pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/llsupport/regalloc.py Tue Nov 23 12:10:22 2010 @@ -1,4 +1,4 @@ - +import os from pypy.jit.metainterp.history import Const, Box from pypy.rlib.objectmodel import we_are_translated From david at codespeak.net Tue Nov 23 12:18:00 2010 From: david at codespeak.net (david at codespeak.net) Date: Tue, 23 Nov 2010 12:18:00 +0100 (CET) Subject: [pypy-svn] r79388 - pypy/branch/arm-backend/pypy/jit/backend/arm/test Message-ID: <20101123111800.C7E64282BD4@codespeak.net> Author: david Date: Tue Nov 23 12:17:52 2010 New Revision: 79388 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py Log: refactor test_result_is_spilled Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py Tue Nov 23 12:17:52 2010 @@ -27,17 +27,24 @@ def test_result_is_spilled(self): cpu = self.cpu inp = [BoxInt(i) for i in range(1, 15)] - out = list(inp) - out.reverse() + out = [BoxInt(i) for i in range(1, 15)] + #out.reverse() looptoken = LoopToken() operations = [ - ResOperation(rop.INT_ADD, [inp[0], inp[1]], inp[2]), - ResOperation(rop.INT_SUB, [inp[1], ConstInt(1)], inp[4]), - ResOperation(rop.INT_ADD, [inp[4], ConstInt(0)], inp[3]), - ResOperation(rop.INT_ADD, [inp[5], inp[6]], inp[7]), - ResOperation(rop.INT_SUB, [inp[8], ConstInt(1)], inp[9]), - ResOperation(rop.INT_ADD, [inp[10], ConstInt(1)], inp[3]), - ResOperation(rop.INT_ADD, [inp[11], inp[12]], inp[13]), + ResOperation(rop.INT_ADD, [inp[0] , inp[1]], out[0]), + ResOperation(rop.INT_ADD, [inp[2] , inp[3]], out[1]), + ResOperation(rop.INT_ADD, [inp[4] , inp[5]], out[2]), + ResOperation(rop.INT_ADD, [inp[6] , inp[7]], out[3]), + ResOperation(rop.INT_ADD, [inp[8] , inp[9]], out[4]), + ResOperation(rop.INT_ADD, [inp[10], inp[11]], out[5]), + ResOperation(rop.INT_ADD, [inp[12], inp[13]], out[6]), + ResOperation(rop.INT_ADD, [inp[0] , inp[1]], out[7]), + ResOperation(rop.INT_ADD, [inp[2] , inp[3]], out[8]), + ResOperation(rop.INT_ADD, [inp[4] , inp[5]], out[9]), + ResOperation(rop.INT_ADD, [inp[6] , inp[7]], out[10]), + ResOperation(rop.INT_ADD, [inp[8] , inp[9]], out[11]), + ResOperation(rop.INT_ADD, [inp[10], inp[11]], out[12]), + ResOperation(rop.INT_ADD, [inp[12], inp[13]], out[13]), ResOperation(rop.FINISH, out, None, descr=BasicFailDescr(1)), ] cpu.compile_loop(inp, operations, looptoken) @@ -45,5 +52,5 @@ self.cpu.set_future_value_int(i-1, i) res = self.cpu.execute_token(looptoken) output = [self.cpu.get_latest_value_int(i-1) for i in range(1, 15)] - expected = [25,13,12,11,8,9,13,7,6,1,12,3,2,1] + expected = [3, 7, 11, 15, 19, 23, 27, 3, 7, 11, 15, 19, 23, 27] assert output == expected From antocuni at codespeak.net Tue Nov 23 13:26:41 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 23 Nov 2010 13:26:41 +0100 (CET) Subject: [pypy-svn] r79389 - in pypy/branch/jitypes2: . lib-python/modified-2.5.2/distutils lib_pypy/_ctypes lib_pypy/ctypes_config_cache/test pypy pypy/annotation pypy/config pypy/doc pypy/doc/config pypy/interpreter pypy/interpreter/test pypy/jit/backend/llsupport pypy/jit/backend/llsupport/test pypy/jit/backend/test pypy/jit/backend/x86 pypy/jit/codewriter pypy/jit/codewriter/test pypy/jit/metainterp pypy/jit/metainterp/optimizeopt pypy/jit/metainterp/test pypy/jit/tool pypy/jit/tool/test pypy/module/__pypy__ pypy/module/_rawffi/test pypy/module/_stackless/test pypy/module/array pypy/module/array/benchmark pypy/module/array/test pypy/module/cpyext pypy/module/cpyext/test pypy/module/gc pypy/module/gc/test pypy/module/imp pypy/module/imp/test pypy/module/posix pypy/module/pypyjit pypy/module/pypyjit/test pypy/module/rctime pypy/module/sys pypy/module/sys/test pypy/objspace/flow pypy/objspace/std pypy/objspace/std/test pypy/rlib pypy/rlib/test pypy/rpython pypy/rpython/lltypesystem pypy/rpython/memory pypy/rpython/memory/gc pypy/rpython/memory/gc/test pypy/rpython/memory/gctransform pypy/rpython/memory/test pypy/rpython/test pypy/tool pypy/tool/release pypy/tool/release/test pypy/translator/c pypy/translator/c/gcc pypy/translator/c/gcc/test pypy/translator/c/src pypy/translator/c/test pypy/translator/cli/src pypy/translator/goal pypy/translator/goal/test2 pypy/translator/platform Message-ID: <20101123122641.C3B0C282BD4@codespeak.net> Author: antocuni Date: Tue Nov 23 13:26:19 2010 New Revision: 79389 Added: pypy/branch/jitypes2/pypy/doc/config/objspace.soabi.txt - copied unchanged from r79388, pypy/trunk/pypy/doc/config/objspace.soabi.txt pypy/branch/jitypes2/pypy/doc/config/objspace.translationmodules.txt - copied unchanged from r79388, pypy/trunk/pypy/doc/config/objspace.translationmodules.txt pypy/branch/jitypes2/pypy/doc/release-1.4.0beta.txt - copied unchanged from r79388, pypy/trunk/pypy/doc/release-1.4.0beta.txt pypy/branch/jitypes2/pypy/jit/tool/log-template.gnumeric - copied unchanged from r79388, pypy/trunk/pypy/jit/tool/log-template.gnumeric pypy/branch/jitypes2/pypy/jit/tool/log2gnumeric.py - copied unchanged from r79388, pypy/trunk/pypy/jit/tool/log2gnumeric.py pypy/branch/jitypes2/pypy/jit/tool/loopcounter.py - copied unchanged from r79388, pypy/trunk/pypy/jit/tool/loopcounter.py pypy/branch/jitypes2/pypy/jit/tool/test/test_loopcounter.py - copied unchanged from r79388, pypy/trunk/pypy/jit/tool/test/test_loopcounter.py pypy/branch/jitypes2/pypy/jit/tool/test/test_oparser.py - copied unchanged from r79388, pypy/trunk/pypy/jit/tool/test/test_oparser.py pypy/branch/jitypes2/pypy/module/_stackless/test/conftest.py - copied unchanged from r79388, pypy/trunk/pypy/module/_stackless/test/conftest.py pypy/branch/jitypes2/pypy/tool/gcdump.py - copied unchanged from r79388, pypy/trunk/pypy/tool/gcdump.py pypy/branch/jitypes2/pypy/translator/c/src/asm_gcc_x86_64.h - copied unchanged from r79388, pypy/trunk/pypy/translator/c/src/asm_gcc_x86_64.h Removed: pypy/branch/jitypes2/pypy/jit/metainterp/test/test_oparser.py Modified: pypy/branch/jitypes2/ (props changed) pypy/branch/jitypes2/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py pypy/branch/jitypes2/lib_pypy/_ctypes/function.py pypy/branch/jitypes2/lib_pypy/ctypes_config_cache/test/test_cache.py pypy/branch/jitypes2/pypy/ (props changed) pypy/branch/jitypes2/pypy/annotation/bookkeeper.py pypy/branch/jitypes2/pypy/annotation/builtin.py pypy/branch/jitypes2/pypy/config/pypyoption.py pypy/branch/jitypes2/pypy/config/translationoption.py pypy/branch/jitypes2/pypy/conftest.py pypy/branch/jitypes2/pypy/doc/release-1.4.0.txt pypy/branch/jitypes2/pypy/interpreter/argument.py pypy/branch/jitypes2/pypy/interpreter/test/test_argument.py pypy/branch/jitypes2/pypy/jit/backend/llsupport/descr.py pypy/branch/jitypes2/pypy/jit/backend/llsupport/gc.py pypy/branch/jitypes2/pypy/jit/backend/llsupport/test/test_descr.py pypy/branch/jitypes2/pypy/jit/backend/llsupport/test/test_gc.py pypy/branch/jitypes2/pypy/jit/backend/test/runner_test.py pypy/branch/jitypes2/pypy/jit/backend/x86/assembler.py pypy/branch/jitypes2/pypy/jit/codewriter/assembler.py pypy/branch/jitypes2/pypy/jit/codewriter/call.py pypy/branch/jitypes2/pypy/jit/codewriter/codewriter.py pypy/branch/jitypes2/pypy/jit/codewriter/effectinfo.py pypy/branch/jitypes2/pypy/jit/codewriter/jtransform.py pypy/branch/jitypes2/pypy/jit/codewriter/test/test_jtransform.py pypy/branch/jitypes2/pypy/jit/metainterp/compile.py pypy/branch/jitypes2/pypy/jit/metainterp/optimizeopt/optimizer.py (contents, props changed) pypy/branch/jitypes2/pypy/jit/metainterp/optimizeopt/string.py pypy/branch/jitypes2/pypy/jit/metainterp/pyjitpl.py pypy/branch/jitypes2/pypy/jit/metainterp/resoperation.py pypy/branch/jitypes2/pypy/jit/metainterp/resume.py pypy/branch/jitypes2/pypy/jit/metainterp/test/test_compile.py pypy/branch/jitypes2/pypy/jit/metainterp/test/test_exception.py pypy/branch/jitypes2/pypy/jit/metainterp/test/test_optimizeopt.py pypy/branch/jitypes2/pypy/jit/metainterp/test/test_recursive.py pypy/branch/jitypes2/pypy/jit/metainterp/test/test_resoperation.py pypy/branch/jitypes2/pypy/jit/metainterp/test/test_resume.py pypy/branch/jitypes2/pypy/jit/metainterp/test/test_virtualref.py pypy/branch/jitypes2/pypy/jit/metainterp/test/test_ztranslation.py pypy/branch/jitypes2/pypy/jit/metainterp/virtualizable.py pypy/branch/jitypes2/pypy/jit/metainterp/warmspot.py pypy/branch/jitypes2/pypy/jit/tool/pypytrace-mode.el pypy/branch/jitypes2/pypy/module/__pypy__/__init__.py pypy/branch/jitypes2/pypy/module/__pypy__/interp_magic.py pypy/branch/jitypes2/pypy/module/_rawffi/test/test__rawffi.py pypy/branch/jitypes2/pypy/module/_rawffi/test/test_nested.py pypy/branch/jitypes2/pypy/module/array/benchmark/Makefile (props changed) pypy/branch/jitypes2/pypy/module/array/benchmark/intimg.c (props changed) pypy/branch/jitypes2/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/branch/jitypes2/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/branch/jitypes2/pypy/module/array/benchmark/loop.c (props changed) pypy/branch/jitypes2/pypy/module/array/benchmark/sum.c (props changed) pypy/branch/jitypes2/pypy/module/array/benchmark/sumtst.c (props changed) pypy/branch/jitypes2/pypy/module/array/benchmark/sumtst.py (props changed) pypy/branch/jitypes2/pypy/module/array/interp_array.py pypy/branch/jitypes2/pypy/module/array/test/test_array_old.py (props changed) pypy/branch/jitypes2/pypy/module/cpyext/api.py pypy/branch/jitypes2/pypy/module/cpyext/cdatetime.py pypy/branch/jitypes2/pypy/module/cpyext/presetup.py pypy/branch/jitypes2/pypy/module/cpyext/pyobject.py pypy/branch/jitypes2/pypy/module/cpyext/test/test_borrow.py pypy/branch/jitypes2/pypy/module/cpyext/test/test_cpyext.py pypy/branch/jitypes2/pypy/module/cpyext/typeobject.py pypy/branch/jitypes2/pypy/module/gc/__init__.py pypy/branch/jitypes2/pypy/module/gc/app_referents.py pypy/branch/jitypes2/pypy/module/gc/interp_gc.py pypy/branch/jitypes2/pypy/module/gc/referents.py pypy/branch/jitypes2/pypy/module/gc/test/test_gc.py pypy/branch/jitypes2/pypy/module/imp/importing.py pypy/branch/jitypes2/pypy/module/imp/interp_imp.py pypy/branch/jitypes2/pypy/module/imp/test/test_app.py pypy/branch/jitypes2/pypy/module/imp/test/test_import.py pypy/branch/jitypes2/pypy/module/posix/interp_posix.py pypy/branch/jitypes2/pypy/module/pypyjit/__init__.py pypy/branch/jitypes2/pypy/module/pypyjit/test/test_pypy_c.py pypy/branch/jitypes2/pypy/module/rctime/interp_time.py pypy/branch/jitypes2/pypy/module/sys/state.py pypy/branch/jitypes2/pypy/module/sys/test/test_initialpath.py pypy/branch/jitypes2/pypy/objspace/flow/model.py pypy/branch/jitypes2/pypy/objspace/std/callmethod.py pypy/branch/jitypes2/pypy/objspace/std/mapdict.py pypy/branch/jitypes2/pypy/objspace/std/test/test_dictmultiobject.py pypy/branch/jitypes2/pypy/objspace/std/test/test_mapdict.py pypy/branch/jitypes2/pypy/rlib/clibffi.py pypy/branch/jitypes2/pypy/rlib/rerased.py (props changed) pypy/branch/jitypes2/pypy/rlib/rgc.py pypy/branch/jitypes2/pypy/rlib/test/test_rerased.py (props changed) pypy/branch/jitypes2/pypy/rpython/annlowlevel.py pypy/branch/jitypes2/pypy/rpython/llinterp.py pypy/branch/jitypes2/pypy/rpython/lltypesystem/llarena.py pypy/branch/jitypes2/pypy/rpython/lltypesystem/llmemory.py pypy/branch/jitypes2/pypy/rpython/lltypesystem/lloperation.py pypy/branch/jitypes2/pypy/rpython/lltypesystem/lltype.py pypy/branch/jitypes2/pypy/rpython/lltypesystem/rffi.py pypy/branch/jitypes2/pypy/rpython/memory/gc/base.py pypy/branch/jitypes2/pypy/rpython/memory/gc/generation.py pypy/branch/jitypes2/pypy/rpython/memory/gc/inspector.py pypy/branch/jitypes2/pypy/rpython/memory/gc/minimark.py pypy/branch/jitypes2/pypy/rpython/memory/gc/semispace.py pypy/branch/jitypes2/pypy/rpython/memory/gc/test/test_minimark.py pypy/branch/jitypes2/pypy/rpython/memory/gctransform/framework.py pypy/branch/jitypes2/pypy/rpython/memory/support.py pypy/branch/jitypes2/pypy/rpython/memory/test/test_gc.py pypy/branch/jitypes2/pypy/rpython/rbuiltin.py pypy/branch/jitypes2/pypy/rpython/test/test_rpbc.py pypy/branch/jitypes2/pypy/tool/release/force-builds.py pypy/branch/jitypes2/pypy/tool/release/make_release.py pypy/branch/jitypes2/pypy/tool/release/package.py pypy/branch/jitypes2/pypy/tool/release/test/test_package.py pypy/branch/jitypes2/pypy/tool/terminal.py pypy/branch/jitypes2/pypy/translator/c/gcc/test/test_trackgcroot.py pypy/branch/jitypes2/pypy/translator/c/gcc/trackgcroot.py pypy/branch/jitypes2/pypy/translator/c/genc.py pypy/branch/jitypes2/pypy/translator/c/node.py pypy/branch/jitypes2/pypy/translator/c/src/asm_gcc_x86.h pypy/branch/jitypes2/pypy/translator/c/src/debug_print.h pypy/branch/jitypes2/pypy/translator/c/src/g_include.h pypy/branch/jitypes2/pypy/translator/c/test/test_lltyped.py pypy/branch/jitypes2/pypy/translator/c/test/test_newgc.py pypy/branch/jitypes2/pypy/translator/c/test/test_standalone.py pypy/branch/jitypes2/pypy/translator/cli/src/pypylib.cs pypy/branch/jitypes2/pypy/translator/goal/app_main.py pypy/branch/jitypes2/pypy/translator/goal/targetpypystandalone.py pypy/branch/jitypes2/pypy/translator/goal/test2/test_app_main.py pypy/branch/jitypes2/pypy/translator/platform/linux.py pypy/branch/jitypes2/pypy/translator/platform/posix.py Log: merge from trunk: svn merge svn+ssh://codespeak.net/svn/pypy/trunk -r78974:HEAD Modified: pypy/branch/jitypes2/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py ============================================================================== --- pypy/branch/jitypes2/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py (original) +++ pypy/branch/jitypes2/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py Tue Nov 23 13:26:19 2010 @@ -3,6 +3,7 @@ import sys import os +import imp from distutils.errors import DistutilsPlatformError @@ -47,11 +48,17 @@ _config_vars = None +def _get_so_extension(): + for ext, mod, typ in imp.get_suffixes(): + if typ == imp.C_EXTENSION: + return ext + def _init_posix(): """Initialize the module as appropriate for POSIX systems.""" g = {} g['EXE'] = "" - g['SO'] = ".so" + g['SO'] = _get_so_extension() or ".so" + g['SOABI'] = g['SO'].rsplit('.')[0] global _config_vars _config_vars = g @@ -61,7 +68,8 @@ """Initialize the module as appropriate for NT""" g = {} g['EXE'] = ".exe" - g['SO'] = ".pyd" + g['SO'] = _get_so_extension() or ".pyd" + g['SOABI'] = g['SO'].rsplit('.')[0] global _config_vars _config_vars = g Modified: pypy/branch/jitypes2/lib_pypy/_ctypes/function.py ============================================================================== --- pypy/branch/jitypes2/lib_pypy/_ctypes/function.py (original) +++ pypy/branch/jitypes2/lib_pypy/_ctypes/function.py Tue Nov 23 13:26:19 2010 @@ -30,6 +30,7 @@ from_address = cdata_from_address + class CFuncPtr(_CData): __metaclass__ = CFuncPtrType @@ -174,8 +175,8 @@ restype = self._restype_ funcptr = self._getfuncptr(argtypes, restype, thisarg) - return funcptr(*args) - #return funcptr(args[0]) + #return funcptr(*args) + return funcptr(args[0]) #resbuffer = funcptr(*[arg._buffer for _, arg in argsandobjs]) #return self._build_result(restype, resbuffer, argtypes, argsandobjs) @@ -217,6 +218,8 @@ ffi_argtypes = [self._shape_to_ffi_type(shape) for shape in argshapes] ffi_restype = self._shape_to_ffi_type(resshape) self._ptr = cdll.getfunc(self.name, ffi_argtypes, ffi_restype) + if len(ffi_argtypes) == 1: + self.__class__ = _FuncPtr_1_arg # black magic return self._ptr except AttributeError: if self._flags_ & _rawffi.FUNCFLAG_CDECL: @@ -393,3 +396,26 @@ self._ptr.free() self._ptr = None self._needs_free = False + + +# hack hack + +#import ctypes +class _FuncPtr_1_arg(CFuncPtr): + + def _rollback(self, *args): + self.__class__ = CFuncPtr + return self(*args) + + def __call__(self, arg0): + if self.callable is not None or self._com_index: + return self._rollback(arg0) + + #argtypes, argsandobjs = self._wrap_args(argtypes, args) + + argtypes = self._argtypes_ + restype = self._restype_ + thisarg = None + funcptr = self._getfuncptr(argtypes, restype, thisarg) + return funcptr(arg0) + Modified: pypy/branch/jitypes2/lib_pypy/ctypes_config_cache/test/test_cache.py ============================================================================== --- pypy/branch/jitypes2/lib_pypy/ctypes_config_cache/test/test_cache.py (original) +++ pypy/branch/jitypes2/lib_pypy/ctypes_config_cache/test/test_cache.py Tue Nov 23 13:26:19 2010 @@ -7,9 +7,8 @@ def run(filename, outputname): filepath = dirpath.join(filename) - tmpdir2 = udir.ensure('testcache-' + filename, dir=True) - tmpdir = tmpdir2.ensure('ctypes_config_cache', dir=True) - tmpdir.join('__init__.py').write('\n') + tmpdir = udir.ensure('testcache-' + os.path.splitext(filename)[0], + dir=True) tmpdir.join('dumpcache.py').write(dirpath.join('dumpcache.py').read()) path = sys.path[:] try: @@ -23,14 +22,12 @@ assert outputpath.check(exists=1) modname = os.path.splitext(outputname)[0] try: - sys.path.insert(0, str(tmpdir2)) + sys.path.insert(0, str(tmpdir)) d = {} - exec "from ctypes_config_cache import %s" % modname in d - mod = d[modname] + execfile(str(outputpath), d) finally: sys.path[:] = path - sys.modules.pop('ctypes_config_cache', None) - return mod.__dict__ + return d def test_syslog(): Modified: pypy/branch/jitypes2/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/branch/jitypes2/pypy/annotation/bookkeeper.py (original) +++ pypy/branch/jitypes2/pypy/annotation/bookkeeper.py Tue Nov 23 13:26:19 2010 @@ -87,13 +87,6 @@ else: return obj.knowntype.__name__ - def consider_tuple_iter(self, tup): - ctxt = "[%s]" % sys._getframe(4).f_code.co_name - if tup.is_constant(): - return ctxt, tup.const - else: - return ctxt, tuple([self.typerepr(x) for x in tup.items]) - def consider_tuple_random_getitem(self, tup): return tuple([self.typerepr(x) for x in tup.items]) Modified: pypy/branch/jitypes2/pypy/annotation/builtin.py ============================================================================== --- pypy/branch/jitypes2/pypy/annotation/builtin.py (original) +++ pypy/branch/jitypes2/pypy/annotation/builtin.py Tue Nov 23 13:26:19 2010 @@ -453,6 +453,9 @@ #p = lltype.malloc(T, flavor=s_flavor.const) #lltype.free(p, flavor=s_flavor.const) +def render_immortal(s_p, s_track_allocation=None): + assert s_track_allocation is None or s_track_allocation.is_constant() + def typeOf(s_val): lltype = annotation_to_lltype(s_val, info="in typeOf(): ") return immutablevalue(lltype) @@ -520,6 +523,7 @@ BUILTIN_ANALYZERS[lltype.malloc] = malloc BUILTIN_ANALYZERS[lltype.free] = free +BUILTIN_ANALYZERS[lltype.render_immortal] = render_immortal BUILTIN_ANALYZERS[lltype.typeOf] = typeOf BUILTIN_ANALYZERS[lltype.cast_primitive] = cast_primitive BUILTIN_ANALYZERS[lltype.nullptr] = nullptr Modified: pypy/branch/jitypes2/pypy/config/pypyoption.py ============================================================================== --- pypy/branch/jitypes2/pypy/config/pypyoption.py (original) +++ pypy/branch/jitypes2/pypy/config/pypyoption.py Tue Nov 23 13:26:19 2010 @@ -34,6 +34,11 @@ "_bisect", "_ffi"] )) +translation_modules = default_modules.copy() +translation_modules.update(dict.fromkeys( + ["fcntl", "rctime", "select", "signal", "_rawffi", "zlib", + "struct", "md5", "cStringIO", "array"])) + working_oo_modules = default_modules.copy() working_oo_modules.update(dict.fromkeys( ["md5", "sha", "cStringIO", "itertools"] @@ -149,6 +154,12 @@ cmdline="--allworkingmodules", negation=True), + BoolOption("translationmodules", + "use only those modules that are needed to run translate.py on pypy", + default=False, + cmdline="--translationmodules", + suggests=[("objspace.allworkingmodules", False)]), + BoolOption("geninterp", "specify whether geninterp should be used", cmdline=None, default=True), @@ -164,6 +175,11 @@ default=False, requires=[("objspace.usepycfiles", True)]), + StrOption("soabi", + "Tag to differentiate extension modules built for different Python interpreters", + cmdline="--soabi", + default=None), + BoolOption("honor__builtins__", "Honor the __builtins__ key of a module dictionary", default=False), @@ -369,6 +385,11 @@ modules = [name for name in modules if name not in essential_modules] config.objspace.usemodules.suggest(**dict.fromkeys(modules, True)) +def enable_translationmodules(config): + modules = translation_modules + modules = [name for name in modules if name not in essential_modules] + config.objspace.usemodules.suggest(**dict.fromkeys(modules, True)) + if __name__ == '__main__': config = get_pypy_config() Modified: pypy/branch/jitypes2/pypy/config/translationoption.py ============================================================================== --- pypy/branch/jitypes2/pypy/config/translationoption.py (original) +++ pypy/branch/jitypes2/pypy/config/translationoption.py Tue Nov 23 13:26:19 2010 @@ -115,7 +115,7 @@ default="auto", cmdline="--jit-backend"), ChoiceOption("jit_debug", "the amount of debugging dumps made by the JIT", ["off", "profile", "steps", "detailed"], - default="profile", # XXX for now + default="off", cmdline="--jit-debug"), ChoiceOption("jit_profiler", "integrate profiler support into the JIT", ["off", "oprofile"], Modified: pypy/branch/jitypes2/pypy/conftest.py ============================================================================== --- pypy/branch/jitypes2/pypy/conftest.py (original) +++ pypy/branch/jitypes2/pypy/conftest.py Tue Nov 23 13:26:19 2010 @@ -327,6 +327,31 @@ class PyPyTestFunction(py.test.collect.Function): # All PyPy test items catch and display OperationErrors specially. # + def runtest(self): + self.runtest_open() + try: + self.runtest_perform() + finally: + self.runtest_close() + self.runtest_finish() + + def runtest_open(self): + leakfinder.start_tracking_allocations() + + def runtest_perform(self): + super(PyPyTestFunction, self).runtest() + + def runtest_close(self): + if leakfinder.TRACK_ALLOCATIONS: + self._pypytest_leaks = leakfinder.stop_tracking_allocations(False) + else: # stop_tracking_allocations() already called + self._pypytest_leaks = None + + def runtest_finish(self): + # check for leaks, but only if the test passed so far + if self._pypytest_leaks: + raise leakfinder.MallocMismatch(self._pypytest_leaks) + def execute_appex(self, space, target, *args): try: target(*args) @@ -353,16 +378,9 @@ def _keywords(self): return super(IntTestFunction, self)._keywords() + ['interplevel'] - def runtest(self): + def runtest_perform(self): try: - leakfinder.start_tracking_allocations() - try: - super(IntTestFunction, self).runtest() - finally: - if leakfinder.TRACK_ALLOCATIONS: - leaks = leakfinder.stop_tracking_allocations(False) - else: - leaks = None # stop_tracking_allocations() already called + super(IntTestFunction, self).runtest_perform() except OperationError, e: check_keyboard_interrupt(e) raise @@ -375,14 +393,15 @@ py.test.skip('%s: %s' % (e.__class__.__name__, e)) cls = cls.__bases__[0] raise + + def runtest_finish(self): if 'pygame' in sys.modules: global _pygame_imported if not _pygame_imported: _pygame_imported = True assert option.view, ("should not invoke Pygame " "if conftest.option.view is False") - if leaks: # check for leaks, but only if the test passed so far - raise leakfinder.MallocMismatch(leaks) + super(IntTestFunction, self).runtest_finish() class AppTestFunction(PyPyTestFunction): def _prunetraceback(self, traceback): @@ -395,7 +414,7 @@ def _keywords(self): return ['applevel'] + super(AppTestFunction, self)._keywords() - def runtest(self): + def runtest_perform(self): target = self.obj if option.runappdirect: return target() @@ -427,7 +446,7 @@ space.setattr(w_instance, space.wrap(name[2:]), getattr(instance, name)) - def runtest(self): + def runtest_perform(self): target = self.obj if option.runappdirect: return target() Modified: pypy/branch/jitypes2/pypy/doc/release-1.4.0.txt ============================================================================== --- pypy/branch/jitypes2/pypy/doc/release-1.4.0.txt (original) +++ pypy/branch/jitypes2/pypy/doc/release-1.4.0.txt Tue Nov 23 13:26:19 2010 @@ -4,21 +4,25 @@ Hello. -We're pleased to announce release of PyPy 1.4. This is a major breaktrhough -in our long journey, since PyPy 1.4 is the first PyPy release that can translate -itself faster than CPython. Since today, we plan to start using -PyPy for our own development. +We're pleased to announce the 1.4 release of PyPy. This is a major breakthrough +in our long journey, as PyPy 1.4 is the first PyPy release that can translate +itself faster than CPython. Starting today, we plan to start using PyPy for our +own development. Among other features, this release includes numerous performance improvements -(which made fast self-hosting possible), 64jit backend as well as serious -stabilization. As of now, we can consider 32bit version of PyPy stable enough -to run in production. +(which made fast self-hosting possible), a 64-bit JIT backend, as well +as serious stabilization. As of now, we can consider the 32-bit and 64-bit +linux versions of PyPy stable enoughto run in production. More highlights =============== -Virtualenv support: now PyPy is fully compatible with virtualenv_: note that -to use it, you need a recent version of virtualenv (>= 1.5). +* Virtualenv support: now PyPy is fully compatible with virtualenv_: note that + to use it, you need a recent version of virtualenv (>= 1.5). -XXX write me: better regular expressions +* Faster (and JITted) regular expressions - huge boost in speeding up + sre module. +* Faster (and JITted) calls to functions like map(). + +.. _virtualenv: http://pypi.python.org/pypi/virtualenv Modified: pypy/branch/jitypes2/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/jitypes2/pypy/interpreter/argument.py (original) +++ pypy/branch/jitypes2/pypy/interpreter/argument.py Tue Nov 23 13:26:19 2010 @@ -64,7 +64,7 @@ return not self == other - # make it look tuply for the annotator + # make it look tuply for its use in the annotator def __len__(self): return 3 @@ -103,10 +103,11 @@ make_sure_not_resized(self.keywords_w) make_sure_not_resized(self.arguments_w) - if ((w_stararg is not None and w_stararg) or - (w_starstararg is not None and w_starstararg)): - self._combine_wrapped(w_stararg, w_starstararg) - # if we have a call where * or ** args are used at the callsite + if w_stararg is not None and space.is_true(w_stararg): + self._combine_starargs_wrapped(w_stararg) + if w_starstararg is not None and space.is_true(w_starstararg): + self._combine_starstarargs_wrapped(w_starstararg) + # if we have a call where **args are used at the callsite # we shouldn't let the JIT see the argument matching self._dont_jit = True else: @@ -142,42 +143,48 @@ def _combine_wrapped(self, w_stararg, w_starstararg): "unpack the *arg and **kwd into arguments_w and keywords_w" - # unpack the * arguments if w_stararg is not None: - self.arguments_w = (self.arguments_w + - self.space.fixedview(w_stararg)) - # unpack the ** arguments + self._combine_starargs_wrapped(w_stararg) if w_starstararg is not None: - space = self.space - if not space.is_true(space.isinstance(w_starstararg, space.w_dict)): + self._combine_starstarargs_wrapped(w_starstararg) + + def _combine_starargs_wrapped(self, w_stararg): + # unpack the * arguments + self.arguments_w = (self.arguments_w + + self.space.fixedview(w_stararg)) + + def _combine_starstarargs_wrapped(self, w_starstararg): + # unpack the ** arguments + space = self.space + if not space.is_true(space.isinstance(w_starstararg, space.w_dict)): + raise OperationError(space.w_TypeError, + space.wrap("argument after ** must be " + "a dictionary")) + keywords_w = [None] * space.int_w(space.len(w_starstararg)) + keywords = [None] * space.int_w(space.len(w_starstararg)) + i = 0 + for w_key in space.unpackiterable(w_starstararg): + try: + key = space.str_w(w_key) + except OperationError, e: + if not e.match(space, space.w_TypeError): + raise raise OperationError(space.w_TypeError, - space.wrap("argument after ** must be " - "a dictionary")) - keywords_w = [None] * space.int_w(space.len(w_starstararg)) - keywords = [None] * space.int_w(space.len(w_starstararg)) - i = 0 - for w_key in space.unpackiterable(w_starstararg): - try: - key = space.str_w(w_key) - except OperationError, e: - if not e.match(space, space.w_TypeError): - raise - raise OperationError(space.w_TypeError, - space.wrap("keywords must be strings")) - if self.keywords and key in self.keywords: - raise operationerrfmt(self.space.w_TypeError, - "got multiple values " - "for keyword argument " - "'%s'", key) - keywords[i] = key - keywords_w[i] = space.getitem(w_starstararg, w_key) - i += 1 - if self.keywords is None: - self.keywords = keywords - self.keywords_w = keywords_w - else: - self.keywords = self.keywords + keywords - self.keywords_w = self.keywords_w + keywords_w + space.wrap("keywords must be strings")) + if self.keywords and key in self.keywords: + raise operationerrfmt(self.space.w_TypeError, + "got multiple values " + "for keyword argument " + "'%s'", key) + keywords[i] = key + keywords_w[i] = space.getitem(w_starstararg, w_key) + i += 1 + if self.keywords is None: + self.keywords = keywords + self.keywords_w = keywords_w + else: + self.keywords = self.keywords + keywords + self.keywords_w = self.keywords_w + keywords_w def fixedunpack(self, argcount): """The simplest argument parsing: get the 'argcount' arguments, @@ -226,6 +233,10 @@ # argnames = list of formal parameter names # scope_w = resulting list of wrapped values # + + # some comments about the JIT: it assumes that signature is a constant, + # so all values coming from there can be assumed constant. It assumes + # that the length of the defaults_w does not vary too much. co_argcount = signature.num_argnames() # expected formal arguments, without */** has_vararg = signature.has_vararg() has_kwarg = signature.has_kwarg() @@ -245,12 +256,6 @@ args_w = self.arguments_w num_args = len(args_w) - keywords = self.keywords - keywords_w = self.keywords_w - num_kwds = 0 - if keywords is not None: - num_kwds = len(keywords) - avail = num_args + upfront if input_argcount < co_argcount: @@ -260,15 +265,24 @@ else: take = num_args + # letting the JIT unroll this loop is safe, because take is always + # smaller than co_argcount for i in range(take): scope_w[i + input_argcount] = args_w[i] input_argcount += take + keywords = self.keywords + keywords_w = self.keywords_w + num_kwds = 0 + if keywords is not None: + num_kwds = len(keywords) # the code assumes that keywords can potentially be large, but that # argnames is typically not too large num_remainingkwds = num_kwds used_keywords = None if keywords: + # letting JIT unroll the loop is *only* safe if the callsite didn't + # use **args because num_kwds can be arbitrarily large otherwise. used_keywords = [False] * num_kwds for i in range(num_kwds): name = keywords[i] @@ -276,7 +290,7 @@ if j < 0: continue elif j < input_argcount: - # check that no keyword argument conflicts with these note + # check that no keyword argument conflicts with these. note # that for this purpose we ignore the first blindargs, # which were put into place by prepend(). This way, # keywords do not conflict with the hidden extra argument @@ -388,9 +402,10 @@ space = self.space w_args = space.newtuple(self.arguments_w) w_kwds = space.newdict() - for i in range(len(self.keywords)): - space.setitem(w_kwds, space.wrap(self.keywords[i]), - self.keywords_w[i]) + if self.keywords is not None: + for i in range(len(self.keywords)): + space.setitem(w_kwds, space.wrap(self.keywords[i]), + self.keywords_w[i]) return w_args, w_kwds class ArgumentsForTranslation(Arguments): Modified: pypy/branch/jitypes2/pypy/interpreter/test/test_argument.py ============================================================================== --- pypy/branch/jitypes2/pypy/interpreter/test/test_argument.py (original) +++ pypy/branch/jitypes2/pypy/interpreter/test/test_argument.py Tue Nov 23 13:26:19 2010 @@ -52,12 +52,17 @@ assert y == "d" assert z == "e" +class dummy_wrapped_dict(dict): + def __nonzero__(self): + raise NotImplementedError class DummySpace(object): def newtuple(self, items): return tuple(items) def is_true(self, obj): + if isinstance(obj, dummy_wrapped_dict): + return bool(dict(obj)) return bool(obj) def fixedview(self, it): @@ -229,7 +234,7 @@ kwds_w = dict(kwds[:i]) keywords = kwds_w.keys() keywords_w = kwds_w.values() - w_kwds = dict(kwds[i:]) + w_kwds = dummy_wrapped_dict(kwds[i:]) if i == 2: w_kwds = None assert len(keywords) == len(keywords_w) @@ -265,7 +270,7 @@ kwds_w = dict(kwds[:i]) keywords = kwds_w.keys() keywords_w = kwds_w.values() - w_kwds = dict(kwds[i:]) + w_kwds = dummy_wrapped_dict(kwds[i:]) if i == 3: w_kwds = None args = Arguments(space, [1, 2], keywords, keywords_w, w_starstararg=w_kwds) @@ -448,7 +453,11 @@ assert set(args.keywords) == set(['a', 'b']) assert args.keywords_w[args.keywords.index('a')] == 2 assert args.keywords_w[args.keywords.index('b')] == 3 - + + args = Arguments(space, [1]) + w_args, w_kwds = args.topacked() + assert w_args == (1, ) + assert not w_kwds class TestErrorHandling(object): def test_missing_args(self): Modified: pypy/branch/jitypes2/pypy/jit/backend/llsupport/descr.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/backend/llsupport/descr.py (original) +++ pypy/branch/jitypes2/pypy/jit/backend/llsupport/descr.py Tue Nov 23 13:26:19 2010 @@ -130,6 +130,7 @@ # ArrayDescrs _A = lltype.GcArray(lltype.Signed) # a random gcarray +_AF = lltype.GcArray(lltype.Float) # an array of C doubles class BaseArrayDescr(AbstractDescr): @@ -171,16 +172,21 @@ _clsname = 'GcPtrArrayDescr' _is_array_of_pointers = True -_CA = rffi.CArray(lltype.Signed) +class FloatArrayDescr(BaseArrayDescr): + _clsname = 'FloatArrayDescr' + _is_array_of_floats = True + def get_base_size(self, translate_support_code): + basesize, _, _ = symbolic.get_array_token(_AF, translate_support_code) + return basesize + def get_item_size(self, translate_support_code): + return symbolic.get_size(lltype.Float, translate_support_code) class BaseArrayNoLengthDescr(BaseArrayDescr): def get_base_size(self, translate_support_code): - basesize, _, _ = symbolic.get_array_token(_CA, translate_support_code) - return basesize + return 0 def get_ofs_length(self, translate_support_code): - _, _, ofslength = symbolic.get_array_token(_CA, translate_support_code) - return ofslength + return -1 class NonGcPtrArrayNoLengthDescr(BaseArrayNoLengthDescr): _clsname = 'NonGcPtrArrayNoLengthDescr' @@ -192,6 +198,8 @@ _is_array_of_pointers = True def getArrayDescrClass(ARRAY): + if ARRAY.OF is lltype.Float: + return FloatArrayDescr return getDescrClass(ARRAY.OF, BaseArrayDescr, GcPtrArrayDescr, NonGcPtrArrayDescr, 'Array', 'get_item_size', '_is_array_of_floats', '_is_item_signed') @@ -219,7 +227,8 @@ basesize, itemsize, ofslength = symbolic.get_array_token(ARRAY, False) assert basesize == arraydescr.get_base_size(False) assert itemsize == arraydescr.get_item_size(False) - assert ofslength == arraydescr.get_ofs_length(False) + if not ARRAY._hints.get('nolength', False): + assert ofslength == arraydescr.get_ofs_length(False) if isinstance(ARRAY, lltype.GcArray): gccache.init_array_descr(ARRAY, arraydescr) cache[ARRAY] = arraydescr Modified: pypy/branch/jitypes2/pypy/jit/backend/llsupport/gc.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/backend/llsupport/gc.py (original) +++ pypy/branch/jitypes2/pypy/jit/backend/llsupport/gc.py Tue Nov 23 13:26:19 2010 @@ -573,6 +573,9 @@ # GETFIELD_RAW from the array 'gcrefs.list'. # newops = [] + # we can only remember one malloc since the next malloc can possibly + # collect + last_malloc = None for op in operations: if op.getopnum() == rop.DEBUG_MERGE_POINT: continue @@ -590,22 +593,32 @@ [ConstInt(addr)], box, self.single_gcref_descr)) op.setarg(i, box) + if op.is_malloc(): + last_malloc = op.result + elif op.can_malloc(): + last_malloc = None # ---------- write barrier for SETFIELD_GC ---------- if op.getopnum() == rop.SETFIELD_GC: - v = op.getarg(1) - if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and - bool(v.value)): # store a non-NULL - self._gen_write_barrier(newops, op.getarg(0), v) - op = op.copy_and_change(rop.SETFIELD_RAW) + val = op.getarg(0) + # no need for a write barrier in the case of previous malloc + if val is not last_malloc: + v = op.getarg(1) + if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and + bool(v.value)): # store a non-NULL + self._gen_write_barrier(newops, op.getarg(0), v) + op = op.copy_and_change(rop.SETFIELD_RAW) # ---------- write barrier for SETARRAYITEM_GC ---------- if op.getopnum() == rop.SETARRAYITEM_GC: - v = op.getarg(2) - if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and - bool(v.value)): # store a non-NULL - # XXX detect when we should produce a - # write_barrier_from_array - self._gen_write_barrier(newops, op.getarg(0), v) - op = op.copy_and_change(rop.SETARRAYITEM_RAW) + val = op.getarg(0) + # no need for a write barrier in the case of previous malloc + if val is not last_malloc: + v = op.getarg(2) + if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and + bool(v.value)): # store a non-NULL + # XXX detect when we should produce a + # write_barrier_from_array + self._gen_write_barrier(newops, op.getarg(0), v) + op = op.copy_and_change(rop.SETARRAYITEM_RAW) # ---------- newops.append(op) del operations[:] Modified: pypy/branch/jitypes2/pypy/jit/backend/llsupport/test/test_descr.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/backend/llsupport/test/test_descr.py (original) +++ pypy/branch/jitypes2/pypy/jit/backend/llsupport/test/test_descr.py Tue Nov 23 13:26:19 2010 @@ -5,6 +5,7 @@ from pypy.rpython.annlowlevel import llhelper from pypy.jit.metainterp.history import BoxInt, BoxFloat, BoxPtr from pypy.jit.metainterp import history +import struct def test_get_size_descr(): c0 = GcCache(False) @@ -130,11 +131,13 @@ assert not descr3.is_array_of_floats() assert descr4.is_array_of_floats() # - WORD = rffi.sizeof(lltype.Signed) - assert descr1.get_base_size(False) == WORD - assert descr2.get_base_size(False) == WORD - assert descr3.get_base_size(False) == WORD - assert descr4.get_base_size(False) == WORD + def get_alignment(code): + # Retrieve default alignment for the compiler/platform + return struct.calcsize('l' + code) - struct.calcsize(code) + assert descr1.get_base_size(False) == get_alignment('c') + assert descr2.get_base_size(False) == get_alignment('p') + assert descr3.get_base_size(False) == get_alignment('p') + assert descr4.get_base_size(False) == get_alignment('d') assert descr1.get_ofs_length(False) == 0 assert descr2.get_ofs_length(False) == 0 assert descr3.get_ofs_length(False) == 0 Modified: pypy/branch/jitypes2/pypy/jit/backend/llsupport/test/test_gc.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/backend/llsupport/test/test_gc.py (original) +++ pypy/branch/jitypes2/pypy/jit/backend/llsupport/test/test_gc.py Tue Nov 23 13:26:19 2010 @@ -6,7 +6,9 @@ from pypy.jit.backend.llsupport.gc import * from pypy.jit.backend.llsupport import symbolic from pypy.jit.metainterp.gc import get_description - +from pypy.jit.tool.oparser import parse +from pypy.rpython.lltypesystem.rclass import OBJECT, OBJECT_VTABLE +from pypy.jit.metainterp.test.test_optimizeopt import equaloplists def test_boehm(): gc_ll_descr = GcLLDescr_boehm(None, None, None) @@ -114,7 +116,7 @@ assert gcrootmap._gcmap[i*2+1] == expected_shapeaddr[i] -class FakeLLOp: +class FakeLLOp(object): def __init__(self): self.record = [] @@ -148,19 +150,19 @@ return llhelper(FPTRTYPE, self._write_barrier_failing_case) -class TestFramework: +class TestFramework(object): gc = 'hybrid' def setup_method(self, meth): - class config_: - class translation: + class config_(object): + class translation(object): gc = self.gc gcrootfinder = 'asmgcc' gctransformer = 'framework' gcremovetypeptr = False - class FakeTranslator: + class FakeTranslator(object): config = config_ - class FakeCPU: + class FakeCPU(object): def cast_adr_to_int(self, adr): ptr = llmemory.cast_adr_to_ptr(adr, gc_ll_descr.WB_FUNCPTR) assert ptr._obj._callable == llop1._write_barrier_failing_case @@ -278,11 +280,11 @@ def test_rewrite_assembler_1(self): # check rewriting of ConstPtrs - class MyFakeCPU: + class MyFakeCPU(object): def cast_adr_to_int(self, adr): assert adr == "some fake address" return 43 - class MyFakeGCRefList: + class MyFakeGCRefList(object): def get_address_of_gcref(self, s_gcref1): assert s_gcref1 == s_gcref return "some fake address" @@ -311,10 +313,10 @@ def test_rewrite_assembler_1_cannot_move(self): # check rewriting of ConstPtrs - class MyFakeCPU: + class MyFakeCPU(object): def cast_adr_to_int(self, adr): xxx # should not be called - class MyFakeGCRefList: + class MyFakeGCRefList(object): def get_address_of_gcref(self, s_gcref1): seen.append(s_gcref1) assert s_gcref1 == s_gcref @@ -394,6 +396,68 @@ assert operations[1].getarg(2) == v_value assert operations[1].getdescr() == array_descr + def test_rewrite_assembler_initialization_store(self): + S = lltype.GcStruct('S', ('parent', OBJECT), + ('x', lltype.Signed)) + s_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True) + xdescr = get_field_descr(self.gc_ll_descr, S, 'x') + ops = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + # no write barrier + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) + + def test_rewrite_assembler_initialization_store_2(self): + S = lltype.GcStruct('S', ('parent', OBJECT), + ('x', lltype.Signed)) + s_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True) + wbdescr = self.gc_ll_descr.write_barrier_descr + xdescr = get_field_descr(self.gc_ll_descr, S, 'x') + ops = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + p3 = new_with_vtable(ConstClass(s_vtable)) + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + p3 = new_with_vtable(ConstClass(s_vtable)) + cond_call_gc_wb(p0, p1, descr=wbdescr) + setfield_raw(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) + + def test_rewrite_assembler_initialization_store_3(self): + A = lltype.GcArray(lltype.Ptr(lltype.GcStruct('S'))) + arraydescr = get_array_descr(self.gc_ll_descr, A) + ops = parse(""" + [p1] + p0 = new_array(3, descr=arraydescr) + setarrayitem_gc(p0, 0, p1, descr=arraydescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_array(3, descr=arraydescr) + setarrayitem_gc(p0, 0, p1, descr=arraydescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) class TestFrameworkMiniMark(TestFramework): gc = 'minimark' Modified: pypy/branch/jitypes2/pypy/jit/backend/test/runner_test.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/backend/test/runner_test.py (original) +++ pypy/branch/jitypes2/pypy/jit/backend/test/runner_test.py Tue Nov 23 13:26:19 2010 @@ -2168,12 +2168,14 @@ # Tested with a function that intentionally does not cast the # result to RESTYPE, but makes sure that we return the whole # value in eax or rax. - eci = ExternalCompilationInfo(separate_module_sources=[""" + eci = ExternalCompilationInfo( + separate_module_sources=[""" long fn_test_result_of_call(long x) { return x + 1; } - """]) + """], + export_symbols=['fn_test_result_of_call']) f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed], RESTYPE, compilation_info=eci, _nowrapper=True) value = intmask(0xFFEEDDCCBBAA9988) @@ -2199,12 +2201,14 @@ # Tested with a function that intentionally does not cast the # result to RESTYPE, but makes sure that we return the whole # value in eax or rax. - eci = ExternalCompilationInfo(separate_module_sources=[""" + eci = ExternalCompilationInfo( + separate_module_sources=[""" long fn_test_result_of_call(long x) { return x + 1; } - """]) + """], + export_symbols=['fn_test_result_of_call']) f = rffi.llexternal('fn_test_result_of_call', [lltype.Signed], RESTYPE, compilation_info=eci, _nowrapper=True) value = intmask(0xFFEEDDCCBBAA9988) Modified: pypy/branch/jitypes2/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/jitypes2/pypy/jit/backend/x86/assembler.py Tue Nov 23 13:26:19 2010 @@ -1773,13 +1773,18 @@ self.implement_guard(guard_token, 'L') def genop_discard_cond_call_gc_wb(self, op, arglocs): - # use 'mc._mc' directly instead of 'mc', to avoid - # bad surprizes if the code buffer is mostly full + # Write code equivalent to write_barrier() in the GC: it checks + # a flag in the object at arglocs[0], and if set, it calls the + # function remember_young_pointer() from the GC. The two arguments + # to the call are in arglocs[:2]. The rest, arglocs[2:], contains + # registers that need to be saved and restored across the call. descr = op.getdescr() if we_are_translated(): cls = self.cpu.gc_ll_descr.has_write_barrier_class() assert cls is not None and isinstance(descr, cls) loc_base = arglocs[0] + # ensure that enough bytes are available to write the whole + # following piece of code atomically (for the JZ) self.mc.ensure_bytes_available(256) self.mc.TEST8_mi((loc_base.value, descr.jit_wb_if_flag_byteofs), descr.jit_wb_if_flag_singlebyte) @@ -1787,36 +1792,39 @@ jz_location = self.mc.get_relative_pos() # the following is supposed to be the slow path, so whenever possible # we choose the most compact encoding over the most efficient one. - # XXX improve a bit, particularly for IS_X86_64. - for i in range(len(arglocs)-1, -1, -1): + if IS_X86_32: + limit = -1 # push all arglocs on the stack + elif IS_X86_64: + limit = 1 # push only arglocs[2:] on the stack + for i in range(len(arglocs)-1, limit, -1): loc = arglocs[i] if isinstance(loc, RegLoc): self.mc.PUSH_r(loc.value) else: - if IS_X86_64: - self.mc.MOV_ri(X86_64_SCRATCH_REG.value, loc.getint()) - self.mc.PUSH_r(X86_64_SCRATCH_REG.value) - else: - self.mc.PUSH_i32(loc.getint()) - + assert not IS_X86_64 # there should only be regs in arglocs[2:] + self.mc.PUSH_i32(loc.getint()) if IS_X86_64: # We clobber these registers to pass the arguments, but that's # okay, because consider_cond_call_gc_wb makes sure that any - # caller-save registers with values in them are present in arglocs, - # so they are saved on the stack above and restored below - self.mc.MOV_rs(edi.value, 0) - self.mc.MOV_rs(esi.value, 8) + # caller-save registers with values in them are present in + # arglocs[2:] too, so they are saved on the stack above and + # restored below. + remap_frame_layout(self, arglocs[:2], [edi, esi], + X86_64_SCRATCH_REG) # misaligned stack in the call, but it's ok because the write barrier # is not going to call anything more. Also, this assumes that the - # write barrier does not touch the xmm registers. + # write barrier does not touch the xmm registers. (Slightly delicate + # assumption, given that the write barrier can end up calling the + # platform's malloc() from AddressStack.append(). XXX may need to + # be done properly) self.mc.CALL(imm(descr.get_write_barrier_fn(self.cpu))) - for i in range(len(arglocs)): + if IS_X86_32: + self.mc.ADD_ri(esp.value, 2*WORD) + for i in range(2, len(arglocs)): loc = arglocs[i] - if isinstance(loc, RegLoc): - self.mc.POP_r(loc.value) - else: - self.mc.ADD_ri(esp.value, WORD) # ignore the pushed constant + assert isinstance(loc, RegLoc) + self.mc.POP_r(loc.value) # patch the JZ above offset = self.mc.get_relative_pos() - jz_location assert 0 < offset <= 127 Modified: pypy/branch/jitypes2/pypy/jit/codewriter/assembler.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/codewriter/assembler.py (original) +++ pypy/branch/jitypes2/pypy/jit/codewriter/assembler.py Tue Nov 23 13:26:19 2010 @@ -233,10 +233,9 @@ addr = llmemory.cast_ptr_to_adr(value) self.list_of_addr2name.append((addr, name)) - def finished(self): + def finished(self, callinfocollection): # Helper called at the end of assembling. Registers the extra # functions shown in _callinfo_for_oopspec. - from pypy.jit.codewriter.effectinfo import _callinfo_for_oopspec - for _, func in _callinfo_for_oopspec.values(): + for func in callinfocollection.all_function_addresses_as_int(): func = heaptracker.int2adr(func) self.see_raw_object(func.ptr) Modified: pypy/branch/jitypes2/pypy/jit/codewriter/call.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/codewriter/call.py (original) +++ pypy/branch/jitypes2/pypy/jit/codewriter/call.py Tue Nov 23 13:26:19 2010 @@ -7,7 +7,7 @@ from pypy.jit.codewriter.jitcode import JitCode from pypy.jit.codewriter.effectinfo import VirtualizableAnalyzer from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze -from pypy.jit.codewriter.effectinfo import EffectInfo +from pypy.jit.codewriter.effectinfo import EffectInfo, CallInfoCollection from pypy.translator.simplify import get_funcobj, get_functype from pypy.rpython.lltypesystem import lltype, llmemory from pypy.translator.backendopt.canraise import RaiseAnalyzer @@ -23,6 +23,7 @@ self.jitdrivers_sd = jitdrivers_sd self.jitcodes = {} # map {graph: jitcode} self.unfinished_graphs = [] # list of graphs with pending jitcodes + self.callinfocollection = CallInfoCollection() if hasattr(cpu, 'rtyper'): # for tests self.rtyper = cpu.rtyper translator = self.rtyper.annotator.translator Modified: pypy/branch/jitypes2/pypy/jit/codewriter/codewriter.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/codewriter/codewriter.py (original) +++ pypy/branch/jitypes2/pypy/jit/codewriter/codewriter.py Tue Nov 23 13:26:19 2010 @@ -73,7 +73,7 @@ count += 1 if not count % 500: log.info("Produced %d jitcodes" % count) - self.assembler.finished() + self.assembler.finished(self.callcontrol.callinfocollection) heaptracker.finish_registering(self.cpu) log.info("there are %d JitCode instances." % count) Modified: pypy/branch/jitypes2/pypy/jit/codewriter/effectinfo.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/codewriter/effectinfo.py (original) +++ pypy/branch/jitypes2/pypy/jit/codewriter/effectinfo.py Tue Nov 23 13:26:19 2010 @@ -144,30 +144,44 @@ # ____________________________________________________________ -_callinfo_for_oopspec = {} # {oopspecindex: (calldescr, func_as_int)} - -def callinfo_for_oopspec(oopspecindex): - """A function that returns the calldescr and the function - address (as an int) of one of the OS_XYZ functions defined above. - Don't use this if there might be several implementations of the same - OS_XYZ specialized by type, e.g. OS_ARRAYCOPY.""" - try: - return _callinfo_for_oopspec[oopspecindex] - except KeyError: - return (None, 0) - - -def _funcptr_for_oopspec_memo(oopspecindex): - from pypy.jit.codewriter import heaptracker - _, func_as_int = callinfo_for_oopspec(oopspecindex) - funcadr = heaptracker.int2adr(func_as_int) - return funcadr.ptr -_funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo' - -def funcptr_for_oopspec(oopspecindex): - """A memo function that returns a pointer to the function described - by OS_XYZ (as a real low-level function pointer).""" - funcptr = _funcptr_for_oopspec_memo(oopspecindex) - assert funcptr - return funcptr -funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(0)' +class CallInfoCollection(object): + def __init__(self): + # {oopspecindex: (calldescr, func_as_int)} + self._callinfo_for_oopspec = {} + + def _freeze_(self): + return True + + def add(self, oopspecindex, calldescr, func_as_int): + self._callinfo_for_oopspec[oopspecindex] = calldescr, func_as_int + + def has_oopspec(self, oopspecindex): + return oopspecindex in self._callinfo_for_oopspec + + def all_function_addresses_as_int(self): + return [func for (_, func) in self._callinfo_for_oopspec.values()] + + def callinfo_for_oopspec(self, oopspecindex): + """A function that returns the calldescr and the function + address (as an int) of one of the OS_XYZ functions defined above. + Don't use this if there might be several implementations of the same + OS_XYZ specialized by type, e.g. OS_ARRAYCOPY.""" + try: + return self._callinfo_for_oopspec[oopspecindex] + except KeyError: + return (None, 0) + + def _funcptr_for_oopspec_memo(self, oopspecindex): + from pypy.jit.codewriter import heaptracker + _, func_as_int = self.callinfo_for_oopspec(oopspecindex) + funcadr = heaptracker.int2adr(func_as_int) + return funcadr.ptr + _funcptr_for_oopspec_memo._annspecialcase_ = 'specialize:memo' + + def funcptr_for_oopspec(self, oopspecindex): + """A memo function that returns a pointer to the function described + by OS_XYZ (as a real low-level function pointer).""" + funcptr = self._funcptr_for_oopspec_memo(oopspecindex) + assert funcptr + return funcptr + funcptr_for_oopspec._annspecialcase_ = 'specialize:arg(1)' Modified: pypy/branch/jitypes2/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/codewriter/jtransform.py (original) +++ pypy/branch/jitypes2/pypy/jit/codewriter/jtransform.py Tue Nov 23 13:26:19 2010 @@ -6,7 +6,7 @@ from pypy.objspace.flow.model import Block, Link, c_last_exception from pypy.jit.codewriter.flatten import ListOfKind, IndirectCallTargets from pypy.jit.codewriter import support, heaptracker -from pypy.jit.codewriter.effectinfo import EffectInfo, _callinfo_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.jit.codewriter.policy import log from pypy.jit.metainterp.typesystem import deref, arrayItem from pypy.rlib import objectmodel @@ -843,9 +843,16 @@ "general mix-up of jitdrivers?") ops = self.promote_greens(op.args[2:], jitdriver) num_green_args = len(jitdriver.greens) + redlists = self.make_three_lists(op.args[2+num_green_args:]) + for redlist in redlists: + for v in redlist: + assert isinstance(v, Variable), ( + "Constant specified red in jit_merge_point()") + assert len(dict.fromkeys(redlist)) == len(list(redlist)), ( + "duplicate red variable on jit_merge_point()") args = ([Constant(self.portal_jd.index, lltype.Signed)] + self.make_three_lists(op.args[2:2+num_green_args]) + - self.make_three_lists(op.args[2+num_green_args:])) + redlists) op1 = SpaceOperation('jit_merge_point', args, None) op2 = SpaceOperation('-live-', [], None) # ^^^ we need a -live- for the case of do_recursive_call() @@ -1084,7 +1091,8 @@ else: func = heaptracker.adr2int( llmemory.cast_ptr_to_adr(op.args[0].value)) - _callinfo_for_oopspec[oopspecindex] = calldescr, func + self.callcontrol.callinfocollection.add(oopspecindex, + calldescr, func) op1 = self.rewrite_call(op, 'residual_call', [op.args[0], calldescr], args=args) @@ -1095,7 +1103,7 @@ def _register_extra_helper(self, oopspecindex, oopspec_name, argtypes, resulttype): # a bit hackish - if oopspecindex in _callinfo_for_oopspec: + if self.callcontrol.callinfocollection.has_oopspec(oopspecindex): return c_func, TP = support.builtin_func_for_spec(self.cpu.rtyper, oopspec_name, argtypes, @@ -1109,7 +1117,7 @@ else: func = heaptracker.adr2int( llmemory.cast_ptr_to_adr(c_func.value)) - _callinfo_for_oopspec[oopspecindex] = calldescr, func + self.callcontrol.callinfocollection.add(oopspecindex, calldescr, func) def _handle_stroruni_call(self, op, oopspec_name, args): SoU = args[0].concretetype # Ptr(STR) or Ptr(UNICODE) Modified: pypy/branch/jitypes2/pypy/jit/codewriter/test/test_jtransform.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/codewriter/test/test_jtransform.py (original) +++ pypy/branch/jitypes2/pypy/jit/codewriter/test/test_jtransform.py Tue Nov 23 13:26:19 2010 @@ -74,7 +74,20 @@ def calldescr_canraise(self, calldescr): return False +class FakeCallInfoCollection: + def __init__(self): + self.seen = [] + def add(self, oopspecindex, calldescr, func): + self.seen.append((oopspecindex, calldescr, func)) + def has_oopspec(self, oopspecindex): + for i, c, f in self.seen: + if i == oopspecindex: + return True + return False + class FakeBuiltinCallControl: + def __init__(self): + self.callinfocollection = FakeCallInfoCollection() def guess_call_kind(self, op): return 'builtin' def getcalldescr(self, op, oopspecindex=None): @@ -810,7 +823,8 @@ v2 = varoftype(PSTR) v3 = varoftype(PSTR) op = SpaceOperation('direct_call', [const(func), v1, v2], v3) - tr = Transformer(FakeCPU(), FakeBuiltinCallControl()) + cc = FakeBuiltinCallControl() + tr = Transformer(FakeCPU(), cc) op1 = tr.rewrite_operation(op) assert op1.opname == 'residual_call_r_r' assert op1.args[0].value == func @@ -819,9 +833,10 @@ assert op1.result == v3 # # check the callinfo_for_oopspec - got = effectinfo.callinfo_for_oopspec(effectinfo.EffectInfo.OS_UNI_CONCAT) - assert got[0] == op1.args[1] # the calldescr - assert heaptracker.int2adr(got[1]) == llmemory.cast_ptr_to_adr(func) + got = cc.callinfocollection.seen[0] + assert got[0] == effectinfo.EffectInfo.OS_UNI_CONCAT + assert got[1] == op1.args[1] # the calldescr + assert heaptracker.int2adr(got[2]) == llmemory.cast_ptr_to_adr(func) def test_str_slice(): # test that the oopspec is present and correctly transformed @@ -893,7 +908,8 @@ v2 = varoftype(PUNICODE) v3 = varoftype(lltype.Bool) op = SpaceOperation('direct_call', [const(func), v1, v2], v3) - tr = Transformer(FakeCPU(), FakeBuiltinCallControl()) + cc = FakeBuiltinCallControl() + tr = Transformer(FakeCPU(), cc) op1 = tr.rewrite_operation(op) assert op1.opname == 'residual_call_r_i' assert op1.args[0].value == func @@ -901,9 +917,9 @@ assert op1.args[2] == ListOfKind('ref', [v1, v2]) assert op1.result == v3 # test that the OS_UNIEQ_* functions are registered - cifo = effectinfo._callinfo_for_oopspec - assert effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL in cifo - assert effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR in cifo + cic = cc.callinfocollection + assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_SLICE_NONNULL) + assert cic.has_oopspec(effectinfo.EffectInfo.OS_UNIEQ_CHECKNULL_CHAR) def test_list_ll_arraycopy(): from pypy.rlib.rgc import ll_arraycopy Modified: pypy/branch/jitypes2/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/compile.py Tue Nov 23 13:26:19 2010 @@ -1,4 +1,5 @@ +from pypy.rpython.lltypesystem import lltype from pypy.rpython.ootypesystem import ootype from pypy.objspace.flow.model import Constant, Variable from pypy.rlib.objectmodel import we_are_translated @@ -14,6 +15,7 @@ from pypy.jit.metainterp.specnode import NotSpecNode, more_general_specnodes from pypy.jit.metainterp.typesystem import llhelper, oohelper from pypy.jit.metainterp.optimizeutil import InvalidLoop +from pypy.jit.metainterp.resume import NUMBERING from pypy.jit.codewriter import heaptracker def giveup(): @@ -48,13 +50,12 @@ # ____________________________________________________________ -def compile_new_loop(metainterp, old_loop_tokens, greenkey, start): +def compile_new_loop(metainterp, old_loop_tokens, start): """Try to compile a new loop by closing the current history back to the first operation. """ history = metainterp.history loop = create_empty_loop(metainterp) - loop.greenkey = greenkey loop.inputargs = history.inputargs for box in loop.inputargs: assert isinstance(box, Box) @@ -207,8 +208,7 @@ } class ResumeDescr(AbstractFailDescr): - def __init__(self, original_greenkey): - self.original_greenkey = original_greenkey + pass class ResumeGuardDescr(ResumeDescr): _counter = 0 # if < 0, there is one counter per value; @@ -217,7 +217,7 @@ # this class also gets the following attributes stored by resume.py code rd_snapshot = None rd_frame_info_list = None - rd_numb = None + rd_numb = lltype.nullptr(NUMBERING) rd_consts = None rd_virtuals = None rd_pendingfields = None @@ -227,8 +227,7 @@ CNT_FLOAT = -0x60000000 CNT_MASK = 0x1FFFFFFF - def __init__(self, metainterp_sd, original_greenkey): - ResumeDescr.__init__(self, original_greenkey) + def __init__(self, metainterp_sd): self.metainterp_sd = metainterp_sd def store_final_boxes(self, guard_op, boxes): @@ -332,14 +331,14 @@ res.rd_pendingfields = self.rd_pendingfields def _clone_if_mutable(self): - res = ResumeGuardDescr(self.metainterp_sd, self.original_greenkey) + res = ResumeGuardDescr(self.metainterp_sd) self.copy_all_attrbutes_into(res) return res class ResumeGuardForcedDescr(ResumeGuardDescr): - def __init__(self, metainterp_sd, original_greenkey, jitdriver_sd): - ResumeGuardDescr.__init__(self, metainterp_sd, original_greenkey) + def __init__(self, metainterp_sd, jitdriver_sd): + ResumeGuardDescr.__init__(self, metainterp_sd) self.jitdriver_sd = jitdriver_sd def handle_fail(self, metainterp_sd, jitdriver_sd): @@ -406,7 +405,6 @@ def _clone_if_mutable(self): res = ResumeGuardForcedDescr(self.metainterp_sd, - self.original_greenkey, self.jitdriver_sd) self.copy_all_attrbutes_into(res) return res @@ -473,9 +471,8 @@ class ResumeFromInterpDescr(ResumeDescr): - def __init__(self, original_greenkey, redkey): - ResumeDescr.__init__(self, original_greenkey) - self.redkey = redkey + def __init__(self, original_greenkey): + self.original_greenkey = original_greenkey def compile_and_attach(self, metainterp, new_loop): # We managed to create a bridge going from the interpreter @@ -484,10 +481,8 @@ # with completely unoptimized arguments, as in the interpreter. metainterp_sd = metainterp.staticdata jitdriver_sd = metainterp.jitdriver_sd - metainterp.history.inputargs = self.redkey - new_loop_token = make_loop_token(len(self.redkey), jitdriver_sd) - new_loop.greenkey = self.original_greenkey - new_loop.inputargs = self.redkey + redargs = new_loop.inputargs + new_loop_token = make_loop_token(len(redargs), jitdriver_sd) new_loop.token = new_loop_token send_loop_to_backend(metainterp_sd, new_loop, "entry bridge") # send the new_loop to warmspot.py, to be called directly the next time Modified: pypy/branch/jitypes2/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/optimizeopt/optimizer.py Tue Nov 23 13:26:19 2010 @@ -299,7 +299,9 @@ return CVAL_ZERO def propagate_all_forward(self): - self.exception_might_have_happened = False + self.exception_might_have_happened = True + # ^^^ at least at the start of bridges. For loops, we could set + # it to False, but we probably don't care self.newoperations = [] self.i = 0 while self.i < len(self.loop.operations): Modified: pypy/branch/jitypes2/pypy/jit/metainterp/optimizeopt/string.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/optimizeopt/string.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/optimizeopt/string.py Tue Nov 23 13:26:19 2010 @@ -9,7 +9,7 @@ from pypy.jit.metainterp.optimizeopt.optimizer import CONST_0, CONST_1 from pypy.jit.metainterp.optimizeopt.optimizer import llhelper from pypy.jit.metainterp.optimizeutil import _findall -from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.jit.codewriter import heaptracker from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.objectmodel import specialize, we_are_translated @@ -593,7 +593,8 @@ def generate_modified_call(self, oopspecindex, args, result, mode): oopspecindex += mode.OS_offset - calldescr, func = callinfo_for_oopspec(oopspecindex) + cic = self.optimizer.metainterp_sd.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(oopspecindex) op = ResOperation(rop.CALL, [ConstInt(func)] + args, result, descr=calldescr) self.optimizer.newoperations.append(op) Modified: pypy/branch/jitypes2/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/pyjitpl.py Tue Nov 23 13:26:19 2010 @@ -14,7 +14,7 @@ from pypy.jit.metainterp.logger import Logger from pypy.jit.metainterp.jitprof import EmptyProfiler from pypy.jit.metainterp.jitprof import GUARDS, RECORDED_OPS, ABORT_ESCAPE -from pypy.jit.metainterp.jitprof import ABORT_TOO_LONG, ABORT_BRIDGE +from pypy.jit.metainterp.jitprof import ABORT_TOO_LONG from pypy.jit.metainterp.jitexc import JitException, get_llexception from pypy.rlib.rarithmetic import intmask from pypy.rlib.objectmodel import specialize @@ -94,6 +94,18 @@ else: raise AssertionError(argcode) outvalue[startindex+i] = reg + def _put_back_list_of_boxes(self, outvalue, startindex, position): + code = self.bytecode + length = ord(code[position]) + position += 1 + for i in range(length): + index = ord(code[position+i]) + box = outvalue[startindex+i] + if box.type == history.INT: self.registers_i[index] = box + elif box.type == history.REF: self.registers_r[index] = box + elif box.type == history.FLOAT: self.registers_f[index] = box + else: raise AssertionError(box.type) + def get_current_position_info(self): return self.jitcode.get_live_vars_info(self.pc) @@ -814,8 +826,9 @@ for i in range(num_green_args): assert isinstance(varargs[i], Const) - @arguments("orgpc", "int", "boxes3", "boxes3") - def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, redboxes): + @arguments("orgpc", "int", "boxes3", "jitcode_position", "boxes3") + def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, + jcposition, redboxes): any_operation = len(self.metainterp.history.operations) > 0 jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex] self.verify_green_args(jitdriver_sd, greenboxes) @@ -843,6 +856,10 @@ self.pc = orgpc self.metainterp.reached_loop_header(greenboxes, redboxes) self.pc = saved_pc + # no exception, which means that the jit_merge_point did not + # close the loop. We have to put the possibly-modified list + # 'redboxes' back into the registers where it comes from. + put_back_list_of_boxes3(self, jcposition, redboxes) else: # warning! careful here. We have to return from the current # frame containing the jit_merge_point, and then use @@ -1048,14 +1065,11 @@ else: moreargs = list(extraargs) metainterp_sd = metainterp.staticdata - original_greenkey = metainterp.resumekey.original_greenkey if opnum == rop.GUARD_NOT_FORCED: resumedescr = compile.ResumeGuardForcedDescr(metainterp_sd, - original_greenkey, metainterp.jitdriver_sd) else: - resumedescr = compile.ResumeGuardDescr(metainterp_sd, - original_greenkey) + resumedescr = compile.ResumeGuardDescr(metainterp_sd) guard_op = metainterp.history.record(opnum, moreargs, None, descr=resumedescr) virtualizable_boxes = None @@ -1261,6 +1275,7 @@ # self.jitdrivers_sd = codewriter.callcontrol.jitdrivers_sd self.virtualref_info = codewriter.callcontrol.virtualref_info + self.callinfocollection = codewriter.callcontrol.callinfocollection self.setup_jitdrivers_sd(optimizer) # # store this information for fastpath of call_assembler @@ -1624,7 +1639,7 @@ assert jitdriver_sd is self.jitdriver_sd self.create_empty_history() try: - original_boxes = self.initialize_original_boxes(jitdriver_sd,*args) + original_boxes = self.initialize_original_boxes(jitdriver_sd, *args) return self._compile_and_run_once(original_boxes) finally: self.staticdata.profiler.end_tracing() @@ -1635,9 +1650,8 @@ self.current_merge_points = [(original_boxes, 0)] num_green_args = self.jitdriver_sd.num_green_args original_greenkey = original_boxes[:num_green_args] - redkey = original_boxes[num_green_args:] - self.resumekey = compile.ResumeFromInterpDescr(original_greenkey, - redkey) + self.resumekey = compile.ResumeFromInterpDescr(original_greenkey) + self.history.inputargs = original_boxes[num_green_args:] self.seen_loop_header_for_jdindex = -1 try: self.interpret() @@ -1659,11 +1673,7 @@ debug_stop('jit-tracing') def _handle_guard_failure(self, key): - original_greenkey = key.original_greenkey - # notice that here we just put the greenkey - # use -1 to mark that we will have to give up - # because we cannot reconstruct the beginning of the proper loop - self.current_merge_points = [(original_greenkey, -1)] + self.current_merge_points = [] self.resumekey = key self.seen_loop_header_for_jdindex = -1 try: @@ -1727,7 +1737,7 @@ num_green_args = self.jitdriver_sd.num_green_args for j in range(len(self.current_merge_points)-1, -1, -1): original_boxes, start = self.current_merge_points[j] - assert len(original_boxes) == len(live_arg_boxes) or start < 0 + assert len(original_boxes) == len(live_arg_boxes) for i in range(num_green_args): box1 = original_boxes[i] box2 = live_arg_boxes[i] @@ -1736,10 +1746,6 @@ break else: # Found! Compile it as a loop. - if start < 0: - # we cannot reconstruct the beginning of the proper loop - raise SwitchToBlackhole(ABORT_BRIDGE) - # raises in case it works -- which is the common case self.compile(original_boxes, live_arg_boxes, start) # creation of the loop was cancelled! @@ -1804,8 +1810,7 @@ greenkey = original_boxes[:num_green_args] old_loop_tokens = self.get_compiled_merge_points(greenkey) self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None) - loop_token = compile.compile_new_loop(self, old_loop_tokens, - greenkey, start) + loop_token = compile.compile_new_loop(self, old_loop_tokens, start) if loop_token is not None: # raise if it *worked* correctly raise GenerateMergePoint(live_arg_boxes, loop_token) self.history.operations.pop() # remove the JUMP @@ -2305,6 +2310,8 @@ else: raise AssertionError("bad argcode") position += 1 + elif argtype == "jitcode_position": + value = position else: raise AssertionError("bad argtype: %r" % (argtype,)) args += (value,) @@ -2349,3 +2356,15 @@ argtypes = unrolling_iterable(unboundmethod.argtypes) handler.func_name = 'handler_' + name return handler + +def put_back_list_of_boxes3(frame, position, newvalue): + code = frame.bytecode + length1 = ord(code[position]) + position2 = position + 1 + length1 + length2 = ord(code[position2]) + position3 = position2 + 1 + length2 + length3 = ord(code[position3]) + assert len(newvalue) == length1 + length2 + length3 + frame._put_back_list_of_boxes(newvalue, 0, position) + frame._put_back_list_of_boxes(newvalue, length1, position2) + frame._put_back_list_of_boxes(newvalue, length1 + length2, position3) Modified: pypy/branch/jitypes2/pypy/jit/metainterp/resoperation.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/resoperation.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/resoperation.py Tue Nov 23 13:26:19 2010 @@ -143,6 +143,16 @@ def can_raise(self): return rop._CANRAISE_FIRST <= self.getopnum() <= rop._CANRAISE_LAST + def is_malloc(self): + # a slightly different meaning from can_malloc + return rop._MALLOC_FIRST <= self.getopnum() <= rop._MALLOC_LAST + + def can_malloc(self): + return self.is_call() or self.is_malloc() + + def is_call(self): + return rop._CALL_FIRST <= self.getopnum() <= rop._CALL_LAST + def is_ovf(self): return rop._OVF_FIRST <= self.getopnum() <= rop._OVF_LAST @@ -441,9 +451,13 @@ 'GETARRAYITEM_RAW/2d', 'GETFIELD_GC/1d', 'GETFIELD_RAW/1d', + '_MALLOC_FIRST', 'NEW/0d', 'NEW_WITH_VTABLE/1', 'NEW_ARRAY/1d', + 'NEWSTR/1', + 'NEWUNICODE/1', + '_MALLOC_LAST', 'FORCE_TOKEN/0', 'VIRTUAL_REF/2', # removed before it's passed to the backend '_NOSIDEEFFECT_LAST', # ----- end of no_side_effect operations ----- @@ -452,10 +466,8 @@ 'SETARRAYITEM_RAW/3d', 'SETFIELD_GC/2d', 'SETFIELD_RAW/2d', - 'NEWSTR/1', 'STRSETITEM/3', 'UNICODESETITEM/3', - 'NEWUNICODE/1', #'RUNTIMENEW/1', # ootype operation 'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier) 'DEBUG_MERGE_POINT/2', # debugging only @@ -465,6 +477,7 @@ 'COPYUNICODECONTENT/5', '_CANRAISE_FIRST', # ----- start of can_raise operations ----- + '_CALL_FIRST', 'CALL/*d', 'CALL_ASSEMBLER/*d', # call already compiled assembler 'CALL_MAY_FORCE/*d', @@ -473,6 +486,7 @@ #'OOSEND_PURE', # ootype operation 'CALL_PURE/*d', # removed before it's passed to the backend # CALL_PURE(result, func, arg_1,..,arg_n) + '_CALL_LAST', '_CANRAISE_LAST', # ----- end of can_raise operations ----- '_OVF_FIRST', # ----- start of is_ovf operations ----- Modified: pypy/branch/jitypes2/pypy/jit/metainterp/resume.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/resume.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/resume.py Tue Nov 23 13:26:19 2010 @@ -4,8 +4,7 @@ from pypy.jit.metainterp.history import INT, REF, FLOAT, HOLE from pypy.jit.metainterp.resoperation import rop from pypy.jit.metainterp import jitprof -from pypy.jit.codewriter.effectinfo import EffectInfo, callinfo_for_oopspec -from pypy.jit.codewriter.effectinfo import funcptr_for_oopspec +from pypy.jit.codewriter.effectinfo import EffectInfo from pypy.rpython.lltypesystem import lltype, llmemory, rffi, rstr from pypy.rlib import rarithmetic from pypy.rlib.objectmodel import we_are_translated, specialize @@ -66,12 +65,21 @@ snapshot = Snapshot(snapshot, boxes) storage.rd_snapshot = snapshot -class Numbering(object): - __slots__ = ('prev', 'nums') - - def __init__(self, prev, nums): - self.prev = prev - self.nums = nums +# +# The following is equivalent to the RPython-level declaration: +# +# class Numbering: __slots__ = ['prev', 'nums'] +# +# except that it is more compact in translated programs, because the +# array 'nums' is inlined in the single NUMBERING object. This is +# important because this is often the biggest single consumer of memory +# in a pypy-c-jit. +# +NUMBERINGP = lltype.Ptr(lltype.GcForwardReference()) +NUMBERING = lltype.GcStruct('Numbering', + ('prev', NUMBERINGP), + ('nums', lltype.Array(rffi.SHORT))) +NUMBERINGP.TO.become(NUMBERING) TAGMASK = 3 @@ -163,7 +171,7 @@ def number(self, values, snapshot): if snapshot is None: - return None, {}, 0 + return lltype.nullptr(NUMBERING), {}, 0 if snapshot in self.numberings: numb, liveboxes, v = self.numberings[snapshot] return numb, liveboxes.copy(), v @@ -172,7 +180,7 @@ n = len(liveboxes)-v boxes = snapshot.boxes length = len(boxes) - nums = [UNASSIGNED] * length + numb = lltype.malloc(NUMBERING, length) for i in range(length): box = boxes[i] value = values.get(box, None) @@ -191,9 +199,9 @@ tagged = tag(n, TAGBOX) n += 1 liveboxes[box] = tagged - nums[i] = tagged + numb.nums[i] = tagged # - numb = Numbering(numb1, nums) + numb.prev = numb1 self.numberings[snapshot] = numb, liveboxes, v return numb, liveboxes.copy(), v @@ -298,7 +306,7 @@ # compute the numbering storage = self.storage # make sure that nobody attached resume data to this guard yet - assert storage.rd_numb is None + assert not storage.rd_numb numb, liveboxes_from_env, v = self.memo.number(values, storage.rd_snapshot) self.liveboxes_from_env = liveboxes_from_env @@ -723,34 +731,36 @@ self.boxes_f = boxes_f self._prepare_next_section(info) - def consume_virtualizable_boxes(self, vinfo, nums): + def consume_virtualizable_boxes(self, vinfo, numb): # we have to ignore the initial part of 'nums' (containing vrefs), # find the virtualizable from nums[-1], and use it to know how many # boxes of which type we have to return. This does not write # anything into the virtualizable. - virtualizablebox = self.decode_ref(nums[-1]) + index = len(numb.nums) - 1 + virtualizablebox = self.decode_ref(numb.nums[index]) virtualizable = vinfo.unwrap_virtualizable_box(virtualizablebox) - return vinfo.load_list_of_boxes(virtualizable, self, nums) + return vinfo.load_list_of_boxes(virtualizable, self, numb) - def consume_virtualref_boxes(self, nums, end): + def consume_virtualref_boxes(self, numb, end): # Returns a list of boxes, assumed to be all BoxPtrs. # We leave up to the caller to call vrefinfo.continue_tracing(). assert (end & 1) == 0 - return [self.decode_ref(nums[i]) for i in range(end)] + return [self.decode_ref(numb.nums[i]) for i in range(end)] def consume_vref_and_vable_boxes(self, vinfo, ginfo): - nums = self.cur_numb.nums - self.cur_numb = self.cur_numb.prev + numb = self.cur_numb + self.cur_numb = numb.prev if vinfo is not None: - virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, nums) - end = len(nums) - len(virtualizable_boxes) + virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, numb) + end = len(numb.nums) - len(virtualizable_boxes) elif ginfo is not None: - virtualizable_boxes = [self.decode_ref(nums[-1])] - end = len(nums) - 1 + index = len(numb.nums) - 1 + virtualizable_boxes = [self.decode_ref(numb.nums[index])] + end = len(numb.nums) - 1 else: virtualizable_boxes = None - end = len(nums) - virtualref_boxes = self.consume_virtualref_boxes(nums, end) + end = len(numb.nums) + virtualref_boxes = self.consume_virtualref_boxes(numb, end) return virtualizable_boxes, virtualref_boxes def allocate_with_vtable(self, known_class): @@ -774,14 +784,16 @@ strbox, ConstInt(index), charbox) def concat_strings(self, str1num, str2num): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_CONCAT) str1box = self.decode_box(str1num, REF) str2box = self.decode_box(str2num, REF) return self.metainterp.execute_and_record_varargs( rop.CALL, [ConstInt(func), str1box, str2box], calldescr) def slice_string(self, strnum, startnum, lengthnum): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_STR_SLICE) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_STR_SLICE) strbox = self.decode_box(strnum, REF) startbox = self.decode_box(startnum, INT) lengthbox = self.decode_box(lengthnum, INT) @@ -800,14 +812,16 @@ strbox, ConstInt(index), charbox) def concat_unicodes(self, str1num, str2num): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_CONCAT) str1box = self.decode_box(str1num, REF) str2box = self.decode_box(str2num, REF) return self.metainterp.execute_and_record_varargs( rop.CALL, [ConstInt(func), str1box, str2box], calldescr) def slice_unicode(self, strnum, startnum, lengthnum): - calldescr, func = callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE) + cic = self.metainterp.staticdata.callinfocollection + calldescr, func = cic.callinfo_for_oopspec(EffectInfo.OS_UNI_SLICE) strbox = self.decode_box(strnum, REF) startbox = self.decode_box(startnum, INT) lengthbox = self.decode_box(lengthnum, INT) @@ -903,8 +917,8 @@ def blackhole_from_resumedata(blackholeinterpbuilder, jitdriver_sd, storage, all_virtuals=None): - resumereader = ResumeDataDirectReader(blackholeinterpbuilder.cpu, storage, - all_virtuals) + resumereader = ResumeDataDirectReader(blackholeinterpbuilder.metainterp_sd, + storage, all_virtuals) vinfo = jitdriver_sd.virtualizable_info ginfo = jitdriver_sd.greenfield_info vrefinfo = blackholeinterpbuilder.metainterp_sd.virtualref_info @@ -939,7 +953,7 @@ return firstbh def force_from_resumedata(metainterp_sd, storage, vinfo, ginfo): - resumereader = ResumeDataDirectReader(metainterp_sd.cpu, storage) + resumereader = ResumeDataDirectReader(metainterp_sd, storage) resumereader.handling_async_forcing() vrefinfo = metainterp_sd.virtualref_info resumereader.consume_vref_and_vable(vrefinfo, vinfo, ginfo) @@ -953,8 +967,9 @@ # 1: in handle_async_forcing # 2: resuming from the GUARD_NOT_FORCED - def __init__(self, cpu, storage, all_virtuals=None): - self._init(cpu, storage) + def __init__(self, metainterp_sd, storage, all_virtuals=None): + self._init(metainterp_sd.cpu, storage) + self.callinfocollection = metainterp_sd.callinfocollection if all_virtuals is None: # common case self._prepare(storage) else: @@ -973,23 +988,24 @@ info = blackholeinterp.get_current_position_info() self._prepare_next_section(info) - def consume_virtualref_info(self, vrefinfo, nums, end): + def consume_virtualref_info(self, vrefinfo, numb, end): # we have to decode a list of references containing pairs # [..., virtual, vref, ...] stopping at 'end' assert (end & 1) == 0 for i in range(0, end, 2): - virtual = self.decode_ref(nums[i]) - vref = self.decode_ref(nums[i+1]) + virtual = self.decode_ref(numb.nums[i]) + vref = self.decode_ref(numb.nums[i+1]) # For each pair, we store the virtual inside the vref. vrefinfo.continue_tracing(vref, virtual) - def consume_vable_info(self, vinfo, nums): + def consume_vable_info(self, vinfo, numb): # we have to ignore the initial part of 'nums' (containing vrefs), # find the virtualizable from nums[-1], load all other values # from the CPU stack, and copy them into the virtualizable if vinfo is None: - return len(nums) - virtualizable = self.decode_ref(nums[-1]) + return len(numb.nums) + index = len(numb.nums) - 1 + virtualizable = self.decode_ref(numb.nums[index]) virtualizable = vinfo.cast_gcref_to_vtype(virtualizable) if self.resume_after_guard_not_forced == 1: # in the middle of handle_async_forcing() @@ -1001,7 +1017,7 @@ # is and stays 0. Note the call to reset_vable_token() in # warmstate.py. assert not virtualizable.vable_token - return vinfo.write_from_resume_data_partial(virtualizable, self, nums) + return vinfo.write_from_resume_data_partial(virtualizable, self, numb) def load_value_of_type(self, TYPE, tagged): from pypy.jit.metainterp.warmstate import specialize_value @@ -1018,12 +1034,12 @@ load_value_of_type._annspecialcase_ = 'specialize:arg(1)' def consume_vref_and_vable(self, vrefinfo, vinfo, ginfo): - nums = self.cur_numb.nums - self.cur_numb = self.cur_numb.prev + numb = self.cur_numb + self.cur_numb = numb.prev if self.resume_after_guard_not_forced != 2: - end_vref = self.consume_vable_info(vinfo, nums) + end_vref = self.consume_vable_info(vinfo, numb) if ginfo is not None: end_vref -= 1 - self.consume_virtualref_info(vrefinfo, nums, end_vref) + self.consume_virtualref_info(vrefinfo, numb, end_vref) def allocate_with_vtable(self, known_class): from pypy.jit.metainterp.executor import exec_new_with_vtable @@ -1047,7 +1063,8 @@ str2 = self.decode_ref(str2num) str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str1) str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str2) - funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_CONCAT) result = funcptr(str1, str2) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1056,7 +1073,8 @@ start = self.decode_int(startnum) length = self.decode_int(lengthnum) str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.STR), str) - funcptr = funcptr_for_oopspec(EffectInfo.OS_STR_SLICE) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_STR_SLICE) result = funcptr(str, start, start + length) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1072,7 +1090,8 @@ str2 = self.decode_ref(str2num) str1 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str1) str2 = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str2) - funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_CONCAT) result = funcptr(str1, str2) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1081,7 +1100,8 @@ start = self.decode_int(startnum) length = self.decode_int(lengthnum) str = lltype.cast_opaque_ptr(lltype.Ptr(rstr.UNICODE), str) - funcptr = funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE) + cic = self.callinfocollection + funcptr = cic.funcptr_for_oopspec(EffectInfo.OS_UNI_SLICE) result = funcptr(str, start, start + length) return lltype.cast_opaque_ptr(llmemory.GCREF, result) @@ -1172,8 +1192,9 @@ 'at', compute_unique_id(frameinfo)) frameinfo = frameinfo.prev numb = storage.rd_numb - while numb is not None: - debug_print('\tnumb', str([untag(i) for i in numb.nums]), + while numb: + debug_print('\tnumb', str([untag(numb.nums[i]) + for i in range(len(numb.nums))]), 'at', compute_unique_id(numb)) numb = numb.prev for const in storage.rd_consts: Modified: pypy/branch/jitypes2/pypy/jit/metainterp/test/test_compile.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/test/test_compile.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/test/test_compile.py Tue Nov 23 13:26:19 2010 @@ -85,7 +85,7 @@ metainterp.history.inputargs = loop.inputargs[:] # loop_tokens = [] - loop_token = compile_new_loop(metainterp, loop_tokens, [], 0) + loop_token = compile_new_loop(metainterp, loop_tokens, 0) assert loop_tokens == [loop_token] assert loop_token.number == 1 assert staticdata.globaldata.loopnumbering == 2 @@ -101,7 +101,7 @@ metainterp.history.operations = loop.operations[:] metainterp.history.inputargs = loop.inputargs[:] # - loop_token_2 = compile_new_loop(metainterp, loop_tokens, [], 0) + loop_token_2 = compile_new_loop(metainterp, loop_tokens, 0) assert loop_token_2 is loop_token assert loop_tokens == [loop_token] assert len(cpu.seen) == 0 Modified: pypy/branch/jitypes2/pypy/jit/metainterp/test/test_exception.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/test/test_exception.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/test/test_exception.py Tue Nov 23 13:26:19 2010 @@ -1,6 +1,6 @@ import py, sys from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin -from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE +from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE, dont_look_inside from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask from pypy.jit.codewriter.policy import StopAtXPolicy @@ -588,6 +588,33 @@ res = self.interp_operations(f, [99]) assert res == 21 + def test_bug_exc1_noexc_exc2(self): + myjitdriver = JitDriver(greens=[], reds=['i']) + @dont_look_inside + def rescall(i): + if i < 10: + raise KeyError + if i < 20: + return None + raise ValueError + def f(i): + while i < 30: + myjitdriver.can_enter_jit(i=i) + myjitdriver.jit_merge_point(i=i) + try: + rescall(i) + except KeyError: + assert i < 10 + except ValueError: + assert i >= 20 + else: + assert 10 <= i < 20 + i += 1 + return i + res = self.meta_interp(f, [0], inline=True) + assert res == 30 + + class MyError(Exception): def __init__(self, n): self.n = n Modified: pypy/branch/jitypes2/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/test/test_optimizeopt.py Tue Nov 23 13:26:19 2010 @@ -41,7 +41,7 @@ b1 = BoxInt() opt = optimizeopt.Optimizer(FakeMetaInterpStaticData(LLtypeMixin.cpu), None) - fdescr = ResumeGuardDescr(None, None) + fdescr = ResumeGuardDescr(None) op = ResOperation(rop.GUARD_TRUE, ['dummy'], None, descr=fdescr) # setup rd data fi0 = resume.FrameInfo(None, "code0", 11) @@ -51,12 +51,12 @@ # opt.store_final_boxes_in_guard(op) if op.getfailargs() == [b0, b1]: - assert fdescr.rd_numb.nums == [tag(1, TAGBOX)] - assert fdescr.rd_numb.prev.nums == [tag(0, TAGBOX)] + assert list(fdescr.rd_numb.nums) == [tag(1, TAGBOX)] + assert list(fdescr.rd_numb.prev.nums) == [tag(0, TAGBOX)] else: assert op.getfailargs() == [b1, b0] - assert fdescr.rd_numb.nums == [tag(0, TAGBOX)] - assert fdescr.rd_numb.prev.nums == [tag(1, TAGBOX)] + assert list(fdescr.rd_numb.nums) == [tag(0, TAGBOX)] + assert list(fdescr.rd_numb.prev.nums) == [tag(1, TAGBOX)] assert fdescr.rd_virtuals is None assert fdescr.rd_consts == [] @@ -264,6 +264,8 @@ metainterp_sd = FakeMetaInterpStaticData(self.cpu) if hasattr(self, 'vrefinfo'): metainterp_sd.virtualref_info = self.vrefinfo + if hasattr(self, 'callinfocollection'): + metainterp_sd.callinfocollection = self.callinfocollection optimize_loop_1(metainterp_sd, loop) # expected = self.parse(optops) @@ -787,8 +789,12 @@ i3 = call(i2, descr=nonwritedescr) jump(i1) # the exception is considered lost when we loop back """ + # note that 'guard_no_exception' at the very start must be kept + # around: bridges may start with one. (In case of loops we could + # remove it, but we probably don't care.) expected = """ [i] + guard_no_exception() [] i1 = int_add(i, 3) i2 = call(i1, descr=nonwritedescr) guard_no_exception() [i1, i2] @@ -4180,22 +4186,20 @@ # ---------- def optimize_strunicode_loop_extradescrs(self, ops, spectext, optops): from pypy.jit.metainterp.optimizeopt import string - def my_callinfo_for_oopspec(oopspecindex): - calldescrtype = type(LLtypeMixin.strequaldescr) - for value in LLtypeMixin.__dict__.values(): - if isinstance(value, calldescrtype): - if (value.get_extra_info() and - value.get_extra_info().oopspecindex == oopspecindex): - # returns 0 for 'func' in this test - return value, 0 - raise AssertionError("not found: oopspecindex=%d" % oopspecindex) + class FakeCallInfoCollection: + def callinfo_for_oopspec(self, oopspecindex): + calldescrtype = type(LLtypeMixin.strequaldescr) + for value in LLtypeMixin.__dict__.values(): + if isinstance(value, calldescrtype): + extra = value.get_extra_info() + if extra and extra.oopspecindex == oopspecindex: + # returns 0 for 'func' in this test + return value, 0 + raise AssertionError("not found: oopspecindex=%d" % + oopspecindex) # - saved = string.callinfo_for_oopspec - try: - string.callinfo_for_oopspec = my_callinfo_for_oopspec - self.optimize_strunicode_loop(ops, spectext, optops) - finally: - string.callinfo_for_oopspec = saved + self.callinfocollection = FakeCallInfoCollection() + self.optimize_strunicode_loop(ops, spectext, optops) def test_str_equal_noop1(self): ops = """ Modified: pypy/branch/jitypes2/pypy/jit/metainterp/test/test_recursive.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/test/test_recursive.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/test/test_recursive.py Tue Nov 23 13:26:19 2010 @@ -1142,6 +1142,19 @@ res = self.meta_interp(main, [], inline=True, trace_limit=tlimit) assert ''.join(res.chars) == 'ABCDEFGHIabcdefghijJ' * 5 + def test_no_duplicates_bug(self): + driver = JitDriver(greens = ['codeno'], reds = ['i'], + get_printable_location = lambda codeno: str(codeno)) + def portal(codeno, i): + while i > 0: + driver.can_enter_jit(codeno=codeno, i=i) + driver.jit_merge_point(codeno=codeno, i=i) + if codeno > 0: + break + portal(i, i) + i -= 1 + self.meta_interp(portal, [0, 10], inline=True) + class TestLLtype(RecursiveTests, LLJitMixin): pass Modified: pypy/branch/jitypes2/pypy/jit/metainterp/test/test_resoperation.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/test/test_resoperation.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/test/test_resoperation.py Tue Nov 23 13:26:19 2010 @@ -61,3 +61,10 @@ assert op.getarglist() == ['a', 'b'] assert op.result == 'c' assert op.getdescr() is mydescr + +def test_can_malloc(): + mydescr = AbstractDescr() + assert rop.ResOperation(rop.rop.NEW, [], 'b').can_malloc() + call = rop.ResOperation(rop.rop.CALL, ['a', 'b'], 'c', descr=mydescr) + assert call.can_malloc() + assert not rop.ResOperation(rop.rop.INT_ADD, ['a', 'b'], 'c').can_malloc() Modified: pypy/branch/jitypes2/pypy/jit/metainterp/test/test_resume.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/test/test_resume.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/test/test_resume.py Tue Nov 23 13:26:19 2010 @@ -51,6 +51,7 @@ class MyMetaInterp: _already_allocated_resume_virtuals = None + callinfocollection = None def __init__(self, cpu=None): if cpu is None: @@ -141,6 +142,13 @@ assert bh.written_f == expected_f +def Numbering(prev, nums): + numb = lltype.malloc(NUMBERING, len(nums)) + numb.prev = prev or lltype.nullptr(NUMBERING) + for i in range(len(nums)): + numb.nums[i] = nums[i] + return numb + def test_simple_read(): #b1, b2, b3 = [BoxInt(), BoxPtr(), BoxInt()] c1, c2, c3 = [ConstInt(111), ConstInt(222), ConstInt(333)] @@ -156,12 +164,12 @@ storage.rd_numb = numb # cpu = MyCPU([42, gcref1, -66]) - reader = ResumeDataDirectReader(cpu, storage) + metainterp = MyMetaInterp(cpu) + reader = ResumeDataDirectReader(metainterp, storage) _next_section(reader, 42, 111, gcrefnull, 42, gcref1) _next_section(reader, 222, 333) _next_section(reader, 42, gcref1, -66) # - metainterp = MyMetaInterp(cpu) reader = ResumeDataBoxReader(storage, metainterp) bi, br, bf = [None]*3, [None]*2, [None]*0 info = MyBlackholeInterp([lltype.Signed, lltype.Signed, @@ -193,7 +201,7 @@ storage.rd_numb = numb # cpu = MyCPU([]) - reader = ResumeDataDirectReader(cpu, storage) + reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage) _next_section(reader, 100) @@ -211,7 +219,7 @@ class FakeMetainterp(object): _already_allocated_resume_virtuals = None cpu = None - reader = ResumeDataDirectReader(None, FakeStorage()) + reader = ResumeDataDirectReader(MyMetaInterp(None), FakeStorage()) assert reader.force_all_virtuals() == ["allocated", reader.virtual_default] # ____________________________________________________________ @@ -390,15 +398,15 @@ assert fi1.pc == 3 def test_Numbering_create(): - l = [1, 2] + l = [rffi.r_short(1), rffi.r_short(2)] numb = Numbering(None, l) - assert numb.prev is None - assert numb.nums is l + assert not numb.prev + assert list(numb.nums) == l - l1 = ['b3'] + l1 = [rffi.r_short(3)] numb1 = Numbering(numb, l1) - assert numb1.prev is numb - assert numb1.nums is l1 + assert numb1.prev == numb + assert list(numb1.nums) == l1 def test_capture_resumedata(): b1, b2, b3 = [BoxInt(), BoxPtr(), BoxInt()] @@ -764,11 +772,12 @@ assert liveboxes == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b3: tag(2, TAGBOX)} - assert numb.nums == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), - tag(1, TAGINT)] - assert numb.prev.nums == [tag(0, TAGBOX), tag(1, TAGINT), tag(1, TAGBOX), - tag(0, TAGBOX), tag(2, TAGINT)] - assert numb.prev.prev is None + assert list(numb.nums) == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), + tag(1, TAGINT)] + assert list(numb.prev.nums) == [tag(0, TAGBOX), tag(1, TAGINT), + tag(1, TAGBOX), + tag(0, TAGBOX), tag(2, TAGINT)] + assert not numb.prev.prev numb2, liveboxes2, v = memo.number({}, snap2) assert v == 0 @@ -776,9 +785,9 @@ assert liveboxes2 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b3: tag(2, TAGBOX)} assert liveboxes2 is not liveboxes - assert numb2.nums == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb2.prev is numb.prev + assert list(numb2.nums) == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), + tag(3, TAGINT)] + assert numb2.prev == numb.prev env3 = [c3, b3, b1, c3] snap3 = Snapshot(snap, env3) @@ -799,9 +808,9 @@ assert v == 0 assert liveboxes3 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX)} - assert numb3.nums == [tag(3, TAGINT), tag(4, TAGINT), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb3.prev is numb.prev + assert list(numb3.nums) == [tag(3, TAGINT), tag(4, TAGINT), tag(0, TAGBOX), + tag(3, TAGINT)] + assert numb3.prev == numb.prev # virtual env4 = [c3, b4, b1, c3] @@ -812,9 +821,9 @@ assert liveboxes4 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b4: tag(0, TAGVIRTUAL)} - assert numb4.nums == [tag(3, TAGINT), tag(0, TAGVIRTUAL), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb4.prev is numb.prev + assert list(numb4.nums) == [tag(3, TAGINT), tag(0, TAGVIRTUAL), + tag(0, TAGBOX), tag(3, TAGINT)] + assert numb4.prev == numb.prev env5 = [b1, b4, b5] snap5 = Snapshot(snap4, env5) @@ -825,9 +834,9 @@ assert liveboxes5 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b4: tag(0, TAGVIRTUAL), b5: tag(1, TAGVIRTUAL)} - assert numb5.nums == [tag(0, TAGBOX), tag(0, TAGVIRTUAL), - tag(1, TAGVIRTUAL)] - assert numb5.prev is numb4 + assert list(numb5.nums) == [tag(0, TAGBOX), tag(0, TAGVIRTUAL), + tag(1, TAGVIRTUAL)] + assert numb5.prev == numb4 def test_ResumeDataLoopMemo_number_boxes(): memo = ResumeDataLoopMemo(FakeMetaInterpStaticData()) @@ -925,7 +934,7 @@ liveboxes = modifier.finish({}) assert storage.rd_snapshot is None cpu = MyCPU([]) - reader = ResumeDataDirectReader(cpu, storage) + reader = ResumeDataDirectReader(MyMetaInterp(cpu), storage) _next_section(reader, sys.maxint, 2**16, -65) _next_section(reader, 2, 3) _next_section(reader, sys.maxint, 1, sys.maxint, 2**16) Modified: pypy/branch/jitypes2/pypy/jit/metainterp/test/test_virtualref.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/test/test_virtualref.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/test/test_virtualref.py Tue Nov 23 13:26:19 2010 @@ -88,7 +88,11 @@ cpu.get_latest_value_int = lambda i:guard_op.getfailargs()[i].getint() cpu.get_latest_value_ref = lambda i:guard_op.getfailargs()[i].getref_base() cpu.clear_latest_values = lambda count: None - resumereader = ResumeDataDirectReader(cpu, guard_op.getdescr()) + class FakeMetaInterpSd: + callinfocollection = None + FakeMetaInterpSd.cpu = cpu + resumereader = ResumeDataDirectReader(FakeMetaInterpSd(), + guard_op.getdescr()) vrefinfo = self.metainterp.staticdata.virtualref_info lst = [] vrefinfo.continue_tracing = lambda vref, virtual: \ Modified: pypy/branch/jitypes2/pypy/jit/metainterp/test/test_ztranslation.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/test/test_ztranslation.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/test/test_ztranslation.py Tue Nov 23 13:26:19 2010 @@ -79,7 +79,7 @@ res = ll_meta_interp(main, [40, 5], CPUClass=self.CPUClass, type_system=self.type_system) assert res == main(40, 5) - res = rpython_ll_meta_interp(main, [40, 5], loops=2, + res = rpython_ll_meta_interp(main, [40, 5], CPUClass=self.CPUClass, type_system=self.type_system, optimizer=OPTIMIZER_FULL, @@ -120,7 +120,7 @@ res = ll_meta_interp(main, [40], CPUClass=self.CPUClass, type_system=self.type_system) assert res == main(40) - res = rpython_ll_meta_interp(main, [40], loops=2, CPUClass=self.CPUClass, + res = rpython_ll_meta_interp(main, [40], CPUClass=self.CPUClass, type_system=self.type_system, optimizer=OPTIMIZER_FULL, ProfilerClass=Profiler) Modified: pypy/branch/jitypes2/pypy/jit/metainterp/virtualizable.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/virtualizable.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/virtualizable.py Tue Nov 23 13:26:19 2010 @@ -100,48 +100,48 @@ i = i + 1 assert len(boxes) == i + 1 # - def write_from_resume_data_partial(virtualizable, reader, nums): + def write_from_resume_data_partial(virtualizable, reader, numb): # Load values from the reader (see resume.py) described by # the list of numbers 'nums', and write them in their proper # place in the 'virtualizable'. This works from the end of # the list and returns the index in 'nums' of the start of # the virtualizable data found, allowing the caller to do # further processing with the start of the list. - i = len(nums) - 1 + i = len(numb.nums) - 1 assert i >= 0 for ARRAYITEMTYPE, fieldname in unroll_array_fields_rev: lst = getattr(virtualizable, fieldname) for j in range(getlength(lst)-1, -1, -1): i -= 1 assert i >= 0 - x = reader.load_value_of_type(ARRAYITEMTYPE, nums[i]) + x = reader.load_value_of_type(ARRAYITEMTYPE, numb.nums[i]) setarrayitem(lst, j, x) for FIELDTYPE, fieldname in unroll_static_fields_rev: i -= 1 assert i >= 0 - x = reader.load_value_of_type(FIELDTYPE, nums[i]) + x = reader.load_value_of_type(FIELDTYPE, numb.nums[i]) setattr(virtualizable, fieldname, x) return i # - def load_list_of_boxes(virtualizable, reader, nums): + def load_list_of_boxes(virtualizable, reader, numb): # Uses 'virtualizable' only to know the length of the arrays; # does not write anything into it. The returned list is in # the format expected of virtualizable_boxes, so it ends in # the virtualizable itself. - i = len(nums) - 1 + i = len(numb.nums) - 1 assert i >= 0 - boxes = [reader.decode_box_of_type(self.VTYPEPTR, nums[i])] + boxes = [reader.decode_box_of_type(self.VTYPEPTR, numb.nums[i])] for ARRAYITEMTYPE, fieldname in unroll_array_fields_rev: lst = getattr(virtualizable, fieldname) for j in range(getlength(lst)-1, -1, -1): i -= 1 assert i >= 0 - box = reader.decode_box_of_type(ARRAYITEMTYPE, nums[i]) + box = reader.decode_box_of_type(ARRAYITEMTYPE,numb.nums[i]) boxes.append(box) for FIELDTYPE, fieldname in unroll_static_fields_rev: i -= 1 assert i >= 0 - box = reader.decode_box_of_type(FIELDTYPE, nums[i]) + box = reader.decode_box_of_type(FIELDTYPE, numb.nums[i]) boxes.append(box) boxes.reverse() return boxes Modified: pypy/branch/jitypes2/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/branch/jitypes2/pypy/jit/metainterp/warmspot.py (original) +++ pypy/branch/jitypes2/pypy/jit/metainterp/warmspot.py Tue Nov 23 13:26:19 2010 @@ -98,8 +98,7 @@ repeat -= 1 return res -def rpython_ll_meta_interp(function, args, backendopt=True, - loops='not used right now', **kwds): +def rpython_ll_meta_interp(function, args, backendopt=True, **kwds): return ll_meta_interp(function, args, backendopt=backendopt, translate_support_code=True, **kwds) Modified: pypy/branch/jitypes2/pypy/jit/tool/pypytrace-mode.el ============================================================================== --- pypy/branch/jitypes2/pypy/jit/tool/pypytrace-mode.el (original) +++ pypy/branch/jitypes2/pypy/jit/tool/pypytrace-mode.el Tue Nov 23 13:26:19 2010 @@ -26,7 +26,7 @@ ("<.*FieldDescr \\([^ ]*\\)" (1 'font-lock-variable-name-face)) ;; comment out debug_merge_point, but then highlight specific part of it ("^debug_merge_point.*" . font-lock-comment-face) - ("^\\(debug_merge_point\\).*code object\\(.*\\), file \\('.*'\\), \\(line .*\\)> \\(.*\\)')" + ("^\\(debug_merge_point\\).*code object\\(.*\\), file \\('.*'\\), \\(line .*\\)> \\(.*\\)" (1 'compilation-warning t) (2 'escape-glyph t) (3 'font-lock-string-face t) Modified: pypy/branch/jitypes2/pypy/module/__pypy__/__init__.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/__pypy__/__init__.py (original) +++ pypy/branch/jitypes2/pypy/module/__pypy__/__init__.py Tue Nov 23 13:26:19 2010 @@ -23,6 +23,9 @@ 'interp_magic.method_cache_counter') self.extra_interpdef('reset_method_cache_counter', 'interp_magic.reset_method_cache_counter') + if self.space.config.objspace.std.withmapdict: + self.extra_interpdef('mapdict_cache_counter', + 'interp_magic.mapdict_cache_counter') PYC_MAGIC = get_pyc_magic(self.space) self.extra_interpdef('PYC_MAGIC', 'space.wrap(%d)' % PYC_MAGIC) # Modified: pypy/branch/jitypes2/pypy/module/__pypy__/interp_magic.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/__pypy__/interp_magic.py (original) +++ pypy/branch/jitypes2/pypy/module/__pypy__/interp_magic.py Tue Nov 23 13:26:19 2010 @@ -2,6 +2,7 @@ from pypy.interpreter.gateway import ObjSpace from pypy.rlib.objectmodel import we_are_translated from pypy.objspace.std.typeobject import MethodCache +from pypy.objspace.std.mapdict import IndexCache def internal_repr(space, w_object): return space.wrap('%r' % (w_object,)) @@ -36,4 +37,17 @@ cache = space.fromcache(MethodCache) cache.misses = {} cache.hits = {} - + if space.config.objspace.std.withmapdict: + cache = space.fromcache(IndexCache) + cache.misses = {} + cache.hits = {} + +def mapdict_cache_counter(space, name): + """Return a tuple (index_cache_hits, index_cache_misses) for lookups + in the mapdict cache with the given attribute name.""" + assert space.config.objspace.std.withmethodcachecounter + assert space.config.objspace.std.withmapdict + cache = space.fromcache(IndexCache) + return space.newtuple([space.newint(cache.hits.get(name, 0)), + space.newint(cache.misses.get(name, 0))]) +mapdict_cache_counter.unwrap_spec = [ObjSpace, str] Modified: pypy/branch/jitypes2/pypy/module/_rawffi/test/test__rawffi.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/_rawffi/test/test__rawffi.py (original) +++ pypy/branch/jitypes2/pypy/module/_rawffi/test/test__rawffi.py Tue Nov 23 13:26:19 2010 @@ -296,6 +296,7 @@ assert _rawffi.charp2string(res[0]) is None arg1.free() arg2.free() + a.free() def test_raw_callable(self): import _rawffi Modified: pypy/branch/jitypes2/pypy/module/_rawffi/test/test_nested.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/_rawffi/test/test_nested.py (original) +++ pypy/branch/jitypes2/pypy/module/_rawffi/test/test_nested.py Tue Nov 23 13:26:19 2010 @@ -107,7 +107,6 @@ assert S.fieldoffset('x') == 0 assert S.fieldoffset('ar') == A5alignment s = S() - s = S() s.x = 'G' raises(TypeError, 's.ar') assert s.fieldaddress('ar') == s.buffer + S.fieldoffset('ar') Modified: pypy/branch/jitypes2/pypy/module/array/interp_array.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/array/interp_array.py (original) +++ pypy/branch/jitypes2/pypy/module/array/interp_array.py Tue Nov 23 13:26:19 2010 @@ -27,7 +27,7 @@ typecode = typecode[0] if space.is_w(w_cls, space.gettypeobject(W_ArrayBase.typedef)): - if len(w_args.keywords_w) > 0: + if w_args.keywords: # XXX this might be forbidden fishing msg = 'array.array() does not take keyword arguments' raise OperationError(space.w_TypeError, space.wrap(msg)) Modified: pypy/branch/jitypes2/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/cpyext/api.py (original) +++ pypy/branch/jitypes2/pypy/module/cpyext/api.py Tue Nov 23 13:26:19 2010 @@ -226,7 +226,7 @@ def unwrapper(space, *args): from pypy.module.cpyext.pyobject import Py_DecRef from pypy.module.cpyext.pyobject import make_ref, from_ref - from pypy.module.cpyext.pyobject import BorrowPair + from pypy.module.cpyext.pyobject import Reference newargs = () to_decref = [] assert len(args) == len(api_function.argtypes) @@ -270,8 +270,8 @@ return api_function.error_value if res is None: return None - elif isinstance(res, BorrowPair): - return res.w_borrowed + elif isinstance(res, Reference): + return res.get_wrapped(space) else: return res finally: @@ -473,7 +473,7 @@ @specialize.ll() def wrapper(*args): from pypy.module.cpyext.pyobject import make_ref, from_ref - from pypy.module.cpyext.pyobject import BorrowPair + from pypy.module.cpyext.pyobject import Reference # we hope that malloc removal removes the newtuple() that is # inserted exactly here by the varargs specializer llop.gc_stack_bottom(lltype.Void) # marker for trackgcroot.py @@ -525,7 +525,7 @@ elif is_PyObject(callable.api_func.restype): if result is None: retval = make_ref(space, None) - elif isinstance(result, BorrowPair): + elif isinstance(result, Reference): retval = result.get_ref(space) elif not rffi._isllptr(result): retval = rffi.cast(callable.api_func.restype, @@ -908,8 +908,10 @@ from pypy.rlib import rdynload try: ll_libname = rffi.str2charp(path) - dll = rdynload.dlopen(ll_libname) - lltype.free(ll_libname, flavor='raw') + try: + dll = rdynload.dlopen(ll_libname) + finally: + lltype.free(ll_libname, flavor='raw') except rdynload.DLOpenError, e: raise operationerrfmt( space.w_ImportError, Modified: pypy/branch/jitypes2/pypy/module/cpyext/cdatetime.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/cpyext/cdatetime.py (original) +++ pypy/branch/jitypes2/pypy/module/cpyext/cdatetime.py Tue Nov 23 13:26:19 2010 @@ -4,7 +4,7 @@ from pypy.module.cpyext.api import ( cpython_api, CANNOT_FAIL, cpython_struct, PyObjectFields) from pypy.module.cpyext.import_ import PyImport_Import -from pypy.module.cpyext.typeobject import PyTypeObjectPtr +from pypy.module.cpyext.typeobject import PyTypeObjectPtr, render_immortal from pypy.module.cpyext.state import State from pypy.interpreter.error import OperationError from pypy.tool.sourcetools import func_renamer @@ -22,25 +22,34 @@ @cpython_api([], lltype.Ptr(PyDateTime_CAPI), error=lltype.nullptr(PyDateTime_CAPI)) def _PyDateTime_Import(space): - datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw') + datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw', + track_allocation=False) if not we_are_translated(): datetimeAPI_dealloc(space) space.fromcache(State).datetimeAPI = datetimeAPI w_datetime = PyImport_Import(space, space.wrap("datetime")) + w_type = space.getattr(w_datetime, space.wrap("date")) datetimeAPI.c_DateType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DateType, w_type) + w_type = space.getattr(w_datetime, space.wrap("datetime")) datetimeAPI.c_DateTimeType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DateTimeType, w_type) + w_type = space.getattr(w_datetime, space.wrap("time")) datetimeAPI.c_TimeType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_TimeType, w_type) + w_type = space.getattr(w_datetime, space.wrap("timedelta")) datetimeAPI.c_DeltaType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DeltaType, w_type) return datetimeAPI Modified: pypy/branch/jitypes2/pypy/module/cpyext/presetup.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/cpyext/presetup.py (original) +++ pypy/branch/jitypes2/pypy/module/cpyext/presetup.py Tue Nov 23 13:26:19 2010 @@ -21,6 +21,8 @@ from pypy.conftest import gettestobjspace from pypy.module.cpyext.api import build_bridge +from pypy.module.imp.importing import get_so_extension + usemodules = ['cpyext', 'thread'] if sys.platform == 'win32': usemodules.append('_winreg') # necessary in distutils @@ -35,6 +37,7 @@ def patch_distutils(): sysconfig.get_python_inc = get_python_inc + sysconfig.get_config_vars()['SO'] = get_so_extension(space) patch_distutils() Modified: pypy/branch/jitypes2/pypy/module/cpyext/pyobject.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/cpyext/pyobject.py (original) +++ pypy/branch/jitypes2/pypy/module/cpyext/pyobject.py Tue Nov 23 13:26:19 2010 @@ -424,7 +424,18 @@ state = space.fromcache(RefcountState) return state.make_borrowed(w_container, w_borrowed) -class BorrowPair: +class Reference: + def __init__(self, pyobj): + assert not isinstance(pyobj, W_Root) + self.pyobj = pyobj + + def get_ref(self, space): + return self.pyobj + + def get_wrapped(self, space): + return from_ref(space, self.pyobj) + +class BorrowPair(Reference): """ Delays the creation of a borrowed reference. """ @@ -435,6 +446,9 @@ def get_ref(self, space): return make_borrowed_ref(space, self.w_container, self.w_borrowed) + def get_wrapped(self, space): + return self.w_borrowed + def borrow_from(container, borrowed): return BorrowPair(container, borrowed) Modified: pypy/branch/jitypes2/pypy/module/cpyext/test/test_borrow.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/cpyext/test/test_borrow.py (original) +++ pypy/branch/jitypes2/pypy/module/cpyext/test/test_borrow.py Tue Nov 23 13:26:19 2010 @@ -31,6 +31,7 @@ g = PyTuple_GetItem(t, 0); // borrows reference again printf("Refcnt4: %i\\n", f->ob_refcnt); printf("COMPARE: %i\\n", f == g); + fflush(stdout); Py_DECREF(t); Py_RETURN_TRUE; """), Modified: pypy/branch/jitypes2/pypy/module/cpyext/test/test_cpyext.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/cpyext/test/test_cpyext.py (original) +++ pypy/branch/jitypes2/pypy/module/cpyext/test/test_cpyext.py Tue Nov 23 13:26:19 2010 @@ -42,7 +42,7 @@ raises(ImportError, cpyext.load_module, "missing.file", "foo") raises(ImportError, cpyext.load_module, self.libc, "invalid.function") -def compile_module(modname, **kwds): +def compile_module(space, modname, **kwds): """ Build an extension module and return the filename of the resulting native code file. @@ -65,10 +65,8 @@ [], eci, outputfilename=str(dirname/modname), standalone=False) - if sys.platform == 'win32': - pydname = soname.new(purebasename=modname, ext='.pyd') - else: - pydname = soname.new(purebasename=modname, ext='.so') + from pypy.module.imp.importing import get_so_extension + pydname = soname.new(purebasename=modname, ext=get_so_extension(space)) soname.rename(pydname) return str(pydname) @@ -153,7 +151,7 @@ kwds["link_files"] = [str(api_library + '.so')] if sys.platform == 'linux2': kwds["compile_extra"]=["-Werror=implicit-function-declaration"] - return compile_module(name, **kwds) + return compile_module(self.space, name, **kwds) def import_module(self, name, init=None, body='', load_it=True, filename=None): Modified: pypy/branch/jitypes2/pypy/module/cpyext/typeobject.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/cpyext/typeobject.py (original) +++ pypy/branch/jitypes2/pypy/module/cpyext/typeobject.py Tue Nov 23 13:26:19 2010 @@ -182,10 +182,10 @@ subtype = rffi.cast(PyTypeObjectPtr, make_ref(space, w_subtype)) try: - obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds) + w_obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds) finally: Py_DecRef(space, w_subtype) - return obj + return w_obj @specialize.memo() def get_new_method_def(space): @@ -193,10 +193,14 @@ if state.new_method_def: return state.new_method_def from pypy.module.cpyext.modsupport import PyMethodDef - ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True) + ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True, + immortal=True) ptr.c_ml_name = rffi.str2charp("__new__") + lltype.render_immortal(ptr.c_ml_name) rffi.setintfield(ptr, 'c_ml_flags', METH_VARARGS | METH_KEYWORDS) - ptr.c_ml_doc = rffi.str2charp("T.__new__(S, ...) -> a new object with type S, a subtype of T") + ptr.c_ml_doc = rffi.str2charp( + "T.__new__(S, ...) -> a new object with type S, a subtype of T") + lltype.render_immortal(ptr.c_ml_doc) state.new_method_def = ptr return ptr @@ -429,6 +433,12 @@ finish_type_1(space, pto) finish_type_2(space, pto, w_type) + if space.type(w_type).is_cpytype(): + # XXX Types with a C metatype are never freed, try to see why... + render_immortal(pto, w_type) + lltype.render_immortal(pto) + lltype.render_immortal(pto.c_tp_name) + pto.c_tp_basicsize = rffi.sizeof(typedescr.basestruct) if pto.c_tp_base: if pto.c_tp_base.c_tp_basicsize > pto.c_tp_basicsize: @@ -534,12 +544,25 @@ w_obj.ready() finish_type_2(space, py_type, w_obj) + render_immortal(py_type, w_obj) state = space.fromcache(RefcountState) state.non_heaptypes_w.append(w_obj) return w_obj +def render_immortal(py_type, w_obj): + lltype.render_immortal(py_type.c_tp_bases) + lltype.render_immortal(py_type.c_tp_mro) + + assert isinstance(w_obj, W_TypeObject) + if w_obj.is_cpytype(): + lltype.render_immortal(py_type.c_tp_dict) + else: + lltype.render_immortal(py_type.c_tp_name) + if not w_obj.is_cpytype() and w_obj.is_heaptype(): + lltype.render_immortal(py_type) + def finish_type_1(space, pto): """ Sets up tp_bases, necessary before creating the interpreter type. Modified: pypy/branch/jitypes2/pypy/module/gc/__init__.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/gc/__init__.py (original) +++ pypy/branch/jitypes2/pypy/module/gc/__init__.py Tue Nov 23 13:26:19 2010 @@ -29,6 +29,7 @@ 'get_referents': 'referents.get_referents', 'get_referrers': 'referents.get_referrers', '_dump_rpy_heap': 'referents._dump_rpy_heap', + 'get_typeids_z': 'referents.get_typeids_z', 'GcRef': 'referents.W_GcRef', }) MixedModule.__init__(self, space, w_name) Modified: pypy/branch/jitypes2/pypy/module/gc/app_referents.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/gc/app_referents.py (original) +++ pypy/branch/jitypes2/pypy/module/gc/app_referents.py Tue Nov 23 13:26:19 2010 @@ -14,11 +14,25 @@ and [addr1]..[addrn] are addresses of other objects that this object points to. The full dump is a list of such objects, with a marker [0][0][0][-1] inserted after all GC roots, before all non-roots. + + If the argument is a filename and the 'zlib' module is available, + we also write a 'typeids.txt' in the same directory, if none exists. """ if isinstance(file, str): f = open(file, 'wb') gc._dump_rpy_heap(f.fileno()) f.close() + try: + import zlib, os + except ImportError: + pass + else: + filename2 = os.path.join(os.path.dirname(file), 'typeids.txt') + if not os.path.exists(filename2): + data = zlib.decompress(gc.get_typeids_z()) + f = open(filename2, 'wb') + f.write(data) + f.close() else: if isinstance(file, int): fd = file Modified: pypy/branch/jitypes2/pypy/module/gc/interp_gc.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/gc/interp_gc.py (original) +++ pypy/branch/jitypes2/pypy/module/gc/interp_gc.py Tue Nov 23 13:26:19 2010 @@ -10,6 +10,10 @@ from pypy.objspace.std.typeobject import MethodCache cache = space.fromcache(MethodCache) cache.clear() + if space.config.objspace.std.withmapdict: + from pypy.objspace.std.mapdict import IndexCache + cache = space.fromcache(IndexCache) + cache.clear() rgc.collect() return space.wrap(0) Modified: pypy/branch/jitypes2/pypy/module/gc/referents.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/gc/referents.py (original) +++ pypy/branch/jitypes2/pypy/module/gc/referents.py Tue Nov 23 13:26:19 2010 @@ -177,3 +177,9 @@ if not ok: raise missing_operation(space) _dump_rpy_heap.unwrap_spec = [ObjSpace, int] + +def get_typeids_z(space): + a = rgc.get_typeids_z() + s = ''.join([a[i] for i in range(len(a))]) + return space.wrap(s) +get_typeids_z.unwrap_spec = [ObjSpace] Modified: pypy/branch/jitypes2/pypy/module/gc/test/test_gc.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/gc/test/test_gc.py (original) +++ pypy/branch/jitypes2/pypy/module/gc/test/test_gc.py Tue Nov 23 13:26:19 2010 @@ -117,6 +117,33 @@ pass C().f() # Fill the method cache rlist.append(weakref.ref(C)) + for i in range(10): + f() + gc.collect() # the classes C should all go away here + # the last class won't go in mapdict, as long as the code object of f + # is around + rlist.pop() + for r in rlist: + assert r() is None + +class AppTestGcMapDictIndexCache(AppTestGcMethodCache): + def setup_class(cls): + cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True, + "objspace.std.withmapdict": True}) + + + def test_clear_index_cache(self): + import gc, weakref + rlist = [] + def f(): + class C(object): + def f(self): + pass + c = C() + c.x = 1 + getattr(c, "x") # fill the index cache without using the local cache + getattr(c, "x") + rlist.append(weakref.ref(C)) for i in range(5): f() gc.collect() # the classes C should all go away here Modified: pypy/branch/jitypes2/pypy/module/imp/importing.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/imp/importing.py (original) +++ pypy/branch/jitypes2/pypy/module/imp/importing.py Tue Nov 23 13:26:19 2010 @@ -12,7 +12,7 @@ from pypy.rlib import streamio, jit from pypy.rlib.streamio import StreamErrors from pypy.rlib.rarithmetic import intmask -from pypy.rlib.objectmodel import we_are_translated +from pypy.rlib.objectmodel import we_are_translated, specialize SEARCH_ERROR = 0 PY_SOURCE = 1 @@ -25,10 +25,26 @@ # PY_CODERESOURCE = 8 IMP_HOOK = 9 -if sys.platform.startswith('win'): - so_extension = ".pyd" +if sys.platform == 'win32': + SO = ".pyd" else: - so_extension = ".so" + SO = ".so" +DEFAULT_SOABI = 'pypy-14' + + at specialize.memo() +def get_so_extension(space): + if space.config.objspace.soabi is not None: + soabi = space.config.objspace.soabi + else: + soabi = DEFAULT_SOABI + + if not soabi: + return SO + + if not space.config.translating: + soabi += 'i' + + return '.' + soabi + SO def find_modtype(space, filepart): """Check which kind of module to import for the given filepart, @@ -53,6 +69,7 @@ return PY_COMPILED, ".pyc", "rb" if space.config.objspace.usemodules.cpyext: + so_extension = get_so_extension(space) pydfile = filepart + so_extension if os.path.exists(pydfile) and case_ok(pydfile): return C_EXTENSION, so_extension, "rb" Modified: pypy/branch/jitypes2/pypy/module/imp/interp_imp.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/imp/interp_imp.py (original) +++ pypy/branch/jitypes2/pypy/module/imp/interp_imp.py Tue Nov 23 13:26:19 2010 @@ -8,10 +8,16 @@ def get_suffixes(space): w = space.wrap - return space.newlist([ + suffixes_w = [] + if space.config.objspace.usemodules.cpyext: + suffixes_w.append( + space.newtuple([w(importing.get_so_extension(space)), + w('rb'), w(importing.C_EXTENSION)])) + suffixes_w.extend([ space.newtuple([w('.py'), w('U'), w(importing.PY_SOURCE)]), space.newtuple([w('.pyc'), w('rb'), w(importing.PY_COMPILED)]), ]) + return space.newlist(suffixes_w) def get_magic(space): x = importing.get_pyc_magic(space) Modified: pypy/branch/jitypes2/pypy/module/imp/test/test_app.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/imp/test/test_app.py (original) +++ pypy/branch/jitypes2/pypy/module/imp/test/test_app.py Tue Nov 23 13:26:19 2010 @@ -47,6 +47,9 @@ elif mode == self.imp.PY_COMPILED: assert suffix in ('.pyc', '.pyo') assert type == 'rb' + elif mode == self.imp.C_EXTENSION: + assert suffix.endswith(('.pyd', '.so')) + assert type == 'rb' def test_obscure_functions(self): Modified: pypy/branch/jitypes2/pypy/module/imp/test/test_import.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/imp/test/test_import.py (original) +++ pypy/branch/jitypes2/pypy/module/imp/test/test_import.py Tue Nov 23 13:26:19 2010 @@ -473,6 +473,17 @@ except ImportError: pass +class TestAbi: + def test_abi_tag(self): + space1 = gettestobjspace(soabi='TEST') + space2 = gettestobjspace(soabi='') + if sys.platform == 'win32': + assert importing.get_so_extension(space1) == '.TESTi.pyd' + assert importing.get_so_extension(space2) == '.pyd' + else: + assert importing.get_so_extension(space1) == '.TESTi.so' + assert importing.get_so_extension(space2) == '.so' + def _getlong(data): x = marshal.dumps(data) return x[-4:] Modified: pypy/branch/jitypes2/pypy/module/posix/interp_posix.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/posix/interp_posix.py (original) +++ pypy/branch/jitypes2/pypy/module/posix/interp_posix.py Tue Nov 23 13:26:19 2010 @@ -454,7 +454,8 @@ self.w_environ = space.newdict() if _WIN: self.cryptProviderPtr = lltype.malloc( - rffi.CArray(HCRYPTPROV), 1, zero=True, flavor='raw') + rffi.CArray(HCRYPTPROV), 1, zero=True, + flavor='raw', immortal=True) def startup(self, space): _convertenviron(space, self.w_environ) def _freeze_(self): Modified: pypy/branch/jitypes2/pypy/module/pypyjit/__init__.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/pypyjit/__init__.py (original) +++ pypy/branch/jitypes2/pypy/module/pypyjit/__init__.py Tue Nov 23 13:26:19 2010 @@ -15,6 +15,5 @@ # add the 'defaults' attribute from pypy.rlib.jit import PARAMETERS space = self.space - # XXX this is not really the default compiled into a pypy-c-jit XXX w_obj = space.wrap(PARAMETERS) space.setattr(space.wrap(self), space.wrap('defaults'), w_obj) Modified: pypy/branch/jitypes2/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jitypes2/pypy/module/pypyjit/test/test_pypy_c.py Tue Nov 23 13:26:19 2010 @@ -377,10 +377,75 @@ ([1000], 49500), ([10000], 495000), ([100000], 4950000)) - assert len(self.loops) == 2 + assert len(self.loops) == 3 op, = self.get_by_bytecode("CALL_FUNCTION_KW") # XXX a bit too many guards, but better than before - assert len(op.get_opnames("guard")) <= 10 + assert len(op.get_opnames("guard")) <= 12 + + def test_stararg_virtual(self): + self.run_source(''' + d = {} + + def g(*args): + return len(args) + def h(a, b, c): + return c + + def main(x): + s = 0 + for i in range(x): + l = [i, x, 2] + s += g(*l) + s += h(*l) + s += g(i, x, 2) + for i in range(x): + l = [x, 2] + s += g(i, *l) + s += h(i, *l) + return s + ''', 100000, ([100], 1300), + ([1000], 13000), + ([10000], 130000), + ([100000], 1300000)) + assert len(self.loops) == 2 + ops = self.get_by_bytecode("CALL_FUNCTION_VAR") + assert len(ops) == 4 + for op in ops: + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 + + ops = self.get_by_bytecode("CALL_FUNCTION") + for op in ops: + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 + + def test_stararg(self): + self.run_source(''' + d = {} + + def g(*args): + return args[-1] + def h(*args): + return len(args) + + def main(x): + s = 0 + l = [] + i = 0 + while i < x: + l.append(1) + s += g(*l) + i = h(*l) + return s + ''', 100000, ([100], 100), + ([1000], 1000), + ([2000], 2000), + ([4000], 4000)) + assert len(self.loops) == 1 + ops = self.get_by_bytecode("CALL_FUNCTION_VAR") + for op in ops: + assert len(op.get_opnames("new_with_vtable")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 def test_virtual_instance(self): self.run_source(''' Modified: pypy/branch/jitypes2/pypy/module/rctime/interp_time.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/rctime/interp_time.py (original) +++ pypy/branch/jitypes2/pypy/module/rctime/interp_time.py Tue Nov 23 13:26:19 2010 @@ -69,7 +69,7 @@ CLOCKS_PER_SEC = cConfig.CLOCKS_PER_SEC clock_t = cConfig.clock_t tm = cConfig.tm -glob_buf = lltype.malloc(tm, flavor='raw', zero=True) +glob_buf = lltype.malloc(tm, flavor='raw', zero=True, immortal=True) if cConfig.has_gettimeofday: c_gettimeofday = external('gettimeofday', [rffi.VOIDP, rffi.VOIDP], rffi.INT) Modified: pypy/branch/jitypes2/pypy/module/sys/state.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/sys/state.py (original) +++ pypy/branch/jitypes2/pypy/module/sys/state.py Tue Nov 23 13:26:19 2010 @@ -33,6 +33,8 @@ raise OSError(errno.ENOTDIR, path) +platform = sys.platform + def getinitialpath(prefix): from pypy.module.sys.version import CPYTHON_VERSION dirname = '%d.%d.%d' % (CPYTHON_VERSION[0], @@ -51,6 +53,15 @@ importlist.append(lib_pypy) importlist.append(python_std_lib_modified) importlist.append(python_std_lib) + # + # List here the extra platform-specific paths. + if platform != 'win32': + importlist.append(os.path.join(python_std_lib, 'plat-'+platform)) + if platform == 'darwin': + platmac = os.path.join(python_std_lib, 'plat-mac') + importlist.append(platmac) + importlist.append(os.path.join(platmac, 'lib-scriptpackages')) + # return importlist def pypy_initial_path(space, srcdir): Modified: pypy/branch/jitypes2/pypy/module/sys/test/test_initialpath.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/sys/test/test_initialpath.py (original) +++ pypy/branch/jitypes2/pypy/module/sys/test/test_initialpath.py Tue Nov 23 13:26:19 2010 @@ -15,4 +15,5 @@ def test_stdlib_in_prefix(tmpdir): dirs = build_hierarchy(tmpdir) path = getinitialpath(str(tmpdir)) - assert path == map(str, dirs) + # we get at least 'dirs', and maybe more (e.g. plat-linux2) + assert path[:len(dirs)] == map(str, dirs) Modified: pypy/branch/jitypes2/pypy/objspace/flow/model.py ============================================================================== --- pypy/branch/jitypes2/pypy/objspace/flow/model.py (original) +++ pypy/branch/jitypes2/pypy/objspace/flow/model.py Tue Nov 23 13:26:19 2010 @@ -355,7 +355,7 @@ return "%r = %s(%s)" % (self.result, self.opname, ", ".join(map(repr, self.args))) -class Atom: +class Atom(object): def __init__(self, name): self.__name__ = name # make save_global happy def __repr__(self): Modified: pypy/branch/jitypes2/pypy/objspace/std/callmethod.py ============================================================================== --- pypy/branch/jitypes2/pypy/objspace/std/callmethod.py (original) +++ pypy/branch/jitypes2/pypy/objspace/std/callmethod.py Tue Nov 23 13:26:19 2010 @@ -13,6 +13,9 @@ from pypy.interpreter import function from pypy.objspace.descroperation import object_getattribute from pypy.rlib import jit, rstack # for resume points +from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict, \ + LOOKUP_METHOD_mapdict_fill_cache_method + # This module exports two extra methods for StdObjSpaceFrame implementing # the LOOKUP_METHOD and CALL_METHOD opcodes in an efficient way, as well @@ -30,6 +33,13 @@ # space = f.space w_obj = f.popvalue() + + if space.config.objspace.std.withmapdict and not jit.we_are_jitted(): + # mapdict has an extra-fast version of this function + from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict + if LOOKUP_METHOD_mapdict(f, nameindex, w_obj): + return + w_name = f.getname_w(nameindex) w_value = None @@ -50,6 +60,11 @@ # nothing in the instance f.pushvalue(w_descr) f.pushvalue(w_obj) + if (space.config.objspace.std.withmapdict and + not jit.we_are_jitted()): + # let mapdict cache stuff + LOOKUP_METHOD_mapdict_fill_cache_method( + f.getcode(), nameindex, w_obj, w_type, w_descr) return if w_value is None: w_value = space.getattr(w_obj, w_name) Modified: pypy/branch/jitypes2/pypy/objspace/std/mapdict.py ============================================================================== --- pypy/branch/jitypes2/pypy/objspace/std/mapdict.py (original) +++ pypy/branch/jitypes2/pypy/objspace/std/mapdict.py Tue Nov 23 13:26:19 2010 @@ -1,4 +1,5 @@ from pypy.rlib import jit, objectmodel, debug +from pypy.rlib.rarithmetic import intmask, r_uint from pypy.interpreter.baseobjspace import W_Root from pypy.objspace.std.dictmultiobject import W_DictMultiObject @@ -15,24 +16,70 @@ # we want to propagate knowledge that the result cannot be negative class AbstractAttribute(object): - _immutable_fields_ = ['w_cls'] + _immutable_fields_ = ['terminator'] cache_attrs = None _size_estimate = 0 - def __init__(self, space, w_cls): + def __init__(self, space, terminator): self.space = space - self.w_cls = w_cls + assert isinstance(terminator, Terminator) + self.terminator = terminator def read(self, obj, selector): - raise NotImplementedError("abstract base class") + index = self.index(selector) + if index < 0: + return self.terminator._read_terminator(obj, selector) + return obj._mapdict_read_storage(index) def write(self, obj, selector, w_value): - raise NotImplementedError("abstract base class") + index = self.index(selector) + if index < 0: + return self.terminator._write_terminator(obj, selector, w_value) + obj._mapdict_write_storage(index, w_value) + return True def delete(self, obj, selector): return None def index(self, selector): + if (self.space.config.objspace.std.withmethodcache and + not jit.we_are_jitted()): + return self._index_cache(selector) + else: + return self._index(selector) + + @jit.dont_look_inside + def _index_cache(self, selector): + space = self.space + cache = space.fromcache(IndexCache) + SHIFT2 = r_uint.BITS - space.config.objspace.std.methodcachesizeexp + SHIFT1 = SHIFT2 - 5 + attrs_as_int = objectmodel.current_object_addr_as_int(self) + # ^^^Note: see comment in typeobject.py for + # _pure_lookup_where_with_method_cache() + hash_selector = objectmodel.compute_hash(selector) + product = intmask(attrs_as_int * hash_selector) + index_hash = (r_uint(product) ^ (r_uint(product) << SHIFT1)) >> SHIFT2 + # ^^^Note2: same comment too + cached_attr = cache.attrs[index_hash] + if cached_attr is self: + cached_selector = cache.selectors[index_hash] + if cached_selector == selector: + index = cache.indices[index_hash] + if space.config.objspace.std.withmethodcachecounter: + name = selector[0] + cache.hits[name] = cache.hits.get(name, 0) + 1 + return index + index = self._index(selector) + cache.attrs[index_hash] = self + cache.selectors[index_hash] = selector + cache.indices[index_hash] = index + if space.config.objspace.std.withmethodcachecounter: + name = selector[0] + cache.misses[name] = cache.misses.get(name, 0) + 1 + return index + + def _index(self, selector): return -1 def copy(self, obj): @@ -42,7 +89,7 @@ raise NotImplementedError("abstract base class") def get_terminator(self): - raise NotImplementedError("abstract base class") + return self.terminator def set_terminator(self, obj, terminator): raise NotImplementedError("abstract base class") @@ -95,15 +142,20 @@ raise NotImplementedError("abstract base class") def __repr__(self): - return "<%s w_cls=%s>" % (self.__class__.__name__, self.w_cls) + return "<%s>" % (self.__class__.__name__,) class Terminator(AbstractAttribute): + _immutable_fields_ = ['w_cls'] - def read(self, obj, selector): + def __init__(self, space, w_cls): + AbstractAttribute.__init__(self, space, self) + self.w_cls = w_cls + + def _read_terminator(self, obj, selector): return None - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): obj._get_mapdict_map().add_attr(obj, selector, w_value) return True @@ -116,9 +168,6 @@ def length(self): return 0 - def get_terminator(self): - return self - def set_terminator(self, obj, terminator): result = Object() result.space = self.space @@ -128,6 +177,9 @@ def remove_dict_entries(self, obj): return self.copy(obj) + def __repr__(self): + return "<%s w_cls=%s>" % (self.__class__.__name__, self.w_cls) + class DictTerminator(Terminator): _immutable_fields_ = ['devolved_dict_terminator'] def __init__(self, space, w_cls): @@ -142,27 +194,27 @@ class NoDictTerminator(Terminator): - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): if selector[1] == DICT: return False - return Terminator.write(self, obj, selector, w_value) + return Terminator._write_terminator(self, obj, selector, w_value) class DevolvedDictTerminator(Terminator): - def read(self, obj, selector): + def _read_terminator(self, obj, selector): if selector[1] == DICT: w_dict = obj.getdict() space = self.space return space.finditem_str(w_dict, selector[0]) - return Terminator.read(self, obj, selector) + return Terminator._read_terminator(self, obj, selector) - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): if selector[1] == DICT: w_dict = obj.getdict() space = self.space space.setitem_str(w_dict, selector[0], w_value) return True - return Terminator.write(self, obj, selector, w_value) + return Terminator._write_terminator(self, obj, selector, w_value) def delete(self, obj, selector): from pypy.interpreter.error import OperationError @@ -189,7 +241,7 @@ class PlainAttribute(AbstractAttribute): _immutable_fields_ = ['selector', 'position', 'back'] def __init__(self, selector, back): - AbstractAttribute.__init__(self, back.space, back.w_cls) + AbstractAttribute.__init__(self, back.space, back.terminator) self.selector = selector self.position = back.length() self.back = back @@ -199,17 +251,6 @@ w_value = self.read(obj, self.selector) new_obj._get_mapdict_map().add_attr(new_obj, self.selector, w_value) - def read(self, obj, selector): - if selector == self.selector: - return obj._mapdict_read_storage(self.position) - return self.back.read(obj, selector) - - def write(self, obj, selector, w_value): - if selector == self.selector: - obj._mapdict_write_storage(self.position, w_value) - return True - return self.back.write(obj, selector, w_value) - def delete(self, obj, selector): if selector == self.selector: # ok, attribute is deleted @@ -219,10 +260,10 @@ self._copy_attr(obj, new_obj) return new_obj - def index(self, selector): + def _index(self, selector): if selector == self.selector: return self.position - return self.back.index(selector) + return self.back._index(selector) def copy(self, obj): new_obj = self.back.copy(obj) @@ -232,9 +273,6 @@ def length(self): return self.position + 1 - def get_terminator(self): - return self.back.get_terminator() - def set_terminator(self, obj, terminator): new_obj = self.back.set_terminator(obj, terminator) self._copy_attr(obj, new_obj) @@ -268,6 +306,24 @@ # RPython reasons w_obj._set_mapdict_storage_and_map(new_obj.storage, new_obj.map) +class IndexCache(object): + def __init__(self, space): + assert space.config.objspace.std.withmethodcache + SIZE = 1 << space.config.objspace.std.methodcachesizeexp + self.attrs = [None] * SIZE + self._empty_selector = (None, INVALID) + self.selectors = [self._empty_selector] * SIZE + self.indices = [0] * SIZE + if space.config.objspace.std.withmethodcachecounter: + self.hits = {} + self.misses = {} + + def clear(self): + for i in range(len(self.attrs)): + self.attrs[i] = None + for i in range(len(self.selectors)): + self.selectors[i] = self._empty_selector + # ____________________________________________________________ # object implementation @@ -328,7 +384,7 @@ assert flag def getclass(self, space): - return self._get_mapdict_map().w_cls + return self._get_mapdict_map().terminator.w_cls def setclass(self, space, w_cls): new_obj = self._get_mapdict_map().set_terminator(self, w_cls.terminator) @@ -373,6 +429,7 @@ self.storage = make_sure_not_resized([None] * map.size_estimate()) def _mapdict_read_storage(self, index): + assert index >= 0 return self.storage[index] def _mapdict_write_storage(self, index, value): self.storage[index] = value @@ -440,6 +497,7 @@ return rerased.unerase_fixedsizelist(erased, W_Root) def _mapdict_read_storage(self, index): + assert index >= 0 for i in rangenmin1: if index == i: erased = getattr(self, "_value%s" % i) @@ -604,30 +662,54 @@ map = None version_tag = None index = 0 + w_method = None # for callmethod success_counter = 0 failure_counter = 0 + def is_valid_for_obj(self, w_obj): + map = w_obj._get_mapdict_map() + return self.is_valid_for_map(map) + + def is_valid_for_map(self, map): + if map is self.map: + version_tag = map.terminator.w_cls.version_tag() + if version_tag is self.version_tag: + # everything matches, it's incredibly fast + if map.space.config.objspace.std.withmethodcachecounter: + self.success_counter += 1 + return True + return False + INVALID_CACHE_ENTRY = CacheEntry() INVALID_CACHE_ENTRY.map = objectmodel.instantiate(AbstractAttribute) # different from any real map ^^^ -INVALID_CACHE_ENTRY.map.w_cls = None +INVALID_CACHE_ENTRY.map.terminator = None + def init_mapdict_cache(pycode): num_entries = len(pycode.co_names_w) pycode._mapdict_caches = [INVALID_CACHE_ENTRY] * num_entries +def _fill_cache(pycode, nameindex, map, version_tag, index, w_method=None): + entry = pycode._mapdict_caches[nameindex] + if entry is INVALID_CACHE_ENTRY: + entry = CacheEntry() + pycode._mapdict_caches[nameindex] = entry + entry.map = map + entry.version_tag = version_tag + entry.index = index + entry.w_method = w_method + if pycode.space.config.objspace.std.withmethodcachecounter: + entry.failure_counter += 1 + def LOAD_ATTR_caching(pycode, w_obj, nameindex): # this whole mess is to make the interpreter quite a bit faster; it's not # used if we_are_jitted(). entry = pycode._mapdict_caches[nameindex] map = w_obj._get_mapdict_map() - if map is entry.map: - version_tag = map.w_cls.version_tag() - if version_tag is entry.version_tag: - # everything matches, it's incredibly fast - if pycode.space.config.objspace.std.withmethodcachecounter: - entry.success_counter += 1 - return w_obj._mapdict_read_storage(entry.index) + if entry.is_valid_for_map(map) and entry.w_method is None: + # everything matches, it's incredibly fast + return w_obj._mapdict_read_storage(entry.index) return LOAD_ATTR_slowpath(pycode, w_obj, nameindex, map) LOAD_ATTR_caching._always_inline_ = True @@ -635,7 +717,7 @@ space = pycode.space w_name = pycode.co_names_w[nameindex] if map is not None: - w_type = map.w_cls + w_type = map.terminator.w_cls w_descr = w_type.getattribute_if_not_from_object() if w_descr is not None: return space._handle_getattribute(w_descr, w_obj, w_name) @@ -655,17 +737,30 @@ if selector[1] != INVALID: index = map.index(selector) if index >= 0: - entry = pycode._mapdict_caches[nameindex] - if entry is INVALID_CACHE_ENTRY: - entry = CacheEntry() - pycode._mapdict_caches[nameindex] = entry - entry.map = map - entry.version_tag = version_tag - entry.index = index - if space.config.objspace.std.withmethodcachecounter: - entry.failure_counter += 1 + _fill_cache(pycode, nameindex, map, version_tag, index) return w_obj._mapdict_read_storage(index) if space.config.objspace.std.withmethodcachecounter: INVALID_CACHE_ENTRY.failure_counter += 1 return space.getattr(w_obj, w_name) LOAD_ATTR_slowpath._dont_inline_ = True + +def LOOKUP_METHOD_mapdict(f, nameindex, w_obj): + space = f.space + pycode = f.getcode() + entry = pycode._mapdict_caches[nameindex] + if entry.is_valid_for_obj(w_obj): + w_method = entry.w_method + if w_method is not None: + f.pushvalue(w_method) + f.pushvalue(w_obj) + return True + return False + +def LOOKUP_METHOD_mapdict_fill_cache_method(pycode, nameindex, w_obj, w_type, w_method): + version_tag = w_type.version_tag() + if version_tag is None: + return + map = w_obj._get_mapdict_map() + if map is None: + return + _fill_cache(pycode, nameindex, map, version_tag, -1, w_method) Modified: pypy/branch/jitypes2/pypy/objspace/std/test/test_dictmultiobject.py ============================================================================== --- pypy/branch/jitypes2/pypy/objspace/std/test/test_dictmultiobject.py (original) +++ pypy/branch/jitypes2/pypy/objspace/std/test/test_dictmultiobject.py Tue Nov 23 13:26:19 2010 @@ -616,6 +616,7 @@ withdictmeasurement = False withsmalldicts = False withcelldict = False + withmethodcache = False class opcodes: CALL_LIKELY_BUILTIN = False Modified: pypy/branch/jitypes2/pypy/objspace/std/test/test_mapdict.py ============================================================================== --- pypy/branch/jitypes2/pypy/objspace/std/test/test_mapdict.py (original) +++ pypy/branch/jitypes2/pypy/objspace/std/test/test_mapdict.py Tue Nov 23 13:26:19 2010 @@ -24,13 +24,13 @@ hasdict = False def test_plain_attribute(): - space = " " w_cls = "class" aa = PlainAttribute(("b", DICT), PlainAttribute(("a", DICT), Terminator(space, w_cls))) assert aa.space is space - assert aa.w_cls is w_cls + assert aa.terminator.w_cls is w_cls + assert aa.get_terminator() is aa.terminator obj = Object() obj.map, obj.storage = aa, [10, 20] @@ -604,7 +604,8 @@ from pypy.interpreter import gateway cls.space = gettestobjspace( **{"objspace.std.withmapdict": True, - "objspace.std.withmethodcachecounter": True}) + "objspace.std.withmethodcachecounter": True, + "objspace.opcodes.CALL_METHOD": True}) # def check(space, w_func, name): w_code = space.getattr(w_func, space.wrap('func_code')) @@ -785,6 +786,135 @@ res = self.check(f, 'x') assert res == (0, 0, 1) + def test_call_method_uses_cache(self): + # bit sucky + global C + + class C(object): + def m(*args): + return args + C.sm = staticmethod(C.m.im_func) + C.cm = classmethod(C.m.im_func) + + exec """if 1: + + def f(): + c = C() + res = c.m(1) + assert res == (c, 1) + return 42 + + def g(): + c = C() + res = c.sm(1) + assert res == (1, ) + return 42 + + def h(): + c = C() + res = c.cm(1) + assert res == (C, 1) + return 42 + """ + res = self.check(f, 'm') + assert res == (1, 0, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + + # static methods are not cached + res = self.check(g, 'sm') + assert res == (0, 0, 0) + res = self.check(g, 'sm') + assert res == (0, 0, 0) + + # neither are class methods + res = self.check(h, 'cm') + assert res == (0, 0, 0) + res = self.check(h, 'cm') + assert res == (0, 0, 0) + + def test_mix_cache_bug(self): + # bit sucky + global C + + class C(object): + def m(*args): + return args + + exec """if 1: + + def f(): + c = C() + res = c.m(1) + assert res == (c, 1) + bm = c.m + res = bm(1) + assert res == (c, 1) + return 42 + + """ + res = self.check(f, 'm') + assert res == (1, 1, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + +class AppTestGlobalCaching(AppTestWithMapDict): + def setup_class(cls): + cls.space = gettestobjspace( + **{"objspace.std.withmethodcachecounter": True, + "objspace.std.withmapdict": True, + "objspace.opcodes.CALL_METHOD": True}) + + def test_mix_classes(self): + import __pypy__ + class A(object): + def f(self): + return 42 + class B(object): + def f(self): + return 43 + class C(object): + def f(self): + return 44 + l = [A(), B(), C()] * 10 + __pypy__.reset_method_cache_counter() + # 'exec' to make sure that a.f() is compiled with CALL_METHOD + exec """for i, a in enumerate(l): + assert a.f() == 42 + i % 3 +""" + cache_counter = __pypy__.mapdict_cache_counter("f") + assert cache_counter[0] >= 15 + assert cache_counter[1] >= 3 # should be (27, 3) + assert sum(cache_counter) == 30 + + def test_mix_classes_attribute(self): + import __pypy__ + class A(object): + def __init__(self): + self.x = 42 + class B(object): + def __init__(self): + self.x = 43 + class C(object): + def __init__(self): + self.x = 44 + l = [A(), B(), C()] * 10 + __pypy__.reset_method_cache_counter() + for i, a in enumerate(l): + assert a.x == 42 + i % 3 + cache_counter = __pypy__.mapdict_cache_counter("x") + assert cache_counter[0] >= 15 + assert cache_counter[1] >= 3 # should be (27, 3) + assert sum(cache_counter) == 30 + class TestDictSubclassShortcutBug(object): def setup_class(cls): cls.space = gettestobjspace( Modified: pypy/branch/jitypes2/pypy/rlib/clibffi.py ============================================================================== --- pypy/branch/jitypes2/pypy/rlib/clibffi.py (original) +++ pypy/branch/jitypes2/pypy/rlib/clibffi.py Tue Nov 23 13:26:19 2010 @@ -432,7 +432,8 @@ flags=FUNCFLAG_CDECL): AbstractFuncPtr.__init__(self, "callback", argtypes, restype, flags) self.ll_closure = closureHeap.alloc() - self.ll_userdata = lltype.malloc(USERDATA_P.TO, flavor='raw') + self.ll_userdata = lltype.malloc(USERDATA_P.TO, flavor='raw', + track_allocation=False) self.ll_userdata.callback = rffi.llhelper(CALLBACK_TP, func) self.ll_userdata.addarg = additional_arg res = c_ffi_prep_closure(self.ll_closure, self.ll_cif, @@ -447,7 +448,7 @@ closureHeap.free(self.ll_closure) self.ll_closure = lltype.nullptr(FFI_CLOSUREP.TO) if self.ll_userdata: - lltype.free(self.ll_userdata, flavor='raw') + lltype.free(self.ll_userdata, flavor='raw', track_allocation=False) self.ll_userdata = lltype.nullptr(USERDATA_P.TO) class RawFuncPtr(AbstractFuncPtr): Modified: pypy/branch/jitypes2/pypy/rlib/rgc.py ============================================================================== --- pypy/branch/jitypes2/pypy/rlib/rgc.py (original) +++ pypy/branch/jitypes2/pypy/rlib/rgc.py Tue Nov 23 13:26:19 2010 @@ -379,6 +379,11 @@ "NOT_RPYTHON" raise NotImplementedError +def get_typeids_z(): + "NOT_RPYTHON" + raise NotImplementedError + +ARRAY_OF_CHAR = lltype.Array(lltype.Char) NULL_GCREF = lltype.nullptr(llmemory.GCREF.TO) class _GcRef(object): @@ -530,3 +535,12 @@ vlist = hop.inputargs(lltype.Signed) hop.exception_is_here() return hop.genop('gc_dump_rpy_heap', vlist, resulttype = hop.r_result) + +class Entry(ExtRegistryEntry): + _about_ = get_typeids_z + def compute_result_annotation(self): + from pypy.annotation.model import SomePtr + return SomePtr(lltype.Ptr(ARRAY_OF_CHAR)) + def specialize_call(self, hop): + hop.exception_is_here() + return hop.genop('gc_typeids_z', [], resulttype = hop.r_result) Modified: pypy/branch/jitypes2/pypy/rpython/annlowlevel.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/annlowlevel.py (original) +++ pypy/branch/jitypes2/pypy/rpython/annlowlevel.py Tue Nov 23 13:26:19 2010 @@ -136,7 +136,7 @@ return funcdesc.cachedgraph(TYPE, alt_name=valid_identifier(alt_name)) -class MixLevelHelperAnnotator: +class MixLevelHelperAnnotator(object): def __init__(self, rtyper): self.rtyper = rtyper Modified: pypy/branch/jitypes2/pypy/rpython/llinterp.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/llinterp.py (original) +++ pypy/branch/jitypes2/pypy/rpython/llinterp.py Tue Nov 23 13:26:19 2010 @@ -912,6 +912,9 @@ def op_gc_dump_rpy_heap(self): raise NotImplementedError("gc_dump_rpy_heap") + def op_gc_typeids_z(self): + raise NotImplementedError("gc_typeids_z") + def op_do_malloc_fixedsize_clear(self): raise NotImplementedError("do_malloc_fixedsize_clear") Modified: pypy/branch/jitypes2/pypy/rpython/lltypesystem/llarena.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/lltypesystem/llarena.py (original) +++ pypy/branch/jitypes2/pypy/rpython/lltypesystem/llarena.py Tue Nov 23 13:26:19 2010 @@ -440,7 +440,8 @@ [rffi.INT], rffi.INT, sandboxsafe=True, _nowrapper=True) - _dev_zero = rffi.str2charp_immortal('/dev/zero') # prebuilt + _dev_zero = rffi.str2charp('/dev/zero') # prebuilt + lltype.render_immortal(_dev_zero) def clear_large_memory_chunk(baseaddr, size): # on some Unixy platforms, reading from /dev/zero is the fastest way Modified: pypy/branch/jitypes2/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/branch/jitypes2/pypy/rpython/lltypesystem/llmemory.py Tue Nov 23 13:26:19 2010 @@ -766,8 +766,10 @@ return '<%s>' % (self,) def __str__(self): return 'gctransformed_wref(%s)' % (self._ptr,) - def _normalizedcontainer(self): - return self._ptr._obj + def _normalizedcontainer(self, check=True): + return self._ptr._getobj(check=check)._normalizedcontainer(check=check) + def _was_freed(self): + return self._ptr._was_freed() # ____________________________________________________________ Modified: pypy/branch/jitypes2/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/branch/jitypes2/pypy/rpython/lltypesystem/lloperation.py Tue Nov 23 13:26:19 2010 @@ -476,6 +476,7 @@ 'gc_get_rpy_type_index': LLOp(), 'gc_is_rpy_instance' : LLOp(), 'gc_dump_rpy_heap' : LLOp(), + 'gc_typeids_z' : LLOp(), # ------- JIT & GC interaction, only for some GCs ---------- Modified: pypy/branch/jitypes2/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/branch/jitypes2/pypy/rpython/lltypesystem/lltype.py Tue Nov 23 13:26:19 2010 @@ -1868,6 +1868,13 @@ leakfinder.remember_free(p._obj0) p._obj0._free() +def render_immortal(p, track_allocation=True): + T = typeOf(p) + if not isinstance(T, Ptr) or p._togckind() != 'raw': + raise TypeError, "free(): only for pointers to non-gc containers" + if track_allocation: + leakfinder.remember_free(p._obj0) + def _make_scoped_allocator(T): class ScopedAlloc: def __init__(self, n=None, zero=False): Modified: pypy/branch/jitypes2/pypy/rpython/lltypesystem/rffi.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/lltypesystem/rffi.py (original) +++ pypy/branch/jitypes2/pypy/rpython/lltypesystem/rffi.py Tue Nov 23 13:26:19 2010 @@ -607,15 +607,6 @@ return array str2charp._annenforceargs_ = [strtype] - def str2charp_immortal(s): - "NOT_RPYTHON" - array = lltype.malloc(TYPEP.TO, len(s) + 1, flavor='raw', - immortal=True) - for i in range(len(s)): - array[i] = s[i] - array[len(s)] = lastchar - return array - def free_charp(cp): lltype.free(cp, flavor='raw') @@ -734,19 +725,19 @@ l = [cp[i] for i in range(size)] return emptystr.join(l) - return (str2charp, str2charp_immortal, free_charp, charp2str, + return (str2charp, free_charp, charp2str, get_nonmovingbuffer, free_nonmovingbuffer, alloc_buffer, str_from_buffer, keep_buffer_alive_until_here, charp2strn, charpsize2str, ) -(str2charp, str2charp_immortal, free_charp, charp2str, +(str2charp, free_charp, charp2str, get_nonmovingbuffer, free_nonmovingbuffer, alloc_buffer, str_from_buffer, keep_buffer_alive_until_here, charp2strn, charpsize2str, ) = make_string_mappings(str) -(unicode2wcharp, unicode2wcharp_immortal, free_wcharp, wcharp2unicode, +(unicode2wcharp, free_wcharp, wcharp2unicode, get_nonmoving_unicodebuffer, free_nonmoving_unicodebuffer, alloc_unicodebuffer, unicode_from_buffer, keep_unicodebuffer_alive_until_here, wcharp2unicoden, wcharpsize2unicode, Modified: pypy/branch/jitypes2/pypy/rpython/memory/gc/base.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/memory/gc/base.py (original) +++ pypy/branch/jitypes2/pypy/rpython/memory/gc/base.py Tue Nov 23 13:26:19 2010 @@ -247,6 +247,21 @@ (not self.config.taggedpointers or llmemory.cast_adr_to_int(addr) & 1 == 0)) + def enumerate_all_roots(self, callback, arg): + """For each root object, invoke callback(obj, arg). + 'callback' should not be a bound method. + Note that this method is not suitable for actually doing the + collection in a moving GC, because you cannot write back a + modified address. It is there only for inspection. + """ + # overridden in some subclasses, for GCs which have an additional + # list of last generation roots + callback2, attrname = _convert_callback_formats(callback) # :-/ + setattr(self, attrname, arg) + self.root_walker.walk_roots(callback2, callback2, callback2) + self.run_finalizers.foreach(callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' + def debug_check_consistency(self): """To use after a collection. If self.DEBUG is set, this enumerates all roots and traces all objects to check if we didn't @@ -260,8 +275,7 @@ self._debug_pending = self.AddressStack() if not we_are_translated(): self.root_walker._walk_prebuilt_gc(self._debug_record) - callback = GCBase._debug_callback - self.root_walker.walk_roots(callback, callback, callback) + self.enumerate_all_roots(GCBase._debug_callback, self) pending = self._debug_pending while pending.non_empty(): obj = pending.pop() @@ -275,9 +289,8 @@ seen.add(obj) self.debug_check_object(obj) self._debug_pending.append(obj) - def _debug_callback(self, root): - obj = root.address[0] - ll_assert(bool(obj), "NULL address from walk_roots()") + @staticmethod + def _debug_callback(obj, self): self._debug_record(obj) def _debug_callback2(self, pointer, ignored): obj = pointer.address[0] @@ -432,3 +445,17 @@ if factor != 1: return 0.0 return value + +def _convert_callback_formats(callback): + callback = getattr(callback, 'im_func', callback) + if callback not in _converted_callback_formats: + def callback2(gc, root): + obj = root.address[0] + ll_assert(bool(obj), "NULL address from walk_roots()") + callback(obj, getattr(gc, attrname)) + attrname = '_callback2_arg%d' % len(_converted_callback_formats) + _converted_callback_formats[callback] = callback2, attrname + return _converted_callback_formats[callback] + +_convert_callback_formats._annspecialcase_ = 'specialize:memo' +_converted_callback_formats = {} Modified: pypy/branch/jitypes2/pypy/rpython/memory/gc/generation.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/memory/gc/generation.py (original) +++ pypy/branch/jitypes2/pypy/rpython/memory/gc/generation.py Tue Nov 23 13:26:19 2010 @@ -572,16 +572,10 @@ def _compute_current_nursery_hash(self, obj): return intmask(llmemory.cast_adr_to_int(obj) + self.nursery_hash_base) - def heap_stats_walk_roots(self): - self.last_generation_root_objects.foreach( - self._track_heap_ext, None) - self.root_walker.walk_roots( - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root) - - def _track_heap_ext(self, adr, ignored): - self.trace(adr, self.track_heap_parent, adr) + def enumerate_all_roots(self, callback, arg): + self.last_generation_root_objects.foreach(callback, arg) + SemiSpaceGC.enumerate_all_roots(self, callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' def debug_check_object(self, obj): """Check the invariants about 'obj' that should be true Modified: pypy/branch/jitypes2/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/branch/jitypes2/pypy/rpython/memory/gc/inspector.py Tue Nov 23 13:26:19 2010 @@ -4,25 +4,22 @@ from pypy.rpython.lltypesystem import lltype, llmemory, rffi from pypy.rlib.objectmodel import free_non_gc_object from pypy.rpython.module.ll_os import underscore_on_windows -from pypy.rlib import rposix +from pypy.rlib import rposix, rgc from pypy.rpython.memory.support import AddressDict, get_address_stack # ---------- implementation of pypy.rlib.rgc.get_rpy_roots() ---------- -def _counting_rpy_root(gc, root): +def _counting_rpy_root(obj, gc): gc._count_rpy += 1 def _do_count_rpy_roots(gc): gc._count_rpy = 0 - gc.root_walker.walk_roots( - _counting_rpy_root, - _counting_rpy_root, - _counting_rpy_root) + gc.enumerate_all_roots(_counting_rpy_root, gc) return gc._count_rpy -def _append_rpy_root(gc, root): +def _append_rpy_root(obj, gc): # Can use the gc list, but should not allocate! # It is essential that the list is not resizable! lst = gc._list_rpy @@ -30,15 +27,12 @@ if index >= len(lst): raise ValueError gc._count_rpy = index + 1 - lst[index] = llmemory.cast_adr_to_ptr(root.address[0], llmemory.GCREF) + lst[index] = llmemory.cast_adr_to_ptr(obj, llmemory.GCREF) def _do_append_rpy_roots(gc, lst): gc._count_rpy = 0 gc._list_rpy = lst - gc.root_walker.walk_roots( - _append_rpy_root, - _append_rpy_root, - _append_rpy_root) + gc.enumerate_all_roots(_append_rpy_root, gc) gc._list_rpy = None def get_rpy_roots(gc): @@ -101,8 +95,7 @@ # ---------- raw_os_write = rffi.llexternal(underscore_on_windows+'write', - [rffi.INT, rffi.CArrayPtr(lltype.Signed), - rffi.SIZE_T], + [rffi.INT, llmemory.Address, rffi.SIZE_T], rffi.SIZE_T, sandboxsafe=True, _nowrapper=True) @@ -131,7 +124,7 @@ if self.buf_count > 0: bytes = self.buf_count * rffi.sizeof(rffi.LONG) count = raw_os_write(self.fd, - self.writebuffer, + rffi.cast(llmemory.Address, self.writebuffer), rffi.cast(rffi.SIZE_T, bytes)) if rffi.cast(lltype.Signed, count) != bytes: raise OSError(rposix.get_errno(), "raw_os_write failed") @@ -140,7 +133,7 @@ def write(self, value): x = self.buf_count - self.writebuffer[x] = llmemory.raw_malloc_usage(value) + self.writebuffer[x] = value x += 1 self.buf_count = x if x == self.BUFSIZE: @@ -173,12 +166,7 @@ self.pending.append(obj) def add_roots(self): - self.gc._heap_dumper = self - self.gc.root_walker.walk_roots( - _hd_add_root, - _hd_add_root, - _hd_add_root) - self.gc._heap_dumper = None + self.gc.enumerate_all_roots(_hd_add_root, self) pendingroots = self.pending self.pending = AddressStack() self.walk(pendingroots) @@ -189,8 +177,8 @@ while pending.non_empty(): self.writeobj(pending.pop()) -def _hd_add_root(gc, root): - gc._heap_dumper.add(root.address[0]) +def _hd_add_root(obj, heap_dumper): + heap_dumper.add(obj) def dump_rpy_heap(gc, fd): heapdumper = HeapDumper(gc, fd) @@ -199,3 +187,7 @@ heapdumper.flush() heapdumper.delete() return True + +def get_typeids_z(gc): + srcaddress = gc.root_walker.gcdata.typeids_z + return llmemory.cast_adr_to_ptr(srcaddress, lltype.Ptr(rgc.ARRAY_OF_CHAR)) Modified: pypy/branch/jitypes2/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/branch/jitypes2/pypy/rpython/memory/gc/minimark.py Tue Nov 23 13:26:19 2010 @@ -1,3 +1,36 @@ +""" MiniMark GC. + +Environment variables can be used to fine-tune the following parameters: + + PYPY_GC_NURSERY The nursery size. Defaults to half the size of + the L2 cache. Try values like '1.2MB'. + + PYPY_GC_MAJOR_COLLECT Major collection memory factor. Default is '1.82', + which means trigger a major collection when the + memory consumed equals 1.82 times the memory + really used at the end of the previous major + collection. + + PYPY_GC_GROWTH Major collection threshold's max growth rate. + Default is '1.3'. Useful to collect more often + than normally on sudden memory growth, e.g. when + there is a temporary peak in memory usage. + + PYPY_GC_MAX The max heap size. If coming near this limit, it + will first collect more often, then raise an + RPython MemoryError, and if that is not enough, + crash the program with a fatal error. Try values + like '1.6GB'. + + PYPY_GC_MIN Don't collect while the memory size is below this + limit. Useful to avoid spending all the time in + the GC in very small programs. Defaults to 8 + times the nursery. +""" +# XXX Should find a way to bound the major collection threshold by the +# XXX total addressable size. Maybe by keeping some minimarkpage arenas +# XXX pre-reserved, enough for a few nursery collections? What about +# XXX raw-malloced memory? import sys from pypy.rpython.lltypesystem import lltype, llmemory, llarena, llgroup from pypy.rpython.lltypesystem.lloperation import llop @@ -87,13 +120,8 @@ TRANSLATION_PARAMS = { # Automatically adjust the size of the nursery and the - # 'major_collection_threshold' from the environment. For - # 'nursery_size' it will look it up in the env var - # PYPY_GC_NURSERY and fall back to half the size of - # the L2 cache. For 'major_collection_threshold' it will look - # it up in the env var PYPY_GC_MAJOR_COLLECT. It also sets - # 'max_heap_size' to PYPY_GC_MAX. Finally, PYPY_GC_MIN sets - # the minimal value of 'next_major_collection_threshold'. + # 'major_collection_threshold' from the environment. + # See docstring at the start of the file. "read_from_env": True, # The size of the nursery. Note that this is only used as a @@ -121,6 +149,13 @@ # we trigger the next major collection. "major_collection_threshold": 1.82, + # Threshold to avoid that the total heap size grows by a factor of + # major_collection_threshold at every collection: it can only + # grow at most by the following factor from one collection to the + # next. Used e.g. when there is a sudden, temporary peak in memory + # usage; this avoids that the upper bound grows too fast. + "growth_rate_max": 1.3, + # The number of array indices that are mapped to a single bit in # write_barrier_from_array(). Must be a power of two. The default # value of 128 means that card pages are 512 bytes (1024 on 64-bits) @@ -146,6 +181,7 @@ arena_size=64*WORD, small_request_threshold=5*WORD, major_collection_threshold=2.5, + growth_rate_max=2.5, # for tests card_page_indices=0, large_object=8*WORD, large_object_gcptrs=10*WORD, @@ -157,6 +193,7 @@ self.nursery_size = nursery_size self.small_request_threshold = small_request_threshold self.major_collection_threshold = major_collection_threshold + self.growth_rate_max = growth_rate_max self.num_major_collects = 0 self.min_heap_size = 0.0 self.max_heap_size = 0.0 @@ -257,9 +294,13 @@ newsize = max(newsize, minsize) # major_coll = base.read_float_from_env('PYPY_GC_MAJOR_COLLECT') - if major_coll >= 1.0: + if major_coll > 1.0: self.major_collection_threshold = major_coll # + growth = base.read_float_from_env('PYPY_GC_GROWTH') + if growth > 1.0: + self.growth_rate_max = growth + # min_heap_size = base.read_uint_from_env('PYPY_GC_MIN') if min_heap_size > 0: self.min_heap_size = float(min_heap_size) @@ -295,11 +336,19 @@ # initialize the threshold self.min_heap_size = max(self.min_heap_size, self.nursery_size * self.major_collection_threshold) + self.next_major_collection_threshold = self.min_heap_size self.set_major_threshold_from(0.0) debug_stop("gc-set-nursery-size") - def set_major_threshold_from(self, threshold): + + def set_major_threshold_from(self, threshold, reserving_size=0): # Set the next_major_collection_threshold. + threshold_max = (self.next_major_collection_threshold * + self.growth_rate_max) + if threshold > threshold_max: + threshold = threshold_max + # + threshold += reserving_size if threshold < self.min_heap_size: threshold = self.min_heap_size # @@ -922,7 +971,7 @@ # # Now all live nursery objects should be out. Update the # young weakrefs' targets. - if self.young_objects_with_weakrefs.length() > 0: + if self.young_objects_with_weakrefs.non_empty(): self.invalidate_young_weakrefs() # # Clear this mapping. @@ -1016,7 +1065,7 @@ obj = oldlist.pop() # # Add the flag GCFLAG_NO_YOUNG_PTRS. All live objects should have - # this flag after a nursery collection. + # this flag set after a nursery collection. self.header(obj).tid |= GCFLAG_NO_YOUNG_PTRS # # Trace the 'obj' to replace pointers to nursery with pointers @@ -1079,7 +1128,7 @@ # nursery are kept unchanged in this step. llmemory.raw_memcopy(obj - size_gc_header, newhdr, totalsize) # - # Set the old object's tid to -1 (containing all flags) and + # Set the old object's tid to -42 (containing all flags) and # replace the old object's content with the target address. # A bit of no-ops to convince llarena that we are changing # the layout, in non-translated versions. @@ -1198,8 +1247,8 @@ # have allocated 'major_collection_threshold' times more than # we currently have. bounded = self.set_major_threshold_from( - (self.get_total_memory_used() * self.major_collection_threshold) - + reserving_size) + self.get_total_memory_used() * self.major_collection_threshold, + reserving_size) # # Max heap size: gives an upper bound on the threshold. If we # already have at least this much allocated, raise MemoryError. @@ -1288,6 +1337,11 @@ self.run_finalizers.foreach(self._collect_obj, self.objects_to_trace) + def enumerate_all_roots(self, callback, arg): + self.prebuilt_root_objects.foreach(callback, arg) + MovingGCBase.enumerate_all_roots(self, callback, arg) + enumerate_all_roots._annspecialcase_ = 'specialize:arg(1)' + @staticmethod def _collect_obj(obj, objects_to_trace): objects_to_trace.append(obj) @@ -1339,7 +1393,7 @@ if self.is_valid_gc_object(obj): if self.is_in_nursery(obj): # - # The object not a tagged pointer, and is it still in the + # The object is not a tagged pointer, and it is still in the # nursery. Find or allocate a "shadow" object, which is # where the object will be moved by the next minor # collection Modified: pypy/branch/jitypes2/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/branch/jitypes2/pypy/rpython/memory/gc/semispace.py Tue Nov 23 13:26:19 2010 @@ -693,15 +693,10 @@ self._ll_typeid_map[idx].size += llmemory.raw_malloc_usage(totsize) self.trace(adr, self.track_heap_parent, adr) - def _track_heap_root(self, root): - self.track_heap(root.address[0]) + @staticmethod + def _track_heap_root(obj, self): + self.track_heap(obj) - def heap_stats_walk_roots(self): - self.root_walker.walk_roots( - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root, - SemiSpaceGC._track_heap_root) - def heap_stats(self): self._tracked_dict = self.AddressDict() max_tid = self.root_walker.gcdata.max_type_id @@ -714,7 +709,7 @@ while i < max_tid: self._tracked_dict.add(llmemory.cast_ptr_to_adr(ll_typeid_map[i])) i += 1 - self.heap_stats_walk_roots() + self.enumerate_all_roots(SemiSpaceGC._track_heap_root, self) self._ll_typeid_map = lltype.nullptr(ARRAY_TYPEID_MAP) self._tracked_dict.delete() return ll_typeid_map Modified: pypy/branch/jitypes2/pypy/rpython/memory/gc/test/test_minimark.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/memory/gc/test/test_minimark.py (original) +++ pypy/branch/jitypes2/pypy/rpython/memory/gc/test/test_minimark.py Tue Nov 23 13:26:19 2010 @@ -28,3 +28,42 @@ assert gc.card_marking_bytes_for_length(P+P+1) == 3 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P) == 8 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P+1) == 9 + +def test_set_major_threshold(): + gc = MiniMarkGC(None, major_collection_threshold=2.0, + growth_rate_max=1.5) + gc.min_heap_size = 100.0 + gc.max_heap_size = 300.0 + gc.next_major_collection_threshold = 0.0 + # first, we don't grow past min_heap_size + for i in range(5): + gc.set_major_threshold_from(100.0) + assert gc.next_major_collection_threshold == 100.0 + # then we grow a lot + b = gc.set_major_threshold_from(100 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 150.0 + b = gc.set_major_threshold_from(150 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 225.0 + b = gc.set_major_threshold_from(225 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 # max reached + b = gc.set_major_threshold_from(300 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 + # then we shrink instantly + b = gc.set_major_threshold_from(100.0) + assert b is False + assert gc.next_major_collection_threshold == 100.0 + # then we grow a bit + b = gc.set_major_threshold_from(100 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 125.0 + b = gc.set_major_threshold_from(125 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 156.25 + # check that we cannot shrink below min_heap_size + b = gc.set_major_threshold_from(42.7) + assert b is False + assert gc.next_major_collection_threshold == 100.0 Modified: pypy/branch/jitypes2/pypy/rpython/memory/gctransform/framework.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/memory/gctransform/framework.py (original) +++ pypy/branch/jitypes2/pypy/rpython/memory/gctransform/framework.py Tue Nov 23 13:26:19 2010 @@ -172,6 +172,7 @@ gcdata.static_root_nongcend = a_random_address # patched in finish() gcdata.static_root_end = a_random_address # patched in finish() gcdata.max_type_id = 13 # patched in finish() + gcdata.typeids_z = a_random_address # patched in finish() self.gcdata = gcdata self.malloc_fnptr_cache = {} @@ -212,6 +213,9 @@ data_classdef.generalize_attr( 'max_type_id', annmodel.SomeInteger()) + data_classdef.generalize_attr( + 'typeids_z', + annmodel.SomeAddress()) annhelper = annlowlevel.MixLevelHelperAnnotator(self.translator.rtyper) @@ -415,6 +419,11 @@ [s_gc, annmodel.SomeInteger()], annmodel.s_Bool, minimal_transform=False) + self.get_typeids_z_ptr = getfn(inspector.get_typeids_z, + [s_gc], + annmodel.SomePtr( + lltype.Ptr(rgc.ARRAY_OF_CHAR)), + minimal_transform=False) self.set_max_heap_size_ptr = getfn(GCClass.set_max_heap_size.im_func, [s_gc, @@ -572,7 +581,14 @@ newgcdependencies = [] newgcdependencies.append(ll_static_roots_inside) ll_instance.inst_max_type_id = len(group.members) - self.write_typeid_list() + typeids_z = self.write_typeid_list() + ll_typeids_z = lltype.malloc(rgc.ARRAY_OF_CHAR, + len(typeids_z), + immortal=True) + for i in range(len(typeids_z)): + ll_typeids_z[i] = typeids_z[i] + ll_instance.inst_typeids_z = llmemory.cast_ptr_to_adr(ll_typeids_z) + newgcdependencies.append(ll_typeids_z) return newgcdependencies def get_finish_tables(self): @@ -599,6 +615,11 @@ for index in range(len(self.layoutbuilder.type_info_group.members)): f.write("member%-4d %s\n" % (index, all_ids.get(index, '?'))) f.close() + try: + import zlib + return zlib.compress(udir.join("typeids.txt").read(), 9) + except ImportError: + return '' def transform_graph(self, graph): func = getattr(graph, 'func', None) @@ -988,6 +1009,13 @@ resultvar=hop.spaceop.result) self.pop_roots(hop, livevars) + def gct_gc_typeids_z(self, hop): + livevars = self.push_roots(hop) + hop.genop("direct_call", + [self.get_typeids_z_ptr, self.c_const_gc], + resultvar=hop.spaceop.result) + self.pop_roots(hop, livevars) + def gct_malloc_nonmovable_varsize(self, hop): TYPE = hop.spaceop.result.concretetype if self.gcdata.gc.can_malloc_nonmovable(): @@ -1210,7 +1238,7 @@ sizeofaddr = llmemory.sizeof(llmemory.Address) -class BaseRootWalker: +class BaseRootWalker(object): need_root_stack = False def __init__(self, gctransformer): Modified: pypy/branch/jitypes2/pypy/rpython/memory/support.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/memory/support.py (original) +++ pypy/branch/jitypes2/pypy/rpython/memory/support.py Tue Nov 23 13:26:19 2010 @@ -112,7 +112,7 @@ cur = next free_non_gc_object(self) - def length(self): + def _length_estimate(self): chunk = self.chunk count = self.used_in_last_chunk while chunk: @@ -135,7 +135,7 @@ foreach._annspecialcase_ = 'specialize:arg(1)' def stack2dict(self): - result = AddressDict(self.length()) + result = AddressDict(self._length_estimate()) self.foreach(_add_in_dict, result) return result Modified: pypy/branch/jitypes2/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/branch/jitypes2/pypy/rpython/memory/test/test_gc.py Tue Nov 23 13:26:19 2010 @@ -278,6 +278,105 @@ res = self.interpret(f, []) assert res + def test_bug_1(self): + import weakref + class B(object): + pass + def g(): + b = B() + llop.gc__collect(lltype.Void) # force 'b' to be old + ref = weakref.ref(B()) + b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = (ref() is None) + return result + res = self.interpret(f, []) + assert res + + def test_cycle_with_weakref_and_del(self): + import weakref, gc + class A(object): + count = 0 + a = A() + class B(object): + def __del__(self): + # when __del__ is called, the weakref to c should be dead + if self.ref() is None: + a.count += 10 # ok + else: + a.count = 666 # not ok + class C(object): + pass + def g(): + c = C() + c.b = B() + ref = weakref.ref(c) + c.b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = a.count + (ref() is None) + return result + res = self.interpret(f, []) + assert res == 11 + + def test_weakref_to_object_with_finalizer_ordering(self): + import weakref, gc + class A(object): + count = 0 + a = A() + class B(object): + def __del__(self): + # when __del__ is called, the weakref to myself is still valid + # in RPython (at least with most GCs; this test might be + # skipped for specific GCs) + if self.ref() is self: + a.count += 10 # ok + else: + a.count = 666 # not ok + def g(): + b = B() + ref = weakref.ref(b) + b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = a.count + (ref() is None) + return result + res = self.interpret(f, []) + assert res == 11 + + def test_weakref_bug_1(self): + import weakref + class A(object): + pass + class B(object): + def __del__(self): + self.wref().x += 1 + def g(a): + b = B() + b.wref = weakref.ref(a) + # the only way to reach this weakref is via B, which is an + # object with finalizer (but the weakref itself points to + # a, which does not go away but will move during the next + # gc.collect) + def f(): + a = A() + a.x = 10 + g(a) + llop.gc__collect(lltype.Void) + return a.x + res = self.interpret(f, []) + assert res == 11 + def test_id(self): class A(object): pass @@ -635,6 +734,9 @@ class TestMarkSweepGC(GCTest): from pypy.rpython.memory.gc.marksweep import MarkSweepGC as GCClass + def test_weakref_to_object_with_finalizer_ordering(self): + py.test.skip("Does not work") + class TestSemiSpaceGC(GCTest, snippet.SemiSpaceGCTests): from pypy.rpython.memory.gc.semispace import SemiSpaceGC as GCClass GC_CAN_MOVE = True Modified: pypy/branch/jitypes2/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/rbuiltin.py (original) +++ pypy/branch/jitypes2/pypy/rpython/rbuiltin.py Tue Nov 23 13:26:19 2010 @@ -386,6 +386,14 @@ hop.exception_cannot_occur() hop.genop('free', vlist) +def rtype_render_immortal(hop, i_track_allocation=None): + vlist = [hop.inputarg(hop.args_r[0], arg=0)] + v_track_allocation = parse_kwds(hop, + (i_track_allocation, None)) + hop.exception_cannot_occur() + if i_track_allocation is None or v_track_allocation.value: + hop.genop('track_alloc_stop', vlist) + def rtype_const_result(hop): hop.exception_cannot_occur() return hop.inputconst(hop.r_result.lowleveltype, hop.s_result.const) @@ -523,6 +531,7 @@ BUILTIN_TYPER[lltype.malloc] = rtype_malloc BUILTIN_TYPER[lltype.free] = rtype_free +BUILTIN_TYPER[lltype.render_immortal] = rtype_render_immortal BUILTIN_TYPER[lltype.cast_primitive] = rtype_cast_primitive BUILTIN_TYPER[lltype.cast_pointer] = rtype_cast_pointer BUILTIN_TYPER[lltype.cast_opaque_ptr] = rtype_cast_opaque_ptr Modified: pypy/branch/jitypes2/pypy/rpython/test/test_rpbc.py ============================================================================== --- pypy/branch/jitypes2/pypy/rpython/test/test_rpbc.py (original) +++ pypy/branch/jitypes2/pypy/rpython/test/test_rpbc.py Tue Nov 23 13:26:19 2010 @@ -1547,7 +1547,7 @@ def test_always_raising_methods(self): class Base: def m(self): - raise NotImplementedError + raise KeyError class A(Base): def m(self): return 42 @@ -1560,11 +1560,11 @@ o = B() try: o.m() - except NotImplementedError: - pass + except KeyError: + assert 0 return B().m() - self.interpret_raises(NotImplementedError, f, [7]) + self.interpret_raises(KeyError, f, [7]) def test_possible_missing_attribute_access(self): py.test.skip("Should explode or give some warning") Modified: pypy/branch/jitypes2/pypy/tool/release/force-builds.py ============================================================================== --- pypy/branch/jitypes2/pypy/tool/release/force-builds.py (original) +++ pypy/branch/jitypes2/pypy/tool/release/force-builds.py Tue Nov 23 13:26:19 2010 @@ -20,11 +20,12 @@ 'own-linux-x86-32', 'own-linux-x86-64', # 'own-macosx-x86-32', - 'pypy-c-app-level-linux-x86-32', - 'pypy-c-app-level-linux-x86-64', +# 'pypy-c-app-level-linux-x86-32', +# 'pypy-c-app-level-linux-x86-64', 'pypy-c-stackless-app-level-linux-x86-32', 'pypy-c-app-level-win-x86-32', 'pypy-c-jit-linux-x86-32', + 'pypy-c-jit-linux-x86-64', # 'pypy-c-jit-macosx-x86-32', 'pypy-c-jit-win-x86-32', ] Modified: pypy/branch/jitypes2/pypy/tool/release/make_release.py ============================================================================== --- pypy/branch/jitypes2/pypy/tool/release/make_release.py (original) +++ pypy/branch/jitypes2/pypy/tool/release/make_release.py Tue Nov 23 13:26:19 2010 @@ -4,7 +4,7 @@ into release packages. Note: you must run apropriate buildbots first and make sure there are no failures. Use force-builds.py from the same directory. -Usage: make_release.py release/ +Usage: make_release.py release/ release_version """ import autopath @@ -30,7 +30,8 @@ else: xml = override_xml dom = minidom.parseString(xml) - refs = [node.getAttribute('href') for node in dom.getElementsByTagName('a')] + refs = [node.getAttribute('href') for node in dom.getElementsByTagName('a') + if 'pypy' in node.getAttribute('href')] # all refs are of form: pypy-{type}-{revision}-{platform}.tar.bz2 r = re.compile('pypy-c-([\w\d]+)-(\d+)-([\w\d]+).tar.bz2$') d = {} @@ -76,7 +77,7 @@ t.add('pypy-%s' % release) alltars.append(name) t.close() - shutil.rmtree(str(tmpdir.join('pypy-1.3'))) + shutil.rmtree(str(tmpdir.join('pypy-' + release))) for name in alltars: print "Uploading %s" % name os.system('scp %s codespeak.net:/www/pypy.org/htdocs/download' % name) @@ -84,8 +85,8 @@ os.chdir(olddir) if __name__ == '__main__': - if len(sys.argv) != 2: + if len(sys.argv) != 3: print __doc__ sys.exit(1) - main(sys.argv[1], release='1.3') + main(sys.argv[1], release=sys.argv[2]) Modified: pypy/branch/jitypes2/pypy/tool/release/package.py ============================================================================== --- pypy/branch/jitypes2/pypy/tool/release/package.py (original) +++ pypy/branch/jitypes2/pypy/tool/release/package.py Tue Nov 23 13:26:19 2010 @@ -1,8 +1,12 @@ #!/usr/bin/env python """ A sample script that packages PyPy, provided that it's already built. -Usage: +It uses 'pypy/translator/goal/pypy-c' and parts of the rest of the working +copy. Usage: -package.py pypydir [name-of-archive] [name-of-pypy-c] + package.py root-pypy-dir [name-of-archive] [name-of-pypy-c] + +Usually you would do: package.py ../../.. pypy-VER-PLATFORM. +The output is found in the directory /tmp/usession-YOURNAME/build/. """ import autopath @@ -32,7 +36,7 @@ class PyPyCNotFound(Exception): pass -def package(basedir, name='pypy-nightly', rename_pypy_c='pypy-c', +def package(basedir, name='pypy-nightly', rename_pypy_c='pypy', copy_to_dir = None, override_pypy_c = None): basedir = py.path.local(basedir) if sys.platform == 'win32': @@ -64,6 +68,10 @@ headers = includedir.listdir('*.h') + includedir.listdir('*.inl') for n in headers: shutil.copy(str(n), str(pypydir.join('include'))) + # + spdir = pypydir.ensure('site-packages', dir=True) + shutil.copy(str(basedir.join('site-packages', 'README')), str(spdir)) + # pypydir.ensure('bin', dir=True) archive_pypy_c = pypydir.join('bin', rename_pypy_c) shutil.copy(str(pypy_c), str(archive_pypy_c)) Modified: pypy/branch/jitypes2/pypy/tool/release/test/test_package.py ============================================================================== --- pypy/branch/jitypes2/pypy/tool/release/test/test_package.py (original) +++ pypy/branch/jitypes2/pypy/tool/release/test/test_package.py Tue Nov 23 13:26:19 2010 @@ -18,7 +18,7 @@ prefix = builddir.join(test) cpyver = '%d.%d.%d' % CPYTHON_VERSION[:3] assert prefix.join('lib-python', cpyver, 'test').check() - assert prefix.join('bin', 'pypy-c').check() + assert prefix.join('bin', 'pypy').check() assert prefix.join('lib_pypy', 'syslog.py').check() assert not prefix.join('lib_pypy', 'py').check() assert not prefix.join('lib_pypy', 'ctypes_configure').check() Modified: pypy/branch/jitypes2/pypy/tool/terminal.py ============================================================================== --- pypy/branch/jitypes2/pypy/tool/terminal.py (original) +++ pypy/branch/jitypes2/pypy/tool/terminal.py Tue Nov 23 13:26:19 2010 @@ -62,9 +62,10 @@ for control in CONTROLS: # Set the control escape sequence setattr(MODULE, control, curses.tigetstr(CONTROLS[control]) or '') - for value in VALUES: - # Set terminal related values - setattr(MODULE, value, curses.tigetnum(VALUES[value])) + if hasattr(curses, 'tigetnum'): + for value in VALUES: + # Set terminal related values + setattr(MODULE, value, curses.tigetnum(VALUES[value])) def render(text): """Helper function to apply controls easily @@ -74,7 +75,16 @@ return text % MODULE.__dict__ try: - import curses + if '__pypy__' in sys.builtin_module_names: + # this is not really the whole curses, but our _minimal_curses it's + # better than nothing + import _minimal_curses as curses + # a bit of a hack: we have tigetstr but not tigetnum, so we call + # default() to have default values, then setup() will overwrite the + # ones it can + default() + else: + import curses setup() except Exception, e: # There is a failure; set all attributes to default Modified: pypy/branch/jitypes2/pypy/translator/c/gcc/test/test_trackgcroot.py ============================================================================== --- pypy/branch/jitypes2/pypy/translator/c/gcc/test/test_trackgcroot.py (original) +++ pypy/branch/jitypes2/pypy/translator/c/gcc/test/test_trackgcroot.py Tue Nov 23 13:26:19 2010 @@ -98,14 +98,13 @@ """ lines = source.splitlines(True) parts = list(DarwinAssemblerParser().find_functions(iter(lines))) - assert len(parts) == 7 + assert len(parts) == 6 assert parts[0] == (False, lines[:3]) assert parts[1] == (True, lines[3:7]) assert parts[2] == (True, lines[7:11]) - assert parts[3] == (True, lines[11:13]) - assert parts[4] == (False, lines[13:18]) - assert parts[5] == (True, lines[18:20]) - assert parts[6] == (False, lines[20:]) + assert parts[3] == (True, lines[11:18]) + assert parts[4] == (True, lines[18:20]) + assert parts[5] == (False, lines[20:]) def test_computegcmaptable(): tests = [] Modified: pypy/branch/jitypes2/pypy/translator/c/gcc/trackgcroot.py ============================================================================== --- pypy/branch/jitypes2/pypy/translator/c/gcc/trackgcroot.py (original) +++ pypy/branch/jitypes2/pypy/translator/c/gcc/trackgcroot.py Tue Nov 23 13:26:19 2010 @@ -1405,6 +1405,7 @@ 'const_data' ] r_sectionstart = re.compile(r"\t\.("+'|'.join(OTHERSECTIONS)+").*$") + sections_doesnt_end_function = {'cstring': True, 'const': True} def find_functions(self, iterlines): functionlines = [] @@ -1412,20 +1413,20 @@ in_function = False for n, line in enumerate(iterlines): if self.r_textstart.match(line): - assert not in_text, "unexpected repeated .text start: %d" % n in_text = True elif self.r_sectionstart.match(line): - if in_function: + sectionname = self.r_sectionstart.match(line).group(1) + if (in_function and + sectionname not in self.sections_doesnt_end_function): yield in_function, functionlines functionlines = [] + in_function = False in_text = False - in_function = False elif in_text and self.FunctionGcRootTracker.r_functionstart.match(line): yield in_function, functionlines functionlines = [] in_function = True functionlines.append(line) - if functionlines: yield in_function, functionlines @@ -1442,23 +1443,6 @@ format = "mingw32" FunctionGcRootTracker = Mingw32FunctionGcRootTracker - def find_functions(self, iterlines): - functionlines = [] - in_text = False - in_function = False - for n, line in enumerate(iterlines): - if self.r_textstart.match(line): - in_text = True - elif self.r_sectionstart.match(line): - in_text = False - elif in_text and self.FunctionGcRootTracker.r_functionstart.match(line): - yield in_function, functionlines - functionlines = [] - in_function = True - functionlines.append(line) - if functionlines: - yield in_function, functionlines - class MsvcAssemblerParser(AssemblerParser): format = "msvc" FunctionGcRootTracker = MsvcFunctionGcRootTracker Modified: pypy/branch/jitypes2/pypy/translator/c/genc.py ============================================================================== --- pypy/branch/jitypes2/pypy/translator/c/genc.py (original) +++ pypy/branch/jitypes2/pypy/translator/c/genc.py Tue Nov 23 13:26:19 2010 @@ -553,7 +553,7 @@ ('debug', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT" $(TARGET)'), ('debug_exc', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DDO_LOG_EXC" $(TARGET)'), ('debug_mem', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DTRIVIAL_MALLOC_DEBUG" $(TARGET)'), - ('no_obmalloc', '', '$(MAKE) CFLAGS="-g -O1 -DRPY_ASSERT -DNO_OBMALLOC" $(TARGET)'), + ('no_obmalloc', '', '$(MAKE) CFLAGS="-g -O2 -DRPY_ASSERT -DNO_OBMALLOC" $(TARGET)'), ('linuxmemchk', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DLINUXMEMCHK" $(TARGET)'), ('llsafer', '', '$(MAKE) CFLAGS="-O2 -DRPY_LL_ASSERT" $(TARGET)'), ('lldebug', '', '$(MAKE) CFLAGS="$(DEBUGFLAGS) -DRPY_ASSERT -DRPY_LL_ASSERT" $(TARGET)'), Modified: pypy/branch/jitypes2/pypy/translator/c/node.py ============================================================================== --- pypy/branch/jitypes2/pypy/translator/c/node.py (original) +++ pypy/branch/jitypes2/pypy/translator/c/node.py Tue Nov 23 13:26:19 2010 @@ -714,7 +714,11 @@ s = ''.join([self.obj.getitem(i) for i in range(len(self.obj.items))]) else: s = ''.join(self.obj.items) - yield '\t%s%s' % (length, c_char_array_constant(s)) + array_constant = c_char_array_constant(s) + if array_constant.startswith('{') and barebonearray(T): + assert array_constant.endswith('}') + array_constant = array_constant[1:-1].strip() + yield '\t%s%s' % (length, array_constant) yield '}' else: barebone = barebonearray(T) Modified: pypy/branch/jitypes2/pypy/translator/c/src/asm_gcc_x86.h ============================================================================== --- pypy/branch/jitypes2/pypy/translator/c/src/asm_gcc_x86.h (original) +++ pypy/branch/jitypes2/pypy/translator/c/src/asm_gcc_x86.h Tue Nov 23 13:26:19 2010 @@ -2,6 +2,8 @@ * It replaces some complex macros with native assembler instructions. */ +#if 0 /* --- disabled: does not give any speed-up --- */ + #undef OP_INT_ADD_OVF #define OP_INT_ADD_OVF(x,y,r) \ asm volatile( \ @@ -50,6 +52,13 @@ : "0"(x), "g"(y) /* inputs */ \ : "cc", "memory") /* clobber */ +extern void op_int_overflowed(void) + asm ("_op_int_overflowed") + __attribute__((used)); + +#endif /* 0 */ + + /* Pentium only! */ #define READ_TIMESTAMP(val) \ asm volatile("rdtsc" : "=A" (val)) @@ -62,19 +71,15 @@ // I don't know how important it is, comment talks about time warps -/* prototypes */ - -extern void op_int_overflowed(void) - asm ("_op_int_overflowed") - __attribute__((used)); - /* implementations */ #ifndef PYPY_NOT_MAIN_FILE +# if 0 /* disabled */ void op_int_overflowed(void) { FAIL_OVF("integer operation"); } +# endif #endif Modified: pypy/branch/jitypes2/pypy/translator/c/src/debug_print.h ============================================================================== --- pypy/branch/jitypes2/pypy/translator/c/src/debug_print.h (original) +++ pypy/branch/jitypes2/pypy/translator/c/src/debug_print.h Tue Nov 23 13:26:19 2010 @@ -10,6 +10,7 @@ but not any nested debug_print :fname full logging prefix:fname conditional logging + prefix1,prefix2:fname conditional logging with multiple selections Conditional logging means that it only includes the debug_start/debug_stop sections whose name match 'prefix'. Other sections are ignored, including @@ -70,6 +71,8 @@ static void pypy_debug_open(void) { char *filename = getenv("PYPYLOG"); + if (filename) + unsetenv("PYPYLOG"); /* don't pass it to subprocesses */ if (filename && filename[0]) { char *colon = strchr(filename, ':'); @@ -139,12 +142,22 @@ #endif -static bool_t startswith(const char *str, const char *substr) +static bool_t startswithoneof(const char *str, const char *substr) { - while (*substr) - if (*str++ != *substr++) - return 0; - return 1; + const char *p = str; + for (; *substr; substr++) + { + if (*substr != ',') + { + if (p && *p++ != *substr) + p = NULL; /* mismatch */ + } + else if (p != NULL) + return 1; /* match */ + else + p = str; /* mismatched, retry with the next */ + } + return p != NULL; } #if defined(_MSC_VER) || defined(__MINGW32__) @@ -175,7 +188,7 @@ if (!debug_profile) { /* non-profiling version */ - if (!debug_prefix || !startswith(category, debug_prefix)) + if (!debug_prefix || !startswithoneof(category, debug_prefix)) { /* wrong section name, or no PYPYLOG at all, skip it */ return; Modified: pypy/branch/jitypes2/pypy/translator/c/src/g_include.h ============================================================================== --- pypy/branch/jitypes2/pypy/translator/c/src/g_include.h (original) +++ pypy/branch/jitypes2/pypy/translator/c/src/g_include.h Tue Nov 23 13:26:19 2010 @@ -39,10 +39,13 @@ #include "src/instrument.h" /* optional assembler bits */ -// disabled: does not give any speed-up -//#if defined(__GNUC__) && defined(__i386__) -//# include "src/asm_gcc_x86.h" -//#endif +#if defined(__GNUC__) && defined(__i386__) +# include "src/asm_gcc_x86.h" +#endif + +#if defined(__GNUC__) && defined(__amd64__) +# include "src/asm_gcc_x86_64.h" +#endif #if defined(__GNUC__) && defined(__ppc__) # include "src/asm_ppc.h" Modified: pypy/branch/jitypes2/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/branch/jitypes2/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/branch/jitypes2/pypy/translator/c/test/test_lltyped.py Tue Nov 23 13:26:19 2010 @@ -1,6 +1,7 @@ import py from pypy.rpython.lltypesystem.lltype import * from pypy.translator.c.test import test_typed +from pypy.tool.sourcetools import func_with_new_name class TestLowLevelType(test_typed.CompilationTestCase): @@ -655,6 +656,45 @@ fn = self.getcompiled(llf) fn() + def test_prebuilt_raw_arrays(self): + from pypy.rpython.lltypesystem import rffi, ll2ctypes + # + def make_test_function(cast, haslength, length): + a = malloc(A, length, flavor='raw', immortal=True) + # two cases: a zero-terminated array if length == 6 or 1030, + # a non-zero-terminated array if length == 557 or 1031 + for i in range(length): + a[i] = cast(256 - 5 + i) + def llf(): + for i in range(length): + if a[i] != cast(256 - 5 + i): + return False + if haslength and len(a) != length: + return False + return True + return func_with_new_name(llf, repr((A, haslength, length))) + # + testfns = [] + records = [] + for OF, cast in [(Void, lambda n: None), + (Char, lambda n: chr(n & 0xFF)), + (Signed, lambda n: n)]: + for A, haslength in [(rffi.CArray(OF), False), + (Array(OF), True)]: + for length in [0, 6, 557, 1030, 1031]: + testfns.append(make_test_function(cast, haslength, length)) + records.append((A, haslength, length)) + def llf(): + i = 0 + for fn in testfns: + if not fn(): + return i # returns the index of the failing function + i += 1 + return -42 + fn = self.getcompiled(llf) + res = fn() + assert res == -42, "failing function: %r" % (records[res],) + def test_prebuilt_ll2ctypes_array(self): from pypy.rpython.lltypesystem import rffi, ll2ctypes A = rffi.CArray(Char) @@ -841,3 +881,17 @@ assert res == -98765432 res = fn(1) assert res == -9999999 + + def test_render_immortal(self): + A = FixedSizeArray(Signed, 1) + a1 = malloc(A, flavor='raw') + render_immortal(a1) + a1[0] = 42 + def llf(): + a2 = malloc(A, flavor='raw') + render_immortal(a2) + a2[0] = 3 + return a1[0] + a2[0] + fn = self.getcompiled(llf) + assert fn() == 45 + Modified: pypy/branch/jitypes2/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/branch/jitypes2/pypy/translator/c/test/test_newgc.py (original) +++ pypy/branch/jitypes2/pypy/translator/c/test/test_newgc.py Tue Nov 23 13:26:19 2010 @@ -1096,6 +1096,42 @@ assert os.path.exists(self.filename_dump) assert os.path.getsize(self.filename_dump) > 64 + filename_dump_typeids_z = str(udir.join('test_typeids_z')) + def define_write_typeids_z(self): + U = lltype.GcForwardReference() + U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)), + ('x', lltype.Signed))) + S = lltype.GcStruct('S', ('u', lltype.Ptr(U))) + A = lltype.GcArray(lltype.Ptr(S)) + filename = self.filename_dump_typeids_z + + def fn(): + s = lltype.malloc(S) + s.u = lltype.malloc(U) + s.u.next = lltype.malloc(U) + s.u.next.next = lltype.malloc(U) + a = lltype.malloc(A, 1000) + s2 = lltype.malloc(S) + # + p = rgc.get_typeids_z() + s = ''.join([p[i] for i in range(len(p))]) + fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666) + os.write(fd, s) + os.close(fd) + return 0 + + return fn + + def test_write_typeids_z(self): + self.run("write_typeids_z") + f = open(self.filename_dump_typeids_z) + data_z = f.read() + f.close() + import zlib + data = zlib.decompress(data_z) + assert data.startswith('member0') + assert 'GcArray of * GcStruct S {' in data + class TestSemiSpaceGC(TestUsingFramework, snippet.SemiSpaceGCTestDefines): gcpolicy = "semispace" should_be_moving = True @@ -1179,21 +1215,22 @@ b = 0 c = 0 for i in range(len(tb)): - if tb[i].count == 10: + if tb[i].count == 10: # the type of S a += 1 nr = i for i in range(len(tb)): - if tb[i].count == 3: + if tb[i].count == 3: # the type GcArray(Ptr(S)) b += 1 c += tb[i].links[nr] - # we don't count b here since there can be more singletons, + # b can be 1 or 2 here since _heap_stats() is free to return or + # ignore the three GcStructs that point to the GcArray(Ptr(S)). # important one is c, a is for check return c * 100 + b * 10 + a return f def test_gc_heap_stats(self): res = self.run("gc_heap_stats") - assert res == 3011 + assert res == 3011 or res == 3021 def definestr_string_builder(cls): def fn(_): Modified: pypy/branch/jitypes2/pypy/translator/c/test/test_standalone.py ============================================================================== --- pypy/branch/jitypes2/pypy/translator/c/test/test_standalone.py (original) +++ pypy/branch/jitypes2/pypy/translator/c/test/test_standalone.py Tue Nov 23 13:26:19 2010 @@ -368,12 +368,27 @@ assert not err assert path.check(file=1) data = path.read() - assert 'toplevel' in path.read() - assert 'mycat' not in path.read() - assert 'foo 2 bar 3' not in path.read() + assert 'toplevel' in data + assert 'mycat' not in data + assert 'foo 2 bar 3' not in data assert 'cat2' in data assert 'baz' in data assert 'bok' not in data + # check with PYPYLOG=myc,cat2:somefilename (includes mycat and cat2) + path = udir.join('test_debug_xxx_myc_cat2.log') + out, err = cbuilder.cmdexec("", err=True, + env={'PYPYLOG': 'myc,cat2:%s' % path}) + assert out.strip() == 'got:bcda.' + assert not err + assert path.check(file=1) + data = path.read() + assert 'toplevel' in data + assert '{mycat' in data + assert 'mycat}' in data + assert 'foo 2 bar 3' in data + assert 'cat2' in data + assert 'baz' in data + assert 'bok' in data # # finally, check compiling with logging disabled from pypy.config.pypyoption import get_pypy_config Modified: pypy/branch/jitypes2/pypy/translator/cli/src/pypylib.cs ============================================================================== --- pypy/branch/jitypes2/pypy/translator/cli/src/pypylib.cs (original) +++ pypy/branch/jitypes2/pypy/translator/cli/src/pypylib.cs Tue Nov 23 13:26:19 2010 @@ -83,8 +83,12 @@ return Double.NegativeInfinity; else if (s == "nan") return Double.NaN; - else - return System.Convert.ToDouble(s); + else { + System.Globalization.NumberFormatInfo formatter; + formatter = new System.Globalization.NumberFormatInfo(); + formatter.NumberDecimalSeparator = "."; + return System.Convert.ToDouble(s, formatter); + } } } Modified: pypy/branch/jitypes2/pypy/translator/goal/app_main.py ============================================================================== --- pypy/branch/jitypes2/pypy/translator/goal/app_main.py (original) +++ pypy/branch/jitypes2/pypy/translator/goal/app_main.py Tue Nov 23 13:26:19 2010 @@ -326,10 +326,6 @@ except: print >> sys.stderr, "'import site' failed" - # update sys.path *after* loading site.py, in case there is a - # "site.py" file in the script's directory. - sys.path.insert(0, '') - if warnoptions: sys.warnoptions.append(warnoptions) from warnings import _processoptions @@ -366,6 +362,9 @@ try: if run_command: # handle the "-c" command + # Put '' on sys.path + sys.path.insert(0, '') + def run_it(): exec cmd in mainmodule.__dict__ success = run_toplevel(run_it) @@ -378,6 +377,13 @@ elif run_stdin: # handle the case where no command/filename/module is specified # on the command-line. + + # update sys.path *after* loading site.py, in case there is a + # "site.py" file in the script's directory. Only run this if we're + # executing the interactive prompt, if we're running a script we + # put it's directory on sys.path + sys.path.insert(0, '') + if go_interactive or sys.stdin.isatty(): # If stdin is a tty or if "-i" is specified, we print # a banner and run $PYTHONSTARTUP. Modified: pypy/branch/jitypes2/pypy/translator/goal/targetpypystandalone.py ============================================================================== --- pypy/branch/jitypes2/pypy/translator/goal/targetpypystandalone.py (original) +++ pypy/branch/jitypes2/pypy/translator/goal/targetpypystandalone.py Tue Nov 23 13:26:19 2010 @@ -28,9 +28,14 @@ w_call_finish_gateway = space.wrap(gateway.interp2app(call_finish)) w_call_startup_gateway = space.wrap(gateway.interp2app(call_startup)) w_os = setup_nanos(space) + withjit = space.config.objspace.usemodules.pypyjit def entry_point(argv): space.timer.start("Entrypoint") + if withjit: + from pypy.jit.backend.hlinfo import highleveljitinfo + highleveljitinfo.sys_executable = argv[0] + #debug("entry point starting") #for arg in argv: # debug(" argv -> " + arg) @@ -150,6 +155,9 @@ if config.objspace.allworkingmodules: from pypy.config.pypyoption import enable_allworkingmodules enable_allworkingmodules(config) + if config.objspace.translationmodules: + from pypy.config.pypyoption import enable_translationmodules + enable_translationmodules(config) if config.translation.type_system == 'ootype': config.objspace.usemodules.suggest(rbench=True) Modified: pypy/branch/jitypes2/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/branch/jitypes2/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/branch/jitypes2/pypy/translator/goal/test2/test_app_main.py Tue Nov 23 13:26:19 2010 @@ -95,6 +95,11 @@ child.expect('>>> ') child.sendline('__name__') child.expect("'__main__'") + child.expect('>>> ') + child.sendline('import sys') + child.expect('>>> ') + child.sendline("'' in sys.path") + child.expect("True") def test_run_script(self): child = self.spawn([demo_script]) @@ -463,8 +468,10 @@ yield finally: old_cwd.chdir() - os.putenv('PYTHONPATH', old_pythonpath) - + # Can't call putenv with a None argument. + if old_pythonpath is not None: + os.putenv('PYTHONPATH', old_pythonpath) + tmpdir.join('site.py').write('print "SHOULD NOT RUN"') runme_py = tmpdir.join('runme.py') runme_py.write('print "some text"') @@ -485,9 +492,12 @@ with chdir_and_unset_pythonpath(tmpdir): data = self.run(cmdline2, python_flags='-S') - assert data.startswith("some new text\n") assert repr(str(tmpdir.join('otherpath'))) in data + assert "''" not in data + + data = self.run('-c "import sys; print sys.path"') + assert data.startswith("[''") class AppTestAppMain: @@ -524,7 +534,8 @@ newpath = app_main.get_library_path('/tmp/pypy-c') # stdlib not found assert newpath == sys.path newpath = app_main.get_library_path(self.fake_exe) - assert newpath == self.expected_path + # we get at least 'expected_path', and maybe more (e.g.plat-linux2) + assert newpath[:len(self.expected_path)] == self.expected_path finally: sys.path.pop() @@ -537,7 +548,9 @@ app_main.os = os pypy_c = os.path.join(self.trunkdir, 'pypy', 'translator', 'goal', 'pypy-c') newpath = app_main.get_library_path(pypy_c) - assert len(newpath) == 3 + # we get at least lib_pypy, lib-python/modified-X.Y.Z, + # lib-python/X.Y.Z, and maybe more (e.g. plat-linux2) + assert len(newpath) >= 3 for p in newpath: assert p.startswith(self.trunkdir) finally: Modified: pypy/branch/jitypes2/pypy/translator/platform/linux.py ============================================================================== --- pypy/branch/jitypes2/pypy/translator/platform/linux.py (original) +++ pypy/branch/jitypes2/pypy/translator/platform/linux.py Tue Nov 23 13:26:19 2010 @@ -27,6 +27,8 @@ class Linux(BaseLinux): + shared_only = () # it seems that on 32-bit linux, compiling with -fPIC + # gives assembler that asmgcc is not happy about. def library_dirs_for_libffi_a(self): # places where we need to look for libffi.a return self.library_dirs_for_libffi() + ['/usr/lib'] Modified: pypy/branch/jitypes2/pypy/translator/platform/posix.py ============================================================================== --- pypy/branch/jitypes2/pypy/translator/platform/posix.py (original) +++ pypy/branch/jitypes2/pypy/translator/platform/posix.py Tue Nov 23 13:26:19 2010 @@ -104,12 +104,10 @@ else: target_name = exe_name.basename - cflags = self.cflags - if sys.maxint > 2147483647: # XXX XXX XXX sort this out - if shared: - cflags = self.cflags + self.shared_only - else: - cflags = self.cflags + self.standalone_only + if shared: + cflags = self.cflags + self.shared_only + else: + cflags = self.cflags + self.standalone_only m = GnuMakefile(path) m.exe_name = exe_name From afa at codespeak.net Tue Nov 23 13:39:16 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 23 Nov 2010 13:39:16 +0100 (CET) Subject: [pypy-svn] r79390 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101123123916.D8E49282BD4@codespeak.net> Author: afa Date: Tue Nov 23 13:39:15 2010 New Revision: 79390 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Log: TextIOWrapper: implement write() and seek(). Phew Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Tue Nov 23 13:39:15 2010 @@ -6,8 +6,10 @@ from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root from pypy.interpreter.error import OperationError, operationerrfmt from pypy.rlib.rstring import UnicodeBuilder +from pypy.rlib.rarithmetic import r_ulonglong from pypy.module._codecs import interp_codecs from pypy.module._io.interp_iobase import convert_size +import sys STATE_ZERO, STATE_OK, STATE_DETACHED = range(3) @@ -16,6 +18,8 @@ SEEN_CRLF = 4 SEEN_ALL = SEEN_CR | SEEN_LF | SEEN_CRLF +_WINDOWS = sys.platform == 'win32' + class W_IncrementalNewlineDecoder(Wrappable): seennl = 0 pendingcr = False @@ -225,6 +229,9 @@ encoding = interp_attrproperty_w("w_encoding", W_TextIOBase) ) +class PositionCookie: + pass + class W_TextIOWrapper(W_TextIOBase): def __init__(self, space): W_TextIOBase.__init__(self, space) @@ -233,12 +240,18 @@ self.decoded_chars = None # buffer for text returned from decoder self.decoded_chars_used = 0 # offset into _decoded_chars for read() + self.pending_bytes = None # list of bytes objects waiting to be + # written, or NULL self.chunk_size = 8192 self.readuniversal = False self.readtranslate = False self.readnl = None + self.encodefunc = None # Specialized encoding func (see below) + self.encoding_start_of_stream = False # Whether or not it's the start + # of the stream + @unwrap_spec('self', ObjSpace, W_Root, W_Root, W_Root, W_Root, int) def descr_init(self, space, w_buffer, w_encoding=None, w_errors=None, w_newline=None, line_buffering=0): @@ -284,7 +297,14 @@ self.readuniversal = not newline # null or empty self.readtranslate = newline is None self.readnl = newline - # XXX self.writenl + + self.writetranslate = (newline != u'') + if not self.readuniversal: + self.writenl = self.readnl + if self.writenl == u'\n': + self.writenl = None + elif _WINDOWS: + self.writenl = u"\r\n" # build the decoder object if space.is_true(space.call_method(w_buffer, "readable")): @@ -297,6 +317,16 @@ space.gettypeobject(W_IncrementalNewlineDecoder.typedef), self.w_decoder, space.wrap(self.readtranslate)) + # build the encoder object + if space.is_true(space.call_method(w_buffer, "writable")): + w_codec = interp_codecs.lookup_codec(space, + space.str_w(self.w_encoding)) + self.w_encoder = space.call_method(w_codec, + "incrementalencoder", w_errors) + + self.seekable = space.is_true(space.call_method(w_buffer, "seekable")) + self.telling = self.seekable + self.state = STATE_OK def _check_init(self, space): @@ -334,8 +364,8 @@ @unwrap_spec('self', ObjSpace) def flush_w(self, space): self._check_closed(space) - # XXX self.telling = self.seekable - # XXX self._writeflush(space) + self.telling = self.seekable + self._writeflush(space) space.call_method(self.w_buffer, "flush") @unwrap_spec('self', ObjSpace) @@ -346,6 +376,7 @@ return space.call_method(self.w_buffer, "close") # _____________________________________________________________ + # read methods def _set_decoded_chars(self, chars): self.decoded_chars = chars @@ -481,7 +512,7 @@ @unwrap_spec('self', ObjSpace, W_Root) def readline_w(self, space, w_limit=None): self._check_closed(space) - # XXX self._writeflush(space) + self._writeflush(space) limit = convert_size(space, w_limit) chunked = 0 @@ -566,6 +597,69 @@ else: return space.wrap(u'') + # _____________________________________________________________ + # write methods + + @unwrap_spec('self', ObjSpace, W_Root) + def write_w(self, space, w_text): + self._check_init(space) + self._check_closed(space) + + if not self.w_encoder: + raise OperationError(space.w_IOError, space.wrap("not writable")) + + text = space.unicode_w(w_text) + textlen = len(text) + + haslf = False + if (self.writetranslate and self.writenl) or self.line_buffering: + if text.find(u'\n') >= 0: + haslf = True + if haslf and self.writetranslate and self.writenl: + w_text = space.call_method(w_text, "replace", space.wrap(u'\n'), + space.wrap(self.writenl)) + text = space.unicode_w(w_text) + + needflush = False + if self.line_buffering and (haslf or text.find(u'\r') >= 0): + needflush = True + + # XXX What if we were just reading? + if self.encodefunc: + w_bytes = self.encodefunc(space, w_text, self.errors) + self.encoding_start_of_stream = False + else: + w_bytes = space.call_method(self.w_encoder, "encode", w_text) + + b = space.str_w(w_bytes) + if not self.pending_bytes: + self.pending_bytes = [] + self.pending_bytes_count = 0 + self.pending_bytes.append(b) + self.pending_bytes_count += len(b) + + if self.pending_bytes_count > self.chunk_size or needflush: + self._writeflush(space) + + if needflush: + space.call_method(self.w_buffer, "flush") + + self.snapshot = None + + if self.w_decoder: + space.call_method(self.w_decoder, "reset") + + return space.wrap(textlen) + + def _writeflush(self, space): + if not self.pending_bytes: + return + + pending_bytes = ''.join(self.pending_bytes) + self.pending_bytes = None + + space.call_method(self.w_buffer, "write", space.wrap(pending_bytes)) + @unwrap_spec('self', ObjSpace) def detach_w(self, space): self._check_init(space) @@ -575,6 +669,132 @@ self.state = STATE_DETACHED return w_buffer + # _____________________________________________________________ + # seek/tell + + def _pack_cookie(self, start_pos, dec_flags=0, + bytes_to_feed=0, need_eof=0, chars_to_skip=0): + # The meaning of a tell() cookie is: seek to position, set the + # decoder flags to dec_flags, read bytes_to_feed bytes, feed them + # into the decoder with need_eof as the EOF flag, then skip + # chars_to_skip characters of the decoded result. For most simple + # decoders, tell() will often just give a byte offset in the file. + return (start_pos | (dec_flags<<64) | (bytes_to_feed<<128) | + (chars_to_skip<<192) | bool(need_eof)<<256) + + def _unpack_cookie(self, bigint): + cookie = PositionCookie() + cookie.start_pos = bigint.ulonglongmask() + bigint = bigint.rshift(r_ulonglong.BITS) + cookie.dec_flags = 0 + cookie.bytes_to_feed = 0 + cookie.chars_to_skip = 0 + cookie.need_eof = 0 + return cookie + + def _decoder_setstate(self, space, cookie): + # When seeking to the start of the stream, we call decoder.reset() + # rather than decoder.getstate(). + # This is for a few decoders such as utf-16 for which the state value + # at start is not (b"", 0) but e.g. (b"", 2) (meaning, in the case of + # utf-16, that we are expecting a BOM). + if cookie.start_pos == 0 and cookie.dec_flags == 0: + space.call_method(self.w_decoder, "reset") + self.encoding_start_of_stream = True + else: + space.call_method(self.w_encoder, "setstate", + space.newtuple([space.wrap(""), + space.wrap(cookie.dec_flags)])) + + def _encoder_setstate(self, space, cookie): + if cookie.start_pos == 0 and cookie.dec_flags == 0: + space.call_method(self.w_encoder, "reset") + self.encoding_start_of_stream = True + else: + space.call_method(self.w_encoder, "setstate", space.wrap(0)) + self.encoding_start_of_stream = False + + @unwrap_spec('self', ObjSpace, W_Root, int) + def seek_w(self, space, w_pos, whence=0): + self._check_closed(space) + + if not self.seekable: + raise OperationError(space.w_IOError, space.wrap( + "underlying stream is not seekable")) + + if whence == 1: + # seek relative to current position + if not space.is_true(space.eq(w_pos, space.wrap(0))): + raise OperationError(space.w_IOError, space.wrap( + "can't do nonzero cur-relative seeks")) + # Seeking to the current position should attempt to sync the + # underlying buffer with the current position. + w_pos = space.call_method(self, "tell") + + elif whence == 2: + # seek relative to end of file + if not space.is_true(space.eq(w_pos, space.wrap(0))): + raise OperationError(space.w_IOError, space.wrap( + "can't do nonzero end-relative seeks")) + space.call_method(self, "flush") + self._set_decoded_chars(None) + self.snapshot = None + if self.w_decoder: + space.call_method(self.w_decoder, "reset") + return space.call_method(self.w_buffer, "seek", + w_pos, space.wrap(whence)) + + elif whence != 0: + raise OperationError(space.w_ValueError, space.wrap( + "invalid whence (%d, should be 0, 1 or 2)" % (whence,))) + + if space.is_true(space.lt(w_pos, space.wrap(0))): + r = space.str_w(space.repr(w_pos)) + raise OperationError(space.w_ValueError, space.wrap( + "negative seek position %s" % (r,))) + + space.call_method(self, "flush") + + # The strategy of seek() is to go back to the safe start point and + # replay the effect of read(chars_to_skip) from there. + cookie = self._unpack_cookie(space.bigint_w(w_pos)) + + # Seek back to the safe start point + space.call_method(self.w_buffer, "seek", space.wrap(cookie.start_pos)) + + self._set_decoded_chars(None) + self.snapshot = None + + # Restore the decoder to its state from the safe start point. + if self.w_decoder: + self._decoder_setstate(space, cookie) + + if cookie.chars_to_skip: + # Just like _read_chunk, feed the decoder and save a snapshot. + w_chunk = space.call_method(self.w_buffer, "read", + space.wrap(cookie.chars_to_feed)) + # XXX self.snapshot = cookie.dec_flags, w_chunk + + w_decoded = space.call_method(self.w_decoder, "decode", + w_chunk, space.wrap(cookie.need_eof)) + self._set_decoded_chars(space.unicode_w(w_decoded)) + + # Skip chars_to_skip of the decoded characters + if len(self.decoded_chars) < cookie.chars_to_skip: + raise OperationError(space.w_IOError, space.wrap( + "can't restore logical file position")) + self.decoded_chars_used = cookie.chars_to_skip + else: + # XXX self.snapshot = cookie.dec_flags, space.wrap(u"") + pass + + # Finally, reset the encoder (merely useful for proper BOM handling) + if self.w_encoder: + self._encoder_setstate(space, cookie) + + return w_pos + + W_TextIOWrapper.typedef = TypeDef( 'TextIOWrapper', W_TextIOBase.typedef, __new__ = generic_new_descr(W_TextIOWrapper), @@ -582,6 +802,8 @@ read = interp2app(W_TextIOWrapper.read_w), readline = interp2app(W_TextIOWrapper.readline_w), + write = interp2app(W_TextIOWrapper.write_w), + seek = interp2app(W_TextIOWrapper.seek_w), detach = interp2app(W_TextIOWrapper.detach_w), flush = interp2app(W_TextIOWrapper.flush_w), close = interp2app(W_TextIOWrapper.close_w), Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Tue Nov 23 13:39:15 2010 @@ -85,6 +85,27 @@ reads += t.readline() assert reads == u"abc\ndef\n" + def test_encoded_writes(self): + import _io + data = u"1234567890" + tests = ("utf-16", + "utf-16-le", + "utf-16-be", + "utf-32", + "utf-32-le", + "utf-32-be") + for encoding in tests: + buf = _io.BytesIO() + f = _io.TextIOWrapper(buf, encoding=encoding) + # Check if the BOM is written only once (see issue1753). + f.write(data) + f.write(data) + f.seek(0) + assert f.read() == data * 2 + f.seek(0) + assert f.read() == data * 2 + assert buf.getvalue() == (data * 2).encode(encoding) + class AppTestIncrementalNewlineDecoder: def test_newline_decoder(self): From cfbolz at codespeak.net Tue Nov 23 15:11:44 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 23 Nov 2010 15:11:44 +0100 (CET) Subject: [pypy-svn] r79392 - pypy/branch/reflex-support2 Message-ID: <20101123141144.9EB8D5080B@codespeak.net> Author: cfbolz Date: Tue Nov 23 15:11:43 2010 New Revision: 79392 Added: pypy/branch/reflex-support2/ (props changed) - copied from r79391, pypy/trunk/ Log: branch trunk again to add the cppyy module there From afa at codespeak.net Tue Nov 23 15:17:08 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 23 Nov 2010 15:17:08 +0100 (CET) Subject: [pypy-svn] r79393 - pypy/trunk/pypy/translator/c/src Message-ID: <20101123141708.6E8BB282BD6@codespeak.net> Author: afa Date: Tue Nov 23 15:17:07 2010 New Revision: 79393 Modified: pypy/trunk/pypy/translator/c/src/debug_alloc.h pypy/trunk/pypy/translator/c/src/debug_print.h Log: Fix compilation on Windows Modified: pypy/trunk/pypy/translator/c/src/debug_alloc.h ============================================================================== --- pypy/trunk/pypy/translator/c/src/debug_alloc.h (original) +++ pypy/trunk/pypy/translator/c/src/debug_alloc.h Tue Nov 23 15:17:07 2010 @@ -1,5 +1,5 @@ /**************************************************************/ - /*** tracking raw mallocs and frees for debugging ***/ +/*** tracking raw mallocs and frees for debugging ***/ #ifndef RPY_ASSERT @@ -62,8 +62,8 @@ count++; if (count > 0) { - fprintf(stderr, "debug_alloc.h: %ld mallocs left", count); char *env = getenv("PYPY_ALLOC"); + fprintf(stderr, "debug_alloc.h: %ld mallocs left", count); if (env && *env) { fprintf(stderr, " (most recent first):\n"); Modified: pypy/trunk/pypy/translator/c/src/debug_print.h ============================================================================== --- pypy/trunk/pypy/translator/c/src/debug_print.h (original) +++ pypy/trunk/pypy/translator/c/src/debug_print.h Tue Nov 23 15:17:07 2010 @@ -72,7 +72,11 @@ { char *filename = getenv("PYPYLOG"); if (filename) +#ifndef MS_WINDOWS unsetenv("PYPYLOG"); /* don't pass it to subprocesses */ +#else + putenv("PYPYLOG="); /* don't pass it to subprocesses */ +#endif if (filename && filename[0]) { char *colon = strchr(filename, ':'); From arigo at codespeak.net Tue Nov 23 15:29:14 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 15:29:14 +0100 (CET) Subject: [pypy-svn] r79394 - in pypy/trunk/pypy/rpython/memory/gc: . test Message-ID: <20101123142914.A581D5080F@codespeak.net> Author: arigo Date: Tue Nov 23 15:29:12 2010 New Revision: 79394 Added: pypy/trunk/pypy/rpython/memory/gc/env.py pypy/trunk/pypy/rpython/memory/gc/test/test_env.py Modified: pypy/trunk/pypy/rpython/memory/gc/base.py pypy/trunk/pypy/rpython/memory/gc/generation.py pypy/trunk/pypy/rpython/memory/gc/markcompact.py pypy/trunk/pypy/rpython/memory/gc/minimark.py Log: * Move things like reading env vars and poking at system files to the new file env.py. Test it. * Add PYPY_GC_MAX_DELTA to minimark.py, with comments. These env vars should ideally have some tests too... Modified: pypy/trunk/pypy/rpython/memory/gc/base.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/base.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/base.py Tue Nov 23 15:29:12 2010 @@ -5,7 +5,6 @@ from pypy.rpython.memory.support import get_address_stack, get_address_deque from pypy.rpython.memory.support import AddressDict from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage -from pypy.rlib.rarithmetic import r_uint TYPEID_MAP = lltype.GcStruct('TYPEID_MAP', ('count', lltype.Signed), ('size', lltype.Signed), @@ -410,42 +409,6 @@ GCClass = getattr(module, classname) return GCClass, GCClass.TRANSLATION_PARAMS -def _read_float_and_factor_from_env(varname): - import os - value = os.environ.get(varname) - if value: - if len(value) > 1 and value[-1] in 'bB': - value = value[:-1] - realvalue = value[:-1] - if value[-1] in 'kK': - factor = 1024 - elif value[-1] in 'mM': - factor = 1024*1024 - elif value[-1] in 'gG': - factor = 1024*1024*1024 - else: - factor = 1 - realvalue = value - try: - return (float(realvalue), factor) - except ValueError: - pass - return (0.0, 0) - -def read_from_env(varname): - value, factor = _read_float_and_factor_from_env(varname) - return int(value * factor) - -def read_uint_from_env(varname): - value, factor = _read_float_and_factor_from_env(varname) - return r_uint(value * factor) - -def read_float_from_env(varname): - value, factor = _read_float_and_factor_from_env(varname) - if factor != 1: - return 0.0 - return value - def _convert_callback_formats(callback): callback = getattr(callback, 'im_func', callback) if callback not in _converted_callback_formats: Added: pypy/trunk/pypy/rpython/memory/gc/env.py ============================================================================== --- (empty file) +++ pypy/trunk/pypy/rpython/memory/gc/env.py Tue Nov 23 15:29:12 2010 @@ -0,0 +1,245 @@ +""" +Utilities to get environ variables and platform-specific memory-related values. +""" +import os, sys +from pypy.rlib.rarithmetic import r_uint +from pypy.rlib.debug import debug_print, debug_start, debug_stop + +# ____________________________________________________________ +# Reading env vars. Supports returning ints, uints or floats, +# and in the first two cases accepts the suffixes B, KB, MB and GB +# (lower case or upper case). + +def _read_float_and_factor_from_env(varname): + value = os.environ.get(varname) + if value: + if len(value) > 1 and value[-1] in 'bB': + value = value[:-1] + realvalue = value[:-1] + if value[-1] in 'kK': + factor = 1024 + elif value[-1] in 'mM': + factor = 1024*1024 + elif value[-1] in 'gG': + factor = 1024*1024*1024 + else: + factor = 1 + realvalue = value + try: + return (float(realvalue), factor) + except ValueError: + pass + return (0.0, 0) + +def read_from_env(varname): + value, factor = _read_float_and_factor_from_env(varname) + return int(value * factor) + +def read_uint_from_env(varname): + value, factor = _read_float_and_factor_from_env(varname) + return r_uint(value * factor) + +def read_float_from_env(varname): + value, factor = _read_float_and_factor_from_env(varname) + if factor != 1: + return 0.0 + return value + + +# ____________________________________________________________ +# Get the total amount of RAM installed in a system. +# On 32-bit systems, it will try to return at most the addressable size. +# Note that if unknown it will just return the addressable size, which +# will be huge on 64-bit systems. + +if sys.maxint == 2147483647: # 32-bit + if sys.platform == 'linux2': + addressable_size = float(2**32) # 4GB + elif sys.platform == 'win32': + addressable_size = float(2**31) # 2GB + else: + addressable_size = float(2**31 + 2**30) # 3GB (compromise) +else: + addressable_size = float(2**63) # 64-bit + + +def get_total_memory_linux2(filename): + debug_start("gc-hardware") + result = -1.0 + try: + fd = os.open(filename, os.O_RDONLY, 0644) + try: + buf = os.read(fd, 4096) + finally: + os.close(fd) + except OSError: + pass + else: + if buf.startswith('MemTotal:'): + start = _skipspace(buf, len('MemTotal:')) + stop = start + while stop < len(buf) and buf[stop].isdigit(): + stop += 1 + if start < stop: + result = float(buf[start:stop]) * 1024.0 # assume kB + if result < 0.0: + debug_print("get_total_memory() failed") + result = addressable_size + else: + debug_print("memtotal =", result) + debug_stop("gc-hardware") + return result + + +if sys.platform == 'linux2': + def get_total_memory(): + return get_total_memory_linux2('/proc/meminfo') + +#elif sys.platform == 'darwin': +# ... + +else: + def get_total_memory(): + return addressable_size # XXX implement me for other platforms + + +# ____________________________________________________________ +# Estimation of the nursery size, based on the L2 cache. + +def best_nursery_size_for_L2cache(L2cache): + # Heuristically, the best nursery size to choose is about half + # of the L2 cache. XXX benchmark some more. + if L2cache > 0: + return L2cache // 2 + else: + return -1 + +def get_L2cache_linux2(filename): + debug_start("gc-hardware") + L2cache = sys.maxint + try: + fd = os.open(filename, os.O_RDONLY, 0644) + try: + data = [] + while True: + buf = os.read(fd, 4096) + if not buf: + break + data.append(buf) + finally: + os.close(fd) + except OSError: + pass + else: + data = ''.join(data) + linepos = 0 + while True: + start = _findend(data, '\ncache size', linepos) + if start < 0: + break # done + linepos = _findend(data, '\n', start) + if linepos < 0: + break # no end-of-line?? + # *** data[start:linepos] == " : 2048 KB\n" + start = _skipspace(data, start) + if data[start] != ':': + continue + # *** data[start:linepos] == ": 2048 KB\n" + start = _skipspace(data, start + 1) + # *** data[start:linepos] == "2048 KB\n" + end = start + while '0' <= data[end] <= '9': + end += 1 + # *** data[start:end] == "2048" + if start == end: + continue + number = int(data[start:end]) + # *** data[end:linepos] == " KB\n" + end = _skipspace(data, end) + if data[end] not in ('K', 'k'): # assume kilobytes for now + continue + number = number * 1024 + # for now we look for the smallest of the L2 caches of the CPUs + if number < L2cache: + L2cache = number + + debug_print("L2cache =", L2cache) + debug_stop("gc-hardware") + + if L2cache < sys.maxint: + return L2cache + else: + # Print a top-level warning even in non-debug builds + llop.debug_print(lltype.Void, + "Warning: cannot find your CPU L2 cache size in /proc/cpuinfo") + return -1 + +def _findend(data, pattern, pos): + pos = data.find(pattern, pos) + if pos < 0: + return -1 + return pos + len(pattern) + +def _skipspace(data, pos): + while data[pos] in (' ', '\t'): + pos += 1 + return pos + + +if sys.platform == 'linux2': + def estimate_best_nursery_size(): + """Try to estimate the best nursery size at run-time, depending + on the machine we are running on. Linux code.""" + L2cache = get_L2cache_linux2('/proc/cpuinfo') + return best_nursery_size_for_L2cache(L2cache) + +elif sys.platform == 'darwin': + from pypy.rpython.lltypesystem import rffi + + sysctlbyname = rffi.llexternal('sysctlbyname', + [rffi.CCHARP, rffi.VOIDP, rffi.SIZE_TP, + rffi.VOIDP, rffi.SIZE_T], + rffi.INT, + sandboxsafe=True) + + def estimate_best_nursery_size(): + """Try to estimate the best nursery size at run-time, depending + on the machine we are running on. + """ + debug_start("gc-hardware") + L2cache = 0 + l2cache_p = lltype.malloc(rffi.LONGLONGP.TO, 1, flavor='raw') + try: + len_p = lltype.malloc(rffi.SIZE_TP.TO, 1, flavor='raw') + try: + size = rffi.sizeof(rffi.LONGLONG) + l2cache_p[0] = rffi.cast(rffi.LONGLONG, 0) + len_p[0] = rffi.cast(rffi.SIZE_T, size) + # XXX a hack for llhelper not being robust-enough + result = sysctlbyname("hw.l2cachesize", + rffi.cast(rffi.VOIDP, l2cache_p), + len_p, + lltype.nullptr(rffi.VOIDP.TO), + rffi.cast(rffi.SIZE_T, 0)) + if (rffi.cast(lltype.Signed, result) == 0 and + rffi.cast(lltype.Signed, len_p[0]) == size): + L2cache = rffi.cast(lltype.Signed, l2cache_p[0]) + if rffi.cast(rffi.LONGLONG, L2cache) != l2cache_p[0]: + L2cache = 0 # overflow! + finally: + lltype.free(len_p, flavor='raw') + finally: + lltype.free(l2cache_p, flavor='raw') + debug_print("L2cache =", L2cache) + debug_stop("gc-hardware") + if L2cache > 0: + return L2cache + else: + # Print a top-level warning even in non-debug builds + llop.debug_print(lltype.Void, + "Warning: cannot find your CPU L2 cache size with sysctl()") + return -1 + +else: + def estimate_best_nursery_size(): + return -1 # XXX implement me for other platforms Modified: pypy/trunk/pypy/rpython/memory/gc/generation.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/generation.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/generation.py Tue Nov 23 15:29:12 2010 @@ -2,7 +2,7 @@ from pypy.rpython.memory.gc.semispace import SemiSpaceGC from pypy.rpython.memory.gc.semispace import GCFLAG_EXTERNAL, GCFLAG_FORWARDED from pypy.rpython.memory.gc.semispace import GC_HASH_TAKEN_ADDR -from pypy.rpython.memory.gc.base import read_from_env +from pypy.rpython.memory.gc import env from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage from pypy.rpython.lltypesystem import lltype, llmemory, llarena from pypy.rlib.objectmodel import free_non_gc_object @@ -93,7 +93,7 @@ if self.auto_nursery_size: newsize = nursery_size_from_env() if newsize <= 0: - newsize = estimate_best_nursery_size() + newsize = env.estimate_best_nursery_size() if newsize > 0: self.set_nursery_size(newsize) @@ -633,139 +633,5 @@ # ____________________________________________________________ -import os - def nursery_size_from_env(): - return read_from_env('PYPY_GENERATIONGC_NURSERY') - -def best_nursery_size_for_L2cache(L2cache): - # Heuristically, the best nursery size to choose is about half - # of the L2 cache. XXX benchmark some more. - return L2cache // 2 - - -if sys.platform == 'linux2': - def estimate_best_nursery_size(): - """Try to estimate the best nursery size at run-time, depending - on the machine we are running on. - """ - debug_start("gc-L2cache") - L2cache = sys.maxint - try: - fd = os.open('/proc/cpuinfo', os.O_RDONLY, 0644) - try: - data = [] - while True: - buf = os.read(fd, 4096) - if not buf: - break - data.append(buf) - finally: - os.close(fd) - except OSError: - pass - else: - data = ''.join(data) - linepos = 0 - while True: - start = findend(data, '\ncache size', linepos) - if start < 0: - break # done - linepos = findend(data, '\n', start) - if linepos < 0: - break # no end-of-line?? - # *** data[start:linepos] == " : 2048 KB\n" - start = skipspace(data, start) - if data[start] != ':': - continue - # *** data[start:linepos] == ": 2048 KB\n" - start = skipspace(data, start + 1) - # *** data[start:linepos] == "2048 KB\n" - end = start - while '0' <= data[end] <= '9': - end += 1 - # *** data[start:end] == "2048" - if start == end: - continue - number = int(data[start:end]) - # *** data[end:linepos] == " KB\n" - end = skipspace(data, end) - if data[end] not in ('K', 'k'): # assume kilobytes for now - continue - number = number * 1024 - # for now we look for the smallest of the L2 caches of the CPUs - if number < L2cache: - L2cache = number - - debug_print("L2cache =", L2cache) - debug_stop("gc-L2cache") - - if L2cache < sys.maxint: - return best_nursery_size_for_L2cache(L2cache) - else: - # Print a top-level warning even in non-debug builds - llop.debug_print(lltype.Void, - "Warning: cannot find your CPU L2 cache size in /proc/cpuinfo") - return -1 - - def findend(data, pattern, pos): - pos = data.find(pattern, pos) - if pos < 0: - return -1 - return pos + len(pattern) - - def skipspace(data, pos): - while data[pos] in (' ', '\t'): - pos += 1 - return pos - -elif sys.platform == 'darwin': - from pypy.rpython.lltypesystem import rffi - - sysctlbyname = rffi.llexternal('sysctlbyname', - [rffi.CCHARP, rffi.VOIDP, rffi.SIZE_TP, - rffi.VOIDP, rffi.SIZE_T], - rffi.INT, - sandboxsafe=True) - - def estimate_best_nursery_size(): - """Try to estimate the best nursery size at run-time, depending - on the machine we are running on. - """ - debug_start("gc-L2cache") - L2cache = 0 - l2cache_p = lltype.malloc(rffi.LONGLONGP.TO, 1, flavor='raw') - try: - len_p = lltype.malloc(rffi.SIZE_TP.TO, 1, flavor='raw') - try: - size = rffi.sizeof(rffi.LONGLONG) - l2cache_p[0] = rffi.cast(rffi.LONGLONG, 0) - len_p[0] = rffi.cast(rffi.SIZE_T, size) - # XXX a hack for llhelper not being robust-enough - result = sysctlbyname("hw.l2cachesize", - rffi.cast(rffi.VOIDP, l2cache_p), - len_p, - lltype.nullptr(rffi.VOIDP.TO), - rffi.cast(rffi.SIZE_T, 0)) - if (rffi.cast(lltype.Signed, result) == 0 and - rffi.cast(lltype.Signed, len_p[0]) == size): - L2cache = rffi.cast(lltype.Signed, l2cache_p[0]) - if rffi.cast(rffi.LONGLONG, L2cache) != l2cache_p[0]: - L2cache = 0 # overflow! - finally: - lltype.free(len_p, flavor='raw') - finally: - lltype.free(l2cache_p, flavor='raw') - debug_print("L2cache =", L2cache) - debug_stop("gc-L2cache") - if L2cache > 0: - return best_nursery_size_for_L2cache(L2cache) - else: - # Print a top-level warning even in non-debug builds - llop.debug_print(lltype.Void, - "Warning: cannot find your CPU L2 cache size with sysctl()") - return -1 - -else: - def estimate_best_nursery_size(): - return -1 # XXX implement me for other platforms + return env.read_from_env('PYPY_GENERATIONGC_NURSERY') Modified: pypy/trunk/pypy/rpython/memory/gc/markcompact.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/markcompact.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/markcompact.py Tue Nov 23 15:29:12 2010 @@ -1,5 +1,6 @@ from pypy.rpython.lltypesystem import lltype, llmemory, llarena, llgroup -from pypy.rpython.memory.gc.base import MovingGCBase, read_from_env +from pypy.rpython.memory.gc.base import MovingGCBase +from pypy.rpython.memory.gc import env from pypy.rlib.debug import ll_assert, have_debug_prints from pypy.rlib.debug import debug_print, debug_start, debug_stop from pypy.rpython.memory.support import get_address_stack, get_address_deque @@ -110,10 +111,10 @@ return next def setup(self): - envsize = read_from_env('PYPY_MARKCOMPACTGC_MAX') + envsize = env.read_from_env('PYPY_MARKCOMPACTGC_MAX') if envsize >= 4096: self.space_size = envsize & ~4095 - mincollect = read_from_env('PYPY_MARKCOMPACTGC_MIN') + mincollect = env.read_from_env('PYPY_MARKCOMPACTGC_MIN') if mincollect >= 4096: self.min_next_collect_after = mincollect Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Tue Nov 23 15:29:12 2010 @@ -12,7 +12,7 @@ collection. PYPY_GC_GROWTH Major collection threshold's max growth rate. - Default is '1.3'. Useful to collect more often + Default is '1.4'. Useful to collect more often than normally on sudden memory growth, e.g. when there is a temporary peak in memory usage. @@ -22,6 +22,12 @@ crash the program with a fatal error. Try values like '1.6GB'. + PYPY_GC_MAX_DELTA The major collection threshold will never be set + to more than PYPY_GC_MAX_DELTA the amount really + used after a collection. Defaults to 1/8th of the + total RAM size (or at most 4GB on 32-bit systems). + Try values like '200MB'. + PYPY_GC_MIN Don't collect while the memory size is below this limit. Useful to avoid spending all the time in the GC in very small programs. Defaults to 8 @@ -36,7 +42,7 @@ from pypy.rpython.lltypesystem.lloperation import llop from pypy.rpython.lltypesystem.llmemory import raw_malloc_usage from pypy.rpython.memory.gc.base import GCBase, MovingGCBase -from pypy.rpython.memory.gc import minimarkpage, base, generation +from pypy.rpython.memory.gc import minimarkpage, base, env from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask, r_uint from pypy.rlib.rarithmetic import LONG_BIT_SHIFT from pypy.rlib.debug import ll_assert, debug_print, debug_start, debug_stop @@ -154,7 +160,7 @@ # grow at most by the following factor from one collection to the # next. Used e.g. when there is a sudden, temporary peak in memory # usage; this avoids that the upper bound grows too fast. - "growth_rate_max": 1.3, + "growth_rate_max": 1.4, # The number of array indices that are mapped to a single bit in # write_barrier_from_array(). Must be a power of two. The default @@ -198,6 +204,7 @@ self.min_heap_size = 0.0 self.max_heap_size = 0.0 self.max_heap_size_already_raised = False + self.max_delta = 0.0 # self.card_page_indices = card_page_indices if self.card_page_indices > 0: @@ -288,7 +295,7 @@ # handling of the write barrier. self.debug_always_do_minor_collect = newsize == 1 if newsize <= 0: - newsize = generation.estimate_best_nursery_size() + newsize = env.estimate_best_nursery_size() if newsize <= 0: newsize = defaultsize newsize = max(newsize, minsize) @@ -312,6 +319,12 @@ if max_heap_size > 0: self.max_heap_size = float(max_heap_size) # + max_delta = base.read_uint_from_env('PYPY_GC_MAX_DELTA') + if max_delta > 0: + self.max_delta = float(max_delta) + else: + self.max_delta = 0.125 * env.get_total_memory() + # self.minor_collection() # to empty the nursery llarena.arena_free(self.nursery) self.nursery_size = newsize @@ -345,6 +358,9 @@ # Set the next_major_collection_threshold. threshold_max = (self.next_major_collection_threshold * self.growth_rate_max) + if self.max_delta > 0.0: + threshold_max = min(threshold_max, + self.next_major_collection_threshold + self.max_delta) if threshold > threshold_max: threshold = threshold_max # Added: pypy/trunk/pypy/rpython/memory/gc/test/test_env.py ============================================================================== --- (empty file) +++ pypy/trunk/pypy/rpython/memory/gc/test/test_env.py Tue Nov 23 15:29:12 2010 @@ -0,0 +1,133 @@ +import os, py +from pypy.rpython.memory.gc import env +from pypy.rlib.rarithmetic import r_uint +from pypy.tool.udir import udir + + +class FakeEnviron: + def __init__(self, value): + self._value = value + def get(self, varname): + assert varname == 'FOOBAR' + return self._value + +def check_equal(x, y): + assert x == y + assert type(x) == type(y) + + +def test_read_from_env(): + saved = os.environ + try: + os.environ = FakeEnviron(None) + check_equal(env.read_from_env('FOOBAR'), 0) + check_equal(env.read_uint_from_env('FOOBAR'), r_uint(0)) + check_equal(env.read_float_from_env('FOOBAR'), 0.0) + # + os.environ = FakeEnviron('') + check_equal(env.read_from_env('FOOBAR'), 0) + check_equal(env.read_uint_from_env('FOOBAR'), r_uint(0)) + check_equal(env.read_float_from_env('FOOBAR'), 0.0) + # + os.environ = FakeEnviron('???') + check_equal(env.read_from_env('FOOBAR'), 0) + check_equal(env.read_uint_from_env('FOOBAR'), r_uint(0)) + check_equal(env.read_float_from_env('FOOBAR'), 0.0) + # + os.environ = FakeEnviron('1') + check_equal(env.read_from_env('FOOBAR'), 1) + check_equal(env.read_uint_from_env('FOOBAR'), r_uint(1)) + check_equal(env.read_float_from_env('FOOBAR'), 1.0) + # + os.environ = FakeEnviron('12345678') + check_equal(env.read_from_env('FOOBAR'), 12345678) + check_equal(env.read_uint_from_env('FOOBAR'), r_uint(12345678)) + check_equal(env.read_float_from_env('FOOBAR'), 12345678.0) + # + os.environ = FakeEnviron('1234B') + check_equal(env.read_from_env('FOOBAR'), 1234) + check_equal(env.read_uint_from_env('FOOBAR'), r_uint(1234)) + check_equal(env.read_float_from_env('FOOBAR'), 1234.0) + # + os.environ = FakeEnviron('1.5') + check_equal(env.read_float_from_env('FOOBAR'), 1.5) + # + os.environ = FakeEnviron('1.5Kb') + check_equal(env.read_from_env('FOOBAR'), 1536) + check_equal(env.read_uint_from_env('FOOBAR'), r_uint(1536)) + check_equal(env.read_float_from_env('FOOBAR'), 0.0) + # + os.environ = FakeEnviron('1.5mB') + check_equal(env.read_from_env('FOOBAR'), int(1.5*1024*1024)) + check_equal(env.read_uint_from_env('FOOBAR'), r_uint(1.5*1024*1024)) + check_equal(env.read_float_from_env('FOOBAR'), 0.0) + # + os.environ = FakeEnviron('1.5g') + check_equal(env.read_from_env('FOOBAR'), int(1.5*1024*1024*1024)) + check_equal(env.read_uint_from_env('FOOBAR'), r_uint(1.5*1024*1024*1024)) + check_equal(env.read_float_from_env('FOOBAR'), 0.0) + # + finally: + os.environ = saved + +def test_get_total_memory_linux2(): + filepath = udir.join('get_total_memory_linux2') + filepath.write("""\ +MemTotal: 1976804 kB +MemFree: 32200 kB +Buffers: 144092 kB +Cached: 1385196 kB +SwapCached: 8408 kB +Active: 1181436 kB +etc. +""") + result = env.get_total_memory_linux2(str(filepath)) + assert result == 1976804 * 1024 + +def test_estimate_best_nursery_size_linux2(): + filepath = udir.join('estimate_best_nursery_size_linux2') + filepath.write("""\ +processor : 0 +vendor_id : GenuineIntel +cpu family : 6 +model : 37 +model name : Intel(R) Core(TM) i5 CPU M 540 @ 2.53GHz +stepping : 5 +cpu MHz : 1199.000 +cache size : 3072 KB +physical id : 0 +siblings : 4 +core id : 0 +cpu cores : 2 +apicid : 0 +initial apicid : 0 +fpu : yes +fpu_exception : yes +cpuid level : 11 +wp : yes +flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt aes lahf_lm ida arat tpr_shadow vnmi flexpriority ept vpid +bogomips : 5054.78 +clflush size : 64 +cache_alignment : 64 +address sizes : 36 bits physical, 48 bits virtual +power management: + +processor : 1 +vendor_id : GenuineIntel +cpu family : 6 +model : 37 +model name : Intel(R) Core(TM) i5 CPU M 540 @ 2.53GHz +stepping : 5 +cpu MHz : 2534.000 +cache size : 3072 KB +physical id : 0 +siblings : 4 +core id : 0 +cpu cores : 2 +apicid : 1 +initial apicid : 1 +fpu : yes +etc. +""") + result = env.get_L2cache_linux2(str(filepath)) + assert result == 3072 * 1024 From fijal at codespeak.net Tue Nov 23 15:30:03 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 23 Nov 2010 15:30:03 +0100 (CET) Subject: [pypy-svn] r79395 - pypy/trunk/pypy/jit/backend Message-ID: <20101123143003.6F619282BEA@codespeak.net> Author: fijal Date: Tue Nov 23 15:30:01 2010 New Revision: 79395 Modified: pypy/trunk/pypy/jit/backend/conftest.py Log: remove pointless import Modified: pypy/trunk/pypy/jit/backend/conftest.py ============================================================================== --- pypy/trunk/pypy/jit/backend/conftest.py (original) +++ pypy/trunk/pypy/jit/backend/conftest.py Tue Nov 23 15:30:01 2010 @@ -4,8 +4,6 @@ """ import py, random -option = py.test.config.option - def pytest_addoption(parser): group = parser.getgroup('random test options') group.addoption('--random-seed', action="store", type="int", From fijal at codespeak.net Tue Nov 23 15:30:37 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 23 Nov 2010 15:30:37 +0100 (CET) Subject: [pypy-svn] r79396 - pypy/trunk/pypy/jit Message-ID: <20101123143037.9BE985080E@codespeak.net> Author: fijal Date: Tue Nov 23 15:30:36 2010 New Revision: 79396 Modified: pypy/trunk/pypy/jit/conftest.py Log: another pointless import Modified: pypy/trunk/pypy/jit/conftest.py ============================================================================== --- pypy/trunk/pypy/jit/conftest.py (original) +++ pypy/trunk/pypy/jit/conftest.py Tue Nov 23 15:30:36 2010 @@ -1,7 +1,5 @@ import py -option = py.test.config.option - def pytest_addoption(parser): group = parser.getgroup("JIT options") group.addoption('--slow', action="store_true", From arigo at codespeak.net Tue Nov 23 15:33:53 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 15:33:53 +0100 (CET) Subject: [pypy-svn] r79397 - in pypy/trunk/pypy/rpython/memory/gc: . test Message-ID: <20101123143353.17636282BE3@codespeak.net> Author: arigo Date: Tue Nov 23 15:33:51 2010 New Revision: 79397 Modified: pypy/trunk/pypy/rpython/memory/gc/env.py pypy/trunk/pypy/rpython/memory/gc/minimark.py pypy/trunk/pypy/rpython/memory/gc/test/test_env.py Log: Fix and test for the bound at 4GB. Modified: pypy/trunk/pypy/rpython/memory/gc/env.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/env.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/env.py Tue Nov 23 15:33:51 2010 @@ -87,6 +87,8 @@ result = addressable_size else: debug_print("memtotal =", result) + if result > addressable_size: + result = addressable_size debug_stop("gc-hardware") return result Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Tue Nov 23 15:33:51 2010 @@ -25,8 +25,8 @@ PYPY_GC_MAX_DELTA The major collection threshold will never be set to more than PYPY_GC_MAX_DELTA the amount really used after a collection. Defaults to 1/8th of the - total RAM size (or at most 4GB on 32-bit systems). - Try values like '200MB'. + total RAM size (which is constrained to be at most + 4GB on 32-bit systems). Try values like '200MB'. PYPY_GC_MIN Don't collect while the memory size is below this limit. Useful to avoid spending all the time in Modified: pypy/trunk/pypy/rpython/memory/gc/test/test_env.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/test/test_env.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/test/test_env.py Tue Nov 23 15:33:51 2010 @@ -84,6 +84,24 @@ result = env.get_total_memory_linux2(str(filepath)) assert result == 1976804 * 1024 +def test_get_total_memory_linux2_32bit_limit(): + filepath = udir.join('get_total_memory_linux2') + filepath.write("""\ +MemTotal: 3145728 kB +etc. +""") + saved = env.addressable_size + try: + env.addressable_size = float(2**31) + result = env.get_total_memory_linux2(str(filepath)) + check_equal(result, float(2**31)) # limit hit + # + env.addressable_size = float(2**32) + result = env.get_total_memory_linux2(str(filepath)) + check_equal(result, float(3145728 * 1024)) # limit not hit + finally: + env.addressable_size = saved + def test_estimate_best_nursery_size_linux2(): filepath = udir.join('estimate_best_nursery_size_linux2') filepath.write("""\ From arigo at codespeak.net Tue Nov 23 15:48:03 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 15:48:03 +0100 (CET) Subject: [pypy-svn] r79399 - in pypy/branch/jit-free/pypy/rpython/memory/gc: . test Message-ID: <20101123144803.98A9150813@codespeak.net> Author: arigo Date: Tue Nov 23 15:48:01 2010 New Revision: 79399 Added: pypy/branch/jit-free/pypy/rpython/memory/gc/env.py - copied unchanged from r79398, pypy/trunk/pypy/rpython/memory/gc/env.py pypy/branch/jit-free/pypy/rpython/memory/gc/test/test_env.py - copied unchanged from r79398, pypy/trunk/pypy/rpython/memory/gc/test/test_env.py Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/base.py pypy/branch/jit-free/pypy/rpython/memory/gc/generation.py pypy/branch/jit-free/pypy/rpython/memory/gc/markcompact.py pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py pypy/branch/jit-free/pypy/rpython/memory/gc/test/test_minimark.py Log: Merge from trunk all changes done to pypy/rpython/memory/. Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/base.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/base.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/base.py Tue Nov 23 15:48:01 2010 @@ -5,7 +5,6 @@ from pypy.rpython.memory.support import get_address_stack, get_address_deque from pypy.rpython.memory.support import AddressDict from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage -from pypy.rlib.rarithmetic import r_uint TYPEID_MAP = lltype.GcStruct('TYPEID_MAP', ('count', lltype.Signed), ('size', lltype.Signed), @@ -411,42 +410,6 @@ GCClass = getattr(module, classname) return GCClass, GCClass.TRANSLATION_PARAMS -def _read_float_and_factor_from_env(varname): - import os - value = os.environ.get(varname) - if value: - if len(value) > 1 and value[-1] in 'bB': - value = value[:-1] - realvalue = value[:-1] - if value[-1] in 'kK': - factor = 1024 - elif value[-1] in 'mM': - factor = 1024*1024 - elif value[-1] in 'gG': - factor = 1024*1024*1024 - else: - factor = 1 - realvalue = value - try: - return (float(realvalue), factor) - except ValueError: - pass - return (0.0, 0) - -def read_from_env(varname): - value, factor = _read_float_and_factor_from_env(varname) - return int(value * factor) - -def read_uint_from_env(varname): - value, factor = _read_float_and_factor_from_env(varname) - return r_uint(value * factor) - -def read_float_from_env(varname): - value, factor = _read_float_and_factor_from_env(varname) - if factor != 1: - return 0.0 - return value - def _convert_callback_formats(callback): callback = getattr(callback, 'im_func', callback) if callback not in _converted_callback_formats: Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/generation.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/generation.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/generation.py Tue Nov 23 15:48:01 2010 @@ -2,7 +2,7 @@ from pypy.rpython.memory.gc.semispace import SemiSpaceGC from pypy.rpython.memory.gc.semispace import GCFLAG_EXTERNAL, GCFLAG_FORWARDED from pypy.rpython.memory.gc.semispace import GC_HASH_TAKEN_ADDR -from pypy.rpython.memory.gc.base import read_from_env +from pypy.rpython.memory.gc import env from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage from pypy.rpython.lltypesystem import lltype, llmemory, llarena from pypy.rlib.objectmodel import free_non_gc_object @@ -93,7 +93,7 @@ if self.auto_nursery_size: newsize = nursery_size_from_env() if newsize <= 0: - newsize = estimate_best_nursery_size() + newsize = env.estimate_best_nursery_size() if newsize > 0: self.set_nursery_size(newsize) @@ -633,139 +633,5 @@ # ____________________________________________________________ -import os - def nursery_size_from_env(): - return read_from_env('PYPY_GENERATIONGC_NURSERY') - -def best_nursery_size_for_L2cache(L2cache): - # Heuristically, the best nursery size to choose is about half - # of the L2 cache. XXX benchmark some more. - return L2cache // 2 - - -if sys.platform == 'linux2': - def estimate_best_nursery_size(): - """Try to estimate the best nursery size at run-time, depending - on the machine we are running on. - """ - debug_start("gc-L2cache") - L2cache = sys.maxint - try: - fd = os.open('/proc/cpuinfo', os.O_RDONLY, 0644) - try: - data = [] - while True: - buf = os.read(fd, 4096) - if not buf: - break - data.append(buf) - finally: - os.close(fd) - except OSError: - pass - else: - data = ''.join(data) - linepos = 0 - while True: - start = findend(data, '\ncache size', linepos) - if start < 0: - break # done - linepos = findend(data, '\n', start) - if linepos < 0: - break # no end-of-line?? - # *** data[start:linepos] == " : 2048 KB\n" - start = skipspace(data, start) - if data[start] != ':': - continue - # *** data[start:linepos] == ": 2048 KB\n" - start = skipspace(data, start + 1) - # *** data[start:linepos] == "2048 KB\n" - end = start - while '0' <= data[end] <= '9': - end += 1 - # *** data[start:end] == "2048" - if start == end: - continue - number = int(data[start:end]) - # *** data[end:linepos] == " KB\n" - end = skipspace(data, end) - if data[end] not in ('K', 'k'): # assume kilobytes for now - continue - number = number * 1024 - # for now we look for the smallest of the L2 caches of the CPUs - if number < L2cache: - L2cache = number - - debug_print("L2cache =", L2cache) - debug_stop("gc-L2cache") - - if L2cache < sys.maxint: - return best_nursery_size_for_L2cache(L2cache) - else: - # Print a top-level warning even in non-debug builds - llop.debug_print(lltype.Void, - "Warning: cannot find your CPU L2 cache size in /proc/cpuinfo") - return -1 - - def findend(data, pattern, pos): - pos = data.find(pattern, pos) - if pos < 0: - return -1 - return pos + len(pattern) - - def skipspace(data, pos): - while data[pos] in (' ', '\t'): - pos += 1 - return pos - -elif sys.platform == 'darwin': - from pypy.rpython.lltypesystem import rffi - - sysctlbyname = rffi.llexternal('sysctlbyname', - [rffi.CCHARP, rffi.VOIDP, rffi.SIZE_TP, - rffi.VOIDP, rffi.SIZE_T], - rffi.INT, - sandboxsafe=True) - - def estimate_best_nursery_size(): - """Try to estimate the best nursery size at run-time, depending - on the machine we are running on. - """ - debug_start("gc-L2cache") - L2cache = 0 - l2cache_p = lltype.malloc(rffi.LONGLONGP.TO, 1, flavor='raw') - try: - len_p = lltype.malloc(rffi.SIZE_TP.TO, 1, flavor='raw') - try: - size = rffi.sizeof(rffi.LONGLONG) - l2cache_p[0] = rffi.cast(rffi.LONGLONG, 0) - len_p[0] = rffi.cast(rffi.SIZE_T, size) - # XXX a hack for llhelper not being robust-enough - result = sysctlbyname("hw.l2cachesize", - rffi.cast(rffi.VOIDP, l2cache_p), - len_p, - lltype.nullptr(rffi.VOIDP.TO), - rffi.cast(rffi.SIZE_T, 0)) - if (rffi.cast(lltype.Signed, result) == 0 and - rffi.cast(lltype.Signed, len_p[0]) == size): - L2cache = rffi.cast(lltype.Signed, l2cache_p[0]) - if rffi.cast(rffi.LONGLONG, L2cache) != l2cache_p[0]: - L2cache = 0 # overflow! - finally: - lltype.free(len_p, flavor='raw') - finally: - lltype.free(l2cache_p, flavor='raw') - debug_print("L2cache =", L2cache) - debug_stop("gc-L2cache") - if L2cache > 0: - return best_nursery_size_for_L2cache(L2cache) - else: - # Print a top-level warning even in non-debug builds - llop.debug_print(lltype.Void, - "Warning: cannot find your CPU L2 cache size with sysctl()") - return -1 - -else: - def estimate_best_nursery_size(): - return -1 # XXX implement me for other platforms + return env.read_from_env('PYPY_GENERATIONGC_NURSERY') Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/markcompact.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/markcompact.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/markcompact.py Tue Nov 23 15:48:01 2010 @@ -1,5 +1,6 @@ from pypy.rpython.lltypesystem import lltype, llmemory, llarena, llgroup -from pypy.rpython.memory.gc.base import MovingGCBase, read_from_env +from pypy.rpython.memory.gc.base import MovingGCBase +from pypy.rpython.memory.gc import env from pypy.rlib.debug import ll_assert, have_debug_prints from pypy.rlib.debug import debug_print, debug_start, debug_stop from pypy.rpython.memory.support import get_address_stack, get_address_deque @@ -110,10 +111,10 @@ return next def setup(self): - envsize = read_from_env('PYPY_MARKCOMPACTGC_MAX') + envsize = env.read_from_env('PYPY_MARKCOMPACTGC_MAX') if envsize >= 4096: self.space_size = envsize & ~4095 - mincollect = read_from_env('PYPY_MARKCOMPACTGC_MIN') + mincollect = env.read_from_env('PYPY_MARKCOMPACTGC_MIN') if mincollect >= 4096: self.min_next_collect_after = mincollect Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py Tue Nov 23 15:48:01 2010 @@ -1,9 +1,48 @@ +""" MiniMark GC. + +Environment variables can be used to fine-tune the following parameters: + + PYPY_GC_NURSERY The nursery size. Defaults to half the size of + the L2 cache. Try values like '1.2MB'. + + PYPY_GC_MAJOR_COLLECT Major collection memory factor. Default is '1.82', + which means trigger a major collection when the + memory consumed equals 1.82 times the memory + really used at the end of the previous major + collection. + + PYPY_GC_GROWTH Major collection threshold's max growth rate. + Default is '1.4'. Useful to collect more often + than normally on sudden memory growth, e.g. when + there is a temporary peak in memory usage. + + PYPY_GC_MAX The max heap size. If coming near this limit, it + will first collect more often, then raise an + RPython MemoryError, and if that is not enough, + crash the program with a fatal error. Try values + like '1.6GB'. + + PYPY_GC_MAX_DELTA The major collection threshold will never be set + to more than PYPY_GC_MAX_DELTA the amount really + used after a collection. Defaults to 1/8th of the + total RAM size (which is constrained to be at most + 4GB on 32-bit systems). Try values like '200MB'. + + PYPY_GC_MIN Don't collect while the memory size is below this + limit. Useful to avoid spending all the time in + the GC in very small programs. Defaults to 8 + times the nursery. +""" +# XXX Should find a way to bound the major collection threshold by the +# XXX total addressable size. Maybe by keeping some minimarkpage arenas +# XXX pre-reserved, enough for a few nursery collections? What about +# XXX raw-malloced memory? import sys from pypy.rpython.lltypesystem import lltype, llmemory, llarena, llgroup from pypy.rpython.lltypesystem.lloperation import llop from pypy.rpython.lltypesystem.llmemory import raw_malloc_usage from pypy.rpython.memory.gc.base import GCBase, MovingGCBase -from pypy.rpython.memory.gc import minimarkpage, base, generation +from pypy.rpython.memory.gc import minimarkpage, base, env from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask, r_uint from pypy.rlib.rarithmetic import LONG_BIT_SHIFT from pypy.rlib.debug import ll_assert, debug_print, debug_start, debug_stop @@ -88,13 +127,8 @@ TRANSLATION_PARAMS = { # Automatically adjust the size of the nursery and the - # 'major_collection_threshold' from the environment. For - # 'nursery_size' it will look it up in the env var - # PYPY_GC_NURSERY and fall back to half the size of - # the L2 cache. For 'major_collection_threshold' it will look - # it up in the env var PYPY_GC_MAJOR_COLLECT. It also sets - # 'max_heap_size' to PYPY_GC_MAX. Finally, PYPY_GC_MIN sets - # the minimal value of 'next_major_collection_threshold'. + # 'major_collection_threshold' from the environment. + # See docstring at the start of the file. "read_from_env": True, # The size of the nursery. Note that this is only used as a @@ -122,6 +156,13 @@ # we trigger the next major collection. "major_collection_threshold": 1.82, + # Threshold to avoid that the total heap size grows by a factor of + # major_collection_threshold at every collection: it can only + # grow at most by the following factor from one collection to the + # next. Used e.g. when there is a sudden, temporary peak in memory + # usage; this avoids that the upper bound grows too fast. + "growth_rate_max": 1.4, + # The number of array indices that are mapped to a single bit in # write_barrier_from_array(). Must be a power of two. The default # value of 128 means that card pages are 512 bytes (1024 on 64-bits) @@ -147,6 +188,7 @@ arena_size=64*WORD, small_request_threshold=5*WORD, major_collection_threshold=2.5, + growth_rate_max=2.5, # for tests card_page_indices=0, large_object=8*WORD, large_object_gcptrs=10*WORD, @@ -158,10 +200,12 @@ self.nursery_size = nursery_size self.small_request_threshold = small_request_threshold self.major_collection_threshold = major_collection_threshold + self.growth_rate_max = growth_rate_max self.num_major_collects = 0 self.min_heap_size = 0.0 self.max_heap_size = 0.0 self.max_heap_size_already_raised = False + self.max_delta = 0.0 # self.card_page_indices = card_page_indices if self.card_page_indices > 0: @@ -252,15 +296,19 @@ # handling of the write barrier. self.debug_always_do_minor_collect = newsize == 1 if newsize <= 0: - newsize = generation.estimate_best_nursery_size() + newsize = env.estimate_best_nursery_size() if newsize <= 0: newsize = defaultsize newsize = max(newsize, minsize) # major_coll = base.read_float_from_env('PYPY_GC_MAJOR_COLLECT') - if major_coll >= 1.0: + if major_coll > 1.0: self.major_collection_threshold = major_coll # + growth = base.read_float_from_env('PYPY_GC_GROWTH') + if growth > 1.0: + self.growth_rate_max = growth + # min_heap_size = base.read_uint_from_env('PYPY_GC_MIN') if min_heap_size > 0: self.min_heap_size = float(min_heap_size) @@ -272,6 +320,12 @@ if max_heap_size > 0: self.max_heap_size = float(max_heap_size) # + max_delta = base.read_uint_from_env('PYPY_GC_MAX_DELTA') + if max_delta > 0: + self.max_delta = float(max_delta) + else: + self.max_delta = 0.125 * env.get_total_memory() + # self.minor_collection() # to empty the nursery llarena.arena_free(self.nursery) self.nursery_size = newsize @@ -296,11 +350,22 @@ # initialize the threshold self.min_heap_size = max(self.min_heap_size, self.nursery_size * self.major_collection_threshold) + self.next_major_collection_threshold = self.min_heap_size self.set_major_threshold_from(0.0) debug_stop("gc-set-nursery-size") - def set_major_threshold_from(self, threshold): + + def set_major_threshold_from(self, threshold, reserving_size=0): # Set the next_major_collection_threshold. + threshold_max = (self.next_major_collection_threshold * + self.growth_rate_max) + if self.max_delta > 0.0: + threshold_max = min(threshold_max, + self.next_major_collection_threshold + self.max_delta) + if threshold > threshold_max: + threshold = threshold_max + # + threshold += reserving_size if threshold < self.min_heap_size: threshold = self.min_heap_size # @@ -1199,8 +1264,8 @@ # have allocated 'major_collection_threshold' times more than # we currently have. bounded = self.set_major_threshold_from( - (self.get_total_memory_used() * self.major_collection_threshold) - + reserving_size) + self.get_total_memory_used() * self.major_collection_threshold, + reserving_size) # # Max heap size: gives an upper bound on the threshold. If we # already have at least this much allocated, raise MemoryError. Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/test/test_minimark.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/test/test_minimark.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/test/test_minimark.py Tue Nov 23 15:48:01 2010 @@ -28,3 +28,42 @@ assert gc.card_marking_bytes_for_length(P+P+1) == 3 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P) == 8 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P+1) == 9 + +def test_set_major_threshold(): + gc = MiniMarkGC(None, major_collection_threshold=2.0, + growth_rate_max=1.5) + gc.min_heap_size = 100.0 + gc.max_heap_size = 300.0 + gc.next_major_collection_threshold = 0.0 + # first, we don't grow past min_heap_size + for i in range(5): + gc.set_major_threshold_from(100.0) + assert gc.next_major_collection_threshold == 100.0 + # then we grow a lot + b = gc.set_major_threshold_from(100 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 150.0 + b = gc.set_major_threshold_from(150 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 225.0 + b = gc.set_major_threshold_from(225 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 # max reached + b = gc.set_major_threshold_from(300 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 + # then we shrink instantly + b = gc.set_major_threshold_from(100.0) + assert b is False + assert gc.next_major_collection_threshold == 100.0 + # then we grow a bit + b = gc.set_major_threshold_from(100 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 125.0 + b = gc.set_major_threshold_from(125 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 156.25 + # check that we cannot shrink below min_heap_size + b = gc.set_major_threshold_from(42.7) + assert b is False + assert gc.next_major_collection_threshold == 100.0 From fijal at codespeak.net Tue Nov 23 15:51:40 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 23 Nov 2010 15:51:40 +0100 (CET) Subject: [pypy-svn] r79400 - pypy/trunk/pypy/config Message-ID: <20101123145140.067D4282BEB@codespeak.net> Author: fijal Date: Tue Nov 23 15:51:39 2010 New Revision: 79400 Modified: pypy/trunk/pypy/config/translationoption.py Log: Try to be a bit smart (on linux at least) with make_jobs Modified: pypy/trunk/pypy/config/translationoption.py ============================================================================== --- pypy/trunk/pypy/config/translationoption.py (original) +++ pypy/trunk/pypy/config/translationoption.py Tue Nov 23 15:51:39 2010 @@ -3,6 +3,7 @@ from pypy.config.config import OptionDescription, BoolOption, IntOption, ArbitraryOption, FloatOption from pypy.config.config import ChoiceOption, StrOption, to_optparse, Config from pypy.config.config import ConfigError +from pypy.config.support import detect_number_of_processors DEFL_INLINE_THRESHOLD = 32.4 # just enough to inline add__Int_Int() # and just small enough to prevend inlining of some rlist functions. @@ -171,7 +172,7 @@ default=False, negation=False), IntOption("make_jobs", "Specify -j argument to make for compilation" " (C backend only)", - cmdline="--make-jobs", default=1), + cmdline="--make-jobs", default=detect_number_of_processors()), # Flags of the TranslationContext: BoolOption("simplifying", "Simplify flow graphs", default=True), From fijal at codespeak.net Tue Nov 23 15:52:49 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 23 Nov 2010 15:52:49 +0100 (CET) Subject: [pypy-svn] r79401 - in pypy/trunk/pypy/config: . test Message-ID: <20101123145249.719E550813@codespeak.net> Author: fijal Date: Tue Nov 23 15:52:47 2010 New Revision: 79401 Added: pypy/trunk/pypy/config/support.py pypy/trunk/pypy/config/test/test_support.py Log: ARGH this goes with previous checkin Added: pypy/trunk/pypy/config/support.py ============================================================================== --- (empty file) +++ pypy/trunk/pypy/config/support.py Tue Nov 23 15:52:47 2010 @@ -0,0 +1,17 @@ + +""" Some support code +""" + +import re + +def detect_number_of_processors(filename_or_file='/proc/cpuinfo'): + try: + if isinstance(filename_or_file, str): + f = open(filename_or_file, "r") + else: + f = filename_or_file + return max([int(re.split('processor.*(\d+)', line)[1]) + for line in f.readlines() + if line.startswith('processor')]) + 1 + except: + return 1 # we really don't want to explode here, at worst we have 1 Added: pypy/trunk/pypy/config/test/test_support.py ============================================================================== --- (empty file) +++ pypy/trunk/pypy/config/test/test_support.py Tue Nov 23 15:52:47 2010 @@ -0,0 +1,36 @@ + +from cStringIO import StringIO +from pypy.config.support import detect_number_of_processors + +cpuinfo = """ +processor : 0 + +processor : 1 +vendor_id : GenuineIntel +cpu family : 6 +model : 37 +model name : Intel(R) Core(TM) i7 CPU L 620 @ 2.00GHz +stepping : 2 + +processor : 2 +vendor_id : GenuineIntel +cpu family : 6 +model : 37 +model name : Intel(R) Core(TM) i7 CPU L 620 @ 2.00GHz +stepping : 2 + +processor : 3 +vendor_id : GenuineIntel +cpu family : 6 +model : 37 +model name : Intel(R) Core(TM) i7 CPU L 620 @ 2.00GHz +stepping : 2 +cpu MHz : 1199.000 +cache size : 4096 KB +physical id : 0 +siblings : 4 +""" + +def test_cpuinfo(): + assert detect_number_of_processors(StringIO(cpuinfo)) == 4 + assert detect_number_of_processors('random crap that does not exist') == 1 From fijal at codespeak.net Tue Nov 23 15:58:35 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 23 Nov 2010 15:58:35 +0100 (CET) Subject: [pypy-svn] r79402 - in pypy/trunk/pypy/jit/backend/x86: . test Message-ID: <20101123145835.85CD6282BEB@codespeak.net> Author: fijal Date: Tue Nov 23 15:58:34 2010 New Revision: 79402 Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py pypy/trunk/pypy/jit/backend/x86/test/test_runner.py pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py Log: Move assembler logging counts into debug_log (now that we can specify multiple things in logs) Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/assembler.py (original) +++ pypy/trunk/pypy/jit/backend/x86/assembler.py Tue Nov 23 15:58:34 2010 @@ -30,7 +30,7 @@ from pypy.jit.backend.x86 import rx86, regloc, codebuf from pypy.jit.metainterp.resoperation import rop, ResOperation from pypy.jit.backend.x86.support import values_array -from pypy.rlib.debug import debug_print +from pypy.rlib.debug import debug_print, debug_start, debug_stop from pypy.rlib import rgc from pypy.jit.backend.x86.jump import remap_frame_layout from pypy.rlib.streamio import open_file_as_stream @@ -231,7 +231,6 @@ if s.find(':') != -1: s = s.split(':')[-1] self.set_debug(True) - self._output_loop_log = s + ".count" # Intialize here instead of __init__ to prevent # pending_guard_tokens from being considered a prebuilt object, # which sometimes causes memory leaks since the prebuilt list is @@ -241,13 +240,11 @@ def finish_once(self): if self._debug: - output_log = self._output_loop_log - assert output_log is not None - f = open_file_as_stream(output_log, "w") + debug_start('jit-backend-counts') for i in range(len(self.loop_run_counters)): name, struct = self.loop_run_counters[i] - f.write(str(name) + ":" + str(struct.i) + "\n") - f.close() + debug_print(str(name) + ':' + str(struct.i)) + debug_stop('jit-backend-counts') def _build_float_constants(self): # 44 bytes: 32 bytes for the data, and up to 12 bytes for alignment Modified: pypy/trunk/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/trunk/pypy/jit/backend/x86/test/test_runner.py Tue Nov 23 15:58:34 2010 @@ -495,6 +495,9 @@ os.environ['PYPYLOG'] = self.pypylog def test_debugger_on(self): + from pypy.tool.logparser import parse_log_file, extract_category + from pypy.rlib import debug + loop = """ [i0] debug_merge_point('xyz', 0) @@ -504,17 +507,21 @@ jump(i1) """ ops = parse(loop) - self.cpu.assembler.set_debug(True) - self.cpu.compile_loop(ops.inputargs, ops.operations, ops.token) - self.cpu.set_future_value_int(0, 0) - self.cpu.execute_token(ops.token) - # check debugging info - name, struct = self.cpu.assembler.loop_run_counters[0] - assert name == 0 # 'xyz' - assert struct.i == 10 - self.cpu.finish_once() - lines = py.path.local(self.logfile + ".count").readlines() - assert lines[0] == '0:10\n' # '10 xyz\n' + debug._log = dlog = debug.DebugLog() + try: + self.cpu.assembler.set_debug(True) + self.cpu.compile_loop(ops.inputargs, ops.operations, ops.token) + self.cpu.set_future_value_int(0, 0) + self.cpu.execute_token(ops.token) + # check debugging info + name, struct = self.cpu.assembler.loop_run_counters[0] + assert name == 0 # 'xyz' + assert struct.i == 10 + self.cpu.finish_once() + finally: + debug._log = None + assert dlog[1] == ('jit-backend-counts', [('debug_print', '0:10')]) + def test_debugger_checksum(self): loop = """ Modified: pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py (original) +++ pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py Tue Nov 23 15:58:34 2010 @@ -87,8 +87,13 @@ return int(res) # def main(i, j): - return f(i, j) + libffi_stuff(i, j) - expected = main(40, -49) + res = f(i, j) + libffi_stuff(i, j) + fd = os.open('/tmp/x', os.O_WRONLY | os.O_CREAT, 0666) + rgc.dump_rpy_heap(fd) + os.close(fd) + return res + #expected = main(40, -49) + expected = 3 res = self.meta_interp(main, [40, -49]) assert res == expected From arigo at codespeak.net Tue Nov 23 16:03:06 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 16:03:06 +0100 (CET) Subject: [pypy-svn] r79403 - in pypy/trunk/pypy/config: . test Message-ID: <20101123150306.E15065080F@codespeak.net> Author: arigo Date: Tue Nov 23 16:03:05 2010 New Revision: 79403 Modified: pypy/trunk/pypy/config/support.py pypy/trunk/pypy/config/test/test_support.py Log: Improve. Modified: pypy/trunk/pypy/config/support.py ============================================================================== --- pypy/trunk/pypy/config/support.py (original) +++ pypy/trunk/pypy/config/support.py Tue Nov 23 16:03:05 2010 @@ -2,9 +2,13 @@ """ Some support code """ -import re +import re, sys, os def detect_number_of_processors(filename_or_file='/proc/cpuinfo'): + if sys.platform != 'linux2': + return 1 # implement me + if os.environ.get('MAKEFLAGS'): + return 1 # don't override MAKEFLAGS. This will call 'make' without any '-j' option try: if isinstance(filename_or_file, str): f = open(filename_or_file, "r") Modified: pypy/trunk/pypy/config/test/test_support.py ============================================================================== --- pypy/trunk/pypy/config/test/test_support.py (original) +++ pypy/trunk/pypy/config/test/test_support.py Tue Nov 23 16:03:05 2010 @@ -1,6 +1,7 @@ from cStringIO import StringIO from pypy.config.support import detect_number_of_processors +import os, sys, py cpuinfo = """ processor : 0 @@ -31,6 +32,22 @@ siblings : 4 """ +class FakeEnviron: + def __init__(self, value): + self._value = value + def get(self, varname): + assert varname == 'MAKEFLAGS' + return self._value + def test_cpuinfo(): - assert detect_number_of_processors(StringIO(cpuinfo)) == 4 - assert detect_number_of_processors('random crap that does not exist') == 1 + if sys.platform != 'linux2': + py.test.skip("linux only") + saved = os.environ + try: + os.environ = FakeEnviron(None) + assert detect_number_of_processors(StringIO(cpuinfo)) == 4 + assert detect_number_of_processors('random crap that does not exist') == 1 + os.environ = FakeEnviron('-j2') + assert detect_number_of_processors(StringIO(cpuinfo)) == 1 + finally: + os.environ = saved From arigo at codespeak.net Tue Nov 23 16:07:54 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 16:07:54 +0100 (CET) Subject: [pypy-svn] r79404 - pypy/trunk/pypy/rpython/memory/gc Message-ID: <20101123150754.17E3D282BEC@codespeak.net> Author: arigo Date: Tue Nov 23 16:07:52 2010 New Revision: 79404 Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py Log: Grumble. Fix. Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Tue Nov 23 16:07:52 2010 @@ -42,7 +42,7 @@ from pypy.rpython.lltypesystem.lloperation import llop from pypy.rpython.lltypesystem.llmemory import raw_malloc_usage from pypy.rpython.memory.gc.base import GCBase, MovingGCBase -from pypy.rpython.memory.gc import minimarkpage, base, env +from pypy.rpython.memory.gc import minimarkpage, env from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask, r_uint from pypy.rlib.rarithmetic import LONG_BIT_SHIFT from pypy.rlib.debug import ll_assert, debug_print, debug_start, debug_stop @@ -289,7 +289,7 @@ # # From there on, the GC is fully initialized and the code # below can use it - newsize = base.read_from_env('PYPY_GC_NURSERY') + newsize = env.read_from_env('PYPY_GC_NURSERY') # PYPY_GC_NURSERY=1 forces a minor collect for every malloc. # Useful to debug external factors, like trackgcroot or the # handling of the write barrier. @@ -300,26 +300,26 @@ newsize = defaultsize newsize = max(newsize, minsize) # - major_coll = base.read_float_from_env('PYPY_GC_MAJOR_COLLECT') + major_coll = env.read_float_from_env('PYPY_GC_MAJOR_COLLECT') if major_coll > 1.0: self.major_collection_threshold = major_coll # - growth = base.read_float_from_env('PYPY_GC_GROWTH') + growth = env.read_float_from_env('PYPY_GC_GROWTH') if growth > 1.0: self.growth_rate_max = growth # - min_heap_size = base.read_uint_from_env('PYPY_GC_MIN') + min_heap_size = env.read_uint_from_env('PYPY_GC_MIN') if min_heap_size > 0: self.min_heap_size = float(min_heap_size) else: # defaults to 8 times the nursery self.min_heap_size = newsize * 8 # - max_heap_size = base.read_uint_from_env('PYPY_GC_MAX') + max_heap_size = env.read_uint_from_env('PYPY_GC_MAX') if max_heap_size > 0: self.max_heap_size = float(max_heap_size) # - max_delta = base.read_uint_from_env('PYPY_GC_MAX_DELTA') + max_delta = env.read_uint_from_env('PYPY_GC_MAX_DELTA') if max_delta > 0: self.max_delta = float(max_delta) else: From arigo at codespeak.net Tue Nov 23 16:08:20 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 16:08:20 +0100 (CET) Subject: [pypy-svn] r79405 - pypy/branch/jit-free/pypy/rpython/memory/gc Message-ID: <20101123150820.C8DDB5080F@codespeak.net> Author: arigo Date: Tue Nov 23 16:08:19 2010 New Revision: 79405 Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py Log: Merge from trunk. Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py Tue Nov 23 16:08:19 2010 @@ -42,7 +42,7 @@ from pypy.rpython.lltypesystem.lloperation import llop from pypy.rpython.lltypesystem.llmemory import raw_malloc_usage from pypy.rpython.memory.gc.base import GCBase, MovingGCBase -from pypy.rpython.memory.gc import minimarkpage, base, env +from pypy.rpython.memory.gc import minimarkpage, env from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask, r_uint from pypy.rlib.rarithmetic import LONG_BIT_SHIFT from pypy.rlib.debug import ll_assert, debug_print, debug_start, debug_stop @@ -290,7 +290,7 @@ # # From there on, the GC is fully initialized and the code # below can use it - newsize = base.read_from_env('PYPY_GC_NURSERY') + newsize = env.read_from_env('PYPY_GC_NURSERY') # PYPY_GC_NURSERY=1 forces a minor collect for every malloc. # Useful to debug external factors, like trackgcroot or the # handling of the write barrier. @@ -301,26 +301,26 @@ newsize = defaultsize newsize = max(newsize, minsize) # - major_coll = base.read_float_from_env('PYPY_GC_MAJOR_COLLECT') + major_coll = env.read_float_from_env('PYPY_GC_MAJOR_COLLECT') if major_coll > 1.0: self.major_collection_threshold = major_coll # - growth = base.read_float_from_env('PYPY_GC_GROWTH') + growth = env.read_float_from_env('PYPY_GC_GROWTH') if growth > 1.0: self.growth_rate_max = growth # - min_heap_size = base.read_uint_from_env('PYPY_GC_MIN') + min_heap_size = env.read_uint_from_env('PYPY_GC_MIN') if min_heap_size > 0: self.min_heap_size = float(min_heap_size) else: # defaults to 8 times the nursery self.min_heap_size = newsize * 8 # - max_heap_size = base.read_uint_from_env('PYPY_GC_MAX') + max_heap_size = env.read_uint_from_env('PYPY_GC_MAX') if max_heap_size > 0: self.max_heap_size = float(max_heap_size) # - max_delta = base.read_uint_from_env('PYPY_GC_MAX_DELTA') + max_delta = env.read_uint_from_env('PYPY_GC_MAX_DELTA') if max_delta > 0: self.max_delta = float(max_delta) else: From cfbolz at codespeak.net Tue Nov 23 16:13:24 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 23 Nov 2010 16:13:24 +0100 (CET) Subject: [pypy-svn] r79406 - in pypy/branch/reflex-support2/pypy: config jit/backend/llgraph/test jit/codewriter jit/codewriter/test jit/metainterp jit/metainterp/optimizeopt jit/tl module/cppyy module/pypyjit translator/c translator/tool Message-ID: <20101123151324.EF34C50810@codespeak.net> Author: cfbolz Date: Tue Nov 23 16:13:22 2010 New Revision: 79406 Added: pypy/branch/reflex-support2/pypy/module/cppyy/ (props changed) - copied from r79403, pypy/branch/reflex-support/pypy/module/cppyy/ Modified: pypy/branch/reflex-support2/pypy/config/pypyoption.py pypy/branch/reflex-support2/pypy/jit/backend/llgraph/test/test_llgraph.py pypy/branch/reflex-support2/pypy/jit/codewriter/effectinfo.py pypy/branch/reflex-support2/pypy/jit/codewriter/test/test_effectinfo.py pypy/branch/reflex-support2/pypy/jit/metainterp/optimizeopt/heap.py pypy/branch/reflex-support2/pypy/jit/metainterp/warmspot.py pypy/branch/reflex-support2/pypy/jit/tl/pypyjit_child.py pypy/branch/reflex-support2/pypy/jit/tl/pypyjit_demo.py pypy/branch/reflex-support2/pypy/module/pypyjit/policy.py pypy/branch/reflex-support2/pypy/translator/c/genc.py pypy/branch/reflex-support2/pypy/translator/tool/cbuild.py Log: merge in the changes from reflex-support: ------------------------------------------------------------------------ r79226 | wlav | 2010-11-18 03:10:29 +0100 (Thu, 18 Nov 2010) | 1 line Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py rpython and comment fixes ------------------------------------------------------------------------ r79220 | cfbolz | 2010-11-17 19:47:56 +0100 (Wed, 17 Nov 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py fix another issue ------------------------------------------------------------------------ r79219 | cfbolz | 2010-11-17 19:24:23 +0100 (Wed, 17 Nov 2010) | 4 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py - factor some common code out into a helper function - also use space.interp_w to make sure that the objects in question are really W_CPPInstances. This should help translation. ------------------------------------------------------------------------ r79209 | cfbolz | 2010-11-17 18:57:20 +0100 (Wed, 17 Nov 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py use the official way to check whether w_value is an int ------------------------------------------------------------------------ r79156 | cfbolz | 2010-11-16 17:56:23 +0100 (Tue, 16 Nov 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py try to fix some RPython issues ------------------------------------------------------------------------ r79137 | wlav | 2010-11-16 00:48:06 +0100 (Tue, 16 Nov 2010) | 1 line Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/helper.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_helper.py mods to be rpython compliant ------------------------------------------------------------------------ r78903 | wlav | 2010-11-09 02:18:57 +0100 (Tue, 09 Nov 2010) | 1 line Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py array support for all integer types ------------------------------------------------------------------------ r76862 | wlav | 2010-09-03 20:52:08 +0200 (Fri, 03 Sep 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py Initial long array data member access. ------------------------------------------------------------------------ r76861 | wlav | 2010-09-03 20:50:49 +0200 (Fri, 03 Sep 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py Initial unsigned int array data member access. ------------------------------------------------------------------------ r76860 | wlav | 2010-09-03 20:20:24 +0200 (Fri, 03 Sep 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py Initial int array data member access. ------------------------------------------------------------------------ r76828 | wlav | 2010-09-02 02:31:09 +0200 (Thu, 02 Sep 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py Initial support for unsigned short int data member access. ------------------------------------------------------------------------ r76827 | wlav | 2010-09-02 02:20:18 +0200 (Thu, 02 Sep 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/helper.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/datatypes.h M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_helper.py Initial implementation for passing short int arrays. ------------------------------------------------------------------------ r76674 | wlav | 2010-08-20 02:19:47 +0200 (Fri, 20 Aug 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py Allow setting of boolean data members. ------------------------------------------------------------------------ r76481 | wlav | 2010-08-05 00:10:46 +0200 (Thu, 05 Aug 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py Executors and converters for more integer types ------------------------------------------------------------------------ r76462 | wlav | 2010-08-04 00:33:26 +0200 (Wed, 04 Aug 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py Enable float and double data members ------------------------------------------------------------------------ r76461 | wlav | 2010-08-03 23:49:16 +0200 (Tue, 03 Aug 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py Enable passing int value to C char ------------------------------------------------------------------------ r76460 | wlav | 2010-08-03 23:22:05 +0200 (Tue, 03 Aug 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/helper.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/pythonify.py M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py Support for public data members of type (unsigned) char and renaming various names for consistency. ------------------------------------------------------------------------ r76333 | wlav | 2010-07-24 01:36:35 +0200 (Sat, 24 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py Allow passing of C-floats. ------------------------------------------------------------------------ r76283 | wlav | 2010-07-19 19:56:55 +0200 (Mon, 19 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py Initial converters/executors for char and unsigned char. ------------------------------------------------------------------------ r76234 | wlav | 2010-07-15 12:28:20 +0200 (Thu, 15 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Improved error reporting when an overload fails. ------------------------------------------------------------------------ r76208 | wlav | 2010-07-14 17:29:51 +0200 (Wed, 14 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/Makefile A /pypy/branch/reflex-support/pypy/module/cppyy/test/datatypes.cxx A /pypy/branch/reflex-support/pypy/module/cppyy/test/datatypes.h M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py A /pypy/branch/reflex-support/pypy/module/cppyy/test/test_datatypes.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_pythonify.py Preparation of new test for the builtin datatypes (modified from roottest), and a bool executor/converter. ------------------------------------------------------------------------ r76194 | wlav | 2010-07-14 11:09:13 +0200 (Wed, 14 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/pythonify.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.h M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_pythonify.py Enable returning objects from static classes and some code cleanup/cutification. ------------------------------------------------------------------------ r76188 | wlav | 2010-07-13 17:26:09 +0200 (Tue, 13 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/pythonify.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_pythonify.py Allow methods to return instances. ------------------------------------------------------------------------ r76180 | antocuni | 2010-07-13 14:29:22 +0200 (Tue, 13 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py fix translation ------------------------------------------------------------------------ r76175 | arigo | 2010-07-13 12:16:38 +0200 (Tue, 13 Jul 2010) | 4 lines Changed paths: M /pypy/branch/reflex-support/pypy/jit/codewriter/effectinfo.py M /pypy/branch/reflex-support/pypy/jit/codewriter/test/test_effectinfo.py M /pypy/branch/reflex-support/pypy/jit/metainterp/optimizeopt.py M /pypy/branch/reflex-support/pypy/jit/metainterp/pyjitpl.py Extend the EffectInfo to distinguish between the cases EF_PURE etc., even for cases of general escaping where we don't have any list of fields. ------------------------------------------------------------------------ r76163 | wlav | 2010-07-12 22:26:26 +0200 (Mon, 12 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.h M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_pythonify.py Allow executor calls returning objects to succeed (the resulting return object is not yet usuable, though). ------------------------------------------------------------------------ r76157 | arigo | 2010-07-12 19:08:30 +0200 (Mon, 12 Jul 2010) | 3 lines Changed paths: M /pypy/branch/reflex-support/pypy/jit/codewriter/effectinfo.py M /pypy/branch/reflex-support/pypy/jit/codewriter/test/test_effectinfo.py M /pypy/branch/reflex-support/pypy/translator/backendopt/graphanalyze.py Revert r76155, which does not help, and the test in r76154, which is a bit bogus. ------------------------------------------------------------------------ r76155 | arigo | 2010-07-12 18:43:30 +0200 (Mon, 12 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/jit/codewriter/effectinfo.py M /pypy/branch/reflex-support/pypy/translator/backendopt/graphanalyze.py Fix the bug in a not-completely-satisfying way... ------------------------------------------------------------------------ r76154 | arigo | 2010-07-12 18:29:37 +0200 (Mon, 12 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/jit/codewriter/test/test_effectinfo.py A failing test. ------------------------------------------------------------------------ r76070 | antocuni | 2010-07-09 13:02:29 +0200 (Fri, 09 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py fix incorrect signature ------------------------------------------------------------------------ r76069 | cfbolz | 2010-07-09 12:53:46 +0200 (Fri, 09 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy M /pypy/branch/reflex-support/pypy/module/cppyy/include M /pypy/branch/reflex-support/pypy/module/cppyy/include/cppyy.h M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/pythonify.py M /pypy/branch/reflex-support/pypy/module/cppyy/test M /pypy/branch/reflex-support/pypy/module/cppyy/test/bench1.py fixeol and fix svn properties ------------------------------------------------------------------------ r76064 | wlav | 2010-07-09 11:38:47 +0200 (Fri, 09 Jul 2010) | 2 lines Changed paths: A /pypy/branch/reflex-support/pypy/module/cppyy/test/bench1.cxx Initial C++ benchmark. ------------------------------------------------------------------------ r76062 | arigo | 2010-07-09 11:10:27 +0200 (Fri, 09 Jul 2010) | 2 lines Changed paths: A /pypy/branch/reflex-support/pypy/module/cppyy/test/bench1.py A benchmark. ------------------------------------------------------------------------ r76061 | antocuni | 2010-07-09 11:01:34 +0200 (Fri, 09 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/helper.py translation fix ------------------------------------------------------------------------ r76059 | wlav | 2010-07-09 10:45:56 +0200 (Fri, 09 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/test/Makefile M /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.cxx A /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.h (cfbolz, wlav) Cleanup example01 C++ code in test. ------------------------------------------------------------------------ r76058 | antocuni | 2010-07-09 10:44:15 +0200 (Fri, 09 Jul 2010) | 4 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx (arigo, antocuni): revert r76042 as it makes things slower. Moreover, turn get_methptr_getter into a pure function, to help the jit ------------------------------------------------------------------------ r76053 | wlav | 2010-07-08 20:30:36 +0200 (Thu, 08 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx Make C API consistent. ------------------------------------------------------------------------ r76051 | arigo | 2010-07-08 20:23:57 +0200 (Thu, 08 Jul 2010) | 3 lines Changed paths: M /pypy/branch/reflex-support/pypy/translator/c/genc.py Tentatively fix the Makefile to not call trackgcroot.py on some C files, and on all other files (.cpp, .cxx...) ------------------------------------------------------------------------ r76048 | wlav | 2010-07-08 20:04:20 +0200 (Thu, 08 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_pythonify.py Some more tests. ------------------------------------------------------------------------ r76046 | wlav | 2010-07-08 19:37:02 +0200 (Thu, 08 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py A /pypy/branch/reflex-support/pypy/module/cppyy/helper.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py A /pypy/branch/reflex-support/pypy/module/cppyy/test/test_helper.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_pythonify.py (wlav, cfbolz): work enough to be able to pass an instance as an argument. ------------------------------------------------------------------------ r76044 | arigo | 2010-07-08 18:17:14 +0200 (Thu, 08 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py Translation fix. ------------------------------------------------------------------------ r76043 | antocuni | 2010-07-08 18:06:02 +0200 (Thu, 08 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/config/pypyoption.py always enable this module on the branch ------------------------------------------------------------------------ r76042 | antocuni | 2010-07-08 18:04:42 +0200 (Thu, 08 Jul 2010) | 5 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx (antocuni, arigo around) huge pile of custom hacks to convince the jit that the function pointer to the method that we get is constant, given the dynamic class of the cpp object ------------------------------------------------------------------------ r76041 | arigo | 2010-07-08 18:01:43 +0200 (Thu, 08 Jul 2010) | 3 lines Changed paths: M /pypy/branch/reflex-support/pypy/jit/metainterp/pyjitpl.py M /pypy/branch/reflex-support/pypy/jit/metainterp/warmspot.py M /pypy/branch/reflex-support/pypy/jit/tl/pypyjit_child.py Update pypyjit_child.py to force the jitcodes to be dumped to disk in that case. ------------------------------------------------------------------------ r76040 | wlav | 2010-07-08 17:43:20 +0200 (Thu, 08 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_pythonify.py (cfbolz, wlav) Implementation of overloads for constructors from the python side. ------------------------------------------------------------------------ r76039 | arigo | 2010-07-08 17:35:13 +0200 (Thu, 08 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/jit/tl/pypyjit_demo.py Fixes. ------------------------------------------------------------------------ r76038 | arigo | 2010-07-08 17:23:45 +0200 (Thu, 08 Jul 2010) | 5 lines Changed paths: M /pypy/branch/reflex-support/pypy/translator/tool/cbuild.py Running some tests depending on cbuild would create externmod_1.so, externmod_2.so, etc., in a random directory, the list growing forever. Attempt to fix that by forcing the externmod_x.so to be created in the usession directory. ------------------------------------------------------------------------ r76036 | antocuni | 2010-07-08 17:08:22 +0200 (Thu, 08 Jul 2010) | 4 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx introduce two new types, cppyy_typehandle_t and cppyy_object_t, which are just void* but make the code a bit easier to understand ------------------------------------------------------------------------ r76035 | arigo | 2010-07-08 17:06:40 +0200 (Thu, 08 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/jit/tl/pypyjit_demo.py Fix with full relative path. ------------------------------------------------------------------------ r76033 | arigo | 2010-07-08 16:21:00 +0200 (Thu, 08 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Translation fix. ------------------------------------------------------------------------ r76029 | arigo | 2010-07-08 16:02:20 +0200 (Thu, 08 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Translation fixes. ------------------------------------------------------------------------ r76020 | arigo | 2010-07-08 13:25:00 +0200 (Thu, 08 Jul 2010) | 7 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/Makefile (antocuni, arigo) Hack: manually do the fast-path construction enabled by --with-methptrgetter for methods of signature 'int m(int)' only. The JIT should speed that case up. ------------------------------------------------------------------------ r76019 | wlav | 2010-07-08 13:12:42 +0200 (Thu, 08 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/pythonify.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_pythonify.py (wlav, cfbolz): add support for constructing an instance of a class and calling methods on it. ------------------------------------------------------------------------ r76018 | arigo | 2010-07-08 12:53:20 +0200 (Thu, 08 Jul 2010) | 3 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/Makefile M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py Support two installations: either genreflex is installed, or ROOT is in $ROOTSYS. ------------------------------------------------------------------------ r76017 | wlav | 2010-07-08 12:38:17 +0200 (Thu, 08 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/__init__.py M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/pythonify.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_pythonify.py (wlav, cfbolz): start exposing c++ classes and support static methods on them. ------------------------------------------------------------------------ r76014 | antocuni | 2010-07-08 12:20:39 +0200 (Thu, 08 Jul 2010) | 3 lines Changed paths: A /pypy/branch/reflex-support/pypy/module/cppyy/genreflex-methptrgetter.patch (arigo, antocuni) a patch for genreflex.py that adds the MethPtrGetter property to the FunctionMembers of a class ------------------------------------------------------------------------ r76011 | wlav | 2010-07-08 11:00:42 +0200 (Thu, 08 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/__init__.py A /pypy/branch/reflex-support/pypy/module/cppyy/pythonify.py A /pypy/branch/reflex-support/pypy/module/cppyy/test/test_pythonify.py (cfbolz,wlav) New module pythonify for user interface, and its test. Make load_lib cache resulting libraries. ------------------------------------------------------------------------ r75998 | wlav | 2010-07-07 20:04:02 +0200 (Wed, 07 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py Support calls for double and char* on instances. ------------------------------------------------------------------------ r75991 | antocuni | 2010-07-07 18:28:02 +0200 (Wed, 07 Jul 2010) | 5 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/include/cppyy.h M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx lookup and print the property "MethPtrGetter" for methods; so far it's unused and it's always empty, because genreflex.py does not generate this information yet. ------------------------------------------------------------------------ r75987 | wlav | 2010-07-07 18:13:31 +0200 (Wed, 07 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx Merge static and method call for resolution on the C++ side. ------------------------------------------------------------------------ r75985 | wlav | 2010-07-07 17:48:01 +0200 (Wed, 07 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx Use an opaque handle instead of class name to locate Reflex::Type. ------------------------------------------------------------------------ r75964 | wlav | 2010-07-07 12:46:51 +0200 (Wed, 07 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Made method call also use executors. ------------------------------------------------------------------------ r75963 | cfbolz | 2010-07-07 12:44:06 +0200 (Wed, 07 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h (cfbolz, wlav, arigo): make this work in C ------------------------------------------------------------------------ r75962 | arigo | 2010-07-07 12:16:48 +0200 (Wed, 07 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py Tentative fix of translation. ------------------------------------------------------------------------ r75961 | arigo | 2010-07-07 11:47:53 +0200 (Wed, 07 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/capi.py M /pypy/branch/reflex-support/pypy/module/cppyy/executor.py Remove svn:executable. ------------------------------------------------------------------------ r75960 | wlav | 2010-07-07 11:44:56 +0200 (Wed, 07 Jul 2010) | 2 lines Changed paths: A /pypy/branch/reflex-support/pypy/module/cppyy/capi.py A /pypy/branch/reflex-support/pypy/module/cppyy/executor.py M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py (cfbolz, wlav) Make use of the new executor module and moved all Reflex-C interface code to capi.py. ------------------------------------------------------------------------ r75955 | arigo | 2010-07-07 10:43:49 +0200 (Wed, 07 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/jit/backend/llgraph/llimpl.py Fix for metainterp/test/test_ztranslation. ------------------------------------------------------------------------ r75954 | arigo | 2010-07-07 10:36:41 +0200 (Wed, 07 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/jit/backend/llsupport/support.py Fix for x86/test/test_basic.test_raw_malloc_and_access ------------------------------------------------------------------------ r75953 | cfbolz | 2010-07-07 10:25:38 +0200 (Wed, 07 Jul 2010) | 3 lines Changed paths: M /pypy/branch/reflex-support/pypy/jit/backend/llsupport/descr.py (arigo, cfbolz): add some sanity asserts to check that C arrays don't need a typeid. don't try to get a typeid for raw arrays. ------------------------------------------------------------------------ r75940 | cfbolz | 2010-07-06 19:49:36 +0200 (Tue, 06 Jul 2010) | 3 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py (cfbolz, arigo, antocuni): refactor exception handling to help the JIT for completely unobvious reasons :-( ------------------------------------------------------------------------ r75935 | wlav | 2010-07-06 19:15:42 +0200 (Tue, 06 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py (wlav) Add handling for char* return type on static functions. ------------------------------------------------------------------------ r75934 | cfbolz | 2010-07-06 19:03:23 +0200 (Tue, 06 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/jit/backend/llgraph/llimpl.py M /pypy/branch/reflex-support/pypy/jit/backend/llgraph/test/test_llgraph.py (cfbolz, arigo): slightly obscure: need an unsafe cast in setarrayitem_raw_int ------------------------------------------------------------------------ r75932 | wlav | 2010-07-06 18:45:10 +0200 (Tue, 06 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py (wlav, cfbolz) Add const char* argument passing and refactor converter lookup. ------------------------------------------------------------------------ r75931 | arigo | 2010-07-06 18:26:59 +0200 (Tue, 06 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/annotation/bookkeeper.py M /pypy/branch/reflex-support/pypy/annotation/model.py M /pypy/branch/reflex-support/pypy/annotation/test/test_model.py M /pypy/branch/reflex-support/pypy/jit/backend/llsupport/descr.py M /pypy/branch/reflex-support/pypy/jit/backend/llsupport/llmodel.py M /pypy/branch/reflex-support/pypy/jit/backend/llsupport/symbolic.py M /pypy/branch/reflex-support/pypy/jit/backend/llsupport/test/test_descr.py M /pypy/branch/reflex-support/pypy/jit/backend/llsupport/test/test_symbolic.py M /pypy/branch/reflex-support/pypy/jit/backend/x86/assembler.py M /pypy/branch/reflex-support/pypy/jit/backend/x86/regalloc.py M /pypy/branch/reflex-support/pypy/module/_locale/interp_locale.py M /pypy/branch/reflex-support/pypy/rlib/rlocale.py M /pypy/branch/reflex-support/pypy/rpython/lltypesystem/rffi.py M /pypy/branch/reflex-support/pypy/rpython/lltypesystem/test/test_rffi.py M /pypy/branch/reflex-support/pypy/rpython/memory/lltypelayout.py Merge trunk, r75828:HEAD. ------------------------------------------------------------------------ r75928 | cfbolz | 2010-07-06 17:36:01 +0200 (Tue, 06 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py add some more hints ------------------------------------------------------------------------ r75925 | cfbolz | 2010-07-06 17:17:10 +0200 (Tue, 06 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/jit/codewriter/jtransform.py M /pypy/branch/reflex-support/pypy/jit/codewriter/test/test_flatten.py start some support for very simple force_cast cases ------------------------------------------------------------------------ r75917 | cfbolz | 2010-07-06 16:31:31 +0200 (Tue, 06 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/pypyjit/policy.py let the JIT see our new module ------------------------------------------------------------------------ r75913 | cfbolz | 2010-07-06 16:12:39 +0200 (Tue, 06 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/jit/tl/pypyjit.py M /pypy/branch/reflex-support/pypy/jit/tl/pypyjit_demo.py the code that we would like to make fast with the JIT ------------------------------------------------------------------------ r75910 | cfbolz | 2010-07-06 16:01:27 +0200 (Tue, 06 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py (wlav, cfbolz, antocuni, arigo): refactor stuff to hopefully be more JIT-friendly ------------------------------------------------------------------------ r75904 | cfbolz | 2010-07-06 15:09:56 +0200 (Tue, 06 Jul 2010) | 3 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py (antocuni, cfbolz, wlav, arigo): fix one annotation problem, not sure it's the only one. Prefix all C-functions with "c_". ------------------------------------------------------------------------ r75899 | cfbolz | 2010-07-06 13:13:14 +0200 (Tue, 06 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py (wlav, cfbolz, antocuni, arigo around): test and fix ------------------------------------------------------------------------ r75896 | cfbolz | 2010-07-06 12:50:11 +0200 (Tue, 06 Jul 2010) | 3 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py (antocuni, wlav, cfbolz, arigo around): fix memory leak. test overloading by number of arguments. ------------------------------------------------------------------------ r75895 | cfbolz | 2010-07-06 12:37:23 +0200 (Tue, 06 Jul 2010) | 3 lines Changed paths: A /pypy/branch/reflex-support/pypy/module/cppyy/converter.py M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py (antocuni, cfbolz, wlav, arigo around): start going in the direction of supporting overloading. support returning doubles from static methods. ------------------------------------------------------------------------ r75882 | arigo | 2010-07-06 09:11:59 +0200 (Tue, 06 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.cxx Kill the svn:executable property on this file. ------------------------------------------------------------------------ r75847 | cfbolz | 2010-07-05 19:09:53 +0200 (Mon, 05 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py (antocuni, cfbolz, wlav, arigo): call the destructor too. ------------------------------------------------------------------------ r75843 | cfbolz | 2010-07-05 18:48:06 +0200 (Mon, 05 Jul 2010) | 2 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/include/cppyy.h M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx (arigo, antocuni, wlav, cfbolz): clean up the calling of constructors. ------------------------------------------------------------------------ r75839 | cfbolz | 2010-07-05 18:23:24 +0200 (Mon, 05 Jul 2010) | 4 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/include/cppyy.h M /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx M /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py (arigo, wlav, antocuni, cfbolz): hack some more until we are able to construct an object (with a complete hack) and call a method on it. This still leaks memory. ------------------------------------------------------------------------ r75835 | cfbolz | 2010-07-05 17:19:56 +0200 (Mon, 05 Jul 2010) | 3 lines Changed paths: A /pypy/branch/reflex-support/pypy/module/cppyy/include A /pypy/branch/reflex-support/pypy/module/cppyy/include/cppyy.h A /pypy/branch/reflex-support/pypy/module/cppyy/include/reflexcwrapper.h M /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py A /pypy/branch/reflex-support/pypy/module/cppyy/src A /pypy/branch/reflex-support/pypy/module/cppyy/src/reflexcwrapper.cxx (cfbolz, wlav, antocuni, arigo): write just enough code to be able to call a static method ------------------------------------------------------------------------ r75834 | cfbolz | 2010-07-05 15:25:54 +0200 (Mon, 05 Jul 2010) | 3 lines Changed paths: M /pypy/branch/reflex-support/pypy/module/cppyy/__init__.py A /pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py M /pypy/branch/reflex-support/pypy/module/cppyy/test A /pypy/branch/reflex-support/pypy/module/cppyy/test/Makefile A /pypy/branch/reflex-support/pypy/module/cppyy/test/example01.cxx A /pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py (arigo, antocuni, cfbolz, wlav): a tiny C++ class and a test of how we want the interface to look for now. Now the real fun starts. ------------------------------------------------------------------------ r75831 | cfbolz | 2010-07-05 13:54:02 +0200 (Mon, 05 Jul 2010) | 2 lines Changed paths: A /pypy/branch/reflex-support/pypy/module/cppyy A /pypy/branch/reflex-support/pypy/module/cppyy/__init__.py A /pypy/branch/reflex-support/pypy/module/cppyy/test A /pypy/branch/reflex-support/pypy/module/cppyy/test/__init__.py (arigo, antocuni, cfbolz, wlav) the empty module that we want to write ------------------------------------------------------------------------ r75828 | cfbolz | 2010-07-05 11:56:12 +0200 (Mon, 05 Jul 2010) | 2 lines Changed paths: A /pypy/branch/reflex-support (from /pypy/trunk:75827) branch for the CERN sprint ------------------------------------------------------------------------ Modified: pypy/branch/reflex-support2/pypy/config/pypyoption.py ============================================================================== --- pypy/branch/reflex-support2/pypy/config/pypyoption.py (original) +++ pypy/branch/reflex-support2/pypy/config/pypyoption.py Tue Nov 23 16:13:22 2010 @@ -31,7 +31,7 @@ "crypt", "signal", "_rawffi", "termios", "zlib", "struct", "md5", "sha", "bz2", "_minimal_curses", "cStringIO", "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array", - "_bisect"] + "_bisect", "cppyy"] )) translation_modules = default_modules.copy() Modified: pypy/branch/reflex-support2/pypy/jit/backend/llgraph/test/test_llgraph.py ============================================================================== --- pypy/branch/reflex-support2/pypy/jit/backend/llgraph/test/test_llgraph.py (original) +++ pypy/branch/reflex-support2/pypy/jit/backend/llgraph/test/test_llgraph.py Tue Nov 23 16:13:22 2010 @@ -32,6 +32,21 @@ assert heaptracker.adr2int(llmemory.NULL) == 0 assert heaptracker.int2adr(0) == llmemory.NULL +def test_force_cast_ptr_to_other_ptr(): + from pypy.jit.backend.llgraph import llimpl + from pypy.rpython.lltypesystem import rffi, llmemory + x = lltype.malloc(rffi.DOUBLEP.TO, 1, flavor='raw') + x[0] = 0.1 + args = lltype.malloc(rffi.CArray(rffi.VOIDP), 1, flavor='raw', zero=True) + assert not args[0] + arrayasint = llmemory.cast_adr_to_int(llmemory.cast_ptr_to_adr(args), mode="symbolic") + ptrasint = llmemory.cast_adr_to_int(llmemory.cast_ptr_to_adr(x), mode="symbolic") + llimpl.do_setarrayitem_raw_int(arrayasint, 0, ptrasint) + assert args[0] + lltype.free(x, flavor='raw') + lltype.free(args, flavor='raw') + + ## these tests never worked ## class TestOOTypeLLGraph(LLGraphTest): ## from pypy.jit.backend.llgraph.runner import OOtypeCPU as cpu_type Modified: pypy/branch/reflex-support2/pypy/jit/codewriter/effectinfo.py ============================================================================== --- pypy/branch/reflex-support2/pypy/jit/codewriter/effectinfo.py (original) +++ pypy/branch/reflex-support2/pypy/jit/codewriter/effectinfo.py Tue Nov 23 16:13:22 2010 @@ -51,9 +51,9 @@ write_descrs_fields, write_descrs_arrays, extraeffect=EF_CAN_RAISE, oopspecindex=OS_NONE): - key = (frozenset(readonly_descrs_fields), - frozenset(write_descrs_fields), - frozenset(write_descrs_arrays), + key = (_frozenset_or_none(readonly_descrs_fields), + _frozenset_or_none(write_descrs_fields), + _frozenset_or_none(write_descrs_arrays), extraeffect, oopspecindex) if key in cls._cache: @@ -70,12 +70,17 @@ def check_forces_virtual_or_virtualizable(self): return self.extraeffect >= self.EF_FORCES_VIRTUAL_OR_VIRTUALIZABLE + +def _frozenset_or_none(x): + if x is None: return None + return frozenset(x) + def effectinfo_from_writeanalyze(effects, cpu, extraeffect=EffectInfo.EF_CAN_RAISE, oopspecindex=EffectInfo.OS_NONE): from pypy.translator.backendopt.writeanalyze import top_set if effects is top_set: - return None + return EffectInfo(None, None, None, extraeffect) readonly_descrs_fields = [] # readonly_descrs_arrays = [] --- not enabled for now write_descrs_fields = [] Modified: pypy/branch/reflex-support2/pypy/jit/codewriter/test/test_effectinfo.py ============================================================================== --- pypy/branch/reflex-support2/pypy/jit/codewriter/test/test_effectinfo.py (original) +++ pypy/branch/reflex-support2/pypy/jit/codewriter/test/test_effectinfo.py Tue Nov 23 16:13:22 2010 @@ -3,6 +3,7 @@ from pypy.rpython.ootypesystem import ootype from pypy.jit.codewriter.effectinfo import effectinfo_from_writeanalyze,\ EffectInfo +from pypy.translator.backendopt.writeanalyze import top_set class FakeCPU: def fielddescrof(self, T, fieldname): @@ -86,3 +87,19 @@ assert not effectinfo.readonly_descrs_fields assert not effectinfo.write_descrs_fields assert not effectinfo.write_descrs_arrays + +def test_no_effectinfo(): + effectinfo = effectinfo_from_writeanalyze(top_set, None, + EffectInfo.EF_CANNOT_RAISE) + assert effectinfo.readonly_descrs_fields is None + assert effectinfo.write_descrs_fields is None + assert effectinfo.write_descrs_arrays is None + assert effectinfo.extraeffect == EffectInfo.EF_CANNOT_RAISE + # + effectinfo2 = effectinfo_from_writeanalyze(top_set, None, + EffectInfo.EF_CANNOT_RAISE) + assert effectinfo2 is effectinfo + # + effectinfo3 = effectinfo_from_writeanalyze(top_set, None, + EffectInfo.EF_PURE) + assert effectinfo3.extraeffect == EffectInfo.EF_PURE Modified: pypy/branch/reflex-support2/pypy/jit/metainterp/optimizeopt/heap.py ============================================================================== --- pypy/branch/reflex-support2/pypy/jit/metainterp/optimizeopt/heap.py (original) +++ pypy/branch/reflex-support2/pypy/jit/metainterp/optimizeopt/heap.py Tue Nov 23 16:13:22 2010 @@ -118,7 +118,8 @@ effectinfo = None else: effectinfo = op.getdescr().get_extra_info() - if effectinfo is not None: + if (effectinfo is not None and + effectinfo.readonly_descrs_fields is not None): # XXX we can get the wrong complexity here, if the lists # XXX stored on effectinfo are large for fielddescr in effectinfo.readonly_descrs_fields: Modified: pypy/branch/reflex-support2/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/branch/reflex-support2/pypy/jit/metainterp/warmspot.py (original) +++ pypy/branch/reflex-support2/pypy/jit/metainterp/warmspot.py Tue Nov 23 16:13:22 2010 @@ -148,7 +148,8 @@ class WarmRunnerDesc(object): def __init__(self, translator, policy=None, backendopt=True, CPUClass=None, - optimizer=None, ProfilerClass=EmptyProfiler, **kwds): + optimizer=None, ProfilerClass=EmptyProfiler, + write_jitcodes_directory=False, **kwds): pyjitpl._warmrunnerdesc = self # this is a global for debugging only! self.set_translator(translator) self.build_cpu(CPUClass, **kwds) @@ -177,6 +178,8 @@ self.rewrite_jit_merge_points(policy) verbose = not self.cpu.translate_support_code + if write_jitcodes_directory: + verbose = False self.codewriter.make_jitcodes(verbose=verbose) self.rewrite_can_enter_jits() self.rewrite_set_param() Modified: pypy/branch/reflex-support2/pypy/jit/tl/pypyjit_child.py ============================================================================== --- pypy/branch/reflex-support2/pypy/jit/tl/pypyjit_child.py (original) +++ pypy/branch/reflex-support2/pypy/jit/tl/pypyjit_child.py Tue Nov 23 16:13:22 2010 @@ -35,5 +35,6 @@ warmspot.jittify_and_run(interp, graph, [], policy=policy, listops=True, CPUClass=CPUClass, backendopt=True, inline=True, + write_jitcodes_directory=True, optimizer=OPTIMIZER_FULL) Modified: pypy/branch/reflex-support2/pypy/jit/tl/pypyjit_demo.py ============================================================================== --- pypy/branch/reflex-support2/pypy/jit/tl/pypyjit_demo.py (original) +++ pypy/branch/reflex-support2/pypy/jit/tl/pypyjit_demo.py Tue Nov 23 16:13:22 2010 @@ -1,72 +1,14 @@ -## base = object +import cppyy -## class Number(base): -## __slots__ = ('val', ) -## def __init__(self, val=0): -## self.val = val - -## def __add__(self, other): -## if not isinstance(other, int): -## other = other.val -## return Number(val=self.val + other) - -## def __cmp__(self, other): -## val = self.val -## if not isinstance(other, int): -## other = other.val -## return cmp(val, other) - -## def __nonzero__(self): -## return bool(self.val) - -## def g(x, inc=2): -## return x + inc - -## def f(n, x, inc): -## while x < n: -## x = g(x, inc=1) -## return x - -## import time -## #t1 = time.time() -## #f(10000000, Number(), 1) -## #t2 = time.time() -## #print t2 - t1 -## t1 = time.time() -## f(10000000, 0, 1) -## t2 = time.time() -## print t2 - t1 - -try: - from array import array - - def coords(w,h): - y = 0 - while y < h: - x = 0 - while x < w: - yield x,y - x += 1 - y += 1 - - def f(img): - sa=0 - for x, y in coords(4,4): - sa += x * y - return sa - - #img=array('h',(1,2,3,4)) - print f(3) -except Exception, e: - print "Exception: ", type(e) - print e - -## def f(): -## a=7 -## i=0 -## while i<4: -## if i<0: break -## if i<0: break -## i+=1 - -## f() +import time +import cppyy +lib = cppyy.load_lib("../../module/cppyy/test/example01Dict.so") +cls = cppyy._type_byname('example01') +inst = cls.construct(-17) + +t1 = time.time() +res = 0 +for i in range(10): + res += inst.invoke("addDataToInt", i) +t2 = time.time() +print t2 - t1 Modified: pypy/branch/reflex-support2/pypy/module/pypyjit/policy.py ============================================================================== --- pypy/branch/reflex-support2/pypy/module/pypyjit/policy.py (original) +++ pypy/branch/reflex-support2/pypy/module/pypyjit/policy.py Tue Nov 23 16:13:22 2010 @@ -12,7 +12,7 @@ if '.' in modname: modname, _ = modname.split('.', 1) if modname in ['pypyjit', 'signal', 'micronumpy', 'math', 'exceptions', - 'imp', 'sys', 'array', '_ffi']: + 'imp', 'sys', 'array', '_ffi', 'cppyy']: return True return False Modified: pypy/branch/reflex-support2/pypy/translator/c/genc.py ============================================================================== --- pypy/branch/reflex-support2/pypy/translator/c/genc.py (original) +++ pypy/branch/reflex-support2/pypy/translator/c/genc.py Tue Nov 23 16:13:22 2010 @@ -571,16 +571,15 @@ mk.rule(*rule) if self.config.translation.gcrootfinder == 'asmgcc': - trackgcfiles = [cfile[:-2] for cfile in mk.cfiles] - if self.translator.platform.name == 'msvc': + trackgcfiles = [cfile[:-2] for cfile in mk.cfiles + if cfile.endswith('.c')] + if 1: # XXX do that more cleanly trackgcfiles = [f for f in trackgcfiles if f.startswith(('implement', 'testing', '../module_cache/module'))] sfiles = ['%s.s' % (c,) for c in trackgcfiles] - lblsfiles = ['%s.lbl.s' % (c,) for c in trackgcfiles] gcmapfiles = ['%s.gcmap' % (c,) for c in trackgcfiles] mk.definition('ASMFILES', sfiles) - mk.definition('ASMLBLFILES', lblsfiles) mk.definition('GCMAPFILES', gcmapfiles) mk.definition('DEBUGFLAGS', '-O2 -fomit-frame-pointer -g') @@ -595,17 +594,32 @@ python = sys.executable + ' ' if self.translator.platform.name == 'msvc': - lblofiles = [] - for cfile in mk.cfiles: - f = cfile[:-2] - if f in trackgcfiles: - ofile = '%s.lbl.obj' % (f,) - else: - ofile = '%s.obj' % (f,) - - lblofiles.append(ofile) - mk.definition('ASMLBLOBJFILES', lblofiles) - mk.definition('OBJECTS', 'gcmaptable.obj $(ASMLBLOBJFILES)') + o_ext = '.obj' + lbl_ext = '.lbl.obj' + else: + o_ext = '.o' + lbl_ext = '.lbl.s' + + objectfiles = [] + nontrackgcexts = set() + for cfile in mk.cfiles: + f = cfile[:-2] + if f in trackgcfiles: + ofile = f + lbl_ext + else: + f, ext = os.path.splitext(cfile) + nontrackgcexts.add(ext) + ofile = f + o_ext + objectfiles.append(ofile) + if self.translator.platform.name == 'msvc': + objectfiles.append('gcmaptable.obj') + else: + objectfiles.append('gcmaptable.s') + mk.definition('OBJECTS', objectfiles) + nontrackgcexts = list(nontrackgcexts) + nontrackgcexts.sort() + + if self.translator.platform.name == 'msvc': # /Oi (enable intrinsics) and /Ob1 (some inlining) are mandatory # even in debug builds mk.definition('ASM_CFLAGS', '$(CFLAGS) $(CFLAGSEXTRA) /Oi /Ob1') @@ -618,9 +632,13 @@ ) mk.rule('gcmaptable.c', '$(GCMAPFILES)', 'cmd /c ' + python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py -fmsvc $(GCMAPFILES) > $@') + mk.rule('gcmaptable.c', '$(GCMAPFILES)', + 'cmd /c ' + python + '$(PYPYDIR)/translator/c/gcc/trackgcroot.py -fmsvc $(GCMAPFILES) > $@') + for ext in nontrackgcexts: + mk.rule(ext + '.obj', '', + '$(CC) /nologo $(CFLAGS) $(CFLAGSEXTRA) /Fo$@ /c $< $(INCLUDEDIRS)') else: - mk.definition('OBJECTS', '$(ASMLBLFILES) gcmaptable.s') mk.rule('%.s', '%.c', '$(CC) $(CFLAGS) $(CFLAGSEXTRA) -frandom-seed=$< -o $@ -S $< $(INCLUDEDIRS)') mk.rule('%.lbl.s %.gcmap', '%.s', [python + @@ -633,6 +651,9 @@ '$(GCMAPFILES) > $@.tmp', 'mv $@.tmp $@']) mk.rule('.PRECIOUS', '%.s', "# don't remove .s files if Ctrl-C'ed") + for ext in nontrackgcexts: + mk.rule('%.o', '%' + ext, + '$(CC) $(CFLAGS) $(CFLAGSEXTRA) -o $@ -c $< $(INCLUDEDIRS)') else: mk.definition('DEBUGFLAGS', '-O1 -g') Modified: pypy/branch/reflex-support2/pypy/translator/tool/cbuild.py ============================================================================== --- pypy/branch/reflex-support2/pypy/translator/tool/cbuild.py (original) +++ pypy/branch/reflex-support2/pypy/translator/tool/cbuild.py Tue Nov 23 16:13:22 2010 @@ -271,15 +271,10 @@ if not self.separate_module_files: return self if outputfilename is None: - # find more or less unique name there - basepath = py.path.local(self.separate_module_files[0]).dirpath() - pth = basepath.join('externmod').new(ext=host.so_ext) - num = 0 - while pth.check(): - pth = basepath.join( - 'externmod_%d' % (num,)).new(ext=host.so_ext) - num += 1 - outputfilename=pth.purebasename + global _counter_so_names + counter = _counter_so_names + _counter_so_names = counter + 1 + outputfilename = str(udir.join('externmod_%d' % counter)) lib = str(host.compile([], self, outputfilename=outputfilename, standalone=False)) d = self._copy_attributes() @@ -288,6 +283,8 @@ d['separate_module_sources'] = () return ExternalCompilationInfo(**d) +_counter_so_names = 0 + # ____________________________________________________________ # From cfbolz at codespeak.net Tue Nov 23 16:13:50 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 23 Nov 2010 16:13:50 +0100 (CET) Subject: [pypy-svn] r79407 - pypy/branch/reflex-support Message-ID: <20101123151350.7B06550819@codespeak.net> Author: cfbolz Date: Tue Nov 23 16:13:49 2010 New Revision: 79407 Removed: pypy/branch/reflex-support/ Log: kill old branch From cfbolz at codespeak.net Tue Nov 23 16:14:01 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 23 Nov 2010 16:14:01 +0100 (CET) Subject: [pypy-svn] r79408 - in pypy/branch: reflex-support reflex-support2 Message-ID: <20101123151401.DCCFA50821@codespeak.net> Author: cfbolz Date: Tue Nov 23 16:14:00 2010 New Revision: 79408 Added: pypy/branch/reflex-support/ (props changed) - copied from r79407, pypy/branch/reflex-support2/ Removed: pypy/branch/reflex-support2/ Log: rename new branch From afa at codespeak.net Tue Nov 23 16:24:51 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 23 Nov 2010 16:24:51 +0100 (CET) Subject: [pypy-svn] r79409 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101123152451.39A3C50811@codespeak.net> Author: afa Date: Tue Nov 23 16:24:49 2010 New Revision: 79409 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py pypy/branch/fast-forward/pypy/module/_io/interp_textio.py pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Log: Test and fix around TextIOWrapper.close() and __del__() Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bytesio.py Tue Nov 23 16:24:49 2010 @@ -1,5 +1,5 @@ from pypy.interpreter.typedef import ( - TypeDef, generic_new_descr) + TypeDef, generic_new_descr, GetSetProperty) from pypy.interpreter.gateway import interp2app, unwrap_spec from pypy.interpreter.baseobjspace import ObjSpace, W_Root from pypy.interpreter.error import OperationError, operationerrfmt @@ -20,11 +20,12 @@ W_BufferedIOBase.__init__(self, space) self.pos = 0 self.string_size = 0 - self.buf = [] + self.buf = None @unwrap_spec('self', ObjSpace, W_Root) def descr_init(self, space, w_initvalue=None): # In case __init__ is called multiple times + self.buf = [] self.string_size = 0 self.pos = 0 @@ -32,6 +33,12 @@ self.write_w(space, w_initvalue) self.pos = 0 + def _check_closed(self, space, message=None): + if self.buf is None: + if message is None: + message = "I/O operation on closed file" + raise OperationError(space.w_ValueError, space.wrap(message)) + @unwrap_spec('self', ObjSpace, W_Root) def read_w(self, space, w_size=None): self._check_closed(space) @@ -169,6 +176,13 @@ def seekable_w(self, space): return space.w_True + @unwrap_spec('self', ObjSpace) + def close_w(self, space): + self.buf = None + + def closed_get_w(space, self): + return space.wrap(self.buf is None) + W_BytesIO.typedef = TypeDef( 'BytesIO', W_BufferedIOBase.typedef, __new__ = generic_new_descr(W_BytesIO), @@ -185,5 +199,7 @@ readable = interp2app(W_BytesIO.readable_w), writable = interp2app(W_BytesIO.writable_w), seekable = interp2app(W_BytesIO.seekable_w), + close = interp2app(W_BytesIO.close_w), + closed = GetSetProperty(W_BytesIO.closed_get_w), ) Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Tue Nov 23 16:24:49 2010 @@ -361,6 +361,10 @@ self._check_init(space) return space.call_method(self.w_buffer, "fileno") + def closed_get_w(space, self): + self._check_init(space) + return space.getattr(self.w_buffer, space.wrap("closed")) + @unwrap_spec('self', ObjSpace) def flush_w(self, space): self._check_closed(space) @@ -371,8 +375,9 @@ @unwrap_spec('self', ObjSpace) def close_w(self, space): self._check_init(space) - if not self._closed(space): - return space.call_method(self, "flush") + if not space.is_true(space.getattr(self.w_buffer, + space.wrap("closed"))): + space.call_method(self, "flush") return space.call_method(self.w_buffer, "close") # _____________________________________________________________ @@ -813,4 +818,5 @@ writable = interp2app(W_TextIOWrapper.writable_w), seekable = interp2app(W_TextIOWrapper.seekable_w), fileno = interp2app(W_TextIOWrapper.fileno_w), + closed = GetSetProperty(W_TextIOWrapper.closed_get_w), ) Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Tue Nov 23 16:24:49 2010 @@ -106,6 +106,20 @@ assert f.read() == data * 2 assert buf.getvalue() == (data * 2).encode(encoding) + def test_destructor(self): + import _io + l = [] + class MyBytesIO(_io.BytesIO): + def close(self): + l.append(self.getvalue()) + _io.BytesIO.close(self) + b = MyBytesIO() + t = _io.TextIOWrapper(b, encoding="ascii") + t.write(u"abc") + del t + import gc; gc.collect() + assert l == ["abc"] + class AppTestIncrementalNewlineDecoder: def test_newline_decoder(self): From arigo at codespeak.net Tue Nov 23 16:56:49 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 16:56:49 +0100 (CET) Subject: [pypy-svn] r79410 - pypy/branch/jit-free/pypy/rpython/memory/gc Message-ID: <20101123155649.8EBE85080C@codespeak.net> Author: arigo Date: Tue Nov 23 16:56:48 2010 New Revision: 79410 Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py Log: Cover more cases in the doc. Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py Tue Nov 23 16:56:48 2010 @@ -26,7 +26,7 @@ to more than PYPY_GC_MAX_DELTA the amount really used after a collection. Defaults to 1/8th of the total RAM size (which is constrained to be at most - 4GB on 32-bit systems). Try values like '200MB'. + 2/3/4GB on 32-bit systems). Try values like '200MB'. PYPY_GC_MIN Don't collect while the memory size is below this limit. Useful to avoid spending all the time in From arigo at codespeak.net Tue Nov 23 17:00:11 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 17:00:11 +0100 (CET) Subject: [pypy-svn] r79411 - pypy/trunk/pypy/rpython/memory/gc Message-ID: <20101123160011.E4DAD50812@codespeak.net> Author: arigo Date: Tue Nov 23 17:00:10 2010 New Revision: 79411 Modified: pypy/trunk/pypy/rpython/memory/gc/env.py Log: Missing imports. Modified: pypy/trunk/pypy/rpython/memory/gc/env.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/env.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/env.py Tue Nov 23 17:00:10 2010 @@ -4,6 +4,8 @@ import os, sys from pypy.rlib.rarithmetic import r_uint from pypy.rlib.debug import debug_print, debug_start, debug_stop +from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem.lloperation import llop # ____________________________________________________________ # Reading env vars. Supports returning ints, uints or floats, From fijal at codespeak.net Tue Nov 23 17:00:37 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 23 Nov 2010 17:00:37 +0100 (CET) Subject: [pypy-svn] r79412 - pypy/extradoc/planning Message-ID: <20101123160037.8C9D550818@codespeak.net> Author: fijal Date: Tue Nov 23 17:00:36 2010 New Revision: 79412 Added: pypy/extradoc/planning/pypyonpypy.txt Log: My attempt to list issues running pypy on pypy (so they don't get forgotten) Added: pypy/extradoc/planning/pypyonpypy.txt ============================================================================== --- (empty file) +++ pypy/extradoc/planning/pypyonpypy.txt Tue Nov 23 17:00:36 2010 @@ -0,0 +1 @@ +* rpython/test/test_rclass.py fails for GC reasons From arigo at codespeak.net Tue Nov 23 17:00:55 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 17:00:55 +0100 (CET) Subject: [pypy-svn] r79413 - pypy/branch/jit-free/pypy/rpython/memory/gc Message-ID: <20101123160055.901E650823@codespeak.net> Author: arigo Date: Tue Nov 23 17:00:54 2010 New Revision: 79413 Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/env.py Log: Merge r79411 from trunk. Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/env.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/env.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/env.py Tue Nov 23 17:00:54 2010 @@ -4,6 +4,8 @@ import os, sys from pypy.rlib.rarithmetic import r_uint from pypy.rlib.debug import debug_print, debug_start, debug_stop +from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem.lloperation import llop # ____________________________________________________________ # Reading env vars. Supports returning ints, uints or floats, From fijal at codespeak.net Tue Nov 23 17:01:21 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 23 Nov 2010 17:01:21 +0100 (CET) Subject: [pypy-svn] r79414 - pypy/extradoc/planning Message-ID: <20101123160121.A6F7A50824@codespeak.net> Author: fijal Date: Tue Nov 23 17:01:20 2010 New Revision: 79414 Modified: pypy/extradoc/planning/pypyonpypy.txt Log: another issue Modified: pypy/extradoc/planning/pypyonpypy.txt ============================================================================== --- pypy/extradoc/planning/pypyonpypy.txt (original) +++ pypy/extradoc/planning/pypyonpypy.txt Tue Nov 23 17:01:20 2010 @@ -1 +1,4 @@ * rpython/test/test_rclass.py fails for GC reasons + +* ctypes has bugs (exposed by test_ctypes, but also backend/x86 tests + segfaulting every now and then) From fijal at codespeak.net Tue Nov 23 17:06:36 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 23 Nov 2010 17:06:36 +0100 (CET) Subject: [pypy-svn] r79415 - in pypy/branch: ouf-of-line-guards out-of-line-guards Message-ID: <20101123160636.4938450811@codespeak.net> Author: fijal Date: Tue Nov 23 17:06:34 2010 New Revision: 79415 Added: pypy/branch/out-of-line-guards/ (props changed) - copied from r79414, pypy/branch/ouf-of-line-guards/ Removed: pypy/branch/ouf-of-line-guards/ Log: typo in branch name From fijal at codespeak.net Tue Nov 23 17:09:27 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 23 Nov 2010 17:09:27 +0100 (CET) Subject: [pypy-svn] r79416 - in pypy/branch/out-of-line-guards/pypy/rpython: . lltypesystem ootypesystem test Message-ID: <20101123160927.310E650825@codespeak.net> Author: fijal Date: Tue Nov 23 17:09:25 2010 New Revision: 79416 Modified: pypy/branch/out-of-line-guards/pypy/rpython/lltypesystem/rclass.py pypy/branch/out-of-line-guards/pypy/rpython/ootypesystem/rclass.py pypy/branch/out-of-line-guards/pypy/rpython/rclass.py pypy/branch/out-of-line-guards/pypy/rpython/test/test_rclass.py Log: Implement _jit_invariant_fields_ on rclass Modified: pypy/branch/out-of-line-guards/pypy/rpython/lltypesystem/rclass.py ============================================================================== --- pypy/branch/out-of-line-guards/pypy/rpython/lltypesystem/rclass.py (original) +++ pypy/branch/out-of-line-guards/pypy/rpython/lltypesystem/rclass.py Tue Nov 23 17:09:25 2010 @@ -366,7 +366,7 @@ adtmeths = {} if hints is None: hints = {} - hints = self._check_for_immutable_hints(hints) + hints = self._check_for_hints(hints) kwds = {} if self.gcflavor == 'gc': kwds['rtti'] = True Modified: pypy/branch/out-of-line-guards/pypy/rpython/ootypesystem/rclass.py ============================================================================== --- pypy/branch/out-of-line-guards/pypy/rpython/ootypesystem/rclass.py (original) +++ pypy/branch/out-of-line-guards/pypy/rpython/ootypesystem/rclass.py Tue Nov 23 17:09:25 2010 @@ -183,7 +183,7 @@ hints = classdef.classdesc.pyobj._rpython_hints else: hints = {} - hints = self._check_for_immutable_hints(hints) + hints = self._check_for_hints(hints) self.lowleveltype = ootype.Instance(classdef.name, b, {}, {}, _hints = hints) self.iprebuiltinstances = identity_dict() self.object_type = self.lowleveltype Modified: pypy/branch/out-of-line-guards/pypy/rpython/rclass.py ============================================================================== --- pypy/branch/out-of-line-guards/pypy/rpython/rclass.py (original) +++ pypy/branch/out-of-line-guards/pypy/rpython/rclass.py Tue Nov 23 17:09:25 2010 @@ -157,19 +157,27 @@ def _setup_repr(self): pass - def _check_for_immutable_hints(self, hints): + def _check_for_hints(self, hints): if self.classdef.classdesc.lookup('_immutable_') is not None: hints = hints.copy() hints['immutable'] = True - self.immutable_field_list = [] # unless overwritten below + self.immutable_fields = [] # unless overwritten below + self.jit_invariant_fields = [] if self.classdef.classdesc.lookup('_immutable_fields_') is not None: hints = hints.copy() immutable_fields = self.classdef.classdesc.classdict.get( '_immutable_fields_') if immutable_fields is not None: - self.immutable_field_list = immutable_fields.value + self.immutable_fields = immutable_fields.value accessor = FieldListAccessor() hints['immutable_fields'] = accessor + if self.classdef.classdesc.lookup('_jit_invariant_fields_') is not None: + hints = hints.copy() + invariant_fields = self.classdef.classdesc.classdict.get( + '_jit_invariant_fields_') + self.jit_invariant_fields = invariant_fields.value + accessor = FieldListAccessor() + hints['jit_invariant_fields'] = accessor return hints def __repr__(self): @@ -187,20 +195,21 @@ return 'InstanceR %s' % (clsname,) def _setup_repr_final(self): - self._setup_immutable_field_list() + self._setup_field_list('immutable_fields') self._check_for_immutable_conflicts() + self._setup_field_list('jit_invariant_fields') - def _setup_immutable_field_list(self): + def _setup_field_list(self, varname): hints = self.object_type._hints - if "immutable_fields" in hints: - accessor = hints["immutable_fields"] + if varname in hints: + accessor = hints[varname] if not hasattr(accessor, 'fields'): - immutable_fields = [] + fields = [] rbase = self while rbase.classdef is not None: - immutable_fields += rbase.immutable_field_list + fields += getattr(rbase, varname) rbase = rbase.rbase - self._parse_field_list(immutable_fields, accessor) + self._parse_field_list(fields, accessor) def _parse_field_list(self, fields, accessor): with_suffix = {} @@ -233,7 +242,7 @@ continue if r.lowleveltype == Void: continue - base._setup_immutable_field_list() + base._setup_field_list('immutable_fields') if base.object_type._immutable_field(mangled): continue # 'fieldname' is a mutable, non-Void field in the parent @@ -242,7 +251,7 @@ "class %r has _immutable_=True, but parent class %r " "defines (at least) the mutable field %r" % ( self, base, fieldname)) - if fieldname in self.immutable_field_list: + if fieldname in self.immutable_fields: raise ImmutableConflictError( "field %r is defined mutable in class %r, but " "listed in _immutable_fields_ in subclass %r" % ( Modified: pypy/branch/out-of-line-guards/pypy/rpython/test/test_rclass.py ============================================================================== --- pypy/branch/out-of-line-guards/pypy/rpython/test/test_rclass.py (original) +++ pypy/branch/out-of-line-guards/pypy/rpython/test/test_rclass.py Tue Nov 23 17:09:25 2010 @@ -6,6 +6,7 @@ from pypy.rlib.rarithmetic import intmask, r_longlong from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin from pypy.objspace.flow.model import summary +from pypy.jit.metainterp.typesystem import deref class EmptyBase(object): pass @@ -733,7 +734,6 @@ assert summary(graph) == {} def test_immutable_fields(self): - from pypy.jit.metainterp.typesystem import deref class A(object): _immutable_fields_ = ["x", "y[*]"] @@ -749,8 +749,24 @@ assert accessor.fields == {"inst_x" : "", "inst_y" : "[*]"} or \ accessor.fields == {"ox" : "", "oy" : "[*]"} # for ootype + def test_jit_invariant_fields(self): + class A(object): + + _jit_invariant_fields_ = ['x'] + + def __init__(self, x): + self.x = x + + def f(): + return A(3) + + t, typer, graph = self.gengraph(f, []) + A_TYPE = deref(graph.getreturnvar().concretetype) + accessor = A_TYPE._hints["jit_invariant_fields"] + assert accessor.fields == {'inst_x': ''} or \ + accessor.fields == {'ox': ''} + def test_immutable_fields_subclass_1(self): - from pypy.jit.metainterp.typesystem import deref class A(object): _immutable_fields_ = ["x"] def __init__(self, x): @@ -769,7 +785,6 @@ accessor.fields == {"ox" : ""} # for ootype def test_immutable_fields_subclass_2(self): - from pypy.jit.metainterp.typesystem import deref class A(object): _immutable_fields_ = ["x"] def __init__(self, x): @@ -789,7 +804,6 @@ accessor.fields == {"ox" : "", "oy" : ""} # for ootype def test_immutable_fields_only_in_subclass(self): - from pypy.jit.metainterp.typesystem import deref class A(object): def __init__(self, x): self.x = x @@ -832,7 +846,6 @@ py.test.raises(ImmutableConflictError, self.gengraph, f, []) def test_immutable_ok_inheritance_2(self): - from pypy.jit.metainterp.typesystem import deref class A(object): _immutable_fields_ = ['v'] class B(A): @@ -853,7 +866,6 @@ accessor.fields == {"ov" : ""} # for ootype def test_immutable_subclass_1(self): - from pypy.jit.metainterp.typesystem import deref class A(object): _immutable_ = True class B(A): @@ -866,7 +878,6 @@ assert B_TYPE._hints["immutable"] # inherited from A def test_immutable_subclass_2(self): - from pypy.jit.metainterp.typesystem import deref class A(object): pass class B(A): @@ -879,7 +890,6 @@ assert B_TYPE._hints["immutable"] def test_immutable_subclass_void(self): - from pypy.jit.metainterp.typesystem import deref class A(object): pass class B(A): @@ -917,6 +927,7 @@ assert destrptr is not None def test_del_inheritance(self): + import gc class State: pass s = State() @@ -937,6 +948,8 @@ A() B() C() + gc.collect() + gc.collect() return s.a_dels * 10 + s.b_dels res = f() assert res == 42 From arigo at codespeak.net Tue Nov 23 17:14:53 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 17:14:53 +0100 (CET) Subject: [pypy-svn] r79417 - pypy/trunk/pypy/rpython/test Message-ID: <20101123161453.2EFD150827@codespeak.net> Author: arigo Date: Tue Nov 23 17:14:51 2010 New Revision: 79417 Modified: pypy/trunk/pypy/rpython/test/test_rclass.py Log: Add missing gc.collect(). Modified: pypy/trunk/pypy/rpython/test/test_rclass.py ============================================================================== --- pypy/trunk/pypy/rpython/test/test_rclass.py (original) +++ pypy/trunk/pypy/rpython/test/test_rclass.py Tue Nov 23 17:14:51 2010 @@ -917,6 +917,7 @@ assert destrptr is not None def test_del_inheritance(self): + from pypy.rlib import rgc class State: pass s = State() @@ -937,6 +938,7 @@ A() B() C() + rgc.collect() return s.a_dels * 10 + s.b_dels res = f() assert res == 42 @@ -1067,6 +1069,7 @@ assert meth.finalizer def test_del_inheritance(self): + from pypy.rlib import rgc class State: pass s = State() @@ -1087,6 +1090,7 @@ A() B() C() + rgc.collect() return s.a_dels * 10 + s.b_dels res = f() assert res == 42 From afa at codespeak.net Tue Nov 23 17:33:06 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 23 Nov 2010 17:33:06 +0100 (CET) Subject: [pypy-svn] r79418 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101123163306.A13335080C@codespeak.net> Author: afa Date: Tue Nov 23 17:33:04 2010 New Revision: 79418 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Log: Tests and fixes Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Tue Nov 23 17:33:04 2010 @@ -384,15 +384,18 @@ self._check_closed(space, "read of closed file") size = convert_size(space, w_size) - if size < 0: + if size == -1: # read until the end of stream with self.lock: res = self._read_all(space) - else: + elif size >= 0: res = self._read_fast(size) if res is None: with self.lock: res = self._read_generic(space, size) + else: + raise OperationError(space.w_ValueError, space.wrap( + "read length must be positive or -1")) return space.wrap(res) @unwrap_spec('self', ObjSpace, int) Modified: pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_iobase.py Tue Nov 23 17:33:04 2010 @@ -284,6 +284,8 @@ w_buffer = space.call_function(space.w_bytearray, w_size) w_length = space.call_method(self, "readinto", w_buffer) + if space.is_w(w_length, space.w_None): + return w_length space.delslice(w_buffer, w_length, space.len(w_buffer)) return space.str(w_buffer) Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Tue Nov 23 17:33:04 2010 @@ -14,6 +14,7 @@ raw = _io.FileIO(self.tmpfile) f = _io.BufferedReader(raw) assert f.read() == "a\nb\nc" + raises(ValueError, f.read, -2) f.close() # raw = _io.FileIO(self.tmpfile) Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Tue Nov 23 17:33:04 2010 @@ -113,6 +113,29 @@ return len(data) assert MockRawIO().read() == 'abcde' + def test_rawio_read_pieces(self): + import _io + class MockRawIO(_io._RawIOBase): + stack = ['abc', 'de', None, 'fg', ''] + def readinto(self, buf): + data = self.stack.pop(0) + if data is None: + return None + if len(data) <= len(buf): + buf[:len(data)] = data + return len(data) + else: + buf[:] = data[:len(buf)] + self.stack.insert(0, data[len(buf):]) + return len(buf) + r = MockRawIO() + assert r.read(2) == 'ab' + assert r.read(2) == 'c' + assert r.read(2) == 'de' + assert r.read(2) == None + assert r.read(2) == 'fg' + assert r.read(2) == '' + class AppTestOpen: def setup_class(cls): tmpfile = udir.join('tmpfile').ensure() From cfbolz at codespeak.net Tue Nov 23 17:33:57 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 23 Nov 2010 17:33:57 +0100 (CET) Subject: [pypy-svn] r79419 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101123163357.ED6C850811@codespeak.net> Author: cfbolz Date: Tue Nov 23 17:33:56 2010 New Revision: 79419 Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Log: Disable the call hack, as it breaks things now that NotImplementedError cannot be caught. Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Tue Nov 23 17:33:56 2010 @@ -71,7 +71,7 @@ self.executor = executor.get_executor(self.space, result_type) self.arg_converters = None # - self.hack_call = arg_types == ['int'] and result_type == 'int' + self.hack_call = False #arg_types == ['int'] and result_type == 'int' # def call(self, cppthis, args_w): From cfbolz at codespeak.net Tue Nov 23 17:43:39 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 23 Nov 2010 17:43:39 +0100 (CET) Subject: [pypy-svn] r79420 - pypy/branch/reflex-support/pypy/jit/tl Message-ID: <20101123164339.4930050828@codespeak.net> Author: cfbolz Date: Tue Nov 23 17:43:37 2010 New Revision: 79420 Modified: pypy/branch/reflex-support/pypy/jit/tl/pypyjit.py Log: enable cppyy Modified: pypy/branch/reflex-support/pypy/jit/tl/pypyjit.py ============================================================================== --- pypy/branch/reflex-support/pypy/jit/tl/pypyjit.py (original) +++ pypy/branch/reflex-support/pypy/jit/tl/pypyjit.py Tue Nov 23 17:43:37 2010 @@ -42,6 +42,7 @@ config.objspace.usemodules._sre = False # config.objspace.usemodules._ffi = True +config.objspace.usemodules.cppyy = True # set_pypy_opt_level(config, level='jit') From afa at codespeak.net Tue Nov 23 17:58:23 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 23 Nov 2010 17:58:23 +0100 (CET) Subject: [pypy-svn] r79421 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101123165823.805CA50833@codespeak.net> Author: afa Date: Tue Nov 23 17:58:22 2010 New Revision: 79421 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Log: Test and fix in reading newlines Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Tue Nov 23 17:58:22 2010 @@ -488,7 +488,7 @@ while True: # Fast path for non-control chars. The loop always ends # since the Py_UNICODE storage is NUL-terminated. - while line[start + i] > '\r': + while i < size and line[start + i] > '\r': i += 1 if i >= size: return -1, size @@ -507,11 +507,9 @@ if pos >= 0: return pos - start + len(self.readnl), 0 else: - restart = end - len(self.readnl) - if restart >= 0: - pos = line.find(self.readnl[0], restart, end) - if pos >= 0: - return -1, pos - start + pos = line.find(self.readnl[0], start, end) + if pos >= 0: + return -1, pos - start return -1, size @unwrap_spec('self', ObjSpace, W_Root) Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Tue Nov 23 17:58:22 2010 @@ -120,6 +120,44 @@ import gc; gc.collect() assert l == ["abc"] + def test_newlines(self): + import _io + input_lines = [ "unix\n", "windows\r\n", "os9\r", "last\n", "nonl" ] + + tests = [ + [ None, [ 'unix\n', 'windows\n', 'os9\n', 'last\n', 'nonl' ] ], + [ '', input_lines ], + [ '\n', [ "unix\n", "windows\r\n", "os9\rlast\n", "nonl" ] ], + [ '\r\n', [ "unix\nwindows\r\n", "os9\rlast\nnonl" ] ], + [ '\r', [ "unix\nwindows\r", "\nos9\r", "last\nnonl" ] ], + ] + + # Try a range of buffer sizes to test the case where \r is the last + # character in TextIOWrapper._pending_line. + encoding = "ascii" + # XXX: str.encode() should return bytes + data = bytes(''.join(input_lines).encode(encoding)) + for do_reads in (False, True): + for bufsize in range(1, 10): + for newline, exp_lines in tests: + bufio = _io.BufferedReader(_io.BytesIO(data), bufsize) + textio = _io.TextIOWrapper(bufio, newline=newline, + encoding=encoding) + if do_reads: + got_lines = [] + while True: + c2 = textio.read(2) + if c2 == '': + break + len(c2) == 2 + got_lines.append(c2 + textio.readline()) + else: + got_lines = list(textio) + + for got_line, exp_line in zip(got_lines, exp_lines): + assert got_line == exp_line + assert len(got_lines) == len(exp_lines) + class AppTestIncrementalNewlineDecoder: def test_newline_decoder(self): From cfbolz at codespeak.net Tue Nov 23 17:58:56 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 23 Nov 2010 17:58:56 +0100 (CET) Subject: [pypy-svn] r79422 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101123165856.C179550835@codespeak.net> Author: cfbolz Date: Tue Nov 23 17:58:55 2010 New Revision: 79422 Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Log: disable this differently Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Tue Nov 23 17:58:55 2010 @@ -71,18 +71,18 @@ self.executor = executor.get_executor(self.space, result_type) self.arg_converters = None # - self.hack_call = False #arg_types == ['int'] and result_type == 'int' + #self.hack_call = arg_types == ['int'] and result_type == 'int' # def call(self, cppthis, args_w): if self.executor is None: raise OperationError(self.space.w_TypeError, self.space.wrap("return type not handled")) - if self.hack_call: - try: - return self.do_hack_call(cppthis, args_w) - except NotImplementedError: - pass + #if self.hack_call: + # try: + # return self.do_hack_call(cppthis, args_w) + # except NotImplementedError: + # pass args = self.prepare_arguments(args_w) try: From afa at codespeak.net Tue Nov 23 18:25:41 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 23 Nov 2010 18:25:41 +0100 (CET) Subject: [pypy-svn] r79423 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101123172541.7A46F282BE3@codespeak.net> Author: afa Date: Tue Nov 23 18:25:40 2010 New Revision: 79423 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Log: FileIO.write() should accept any buffer object Modified: pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_fileio.py Tue Nov 23 18:25:40 2010 @@ -340,7 +340,7 @@ def write_w(self, space, w_data): self._check_closed(space) self._check_writable(space) - data = space.str_w(w_data) + data = space.bufferstr_w(w_data) try: n = os.write(self.fd, data) Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_io.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_io.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_io.py Tue Nov 23 18:25:40 2010 @@ -153,3 +153,10 @@ f = io.open(self.tmpfile, "w+b") f.close() + def test_array_write(self): + import _io, array + a = array.array(b'i', range(10)) + n = len(a.tostring()) + with _io.open(self.tmpfile, "wb", 0) as f: + assert f.write(a) == n + From hakanardo at codespeak.net Tue Nov 23 18:25:47 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Tue, 23 Nov 2010 18:25:47 +0100 (CET) Subject: [pypy-svn] r79424 - in pypy/branch/jit-unroll-loops/pypy/jit/metainterp: . optimizeopt Message-ID: <20101123172547.1875E5080C@codespeak.net> Author: hakanardo Date: Tue Nov 23 18:25:44 2010 New Revision: 79424 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/logger.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Log: added some logging Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py Tue Nov 23 18:25:44 2010 @@ -108,6 +108,11 @@ globaldata.loopnumbering += 1 metainterp_sd.logger_ops.log_loop(loop.inputargs, loop.operations, n, type) + short = loop.token.short_preamble + if short: + metainterp_sd.logger_ops.log_short_preamble(short.inputargs, + short.operations) + if not we_are_translated(): show_loop(metainterp_sd, loop) loop.check_consistency() Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/logger.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/logger.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/logger.py Tue Nov 23 18:25:44 2010 @@ -38,6 +38,11 @@ self._log_operations(inputargs, operations) debug_stop("jit-log-opt-bridge") + def log_short_preamble(self, inputargs, operations): + debug_start("jit-log-short-preamble") + self._log_operations(inputargs, operations) + debug_stop("jit-log-short-preamble") + def repr_of_descr(self, descr): return descr.repr_of_descr() Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Tue Nov 23 18:25:44 2010 @@ -3,6 +3,7 @@ from pypy.jit.metainterp.compile import ResumeGuardDescr from pypy.jit.metainterp.resume import Snapshot from pypy.jit.metainterp.history import TreeLoop, LoopToken +from pypy.rlib.debug import debug_start, debug_stop, debug_print # FXIME: Introduce some VirtualOptimizer super class instead @@ -201,28 +202,26 @@ state = ExeState() short_preamble = [] loop_i = preamble_i = 0 - while loop_i < len(loop)-1 and preamble_i < len(preamble)-1: - if self.sameop(preamble[preamble_i], loop[loop_i]): + while preamble_i < len(preamble)-1: + if self.sameop(preamble[preamble_i], loop[loop_i]) \ + and loop_i < len(loop)-1: loop_i += 1 else: if not state.safe_to_move(preamble[preamble_i]): - #print "Unsafe heap operation." + debug_print("create_short_preamble failed due to", + "unsafe op:", preamble[preamble_i].getopnum(), + "at position: ", preamble_i) return None short_preamble.append(preamble[preamble_i]) state.update(preamble[preamble_i]) preamble_i += 1 - if loop_i < len(loop)-1: - #print "Loop contains ops not in preamble???" + if loop_i < len(loop)-1: + debug_print("create_short_preamble failed due to", + "loop contaning ops not in preamble" + "at position", loop_i) return None - while preamble_i < len(preamble)-1: - if not state.safe_to_move(preamble[preamble_i]): - #print "Unsafe heap operation." - return None - short_preamble.append(preamble[preamble_i]) - state.update(preamble[preamble_i]) - preamble_i += 1 jumpargs = [None] * len(inputargs) allboxes = preambleargs[:] @@ -240,7 +239,8 @@ for a in jumpargs: if a is None: - #print "Unable to find all input arguments???" + debug_print("create_short_preamble failed due to", + "input arguments not located") return None jmp = ResOperation(rop.JUMP, jumpargs[:], None) @@ -257,7 +257,8 @@ for op in short_preamble: for box in op.getarglist(): if box not in seen: - #print "Op arguments not produced???" + debug_print("create_short_preamble failed due to", + "op arguments not produced") return None if op.result: seen[op.result] = True From arigo at codespeak.net Tue Nov 23 18:34:29 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 18:34:29 +0100 (CET) Subject: [pypy-svn] r79425 - pypy/trunk/pypy/tool Message-ID: <20101123173429.14BB7282BD4@codespeak.net> Author: arigo Date: Tue Nov 23 18:34:28 2010 New Revision: 79425 Modified: pypy/trunk/pypy/tool/logparser.py Log: Adapt a bit. Modified: pypy/trunk/pypy/tool/logparser.py ============================================================================== --- pypy/trunk/pypy/tool/logparser.py (original) +++ pypy/trunk/pypy/tool/logparser.py Tue Nov 23 18:34:28 2010 @@ -12,14 +12,6 @@ from pypy.tool import progressbar def parse_log_file(filename, verbose=True): - r_start = re.compile(r"\[([0-9a-fA-F]+)\] \{([\w-]+)$") - r_stop = re.compile(r"\[([0-9a-fA-F]+)\] ([\w-]+)\}$") - lasttime = 0 - log = DebugLog() - time_decrase = False - performance_log = True - nested = 0 - # f = open(filename, 'r') if f.read(2) == 'BZ': f.close() @@ -30,19 +22,32 @@ lines = f.readlines() f.close() # - if sys.stdout.isatty(): + return parse_log(lines, verbose=verbose) + +def parse_log(lines, verbose=False): + r_start = re.compile(r"\[([0-9a-fA-F]+)\] \{([\w-]+)$") + r_stop = re.compile(r"\[([0-9a-fA-F]+)\] ([\w-]+)\}$") + lasttime = 0 + log = DebugLog() + time_decrase = False + performance_log = True + nested = 0 + # + if verbose and sys.stdout.isatty(): progress = progressbar.ProgressBar(color='green') + counter = 0 + else: + progress = None single_percent = len(lines) / 100 if verbose: - vnext = single_percent + vnext = 0 else: - vnext = len(lines) - counter = 0 + vnext = -1 for i, line in enumerate(lines): if i == vnext: - counter += 1 - if sys.stdout.isatty(): + if progress is not None: progress.render(counter) + counter += 1 vnext += single_percent else: sys.stderr.write('%d%%..' % int(100.0*i/len(lines))) From arigo at codespeak.net Tue Nov 23 18:45:23 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 18:45:23 +0100 (CET) Subject: [pypy-svn] r79426 - in pypy/trunk/pypy: rlib tool Message-ID: <20101123174523.B85A25081A@codespeak.net> Author: arigo Date: Tue Nov 23 18:45:22 2010 New Revision: 79426 Modified: pypy/trunk/pypy/rlib/debug.py pypy/trunk/pypy/tool/logparser.py Log: More whacking in order to make log_parse() be able to parse logs captured during testing too. Modified: pypy/trunk/pypy/rlib/debug.py ============================================================================== --- pypy/trunk/pypy/rlib/debug.py (original) +++ pypy/trunk/pypy/rlib/debug.py Tue Nov 23 18:45:22 2010 @@ -87,13 +87,15 @@ _stop_colors = "" def debug_start(category): - print >> sys.stderr, '%s[%s] {%s%s' % (_start_colors_1, time.clock(), + c = int(time.clock() * 100) + print >> sys.stderr, '%s[%x] {%s%s' % (_start_colors_1, c, category, _stop_colors) if _log is not None: _log.debug_start(category) def debug_stop(category): - print >> sys.stderr, '%s[%s] %s}%s' % (_start_colors_2, time.clock(), + c = int(time.clock() * 100) + print >> sys.stderr, '%s[%x] %s}%s' % (_start_colors_2, c, category, _stop_colors) if _log is not None: _log.debug_stop(category) Modified: pypy/trunk/pypy/tool/logparser.py ============================================================================== --- pypy/trunk/pypy/tool/logparser.py (original) +++ pypy/trunk/pypy/tool/logparser.py Tue Nov 23 18:45:22 2010 @@ -25,8 +25,9 @@ return parse_log(lines, verbose=verbose) def parse_log(lines, verbose=False): - r_start = re.compile(r"\[([0-9a-fA-F]+)\] \{([\w-]+)$") - r_stop = re.compile(r"\[([0-9a-fA-F]+)\] ([\w-]+)\}$") + color = "(?:\x1b.*?m)" + r_start = re.compile(color + r"\[([0-9a-fA-F]+)\] \{([\w-]+)" + color + "$") + r_stop = re.compile(color + r"\[([0-9a-fA-F]+)\] ([\w-]+)\}" + color + "$") lasttime = 0 log = DebugLog() time_decrase = False From arigo at codespeak.net Tue Nov 23 19:03:19 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 19:03:19 +0100 (CET) Subject: [pypy-svn] r79427 - in pypy/trunk/pypy: config jit/metainterp jit/metainterp/test jit/tool/test rlib translator Message-ID: <20101123180319.BC61C282BD6@codespeak.net> Author: arigo Date: Tue Nov 23 19:03:17 2010 New Revision: 79427 Modified: pypy/trunk/pypy/config/translationoption.py pypy/trunk/pypy/jit/metainterp/jitprof.py pypy/trunk/pypy/jit/metainterp/pyjitpl.py pypy/trunk/pypy/jit/metainterp/test/test_basic.py pypy/trunk/pypy/jit/metainterp/test/test_compile.py pypy/trunk/pypy/jit/metainterp/test/test_warmspot.py pypy/trunk/pypy/jit/metainterp/warmspot.py pypy/trunk/pypy/jit/metainterp/warmstate.py pypy/trunk/pypy/jit/tool/test/test_jitoutput.py pypy/trunk/pypy/rlib/jit.py pypy/trunk/pypy/translator/driver.py Log: Remove the old jit-debug option completely, as well as all places using it -- there was actually only the Profiler subclasses. Now instead, write the summary at the end in the normal (logged) way in the category "jit-summary". To enable, you can set the env var PYPYLOG=jit-summary:- Modified: pypy/trunk/pypy/config/translationoption.py ============================================================================== --- pypy/trunk/pypy/config/translationoption.py (original) +++ pypy/trunk/pypy/config/translationoption.py Tue Nov 23 19:03:17 2010 @@ -114,10 +114,6 @@ ChoiceOption("jit_backend", "choose the backend for the JIT", ["auto", "x86", "x86-without-sse2", "llvm"], default="auto", cmdline="--jit-backend"), - ChoiceOption("jit_debug", "the amount of debugging dumps made by the JIT", - ["off", "profile", "steps", "detailed"], - default="off", - cmdline="--jit-debug"), ChoiceOption("jit_profiler", "integrate profiler support into the JIT", ["off", "oprofile"], default="off"), Modified: pypy/trunk/pypy/jit/metainterp/jitprof.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/jitprof.py (original) +++ pypy/trunk/pypy/jit/metainterp/jitprof.py Tue Nov 23 19:03:17 2010 @@ -2,9 +2,9 @@ """ A small helper module for profiling JIT """ -import os import time -from pypy.rlib.debug import debug_print +from pypy.rlib.debug import debug_print, debug_start, debug_stop +from pypy.rlib.debug import have_debug_prints from pypy.jit.metainterp.jitexc import JitException counters=""" @@ -48,9 +48,6 @@ def finish(self): pass - def set_printing(self, printing): - pass - def start_tracing(self): pass @@ -90,7 +87,6 @@ counters = None calls = 0 current = None - printing = True def start(self): self.starttime = self.timer() @@ -102,11 +98,7 @@ def finish(self): self.tk = self.timer() - if self.printing: - self.print_stats() - - def set_printing(self, printing): - self.printing = printing + self.print_stats() def _start(self, event): t0 = self.t1 @@ -154,6 +146,12 @@ self.calls += 1 def print_stats(self): + debug_start("jit-summary") + if have_debug_prints(): + self._print_stats() + debug_stop("jit-summary") + + def _print_stats(self): cnt = self.counters tim = self.times calls = self.calls @@ -161,8 +159,8 @@ self._print_line_time("Backend", cnt[BACKEND], tim[BACKEND]) self._print_intline("Running asm", cnt[RUNNING]) self._print_intline("Blackhole", cnt[BLACKHOLE]) - line = "TOTAL: \t\t%f\n" % (self.tk - self.starttime, ) - os.write(2, line) + line = "TOTAL: \t\t%f" % (self.tk - self.starttime, ) + debug_print(line) self._print_intline("ops", cnt[OPS]) self._print_intline("recorded ops", cnt[RECORDED_OPS]) self._print_intline(" calls", calls) @@ -178,15 +176,14 @@ self._print_intline("nvreused", cnt[NVREUSED]) def _print_line_time(self, string, i, tim): - final = "%s:%s\t%d\t%f\n" % (string, " " * max(0, 13-len(string)), i, tim) - os.write(2, final) + final = "%s:%s\t%d\t%f" % (string, " " * max(0, 13-len(string)), i, tim) + debug_print(final) def _print_intline(self, string, i): final = string + ':' + " " * max(0, 16-len(string)) - final += '\t' + str(i) + '\n' - os.write(2, final) - - + final += '\t' + str(i) + debug_print(final) + class BrokenProfilerData(JitException): pass Modified: pypy/trunk/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/trunk/pypy/jit/metainterp/pyjitpl.py Tue Nov 23 19:03:17 2010 @@ -18,7 +18,6 @@ from pypy.jit.metainterp.jitexc import JitException, get_llexception from pypy.rlib.rarithmetic import intmask from pypy.rlib.objectmodel import specialize -from pypy.rlib.jit import DEBUG_OFF, DEBUG_PROFILE, DEBUG_STEPS, DEBUG_DETAILED from pypy.jit.codewriter.jitcode import JitCode, SwitchDictDescr from pypy.jit.codewriter import heaptracker Modified: pypy/trunk/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_basic.py Tue Nov 23 19:03:17 2010 @@ -35,7 +35,6 @@ optimize_bridge = staticmethod(simple_optimize.optimize_bridge) trace_limit = sys.maxint - debug_level = 2 func._jit_unroll_safe_ = True rtyper = support.annotate(func, values, type_system=type_system) Modified: pypy/trunk/pypy/jit/metainterp/test/test_compile.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_compile.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_compile.py Tue Nov 23 19:03:17 2010 @@ -42,7 +42,6 @@ class FakeState: optimize_loop = staticmethod(optimize.optimize_loop) - debug_level = 0 class FakeGlobalData: loopnumbering = 0 Modified: pypy/trunk/pypy/jit/metainterp/test/test_warmspot.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_warmspot.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_warmspot.py Tue Nov 23 19:03:17 2010 @@ -132,91 +132,6 @@ assert state.optimize_loop is optimize.optimize_loop assert state.optimize_bridge is optimize.optimize_bridge - def test_static_debug_level(self, capfd): - py.test.skip("debug_level is being deprecated") - from pypy.rlib.jit import DEBUG_PROFILE, DEBUG_OFF, DEBUG_STEPS - from pypy.jit.metainterp.jitprof import EmptyProfiler, Profiler - - myjitdriver = JitDriver(greens = [], reds = ['n']) - def f(n): - while n > 0: - myjitdriver.can_enter_jit(n=n) - myjitdriver.jit_merge_point(n=n) - n -= 1 - return n - - capfd.readouterr() - self.meta_interp(f, [10], debug_level=DEBUG_OFF, - ProfilerClass=Profiler) - out, err = capfd.readouterr() - assert not 'ENTER' in err - assert not 'LEAVE' in err - assert not "Running asm" in err - self.meta_interp(f, [10], debug_level=DEBUG_PROFILE, - ProfilerClass=Profiler) - out, err = capfd.readouterr() - assert not 'ENTER' in err - assert not 'LEAVE' in err - assert not 'compiled new' in err - assert "Running asm" in err - - self.meta_interp(f, [10], debug_level=DEBUG_STEPS, - ProfilerClass=Profiler) - out, err = capfd.readouterr() - assert 'ENTER' in err - assert 'LEAVE' in err - assert "Running asm" in err - - self.meta_interp(f, [10], debug_level=DEBUG_STEPS, - ProfilerClass=EmptyProfiler) - out, err = capfd.readouterr() - assert 'ENTER' in err - assert 'LEAVE' in err - assert not "Running asm" in err - - def test_set_param_debug(self): - py.test.skip("debug_level is being deprecated") - from pypy.rlib.jit import DEBUG_PROFILE, DEBUG_OFF, DEBUG_STEPS - from pypy.jit.metainterp.jitprof import EmptyProfiler, Profiler - - myjitdriver = JitDriver(greens = [], reds = ['n']) - def f(n): - while n > 0: - myjitdriver.can_enter_jit(n=n) - myjitdriver.jit_merge_point(n=n) - n -= 1 - return n - - def main(n, debug): - myjitdriver.set_param("debug", debug) - print f(n) - - outerr = py.io.StdCaptureFD() - self.meta_interp(main, [10, DEBUG_OFF], debug_level=DEBUG_STEPS, - ProfilerClass=Profiler) - out, errf = outerr.done() - err = errf.read() - assert not 'ENTER' in err - assert not 'LEAVE' in err - assert not "Running asm" in err - outerr = py.io.StdCaptureFD() - self.meta_interp(main, [10, DEBUG_PROFILE], debug_level=DEBUG_STEPS, - ProfilerClass=Profiler) - out, errf = outerr.done() - err = errf.read() - assert not 'ENTER' in err - assert not 'LEAVE' in err - assert not 'compiled new' in err - assert "Running asm" in err - outerr = py.io.StdCaptureFD() - self.meta_interp(main, [10, DEBUG_STEPS], debug_level=DEBUG_OFF, - ProfilerClass=Profiler) - out, errf = outerr.done() - err = errf.read() - assert 'ENTER' in err - assert 'LEAVE' in err - assert "Running asm" in err - def test_unwanted_loops(self): mydriver = JitDriver(reds = ['n', 'total', 'm'], greens = []) Modified: pypy/trunk/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/warmspot.py (original) +++ pypy/trunk/pypy/jit/metainterp/warmspot.py Tue Nov 23 19:03:17 2010 @@ -24,21 +24,17 @@ from pypy.jit.metainterp.jitdriver import JitDriverStaticData from pypy.jit.codewriter import support, codewriter from pypy.jit.codewriter.policy import JitPolicy -from pypy.rlib.jit import DEBUG_STEPS, DEBUG_DETAILED, DEBUG_OFF, DEBUG_PROFILE # ____________________________________________________________ # Bootstrapping -def apply_jit(translator, backend_name="auto", debug_level=DEBUG_STEPS, - inline=False, - **kwds): +def apply_jit(translator, backend_name="auto", inline=False, **kwds): if 'CPUClass' not in kwds: from pypy.jit.backend.detect_cpu import getcpuclass kwds['CPUClass'] = getcpuclass(backend_name) - if debug_level > DEBUG_OFF: - ProfilerClass = Profiler - else: - ProfilerClass = EmptyProfiler + ProfilerClass = Profiler + # Always use Profiler here, which should have a very low impact. + # Otherwise you can try with ProfilerClass = EmptyProfiler. warmrunnerdesc = WarmRunnerDesc(translator, translate_support_code=True, listops=True, @@ -47,7 +43,6 @@ **kwds) for jd in warmrunnerdesc.jitdrivers_sd: jd.warmstate.set_param_inlining(inline) - jd.warmstate.set_param_debug(debug_level) warmrunnerdesc.finish() translator.warmrunnerdesc = warmrunnerdesc # for later debugging @@ -66,7 +61,7 @@ def jittify_and_run(interp, graph, args, repeat=1, backendopt=False, trace_limit=sys.maxint, - debug_level=DEBUG_STEPS, inline=False, **kwds): + inline=False, **kwds): from pypy.config.config import ConfigError translator = interp.typer.annotator.translator try: @@ -83,7 +78,6 @@ jd.warmstate.set_param_trace_eagerness(2) # for tests jd.warmstate.set_param_trace_limit(trace_limit) jd.warmstate.set_param_inlining(inline) - jd.warmstate.set_param_debug(debug_level) warmrunnerdesc.finish() res = interp.eval_graph(graph, args) if not kwds.get('translate_support_code', False): Modified: pypy/trunk/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/warmstate.py (original) +++ pypy/trunk/pypy/jit/metainterp/warmstate.py Tue Nov 23 19:03:17 2010 @@ -9,7 +9,6 @@ from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.jit import (PARAMETERS, OPTIMIZER_SIMPLE, OPTIMIZER_FULL, OPTIMIZER_NO_PERFECTSPEC) -from pypy.rlib.jit import DEBUG_PROFILE from pypy.rlib.jit import BaseJitCell from pypy.rlib.debug import debug_start, debug_stop, debug_print from pypy.jit.metainterp import history @@ -209,11 +208,6 @@ else: raise ValueError("unknown optimizer") - def set_param_debug(self, value): - self.debug_level = value - if self.profiler is not None: - self.profiler.set_printing(value >= DEBUG_PROFILE) - def disable_noninlinable_function(self, greenkey): cell = self.jit_cell_at_key(greenkey) cell.dont_trace_here = True Modified: pypy/trunk/pypy/jit/tool/test/test_jitoutput.py ============================================================================== --- pypy/trunk/pypy/jit/tool/test/test_jitoutput.py (original) +++ pypy/trunk/pypy/jit/tool/test/test_jitoutput.py Tue Nov 23 19:03:17 2010 @@ -1,10 +1,11 @@ import py from pypy.jit.metainterp.warmspot import ll_meta_interp -from pypy.rlib.jit import JitDriver, DEBUG_PROFILE +from pypy.rlib.jit import JitDriver from pypy.jit.backend.llgraph import runner from pypy.jit.metainterp.jitprof import Profiler, JITPROF_LINES from pypy.jit.tool.jitoutput import parse_prof +from pypy.tool.logparser import parse_log, extract_category def test_really_run(): """ This test checks whether output of jitprof did not change. @@ -21,13 +22,15 @@ cap = py.io.StdCaptureFD() try: ll_meta_interp(f, [10], CPUClass=runner.LLtypeCPU, type_system='lltype', - ProfilerClass=Profiler, debug_level=DEBUG_PROFILE) + ProfilerClass=Profiler) finally: out, err = cap.reset() - err = "\n".join(err.splitlines()[-JITPROF_LINES:]) - print err - assert err.count("\n") == JITPROF_LINES - 1 - info = parse_prof(err) + + log = parse_log(err.splitlines(True)) + err_sections = list(extract_category(log, 'jit-summary')) + [err1] = err_sections # there should be exactly one jit-summary + assert err1.count("\n") == JITPROF_LINES + info = parse_prof(err1) # assert did not crash # asserts below are a bit delicate, possibly they might be deleted assert info.tracing_no == 1 Modified: pypy/trunk/pypy/rlib/jit.py ============================================================================== --- pypy/trunk/pypy/rlib/jit.py (original) +++ pypy/trunk/pypy/rlib/jit.py Tue Nov 23 19:03:17 2010 @@ -260,17 +260,11 @@ OPTIMIZER_NO_PERFECTSPEC = 1 OPTIMIZER_FULL = 2 -DEBUG_OFF = 0 -DEBUG_PROFILE = 1 -DEBUG_STEPS = 2 -DEBUG_DETAILED = 3 - PARAMETERS = {'threshold': 1000, 'trace_eagerness': 200, 'trace_limit': 10000, 'inlining': False, 'optimizer': OPTIMIZER_FULL, - 'debug' : DEBUG_STEPS, } unroll_parameters = unrolling_iterable(PARAMETERS.keys()) Modified: pypy/trunk/pypy/translator/driver.py ============================================================================== --- pypy/trunk/pypy/translator/driver.py (original) +++ pypy/trunk/pypy/translator/driver.py Tue Nov 23 19:03:17 2010 @@ -11,7 +11,6 @@ from pypy.annotation import policy as annpolicy import optparse from pypy.tool.udir import udir -from pypy.rlib.jit import DEBUG_OFF, DEBUG_DETAILED, DEBUG_PROFILE, DEBUG_STEPS from pypy.rlib.entrypoint import secondary_entrypoints import py @@ -37,13 +36,6 @@ 'c': 'lltype', } -JIT_DEBUG = { - 'off' : DEBUG_OFF, - 'profile' : DEBUG_PROFILE, - 'steps' : DEBUG_STEPS, - 'detailed' : DEBUG_DETAILED, -} - def backend_to_typesystem(backend): return _BACKEND_TO_TYPESYSTEM.get(backend, 'ootype') @@ -399,7 +391,6 @@ # from pypy.jit.metainterp.warmspot import apply_jit apply_jit(self.translator, policy=self.jitpolicy, - debug_level=JIT_DEBUG[self.config.translation.jit_debug], backend_name=self.config.translation.jit_backend, inline=True) # self.log.info("the JIT compiler was generated") @@ -417,7 +408,6 @@ # from pypy.jit.metainterp.warmspot import apply_jit apply_jit(self.translator, policy=self.jitpolicy, - debug_level=JIT_DEBUG[self.config.translation.jit_debug], backend_name='cli', inline=True) #XXX # self.log.info("the JIT compiler was generated") From antocuni at codespeak.net Tue Nov 23 19:03:54 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 23 Nov 2010 19:03:54 +0100 (CET) Subject: [pypy-svn] r79428 - in pypy/branch/jit-free/pypy: tool translator Message-ID: <20101123180354.8FC575080C@codespeak.net> Author: antocuni Date: Tue Nov 23 19:03:53 2010 New Revision: 79428 Added: pypy/branch/jit-free/pypy/tool/debug_print.py (contents, props changed) Modified: pypy/branch/jit-free/pypy/translator/driver.py Log: - add a tool.debug_print module so we can freely use __pypy__.debug_* also on cpython (they are ignored there) - log the start and end of each translation task Added: pypy/branch/jit-free/pypy/tool/debug_print.py ============================================================================== --- (empty file) +++ pypy/branch/jit-free/pypy/tool/debug_print.py Tue Nov 23 19:03:53 2010 @@ -0,0 +1,20 @@ +""" +If we are on pypy, use the real debug_* functions defined in __pypy__. + +Else, use empty functions +""" + +try: + from __pypy__ import debug_start, debug_stop, debug_print, debug_print_once +except ImportError: + def debug_start(*args): + pass + + def debug_stop(*args): + pass + + def debug_print(*args): + pass + + def debug_print_once(*args): + pass Modified: pypy/branch/jit-free/pypy/translator/driver.py ============================================================================== --- pypy/branch/jit-free/pypy/translator/driver.py (original) +++ pypy/branch/jit-free/pypy/translator/driver.py Tue Nov 23 19:03:53 2010 @@ -11,6 +11,7 @@ from pypy.annotation import policy as annpolicy import optparse from pypy.tool.udir import udir +from pypy.tool.debug_print import debug_start, debug_print, debug_stop from pypy.rlib.jit import DEBUG_OFF, DEBUG_DETAILED, DEBUG_PROFILE, DEBUG_STEPS from pypy.rlib.entrypoint import secondary_entrypoints @@ -19,11 +20,6 @@ log = py.log.Producer("translation") py.log.setconsumer("translation", ansi_log) -try: - from __pypy__ import debug_print_once -except ImportError: - def debug_print_once(*args): - pass def taskdef(taskfunc, deps, title, new_state=None, expected_states=[], idemp=False, earlycheck=None): @@ -288,7 +284,8 @@ return else: self.log.info("%s..." % title) - debug_print_once('translation-task', 'starting', goal) + debug_start('translation-task') + debug_print('starting', goal) self.timer.start_event(goal) try: instrument = False @@ -306,6 +303,7 @@ assert False, 'we should not get here' finally: try: + debug_stop('translation-task') self.timer.end_event(goal) except (KeyboardInterrupt, SystemExit): raise From antocuni at codespeak.net Tue Nov 23 19:08:16 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 23 Nov 2010 19:08:16 +0100 (CET) Subject: [pypy-svn] r79429 - pypy/trunk/pypy/jit/tool Message-ID: <20101123180816.4E14350823@codespeak.net> Author: antocuni Date: Tue Nov 23 19:08:14 2010 New Revision: 79429 Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric pypy/trunk/pypy/jit/tool/log2gnumeric.py Log: (in-progress): add also a memusage sheet to show the VmRSS; so far that particular sheet is broken Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric ============================================================================== Binary files. No diff available. Modified: pypy/trunk/pypy/jit/tool/log2gnumeric.py ============================================================================== --- pypy/trunk/pypy/jit/tool/log2gnumeric.py (original) +++ pypy/trunk/pypy/jit/tool/log2gnumeric.py Tue Nov 23 19:08:14 2010 @@ -15,15 +15,18 @@ import re, sys import gzip + def main(): logname = sys.argv[1] outname = logname + '.gnumeric' data = open(logname).read() data = data.replace('\n', '') - xml = gzip.open('log-template.gnumeric').read() + maxclock = find_max_clock(data) # + xml = gzip.open('log-template.gnumeric').read() xml = replace_sheet(xml, 'translation-task', tasks_rows(data)) xml = replace_sheet(xml, 'gc-collect', gc_collect_rows(data)) + xml = replace_sheet(xml, 'memusage', memusage_rows(logname + '.memusage', maxclock)) # out = gzip.open(outname, 'wb') out.write(xml) @@ -31,25 +34,41 @@ def replace_sheet(xml, sheet_name, data): - pattern = '(%s.*?).*?' + pattern = '%s.*?(.*?)' regex = re.compile(pattern % sheet_name, re.DOTALL) cells = gen_cells(data) - xml2 = regex.sub(r'\1%s' % cells, xml) - assert xml != xml2 + match = regex.search(xml) + if not match: + print 'Cannot find sheet %s' % sheet_name + return xml + a, b = match.span(1) + xml2 = xml[:a] + cells + xml[b:] return xml2 def gen_cells(data): + # values for the ValueType attribute + ValueType_Empty = 'ValueType="10"' + ValueType_Number = 'ValueType="40"' + ValueType_String = 'ValueType="60"' + # parts = [] - parts.append(' ') + parts.append('') for i, row in enumerate(data): for j, val in enumerate(row): - cell = ' %s' - parts.append(cell % (i, j, val)) + if val is None: + attr = ValueType_Empty + val = '' + elif isinstance(val, (int, long, float)): + attr = ValueType_Number + else: + attr = ValueType_String + cell = ' %s' + parts.append(cell % (i, j, attr, val)) parts.append(' ') - return ''.join(parts) + return '\n'.join(parts) -def gc_collect_rows(data, show_header=True): +def gc_collect_rows(data): s = r""" ----------- Full collection ------------------ \| used before collection: @@ -63,6 +82,7 @@ \[([0-9a-f]+)\] gc-collect\}""" # r = re.compile(s.replace('\n', '')) + yield 'clock', 'gc-before', 'gc-after' for a,b,c,d,e,f in r.findall(data): yield int(f, 16), int(a)+int(b), int(c)+int(d) @@ -73,9 +93,26 @@ \[([0-9a-f]+)\] translation-task\}""" # r = re.compile(s.replace('\n', '')) + yield 'clock', None, 'task' for a,b in r.findall(data): yield int(b, 16), 1, a +def memusage_rows(filename, maxclock): + try: + lines = open(filename) + except IOError: + print 'Warning: cannot find file %s, skipping the memusage sheet' + lines = [] + yield 'n', 'computed clock', 'VmRSS' + for i, line in enumerate(lines): + mem = int(line) + yield i, "=max('gc-collect'!$A$1:$A$65536)*(A2/max($A$1:$A$65536)))", mem + +def find_max_clock(data): + s = r"\[([0-9a-f]+)\] " + r = re.compile(s) + clocks = [int(x, 16) for x in r.findall(data)] + return max(clocks) if __name__ == '__main__': main() From arigo at codespeak.net Tue Nov 23 19:10:30 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 19:10:30 +0100 (CET) Subject: [pypy-svn] r79430 - pypy/branch/jit-free/pypy/module/__pypy__/test Message-ID: <20101123181030.2E1C150822@codespeak.net> Author: arigo Date: Tue Nov 23 19:10:29 2010 New Revision: 79430 Modified: pypy/branch/jit-free/pypy/module/__pypy__/test/test_debug.py Log: Fix test. Modified: pypy/branch/jit-free/pypy/module/__pypy__/test/test_debug.py ============================================================================== --- pypy/branch/jit-free/pypy/module/__pypy__/test/test_debug.py (original) +++ pypy/branch/jit-free/pypy/module/__pypy__/test/test_debug.py Tue Nov 23 19:10:29 2010 @@ -1,9 +1,11 @@ import py -from pypy.conftest import gettestobjspace +from pypy.conftest import gettestobjspace, option from pypy.rlib import debug class AppTestDebug: def setup_class(cls): + if option.runappdirect: + py.test.skip("not meant to be run with -A") cls.space = gettestobjspace(usemodules=['__pypy__']) space = cls.space cls.w_check_log = cls.space.wrap(cls.check_log) From arigo at codespeak.net Tue Nov 23 19:50:23 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 19:50:23 +0100 (CET) Subject: [pypy-svn] r79431 - pypy/branch/jit-free/pypy/jit/metainterp/test Message-ID: <20101123185023.E790A282BE3@codespeak.net> Author: arigo Date: Tue Nov 23 19:50:19 2010 New Revision: 79431 Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Log: A bit obscure hacks to speed up this test, which takes ages if run together with all other tests -- because it does a lot of calls to gc.collect(). Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py Tue Nov 23 19:50:19 2010 @@ -1,3 +1,9 @@ +import sys, py +if len(sys.argv) >= 4 and sys.argv[1] == '--sub': + sys.path[:] = eval(sys.argv[2]) # hacks for test_integration + from pypy.conftest import option + option.__dict__.update(eval(sys.argv[3])) + from pypy.jit.metainterp.memmgr import MemoryManager from pypy.jit.metainterp.test.test_basic import LLJitMixin from pypy.rlib.jit import JitDriver, dont_look_inside @@ -7,7 +13,12 @@ generation = 0 -class TestMemoryManager: +class _TestMemoryManager: + # We spawn a fresh process below to lower the time it takes to do + # all these gc.collect() in memmgr.py. This issue is particularly + # important when running all the tests, because test_[a-l]*.py have + # left tons of stuff in memory. To get temporarily the normal + # behavior just rename this class to TestMemoryManager. def test_disabled(self): memmgr = MemoryManager() @@ -58,7 +69,9 @@ assert tokens[i] in memmgr.alive_loops -class TestIntegration(LLJitMixin): +class _TestIntegration(LLJitMixin): + # See comments in TestMemoryManager. To get temporarily the normal + # behavior just rename this class to TestIntegration. def test_loop_kept_alive(self): myjitdriver = JitDriver(greens=[], reds=['n']) @@ -178,3 +191,28 @@ res = self.meta_interp(f, [1], loop_longevity=4, inline=True) assert res == 42 self.check_tree_loop_count(8) + +# ____________________________________________________________ + +def test_all(): + if sys.platform == 'win32': + py.test.skip( + "passing repr() to subprocess.Popen probably doesn't work") + import os, subprocess + from pypy.conftest import option + thisfile = os.path.abspath(__file__) + p = subprocess.Popen([sys.executable, thisfile, + '--sub', repr(sys.path), repr(option.__dict__)]) + result = p.wait() + assert result == 0 + +if __name__ == '__main__': + # occurs in the subprocess + for test in [_TestMemoryManager(), _TestIntegration()]: + for name in dir(test): + if name.startswith('test_'): + print + print '-'*79 + print '----- Now running test', name, '-----' + print + getattr(test, name)() From afa at codespeak.net Tue Nov 23 19:57:32 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 23 Nov 2010 19:57:32 +0100 (CET) Subject: [pypy-svn] r79432 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101123185732.24540282BD4@codespeak.net> Author: afa Date: Tue Nov 23 19:57:30 2010 New Revision: 79432 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Log: Implement the most complex tell() function in the world. Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Tue Nov 23 19:57:30 2010 @@ -230,7 +230,30 @@ ) class PositionCookie: - pass + def __init__(self, bigint): + self.start_pos = bigint.ulonglongmask() + bigint = bigint.rshift(r_ulonglong.BITS) + self.dec_flags = 0 + self.bytes_to_feed = 0 + self.chars_to_skip = 0 + self.need_eof = 0 + + def pack(self): + # The meaning of a tell() cookie is: seek to position, set the + # decoder flags to dec_flags, read bytes_to_feed bytes, feed them + # into the decoder with need_eof as the EOF flag, then skip + # chars_to_skip characters of the decoded result. For most simple + # decoders, tell() will often just give a byte offset in the file. + return (self.start_pos | + (self.dec_flags<<64) | + (self.bytes_to_feed<<128) | + (self.chars_to_skip<<192) | + bool(self.need_eof)<<256) + +class PositionSnapshot: + def __init__(self, flags, input): + self.flags = flags + self.input = input class W_TextIOWrapper(W_TextIOBase): def __init__(self, space): @@ -251,6 +274,7 @@ self.encodefunc = None # Specialized encoding func (see below) self.encoding_start_of_stream = False # Whether or not it's the start # of the stream + self.snapshot = None @unwrap_spec('self', ObjSpace, W_Root, W_Root, W_Root, W_Root, int) def descr_init(self, space, w_buffer, w_encoding=None, @@ -418,8 +442,18 @@ if not self.w_decoder: raise OperationError(space.w_IOError, space.wrap("not readable")) - # XXX - # if self.telling... + if self.telling: + # To prepare for tell(), we need to snapshot a point in the file + # where the decoder's input buffer is empty. + w_state = space.call_method(self.w_decoder, "getstate") + # Given this, we know there was a valid snapshot point + # len(dec_buffer) bytes ago with decoder state (b'', dec_flags). + w_dec_buffer, w_dec_flags = space.unpackiterable(w_state, 2) + dec_buffer = space.str_w(w_dec_buffer) + dec_flags = space.int_w(w_dec_flags) + else: + dec_buffer = None + dec_flags = 0 # Read a chunk, decode it, and put the result in self._decoded_chars w_input = space.call_method(self.w_buffer, "read1", @@ -431,8 +465,11 @@ if space.int_w(space.len(w_decoded)) > 0: eof = False - # XXX - # if self.telling... + if self.telling: + # At the snapshot point, len(dec_buffer) bytes before the read, + # the next input to be decoded is dec_buffer + input_chunk. + next_input = dec_buffer + space.str_w(w_input) + self.snapshot = PositionSnapshot(dec_flags, next_input) return not eof @@ -675,26 +712,6 @@ # _____________________________________________________________ # seek/tell - def _pack_cookie(self, start_pos, dec_flags=0, - bytes_to_feed=0, need_eof=0, chars_to_skip=0): - # The meaning of a tell() cookie is: seek to position, set the - # decoder flags to dec_flags, read bytes_to_feed bytes, feed them - # into the decoder with need_eof as the EOF flag, then skip - # chars_to_skip characters of the decoded result. For most simple - # decoders, tell() will often just give a byte offset in the file. - return (start_pos | (dec_flags<<64) | (bytes_to_feed<<128) | - (chars_to_skip<<192) | bool(need_eof)<<256) - - def _unpack_cookie(self, bigint): - cookie = PositionCookie() - cookie.start_pos = bigint.ulonglongmask() - bigint = bigint.rshift(r_ulonglong.BITS) - cookie.dec_flags = 0 - cookie.bytes_to_feed = 0 - cookie.chars_to_skip = 0 - cookie.need_eof = 0 - return cookie - def _decoder_setstate(self, space, cookie): # When seeking to the start of the stream, we call decoder.reset() # rather than decoder.getstate(). @@ -760,7 +777,7 @@ # The strategy of seek() is to go back to the safe start point and # replay the effect of read(chars_to_skip) from there. - cookie = self._unpack_cookie(space.bigint_w(w_pos)) + cookie = PositionCookie(space.bigint_w(w_pos)) # Seek back to the safe start point space.call_method(self.w_buffer, "seek", space.wrap(cookie.start_pos)) @@ -775,8 +792,9 @@ if cookie.chars_to_skip: # Just like _read_chunk, feed the decoder and save a snapshot. w_chunk = space.call_method(self.w_buffer, "read", - space.wrap(cookie.chars_to_feed)) - # XXX self.snapshot = cookie.dec_flags, w_chunk + space.wrap(cookie.bytes_to_feed)) + self.snapshot = PositionSnapshot(cookie.dec_flags, + space.str_w(w_chunk)) w_decoded = space.call_method(self.w_decoder, "decode", w_chunk, space.wrap(cookie.need_eof)) @@ -788,8 +806,7 @@ "can't restore logical file position")) self.decoded_chars_used = cookie.chars_to_skip else: - # XXX self.snapshot = cookie.dec_flags, space.wrap(u"") - pass + self.snapshot = PositionSnapshot(cookie.dec_flags, "") # Finally, reset the encoder (merely useful for proper BOM handling) if self.w_encoder: @@ -797,6 +814,95 @@ return w_pos + @unwrap_spec('self', ObjSpace) + def tell_w(self, space): + self._check_closed(space) + + if not self.seekable: + raise OperationError(space.w_IOError, space.wrap( + "underlying stream is not seekable")) + + if not self.telling: + raise OperationError(space.w_IOError, space.wrap( + "telling position disabled by next() call")) + + self._writeflush(space) + space.call_method(self, "flush") + + w_pos = space.call_method(self.w_buffer, "tell") + + if self.w_decoder is None or self.snapshot is None: + assert not self.decoded_chars + return w_pos + + cookie = PositionCookie(space.bigint_w(w_pos)) + + # Skip backward to the snapshot point (see _read_chunk) + cookie.dec_flags = self.snapshot.flags + input = self.snapshot.input + cookie.start_pos -= len(input) + + # How many decoded characters have been used up since the snapshot? + if not self.decoded_chars_used: + # We haven't moved from the snapshot point. + return space.wrap(cookie.pack()) + + chars_to_skip = self.decoded_chars_used + + # Starting from the snapshot position, we will walk the decoder + # forward until it gives us enough decoded characters. + w_saved_state = space.call_method(self.w_decoder, "getstate") + + try: + # Note our initial start point + self._decoder_setstate(space, cookie) + + # Feed the decoder one byte at a time. As we go, note the nearest + # "safe start point" before the current location (a point where + # the decoder has nothing buffered, so seek() can safely start + # from there and advance to this location). + + chars_decoded = 0 + i = 0 + while i < len(input): + w_decoded = space.call_method(self.w_decoder, "decode", + space.wrap(input[i])) + chars_decoded += len(space.unicode_w(w_decoded)) + + cookie.bytes_to_feed += 1 + + w_state = space.call_method(self.w_decoder, "getstate") + w_dec_buffer, w_flags = space.unpackiterable(w_state, 2) + dec_buffer_len = len(space.str_w(w_dec_buffer)) + + if dec_buffer_len == 0 and chars_decoded <= chars_to_skip: + # Decoder buffer is empty, so this is a safe start point. + cookie.start_pos += cookie.bytes_to_feed + chars_to_skip -= chars_decoded + assert chars_to_skip >= 0 + cookie.dec_flags = space.int_w(w_flags) + cookie.bytes_to_feed = 0 + chars_decoded = 0 + if chars_decoded >= chars_to_skip: + break + i += 1 + else: + # We didn't get enough decoded data; signal EOF to get more. + w_decoded = space.call_method(self.w_decoder, "decode", + space.wrap(""), + space.wrap(1)) # final=1 + chars_decoded += len(space.unicode_w(w_decoded)) + cookie.need_eof = 1 + + if chars_decoded < chars_to_skip: + raise OperationError(space.w_IOError, space.wrap( + "can't reconstruct logical file position")) + finally: + space.call_method(self.w_decoder, "setstate", w_saved_state) + + # The returned cookie corresponds to the last safe start point. + cookie.chars_to_skip = chars_to_skip + return space.wrap(cookie.pack()) W_TextIOWrapper.typedef = TypeDef( 'TextIOWrapper', W_TextIOBase.typedef, @@ -807,6 +913,7 @@ readline = interp2app(W_TextIOWrapper.readline_w), write = interp2app(W_TextIOWrapper.write_w), seek = interp2app(W_TextIOWrapper.seek_w), + tell = interp2app(W_TextIOWrapper.tell_w), detach = interp2app(W_TextIOWrapper.detach_w), flush = interp2app(W_TextIOWrapper.flush_w), close = interp2app(W_TextIOWrapper.close_w), Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_textio.py Tue Nov 23 19:57:30 2010 @@ -106,6 +106,14 @@ assert f.read() == data * 2 assert buf.getvalue() == (data * 2).encode(encoding) + def test_tell(self): + import _io + r = _io.BytesIO("abc\ndef\n") + t = _io.TextIOWrapper(r) + assert t.tell() == 0 + t.read(4) + assert t.tell() == 4 + def test_destructor(self): import _io l = [] From arigo at codespeak.net Tue Nov 23 20:07:26 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 23 Nov 2010 20:07:26 +0100 (CET) Subject: [pypy-svn] r79433 - in pypy/branch/jit-free/pypy: jit/metainterp/test rlib Message-ID: <20101123190726.C90A4282BD4@codespeak.net> Author: arigo Date: Tue Nov 23 20:07:24 2010 New Revision: 79433 Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_logger.py pypy/branch/jit-free/pypy/rlib/debug.py Log: Remove one global that can be patched by tests, and fix the only test that patches it. Modified: pypy/branch/jit-free/pypy/jit/metainterp/test/test_logger.py ============================================================================== --- pypy/branch/jit-free/pypy/jit/metainterp/test/test_logger.py (original) +++ pypy/branch/jit-free/pypy/jit/metainterp/test/test_logger.py Tue Nov 23 20:07:24 2010 @@ -14,11 +14,20 @@ def capturing(func, *args, **kwds): log_stream = StringIO() - debug._stderr = log_stream + class MyDebugLog: + def debug_print(self, *args): + for arg in args: + print >> log_stream, arg, + print >> log_stream + def debug_start(self, *args): + pass + def debug_stop(self, *args): + pass try: + debug._log = MyDebugLog() func(*args, **kwds) finally: - debug._stderr = sys.stderr + debug._log = None return log_stream.getvalue() class Logger(logger.Logger): Modified: pypy/branch/jit-free/pypy/rlib/debug.py ============================================================================== --- pypy/branch/jit-free/pypy/rlib/debug.py (original) +++ pypy/branch/jit-free/pypy/rlib/debug.py Tue Nov 23 20:07:24 2010 @@ -53,13 +53,11 @@ _log = None # patched from tests to be an object of class DebugLog # or compatible -_stderr = sys.stderr # alternatively, this is patched from tests - # (redirects debug_print(), but not debug_start/stop) def debug_print(*args): for arg in args: - print >> _stderr, arg, - print >> _stderr + print >> sys.stderr, arg, + print >> sys.stderr if _log is not None: _log.debug_print(*args) From wlav at codespeak.net Tue Nov 23 20:19:34 2010 From: wlav at codespeak.net (wlav at codespeak.net) Date: Tue, 23 Nov 2010 20:19:34 +0100 (CET) Subject: [pypy-svn] r79434 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101123191934.9A0E3282BD6@codespeak.net> Author: wlav Date: Tue Nov 23 20:19:33 2010 New Revision: 79434 Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py Log: make immutable markers consistent Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/converter.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/converter.py Tue Nov 23 20:19:33 2010 @@ -44,6 +44,7 @@ class ArrayTypeConverter(TypeConverter): + _immutable_ = True def __init__(self, space, array_size): if array_size <= 0: self.size = sys.maxint @@ -208,6 +209,7 @@ # byteptr[0] = space.unwrap(space.id(w_value.getslotvalue(2))) class ShortArrayConverter(ShortPtrConverter): + _immutable_ = True def to_memory(self, space, w_obj, w_value, offset): # copy the full array (uses byte copy for now) fieldptr = self._get_fieldptr(space, w_obj, offset) @@ -236,6 +238,7 @@ # byteptr[0] = space.unwrap(space.id(w_value.getslotvalue(2))) class LongArrayConverter(LongPtrConverter): + _immutable_ = True def to_memory(self, space, w_obj, w_value, offset): # copy the full array (uses byte copy for now) fieldptr = self._get_fieldptr(space, w_obj, offset) From david at codespeak.net Tue Nov 23 20:57:27 2010 From: david at codespeak.net (david at codespeak.net) Date: Tue, 23 Nov 2010 20:57:27 +0100 (CET) Subject: [pypy-svn] r79435 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . test Message-ID: <20101123195727.8091F282BE3@codespeak.net> Author: david Date: Tue Nov 23 20:57:25 2010 New Revision: 79435 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py Log: Fix a register allocation issue in emit_guard_int_mul_ovf Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Tue Nov 23 20:57:25 2010 @@ -89,7 +89,7 @@ def emit_op_int_mul(self, op, regalloc, fcond): a0 = op.getarg(0) a1 = op.getarg(1) - reg1 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + reg1 = regalloc.make_sure_var_in_reg(a0, [a1], imm_fine=False) reg2 = regalloc.make_sure_var_in_reg(a1, [a0], imm_fine=False) res = regalloc.force_allocate_reg(op.result, [a0, a1]) self.mc.MUL(res.value, reg1.value, reg2.value) @@ -98,11 +98,10 @@ return fcond #ref: http://blogs.arm.com/software-enablement/detecting-overflow-from-mul/ - f = False def emit_guard_int_mul_ovf(self, op, guard, regalloc, fcond): a0 = op.getarg(0) a1 = op.getarg(1) - reg1 = regalloc.make_sure_var_in_reg(a0, imm_fine=False) + reg1 = regalloc.make_sure_var_in_reg(a0, [a1], imm_fine=False) reg2 = regalloc.make_sure_var_in_reg(a1, [a0], imm_fine=False) res = regalloc.force_allocate_reg(op.result, [a0, a1]) self.mc.SMULL(res.value, r.ip.value, reg1.value, reg2.value, cond=fcond) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py Tue Nov 23 20:57:25 2010 @@ -28,7 +28,6 @@ cpu = self.cpu inp = [BoxInt(i) for i in range(1, 15)] out = [BoxInt(i) for i in range(1, 15)] - #out.reverse() looptoken = LoopToken() operations = [ ResOperation(rop.INT_ADD, [inp[0] , inp[1]], out[0]), From hakanardo at codespeak.net Tue Nov 23 21:15:42 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Tue, 23 Nov 2010 21:15:42 +0100 (CET) Subject: [pypy-svn] r79436 - in pypy/branch/jit-unroll-loops/pypy/jit: backend/llgraph metainterp metainterp/optimizeopt Message-ID: <20101123201542.4C123282BEA@codespeak.net> Author: hakanardo Date: Tue Nov 23 21:15:26 2010 New Revision: 79436 Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/runner.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resoperation.py Log: Allow guards in the short inlined preamble by connecting them to jump to the original preamble on failure already when they are first emitted. What would it take to add this feature in the other backends aswell? Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py Tue Nov 23 21:15:26 2010 @@ -365,6 +365,13 @@ else: log.info("compiling new bridge") +def compile_add_guard_jump_target(loop, loop_target): + loop = _from_opaque(loop) + loop_target = _from_opaque(loop_target) + op = loop.operations[-1] + assert op.is_guard() + op.jump_target = loop_target + def compile_add_fail(loop, fail_index): loop = _from_opaque(loop) index = len(loop.operations)-1 Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/runner.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/runner.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/runner.py Tue Nov 23 21:15:26 2010 @@ -191,6 +191,12 @@ llimpl.compile_add_fail_arg(c, var2index[box]) else: llimpl.compile_add_fail_arg(c, -1) + token = op.getjumptarget() + if token: + assert isinstance(token, history.LoopToken) + compiled_version = token._llgraph_compiled_version + llimpl.compile_add_guard_jump_target(c, compiled_version) + x = op.result if x is not None: if isinstance(x, history.BoxInt): Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Tue Nov 23 21:15:26 2010 @@ -419,6 +419,8 @@ def store_final_boxes_in_guard(self, op): ###pendingfields = self.heap_op_optimizer.force_lazy_setfields_for_guard() + if op.getjumptarget(): + return op descr = op.getdescr() assert isinstance(descr, compile.ResumeGuardDescr) modifier = resume.ResumeDataVirtualAdder(descr, self.resumedata_memo) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Tue Nov 23 21:15:26 2010 @@ -55,6 +55,7 @@ short = self.create_short_preamble(loop.preamble.operations, loop.preamble.inputargs, + loop.preamble.token, loop.operations, loop.inputargs, loop.token) @@ -72,6 +73,8 @@ assert isinstance(loop.preamble.token, LoopToken) loop.preamble.token.short_preamble = short_loop + + # Clone ops and boxes to get private versions and forget the # values to allow them to be freed boxmap = {} @@ -90,6 +93,13 @@ a = boxmap[a] args.append(a) op.initarglist(args) + if op.is_guard(): + args = [] + for a in op.getfailargs(): + if not isinstance(a, Const): + a = boxmap[a] + args.append(a) + op.setfailargs(args) box = op.result if box: newbox = box.clonebox() @@ -195,7 +205,7 @@ except KeyError: return False - def create_short_preamble(self, preamble, preambleargs, + def create_short_preamble(self, preamble, preambleargs, preamble_token, loop, inputargs, token): #return None # Dissable @@ -203,17 +213,18 @@ short_preamble = [] loop_i = preamble_i = 0 while preamble_i < len(preamble)-1: - if self.sameop(preamble[preamble_i], loop[loop_i]) \ + op = preamble[preamble_i] + if self.sameop(op, loop[loop_i]) \ and loop_i < len(loop)-1: loop_i += 1 else: - if not state.safe_to_move(preamble[preamble_i]): + if not state.safe_to_move(op): debug_print("create_short_preamble failed due to", - "unsafe op:", preamble[preamble_i].getopnum(), + "unsafe op:", op.getopnum(), "at position: ", preamble_i) return None - short_preamble.append(preamble[preamble_i]) - state.update(preamble[preamble_i]) + short_preamble.append(op) + state.update(op) preamble_i += 1 @@ -247,10 +258,7 @@ jmp.setdescr(token) short_preamble.append(jmp) - # FIXME: Turn guards into conditional jumps to the preamble - - # Check that boxes used as arguemts are produced. Might not be - # needed, but let's play it safe. + # Check that boxes used as arguemts are produced. seen = {} for box in preambleargs: seen[box] = True @@ -263,6 +271,14 @@ if op.result: seen[op.result] = True + # Turn guards into conditional jumps to the preamble + for i in range(len(short_preamble)): + op = short_preamble[i] + if op.is_guard(): + op = op.clone() + op.setfailargs(preambleargs) + op.setjumptarget(preamble_token) + short_preamble[i] = op return short_preamble @@ -276,7 +292,7 @@ # to preamble def safe_to_move(self, op): opnum = op.getopnum() - if op.is_always_pure(): + if op.is_always_pure() or op.is_foldable_guard(): return True elif opnum == rop.JUMP: return True @@ -329,6 +345,10 @@ newop = op.clone() args = newop.getarglist() newop.initarglist([self.inline_arg(a) for a in args]) + + if op.is_guard(): + args = newop.getfailargs() + newop.setfailargs([self.inline_arg(a) for a in args]) if newop.result: old_result = newop.result Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resoperation.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resoperation.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/resoperation.py Tue Nov 23 21:15:26 2010 @@ -198,6 +198,7 @@ class GuardResOp(ResOpWithDescr): _fail_args = None + _jump_target = None def getfailargs(self): return self._fail_args @@ -205,14 +206,22 @@ def setfailargs(self, fail_args): self._fail_args = fail_args + def getjumptarget(self): + return self._jump_target + + def setjumptarget(self, jump_target): + self._jump_target = jump_target + def copy_and_change(self, opnum, args=None, result=None, descr=None): newop = AbstractResOp.copy_and_change(self, opnum, args, result, descr) newop.setfailargs(self.getfailargs()) + newop.setjumptarget(self.getjumptarget()) return newop def clone(self): newop = AbstractResOp.clone(self) newop.setfailargs(self.getfailargs()) + newop.setjumptarget(self.getjumptarget()) return newop From afa at codespeak.net Tue Nov 23 23:43:41 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 23 Nov 2010 23:43:41 +0100 (CET) Subject: [pypy-svn] r79437 - in pypy/branch/fast-forward/pypy/objspace/std: . test Message-ID: <20101123224341.CCEA8282B90@codespeak.net> Author: afa Date: Tue Nov 23 23:43:38 2010 New Revision: 79437 Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py Log: bytearray.extend shoud accept any buffer object Modified: pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/bytearrayobject.py Tue Nov 23 23:43:38 2010 @@ -365,7 +365,7 @@ w_bytearray.data += w_other.data def list_extend__Bytearray_ANY(space, w_bytearray, w_other): - w_bytearray.data += [c for c in space.str_w(w_other)] + w_bytearray.data += [c for c in space.bufferstr_w(w_other)] def inplace_add__Bytearray_Bytearray(space, w_bytearray1, w_bytearray2): list_extend__Bytearray_Bytearray(space, w_bytearray1, w_bytearray2) Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_bytes.py Tue Nov 23 23:43:38 2010 @@ -154,6 +154,8 @@ b.extend(bytearray('def')) b.extend('ghi') assert b == 'abcdefghi' + b.extend(buffer('jkl')) + assert b == 'abcdefghijkl' def test_delslice(self): b = bytearray('abcdefghi') From dstromberg at codespeak.net Wed Nov 24 00:38:40 2010 From: dstromberg at codespeak.net (dstromberg at codespeak.net) Date: Wed, 24 Nov 2010 00:38:40 +0100 (CET) Subject: [pypy-svn] r79438 - pypy/branch/gdbm Message-ID: <20101123233840.50092282BEA@codespeak.net> Author: dstromberg Date: Wed Nov 24 00:38:37 2010 New Revision: 79438 Added: pypy/branch/gdbm/ (props changed) - copied from r79437, pypy/trunk/ Log: Branching from trunk to merge gdbm module, rename dbm module to someplace unused. From dstromberg at codespeak.net Wed Nov 24 00:51:18 2010 From: dstromberg at codespeak.net (dstromberg at codespeak.net) Date: Wed, 24 Nov 2010 00:51:18 +0100 (CET) Subject: [pypy-svn] r79439 - pypy/branch/gdbm/lib_pypy Message-ID: <20101123235118.8AFBA282B90@codespeak.net> Author: dstromberg Date: Wed Nov 24 00:51:16 2010 New Revision: 79439 Added: pypy/branch/gdbm/lib_pypy/gdbm.py Log: Initial checkin: gdbm.py Added: pypy/branch/gdbm/lib_pypy/gdbm.py ============================================================================== --- (empty file) +++ pypy/branch/gdbm/lib_pypy/gdbm.py Wed Nov 24 00:51:16 2010 @@ -0,0 +1,245 @@ + +'''libgdbm wrapper using ctypes''' + +from __future__ import with_statement + +#mport sys +import ctypes +import ctypes.util + +class error(Exception): + '''Exception for gdbm errors''' + def __init__(self, msg): + Exception.__init__(self) + self.msg = msg + + def __str__(self): + return self.msg + +class datum(ctypes.Structure): + # pylint: disable=R0903 + # R0903: We don't need a lot of public methods + '''Hold a c string (with explicit length - not null terminated) describing a key or value''' + + _fields_ = [ + ('dptr', ctypes.POINTER(ctypes.c_char)), + ('dsize', ctypes.c_int), + ] + + def __init__(self, text=None): + #sys.stderr.write('blee creating a datum from %s\n' % text) + if text is None: + self.free_when_done = True + self.dptr = None + self.dsize = -1 + print 'No text, dptr is', bool(self.dptr) + else: + self.free_when_done = False + if isinstance(text, str): + ctypes.Structure.__init__(self, ctypes.create_string_buffer(text), len(text)) + else: + raise TypeError, "text not None or str" + + def __nonzero__(self): + if self.dptr: + return True + else: + return False + + __bool__ = __nonzero__ + + def __str__(self): + char_star = ctypes.cast(self.dptr, ctypes.POINTER(ctypes.c_char)) + return char_star[:self.dsize] + + def __enter__(self): + self.free_when_done = True + return self + + def free(self): + '''Free, in the style of C, the malloc'd memory associated with this datum (key or value)''' + address = ctypes.cast(self.dptr, ctypes.POINTER(ctypes.c_char)) + FREE(address) + + def __exit__(self, *args): + # pylint: disable=W0212 + # W0212: We seem to need _b_needsfree_ + + # if dptr isn't a NULL pointer, free what it points at + #sys.stderr.write('blee: self.dptr %s\n' % self.dptr) + #sys.stderr.write('blee: dir(self.dptr) %s\n' % dir(self.dptr)) + # sys.stderr.write('blee: self.dptr.getcontents() %s\n' % self.dptr.getcontents()) + #sys.stderr.write('blee: ctypes.addressof(self.dptr) %s\n' % hex(ctypes.addressof(self.dptr))) + #sys.stderr.write('blee: ctypes.byref(self.dptr) %s\n' % ctypes.byref(self.dptr)) + if self.free_when_done and self.dptr: + self.free() + +class dbm(object): + '''A database object - providing access to gdbm tables''' + def __init__(self, dbmobj): + self._aobj = dbmobj + + def close(self): + '''Close a table''' + if not self._aobj: + raise error('gdbm object has already been closed') + # Note that gdbm_close will free the memory malloc'd for the open database (but not for keys or values) + getattr(GDBM_LIB, FUNCS['close'])(self._aobj) + self._aobj = None + + def __del__(self): + if self._aobj: + self.close() + + def keys(self): + '''Get a list of the keys in the database''' + if not self._aobj: + raise error('gdbm object has already been closed') + allkeys = [] + prior_key = getattr(GDBM_LIB, FUNCS['firstkey'])(self._aobj) + while prior_key: + key = str(prior_key) + allkeys.append(key) + new_key = getattr(GDBM_LIB, FUNCS['nextkey'])(self._aobj, prior_key) + prior_key.free() + prior_key = new_key + return allkeys + + def get(self, key, default=None): + '''Get a key from the table''' + if not self._aobj: + raise error('gdbm object has already been closed') + key_datum = datum(key) + with getattr(GDBM_LIB, FUNCS['fetch'])(self._aobj, key_datum) as value_datum: + if value_datum: + result = str(value_datum) + return result + return default + + def __len__(self): + return len(self.keys()) + + def __getitem__(self, key): + # It's OK to use self.get here, because gdbm itself can never return a None; + # It can only return strings (including the empty string), so None is a distinct sentinel value + value = self.get(key) + if value is None: + raise KeyError(key) + return value + + def __setitem__(self, key, value): + if not self._aobj: + raise error('gdbm object has already been closed') + key_datum = datum(key) + value = datum(value) + status = getattr(GDBM_LIB, FUNCS['store'])(self._aobj, key_datum, value, GDBM_LIB.DBM_REPLACE) + return status + + def setdefault(self, key, default=''): + '''Get and maybe set a default''' + if not self._aobj: + raise error('gdbm object has already been closed') + value = self.get(key) + if value is not None: + return value + status = self[key] = default + if status < 0: + raise error("cannot add item to database") + return default + + def has_key(self, key): + '''Does the table contain the requested key?''' + if not self._aobj: + raise error('gdbm object has already been closed') + value = self.get(key) + if value is None: + return False + else: + return True + + def __delitem__(self, key): + if not self._aobj: + raise error('gdbm object has already been closed') + key_datum = datum(key) + status = getattr(GDBM_LIB, FUNCS['delete'])(self._aobj, key_datum) + if status < 0: + raise KeyError(key) + +### initialization + +def _init_func(name, argtypes=None, restype=None): + '''Extract functions from libgdbm''' + try: + func = getattr(GDBM_LIB, '_gdbm_' + name) + FUNCS[name] = '_gdbm_' + name + except AttributeError: + func = getattr(GDBM_LIB, 'gdbm_' + name) + FUNCS[name] = 'gdbm_' + name + if argtypes is not None: + func.argtypes = argtypes + if restype is not None: + func.restype = restype + +GDBM_LIBPATH = ctypes.util.find_library('gdbm') +if not GDBM_LIBPATH: + raise ImportError("Cannot find gdbm library") +GDBM_LIB = ctypes.CDLL(GDBM_LIBPATH) # Linux + +FUNCS = {} +_init_func('open', argtypes=(ctypes.c_char_p, ctypes.c_int, ctypes.c_int)) +_init_func('close', restype=ctypes.c_void_p) +_init_func('firstkey', restype=datum) +_init_func('nextkey', argtypes=(ctypes.c_void_p, datum), restype=datum) +_init_func('fetch', restype=datum) +_init_func('store', restype=ctypes.c_int) +_init_func('delete', restype=ctypes.c_int) + +C_LIBPATH = ctypes.util.find_library('c') +if not C_LIBPATH: + raise ImportError("Cannot find c library") +C_LIB = ctypes.CDLL(C_LIBPATH) # Linux + +try: + FREE = getattr(C_LIB, '_free') +except AttributeError: + FREE = getattr(C_LIB, 'free') +#FREE.argtypes = [ ctypes.POINTER(ctypes.c_char) ] +FREE.argtypes = [ ctypes.c_void_p ] +FREE.restype = None + +GDBM_LIB.DBM_INSERT = 0 +GDBM_LIB.DBM_REPLACE = 1 + +# pylint: disable=W0622 +# W0622: We need to redefine open this time - it's part of the API +def open(filename, flag='r', mode=0666): + "open a gdbm database" + if not isinstance(filename, str): + raise TypeError("expected string") + + openflag = 0 + + gdbm_reader = 0 # A reader. + gdbm_writer = 1 # A writer. + gdbm_wrcreat = 2 # A writer. Create the db if needed. + gdbm_newdb = 3 # A writer. Always create a new db. + + try: + openflag = { + 'r': gdbm_reader, + 'rw': gdbm_writer, + 'w': gdbm_writer, + 'c': gdbm_wrcreat, + 'n': gdbm_newdb, + }[flag] + except KeyError: + raise error("arg 2 to open should be 'r', 'w', 'rw', 'c', or 'n'") + + # filename, block_size, read_write, mode, fatal_func + a_db = getattr(GDBM_LIB, FUNCS['open'])(filename, 2**18, openflag, mode, 0) + if a_db == 0: + raise error("Could not open file %s" % filename) + return dbm(a_db) + +__all__ = ('datum', 'dbm', 'error', 'open') + From dstromberg at codespeak.net Wed Nov 24 00:53:48 2010 From: dstromberg at codespeak.net (dstromberg at codespeak.net) Date: Wed, 24 Nov 2010 00:53:48 +0100 (CET) Subject: [pypy-svn] r79440 - in pypy/branch/gdbm: attic lib_pypy Message-ID: <20101123235348.1973B282BEA@codespeak.net> Author: dstromberg Date: Wed Nov 24 00:53:46 2010 New Revision: 79440 Added: pypy/branch/gdbm/attic/ pypy/branch/gdbm/attic/dbm.py - copied unchanged from r79438, pypy/branch/gdbm/lib_pypy/dbm.py Removed: pypy/branch/gdbm/lib_pypy/dbm.py Log: Moved lib_py/dbm.py to attack/dbm.py, because: 1) Although it works fine when you use it from anydbm 2) If you import it directly, as some will surely do, it seems to be NDBM 3) But it's really a Berkeley DB under the covers, and will appear as such in the filesystem 4) The API isn't really what people will be expecting when importing it directly as a bsddb anydbm should continue to function, because it'll be able to use gdbm.py - previous checkin. From dstromberg at codespeak.net Wed Nov 24 01:07:59 2010 From: dstromberg at codespeak.net (dstromberg at codespeak.net) Date: Wed, 24 Nov 2010 01:07:59 +0100 (CET) Subject: [pypy-svn] r79441 - pypy/branch/gdbm/lib_pypy Message-ID: <20101124000759.A41FB5080B@codespeak.net> Author: dstromberg Date: Wed Nov 24 01:07:57 2010 New Revision: 79441 Modified: pypy/branch/gdbm/lib_pypy/gdbm.py Log: Convert from char * to str more carefully; this enables the gdbm.py module to pass anydbm's tests. Modified: pypy/branch/gdbm/lib_pypy/gdbm.py ============================================================================== --- pypy/branch/gdbm/lib_pypy/gdbm.py (original) +++ pypy/branch/gdbm/lib_pypy/gdbm.py Wed Nov 24 01:07:57 2010 @@ -50,7 +50,11 @@ def __str__(self): char_star = ctypes.cast(self.dptr, ctypes.POINTER(ctypes.c_char)) - return char_star[:self.dsize] + # we can't just slice a char *, so we convert to str, one character (byte) at a time + result = [] + for index in xrange(self.dsize): + result.append(char_star[index]) + return ''.join(result) def __enter__(self): self.free_when_done = True From hakanardo at codespeak.net Wed Nov 24 07:59:10 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Wed, 24 Nov 2010 07:59:10 +0100 (CET) Subject: [pypy-svn] r79442 - pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph Message-ID: <20101124065910.B9892282B90@codespeak.net> Author: hakanardo Date: Wed Nov 24 07:59:08 2010 New Revision: 79442 Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py Log: fixed translation Modified: pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/backend/llgraph/llimpl.py Wed Nov 24 07:59:08 2010 @@ -1613,6 +1613,7 @@ setannotation(compile_add_ref_result, annmodel.SomeInteger()) setannotation(compile_add_float_result, annmodel.SomeInteger()) setannotation(compile_add_jump_target, annmodel.s_None) +setannotation(compile_add_guard_jump_target, annmodel.s_None) setannotation(compile_add_fail, annmodel.SomeInteger()) setannotation(compile_add_fail_arg, annmodel.s_None) setannotation(compile_redirect_fail, annmodel.s_None) From fijal at codespeak.net Wed Nov 24 09:48:44 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 24 Nov 2010 09:48:44 +0100 (CET) Subject: [pypy-svn] r79443 - pypy/extradoc/talk/ctpug2010 Message-ID: <20101124084844.5B1D5282B9D@codespeak.net> Author: fijal Date: Wed Nov 24 09:48:42 2010 New Revision: 79443 Added: pypy/extradoc/talk/ctpug2010/ Log: Cape Town Python User Group 2010 From cfbolz at codespeak.net Wed Nov 24 10:29:06 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 24 Nov 2010 10:29:06 +0100 (CET) Subject: [pypy-svn] r79444 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101124092906.4A35C282B9D@codespeak.net> Author: cfbolz Date: Wed Nov 24 10:29:04 2010 New Revision: 79444 Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py Log: For now just hide the direct_ptradd. Should be fixed later. Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/converter.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/converter.py Wed Nov 24 10:29:04 2010 @@ -3,6 +3,7 @@ from pypy.interpreter.buffer import Buffer from pypy.rpython.lltypesystem import rffi, lltype from pypy.rlib.rarithmetic import r_singlefloat +from pypy.rlib import jit from pypy.module._rawffi.interp_rawffi import unpack_simple_shape from pypy.module._rawffi.array import W_Array @@ -22,6 +23,7 @@ def __init__(self, space, array_size): pass + @jit.dont_look_inside def _get_fieldptr(self, space, w_obj, offset): rawobject = get_rawobject(space, w_obj) return lltype.direct_ptradd(rawobject, offset) From cfbolz at codespeak.net Wed Nov 24 10:39:06 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 24 Nov 2010 10:39:06 +0100 (CET) Subject: [pypy-svn] r79445 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101124093906.50565282B9E@codespeak.net> Author: cfbolz Date: Wed Nov 24 10:39:04 2010 New Revision: 79445 Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Log: re-enabled hack call, using a different exception than NotImplementedError Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Wed Nov 24 10:39:04 2010 @@ -12,6 +12,8 @@ from pypy.module.cppyy import converter, executor +class HackCallNotPossible(Exception): + pass NULL_VOIDP = lltype.nullptr(rffi.VOIDP.TO) @@ -71,18 +73,18 @@ self.executor = executor.get_executor(self.space, result_type) self.arg_converters = None # - #self.hack_call = arg_types == ['int'] and result_type == 'int' + self.hack_call = arg_types == ['int'] and result_type == 'int' # def call(self, cppthis, args_w): if self.executor is None: raise OperationError(self.space.w_TypeError, self.space.wrap("return type not handled")) - #if self.hack_call: - # try: - # return self.do_hack_call(cppthis, args_w) - # except NotImplementedError: - # pass + if self.hack_call: + try: + return self.do_hack_call(cppthis, args_w) + except HackCallNotPossible: + pass args = self.prepare_arguments(args_w) try: @@ -101,7 +103,7 @@ methgetter = get_methptr_getter(self.cpptype.handle, self.method_index) if not methgetter: - raise NotImplementedError + raise HackCallNotPossible funcptr = methgetter(cppthis) funcptr = rffi.cast(self.INT_2_INT_FNPTR, funcptr) result = funcptr(cppthis, arg) From arigo at codespeak.net Wed Nov 24 11:42:19 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 11:42:19 +0100 (CET) Subject: [pypy-svn] r79446 - pypy/trunk/pypy/rpython/memory/gc Message-ID: <20101124104219.9414536C221@codespeak.net> Author: arigo Date: Wed Nov 24 11:42:17 2010 New Revision: 79446 Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py Log: Fix max_delta. It was broken. No tests yet -- should start writing tests for the env vars... Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Wed Nov 24 11:42:17 2010 @@ -204,7 +204,7 @@ self.min_heap_size = 0.0 self.max_heap_size = 0.0 self.max_heap_size_already_raised = False - self.max_delta = 0.0 + self.max_delta = float(r_uint(-1)) # self.card_page_indices = card_page_indices if self.card_page_indices > 0: @@ -358,9 +358,6 @@ # Set the next_major_collection_threshold. threshold_max = (self.next_major_collection_threshold * self.growth_rate_max) - if self.max_delta > 0.0: - threshold_max = min(threshold_max, - self.next_major_collection_threshold + self.max_delta) if threshold > threshold_max: threshold = threshold_max # @@ -1261,9 +1258,12 @@ # # Set the threshold for the next major collection to be when we # have allocated 'major_collection_threshold' times more than + # we currently have -- but no more than 'max_delta' more than # we currently have. + total_memory_used = float(self.get_total_memory_used()) bounded = self.set_major_threshold_from( - self.get_total_memory_used() * self.major_collection_threshold, + min(total_memory_used * self.major_collection_threshold, + total_memory_used + self.max_delta), reserving_size) # # Max heap size: gives an upper bound on the threshold. If we From arigo at codespeak.net Wed Nov 24 11:42:46 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 11:42:46 +0100 (CET) Subject: [pypy-svn] r79447 - pypy/branch/jit-free/pypy/rpython/memory/gc Message-ID: <20101124104246.F08D736C222@codespeak.net> Author: arigo Date: Wed Nov 24 11:42:45 2010 New Revision: 79447 Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py Log: Merge r79446. Modified: pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/branch/jit-free/pypy/rpython/memory/gc/minimark.py Wed Nov 24 11:42:45 2010 @@ -205,7 +205,7 @@ self.min_heap_size = 0.0 self.max_heap_size = 0.0 self.max_heap_size_already_raised = False - self.max_delta = 0.0 + self.max_delta = float(r_uint(-1)) # self.card_page_indices = card_page_indices if self.card_page_indices > 0: @@ -359,9 +359,6 @@ # Set the next_major_collection_threshold. threshold_max = (self.next_major_collection_threshold * self.growth_rate_max) - if self.max_delta > 0.0: - threshold_max = min(threshold_max, - self.next_major_collection_threshold + self.max_delta) if threshold > threshold_max: threshold = threshold_max # @@ -1262,9 +1259,12 @@ # # Set the threshold for the next major collection to be when we # have allocated 'major_collection_threshold' times more than + # we currently have -- but no more than 'max_delta' more than # we currently have. + total_memory_used = float(self.get_total_memory_used()) bounded = self.set_major_threshold_from( - self.get_total_memory_used() * self.major_collection_threshold, + min(total_memory_used * self.major_collection_threshold, + total_memory_used + self.max_delta), reserving_size) # # Max heap size: gives an upper bound on the threshold. If we From arigo at codespeak.net Wed Nov 24 12:44:08 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 12:44:08 +0100 (CET) Subject: [pypy-svn] r79448 - pypy/trunk/site-packages Message-ID: <20101124114408.F3A1436C220@codespeak.net> Author: arigo Date: Wed Nov 24 12:44:07 2010 New Revision: 79448 Modified: pypy/trunk/site-packages/ (props changed) Log: svn:ignore. From arigo at codespeak.net Wed Nov 24 13:15:21 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 13:15:21 +0100 (CET) Subject: [pypy-svn] r79449 - in pypy/trunk/pypy: interpreter jit/backend jit/backend/llgraph jit/backend/llsupport jit/backend/test jit/backend/x86 jit/backend/x86/test jit/metainterp jit/metainterp/test jit/tool jit/tool/test module/__pypy__ module/__pypy__/test module/_pickle_support rlib rpython/memory/gc tool translator translator/c translator/c/test translator/tool Message-ID: <20101124121521.007E85080B@codespeak.net> Author: arigo Date: Wed Nov 24 13:15:18 2010 New Revision: 79449 Added: pypy/trunk/pypy/jit/metainterp/memmgr.py - copied unchanged from r79448, pypy/branch/jit-free/pypy/jit/metainterp/memmgr.py pypy/trunk/pypy/jit/metainterp/test/test_memmgr.py - copied unchanged from r79448, pypy/branch/jit-free/pypy/jit/metainterp/test/test_memmgr.py pypy/trunk/pypy/module/__pypy__/interp_debug.py - copied unchanged from r79448, pypy/branch/jit-free/pypy/module/__pypy__/interp_debug.py pypy/trunk/pypy/module/__pypy__/test/test_debug.py - copied unchanged from r79448, pypy/branch/jit-free/pypy/module/__pypy__/test/test_debug.py pypy/trunk/pypy/tool/debug_print.py - copied unchanged from r79448, pypy/branch/jit-free/pypy/tool/debug_print.py Modified: pypy/trunk/pypy/interpreter/baseobjspace.py pypy/trunk/pypy/interpreter/generator.py pypy/trunk/pypy/jit/backend/llgraph/llimpl.py pypy/trunk/pypy/jit/backend/llgraph/runner.py pypy/trunk/pypy/jit/backend/llsupport/gc.py pypy/trunk/pypy/jit/backend/model.py pypy/trunk/pypy/jit/backend/test/runner_test.py pypy/trunk/pypy/jit/backend/test/test_random.py pypy/trunk/pypy/jit/backend/x86/assembler.py pypy/trunk/pypy/jit/backend/x86/regalloc.py pypy/trunk/pypy/jit/backend/x86/runner.py pypy/trunk/pypy/jit/backend/x86/test/test_regalloc.py pypy/trunk/pypy/jit/backend/x86/test/test_runner.py pypy/trunk/pypy/jit/metainterp/compile.py pypy/trunk/pypy/jit/metainterp/history.py pypy/trunk/pypy/jit/metainterp/jitprof.py pypy/trunk/pypy/jit/metainterp/pyjitpl.py pypy/trunk/pypy/jit/metainterp/test/test_basic.py pypy/trunk/pypy/jit/metainterp/test/test_compile.py pypy/trunk/pypy/jit/metainterp/test/test_logger.py pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py pypy/trunk/pypy/jit/metainterp/test/test_pyjitpl.py pypy/trunk/pypy/jit/metainterp/test/test_warmspot.py pypy/trunk/pypy/jit/metainterp/test/test_warmstate.py pypy/trunk/pypy/jit/metainterp/warmspot.py pypy/trunk/pypy/jit/metainterp/warmstate.py pypy/trunk/pypy/jit/tool/jitoutput.py pypy/trunk/pypy/jit/tool/test/test_jitoutput.py pypy/trunk/pypy/module/__pypy__/__init__.py pypy/trunk/pypy/module/_pickle_support/maker.py pypy/trunk/pypy/rlib/debug.py pypy/trunk/pypy/rlib/jit.py pypy/trunk/pypy/rpython/memory/gc/base.py pypy/trunk/pypy/rpython/memory/gc/inspector.py pypy/trunk/pypy/rpython/memory/gc/minimark.py pypy/trunk/pypy/rpython/memory/gc/semispace.py pypy/trunk/pypy/translator/c/funcgen.py pypy/trunk/pypy/translator/c/test/test_newgc.py pypy/trunk/pypy/translator/c/test/test_standalone.py pypy/trunk/pypy/translator/driver.py pypy/trunk/pypy/translator/tool/reftracker.py Log: (antocuni, arigo) Merge branch/jit-free. The major point of the branch is to reduce memory usage of long-running pypy's with the JIT (such as "pypy translate.py"). Done by playing with weakrefs and having a __del__ that kills the ResumeGuardDescrs of old loops. This is controlled by "--jit loop_longevity=N". (Actually freeing the assembler code would be the next step, but is less important, because the ResumeDescrs take more memory.) In addition, this branch contains a number of semi-related things: * Generators: set their 'frame' to None when they finish. This helps a lot the GC. (The issue is that generators need an interp-level __del__; this let the GC free the frame earlier.) * rgc.dump_rpy_heap() now works with almost no need for additional memory. * pypy.rlib.debug.debug_print(r_longlong(...)). * add debug_{start,stop,print} to the __pypy__ module. This lets you write to the PYPYLOG file from app-level. * use this new interface in translator/driver, to record the begin and end of tasks (via pypy.tool.debug_print so that it does not crash on CPython). * improve a bit reftracker.py. The net result is that you can now translate pypy with a pypy-jit on 32-bits, using roughly 2.5 GB of RAM. It is twice as big (for now) as CPython, but twice as fast :-) Modified: pypy/trunk/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/trunk/pypy/interpreter/baseobjspace.py (original) +++ pypy/trunk/pypy/interpreter/baseobjspace.py Wed Nov 24 13:15:18 2010 @@ -147,7 +147,7 @@ __already_enqueued_for_destruction = False - def _enqueue_for_destruction(self, space): + def _enqueue_for_destruction(self, space, call_user_del=True): """Put the object in the destructor queue of the space. At a later, safe point in time, UserDelAction will use space.userdel() to call the object's app-level __del__ method. @@ -160,7 +160,8 @@ return self.__already_enqueued_for_destruction = True self.clear_all_weakrefs() - space.user_del_action.register_dying_object(self) + if call_user_del: + space.user_del_action.register_dying_object(self) def _call_builtin_destructor(self): pass # method overridden in typedef.py Modified: pypy/trunk/pypy/interpreter/generator.py ============================================================================== --- pypy/trunk/pypy/interpreter/generator.py (original) +++ pypy/trunk/pypy/interpreter/generator.py Wed Nov 24 13:15:18 2010 @@ -10,7 +10,7 @@ def __init__(self, frame): self.space = frame.space - self.frame = frame + self.frame = frame # turned into None when frame_finished_execution self.running = False def descr__reduce__(self, space): @@ -19,9 +19,13 @@ mod = space.interp_w(MixedModule, w_mod) new_inst = mod.get('generator_new') w = space.wrap + if self.frame: + w_frame = w(self.frame) + else: + w_frame = space.w_None tup = [ - w(self.frame), + w_frame, w(self.running), ] @@ -41,7 +45,8 @@ if self.running: raise OperationError(space.w_ValueError, space.wrap('generator already executing')) - if self.frame.frame_finished_execution: + frame = self.frame + if frame is None: # xxx a bit ad-hoc, but we don't want to go inside # execute_generator_frame() if the frame is actually finished if operr is None: @@ -49,7 +54,7 @@ raise operr # XXX it's not clear that last_instr should be promoted at all # but as long as it is necessary for call_assembler, let's do it early - last_instr = jit.hint(self.frame.last_instr, promote=True) + last_instr = jit.hint(frame.last_instr, promote=True) if last_instr == -1: if w_arg and not space.is_w(w_arg, space.w_None): msg = "can't send non-None value to a just-started generator" @@ -60,18 +65,19 @@ self.running = True try: try: - w_result = self.frame.execute_generator_frame(w_arg, operr) + w_result = frame.execute_generator_frame(w_arg, operr) except OperationError: # errors finish a frame - self.frame.frame_finished_execution = True + self.frame = None raise # if the frame is now marked as finished, it was RETURNed from - if self.frame.frame_finished_execution: + if frame.frame_finished_execution: + self.frame = None raise OperationError(space.w_StopIteration, space.w_None) else: return w_result # YIELDed finally: - self.frame.f_backref = jit.vref_None + frame.f_backref = jit.vref_None self.running = False def descr_throw(self, w_type, w_val=None, w_tb=None): @@ -115,7 +121,7 @@ raise OperationError(space.w_RuntimeError, space.wrap(msg)) def descr_gi_frame(space, self): - if not self.frame.frame_finished_execution: + if self.frame is not None and not self.frame.frame_finished_execution: return self.frame else: return space.w_None @@ -125,15 +131,17 @@ applevel __del__, which is called at a safe point after the interp-level __del__ enqueued the object for destruction """ - # Only bother raising an exception if the frame is still not - # finished and finally or except blocks are present. - if not self.frame.frame_finished_execution: + self.descr_close() + + def __del__(self): + # Only bother enqueuing self to raise an exception if the frame is + # still not finished and finally or except blocks are present. + must_call_close = False + if self.frame is not None: block = self.frame.lastblock while block is not None: if not isinstance(block, LoopBlock): - self.descr_close() - return + must_call_close = True + break block = block.previous - - def __del__(self): - self._enqueue_for_destruction(self.space) + self._enqueue_for_destruction(self.space, must_call_close) Modified: pypy/trunk/pypy/jit/backend/llgraph/llimpl.py ============================================================================== --- pypy/trunk/pypy/jit/backend/llgraph/llimpl.py (original) +++ pypy/trunk/pypy/jit/backend/llgraph/llimpl.py Wed Nov 24 13:15:18 2010 @@ -4,6 +4,7 @@ when executing on top of the llinterpreter. """ +import weakref from pypy.objspace.flow.model import Variable, Constant from pypy.annotation import model as annmodel from pypy.jit.metainterp.history import (ConstInt, ConstPtr, @@ -161,6 +162,8 @@ # ____________________________________________________________ class CompiledLoop(object): + has_been_freed = False + def __init__(self): self.inputargs = [] self.operations = [] @@ -285,6 +288,11 @@ del _variables[:] return _to_opaque(CompiledLoop()) +def mark_as_free(loop): + loop = _from_opaque(loop) + assert not loop.has_been_freed + loop.has_been_freed = True + def compile_start_int_var(loop): return compile_start_ref_var(loop, lltype.Signed) @@ -317,7 +325,7 @@ raise ValueError("CALL_ASSEMBLER not supported") loop = _from_opaque(loop) op = loop.operations[-1] - op.descr = descr + op.descr = weakref.ref(descr) def compile_add_var(loop, intvar): loop = _from_opaque(loop) @@ -429,6 +437,7 @@ verbose = True self.opindex = 0 while True: + assert not self.loop.has_been_freed op = self.loop.operations[self.opindex] args = [self.getenv(v) for v in op.args] if not op.is_final(): @@ -440,7 +449,10 @@ _stats.exec_conditional_jumps += 1 if op.jump_target is not None: # a patched guard, pointing to further code - args = [self.getenv(v) for v in op.fail_args if v] + if op.fail_args: + args = [self.getenv(v) for v in op.fail_args if v] + else: + args = [] assert len(op.jump_target.inputargs) == len(args) self.env = dict(zip(op.jump_target.inputargs, args)) self.loop = op.jump_target @@ -839,14 +851,22 @@ finally: self._may_force = -1 - def op_call_assembler(self, loop_token, *args): + def op_call_assembler(self, wref_loop_token, *args): + if we_are_translated(): + raise ValueError("CALL_ASSEMBLER not supported") + return self._do_call_assembler(wref_loop_token, *args) + + def _do_call_assembler(self, wref_loop_token, *args): global _last_exception + loop_token = wref_loop_token() + assert loop_token, "CALL_ASSEMBLER to a target that already died" + ctl = loop_token.compiled_loop_token + if hasattr(ctl, 'redirected'): + return self._do_call_assembler(ctl.redirected, *args) assert not self._forced - loop_token = self.cpu._redirected_call_assembler.get(loop_token, - loop_token) self._may_force = self.opindex try: - inpargs = _from_opaque(loop_token._llgraph_compiled_version).inputargs + inpargs = _from_opaque(ctl.compiled_version).inputargs for i, inparg in enumerate(inpargs): TYPE = inparg.concretetype if TYPE is lltype.Signed: @@ -1539,10 +1559,13 @@ do_setfield_gc_int(vable, fielddescr.ofs, 0) def redirect_call_assembler(cpu, oldlooptoken, newlooptoken): - OLD = _from_opaque(oldlooptoken._llgraph_compiled_version).getargtypes() - NEW = _from_opaque(newlooptoken._llgraph_compiled_version).getargtypes() + oldclt = oldlooptoken.compiled_loop_token + newclt = newlooptoken.compiled_loop_token + OLD = _from_opaque(oldclt.compiled_version).getargtypes() + NEW = _from_opaque(newclt.compiled_version).getargtypes() assert OLD == NEW - cpu._redirected_call_assembler[oldlooptoken] = newlooptoken + assert not hasattr(oldclt, 'redirected') + oldclt.redirected = weakref.ref(newlooptoken) # ____________________________________________________________ @@ -1609,6 +1632,7 @@ setannotation(compile_add_fail, annmodel.SomeInteger()) setannotation(compile_add_fail_arg, annmodel.s_None) setannotation(compile_redirect_fail, annmodel.s_None) +setannotation(mark_as_free, annmodel.s_None) setannotation(new_frame, s_Frame) setannotation(frame_clear, annmodel.s_None) Modified: pypy/trunk/pypy/jit/backend/llgraph/runner.py ============================================================================== --- pypy/trunk/pypy/jit/backend/llgraph/runner.py (original) +++ pypy/trunk/pypy/jit/backend/llgraph/runner.py Wed Nov 24 13:15:18 2010 @@ -102,7 +102,6 @@ llimpl._llinterp = LLInterpreter(self.rtyper) self._future_values = [] self._descrs = {} - self._redirected_call_assembler = {} def _freeze_(self): assert self.translate_support_code @@ -118,22 +117,34 @@ self._descrs[key] = descr return descr - def compile_bridge(self, faildescr, inputargs, operations, log=True): + def compile_bridge(self, faildescr, inputargs, operations, + original_loop_token, log=True): c = llimpl.compile_start() + clt = original_loop_token.compiled_loop_token + clt.loop_and_bridges.append(c) + clt.compiling_a_bridge() self._compile_loop_or_bridge(c, inputargs, operations) old, oldindex = faildescr._compiled_fail llimpl.compile_redirect_fail(old, oldindex, c) - def compile_loop(self, inputargs, operations, loopdescr, log=True): + def compile_loop(self, inputargs, operations, looptoken, log=True): """In a real assembler backend, this should assemble the given list of operations. Here we just generate a similar CompiledLoop instance. The code here is RPython, whereas the code in llimpl is not. """ c = llimpl.compile_start() - loopdescr._llgraph_compiled_version = c + clt = model.CompiledLoopToken(self, looptoken.number) + clt.loop_and_bridges = [c] + clt.compiled_version = c + looptoken.compiled_loop_token = clt self._compile_loop_or_bridge(c, inputargs, operations) + def free_loop_and_bridges(self, compiled_loop_token): + for c in compiled_loop_token.loop_and_bridges: + llimpl.mark_as_free(c) + model.AbstractCPU.free_loop_and_bridges(self, compiled_loop_token) + def _compile_loop_or_bridge(self, c, inputargs, operations): var2index = {} for box in inputargs: @@ -207,7 +218,7 @@ if op.getopnum() == rop.JUMP: targettoken = op.getdescr() assert isinstance(targettoken, history.LoopToken) - compiled_version = targettoken._llgraph_compiled_version + compiled_version = targettoken.compiled_loop_token.compiled_version llimpl.compile_add_jump_target(c, compiled_version) elif op.getopnum() == rop.FINISH: faildescr = op.getdescr() @@ -217,7 +228,7 @@ assert False, "unknown operation" def _execute_token(self, loop_token): - compiled_version = loop_token._llgraph_compiled_version + compiled_version = loop_token.compiled_loop_token.compiled_version frame = llimpl.new_frame(self.is_oo, self) # setup the frame llimpl.frame_clear(frame, compiled_version) Modified: pypy/trunk/pypy/jit/backend/llsupport/gc.py ============================================================================== --- pypy/trunk/pypy/jit/backend/llsupport/gc.py (original) +++ pypy/trunk/pypy/jit/backend/llsupport/gc.py Wed Nov 24 13:15:18 2010 @@ -254,7 +254,7 @@ def _enlarge_gcmap(self): newlength = 250 + self._gcmap_maxlength * 2 newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw', - track_allocation=False) + track_allocation=False) # YYY leak oldgcmap = self._gcmap for i in range(self._gcmap_curlength): newgcmap[i] = oldgcmap[i] @@ -311,7 +311,7 @@ length = len(shape) compressed = lltype.malloc(self.CALLSHAPE_ARRAY, length, flavor='raw', - track_allocation=False) # memory leak + track_allocation=False) # YYY leak for i in range(length): compressed[length-1-i] = rffi.cast(rffi.UCHAR, shape[i]) return llmemory.cast_ptr_to_adr(compressed) Modified: pypy/trunk/pypy/jit/backend/model.py ============================================================================== --- pypy/trunk/pypy/jit/backend/model.py (original) +++ pypy/trunk/pypy/jit/backend/model.py Wed Nov 24 13:15:18 2010 @@ -1,3 +1,4 @@ +from pypy.rlib.debug import debug_start, debug_print, debug_stop from pypy.jit.metainterp import history, compile @@ -7,17 +8,27 @@ done_with_this_frame_int_v = -1 done_with_this_frame_ref_v = -1 done_with_this_frame_float_v = -1 + total_compiled_loops = 0 + total_compiled_bridges = 0 + total_freed_loops = 0 + total_freed_bridges = 0 def __init__(self): self.fail_descr_list = [] + self.fail_descr_free_list = [] def get_fail_descr_number(self, descr): assert isinstance(descr, history.AbstractFailDescr) n = descr.index if n < 0: lst = self.fail_descr_list - n = len(lst) - lst.append(descr) + if len(self.fail_descr_free_list) > 0: + n = self.fail_descr_free_list.pop() + assert lst[n] is None + lst[n] = descr + else: + n = len(lst) + lst.append(descr) descr.index = n return n @@ -35,12 +46,14 @@ def compile_loop(self, inputargs, operations, looptoken, log=True): """Assemble the given loop. - Extra attributes should be put in the LoopToken to - point to the compiled loop in assembler. + Should create and attach a fresh CompiledLoopToken to + looptoken.compiled_loop_token and stick extra attributes + on it to point to the compiled loop in assembler. """ raise NotImplementedError - def compile_bridge(self, faildescr, inputargs, operations, log=True): + def compile_bridge(self, faildescr, inputargs, operations, + original_loop_token, log=True): """Assemble the bridge. The FailDescr is the descr of the original guard that failed. """ @@ -113,6 +126,24 @@ oldlooptoken so that from now own they will call newlooptoken.""" raise NotImplementedError + def free_loop_and_bridges(self, compiled_loop_token): + """This method is called to free resources (machine code, + references to resume guards, etc.) allocated by the compilation + of a loop and all bridges attached to it. After this call, the + frontend cannot use this compiled loop any more; in fact, it + guarantees that at the point of the call to free_code_group(), + none of the corresponding assembler is currently running. + """ + # The base class provides a limited implementation: freeing the + # resume descrs. This is already quite helpful, because the + # resume descrs are the largest consumers of memory (about 3x + # more than the assembler, in the case of the x86 backend). + lst = self.fail_descr_list + for n in compiled_loop_token.faildescr_indices: + lst[n] = None + self.fail_descr_free_list.extend(compiled_loop_token.faildescr_indices) + # We expect 'compiled_loop_token' to be itself garbage-collected soon. + @staticmethod def sizeof(S): raise NotImplementedError @@ -237,3 +268,37 @@ def force(self, force_token): raise NotImplementedError + + +class CompiledLoopToken(object): + def __init__(self, cpu, number): + cpu.total_compiled_loops += 1 + self.cpu = cpu + self.number = number + self.bridges_count = 0 + # This growing list gives the 'descr_number' of all fail descrs + # that belong to this loop or to a bridge attached to it. + # Filled by the frontend calling record_faildescr_index(). + self.faildescr_indices = [] + debug_start("jit-mem-looptoken-alloc") + debug_print("allocating Loop #", self.number) + debug_stop("jit-mem-looptoken-alloc") + + def record_faildescr_index(self, n): + self.faildescr_indices.append(n) + + def compiling_a_bridge(self): + self.cpu.total_compiled_bridges += 1 + self.bridges_count += 1 + debug_start("jit-mem-looptoken-alloc") + debug_print("allocating Bridge #", self.bridges_count, "of Loop #", self.number) + debug_stop("jit-mem-looptoken-alloc") + + def __del__(self): + debug_start("jit-mem-looptoken-free") + debug_print("freeing Loop #", self.number, 'with', + self.bridges_count, 'attached bridges') + self.cpu.free_loop_and_bridges(self) + self.cpu.total_freed_loops += 1 + self.cpu.total_freed_bridges += self.bridges_count + debug_stop("jit-mem-looptoken-free") Modified: pypy/trunk/pypy/jit/backend/test/runner_test.py ============================================================================== --- pypy/trunk/pypy/jit/backend/test/runner_test.py (original) +++ pypy/trunk/pypy/jit/backend/test/runner_test.py Wed Nov 24 13:15:18 2010 @@ -174,6 +174,8 @@ assert not wr_i1() and not wr_guard() def test_compile_bridge(self): + self.cpu.total_compiled_loops = 0 + self.cpu.total_compiled_bridges = 0 i0 = BoxInt() i1 = BoxInt() i2 = BoxInt() @@ -199,7 +201,7 @@ ] bridge[1].setfailargs([i1b]) - self.cpu.compile_bridge(faildescr1, [i1b], bridge) + self.cpu.compile_bridge(faildescr1, [i1b], bridge, looptoken) self.cpu.set_future_value_int(0, 2) fail = self.cpu.execute_token(looptoken) @@ -207,6 +209,9 @@ res = self.cpu.get_latest_value_int(0) assert res == 20 + assert self.cpu.total_compiled_loops == 1 + assert self.cpu.total_compiled_bridges == 1 + def test_compile_bridge_with_holes(self): i0 = BoxInt() i1 = BoxInt() @@ -233,7 +238,7 @@ ] bridge[1].setfailargs([i1b]) - self.cpu.compile_bridge(faildescr1, [i1b], bridge) + self.cpu.compile_bridge(faildescr1, [i1b], bridge, looptoken) self.cpu.set_future_value_int(0, 2) fail = self.cpu.execute_token(looptoken) @@ -1050,7 +1055,7 @@ ResOperation(rop.JUMP, [f3] + fboxes2[1:], None, descr=looptoken), ] - self.cpu.compile_bridge(faildescr1, fboxes2, bridge) + self.cpu.compile_bridge(faildescr1, fboxes2, bridge, looptoken) for i in range(len(fboxes)): self.cpu.set_future_value_float(i, 13.5 + 6.73 * i) Modified: pypy/trunk/pypy/jit/backend/test/test_random.py ============================================================================== --- pypy/trunk/pypy/jit/backend/test/test_random.py (original) +++ pypy/trunk/pypy/jit/backend/test/test_random.py Wed Nov 24 13:15:18 2010 @@ -524,7 +524,7 @@ self.prebuilt_ptr_consts = [] self.r = r self.build_random_loop(cpu, builder_factory, r, startvars) - + def build_random_loop(self, cpu, builder_factory, r, startvars): loop = TreeLoop('test_random_function') @@ -685,11 +685,12 @@ subloop.operations[-1] = jump_op self.guard_op = rl.guard_op self.prebuilt_ptr_consts += rl.prebuilt_ptr_consts + self.loop.token.record_jump_to(rl.loop.token) self.dont_generate_more = True if r.random() < .05: return False self.builder.cpu.compile_bridge(fail_descr, fail_args, - subloop.operations) + subloop.operations, self.loop.token) return True def check_random_function(cpu, BuilderClass, r, num=None, max=None): Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/assembler.py (original) +++ pypy/trunk/pypy/jit/backend/x86/assembler.py Wed Nov 24 13:15:18 2010 @@ -7,6 +7,7 @@ from pypy.rpython.lltypesystem.lloperation import llop from pypy.rpython.annlowlevel import llhelper from pypy.tool.uid import fixid +from pypy.jit.backend.model import CompiledLoopToken from pypy.jit.backend.x86.regalloc import (RegAlloc, X86RegisterManager, X86XMMRegisterManager, get_ebp_ofs, _get_scale) @@ -33,7 +34,6 @@ from pypy.rlib.debug import debug_print, debug_start, debug_stop from pypy.rlib import rgc from pypy.jit.backend.x86.jump import remap_frame_layout -from pypy.rlib.streamio import open_file_as_stream from pypy.jit.metainterp.history import ConstInt, BoxInt # darwin requires the stack to be 16 bytes aligned on calls. Same for gcc 4.5.0, @@ -302,6 +302,8 @@ _x86_arglocs _x86_debug_checksum """ + looptoken.compiled_loop_token = CompiledLoopToken(self.cpu, + looptoken.number) if not we_are_translated(): # Arguments should be unique assert len(set(inputargs)) == len(inputargs) @@ -406,8 +408,9 @@ def _register_counter(self): if self._debug: + # YYY leak -- just put it in self.mc instead struct = lltype.malloc(DEBUG_COUNTER, flavor='raw', - track_allocation=False) # known to leak + track_allocation=False) struct.i = 0 self.loop_run_counters.append((len(self.loop_run_counters), struct)) Modified: pypy/trunk/pypy/jit/backend/x86/regalloc.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/regalloc.py (original) +++ pypy/trunk/pypy/jit/backend/x86/regalloc.py Wed Nov 24 13:15:18 2010 @@ -70,7 +70,7 @@ def _get_new_array(self): n = self.BASE_CONSTANT_SIZE # known to leak - self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n, + self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n, # YYY leak flavor='raw', track_allocation=False) self.cur_array_free = n _get_new_array._dont_inline_ = True Modified: pypy/trunk/pypy/jit/backend/x86/runner.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/runner.py (original) +++ pypy/trunk/pypy/jit/backend/x86/runner.py Wed Nov 24 13:15:18 2010 @@ -53,7 +53,10 @@ self.assembler.assemble_loop(inputargs, operations, looptoken, log=log) - def compile_bridge(self, faildescr, inputargs, operations, log=True): + def compile_bridge(self, faildescr, inputargs, operations, + original_loop_token, log=True): + clt = original_loop_token.compiled_loop_token + clt.compiling_a_bridge() self.assembler.assemble_bridge(faildescr, inputargs, operations, log=log) Modified: pypy/trunk/pypy/jit/backend/x86/test/test_regalloc.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/test/test_regalloc.py (original) +++ pypy/trunk/pypy/jit/backend/x86/test/test_regalloc.py Wed Nov 24 13:15:18 2010 @@ -166,7 +166,8 @@ assert ([box.type for box in bridge.inputargs] == [box.type for box in guard_op.getfailargs()]) faildescr = guard_op.getdescr() - self.cpu.compile_bridge(faildescr, bridge.inputargs, bridge.operations) + self.cpu.compile_bridge(faildescr, bridge.inputargs, bridge.operations, + loop.token) return bridge def run(self, loop): Modified: pypy/trunk/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/trunk/pypy/jit/backend/x86/test/test_runner.py Wed Nov 24 13:15:18 2010 @@ -338,6 +338,7 @@ faildescr1 = BasicFailDescr(1) faildescr2 = BasicFailDescr(2) looptoken = LoopToken() + looptoken.number = 17 class FakeString(object): def __init__(self, val): self.val = val @@ -356,7 +357,7 @@ operations[3].setfailargs([i1]) self.cpu.compile_loop(inputargs, operations, looptoken) name, loopaddress, loopsize = agent.functions[0] - assert name == "Loop # 0: hello" + assert name == "Loop # 17: hello" assert loopaddress <= looptoken._x86_loop_code assert loopsize >= 40 # randomish number @@ -370,7 +371,7 @@ ] bridge[1].setfailargs([i1b]) - self.cpu.compile_bridge(faildescr1, [i1b], bridge) + self.cpu.compile_bridge(faildescr1, [i1b], bridge, looptoken) name, address, size = agent.functions[1] assert name == "Bridge # 0: bye" # Would be exactly ==, but there are some guard failure recovery Modified: pypy/trunk/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/compile.py (original) +++ pypy/trunk/pypy/jit/metainterp/compile.py Wed Nov 24 13:15:18 2010 @@ -1,4 +1,4 @@ - +import weakref from pypy.rpython.lltypesystem import lltype from pypy.rpython.ootypesystem import ootype from pypy.objspace.flow.model import Constant, Variable @@ -48,6 +48,32 @@ loop_token.outermost_jitdriver_sd = jitdriver_sd return loop_token +def record_loop_or_bridge(loop): + """Do post-backend recordings and cleanups on 'loop'. + """ + # get the original loop token (corresponding to 'loop', or if that is + # a bridge, to the loop that this bridge belongs to) + looptoken = loop.token + assert looptoken is not None + wref = weakref.ref(looptoken) + for op in loop.operations: + descr = op.getdescr() + if isinstance(descr, ResumeDescr): + descr.wref_original_loop_token = wref # stick it there + n = descr.index + if n >= 0: # we also record the resumedescr number + looptoken.compiled_loop_token.record_faildescr_index(n) + elif isinstance(descr, LoopToken): + # for a JUMP or a CALL_ASSEMBLER: record it as a potential jump. + # (the following test is not enough to prevent more complicated + # cases of cycles, but at least it helps in simple tests of + # test_memgr.py) + if descr is not looptoken: + looptoken.record_jump_to(descr) + op.setdescr(None) # clear reference, mostly for tests + # mostly for tests: make sure we don't keep a reference to the LoopToken + loop.token = None + # ____________________________________________________________ def compile_new_loop(metainterp, old_loop_tokens, start): @@ -77,6 +103,7 @@ return old_loop_token send_loop_to_backend(metainterp_sd, loop, "loop") insert_loop_token(old_loop_tokens, loop_token) + record_loop_or_bridge(loop) return loop_token def insert_loop_token(old_loop_tokens, loop_token): @@ -117,8 +144,11 @@ else: loop._ignore_during_counting = True metainterp_sd.log("compiled new " + type) + if metainterp_sd.warmrunnerdesc is not None: # for tests + metainterp_sd.warmrunnerdesc.memory_manager.keep_loop_alive(loop.token) -def send_bridge_to_backend(metainterp_sd, faildescr, inputargs, operations): +def send_bridge_to_backend(metainterp_sd, faildescr, inputargs, operations, + original_loop_token): n = metainterp_sd.cpu.get_fail_descr_number(faildescr) metainterp_sd.logger_ops.log_bridge(inputargs, operations, n) if not we_are_translated(): @@ -127,7 +157,8 @@ metainterp_sd.profiler.start_backend() debug_start("jit-backend") try: - metainterp_sd.cpu.compile_bridge(faildescr, inputargs, operations) + metainterp_sd.cpu.compile_bridge(faildescr, inputargs, operations, + original_loop_token) finally: debug_stop("jit-backend") metainterp_sd.profiler.end_backend() @@ -227,9 +258,6 @@ CNT_FLOAT = -0x60000000 CNT_MASK = 0x1FFFFFFF - def __init__(self, metainterp_sd): - self.metainterp_sd = metainterp_sd - def store_final_boxes(self, guard_op, boxes): guard_op.setfailargs(boxes) self.guard_opnum = guard_op.getopnum() @@ -314,12 +342,14 @@ def compile_and_attach(self, metainterp, new_loop): # We managed to create a bridge. Attach the new operations - # to the corrsponding guard_op and compile from there + # to the corresponding guard_op and compile from there + assert metainterp.resumekey_original_loop_token is not None + new_loop.token = metainterp.resumekey_original_loop_token inputargs = metainterp.history.inputargs if not we_are_translated(): self._debug_suboperations = new_loop.operations send_bridge_to_backend(metainterp.staticdata, self, inputargs, - new_loop.operations) + new_loop.operations, new_loop.token) def copy_all_attrbutes_into(self, res): # XXX a bit ugly to have to list them all here @@ -331,14 +361,14 @@ res.rd_pendingfields = self.rd_pendingfields def _clone_if_mutable(self): - res = ResumeGuardDescr(self.metainterp_sd) + res = ResumeGuardDescr() self.copy_all_attrbutes_into(res) return res class ResumeGuardForcedDescr(ResumeGuardDescr): def __init__(self, metainterp_sd, jitdriver_sd): - ResumeGuardDescr.__init__(self, metainterp_sd) + self.metainterp_sd = metainterp_sd self.jitdriver_sd = jitdriver_sd def handle_fail(self, metainterp_sd, jitdriver_sd): @@ -482,6 +512,8 @@ metainterp_sd = metainterp.staticdata jitdriver_sd = metainterp.jitdriver_sd redargs = new_loop.inputargs + # We make a new LoopToken for this entry bridge, and stick it + # to every guard in the loop. new_loop_token = make_loop_token(len(redargs), jitdriver_sd) new_loop.token = new_loop_token send_loop_to_backend(metainterp_sd, new_loop, "entry bridge") @@ -489,12 +521,14 @@ jitdriver_sd.warmstate.attach_unoptimized_bridge_from_interp( self.original_greenkey, new_loop_token) - # store the new loop in compiled_merge_points too + # store the new loop in compiled_merge_points_wref too old_loop_tokens = metainterp.get_compiled_merge_points( self.original_greenkey) # it always goes at the end of the list, as it is the most # general loop token old_loop_tokens.append(new_loop_token) + metainterp.set_compiled_merge_points(self.original_greenkey, + old_loop_tokens) def reset_counter_from_failure(self): pass @@ -529,6 +563,7 @@ # know exactly what we must do (ResumeGuardDescr/ResumeFromInterpDescr) prepare_last_operation(new_loop, target_loop_token) resumekey.compile_and_attach(metainterp, new_loop) + record_loop_or_bridge(new_loop) return target_loop_token def prepare_last_operation(new_loop, target_loop_token): @@ -555,7 +590,8 @@ propagate_exception_descr = PropagateExceptionDescr() -def compile_tmp_callback(cpu, jitdriver_sd, greenboxes, redboxes): +def compile_tmp_callback(cpu, jitdriver_sd, greenboxes, redboxes, + memory_manager=None): """Make a LoopToken that corresponds to assembler code that just calls back the interpreter. Used temporarily: a fully compiled version of the code may end up replacing it. @@ -595,4 +631,6 @@ ] operations[1].setfailargs([]) cpu.compile_loop(inputargs, operations, loop_token, log=False) + if memory_manager is not None: # for tests + memory_manager.keep_loop_alive(loop_token) return loop_token Modified: pypy/trunk/pypy/jit/metainterp/history.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/history.py (original) +++ pypy/trunk/pypy/jit/metainterp/history.py Wed Nov 24 13:15:18 2010 @@ -4,7 +4,7 @@ from pypy.rpython.ootypesystem import ootype from pypy.rlib.objectmodel import we_are_translated, r_dict, Symbolic from pypy.rlib.objectmodel import compute_hash, compute_unique_id -from pypy.rlib.rarithmetic import intmask +from pypy.rlib.rarithmetic import intmask, r_longlong from pypy.tool.uid import uid from pypy.conftest import option @@ -730,14 +730,28 @@ outermost_jitdriver_sd = None # specnodes = ... # and more data specified by the backend when the loop is compiled - number = 0 + number = -1 + generation = r_longlong(0) + # one purpose of LoopToken is to keep alive the CompiledLoopToken + # returned by the backend. When the LoopToken goes away, the + # CompiledLoopToken has its __del__ called, which frees the assembler + # memory and the ResumeGuards. + compiled_loop_token = None - def __init__(self, number=0): - self.number = number + def __init__(self): + # For memory management of assembled loops + self._keepalive_target_looktokens = {} # set of other LoopTokens + + def record_jump_to(self, target_loop_token): + self._keepalive_target_looktokens[target_loop_token] = None + + def __repr__(self): + return '' % (self.number, self.generation) def repr_of_descr(self): return '' % self.number + class TreeLoop(object): inputargs = None operations = None Modified: pypy/trunk/pypy/jit/metainterp/jitprof.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/jitprof.py (original) +++ pypy/trunk/pypy/jit/metainterp/jitprof.py Wed Nov 24 13:15:18 2010 @@ -24,6 +24,10 @@ NVIRTUALS NVHOLES NVREUSED +TOTAL_COMPILED_LOOPS +TOTAL_COMPILED_BRIDGES +TOTAL_FREED_LOOPS +TOTAL_FREED_BRIDGES """ def _setup(): @@ -35,6 +39,7 @@ _setup() JITPROF_LINES = ncounters + 1 + 1 # one for TOTAL, 1 for calls, update if needed +_CPU_LINES = 4 # the last 4 lines are stored on the cpu class BaseProfiler(object): pass @@ -87,12 +92,13 @@ counters = None calls = 0 current = None + cpu = None def start(self): self.starttime = self.timer() self.t1 = self.starttime self.times = [0, 0] - self.counters = [0] * ncounters + self.counters = [0] * (ncounters - _CPU_LINES) self.calls = 0 self.current = [] @@ -174,6 +180,16 @@ self._print_intline("nvirtuals", cnt[NVIRTUALS]) self._print_intline("nvholes", cnt[NVHOLES]) self._print_intline("nvreused", cnt[NVREUSED]) + cpu = self.cpu + if cpu is not None: # for some tests + self._print_intline("Total # of loops", + cpu.total_compiled_loops) + self._print_intline("Total # of bridges", + cpu.total_compiled_bridges) + self._print_intline("Freed # of loops", + cpu.total_freed_loops) + self._print_intline("Freed # of bridges", + cpu.total_freed_bridges) def _print_line_time(self, string, i, tim): final = "%s:%s\t%d\t%f" % (string, " " * max(0, 13-len(string)), i, tim) Modified: pypy/trunk/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/trunk/pypy/jit/metainterp/pyjitpl.py Wed Nov 24 13:15:18 2010 @@ -14,7 +14,7 @@ from pypy.jit.metainterp.logger import Logger from pypy.jit.metainterp.jitprof import EmptyProfiler from pypy.jit.metainterp.jitprof import GUARDS, RECORDED_OPS, ABORT_ESCAPE -from pypy.jit.metainterp.jitprof import ABORT_TOO_LONG +from pypy.jit.metainterp.jitprof import ABORT_TOO_LONG, ABORT_BRIDGE from pypy.jit.metainterp.jitexc import JitException, get_llexception from pypy.rlib.rarithmetic import intmask from pypy.rlib.objectmodel import specialize @@ -1068,7 +1068,7 @@ resumedescr = compile.ResumeGuardForcedDescr(metainterp_sd, metainterp.jitdriver_sd) else: - resumedescr = compile.ResumeGuardDescr(metainterp_sd) + resumedescr = compile.ResumeGuardDescr() guard_op = metainterp.history.record(opnum, moreargs, None, descr=resumedescr) virtualizable_boxes = None @@ -1222,6 +1222,7 @@ self.logger_ops = Logger(self, guard_number=True) self.profiler = ProfilerClass() + self.profiler.cpu = cpu self.warmrunnerdesc = warmrunnerdesc backendmodule = self.cpu.__module__ @@ -1340,6 +1341,11 @@ return jitcode return None + def try_to_free_some_loops(self): + # Increase here the generation recorded by the memory manager. + if self.warmrunnerdesc is not None: # for tests + self.warmrunnerdesc.memory_manager.next_generation() + # ---------------- logging ------------------------ def log(self, msg): @@ -1636,6 +1642,7 @@ self.staticdata._setup_once() self.staticdata.profiler.start_tracing() assert jitdriver_sd is self.jitdriver_sd + self.staticdata.try_to_free_some_loops() self.create_empty_history() try: original_boxes = self.initialize_original_boxes(jitdriver_sd, *args) @@ -1664,10 +1671,15 @@ debug_start('jit-tracing') self.staticdata.profiler.start_tracing() assert isinstance(key, compile.ResumeGuardDescr) + # store the resumekey.wref_original_loop_token() on 'self' to make + # sure that it stays alive as long as this MetaInterp + self.resumekey_original_loop_token = key.wref_original_loop_token() + self.staticdata.try_to_free_some_loops() self.initialize_state_from_guard_failure(key) try: return self._handle_guard_failure(key) finally: + self.resumekey_original_loop_token = None self.staticdata.profiler.end_tracing() debug_stop('jit-tracing') @@ -1677,6 +1689,8 @@ self.seen_loop_header_for_jdindex = -1 try: self.prepare_resume_from_failure(key.guard_opnum) + if self.resumekey_original_loop_token is None: # very rare case + raise SwitchToBlackhole(ABORT_BRIDGE) self.interpret() except GenerateMergePoint, gmp: return self.designate_target_loop(gmp) @@ -1798,10 +1812,15 @@ raise NotImplementedError(opname[opnum]) def get_compiled_merge_points(self, greenkey): + """Get the list of looptokens corresponding to the greenkey. + Turns the (internal) list of weakrefs into regular refs. + """ + cell = self.jitdriver_sd.warmstate.jit_cell_at_key(greenkey) + return cell.get_compiled_merge_points() + + def set_compiled_merge_points(self, greenkey, looptokens): cell = self.jitdriver_sd.warmstate.jit_cell_at_key(greenkey) - if cell.compiled_merge_points is None: - cell.compiled_merge_points = [] - return cell.compiled_merge_points + cell.set_compiled_merge_points(looptokens) def compile(self, original_boxes, live_arg_boxes, start): num_green_args = self.jitdriver_sd.num_green_args @@ -1811,6 +1830,7 @@ self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None) loop_token = compile.compile_new_loop(self, old_loop_tokens, start) if loop_token is not None: # raise if it *worked* correctly + self.set_compiled_merge_points(greenkey, old_loop_tokens) raise GenerateMergePoint(live_arg_boxes, loop_token) self.history.operations.pop() # remove the JUMP Modified: pypy/trunk/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_basic.py Wed Nov 24 13:15:18 2010 @@ -19,7 +19,11 @@ from pypy.jit.metainterp import simple_optimize class FakeJitCell: - compiled_merge_points = None + __compiled_merge_points = [] + def get_compiled_merge_points(self): + return self.__compiled_merge_points[:] + def set_compiled_merge_points(self, lst): + self.__compiled_merge_points = lst class FakeWarmRunnerState: def attach_unoptimized_bridge_from_interp(self, greenkey, newloop): Modified: pypy/trunk/pypy/jit/metainterp/test/test_compile.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_compile.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_compile.py Wed Nov 24 13:15:18 2010 @@ -53,6 +53,7 @@ stats = Stats() profiler = jitprof.EmptyProfiler() + warmrunnerdesc = None def log(self, msg, event_kind=None): pass @@ -206,14 +207,12 @@ class ExitFrameWithExceptionRef(Exception): pass FakeMetaInterpSD.cpu = cpu - class FakeJitDriverSD: - pass cpu.set_future_value_int(0, -156) cpu.set_future_value_int(1, -178) cpu.set_future_value_int(2, -190) fail_descr = cpu.execute_token(loop_token) try: - fail_descr.handle_fail(FakeMetaInterpSD(), FakeJitDriverSD()) + fail_descr.handle_fail(FakeMetaInterpSD(), None) except FakeMetaInterpSD.ExitFrameWithExceptionRef, e: assert lltype.cast_opaque_ptr(lltype.Ptr(EXC), e.args[1]) == llexc else: Modified: pypy/trunk/pypy/jit/metainterp/test/test_logger.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_logger.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_logger.py Wed Nov 24 13:15:18 2010 @@ -14,11 +14,20 @@ def capturing(func, *args, **kwds): log_stream = StringIO() - debug._stderr = log_stream + class MyDebugLog: + def debug_print(self, *args): + for arg in args: + print >> log_stream, arg, + print >> log_stream + def debug_start(self, *args): + pass + def debug_stop(self, *args): + pass try: + debug._log = MyDebugLog() func(*args, **kwds) finally: - debug._stderr = sys.stderr + debug._log = None return log_stream.getvalue() class Logger(logger.Logger): @@ -112,7 +121,8 @@ equaloplists(loop.operations, oloop.operations) def test_jump(self): - namespace = {'target': LoopToken(3)} + namespace = {'target': LoopToken()} + namespace['target'].number = 3 inp = ''' [i0] jump(i0, descr=target) Modified: pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py Wed Nov 24 13:15:18 2010 @@ -41,7 +41,7 @@ b1 = BoxInt() opt = optimizeopt.Optimizer(FakeMetaInterpStaticData(LLtypeMixin.cpu), None) - fdescr = ResumeGuardDescr(None) + fdescr = ResumeGuardDescr() op = ResOperation(rop.GUARD_TRUE, ['dummy'], None, descr=fdescr) # setup rd data fi0 = resume.FrameInfo(None, "code0", 11) Modified: pypy/trunk/pypy/jit/metainterp/test/test_pyjitpl.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_pyjitpl.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_pyjitpl.py Wed Nov 24 13:15:18 2010 @@ -17,6 +17,7 @@ portal.setup(None) class FakeStaticData: cpu = None + warmrunnerdesc = None metainterp = pyjitpl.MetaInterp(FakeStaticData(), None) metainterp.framestack = [] @@ -53,6 +54,7 @@ def test_remove_consts_and_duplicates(): class FakeStaticData: cpu = None + warmrunnerdesc = None def is_another_box_like(box, referencebox): assert box is not referencebox assert isinstance(box, referencebox.clonebox().__class__) Modified: pypy/trunk/pypy/jit/metainterp/test/test_warmspot.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_warmspot.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_warmspot.py Wed Nov 24 13:15:18 2010 @@ -293,23 +293,30 @@ exc_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True) cls.exc_vtable = exc_vtable - class FakeFailDescr(object): + class FakeLoopToken: def __init__(self, no): self.no = no + self.generation = 0 + + class FakeFailDescr(object): + def __init__(self, looptoken): + assert isinstance(looptoken, FakeLoopToken) + self.looptoken = looptoken def handle_fail(self, metainterp_sd, jitdrivers_sd): - if self.no == 0: + no = self.looptoken.no + if no == 0: raise metainterp_sd.warmrunnerdesc.DoneWithThisFrameInt(3) - if self.no == 1: + if no == 1: raise metainterp_sd.warmrunnerdesc.ContinueRunningNormally( [0], [], [], [1], [], []) - if self.no == 3: + if no == 3: exc = lltype.malloc(OBJECT) exc.typeptr = exc_vtable raise metainterp_sd.warmrunnerdesc.ExitFrameWithExceptionRef( metainterp_sd.cpu, lltype.cast_opaque_ptr(llmemory.GCREF, exc)) - return self.no + return self.looptoken class FakeDescr: def as_vtable_size_descr(self): @@ -334,11 +341,11 @@ sizeof = nodescr def get_fail_descr_from_number(self, no): - return FakeFailDescr(no) + return FakeFailDescr(FakeLoopToken(no)) def execute_token(self, token): - assert token == 2 - return FakeFailDescr(1) + assert token.no == 2 + return FakeFailDescr(FakeLoopToken(1)) driver = JitDriver(reds = ['red'], greens = ['green']) Modified: pypy/trunk/pypy/jit/metainterp/test/test_warmstate.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_warmstate.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_warmstate.py Wed Nov 24 13:15:18 2010 @@ -99,6 +99,8 @@ lltype.Float], lltype.Void)) class FakeWarmRunnerDesc: rtyper = FakeRTyper() + cpu = None + memory_manager = None class FakeJitDriverSD: _get_jitcell_at_ptr = llhelper(GETTER, getter) _set_jitcell_at_ptr = llhelper(SETTER, setter) @@ -126,6 +128,7 @@ future_values[j] = "float", value class FakeWarmRunnerDesc: cpu = FakeCPU() + memory_manager = None class FakeJitDriverSD: _red_args_types = ["int", "float"] virtualizable_info = None @@ -154,16 +157,20 @@ _get_jitcell_at_ptr = None state = WarmEnterState(None, FakeJitDriverSD()) get_jitcell = state.make_jitcell_getter() + class FakeLoopToken(object): + pass + looptoken = FakeLoopToken() state.attach_unoptimized_bridge_from_interp([ConstInt(5), ConstFloat(2.25)], - "entry loop token") + looptoken) cell1 = get_jitcell(True, 5, 2.25) assert cell1.counter < 0 - assert cell1.entry_loop_token == "entry loop token" + assert cell1.get_entry_loop_token() is looptoken def test_make_jitdriver_callbacks_1(): class FakeWarmRunnerDesc: cpu = None + memory_manager = None class FakeJitDriverSD: _green_args_spec = [lltype.Signed, lltype.Float] _get_printable_location_ptr = None @@ -189,6 +196,7 @@ class FakeWarmRunnerDesc: rtyper = None cpu = None + memory_manager = None class FakeJitDriverSD: _green_args_spec = [lltype.Signed, lltype.Float] _get_printable_location_ptr = llhelper(GET_LOCATION, get_location) @@ -211,6 +219,7 @@ class FakeWarmRunnerDesc: rtyper = None cpu = None + memory_manager = None class FakeJitDriverSD: _green_args_spec = [lltype.Signed, lltype.Float] _get_printable_location_ptr = None @@ -233,6 +242,7 @@ class FakeWarmRunnerDesc: rtyper = None cpu = None + memory_manager = None class FakeJitDriverSD: _green_args_spec = [lltype.Signed, lltype.Float] _get_printable_location_ptr = None Modified: pypy/trunk/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/warmspot.py (original) +++ pypy/trunk/pypy/jit/metainterp/warmspot.py Wed Nov 24 13:15:18 2010 @@ -12,11 +12,12 @@ from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.rarithmetic import r_uint, intmask from pypy.rlib.debug import debug_print, fatalerror +from pypy.rlib.debug import debug_start, debug_stop from pypy.rpython.lltypesystem.lloperation import llop from pypy.translator.simplify import get_funcobj, get_functype from pypy.translator.unsimplify import call_final_function -from pypy.jit.metainterp import history, pyjitpl, gc +from pypy.jit.metainterp import history, pyjitpl, gc, memmgr from pypy.jit.metainterp.pyjitpl import MetaInterpStaticData, MetaInterp from pypy.jit.metainterp.typesystem import LLTypeHelper, OOTypeHelper from pypy.jit.metainterp.jitprof import Profiler, EmptyProfiler @@ -61,7 +62,7 @@ def jittify_and_run(interp, graph, args, repeat=1, backendopt=False, trace_limit=sys.maxint, - inline=False, **kwds): + inline=False, loop_longevity=0, **kwds): from pypy.config.config import ConfigError translator = interp.typer.annotator.translator try: @@ -78,6 +79,7 @@ jd.warmstate.set_param_trace_eagerness(2) # for tests jd.warmstate.set_param_trace_limit(trace_limit) jd.warmstate.set_param_inlining(inline) + jd.warmstate.set_param_loop_longevity(loop_longevity) warmrunnerdesc.finish() res = interp.eval_graph(graph, args) if not kwds.get('translate_support_code', False): @@ -145,6 +147,7 @@ optimizer=None, ProfilerClass=EmptyProfiler, **kwds): pyjitpl._warmrunnerdesc = self # this is a global for debugging only! self.set_translator(translator) + self.memory_manager = memmgr.MemoryManager() self.build_cpu(CPUClass, **kwds) self.find_portals() self.codewriter = codewriter.CodeWriter(self.cpu, self.jitdrivers_sd) @@ -707,7 +710,7 @@ loop_token = fail_descr.handle_fail(self.metainterp_sd, jd) except JitException, e: return handle_jitexception(e) - fail_descr = self.cpu.execute_token(loop_token) + fail_descr = self.execute_token(loop_token) jd._assembler_call_helper = assembler_call_helper # for debugging jd._assembler_helper_ptr = self.helper_func( @@ -801,3 +804,10 @@ py.test.skip("rewrite_force_virtual: port it to ootype") all_graphs = self.translator.graphs vrefinfo.replace_force_virtual_with_call(all_graphs) + + # ____________________________________________________________ + + def execute_token(self, loop_token): + fail_descr = self.cpu.execute_token(loop_token) + self.memory_manager.keep_loop_alive(loop_token) + return fail_descr Modified: pypy/trunk/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/warmstate.py (original) +++ pypy/trunk/pypy/jit/metainterp/warmstate.py Wed Nov 24 13:15:18 2010 @@ -1,4 +1,4 @@ -import sys +import sys, weakref from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi from pypy.rpython.ootypesystem import ootype from pypy.rpython.annlowlevel import hlstr, llstr, cast_base_ptr_to_instance @@ -149,9 +149,34 @@ # counter == -1: there is an entry bridge for this cell # counter == -2: tracing is currently going on for this cell counter = 0 - compiled_merge_points = None + compiled_merge_points_wref = None # list of weakrefs to LoopToken dont_trace_here = False - entry_loop_token = None + wref_entry_loop_token = None # (possibly) one weakref to LoopToken + + def get_compiled_merge_points(self): + result = [] + if self.compiled_merge_points_wref is not None: + for wref in self.compiled_merge_points_wref: + looptoken = wref() + if looptoken is not None: + result.append(looptoken) + return result + + def set_compiled_merge_points(self, looptokens): + self.compiled_merge_points_wref = [self._makeref(token) + for token in looptokens] + + def get_entry_loop_token(self): + if self.wref_entry_loop_token is not None: + return self.wref_entry_loop_token() + return None + + def set_entry_loop_token(self, looptoken): + self.wref_entry_loop_token = self._makeref(looptoken) + + def _makeref(self, looptoken): + assert looptoken is not None + return weakref.ref(looptoken) # ____________________________________________________________ @@ -164,6 +189,8 @@ "NOT_RPYTHON" self.warmrunnerdesc = warmrunnerdesc self.jitdriver_sd = jitdriver_sd + if warmrunnerdesc is not None: # for tests + self.cpu = warmrunnerdesc.cpu try: self.profiler = warmrunnerdesc.metainterp_sd.profiler except AttributeError: # for tests @@ -208,6 +235,12 @@ else: raise ValueError("unknown optimizer") + def set_param_loop_longevity(self, value): + # note: it's a global parameter, not a per-jitdriver one + if (self.warmrunnerdesc is not None and + self.warmrunnerdesc.memory_manager is not None): # all for tests + self.warmrunnerdesc.memory_manager.set_max_age(value) + def disable_noninlinable_function(self, greenkey): cell = self.jit_cell_at_key(greenkey) cell.dont_trace_here = True @@ -219,12 +252,15 @@ def attach_unoptimized_bridge_from_interp(self, greenkey, entry_loop_token): cell = self.jit_cell_at_key(greenkey) - cell.counter = -1 - old_token = cell.entry_loop_token - cell.entry_loop_token = entry_loop_token + old_token = cell.get_entry_loop_token() + cell.set_entry_loop_token(entry_loop_token) + cell.counter = -1 # valid entry bridge attached if old_token is not None: - cpu = self.warmrunnerdesc.cpu - cpu.redirect_call_assembler(old_token, entry_loop_token) + self.cpu.redirect_call_assembler(old_token, entry_loop_token) + # entry_loop_token is also kept alive by any loop that used + # to point to old_token. Actually freeing old_token early + # is a pointless optimization (it is tiny). + old_token.record_jump_to(entry_loop_token) # ---------- @@ -233,7 +269,8 @@ if hasattr(self, 'maybe_compile_and_run'): return self.maybe_compile_and_run - metainterp_sd = self.warmrunnerdesc.metainterp_sd + warmrunnerdesc = self.warmrunnerdesc + metainterp_sd = warmrunnerdesc.metainterp_sd jitdriver_sd = self.jitdriver_sd vinfo = jitdriver_sd.virtualizable_info index_of_virtualizable = jitdriver_sd.index_of_virtualizable @@ -291,23 +328,27 @@ assert cell.counter == -1 if not confirm_enter_jit(*args): return + loop_token = cell.get_entry_loop_token() + if loop_token is None: # it was a weakref that has been freed + cell.counter = 0 + return # machine code was already compiled for these greenargs # get the assembler and fill in the boxes set_future_values(*args[num_green_args:]) - loop_token = cell.entry_loop_token # ---------- execute assembler ---------- while True: # until interrupted by an exception metainterp_sd.profiler.start_running() debug_start("jit-running") - fail_descr = metainterp_sd.cpu.execute_token(loop_token) + fail_descr = warmrunnerdesc.execute_token(loop_token) debug_stop("jit-running") metainterp_sd.profiler.end_running() + loop_token = None # for test_memmgr if vinfo is not None: vinfo.reset_vable_token(virtualizable) loop_token = fail_descr.handle_fail(metainterp_sd, jitdriver_sd) - + maybe_compile_and_run._dont_inline_ = True self.maybe_compile_and_run = maybe_compile_and_run return maybe_compile_and_run @@ -453,7 +494,7 @@ warmrunnerdesc = self.warmrunnerdesc jitdriver_sd = self.jitdriver_sd - cpu = warmrunnerdesc.cpu + cpu = self.cpu vinfo = jitdriver_sd.virtualizable_info red_args_types = unrolling_iterable(jitdriver_sd._red_args_types) # @@ -502,10 +543,11 @@ if hasattr(self, 'get_location_str'): return # + warmrunnerdesc = self.warmrunnerdesc unwrap_greenkey = self.make_unwrap_greenkey() jit_getter = self.make_jitcell_getter() jd = self.jitdriver_sd - cpu = self.warmrunnerdesc.cpu + cpu = self.cpu def can_inline_greenargs(*greenargs): if can_never_inline(*greenargs): @@ -523,11 +565,16 @@ def get_assembler_token(greenkey, redboxes): # 'redboxes' is only used to know the types of red arguments cell = self.jit_cell_at_key(greenkey) - if cell.entry_loop_token is None: + entry_loop_token = cell.get_entry_loop_token() + if entry_loop_token is None: from pypy.jit.metainterp.compile import compile_tmp_callback - cell.entry_loop_token = compile_tmp_callback(cpu, jd, greenkey, - redboxes) - return cell.entry_loop_token + if cell.counter == -1: # used to be a valid entry bridge, + cell.counter = 0 # but was freed in the meantime. + memmgr = warmrunnerdesc.memory_manager + entry_loop_token = compile_tmp_callback(cpu, jd, greenkey, + redboxes, memmgr) + cell.set_entry_loop_token(entry_loop_token) + return entry_loop_token self.get_assembler_token = get_assembler_token # Modified: pypy/trunk/pypy/jit/tool/jitoutput.py ============================================================================== --- pypy/trunk/pypy/jit/tool/jitoutput.py (original) +++ pypy/trunk/pypy/jit/tool/jitoutput.py Wed Nov 24 13:15:18 2010 @@ -27,6 +27,10 @@ (('nvirtuals',), '^nvirtuals:\s+(\d+)$'), (('nvholes',), '^nvholes:\s+(\d+)$'), (('nvreused',), '^nvreused:\s+(\d+)$'), + (('total_compiled_loops',), '^Total # of loops:\s+(\d+)$'), + (('total_compiled_bridges',), '^Total # of bridges:\s+(\d+)$'), + (('total_freed_loops',), '^Freed # of loops:\s+(\d+)$'), + (('total_freed_bridges',), '^Freed # of bridges:\s+(\d+)$'), ] class Ops(object): Modified: pypy/trunk/pypy/jit/tool/test/test_jitoutput.py ============================================================================== --- pypy/trunk/pypy/jit/tool/test/test_jitoutput.py (original) +++ pypy/trunk/pypy/jit/tool/test/test_jitoutput.py Wed Nov 24 13:15:18 2010 @@ -63,6 +63,10 @@ nvirtuals: 13 nvholes: 14 nvreused: 15 +Total # of loops: 100 +Total # of bridges: 300 +Freed # of loops: 99 +Freed # of bridges: 299 ''' def test_parse(): Modified: pypy/trunk/pypy/module/__pypy__/__init__.py ============================================================================== --- pypy/trunk/pypy/module/__pypy__/__init__.py (original) +++ pypy/trunk/pypy/module/__pypy__/__init__.py Wed Nov 24 13:15:18 2010 @@ -11,6 +11,10 @@ 'internal_repr' : 'interp_magic.internal_repr', 'bytebuffer' : 'bytebuffer.bytebuffer', 'identity_dict' : 'interp_identitydict.W_IdentityDict', + 'debug_start' : 'interp_debug.debug_start', + 'debug_print' : 'interp_debug.debug_print', + 'debug_stop' : 'interp_debug.debug_stop', + 'debug_print_once' : 'interp_debug.debug_print_once', } def setup_after_space_initialization(self): Modified: pypy/trunk/pypy/module/_pickle_support/maker.py ============================================================================== --- pypy/trunk/pypy/module/_pickle_support/maker.py (original) +++ pypy/trunk/pypy/module/_pickle_support/maker.py Wed Nov 24 13:15:18 2010 @@ -67,11 +67,12 @@ return space.wrap(tb) traceback_new.unwrap_spec = [ObjSpace] -def generator_new(space, frame, running): +def generator_new(space, w_frame, running): + frame = space.interp_w(PyFrame, w_frame, can_be_None=True) new_generator = GeneratorIterator(frame) new_generator.running = running return space.wrap(new_generator) -generator_new.unwrap_spec = [ObjSpace, PyFrame, int] +generator_new.unwrap_spec = [ObjSpace, W_Root, int] def xrangeiter_new(space, current, remaining, step): from pypy.module.__builtin__.functional import W_XRangeIterator Modified: pypy/trunk/pypy/rlib/debug.py ============================================================================== --- pypy/trunk/pypy/rlib/debug.py (original) +++ pypy/trunk/pypy/rlib/debug.py Wed Nov 24 13:15:18 2010 @@ -53,13 +53,11 @@ _log = None # patched from tests to be an object of class DebugLog # or compatible -_stderr = sys.stderr # alternatively, this is patched from tests - # (redirects debug_print(), but not debug_start/stop) def debug_print(*args): for arg in args: - print >> _stderr, arg, - print >> _stderr + print >> sys.stderr, arg, + print >> sys.stderr if _log is not None: _log.debug_print(*args) Modified: pypy/trunk/pypy/rlib/jit.py ============================================================================== --- pypy/trunk/pypy/rlib/jit.py (original) +++ pypy/trunk/pypy/rlib/jit.py Wed Nov 24 13:15:18 2010 @@ -265,6 +265,7 @@ 'trace_limit': 10000, 'inlining': False, 'optimizer': OPTIMIZER_FULL, + 'loop_longevity': 1000, } unroll_parameters = unrolling_iterable(PARAMETERS.keys()) Modified: pypy/trunk/pypy/rpython/memory/gc/base.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/base.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/base.py Wed Nov 24 13:15:18 2010 @@ -18,6 +18,7 @@ malloc_zero_filled = False prebuilt_gc_objects_are_static_roots = True object_minimal_size = 0 + gcflag_extra = 0 # or a real GC flag that is always 0 when not collecting def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, translated_to_c=True): Modified: pypy/trunk/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/inspector.py Wed Nov 24 13:15:18 2010 @@ -107,15 +107,18 @@ def __init__(self, gc, fd): self.gc = gc + self.gcflag = gc.gcflag_extra self.fd = rffi.cast(rffi.INT, fd) self.writebuffer = lltype.malloc(rffi.LONGP.TO, self.BUFSIZE, flavor='raw') self.buf_count = 0 - self.seen = AddressDict() + if self.gcflag == 0: + self.seen = AddressDict() self.pending = AddressStack() def delete(self): - self.seen.delete() + if self.gcflag == 0: + self.seen.delete() self.pending.delete() lltype.free(self.writebuffer, flavor='raw') free_non_gc_object(self) @@ -140,6 +143,8 @@ self.flush() write._always_inline_ = True + # ---------- + def write_marker(self): self.write(0) self.write(0) @@ -161,9 +166,15 @@ self.add(obj) def add(self, obj): - if not self.seen.contains(obj): - self.seen.setitem(obj, obj) - self.pending.append(obj) + if self.gcflag == 0: + if not self.seen.contains(obj): + self.seen.setitem(obj, obj) + self.pending.append(obj) + else: + hdr = self.gc.header(obj) + if (hdr.tid & self.gcflag) == 0: + hdr.tid |= self.gcflag + self.pending.append(obj) def add_roots(self): self.gc.enumerate_all_roots(_hd_add_root, self) @@ -177,14 +188,50 @@ while pending.non_empty(): self.writeobj(pending.pop()) + # ---------- + # A simplified copy of the above, to make sure we walk again all the + # objects to clear the 'gcflag'. + + def unwriteobj(self, obj): + gc = self.gc + gc.trace(obj, self._unwriteref, None) + + def _unwriteref(self, pointer, _): + obj = pointer.address[0] + self.unadd(obj) + + def unadd(self, obj): + assert self.gcflag != 0 + hdr = self.gc.header(obj) + if (hdr.tid & self.gcflag) != 0: + hdr.tid &= ~self.gcflag + self.pending.append(obj) + + def clear_gcflag_again(self): + self.gc.enumerate_all_roots(_hd_unadd_root, self) + pendingroots = self.pending + self.pending = AddressStack() + self.unwalk(pendingroots) + pendingroots.delete() + + def unwalk(self, pending): + while pending.non_empty(): + self.unwriteobj(pending.pop()) + def _hd_add_root(obj, heap_dumper): heap_dumper.add(obj) +def _hd_unadd_root(obj, heap_dumper): + heap_dumper.unadd(obj) + def dump_rpy_heap(gc, fd): heapdumper = HeapDumper(gc, fd) heapdumper.add_roots() heapdumper.walk(heapdumper.pending) heapdumper.flush() + if heapdumper.gcflag != 0: + heapdumper.clear_gcflag_again() + heapdumper.unwalk(heapdumper.pending) heapdumper.delete() return True Modified: pypy/trunk/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/minimark.py Wed Nov 24 13:15:18 2010 @@ -26,7 +26,7 @@ to more than PYPY_GC_MAX_DELTA the amount really used after a collection. Defaults to 1/8th of the total RAM size (which is constrained to be at most - 4GB on 32-bit systems). Try values like '200MB'. + 2/3/4GB on 32-bit systems). Try values like '200MB'. PYPY_GC_MIN Don't collect while the memory size is below this limit. Useful to avoid spending all the time in @@ -102,6 +102,7 @@ needs_write_barrier = True prebuilt_gc_objects_are_static_roots = False malloc_zero_filled = True # xxx experiment with False + gcflag_extra = GCFLAG_FINALIZATION_ORDERING # All objects start with a HDR, i.e. with a field 'tid' which contains # a word. This word is divided in two halves: the lower half contains Modified: pypy/trunk/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/trunk/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/trunk/pypy/rpython/memory/gc/semispace.py Wed Nov 24 13:15:18 2010 @@ -42,6 +42,7 @@ inline_simple_malloc_varsize = True malloc_zero_filled = True first_unused_gcflag = first_gcflag << 5 + gcflag_extra = GCFLAG_FINALIZATION_ORDERING HDR = lltype.Struct('header', ('tid', lltype.Signed)) # XXX or rffi.INT? typeid_is_in_field = 'tid' Modified: pypy/trunk/pypy/translator/c/funcgen.py ============================================================================== --- pypy/trunk/pypy/translator/c/funcgen.py (original) +++ pypy/trunk/pypy/translator/c/funcgen.py Wed Nov 24 13:15:18 2010 @@ -1,3 +1,4 @@ +import sys from pypy.translator.c.support import USESLOTS # set to False if necessary while refactoring from pypy.translator.c.support import cdecl from pypy.translator.c.support import llvalue_from_constant, gen_assignments @@ -757,6 +758,16 @@ format.append('%s') argv.append('(%s) ? "True" : "False"' % self.expr(arg)) continue + elif T == SignedLongLong: + if sys.platform == 'win32': + format.append('%I64d') + else: + format.append('%lld') + elif T == UnsignedLongLong: + if sys.platform == 'win32': + format.append('%I64u') + else: + format.append('%llu') else: raise Exception("don't know how to debug_print %r" % (T,)) argv.append(self.expr(arg)) @@ -765,17 +776,20 @@ "if (PYPY_HAVE_DEBUG_PRINTS) { fprintf(PYPY_DEBUG_FILE, %s); %s}" % (', '.join(argv), free_line)) + def _op_debug(self, opname, arg): + if isinstance(arg, Constant): + string_literal = c_string_constant(''.join(arg.value.chars)) + return "%s(%s);" % (opname, string_literal) + else: + x = "%s(RPyString_AsCharP(%s));\n" % (opname, self.expr(arg)) + x += "RPyString_FreeCache();" + return x + def OP_DEBUG_START(self, op): - arg = op.args[0] - assert isinstance(arg, Constant) - return "PYPY_DEBUG_START(%s);" % ( - c_string_constant(''.join(arg.value.chars)),) + return self._op_debug('PYPY_DEBUG_START', op.args[0]) def OP_DEBUG_STOP(self, op): - arg = op.args[0] - assert isinstance(arg, Constant) - return "PYPY_DEBUG_STOP(%s);" % ( - c_string_constant(''.join(arg.value.chars)),) + return self._op_debug('PYPY_DEBUG_STOP', op.args[0]) def OP_DEBUG_ASSERT(self, op): return 'RPyAssert(%s, %s);' % (self.expr(op.args[0]), Modified: pypy/trunk/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/trunk/pypy/translator/c/test/test_newgc.py (original) +++ pypy/trunk/pypy/translator/c/test/test_newgc.py Wed Nov 24 13:15:18 2010 @@ -1064,14 +1064,16 @@ def test_get_rpy_type_index(self): self.run("get_rpy_type_index") - filename_dump = str(udir.join('test_dump_rpy_heap')) + filename1_dump = str(udir.join('test_dump_rpy_heap.1')) + filename2_dump = str(udir.join('test_dump_rpy_heap.2')) def define_dump_rpy_heap(self): U = lltype.GcForwardReference() U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)), ('x', lltype.Signed))) S = lltype.GcStruct('S', ('u', lltype.Ptr(U))) A = lltype.GcArray(lltype.Ptr(S)) - filename = self.filename_dump + filename1 = self.filename1_dump + filename2 = self.filename2_dump def fn(): s = lltype.malloc(S) @@ -1081,20 +1083,31 @@ a = lltype.malloc(A, 1000) s2 = lltype.malloc(S) # - fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666) - rgc.dump_rpy_heap(fd) + fd1 = os.open(filename1, os.O_WRONLY | os.O_CREAT, 0666) + fd2 = os.open(filename2, os.O_WRONLY | os.O_CREAT, 0666) + rgc.dump_rpy_heap(fd1) + rgc.dump_rpy_heap(fd2) # try twice in a row keepalive_until_here(s2) keepalive_until_here(s) keepalive_until_here(a) - os.close(fd) + os.close(fd1) + os.close(fd2) return 0 return fn def test_dump_rpy_heap(self): self.run("dump_rpy_heap") - assert os.path.exists(self.filename_dump) - assert os.path.getsize(self.filename_dump) > 64 + for fn in [self.filename1_dump, self.filename2_dump]: + assert os.path.exists(fn) + assert os.path.getsize(fn) > 64 + f = open(self.filename1_dump) + data1 = f.read() + f.close() + f = open(self.filename2_dump) + data2 = f.read() + f.close() + assert data1 == data2 filename_dump_typeids_z = str(udir.join('test_typeids_z')) def define_write_typeids_z(self): Modified: pypy/trunk/pypy/translator/c/test/test_standalone.py ============================================================================== --- pypy/trunk/pypy/translator/c/test/test_standalone.py (original) +++ pypy/trunk/pypy/translator/c/test/test_standalone.py Wed Nov 24 13:15:18 2010 @@ -272,7 +272,7 @@ x = "got:" debug_start ("mycat") if have_debug_prints(): x += "b" - debug_print ("foo", 2, "bar", 3) + debug_print ("foo", r_longlong(2), "bar", 3) debug_start ("cat2") if have_debug_prints(): x += "c" debug_print ("baz") @@ -403,6 +403,20 @@ assert not err assert path.check(file=0) + def test_debug_print_start_stop_nonconst(self): + def entry_point(argv): + debug_start(argv[1]) + debug_print(argv[2]) + debug_stop(argv[1]) + return 0 + t, cbuilder = self.compile(entry_point) + out, err = cbuilder.cmdexec("foo bar", err=True, env={'PYPYLOG': ':-'}) + lines = err.splitlines() + assert '{foo' in lines[0] + assert 'bar' == lines[1] + assert 'foo}' in lines[2] + + def test_fatal_error(self): def g(x): if x == 1: Modified: pypy/trunk/pypy/translator/driver.py ============================================================================== --- pypy/trunk/pypy/translator/driver.py (original) +++ pypy/trunk/pypy/translator/driver.py Wed Nov 24 13:15:18 2010 @@ -11,6 +11,7 @@ from pypy.annotation import policy as annpolicy import optparse from pypy.tool.udir import udir +from pypy.tool.debug_print import debug_start, debug_print, debug_stop from pypy.rlib.entrypoint import secondary_entrypoints import py @@ -275,6 +276,8 @@ return else: self.log.info("%s..." % title) + debug_start('translation-task') + debug_print('starting', goal) self.timer.start_event(goal) try: instrument = False @@ -292,11 +295,13 @@ assert False, 'we should not get here' finally: try: + debug_stop('translation-task') self.timer.end_event(goal) except (KeyboardInterrupt, SystemExit): raise except: pass + #import gc; gc.dump_rpy_heap('rpyheap-after-%s.dump' % goal) return res def task_annotate(self): Modified: pypy/trunk/pypy/translator/tool/reftracker.py ============================================================================== --- pypy/trunk/pypy/translator/tool/reftracker.py (original) +++ pypy/trunk/pypy/translator/tool/reftracker.py Wed Nov 24 13:15:18 2010 @@ -3,7 +3,7 @@ Usage: call track(obj). """ -import autopath, sys, os +import autopath, sys, os, types import gc from pypy.translator.tool.graphpage import GraphPage, DotGen from pypy.tool.uid import uid @@ -39,7 +39,7 @@ if o2 is None: continue addedge(objectlist[i], o2) - id2typename[uid(o2)] = type(o2).__name__ + id2typename[uid(o2)] = self.shortrepr(o2) del o2 for o2 in self.get_referrers(objectlist[i]): if o2 is None: @@ -47,7 +47,7 @@ if type(o2) is list and o2 and o2[0] is MARKER: continue addedge(o2, objectlist[i]) - id2typename[uid(o2)] = type(o2).__name__ + id2typename[uid(o2)] = self.shortrepr(o2) del o2 for ids, label in edges.items(): @@ -82,13 +82,23 @@ return self.newpage(objectlist) def formatobject(self, o): + header = self.shortrepr(o, compact=False) + secondline = repr(o.__class__) + return header, secondline, repr(o) + + def shortrepr(self, o, compact=True): + t = type(o) + if t is types.FrameType: + if compact: + return 'frame %r' % (o.f_code.co_name,) + else: + return 'frame %r' % (o.f_code,) s = repr(o) if len(s) > 50: - linktext = s s = s[:20] + ' ... ' + s[-20:] - else: - linktext = '' - return type(o).__name__, s, linktext + if s.startswith('<') and s.endswith('>'): + s = s[1:-1] + return s def edgelabel(self, o1, o2): return '' From arigo at codespeak.net Wed Nov 24 13:16:59 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 13:16:59 +0100 (CET) Subject: [pypy-svn] r79450 - pypy/trunk/pypy/jit/metainterp Message-ID: <20101124121659.5B073282B90@codespeak.net> Author: arigo Date: Wed Nov 24 13:16:57 2010 New Revision: 79450 Modified: pypy/trunk/pypy/jit/metainterp/warmstate.py Log: Support "--jit threshold=0" to mean the same thing as "--jit threshold=-1", i.e. "disable the JIT", and not the same thing as "--jit threshold=2", i.e. "JIT as much as possible". Modified: pypy/trunk/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/warmstate.py (original) +++ pypy/trunk/pypy/jit/metainterp/warmstate.py Wed Nov 24 13:16:57 2010 @@ -202,7 +202,7 @@ meth(default_value) def set_param_threshold(self, threshold): - if threshold < 0: + if threshold <= 0: self.increment_threshold = 0 # never reach the THRESHOLD_LIMIT return if threshold < 2: From cfbolz at codespeak.net Wed Nov 24 13:22:46 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Wed, 24 Nov 2010 13:22:46 +0100 (CET) Subject: [pypy-svn] r79451 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101124122246.D25EE282BD4@codespeak.net> Author: cfbolz Date: Wed Nov 24 13:22:45 2010 New Revision: 79451 Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Log: this seems to be needed Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Wed Nov 24 13:22:45 2010 @@ -105,7 +105,9 @@ if not methgetter: raise HackCallNotPossible funcptr = methgetter(cppthis) + funcptr = rffi.cast(self.INT_2_INT_FNPTR, funcptr) + funcptr = jit.hint(funcptr, promote=True) result = funcptr(cppthis, arg) return space.wrap(rffi.cast(lltype.Signed, result)) From david at codespeak.net Wed Nov 24 13:23:11 2010 From: david at codespeak.net (david at codespeak.net) Date: Wed, 24 Nov 2010 13:23:11 +0100 (CET) Subject: [pypy-svn] r79452 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101124122311.3AE68282BE3@codespeak.net> Author: david Date: Wed Nov 24 13:23:09 2010 New Revision: 79452 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Log: Fix ops that read fields to correctly zero or one extend the value when it is smaller than a word. Also fix ofset issues with array operations Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Wed Nov 24 13:23:09 2010 @@ -309,7 +309,7 @@ if self._debug_asm: self._dump_trace('loop.asm') - print 'Done assembling' + print 'Done assembling loop with token %r' % looptoken def _prepare_sp_patch_location(self): """Generate NOPs as placeholder to patch the instruction(s) to update the @@ -410,6 +410,27 @@ #XXX check ranges for different operations return isinstance(arg, ConstInt) and arg.getint() <= size and lower_bound + + def _ensure_result_bit_extension(self, resloc, size, signed, regalloc): + if size == 4: + return + if size == 1: + if not signed: #unsigned char + self.mc.AND_ri(resloc.value, resloc.value, 0xFF) + else: + self.mc.LSL_ri(resloc.value, resloc.value, 24) + self.mc.ASR_ri(resloc.value, resloc.value, 24) + elif size == 2: + if not signed: + t = TempBox() + loc = regalloc.force_allocate_reg(t) + self.mc.gen_load_int(loc.value, 0xFFFF) + self.mc.AND_rr(resloc.value, resloc.value, loc.value) + regalloc.possibly_free_var(t) + else: + self.mc.LSL_ri(resloc.value, resloc.value, 16) + self.mc.ASR_ri(resloc.value, resloc.value, 16) + def patch_trace(self, faildescr, bridge_addr, regalloc): # The first instruction (word) is not overwritten, because it is the # one that actually checks the condition Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Wed Nov 24 13:23:09 2010 @@ -251,15 +251,10 @@ descr = op.getdescr() #XXX Hack, Hack, Hack if op.result and not we_are_translated() and not isinstance(descr, LoopToken): - l = regalloc.loc(op.result) - # XXX we need descr.get_result_sign here!!!! + loc = regalloc.loc(op.result) size = descr.get_result_size(False) - # for now just check the size of the value - if size == 1: #unsigned char - self.mc.AND_ri(l.value, l.value, 255) - elif size == 2: # signed short - self.mc.LSL_ri(l.value, l.value, 16) - self.mc.ASR_ri(l.value, l.value, 16) + signed = descr.is_result_signed() + self._ensure_result_bit_extension(loc, size, signed, regalloc) return cond def _emit_call(self, adr, args, regalloc, fcond=c.AL, save_all_regs=False, result=None): @@ -353,6 +348,11 @@ else: assert 0 f(res.value, base_loc.value, ofs) + + #XXX Hack, Hack, Hack + if not we_are_translated(): + signed = op.getdescr().is_field_signed() + self._ensure_result_bit_extension(res, size, signed, regalloc) return fcond emit_op_getfield_raw = emit_op_getfield_gc @@ -397,11 +397,14 @@ #XXX check if imm would be fine here value_loc = regalloc.make_sure_var_in_reg(a2, imm_fine=False) + if scale > 0: + self.mc.LSL_ri(ofs_loc.value, ofs_loc.value, scale) + if ofs > 0: + self.mc.ADD_ri(ofs_loc.value, ofs_loc.value, ofs) + if scale == 2: - self.mc.STR_rr(value_loc.value, base_loc.value, ofs_loc.value, cond=fcond, - imm=scale, shifttype=shift.LSL) + self.mc.STR_rr(value_loc.value, base_loc.value, ofs_loc.value, cond=fcond) elif scale == 1: - self.mc.LSL_ri(ofs_loc.value, ofs_loc.value, scale) self.mc.STRH_rr(value_loc.value, base_loc.value, ofs_loc.value, cond=fcond) elif scale == 0: self.mc.STRB_rr(value_loc.value, base_loc.value, ofs_loc.value, cond=fcond) @@ -419,6 +422,12 @@ base_loc = regalloc.make_sure_var_in_reg(a0, imm_fine=False) ofs_loc = regalloc.make_sure_var_in_reg(a1, imm_fine=False) res = regalloc.force_allocate_reg(op.result) + + if scale > 0: + self.mc.LSL_ri(ofs_loc.value, ofs_loc.value, scale) + if ofs > 0: + self.mc.ADD_ri(ofs_loc.value, ofs_loc.value, imm=ofs) + if scale == 2: f = self.mc.LDR_rr elif scale == 1: @@ -427,9 +436,14 @@ f = self.mc.LDRB_rr else: assert 0 - if scale > 0: - self.mc.LSL_ri(ofs_loc.value, ofs_loc.value, scale) + f(res.value, base_loc.value, ofs_loc.value, cond=fcond) + #XXX Hack, Hack, Hack + if not we_are_translated(): + descr = op.getdescr() + size = descr.get_item_size(False) + signed = descr.is_item_signed() + self._ensure_result_bit_extension(res, size, signed, regalloc) return fcond emit_op_getarrayitem_raw = emit_op_getarrayitem_gc Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Wed Nov 24 13:23:09 2010 @@ -42,7 +42,7 @@ def execute_token(self, executable_token): i = [self.get_latest_value_int(x) for x in range(10)] - print 'Inputargs: ', i + print 'Inputargs: %r for token %r' % (i, executable_token) addr = executable_token._arm_bootstrap_code assert addr % 8 == 0 func = rffi.cast(lltype.Ptr(self.BOOTSTRAP_TP), addr) From fijal at codespeak.net Wed Nov 24 13:30:37 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Wed, 24 Nov 2010 13:30:37 +0100 (CET) Subject: [pypy-svn] r79453 - pypy/extradoc/planning Message-ID: <20101124123037.20FD9282BF0@codespeak.net> Author: fijal Date: Wed Nov 24 13:30:35 2010 New Revision: 79453 Modified: pypy/extradoc/planning/pypyonpypy.txt Log: fixed Modified: pypy/extradoc/planning/pypyonpypy.txt ============================================================================== --- pypy/extradoc/planning/pypyonpypy.txt (original) +++ pypy/extradoc/planning/pypyonpypy.txt Wed Nov 24 13:30:35 2010 @@ -1,4 +1,2 @@ -* rpython/test/test_rclass.py fails for GC reasons - * ctypes has bugs (exposed by test_ctypes, but also backend/x86 tests segfaulting every now and then) From afa at codespeak.net Wed Nov 24 13:35:12 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 24 Nov 2010 13:35:12 +0100 (CET) Subject: [pypy-svn] r79454 - in pypy/branch/fast-forward/pypy/module/_io: . test Message-ID: <20101124123512.01421282BE3@codespeak.net> Author: afa Date: Wed Nov 24 13:35:11 2010 New Revision: 79454 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Log: Test and fix Modified: pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_bufferedio.py Wed Nov 24 13:35:11 2010 @@ -281,7 +281,7 @@ space.call_method(self.w_raw, "close") @unwrap_spec('self', ObjSpace) - def flush_w(self, space): + def simple_flush_w(self, space): self._check_init(space) return space.call_method(self.w_raw, "flush") @@ -387,7 +387,7 @@ if size == -1: # read until the end of stream with self.lock: - res = self._read_all(space) + return self._read_all(space) elif size >= 0: res = self._read_fast(size) if res is None: @@ -484,6 +484,8 @@ # Read until EOF or until read() would block w_data = space.call_method(self.w_raw, "read") if space.is_w(w_data, space.w_None): + if current_size == 0: + return w_data break data = space.str_w(w_data) size = len(data) @@ -493,7 +495,7 @@ current_size += size if self.abs_pos != -1: self.abs_pos += size - return builder.build() + return space.wrap(builder.build()) def _raw_read(self, space, buffer, start, length): length = intmask(length) @@ -759,7 +761,7 @@ seek = interp2app(W_BufferedReader.seek_w), tell = interp2app(W_BufferedReader.tell_w), close = interp2app(W_BufferedReader.close_w), - flush = interp2app(W_BufferedReader.flush_w), + flush = interp2app(W_BufferedReader.simple_flush_w), # Not flush_w! detach = interp2app(W_BufferedReader.detach_w), truncate = interp2app(W_BufferedReader.truncate_w), fileno = interp2app(W_BufferedReader.fileno_w), Modified: pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/test/test_bufferedio.py Wed Nov 24 13:35:11 2010 @@ -339,6 +339,48 @@ # Previously buffered bytes were flushed assert s.startswith("01234567A") + def test_read_non_blocking(self): + import _io + class MockRawIO(_io._RawIOBase): + def __init__(self, read_stack=()): + self._read_stack = list(read_stack) + def readable(self): + return True + def readinto(self, buf): + max_len = len(buf) + try: + data = self._read_stack[0] + except IndexError: + self._extraneous_reads += 1 + return 0 + if data is None: + del self._read_stack[0] + return None + n = len(data) + if len(data) <= max_len: + del self._read_stack[0] + buf[:n] = data + return n + else: + buf[:] = data[:max_len] + self._read_stack[0] = data[max_len:] + return max_len + def read(self, n=None): + try: + return self._read_stack.pop(0) + except IndexError: + return "" + # Inject some None's in there to simulate EWOULDBLOCK + rawio = MockRawIO((b"abc", b"d", None, b"efg", None, None, None)) + bufio = _io.BufferedReader(rawio) + + assert bufio.read(6) == "abcd" + assert bufio.read(1) == "e" + assert bufio.read() == "fg" + assert bufio.peek(1) == "" + assert bufio.read() is None + assert bufio.read() == "" + class AppTestBufferedRWPair: def test_pair(self): import _io From afa at codespeak.net Wed Nov 24 13:48:31 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 24 Nov 2010 13:48:31 +0100 (CET) Subject: [pypy-svn] r79455 - pypy/branch/fast-forward/pypy/translator/c/src Message-ID: <20101124124831.E6E9E282B9D@codespeak.net> Author: afa Date: Wed Nov 24 13:48:29 2010 New Revision: 79455 Modified: pypy/branch/fast-forward/pypy/translator/c/src/signals.h Log: Fix compilation on Windows when many modules are disabled Modified: pypy/branch/fast-forward/pypy/translator/c/src/signals.h ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/src/signals.h (original) +++ pypy/branch/fast-forward/pypy/translator/c/src/signals.h Wed Nov 24 13:48:29 2010 @@ -10,6 +10,7 @@ #ifdef MS_WINDOWS #include +#include #endif #include From antocuni at codespeak.net Wed Nov 24 13:54:36 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 24 Nov 2010 13:54:36 +0100 (CET) Subject: [pypy-svn] r79456 - in pypy/trunk/pypy/jit/tool: . test Message-ID: <20101124125436.2E46A282B9E@codespeak.net> Author: antocuni Date: Wed Nov 24 13:54:35 2010 New Revision: 79456 Added: pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py (contents, props changed) Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric pypy/trunk/pypy/jit/tool/log2gnumeric.py Log: - fix for the new log format - use clocks relative to the "time0", i.e. the first clock value seen in the file - write tests :-) Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric ============================================================================== Binary files. No diff available. Modified: pypy/trunk/pypy/jit/tool/log2gnumeric.py ============================================================================== --- pypy/trunk/pypy/jit/tool/log2gnumeric.py (original) +++ pypy/trunk/pypy/jit/tool/log2gnumeric.py Wed Nov 24 13:54:35 2010 @@ -21,18 +21,24 @@ outname = logname + '.gnumeric' data = open(logname).read() data = data.replace('\n', '') - maxclock = find_max_clock(data) + minclock, maxclock = get_clock_range(data) + time0 = minclock # we want "relative clocks" + maxtime = maxclock-time0 # xml = gzip.open('log-template.gnumeric').read() - xml = replace_sheet(xml, 'translation-task', tasks_rows(data)) - xml = replace_sheet(xml, 'gc-collect', gc_collect_rows(data)) - xml = replace_sheet(xml, 'memusage', memusage_rows(logname + '.memusage', maxclock)) + xml = replace_sheet(xml, 'translation-task', tasks_rows(time0, data)) + xml = replace_sheet(xml, 'gc-collect', gc_collect_rows(time0, data)) + xml = replace_sheet(xml, 'memusage', memusage_rows(logname + '.memusage', maxtime)) # out = gzip.open(outname, 'wb') out.write(xml) out.close() +# ======================================================================== +# functions to manipulate gnumeric files +# ======================================================================== + def replace_sheet(xml, sheet_name, data): pattern = '%s.*?(.*?)' regex = re.compile(pattern % sheet_name, re.DOTALL) @@ -68,7 +74,17 @@ return '\n'.join(parts) -def gc_collect_rows(data): +# ======================================================================== +# functions to extract various data from the logs +# ======================================================================== + +def get_clock_range(data): + s = r"\[([0-9a-f]+)\] " + r = re.compile(s) + clocks = [int(x, 16) for x in r.findall(data)] + return min(clocks), max(clocks) + +def gc_collect_rows(time0, data): s = r""" ----------- Full collection ------------------ \| used before collection: @@ -84,35 +100,34 @@ r = re.compile(s.replace('\n', '')) yield 'clock', 'gc-before', 'gc-after' for a,b,c,d,e,f in r.findall(data): - yield int(f, 16), int(a)+int(b), int(c)+int(d) + clock = int(f, 16) - time0 + yield clock, int(a)+int(b), int(c)+int(d) -def tasks_rows(data): +def tasks_rows(time0, data): s = r""" -\{translation-task +\[([0-9a-f]+)\] \{translation-task starting ([\w-]+) -\[([0-9a-f]+)\] translation-task\}""" +""" # r = re.compile(s.replace('\n', '')) yield 'clock', None, 'task' for a,b in r.findall(data): - yield int(b, 16), 1, a + clock = int(a, 16) - time0 + yield clock, 1, b -def memusage_rows(filename, maxclock): +def memusage_rows(filename, maxtime): try: - lines = open(filename) + lines = open(filename).readlines() except IOError: print 'Warning: cannot find file %s, skipping the memusage sheet' lines = [] - yield 'n', 'computed clock', 'VmRSS' + yield 'inferred clock', 'VmRSS' + numlines = len(lines) for i, line in enumerate(lines): mem = int(line) - yield i, "=max('gc-collect'!$A$1:$A$65536)*(A2/max($A$1:$A$65536)))", mem + clock = maxtime * i // numlines + yield clock, mem -def find_max_clock(data): - s = r"\[([0-9a-f]+)\] " - r = re.compile(s) - clocks = [int(x, 16) for x in r.findall(data)] - return max(clocks) if __name__ == '__main__': main() Added: pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py ============================================================================== --- (empty file) +++ pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py Wed Nov 24 13:54:35 2010 @@ -0,0 +1,65 @@ +from pypy.jit.tool import log2gnumeric + +log = """ +[1000] ... +[2000] {gc-collect + +.----------- Full collection ------------------ +| used before collection: +| in ArenaCollection: 500 bytes +| raw_malloced: 100 bytes +| used after collection: +| in ArenaCollection: 300 bytes +| raw_malloced: 50 bytes +| number of major collects: 1 +`---------------------------------------------- +[3000] gc-collect} +[4000] {gc-collect + +.----------- Full collection ------------------ +| used before collection: +| in ArenaCollection: 600 bytes +| raw_malloced: 200 bytes +| used after collection: +| in ArenaCollection: 400 bytes +| raw_malloced: 100 bytes +| number of major collects: 1 +`---------------------------------------------- +[5000] gc-collect} +... +... +[6000] {translation-task +starting annotate +... +... +[7000] translation-task} +[8000] {translation-task +starting rtype_lltype +... +... +[9000] translation-task} +... +[a000] ... +""" + +log = log.replace('\n', '') + +def test_get_clock_range(): + minclock, maxclock = log2gnumeric.get_clock_range(log) + assert minclock == 0x1000 + assert maxclock == 0xa000 + + +def test_gc_collect_rows(): + rows = list(log2gnumeric.gc_collect_rows(0x1000, log)) + assert len(rows) == 3 + assert rows[0] == ( 'clock', 'gc-before', 'gc-after') + assert rows[1] == (0x3000-0x1000, 500+100, 300+ 50) + assert rows[2] == (0x5000-0x1000, 600+200, 400+100) + +def test_tasks_rows(): + rows = list(log2gnumeric.tasks_rows(0x1000, log)) + assert len(rows) == 3 + assert rows[0] == ( 'clock', None, 'task') + assert rows[1] == (0x6000-0x1000, 1, 'annotate') + assert rows[2] == (0x8000-0x1000, 1, 'rtype_lltype') From antocuni at codespeak.net Wed Nov 24 14:18:31 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 24 Nov 2010 14:18:31 +0100 (CET) Subject: [pypy-svn] r79457 - in pypy/trunk/pypy/jit/tool: . test Message-ID: <20101124131831.D75045080B@codespeak.net> Author: antocuni Date: Wed Nov 24 14:18:30 2010 New Revision: 79457 Modified: pypy/trunk/pypy/jit/tool/log2gnumeric.py pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py Log: write a test for memusage_rows, and fix the corresponding bug Modified: pypy/trunk/pypy/jit/tool/log2gnumeric.py ============================================================================== --- pypy/trunk/pypy/jit/tool/log2gnumeric.py (original) +++ pypy/trunk/pypy/jit/tool/log2gnumeric.py Wed Nov 24 14:18:30 2010 @@ -121,11 +121,15 @@ except IOError: print 'Warning: cannot find file %s, skipping the memusage sheet' lines = [] + for row in memusage_rows_impl(lines, maxtime): + yield row + +def memusage_rows_impl(lines, maxtime): yield 'inferred clock', 'VmRSS' numlines = len(lines) for i, line in enumerate(lines): mem = int(line) - clock = maxtime * i // numlines + clock = maxtime * i // (numlines-1) yield clock, mem Modified: pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py ============================================================================== --- pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py (original) +++ pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py Wed Nov 24 14:18:30 2010 @@ -63,3 +63,14 @@ assert rows[0] == ( 'clock', None, 'task') assert rows[1] == (0x6000-0x1000, 1, 'annotate') assert rows[2] == (0x8000-0x1000, 1, 'rtype_lltype') + + +def test_memusage_rows(): + lines = ['100', '200', '300'] + rows = list(log2gnumeric.memusage_rows_impl(lines, 2000)) + assert len(rows) == 4 + assert rows[0] == ('inferred clock', 'VmRSS') + assert rows[1] == (0, 100) + assert rows[2] == (1000, 200) + assert rows[3] == (2000, 300) + From afa at codespeak.net Wed Nov 24 14:21:07 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 24 Nov 2010 14:21:07 +0100 (CET) Subject: [pypy-svn] r79458 - in pypy/branch/fast-forward: . lib-python/modified-2.5.2/distutils pypy pypy/annotation pypy/config pypy/config/test pypy/doc pypy/doc/config pypy/interpreter pypy/interpreter/test pypy/jit pypy/jit/backend pypy/jit/backend/llgraph pypy/jit/backend/llsupport pypy/jit/backend/llsupport/test pypy/jit/backend/test pypy/jit/backend/x86 pypy/jit/backend/x86/test pypy/jit/codewriter pypy/jit/metainterp pypy/jit/metainterp/optimizeopt pypy/jit/metainterp/test pypy/jit/tool pypy/jit/tool/test pypy/module/__pypy__ pypy/module/__pypy__/test pypy/module/_pickle_support pypy/module/_rawffi/test pypy/module/array pypy/module/array/benchmark pypy/module/array/test pypy/module/cpyext pypy/module/cpyext/test pypy/module/gc pypy/module/gc/test pypy/module/imp pypy/module/imp/test pypy/module/pypyjit pypy/module/pypyjit/test pypy/module/rctime pypy/module/sys pypy/module/sys/test pypy/objspace/flow pypy/objspace/std pypy/objspace/std/test pypy/rlib pypy/rlib/test pypy/rpython pypy/rpython/lltypesystem pypy/rpython/memory pypy/rpython/memory/gc pypy/rpython/memory/gc/test pypy/rpython/memory/gctransform pypy/rpython/memory/test pypy/rpython/test pypy/tool pypy/tool/release pypy/tool/release/test pypy/translator pypy/translator/c pypy/translator/c/gcc pypy/translator/c/gcc/test pypy/translator/c/src pypy/translator/c/test pypy/translator/goal pypy/translator/goal/test2 pypy/translator/platform pypy/translator/tool site-packages Message-ID: <20101124132107.E8A48282B9D@codespeak.net> Author: afa Date: Wed Nov 24 14:21:00 2010 New Revision: 79458 Added: pypy/branch/fast-forward/pypy/config/support.py - copied unchanged from r79455, pypy/trunk/pypy/config/support.py pypy/branch/fast-forward/pypy/config/test/test_support.py - copied unchanged from r79455, pypy/trunk/pypy/config/test/test_support.py pypy/branch/fast-forward/pypy/doc/config/objspace.soabi.txt - copied unchanged from r79455, pypy/trunk/pypy/doc/config/objspace.soabi.txt pypy/branch/fast-forward/pypy/doc/config/objspace.translationmodules.txt - copied unchanged from r79455, pypy/trunk/pypy/doc/config/objspace.translationmodules.txt pypy/branch/fast-forward/pypy/doc/release-1.4.0beta.txt - copied unchanged from r79455, pypy/trunk/pypy/doc/release-1.4.0beta.txt pypy/branch/fast-forward/pypy/jit/metainterp/memmgr.py - copied unchanged from r79455, pypy/trunk/pypy/jit/metainterp/memmgr.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_memmgr.py - copied unchanged from r79455, pypy/trunk/pypy/jit/metainterp/test/test_memmgr.py pypy/branch/fast-forward/pypy/jit/tool/log-template.gnumeric - copied unchanged from r79455, pypy/trunk/pypy/jit/tool/log-template.gnumeric pypy/branch/fast-forward/pypy/jit/tool/log2gnumeric.py - copied unchanged from r79455, pypy/trunk/pypy/jit/tool/log2gnumeric.py pypy/branch/fast-forward/pypy/jit/tool/loopcounter.py - copied unchanged from r79455, pypy/trunk/pypy/jit/tool/loopcounter.py pypy/branch/fast-forward/pypy/jit/tool/test/test_loopcounter.py - copied unchanged from r79455, pypy/trunk/pypy/jit/tool/test/test_loopcounter.py pypy/branch/fast-forward/pypy/jit/tool/test/test_oparser.py - copied unchanged from r79455, pypy/trunk/pypy/jit/tool/test/test_oparser.py pypy/branch/fast-forward/pypy/module/__pypy__/interp_debug.py - copied unchanged from r79455, pypy/trunk/pypy/module/__pypy__/interp_debug.py pypy/branch/fast-forward/pypy/module/__pypy__/test/test_debug.py - copied unchanged from r79455, pypy/trunk/pypy/module/__pypy__/test/test_debug.py pypy/branch/fast-forward/pypy/rpython/memory/gc/env.py - copied unchanged from r79455, pypy/trunk/pypy/rpython/memory/gc/env.py pypy/branch/fast-forward/pypy/rpython/memory/gc/test/test_env.py - copied unchanged from r79455, pypy/trunk/pypy/rpython/memory/gc/test/test_env.py pypy/branch/fast-forward/pypy/tool/debug_print.py - copied unchanged from r79455, pypy/trunk/pypy/tool/debug_print.py pypy/branch/fast-forward/pypy/tool/gcdump.py - copied unchanged from r79455, pypy/trunk/pypy/tool/gcdump.py pypy/branch/fast-forward/pypy/translator/c/src/asm_gcc_x86_64.h - copied unchanged from r79455, pypy/trunk/pypy/translator/c/src/asm_gcc_x86_64.h Removed: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_oparser.py Modified: pypy/branch/fast-forward/ (props changed) pypy/branch/fast-forward/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py pypy/branch/fast-forward/pypy/ (props changed) pypy/branch/fast-forward/pypy/annotation/bookkeeper.py pypy/branch/fast-forward/pypy/annotation/builtin.py pypy/branch/fast-forward/pypy/config/pypyoption.py pypy/branch/fast-forward/pypy/config/translationoption.py pypy/branch/fast-forward/pypy/conftest.py pypy/branch/fast-forward/pypy/doc/config/objspace.usemodules.array.txt (props changed) pypy/branch/fast-forward/pypy/doc/release-1.4.0.txt pypy/branch/fast-forward/pypy/interpreter/argument.py pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py pypy/branch/fast-forward/pypy/interpreter/generator.py pypy/branch/fast-forward/pypy/interpreter/test/test_argument.py pypy/branch/fast-forward/pypy/jit/backend/conftest.py pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_gc.py pypy/branch/fast-forward/pypy/jit/backend/model.py pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py pypy/branch/fast-forward/pypy/jit/backend/test/test_random.py pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_regalloc.py pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py pypy/branch/fast-forward/pypy/jit/conftest.py pypy/branch/fast-forward/pypy/jit/metainterp/compile.py pypy/branch/fast-forward/pypy/jit/metainterp/history.py pypy/branch/fast-forward/pypy/jit/metainterp/jitprof.py pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/optimizer.py (contents, props changed) pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py pypy/branch/fast-forward/pypy/jit/metainterp/resume.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_compile.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_exception.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_pyjitpl.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_recursive.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resoperation.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmspot.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmstate.py pypy/branch/fast-forward/pypy/jit/metainterp/test/test_ztranslation.py pypy/branch/fast-forward/pypy/jit/metainterp/virtualizable.py pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py pypy/branch/fast-forward/pypy/jit/metainterp/warmstate.py pypy/branch/fast-forward/pypy/jit/tool/jitoutput.py pypy/branch/fast-forward/pypy/jit/tool/test/test_jitoutput.py pypy/branch/fast-forward/pypy/module/__pypy__/__init__.py pypy/branch/fast-forward/pypy/module/__pypy__/interp_magic.py pypy/branch/fast-forward/pypy/module/_pickle_support/maker.py pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py pypy/branch/fast-forward/pypy/module/_rawffi/test/test_nested.py pypy/branch/fast-forward/pypy/module/array/benchmark/Makefile (props changed) pypy/branch/fast-forward/pypy/module/array/benchmark/intimg.c (props changed) pypy/branch/fast-forward/pypy/module/array/benchmark/intimgtst.c (props changed) pypy/branch/fast-forward/pypy/module/array/benchmark/intimgtst.py (props changed) pypy/branch/fast-forward/pypy/module/array/benchmark/loop.c (props changed) pypy/branch/fast-forward/pypy/module/array/benchmark/sum.c (props changed) pypy/branch/fast-forward/pypy/module/array/benchmark/sumtst.c (props changed) pypy/branch/fast-forward/pypy/module/array/benchmark/sumtst.py (props changed) pypy/branch/fast-forward/pypy/module/array/interp_array.py pypy/branch/fast-forward/pypy/module/array/test/test_array_old.py (props changed) pypy/branch/fast-forward/pypy/module/cpyext/api.py pypy/branch/fast-forward/pypy/module/cpyext/cdatetime.py pypy/branch/fast-forward/pypy/module/cpyext/presetup.py pypy/branch/fast-forward/pypy/module/cpyext/test/test_borrow.py pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py pypy/branch/fast-forward/pypy/module/cpyext/typeobject.py pypy/branch/fast-forward/pypy/module/gc/__init__.py pypy/branch/fast-forward/pypy/module/gc/app_referents.py pypy/branch/fast-forward/pypy/module/gc/interp_gc.py pypy/branch/fast-forward/pypy/module/gc/referents.py pypy/branch/fast-forward/pypy/module/gc/test/test_gc.py pypy/branch/fast-forward/pypy/module/imp/importing.py pypy/branch/fast-forward/pypy/module/imp/interp_imp.py pypy/branch/fast-forward/pypy/module/imp/test/test_app.py pypy/branch/fast-forward/pypy/module/imp/test/test_import.py pypy/branch/fast-forward/pypy/module/pypyjit/__init__.py pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py pypy/branch/fast-forward/pypy/module/rctime/interp_time.py pypy/branch/fast-forward/pypy/module/sys/state.py pypy/branch/fast-forward/pypy/module/sys/test/test_initialpath.py pypy/branch/fast-forward/pypy/objspace/flow/model.py pypy/branch/fast-forward/pypy/objspace/std/callmethod.py pypy/branch/fast-forward/pypy/objspace/std/mapdict.py pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py pypy/branch/fast-forward/pypy/rlib/clibffi.py pypy/branch/fast-forward/pypy/rlib/debug.py pypy/branch/fast-forward/pypy/rlib/jit.py pypy/branch/fast-forward/pypy/rlib/rerased.py (props changed) pypy/branch/fast-forward/pypy/rlib/rgc.py pypy/branch/fast-forward/pypy/rlib/test/test_rerased.py (props changed) pypy/branch/fast-forward/pypy/rpython/annlowlevel.py pypy/branch/fast-forward/pypy/rpython/llinterp.py pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py pypy/branch/fast-forward/pypy/rpython/lltypesystem/llmemory.py pypy/branch/fast-forward/pypy/rpython/lltypesystem/lloperation.py pypy/branch/fast-forward/pypy/rpython/lltypesystem/lltype.py pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py pypy/branch/fast-forward/pypy/rpython/memory/gc/base.py pypy/branch/fast-forward/pypy/rpython/memory/gc/generation.py pypy/branch/fast-forward/pypy/rpython/memory/gc/inspector.py pypy/branch/fast-forward/pypy/rpython/memory/gc/markcompact.py pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py pypy/branch/fast-forward/pypy/rpython/memory/gc/semispace.py pypy/branch/fast-forward/pypy/rpython/memory/gc/test/test_minimark.py pypy/branch/fast-forward/pypy/rpython/memory/gctransform/framework.py pypy/branch/fast-forward/pypy/rpython/memory/support.py pypy/branch/fast-forward/pypy/rpython/memory/test/test_gc.py pypy/branch/fast-forward/pypy/rpython/rbuiltin.py pypy/branch/fast-forward/pypy/rpython/test/test_rclass.py pypy/branch/fast-forward/pypy/tool/logparser.py pypy/branch/fast-forward/pypy/tool/release/force-builds.py pypy/branch/fast-forward/pypy/tool/release/make_release.py pypy/branch/fast-forward/pypy/tool/release/package.py pypy/branch/fast-forward/pypy/tool/release/test/test_package.py pypy/branch/fast-forward/pypy/tool/terminal.py pypy/branch/fast-forward/pypy/translator/c/funcgen.py pypy/branch/fast-forward/pypy/translator/c/gcc/test/test_trackgcroot.py pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py pypy/branch/fast-forward/pypy/translator/c/node.py pypy/branch/fast-forward/pypy/translator/c/src/asm_gcc_x86.h pypy/branch/fast-forward/pypy/translator/c/src/debug_alloc.h pypy/branch/fast-forward/pypy/translator/c/src/debug_print.h pypy/branch/fast-forward/pypy/translator/c/src/g_include.h pypy/branch/fast-forward/pypy/translator/c/test/test_lltyped.py pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py pypy/branch/fast-forward/pypy/translator/c/test/test_standalone.py pypy/branch/fast-forward/pypy/translator/driver.py pypy/branch/fast-forward/pypy/translator/goal/app_main.py pypy/branch/fast-forward/pypy/translator/goal/targetpypystandalone.py pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py pypy/branch/fast-forward/pypy/translator/platform/linux.py pypy/branch/fast-forward/pypy/translator/platform/posix.py pypy/branch/fast-forward/pypy/translator/tool/reftracker.py pypy/branch/fast-forward/site-packages/ (props changed) Log: Merge from trunk: svn merge -r79039:79455 ../trunk Modified: pypy/branch/fast-forward/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py ============================================================================== --- pypy/branch/fast-forward/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py (original) +++ pypy/branch/fast-forward/lib-python/modified-2.5.2/distutils/sysconfig_pypy.py Wed Nov 24 14:21:00 2010 @@ -3,6 +3,7 @@ import sys import os +import imp from distutils.errors import DistutilsPlatformError @@ -47,11 +48,17 @@ _config_vars = None +def _get_so_extension(): + for ext, mod, typ in imp.get_suffixes(): + if typ == imp.C_EXTENSION: + return ext + def _init_posix(): """Initialize the module as appropriate for POSIX systems.""" g = {} g['EXE'] = "" - g['SO'] = ".so" + g['SO'] = _get_so_extension() or ".so" + g['SOABI'] = g['SO'].rsplit('.')[0] global _config_vars _config_vars = g @@ -61,7 +68,8 @@ """Initialize the module as appropriate for NT""" g = {} g['EXE'] = ".exe" - g['SO'] = ".pyd" + g['SO'] = _get_so_extension() or ".pyd" + g['SOABI'] = g['SO'].rsplit('.')[0] global _config_vars _config_vars = g Modified: pypy/branch/fast-forward/pypy/annotation/bookkeeper.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/bookkeeper.py (original) +++ pypy/branch/fast-forward/pypy/annotation/bookkeeper.py Wed Nov 24 14:21:00 2010 @@ -87,13 +87,6 @@ else: return obj.knowntype.__name__ - def consider_tuple_iter(self, tup): - ctxt = "[%s]" % sys._getframe(4).f_code.co_name - if tup.is_constant(): - return ctxt, tup.const - else: - return ctxt, tuple([self.typerepr(x) for x in tup.items]) - def consider_tuple_random_getitem(self, tup): return tuple([self.typerepr(x) for x in tup.items]) Modified: pypy/branch/fast-forward/pypy/annotation/builtin.py ============================================================================== --- pypy/branch/fast-forward/pypy/annotation/builtin.py (original) +++ pypy/branch/fast-forward/pypy/annotation/builtin.py Wed Nov 24 14:21:00 2010 @@ -453,6 +453,9 @@ #p = lltype.malloc(T, flavor=s_flavor.const) #lltype.free(p, flavor=s_flavor.const) +def render_immortal(s_p, s_track_allocation=None): + assert s_track_allocation is None or s_track_allocation.is_constant() + def typeOf(s_val): lltype = annotation_to_lltype(s_val, info="in typeOf(): ") return immutablevalue(lltype) @@ -520,6 +523,7 @@ BUILTIN_ANALYZERS[lltype.malloc] = malloc BUILTIN_ANALYZERS[lltype.free] = free +BUILTIN_ANALYZERS[lltype.render_immortal] = render_immortal BUILTIN_ANALYZERS[lltype.typeOf] = typeOf BUILTIN_ANALYZERS[lltype.cast_primitive] = cast_primitive BUILTIN_ANALYZERS[lltype.nullptr] = nullptr Modified: pypy/branch/fast-forward/pypy/config/pypyoption.py ============================================================================== --- pypy/branch/fast-forward/pypy/config/pypyoption.py (original) +++ pypy/branch/fast-forward/pypy/config/pypyoption.py Wed Nov 24 14:21:00 2010 @@ -34,6 +34,11 @@ "_bisect", "_multiprocessing", '_warnings'] )) +translation_modules = default_modules.copy() +translation_modules.update(dict.fromkeys( + ["fcntl", "rctime", "select", "signal", "_rawffi", "zlib", + "struct", "md5", "cStringIO", "array"])) + working_oo_modules = default_modules.copy() working_oo_modules.update(dict.fromkeys( ["_md5", "_sha", "cStringIO", "itertools"] @@ -151,6 +156,12 @@ cmdline="--allworkingmodules", negation=True), + BoolOption("translationmodules", + "use only those modules that are needed to run translate.py on pypy", + default=False, + cmdline="--translationmodules", + suggests=[("objspace.allworkingmodules", False)]), + BoolOption("geninterp", "specify whether geninterp should be used", cmdline=None, default=True), @@ -166,6 +177,11 @@ default=False, requires=[("objspace.usepycfiles", True)]), + StrOption("soabi", + "Tag to differentiate extension modules built for different Python interpreters", + cmdline="--soabi", + default=None), + BoolOption("honor__builtins__", "Honor the __builtins__ key of a module dictionary", default=False), @@ -371,6 +387,11 @@ modules = [name for name in modules if name not in essential_modules] config.objspace.usemodules.suggest(**dict.fromkeys(modules, True)) +def enable_translationmodules(config): + modules = translation_modules + modules = [name for name in modules if name not in essential_modules] + config.objspace.usemodules.suggest(**dict.fromkeys(modules, True)) + if __name__ == '__main__': config = get_pypy_config() Modified: pypy/branch/fast-forward/pypy/config/translationoption.py ============================================================================== --- pypy/branch/fast-forward/pypy/config/translationoption.py (original) +++ pypy/branch/fast-forward/pypy/config/translationoption.py Wed Nov 24 14:21:00 2010 @@ -3,6 +3,7 @@ from pypy.config.config import OptionDescription, BoolOption, IntOption, ArbitraryOption, FloatOption from pypy.config.config import ChoiceOption, StrOption, to_optparse, Config from pypy.config.config import ConfigError +from pypy.config.support import detect_number_of_processors DEFL_INLINE_THRESHOLD = 32.4 # just enough to inline add__Int_Int() # and just small enough to prevend inlining of some rlist functions. @@ -113,10 +114,6 @@ ChoiceOption("jit_backend", "choose the backend for the JIT", ["auto", "x86", "x86-without-sse2", "llvm"], default="auto", cmdline="--jit-backend"), - ChoiceOption("jit_debug", "the amount of debugging dumps made by the JIT", - ["off", "profile", "steps", "detailed"], - default="profile", # XXX for now - cmdline="--jit-debug"), ChoiceOption("jit_profiler", "integrate profiler support into the JIT", ["off", "oprofile"], default="off"), @@ -171,7 +168,7 @@ default=False, negation=False), IntOption("make_jobs", "Specify -j argument to make for compilation" " (C backend only)", - cmdline="--make-jobs", default=1), + cmdline="--make-jobs", default=detect_number_of_processors()), # Flags of the TranslationContext: BoolOption("simplifying", "Simplify flow graphs", default=True), Modified: pypy/branch/fast-forward/pypy/conftest.py ============================================================================== --- pypy/branch/fast-forward/pypy/conftest.py (original) +++ pypy/branch/fast-forward/pypy/conftest.py Wed Nov 24 14:21:00 2010 @@ -334,6 +334,31 @@ class PyPyTestFunction(py.test.collect.Function): # All PyPy test items catch and display OperationErrors specially. # + def runtest(self): + self.runtest_open() + try: + self.runtest_perform() + finally: + self.runtest_close() + self.runtest_finish() + + def runtest_open(self): + leakfinder.start_tracking_allocations() + + def runtest_perform(self): + super(PyPyTestFunction, self).runtest() + + def runtest_close(self): + if leakfinder.TRACK_ALLOCATIONS: + self._pypytest_leaks = leakfinder.stop_tracking_allocations(False) + else: # stop_tracking_allocations() already called + self._pypytest_leaks = None + + def runtest_finish(self): + # check for leaks, but only if the test passed so far + if self._pypytest_leaks: + raise leakfinder.MallocMismatch(self._pypytest_leaks) + def execute_appex(self, space, target, *args): try: target(*args) @@ -360,16 +385,9 @@ def _keywords(self): return super(IntTestFunction, self)._keywords() + ['interplevel'] - def runtest(self): + def runtest_perform(self): try: - leakfinder.start_tracking_allocations() - try: - super(IntTestFunction, self).runtest() - finally: - if leakfinder.TRACK_ALLOCATIONS: - leaks = leakfinder.stop_tracking_allocations(False) - else: - leaks = None # stop_tracking_allocations() already called + super(IntTestFunction, self).runtest_perform() except OperationError, e: check_keyboard_interrupt(e) raise @@ -382,14 +400,15 @@ py.test.skip('%s: %s' % (e.__class__.__name__, e)) cls = cls.__bases__[0] raise + + def runtest_finish(self): if 'pygame' in sys.modules: global _pygame_imported if not _pygame_imported: _pygame_imported = True assert option.view, ("should not invoke Pygame " "if conftest.option.view is False") - if leaks: # check for leaks, but only if the test passed so far - raise leakfinder.MallocMismatch(leaks) + super(IntTestFunction, self).runtest_finish() class AppTestFunction(PyPyTestFunction): def _prunetraceback(self, traceback): @@ -402,7 +421,7 @@ def _keywords(self): return ['applevel'] + super(AppTestFunction, self)._keywords() - def runtest(self): + def runtest_perform(self): target = self.obj if option.runappdirect: return target() @@ -434,7 +453,7 @@ space.setattr(w_instance, space.wrap(name[2:]), getattr(instance, name)) - def runtest(self): + def runtest_perform(self): target = self.obj if option.runappdirect: return target() Modified: pypy/branch/fast-forward/pypy/doc/release-1.4.0.txt ============================================================================== --- pypy/branch/fast-forward/pypy/doc/release-1.4.0.txt (original) +++ pypy/branch/fast-forward/pypy/doc/release-1.4.0.txt Wed Nov 24 14:21:00 2010 @@ -10,17 +10,19 @@ own development. Among other features, this release includes numerous performance improvements -(which made fast self-hosting possible), 64-bit JIT backend, as well as serious -stabilization. As of now, we can consider 32-bit version of PyPy stable enough -to run in production. +(which made fast self-hosting possible), a 64-bit JIT backend, as well +as serious stabilization. As of now, we can consider the 32-bit and 64-bit +linux versions of PyPy stable enoughto run in production. More highlights =============== -Virtualenv support: now PyPy is fully compatible with virtualenv_: note that -to use it, you need a recent version of virtualenv (>= 1.5). +* Virtualenv support: now PyPy is fully compatible with virtualenv_: note that + to use it, you need a recent version of virtualenv (>= 1.5). -XXX write me: better regular expressions +* Faster (and JITted) regular expressions - huge boost in speeding up + sre module. +* Faster (and JITted) calls to functions like map(). .. _virtualenv: http://pypi.python.org/pypi/virtualenv Modified: pypy/branch/fast-forward/pypy/interpreter/argument.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/argument.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/argument.py Wed Nov 24 14:21:00 2010 @@ -64,7 +64,7 @@ return not self == other - # make it look tuply for the annotator + # make it look tuply for its use in the annotator def __len__(self): return 3 @@ -103,10 +103,11 @@ make_sure_not_resized(self.keywords_w) make_sure_not_resized(self.arguments_w) - if ((w_stararg is not None and w_stararg) or - (w_starstararg is not None and w_starstararg)): - self._combine_wrapped(w_stararg, w_starstararg) - # if we have a call where * or ** args are used at the callsite + if w_stararg is not None and space.is_true(w_stararg): + self._combine_starargs_wrapped(w_stararg) + if w_starstararg is not None and space.is_true(w_starstararg): + self._combine_starstarargs_wrapped(w_starstararg) + # if we have a call where **args are used at the callsite # we shouldn't let the JIT see the argument matching self._dont_jit = True else: @@ -142,53 +143,59 @@ def _combine_wrapped(self, w_stararg, w_starstararg): "unpack the *arg and **kwd into arguments_w and keywords_w" - # unpack the * arguments if w_stararg is not None: - self.arguments_w = (self.arguments_w + - self.space.fixedview(w_stararg)) - # unpack the ** arguments + self._combine_starargs_wrapped(w_stararg) if w_starstararg is not None: - space = self.space - if space.isinstance_w(w_starstararg, space.w_dict): - keys_w = space.unpackiterable(w_starstararg) - else: - try: - w_keys = space.call_method(w_starstararg, "keys") - except OperationError, e: - if e.match(space, space.w_AttributeError): - w_type = space.type(w_starstararg) - typename = w_type.getname(space, '?') - raise OperationError( - space.w_TypeError, - space.wrap("argument after ** must be " - "a mapping, not %s" % (typename,))) + self._combine_starstarargs_wrapped(w_starstararg) + + def _combine_starargs_wrapped(self, w_stararg): + # unpack the * arguments + self.arguments_w = (self.arguments_w + + self.space.fixedview(w_stararg)) + + def _combine_starstarargs_wrapped(self, w_starstararg): + # unpack the ** arguments + space = self.space + if space.isinstance_w(w_starstararg, space.w_dict): + keys_w = space.unpackiterable(w_starstararg) + else: + try: + w_keys = space.call_method(w_starstararg, "keys") + except OperationError, e: + if e.match(space, space.w_AttributeError): + w_type = space.type(w_starstararg) + typename = w_type.getname(space, '?') + raise OperationError( + space.w_TypeError, + space.wrap("argument after ** must be " + "a mapping, not %s" % (typename,))) + raise + keys_w = space.unpackiterable(w_keys) + keywords_w = [None] * len(keys_w) + keywords = [None] * len(keys_w) + i = 0 + for w_key in keys_w: + try: + key = space.str_w(w_key) + except OperationError, e: + if not e.match(space, space.w_TypeError): raise - keys_w = space.unpackiterable(w_keys) - keywords_w = [None] * len(keys_w) - keywords = [None] * len(keys_w) - i = 0 - for w_key in keys_w: - try: - key = space.str_w(w_key) - except OperationError, e: - if not e.match(space, space.w_TypeError): - raise - raise OperationError(space.w_TypeError, - space.wrap("keywords must be strings")) - if self.keywords and key in self.keywords: - raise operationerrfmt(self.space.w_TypeError, - "got multiple values " - "for keyword argument " - "'%s'", key) - keywords[i] = key - keywords_w[i] = space.getitem(w_starstararg, w_key) - i += 1 - if self.keywords is None: - self.keywords = keywords - self.keywords_w = keywords_w - else: - self.keywords = self.keywords + keywords - self.keywords_w = self.keywords_w + keywords_w + raise OperationError(space.w_TypeError, + space.wrap("keywords must be strings")) + if self.keywords and key in self.keywords: + raise operationerrfmt(self.space.w_TypeError, + "got multiple values " + "for keyword argument " + "'%s'", key) + keywords[i] = key + keywords_w[i] = space.getitem(w_starstararg, w_key) + i += 1 + if self.keywords is None: + self.keywords = keywords + self.keywords_w = keywords_w + else: + self.keywords = self.keywords + keywords + self.keywords_w = self.keywords_w + keywords_w def fixedunpack(self, argcount): """The simplest argument parsing: get the 'argcount' arguments, @@ -237,6 +244,10 @@ # argnames = list of formal parameter names # scope_w = resulting list of wrapped values # + + # some comments about the JIT: it assumes that signature is a constant, + # so all values coming from there can be assumed constant. It assumes + # that the length of the defaults_w does not vary too much. co_argcount = signature.num_argnames() # expected formal arguments, without */** has_vararg = signature.has_vararg() has_kwarg = signature.has_kwarg() @@ -256,12 +267,6 @@ args_w = self.arguments_w num_args = len(args_w) - keywords = self.keywords - keywords_w = self.keywords_w - num_kwds = 0 - if keywords is not None: - num_kwds = len(keywords) - avail = num_args + upfront if input_argcount < co_argcount: @@ -271,15 +276,24 @@ else: take = num_args + # letting the JIT unroll this loop is safe, because take is always + # smaller than co_argcount for i in range(take): scope_w[i + input_argcount] = args_w[i] input_argcount += take + keywords = self.keywords + keywords_w = self.keywords_w + num_kwds = 0 + if keywords is not None: + num_kwds = len(keywords) # the code assumes that keywords can potentially be large, but that # argnames is typically not too large num_remainingkwds = num_kwds used_keywords = None if keywords: + # letting JIT unroll the loop is *only* safe if the callsite didn't + # use **args because num_kwds can be arbitrarily large otherwise. used_keywords = [False] * num_kwds for i in range(num_kwds): name = keywords[i] @@ -287,7 +301,7 @@ if j < 0: continue elif j < input_argcount: - # check that no keyword argument conflicts with these note + # check that no keyword argument conflicts with these. note # that for this purpose we ignore the first blindargs, # which were put into place by prepend(). This way, # keywords do not conflict with the hidden extra argument @@ -399,9 +413,10 @@ space = self.space w_args = space.newtuple(self.arguments_w) w_kwds = space.newdict() - for i in range(len(self.keywords)): - space.setitem(w_kwds, space.wrap(self.keywords[i]), - self.keywords_w[i]) + if self.keywords is not None: + for i in range(len(self.keywords)): + space.setitem(w_kwds, space.wrap(self.keywords[i]), + self.keywords_w[i]) return w_args, w_kwds class ArgumentsForTranslation(Arguments): Modified: pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/baseobjspace.py Wed Nov 24 14:21:00 2010 @@ -147,7 +147,7 @@ __already_enqueued_for_destruction = False - def _enqueue_for_destruction(self, space): + def _enqueue_for_destruction(self, space, call_user_del=True): """Put the object in the destructor queue of the space. At a later, safe point in time, UserDelAction will use space.userdel() to call the object's app-level __del__ method. @@ -160,7 +160,8 @@ return self.__already_enqueued_for_destruction = True self.clear_all_weakrefs() - space.user_del_action.register_dying_object(self) + if call_user_del: + space.user_del_action.register_dying_object(self) def _call_builtin_destructor(self): pass # method overridden in typedef.py Modified: pypy/branch/fast-forward/pypy/interpreter/generator.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/generator.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/generator.py Wed Nov 24 14:21:00 2010 @@ -10,7 +10,7 @@ def __init__(self, frame): self.space = frame.space - self.frame = frame + self.frame = frame # turned into None when frame_finished_execution self.running = False def descr__repr__(self, space): @@ -25,9 +25,13 @@ mod = space.interp_w(MixedModule, w_mod) new_inst = mod.get('generator_new') w = space.wrap + if self.frame: + w_frame = w(self.frame) + else: + w_frame = space.w_None tup = [ - w(self.frame), + w_frame, w(self.running), ] @@ -47,7 +51,8 @@ if self.running: raise OperationError(space.w_ValueError, space.wrap('generator already executing')) - if self.frame.frame_finished_execution: + frame = self.frame + if frame is None: # xxx a bit ad-hoc, but we don't want to go inside # execute_generator_frame() if the frame is actually finished if operr is None: @@ -55,7 +60,7 @@ raise operr # XXX it's not clear that last_instr should be promoted at all # but as long as it is necessary for call_assembler, let's do it early - last_instr = jit.hint(self.frame.last_instr, promote=True) + last_instr = jit.hint(frame.last_instr, promote=True) if last_instr == -1: if w_arg and not space.is_w(w_arg, space.w_None): msg = "can't send non-None value to a just-started generator" @@ -66,18 +71,19 @@ self.running = True try: try: - w_result = self.frame.execute_generator_frame(w_arg, operr) + w_result = frame.execute_generator_frame(w_arg, operr) except OperationError: # errors finish a frame - self.frame.frame_finished_execution = True + self.frame = None raise # if the frame is now marked as finished, it was RETURNed from - if self.frame.frame_finished_execution: + if frame.frame_finished_execution: + self.frame = None raise OperationError(space.w_StopIteration, space.w_None) else: return w_result # YIELDed finally: - self.frame.f_backref = jit.vref_None + frame.f_backref = jit.vref_None self.running = False def descr_throw(self, w_type, w_val=None, w_tb=None): @@ -121,7 +127,7 @@ raise OperationError(space.w_RuntimeError, space.wrap(msg)) def descr_gi_frame(space, self): - if not self.frame.frame_finished_execution: + if self.frame is not None and not self.frame.frame_finished_execution: return self.frame else: return space.w_None @@ -138,15 +144,17 @@ applevel __del__, which is called at a safe point after the interp-level __del__ enqueued the object for destruction """ - # Only bother raising an exception if the frame is still not - # finished and finally or except blocks are present. - if not self.frame.frame_finished_execution: + self.descr_close() + + def __del__(self): + # Only bother enqueuing self to raise an exception if the frame is + # still not finished and finally or except blocks are present. + must_call_close = False + if self.frame is not None: block = self.frame.lastblock while block is not None: if not isinstance(block, LoopBlock): - self.descr_close() - return + must_call_close = True + break block = block.previous - - def __del__(self): - self._enqueue_for_destruction(self.space) + self._enqueue_for_destruction(self.space, must_call_close) Modified: pypy/branch/fast-forward/pypy/interpreter/test/test_argument.py ============================================================================== --- pypy/branch/fast-forward/pypy/interpreter/test/test_argument.py (original) +++ pypy/branch/fast-forward/pypy/interpreter/test/test_argument.py Wed Nov 24 14:21:00 2010 @@ -52,12 +52,17 @@ assert y == "d" assert z == "e" +class dummy_wrapped_dict(dict): + def __nonzero__(self): + raise NotImplementedError class DummySpace(object): def newtuple(self, items): return tuple(items) def is_true(self, obj): + if isinstance(obj, dummy_wrapped_dict): + return bool(dict(obj)) return bool(obj) def fixedview(self, it): @@ -244,7 +249,7 @@ kwds_w = dict(kwds[:i]) keywords = kwds_w.keys() keywords_w = kwds_w.values() - w_kwds = dict(kwds[i:]) + w_kwds = dummy_wrapped_dict(kwds[i:]) if i == 2: w_kwds = None assert len(keywords) == len(keywords_w) @@ -280,7 +285,7 @@ kwds_w = dict(kwds[:i]) keywords = kwds_w.keys() keywords_w = kwds_w.values() - w_kwds = dict(kwds[i:]) + w_kwds = dummy_wrapped_dict(kwds[i:]) if i == 3: w_kwds = None args = Arguments(space, [1, 2], keywords, keywords_w, w_starstararg=w_kwds) @@ -463,7 +468,11 @@ assert set(args.keywords) == set(['a', 'b']) assert args.keywords_w[args.keywords.index('a')] == 2 assert args.keywords_w[args.keywords.index('b')] == 3 - + + args = Arguments(space, [1]) + w_args, w_kwds = args.topacked() + assert w_args == (1, ) + assert not w_kwds class TestErrorHandling(object): def test_missing_args(self): Modified: pypy/branch/fast-forward/pypy/jit/backend/conftest.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/conftest.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/conftest.py Wed Nov 24 14:21:00 2010 @@ -4,8 +4,6 @@ """ import py, random -option = py.test.config.option - def pytest_addoption(parser): group = parser.getgroup('random test options') group.addoption('--random-seed', action="store", type="int", Modified: pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/llgraph/llimpl.py Wed Nov 24 14:21:00 2010 @@ -4,6 +4,7 @@ when executing on top of the llinterpreter. """ +import weakref from pypy.objspace.flow.model import Variable, Constant from pypy.annotation import model as annmodel from pypy.jit.metainterp.history import (ConstInt, ConstPtr, @@ -161,6 +162,8 @@ # ____________________________________________________________ class CompiledLoop(object): + has_been_freed = False + def __init__(self): self.inputargs = [] self.operations = [] @@ -285,6 +288,11 @@ del _variables[:] return _to_opaque(CompiledLoop()) +def mark_as_free(loop): + loop = _from_opaque(loop) + assert not loop.has_been_freed + loop.has_been_freed = True + def compile_start_int_var(loop): return compile_start_ref_var(loop, lltype.Signed) @@ -317,7 +325,7 @@ raise ValueError("CALL_ASSEMBLER not supported") loop = _from_opaque(loop) op = loop.operations[-1] - op.descr = descr + op.descr = weakref.ref(descr) def compile_add_var(loop, intvar): loop = _from_opaque(loop) @@ -429,6 +437,7 @@ verbose = True self.opindex = 0 while True: + assert not self.loop.has_been_freed op = self.loop.operations[self.opindex] args = [self.getenv(v) for v in op.args] if not op.is_final(): @@ -440,7 +449,10 @@ _stats.exec_conditional_jumps += 1 if op.jump_target is not None: # a patched guard, pointing to further code - args = [self.getenv(v) for v in op.fail_args if v] + if op.fail_args: + args = [self.getenv(v) for v in op.fail_args if v] + else: + args = [] assert len(op.jump_target.inputargs) == len(args) self.env = dict(zip(op.jump_target.inputargs, args)) self.loop = op.jump_target @@ -839,14 +851,22 @@ finally: self._may_force = -1 - def op_call_assembler(self, loop_token, *args): + def op_call_assembler(self, wref_loop_token, *args): + if we_are_translated(): + raise ValueError("CALL_ASSEMBLER not supported") + return self._do_call_assembler(wref_loop_token, *args) + + def _do_call_assembler(self, wref_loop_token, *args): global _last_exception + loop_token = wref_loop_token() + assert loop_token, "CALL_ASSEMBLER to a target that already died" + ctl = loop_token.compiled_loop_token + if hasattr(ctl, 'redirected'): + return self._do_call_assembler(ctl.redirected, *args) assert not self._forced - loop_token = self.cpu._redirected_call_assembler.get(loop_token, - loop_token) self._may_force = self.opindex try: - inpargs = _from_opaque(loop_token._llgraph_compiled_version).inputargs + inpargs = _from_opaque(ctl.compiled_version).inputargs for i, inparg in enumerate(inpargs): TYPE = inparg.concretetype if TYPE is lltype.Signed: @@ -1539,10 +1559,13 @@ do_setfield_gc_int(vable, fielddescr.ofs, 0) def redirect_call_assembler(cpu, oldlooptoken, newlooptoken): - OLD = _from_opaque(oldlooptoken._llgraph_compiled_version).getargtypes() - NEW = _from_opaque(newlooptoken._llgraph_compiled_version).getargtypes() + oldclt = oldlooptoken.compiled_loop_token + newclt = newlooptoken.compiled_loop_token + OLD = _from_opaque(oldclt.compiled_version).getargtypes() + NEW = _from_opaque(newclt.compiled_version).getargtypes() assert OLD == NEW - cpu._redirected_call_assembler[oldlooptoken] = newlooptoken + assert not hasattr(oldclt, 'redirected') + oldclt.redirected = weakref.ref(newlooptoken) # ____________________________________________________________ @@ -1609,6 +1632,7 @@ setannotation(compile_add_fail, annmodel.SomeInteger()) setannotation(compile_add_fail_arg, annmodel.s_None) setannotation(compile_redirect_fail, annmodel.s_None) +setannotation(mark_as_free, annmodel.s_None) setannotation(new_frame, s_Frame) setannotation(frame_clear, annmodel.s_None) Modified: pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/llgraph/runner.py Wed Nov 24 14:21:00 2010 @@ -102,7 +102,6 @@ llimpl._llinterp = LLInterpreter(self.rtyper) self._future_values = [] self._descrs = {} - self._redirected_call_assembler = {} def _freeze_(self): assert self.translate_support_code @@ -118,22 +117,34 @@ self._descrs[key] = descr return descr - def compile_bridge(self, faildescr, inputargs, operations, log=True): + def compile_bridge(self, faildescr, inputargs, operations, + original_loop_token, log=True): c = llimpl.compile_start() + clt = original_loop_token.compiled_loop_token + clt.loop_and_bridges.append(c) + clt.compiling_a_bridge() self._compile_loop_or_bridge(c, inputargs, operations) old, oldindex = faildescr._compiled_fail llimpl.compile_redirect_fail(old, oldindex, c) - def compile_loop(self, inputargs, operations, loopdescr, log=True): + def compile_loop(self, inputargs, operations, looptoken, log=True): """In a real assembler backend, this should assemble the given list of operations. Here we just generate a similar CompiledLoop instance. The code here is RPython, whereas the code in llimpl is not. """ c = llimpl.compile_start() - loopdescr._llgraph_compiled_version = c + clt = model.CompiledLoopToken(self, looptoken.number) + clt.loop_and_bridges = [c] + clt.compiled_version = c + looptoken.compiled_loop_token = clt self._compile_loop_or_bridge(c, inputargs, operations) + def free_loop_and_bridges(self, compiled_loop_token): + for c in compiled_loop_token.loop_and_bridges: + llimpl.mark_as_free(c) + model.AbstractCPU.free_loop_and_bridges(self, compiled_loop_token) + def _compile_loop_or_bridge(self, c, inputargs, operations): var2index = {} for box in inputargs: @@ -207,7 +218,7 @@ if op.getopnum() == rop.JUMP: targettoken = op.getdescr() assert isinstance(targettoken, history.LoopToken) - compiled_version = targettoken._llgraph_compiled_version + compiled_version = targettoken.compiled_loop_token.compiled_version llimpl.compile_add_jump_target(c, compiled_version) elif op.getopnum() == rop.FINISH: faildescr = op.getdescr() @@ -217,7 +228,7 @@ assert False, "unknown operation" def _execute_token(self, loop_token): - compiled_version = loop_token._llgraph_compiled_version + compiled_version = loop_token.compiled_loop_token.compiled_version frame = llimpl.new_frame(self.is_oo, self) # setup the frame llimpl.frame_clear(frame, compiled_version) Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/gc.py Wed Nov 24 14:21:00 2010 @@ -254,7 +254,7 @@ def _enlarge_gcmap(self): newlength = 250 + self._gcmap_maxlength * 2 newgcmap = lltype.malloc(self.GCMAP_ARRAY, newlength, flavor='raw', - track_allocation=False) + track_allocation=False) # YYY leak oldgcmap = self._gcmap for i in range(self._gcmap_curlength): newgcmap[i] = oldgcmap[i] @@ -311,7 +311,7 @@ length = len(shape) compressed = lltype.malloc(self.CALLSHAPE_ARRAY, length, flavor='raw', - track_allocation=False) # memory leak + track_allocation=False) # YYY leak for i in range(length): compressed[length-1-i] = rffi.cast(rffi.UCHAR, shape[i]) return llmemory.cast_ptr_to_adr(compressed) @@ -573,6 +573,9 @@ # GETFIELD_RAW from the array 'gcrefs.list'. # newops = [] + # we can only remember one malloc since the next malloc can possibly + # collect + last_malloc = None for op in operations: if op.getopnum() == rop.DEBUG_MERGE_POINT: continue @@ -590,22 +593,32 @@ [ConstInt(addr)], box, self.single_gcref_descr)) op.setarg(i, box) + if op.is_malloc(): + last_malloc = op.result + elif op.can_malloc(): + last_malloc = None # ---------- write barrier for SETFIELD_GC ---------- if op.getopnum() == rop.SETFIELD_GC: - v = op.getarg(1) - if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and - bool(v.value)): # store a non-NULL - self._gen_write_barrier(newops, op.getarg(0), v) - op = op.copy_and_change(rop.SETFIELD_RAW) + val = op.getarg(0) + # no need for a write barrier in the case of previous malloc + if val is not last_malloc: + v = op.getarg(1) + if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and + bool(v.value)): # store a non-NULL + self._gen_write_barrier(newops, op.getarg(0), v) + op = op.copy_and_change(rop.SETFIELD_RAW) # ---------- write barrier for SETARRAYITEM_GC ---------- if op.getopnum() == rop.SETARRAYITEM_GC: - v = op.getarg(2) - if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and - bool(v.value)): # store a non-NULL - # XXX detect when we should produce a - # write_barrier_from_array - self._gen_write_barrier(newops, op.getarg(0), v) - op = op.copy_and_change(rop.SETARRAYITEM_RAW) + val = op.getarg(0) + # no need for a write barrier in the case of previous malloc + if val is not last_malloc: + v = op.getarg(2) + if isinstance(v, BoxPtr) or (isinstance(v, ConstPtr) and + bool(v.value)): # store a non-NULL + # XXX detect when we should produce a + # write_barrier_from_array + self._gen_write_barrier(newops, op.getarg(0), v) + op = op.copy_and_change(rop.SETARRAYITEM_RAW) # ---------- newops.append(op) del operations[:] Modified: pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_gc.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_gc.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/llsupport/test/test_gc.py Wed Nov 24 14:21:00 2010 @@ -6,7 +6,9 @@ from pypy.jit.backend.llsupport.gc import * from pypy.jit.backend.llsupport import symbolic from pypy.jit.metainterp.gc import get_description - +from pypy.jit.tool.oparser import parse +from pypy.rpython.lltypesystem.rclass import OBJECT, OBJECT_VTABLE +from pypy.jit.metainterp.test.test_optimizeopt import equaloplists def test_boehm(): gc_ll_descr = GcLLDescr_boehm(None, None, None) @@ -114,7 +116,7 @@ assert gcrootmap._gcmap[i*2+1] == expected_shapeaddr[i] -class FakeLLOp: +class FakeLLOp(object): def __init__(self): self.record = [] @@ -148,19 +150,19 @@ return llhelper(FPTRTYPE, self._write_barrier_failing_case) -class TestFramework: +class TestFramework(object): gc = 'hybrid' def setup_method(self, meth): - class config_: - class translation: + class config_(object): + class translation(object): gc = self.gc gcrootfinder = 'asmgcc' gctransformer = 'framework' gcremovetypeptr = False - class FakeTranslator: + class FakeTranslator(object): config = config_ - class FakeCPU: + class FakeCPU(object): def cast_adr_to_int(self, adr): ptr = llmemory.cast_adr_to_ptr(adr, gc_ll_descr.WB_FUNCPTR) assert ptr._obj._callable == llop1._write_barrier_failing_case @@ -278,11 +280,11 @@ def test_rewrite_assembler_1(self): # check rewriting of ConstPtrs - class MyFakeCPU: + class MyFakeCPU(object): def cast_adr_to_int(self, adr): assert adr == "some fake address" return 43 - class MyFakeGCRefList: + class MyFakeGCRefList(object): def get_address_of_gcref(self, s_gcref1): assert s_gcref1 == s_gcref return "some fake address" @@ -311,10 +313,10 @@ def test_rewrite_assembler_1_cannot_move(self): # check rewriting of ConstPtrs - class MyFakeCPU: + class MyFakeCPU(object): def cast_adr_to_int(self, adr): xxx # should not be called - class MyFakeGCRefList: + class MyFakeGCRefList(object): def get_address_of_gcref(self, s_gcref1): seen.append(s_gcref1) assert s_gcref1 == s_gcref @@ -394,6 +396,68 @@ assert operations[1].getarg(2) == v_value assert operations[1].getdescr() == array_descr + def test_rewrite_assembler_initialization_store(self): + S = lltype.GcStruct('S', ('parent', OBJECT), + ('x', lltype.Signed)) + s_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True) + xdescr = get_field_descr(self.gc_ll_descr, S, 'x') + ops = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + # no write barrier + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) + + def test_rewrite_assembler_initialization_store_2(self): + S = lltype.GcStruct('S', ('parent', OBJECT), + ('x', lltype.Signed)) + s_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True) + wbdescr = self.gc_ll_descr.write_barrier_descr + xdescr = get_field_descr(self.gc_ll_descr, S, 'x') + ops = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + p3 = new_with_vtable(ConstClass(s_vtable)) + setfield_gc(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_with_vtable(ConstClass(s_vtable)) + p3 = new_with_vtable(ConstClass(s_vtable)) + cond_call_gc_wb(p0, p1, descr=wbdescr) + setfield_raw(p0, p1, descr=xdescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) + + def test_rewrite_assembler_initialization_store_3(self): + A = lltype.GcArray(lltype.Ptr(lltype.GcStruct('S'))) + arraydescr = get_array_descr(self.gc_ll_descr, A) + ops = parse(""" + [p1] + p0 = new_array(3, descr=arraydescr) + setarrayitem_gc(p0, 0, p1, descr=arraydescr) + jump() + """, namespace=locals()) + expected = parse(""" + [p1] + p0 = new_array(3, descr=arraydescr) + setarrayitem_gc(p0, 0, p1, descr=arraydescr) + jump() + """, namespace=locals()) + self.gc_ll_descr.rewrite_assembler(self.fake_cpu, ops.operations) + equaloplists(ops.operations, expected.operations) class TestFrameworkMiniMark(TestFramework): gc = 'minimark' Modified: pypy/branch/fast-forward/pypy/jit/backend/model.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/model.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/model.py Wed Nov 24 14:21:00 2010 @@ -1,3 +1,4 @@ +from pypy.rlib.debug import debug_start, debug_print, debug_stop from pypy.jit.metainterp import history, compile @@ -7,17 +8,27 @@ done_with_this_frame_int_v = -1 done_with_this_frame_ref_v = -1 done_with_this_frame_float_v = -1 + total_compiled_loops = 0 + total_compiled_bridges = 0 + total_freed_loops = 0 + total_freed_bridges = 0 def __init__(self): self.fail_descr_list = [] + self.fail_descr_free_list = [] def get_fail_descr_number(self, descr): assert isinstance(descr, history.AbstractFailDescr) n = descr.index if n < 0: lst = self.fail_descr_list - n = len(lst) - lst.append(descr) + if len(self.fail_descr_free_list) > 0: + n = self.fail_descr_free_list.pop() + assert lst[n] is None + lst[n] = descr + else: + n = len(lst) + lst.append(descr) descr.index = n return n @@ -35,12 +46,14 @@ def compile_loop(self, inputargs, operations, looptoken, log=True): """Assemble the given loop. - Extra attributes should be put in the LoopToken to - point to the compiled loop in assembler. + Should create and attach a fresh CompiledLoopToken to + looptoken.compiled_loop_token and stick extra attributes + on it to point to the compiled loop in assembler. """ raise NotImplementedError - def compile_bridge(self, faildescr, inputargs, operations, log=True): + def compile_bridge(self, faildescr, inputargs, operations, + original_loop_token, log=True): """Assemble the bridge. The FailDescr is the descr of the original guard that failed. """ @@ -113,6 +126,24 @@ oldlooptoken so that from now own they will call newlooptoken.""" raise NotImplementedError + def free_loop_and_bridges(self, compiled_loop_token): + """This method is called to free resources (machine code, + references to resume guards, etc.) allocated by the compilation + of a loop and all bridges attached to it. After this call, the + frontend cannot use this compiled loop any more; in fact, it + guarantees that at the point of the call to free_code_group(), + none of the corresponding assembler is currently running. + """ + # The base class provides a limited implementation: freeing the + # resume descrs. This is already quite helpful, because the + # resume descrs are the largest consumers of memory (about 3x + # more than the assembler, in the case of the x86 backend). + lst = self.fail_descr_list + for n in compiled_loop_token.faildescr_indices: + lst[n] = None + self.fail_descr_free_list.extend(compiled_loop_token.faildescr_indices) + # We expect 'compiled_loop_token' to be itself garbage-collected soon. + @staticmethod def sizeof(S): raise NotImplementedError @@ -237,3 +268,37 @@ def force(self, force_token): raise NotImplementedError + + +class CompiledLoopToken(object): + def __init__(self, cpu, number): + cpu.total_compiled_loops += 1 + self.cpu = cpu + self.number = number + self.bridges_count = 0 + # This growing list gives the 'descr_number' of all fail descrs + # that belong to this loop or to a bridge attached to it. + # Filled by the frontend calling record_faildescr_index(). + self.faildescr_indices = [] + debug_start("jit-mem-looptoken-alloc") + debug_print("allocating Loop #", self.number) + debug_stop("jit-mem-looptoken-alloc") + + def record_faildescr_index(self, n): + self.faildescr_indices.append(n) + + def compiling_a_bridge(self): + self.cpu.total_compiled_bridges += 1 + self.bridges_count += 1 + debug_start("jit-mem-looptoken-alloc") + debug_print("allocating Bridge #", self.bridges_count, "of Loop #", self.number) + debug_stop("jit-mem-looptoken-alloc") + + def __del__(self): + debug_start("jit-mem-looptoken-free") + debug_print("freeing Loop #", self.number, 'with', + self.bridges_count, 'attached bridges') + self.cpu.free_loop_and_bridges(self) + self.cpu.total_freed_loops += 1 + self.cpu.total_freed_bridges += self.bridges_count + debug_stop("jit-mem-looptoken-free") Modified: pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/test/runner_test.py Wed Nov 24 14:21:00 2010 @@ -174,6 +174,8 @@ assert not wr_i1() and not wr_guard() def test_compile_bridge(self): + self.cpu.total_compiled_loops = 0 + self.cpu.total_compiled_bridges = 0 i0 = BoxInt() i1 = BoxInt() i2 = BoxInt() @@ -199,7 +201,7 @@ ] bridge[1].setfailargs([i1b]) - self.cpu.compile_bridge(faildescr1, [i1b], bridge) + self.cpu.compile_bridge(faildescr1, [i1b], bridge, looptoken) self.cpu.set_future_value_int(0, 2) fail = self.cpu.execute_token(looptoken) @@ -207,6 +209,9 @@ res = self.cpu.get_latest_value_int(0) assert res == 20 + assert self.cpu.total_compiled_loops == 1 + assert self.cpu.total_compiled_bridges == 1 + def test_compile_bridge_with_holes(self): i0 = BoxInt() i1 = BoxInt() @@ -233,7 +238,7 @@ ] bridge[1].setfailargs([i1b]) - self.cpu.compile_bridge(faildescr1, [i1b], bridge) + self.cpu.compile_bridge(faildescr1, [i1b], bridge, looptoken) self.cpu.set_future_value_int(0, 2) fail = self.cpu.execute_token(looptoken) @@ -1050,7 +1055,7 @@ ResOperation(rop.JUMP, [f3] + fboxes2[1:], None, descr=looptoken), ] - self.cpu.compile_bridge(faildescr1, fboxes2, bridge) + self.cpu.compile_bridge(faildescr1, fboxes2, bridge, looptoken) for i in range(len(fboxes)): self.cpu.set_future_value_float(i, 13.5 + 6.73 * i) Modified: pypy/branch/fast-forward/pypy/jit/backend/test/test_random.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/test/test_random.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/test/test_random.py Wed Nov 24 14:21:00 2010 @@ -524,7 +524,7 @@ self.prebuilt_ptr_consts = [] self.r = r self.build_random_loop(cpu, builder_factory, r, startvars) - + def build_random_loop(self, cpu, builder_factory, r, startvars): loop = TreeLoop('test_random_function') @@ -685,11 +685,12 @@ subloop.operations[-1] = jump_op self.guard_op = rl.guard_op self.prebuilt_ptr_consts += rl.prebuilt_ptr_consts + self.loop.token.record_jump_to(rl.loop.token) self.dont_generate_more = True if r.random() < .05: return False self.builder.cpu.compile_bridge(fail_descr, fail_args, - subloop.operations) + subloop.operations, self.loop.token) return True def check_random_function(cpu, BuilderClass, r, num=None, max=None): Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/x86/assembler.py Wed Nov 24 14:21:00 2010 @@ -7,6 +7,7 @@ from pypy.rpython.lltypesystem.lloperation import llop from pypy.rpython.annlowlevel import llhelper from pypy.tool.uid import fixid +from pypy.jit.backend.model import CompiledLoopToken from pypy.jit.backend.x86.regalloc import (RegAlloc, X86RegisterManager, X86XMMRegisterManager, get_ebp_ofs, _get_scale) @@ -30,10 +31,9 @@ from pypy.jit.backend.x86 import rx86, regloc, codebuf from pypy.jit.metainterp.resoperation import rop, ResOperation from pypy.jit.backend.x86.support import values_array -from pypy.rlib.debug import debug_print +from pypy.rlib.debug import debug_print, debug_start, debug_stop from pypy.rlib import rgc from pypy.jit.backend.x86.jump import remap_frame_layout -from pypy.rlib.streamio import open_file_as_stream from pypy.jit.metainterp.history import ConstInt, BoxInt # darwin requires the stack to be 16 bytes aligned on calls. Same for gcc 4.5.0, @@ -231,7 +231,6 @@ if s.find(':') != -1: s = s.split(':')[-1] self.set_debug(True) - self._output_loop_log = s + ".count" # Intialize here instead of __init__ to prevent # pending_guard_tokens from being considered a prebuilt object, # which sometimes causes memory leaks since the prebuilt list is @@ -241,13 +240,11 @@ def finish_once(self): if self._debug: - output_log = self._output_loop_log - assert output_log is not None - f = open_file_as_stream(output_log, "w") + debug_start('jit-backend-counts') for i in range(len(self.loop_run_counters)): name, struct = self.loop_run_counters[i] - f.write(str(name) + ":" + str(struct.i) + "\n") - f.close() + debug_print(str(name) + ':' + str(struct.i)) + debug_stop('jit-backend-counts') def _build_float_constants(self): # 44 bytes: 32 bytes for the data, and up to 12 bytes for alignment @@ -305,6 +302,8 @@ _x86_arglocs _x86_debug_checksum """ + looptoken.compiled_loop_token = CompiledLoopToken(self.cpu, + looptoken.number) if not we_are_translated(): # Arguments should be unique assert len(set(inputargs)) == len(inputargs) @@ -409,8 +408,9 @@ def _register_counter(self): if self._debug: + # YYY leak -- just put it in self.mc instead struct = lltype.malloc(DEBUG_COUNTER, flavor='raw', - track_allocation=False) # known to leak + track_allocation=False) struct.i = 0 self.loop_run_counters.append((len(self.loop_run_counters), struct)) @@ -1773,13 +1773,18 @@ self.implement_guard(guard_token, 'L') def genop_discard_cond_call_gc_wb(self, op, arglocs): - # use 'mc._mc' directly instead of 'mc', to avoid - # bad surprizes if the code buffer is mostly full + # Write code equivalent to write_barrier() in the GC: it checks + # a flag in the object at arglocs[0], and if set, it calls the + # function remember_young_pointer() from the GC. The two arguments + # to the call are in arglocs[:2]. The rest, arglocs[2:], contains + # registers that need to be saved and restored across the call. descr = op.getdescr() if we_are_translated(): cls = self.cpu.gc_ll_descr.has_write_barrier_class() assert cls is not None and isinstance(descr, cls) loc_base = arglocs[0] + # ensure that enough bytes are available to write the whole + # following piece of code atomically (for the JZ) self.mc.ensure_bytes_available(256) self.mc.TEST8_mi((loc_base.value, descr.jit_wb_if_flag_byteofs), descr.jit_wb_if_flag_singlebyte) @@ -1787,36 +1792,39 @@ jz_location = self.mc.get_relative_pos() # the following is supposed to be the slow path, so whenever possible # we choose the most compact encoding over the most efficient one. - # XXX improve a bit, particularly for IS_X86_64. - for i in range(len(arglocs)-1, -1, -1): + if IS_X86_32: + limit = -1 # push all arglocs on the stack + elif IS_X86_64: + limit = 1 # push only arglocs[2:] on the stack + for i in range(len(arglocs)-1, limit, -1): loc = arglocs[i] if isinstance(loc, RegLoc): self.mc.PUSH_r(loc.value) else: - if IS_X86_64: - self.mc.MOV_ri(X86_64_SCRATCH_REG.value, loc.getint()) - self.mc.PUSH_r(X86_64_SCRATCH_REG.value) - else: - self.mc.PUSH_i32(loc.getint()) - + assert not IS_X86_64 # there should only be regs in arglocs[2:] + self.mc.PUSH_i32(loc.getint()) if IS_X86_64: # We clobber these registers to pass the arguments, but that's # okay, because consider_cond_call_gc_wb makes sure that any - # caller-save registers with values in them are present in arglocs, - # so they are saved on the stack above and restored below - self.mc.MOV_rs(edi.value, 0) - self.mc.MOV_rs(esi.value, 8) + # caller-save registers with values in them are present in + # arglocs[2:] too, so they are saved on the stack above and + # restored below. + remap_frame_layout(self, arglocs[:2], [edi, esi], + X86_64_SCRATCH_REG) # misaligned stack in the call, but it's ok because the write barrier # is not going to call anything more. Also, this assumes that the - # write barrier does not touch the xmm registers. + # write barrier does not touch the xmm registers. (Slightly delicate + # assumption, given that the write barrier can end up calling the + # platform's malloc() from AddressStack.append(). XXX may need to + # be done properly) self.mc.CALL(imm(descr.get_write_barrier_fn(self.cpu))) - for i in range(len(arglocs)): + if IS_X86_32: + self.mc.ADD_ri(esp.value, 2*WORD) + for i in range(2, len(arglocs)): loc = arglocs[i] - if isinstance(loc, RegLoc): - self.mc.POP_r(loc.value) - else: - self.mc.ADD_ri(esp.value, WORD) # ignore the pushed constant + assert isinstance(loc, RegLoc) + self.mc.POP_r(loc.value) # patch the JZ above offset = self.mc.get_relative_pos() - jz_location assert 0 < offset <= 127 Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/x86/regalloc.py Wed Nov 24 14:21:00 2010 @@ -70,7 +70,7 @@ def _get_new_array(self): n = self.BASE_CONSTANT_SIZE # known to leak - self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n, + self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n, # YYY leak flavor='raw', track_allocation=False) self.cur_array_free = n _get_new_array._dont_inline_ = True Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/x86/runner.py Wed Nov 24 14:21:00 2010 @@ -53,7 +53,10 @@ self.assembler.assemble_loop(inputargs, operations, looptoken, log=log) - def compile_bridge(self, faildescr, inputargs, operations, log=True): + def compile_bridge(self, faildescr, inputargs, operations, + original_loop_token, log=True): + clt = original_loop_token.compiled_loop_token + clt.compiling_a_bridge() self.assembler.assemble_bridge(faildescr, inputargs, operations, log=log) Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_regalloc.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_regalloc.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_regalloc.py Wed Nov 24 14:21:00 2010 @@ -166,7 +166,8 @@ assert ([box.type for box in bridge.inputargs] == [box.type for box in guard_op.getfailargs()]) faildescr = guard_op.getdescr() - self.cpu.compile_bridge(faildescr, bridge.inputargs, bridge.operations) + self.cpu.compile_bridge(faildescr, bridge.inputargs, bridge.operations, + loop.token) return bridge def run(self, loop): Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_runner.py Wed Nov 24 14:21:00 2010 @@ -338,6 +338,7 @@ faildescr1 = BasicFailDescr(1) faildescr2 = BasicFailDescr(2) looptoken = LoopToken() + looptoken.number = 17 class FakeString(object): def __init__(self, val): self.val = val @@ -356,7 +357,7 @@ operations[3].setfailargs([i1]) self.cpu.compile_loop(inputargs, operations, looptoken) name, loopaddress, loopsize = agent.functions[0] - assert name == "Loop # 0: hello" + assert name == "Loop # 17: hello" assert loopaddress <= looptoken._x86_loop_code assert loopsize >= 40 # randomish number @@ -370,7 +371,7 @@ ] bridge[1].setfailargs([i1b]) - self.cpu.compile_bridge(faildescr1, [i1b], bridge) + self.cpu.compile_bridge(faildescr1, [i1b], bridge, looptoken) name, address, size = agent.functions[1] assert name == "Bridge # 0: bye" # Would be exactly ==, but there are some guard failure recovery @@ -495,6 +496,9 @@ os.environ['PYPYLOG'] = self.pypylog def test_debugger_on(self): + from pypy.tool.logparser import parse_log_file, extract_category + from pypy.rlib import debug + loop = """ [i0] debug_merge_point('xyz', 0) @@ -504,17 +508,21 @@ jump(i1) """ ops = parse(loop) - self.cpu.assembler.set_debug(True) - self.cpu.compile_loop(ops.inputargs, ops.operations, ops.token) - self.cpu.set_future_value_int(0, 0) - self.cpu.execute_token(ops.token) - # check debugging info - name, struct = self.cpu.assembler.loop_run_counters[0] - assert name == 0 # 'xyz' - assert struct.i == 10 - self.cpu.finish_once() - lines = py.path.local(self.logfile + ".count").readlines() - assert lines[0] == '0:10\n' # '10 xyz\n' + debug._log = dlog = debug.DebugLog() + try: + self.cpu.assembler.set_debug(True) + self.cpu.compile_loop(ops.inputargs, ops.operations, ops.token) + self.cpu.set_future_value_int(0, 0) + self.cpu.execute_token(ops.token) + # check debugging info + name, struct = self.cpu.assembler.loop_run_counters[0] + assert name == 0 # 'xyz' + assert struct.i == 10 + self.cpu.finish_once() + finally: + debug._log = None + assert dlog[1] == ('jit-backend-counts', [('debug_print', '0:10')]) + def test_debugger_checksum(self): loop = """ Modified: pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py (original) +++ pypy/branch/fast-forward/pypy/jit/backend/x86/test/test_ztranslation.py Wed Nov 24 14:21:00 2010 @@ -87,8 +87,13 @@ return int(res) # def main(i, j): - return f(i, j) + libffi_stuff(i, j) - expected = main(40, -49) + res = f(i, j) + libffi_stuff(i, j) + fd = os.open('/tmp/x', os.O_WRONLY | os.O_CREAT, 0666) + rgc.dump_rpy_heap(fd) + os.close(fd) + return res + #expected = main(40, -49) + expected = 3 res = self.meta_interp(main, [40, -49]) assert res == expected Modified: pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py (original) +++ pypy/branch/fast-forward/pypy/jit/codewriter/jtransform.py Wed Nov 24 14:21:00 2010 @@ -843,9 +843,16 @@ "general mix-up of jitdrivers?") ops = self.promote_greens(op.args[2:], jitdriver) num_green_args = len(jitdriver.greens) + redlists = self.make_three_lists(op.args[2+num_green_args:]) + for redlist in redlists: + for v in redlist: + assert isinstance(v, Variable), ( + "Constant specified red in jit_merge_point()") + assert len(dict.fromkeys(redlist)) == len(list(redlist)), ( + "duplicate red variable on jit_merge_point()") args = ([Constant(self.portal_jd.index, lltype.Signed)] + self.make_three_lists(op.args[2:2+num_green_args]) + - self.make_three_lists(op.args[2+num_green_args:])) + redlists) op1 = SpaceOperation('jit_merge_point', args, None) op2 = SpaceOperation('-live-', [], None) # ^^^ we need a -live- for the case of do_recursive_call() Modified: pypy/branch/fast-forward/pypy/jit/conftest.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/conftest.py (original) +++ pypy/branch/fast-forward/pypy/jit/conftest.py Wed Nov 24 14:21:00 2010 @@ -1,7 +1,5 @@ import py -option = py.test.config.option - def pytest_addoption(parser): group = parser.getgroup("JIT options") group.addoption('--slow', action="store_true", Modified: pypy/branch/fast-forward/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/compile.py Wed Nov 24 14:21:00 2010 @@ -1,4 +1,5 @@ - +import weakref +from pypy.rpython.lltypesystem import lltype from pypy.rpython.ootypesystem import ootype from pypy.objspace.flow.model import Constant, Variable from pypy.rlib.objectmodel import we_are_translated @@ -14,6 +15,7 @@ from pypy.jit.metainterp.specnode import NotSpecNode, more_general_specnodes from pypy.jit.metainterp.typesystem import llhelper, oohelper from pypy.jit.metainterp.optimizeutil import InvalidLoop +from pypy.jit.metainterp.resume import NUMBERING from pypy.jit.codewriter import heaptracker def giveup(): @@ -46,15 +48,40 @@ loop_token.outermost_jitdriver_sd = jitdriver_sd return loop_token +def record_loop_or_bridge(loop): + """Do post-backend recordings and cleanups on 'loop'. + """ + # get the original loop token (corresponding to 'loop', or if that is + # a bridge, to the loop that this bridge belongs to) + looptoken = loop.token + assert looptoken is not None + wref = weakref.ref(looptoken) + for op in loop.operations: + descr = op.getdescr() + if isinstance(descr, ResumeDescr): + descr.wref_original_loop_token = wref # stick it there + n = descr.index + if n >= 0: # we also record the resumedescr number + looptoken.compiled_loop_token.record_faildescr_index(n) + elif isinstance(descr, LoopToken): + # for a JUMP or a CALL_ASSEMBLER: record it as a potential jump. + # (the following test is not enough to prevent more complicated + # cases of cycles, but at least it helps in simple tests of + # test_memgr.py) + if descr is not looptoken: + looptoken.record_jump_to(descr) + op.setdescr(None) # clear reference, mostly for tests + # mostly for tests: make sure we don't keep a reference to the LoopToken + loop.token = None + # ____________________________________________________________ -def compile_new_loop(metainterp, old_loop_tokens, greenkey, start): +def compile_new_loop(metainterp, old_loop_tokens, start): """Try to compile a new loop by closing the current history back to the first operation. """ history = metainterp.history loop = create_empty_loop(metainterp) - loop.greenkey = greenkey loop.inputargs = history.inputargs for box in loop.inputargs: assert isinstance(box, Box) @@ -76,6 +103,7 @@ return old_loop_token send_loop_to_backend(metainterp_sd, loop, "loop") insert_loop_token(old_loop_tokens, loop_token) + record_loop_or_bridge(loop) return loop_token def insert_loop_token(old_loop_tokens, loop_token): @@ -116,8 +144,11 @@ else: loop._ignore_during_counting = True metainterp_sd.log("compiled new " + type) + if metainterp_sd.warmrunnerdesc is not None: # for tests + metainterp_sd.warmrunnerdesc.memory_manager.keep_loop_alive(loop.token) -def send_bridge_to_backend(metainterp_sd, faildescr, inputargs, operations): +def send_bridge_to_backend(metainterp_sd, faildescr, inputargs, operations, + original_loop_token): n = metainterp_sd.cpu.get_fail_descr_number(faildescr) metainterp_sd.logger_ops.log_bridge(inputargs, operations, n) if not we_are_translated(): @@ -126,7 +157,8 @@ metainterp_sd.profiler.start_backend() debug_start("jit-backend") try: - metainterp_sd.cpu.compile_bridge(faildescr, inputargs, operations) + metainterp_sd.cpu.compile_bridge(faildescr, inputargs, operations, + original_loop_token) finally: debug_stop("jit-backend") metainterp_sd.profiler.end_backend() @@ -207,8 +239,7 @@ } class ResumeDescr(AbstractFailDescr): - def __init__(self, original_greenkey): - self.original_greenkey = original_greenkey + pass class ResumeGuardDescr(ResumeDescr): _counter = 0 # if < 0, there is one counter per value; @@ -217,7 +248,7 @@ # this class also gets the following attributes stored by resume.py code rd_snapshot = None rd_frame_info_list = None - rd_numb = None + rd_numb = lltype.nullptr(NUMBERING) rd_consts = None rd_virtuals = None rd_pendingfields = None @@ -227,10 +258,6 @@ CNT_FLOAT = -0x60000000 CNT_MASK = 0x1FFFFFFF - def __init__(self, metainterp_sd, original_greenkey): - ResumeDescr.__init__(self, original_greenkey) - self.metainterp_sd = metainterp_sd - def store_final_boxes(self, guard_op, boxes): guard_op.setfailargs(boxes) self.guard_opnum = guard_op.getopnum() @@ -315,12 +342,14 @@ def compile_and_attach(self, metainterp, new_loop): # We managed to create a bridge. Attach the new operations - # to the corrsponding guard_op and compile from there + # to the corresponding guard_op and compile from there + assert metainterp.resumekey_original_loop_token is not None + new_loop.token = metainterp.resumekey_original_loop_token inputargs = metainterp.history.inputargs if not we_are_translated(): self._debug_suboperations = new_loop.operations send_bridge_to_backend(metainterp.staticdata, self, inputargs, - new_loop.operations) + new_loop.operations, new_loop.token) def copy_all_attrbutes_into(self, res): # XXX a bit ugly to have to list them all here @@ -332,14 +361,14 @@ res.rd_pendingfields = self.rd_pendingfields def _clone_if_mutable(self): - res = ResumeGuardDescr(self.metainterp_sd, self.original_greenkey) + res = ResumeGuardDescr() self.copy_all_attrbutes_into(res) return res class ResumeGuardForcedDescr(ResumeGuardDescr): - def __init__(self, metainterp_sd, original_greenkey, jitdriver_sd): - ResumeGuardDescr.__init__(self, metainterp_sd, original_greenkey) + def __init__(self, metainterp_sd, jitdriver_sd): + self.metainterp_sd = metainterp_sd self.jitdriver_sd = jitdriver_sd def handle_fail(self, metainterp_sd, jitdriver_sd): @@ -406,7 +435,6 @@ def _clone_if_mutable(self): res = ResumeGuardForcedDescr(self.metainterp_sd, - self.original_greenkey, self.jitdriver_sd) self.copy_all_attrbutes_into(res) return res @@ -473,9 +501,8 @@ class ResumeFromInterpDescr(ResumeDescr): - def __init__(self, original_greenkey, redkey): - ResumeDescr.__init__(self, original_greenkey) - self.redkey = redkey + def __init__(self, original_greenkey): + self.original_greenkey = original_greenkey def compile_and_attach(self, metainterp, new_loop): # We managed to create a bridge going from the interpreter @@ -484,22 +511,24 @@ # with completely unoptimized arguments, as in the interpreter. metainterp_sd = metainterp.staticdata jitdriver_sd = metainterp.jitdriver_sd - metainterp.history.inputargs = self.redkey - new_loop_token = make_loop_token(len(self.redkey), jitdriver_sd) - new_loop.greenkey = self.original_greenkey - new_loop.inputargs = self.redkey + redargs = new_loop.inputargs + # We make a new LoopToken for this entry bridge, and stick it + # to every guard in the loop. + new_loop_token = make_loop_token(len(redargs), jitdriver_sd) new_loop.token = new_loop_token send_loop_to_backend(metainterp_sd, new_loop, "entry bridge") # send the new_loop to warmspot.py, to be called directly the next time jitdriver_sd.warmstate.attach_unoptimized_bridge_from_interp( self.original_greenkey, new_loop_token) - # store the new loop in compiled_merge_points too + # store the new loop in compiled_merge_points_wref too old_loop_tokens = metainterp.get_compiled_merge_points( self.original_greenkey) # it always goes at the end of the list, as it is the most # general loop token old_loop_tokens.append(new_loop_token) + metainterp.set_compiled_merge_points(self.original_greenkey, + old_loop_tokens) def reset_counter_from_failure(self): pass @@ -534,6 +563,7 @@ # know exactly what we must do (ResumeGuardDescr/ResumeFromInterpDescr) prepare_last_operation(new_loop, target_loop_token) resumekey.compile_and_attach(metainterp, new_loop) + record_loop_or_bridge(new_loop) return target_loop_token def prepare_last_operation(new_loop, target_loop_token): @@ -560,7 +590,8 @@ propagate_exception_descr = PropagateExceptionDescr() -def compile_tmp_callback(cpu, jitdriver_sd, greenboxes, redboxes): +def compile_tmp_callback(cpu, jitdriver_sd, greenboxes, redboxes, + memory_manager=None): """Make a LoopToken that corresponds to assembler code that just calls back the interpreter. Used temporarily: a fully compiled version of the code may end up replacing it. @@ -600,4 +631,6 @@ ] operations[1].setfailargs([]) cpu.compile_loop(inputargs, operations, loop_token, log=False) + if memory_manager is not None: # for tests + memory_manager.keep_loop_alive(loop_token) return loop_token Modified: pypy/branch/fast-forward/pypy/jit/metainterp/history.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/history.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/history.py Wed Nov 24 14:21:00 2010 @@ -4,7 +4,7 @@ from pypy.rpython.ootypesystem import ootype from pypy.rlib.objectmodel import we_are_translated, r_dict, Symbolic from pypy.rlib.objectmodel import compute_hash, compute_unique_id -from pypy.rlib.rarithmetic import intmask +from pypy.rlib.rarithmetic import intmask, r_longlong from pypy.tool.uid import uid from pypy.conftest import option @@ -730,14 +730,28 @@ outermost_jitdriver_sd = None # specnodes = ... # and more data specified by the backend when the loop is compiled - number = 0 + number = -1 + generation = r_longlong(0) + # one purpose of LoopToken is to keep alive the CompiledLoopToken + # returned by the backend. When the LoopToken goes away, the + # CompiledLoopToken has its __del__ called, which frees the assembler + # memory and the ResumeGuards. + compiled_loop_token = None - def __init__(self, number=0): - self.number = number + def __init__(self): + # For memory management of assembled loops + self._keepalive_target_looktokens = {} # set of other LoopTokens + + def record_jump_to(self, target_loop_token): + self._keepalive_target_looktokens[target_loop_token] = None + + def __repr__(self): + return '' % (self.number, self.generation) def repr_of_descr(self): return '' % self.number + class TreeLoop(object): inputargs = None operations = None Modified: pypy/branch/fast-forward/pypy/jit/metainterp/jitprof.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/jitprof.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/jitprof.py Wed Nov 24 14:21:00 2010 @@ -2,9 +2,9 @@ """ A small helper module for profiling JIT """ -import os import time -from pypy.rlib.debug import debug_print +from pypy.rlib.debug import debug_print, debug_start, debug_stop +from pypy.rlib.debug import have_debug_prints from pypy.jit.metainterp.jitexc import JitException counters=""" @@ -24,6 +24,10 @@ NVIRTUALS NVHOLES NVREUSED +TOTAL_COMPILED_LOOPS +TOTAL_COMPILED_BRIDGES +TOTAL_FREED_LOOPS +TOTAL_FREED_BRIDGES """ def _setup(): @@ -35,6 +39,7 @@ _setup() JITPROF_LINES = ncounters + 1 + 1 # one for TOTAL, 1 for calls, update if needed +_CPU_LINES = 4 # the last 4 lines are stored on the cpu class BaseProfiler(object): pass @@ -48,9 +53,6 @@ def finish(self): pass - def set_printing(self, printing): - pass - def start_tracing(self): pass @@ -90,23 +92,19 @@ counters = None calls = 0 current = None - printing = True + cpu = None def start(self): self.starttime = self.timer() self.t1 = self.starttime self.times = [0, 0] - self.counters = [0] * ncounters + self.counters = [0] * (ncounters - _CPU_LINES) self.calls = 0 self.current = [] def finish(self): self.tk = self.timer() - if self.printing: - self.print_stats() - - def set_printing(self, printing): - self.printing = printing + self.print_stats() def _start(self, event): t0 = self.t1 @@ -154,6 +152,12 @@ self.calls += 1 def print_stats(self): + debug_start("jit-summary") + if have_debug_prints(): + self._print_stats() + debug_stop("jit-summary") + + def _print_stats(self): cnt = self.counters tim = self.times calls = self.calls @@ -161,8 +165,8 @@ self._print_line_time("Backend", cnt[BACKEND], tim[BACKEND]) self._print_intline("Running asm", cnt[RUNNING]) self._print_intline("Blackhole", cnt[BLACKHOLE]) - line = "TOTAL: \t\t%f\n" % (self.tk - self.starttime, ) - os.write(2, line) + line = "TOTAL: \t\t%f" % (self.tk - self.starttime, ) + debug_print(line) self._print_intline("ops", cnt[OPS]) self._print_intline("recorded ops", cnt[RECORDED_OPS]) self._print_intline(" calls", calls) @@ -176,17 +180,26 @@ self._print_intline("nvirtuals", cnt[NVIRTUALS]) self._print_intline("nvholes", cnt[NVHOLES]) self._print_intline("nvreused", cnt[NVREUSED]) + cpu = self.cpu + if cpu is not None: # for some tests + self._print_intline("Total # of loops", + cpu.total_compiled_loops) + self._print_intline("Total # of bridges", + cpu.total_compiled_bridges) + self._print_intline("Freed # of loops", + cpu.total_freed_loops) + self._print_intline("Freed # of bridges", + cpu.total_freed_bridges) def _print_line_time(self, string, i, tim): - final = "%s:%s\t%d\t%f\n" % (string, " " * max(0, 13-len(string)), i, tim) - os.write(2, final) + final = "%s:%s\t%d\t%f" % (string, " " * max(0, 13-len(string)), i, tim) + debug_print(final) def _print_intline(self, string, i): final = string + ':' + " " * max(0, 16-len(string)) - final += '\t' + str(i) + '\n' - os.write(2, final) - - + final += '\t' + str(i) + debug_print(final) + class BrokenProfilerData(JitException): pass Modified: pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/optimizeopt/optimizer.py Wed Nov 24 14:21:00 2010 @@ -299,7 +299,9 @@ return CVAL_ZERO def propagate_all_forward(self): - self.exception_might_have_happened = False + self.exception_might_have_happened = True + # ^^^ at least at the start of bridges. For loops, we could set + # it to False, but we probably don't care self.newoperations = [] self.i = 0 while self.i < len(self.loop.operations): Modified: pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/pyjitpl.py Wed Nov 24 14:21:00 2010 @@ -18,7 +18,6 @@ from pypy.jit.metainterp.jitexc import JitException, get_llexception from pypy.rlib.rarithmetic import intmask from pypy.rlib.objectmodel import specialize -from pypy.rlib.jit import DEBUG_OFF, DEBUG_PROFILE, DEBUG_STEPS, DEBUG_DETAILED from pypy.jit.codewriter.jitcode import JitCode, SwitchDictDescr from pypy.jit.codewriter import heaptracker @@ -94,6 +93,18 @@ else: raise AssertionError(argcode) outvalue[startindex+i] = reg + def _put_back_list_of_boxes(self, outvalue, startindex, position): + code = self.bytecode + length = ord(code[position]) + position += 1 + for i in range(length): + index = ord(code[position+i]) + box = outvalue[startindex+i] + if box.type == history.INT: self.registers_i[index] = box + elif box.type == history.REF: self.registers_r[index] = box + elif box.type == history.FLOAT: self.registers_f[index] = box + else: raise AssertionError(box.type) + def get_current_position_info(self): return self.jitcode.get_live_vars_info(self.pc) @@ -814,8 +825,9 @@ for i in range(num_green_args): assert isinstance(varargs[i], Const) - @arguments("orgpc", "int", "boxes3", "boxes3") - def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, redboxes): + @arguments("orgpc", "int", "boxes3", "jitcode_position", "boxes3") + def opimpl_jit_merge_point(self, orgpc, jdindex, greenboxes, + jcposition, redboxes): any_operation = len(self.metainterp.history.operations) > 0 jitdriver_sd = self.metainterp.staticdata.jitdrivers_sd[jdindex] self.verify_green_args(jitdriver_sd, greenboxes) @@ -843,6 +855,10 @@ self.pc = orgpc self.metainterp.reached_loop_header(greenboxes, redboxes) self.pc = saved_pc + # no exception, which means that the jit_merge_point did not + # close the loop. We have to put the possibly-modified list + # 'redboxes' back into the registers where it comes from. + put_back_list_of_boxes3(self, jcposition, redboxes) else: # warning! careful here. We have to return from the current # frame containing the jit_merge_point, and then use @@ -1048,14 +1064,11 @@ else: moreargs = list(extraargs) metainterp_sd = metainterp.staticdata - original_greenkey = metainterp.resumekey.original_greenkey if opnum == rop.GUARD_NOT_FORCED: resumedescr = compile.ResumeGuardForcedDescr(metainterp_sd, - original_greenkey, metainterp.jitdriver_sd) else: - resumedescr = compile.ResumeGuardDescr(metainterp_sd, - original_greenkey) + resumedescr = compile.ResumeGuardDescr() guard_op = metainterp.history.record(opnum, moreargs, None, descr=resumedescr) virtualizable_boxes = None @@ -1209,6 +1222,7 @@ self.logger_ops = Logger(self, guard_number=True) self.profiler = ProfilerClass() + self.profiler.cpu = cpu self.warmrunnerdesc = warmrunnerdesc backendmodule = self.cpu.__module__ @@ -1327,6 +1341,11 @@ return jitcode return None + def try_to_free_some_loops(self): + # Increase here the generation recorded by the memory manager. + if self.warmrunnerdesc is not None: # for tests + self.warmrunnerdesc.memory_manager.next_generation() + # ---------------- logging ------------------------ def log(self, msg): @@ -1623,6 +1642,7 @@ self.staticdata._setup_once() self.staticdata.profiler.start_tracing() assert jitdriver_sd is self.jitdriver_sd + self.staticdata.try_to_free_some_loops() self.create_empty_history() try: original_boxes = self.initialize_original_boxes(jitdriver_sd, *args) @@ -1636,9 +1656,8 @@ self.current_merge_points = [(original_boxes, 0)] num_green_args = self.jitdriver_sd.num_green_args original_greenkey = original_boxes[:num_green_args] - redkey = original_boxes[num_green_args:] - self.resumekey = compile.ResumeFromInterpDescr(original_greenkey, - redkey) + self.resumekey = compile.ResumeFromInterpDescr(original_greenkey) + self.history.inputargs = original_boxes[num_green_args:] self.seen_loop_header_for_jdindex = -1 try: self.interpret() @@ -1652,23 +1671,26 @@ debug_start('jit-tracing') self.staticdata.profiler.start_tracing() assert isinstance(key, compile.ResumeGuardDescr) + # store the resumekey.wref_original_loop_token() on 'self' to make + # sure that it stays alive as long as this MetaInterp + self.resumekey_original_loop_token = key.wref_original_loop_token() + self.staticdata.try_to_free_some_loops() self.initialize_state_from_guard_failure(key) try: return self._handle_guard_failure(key) finally: + self.resumekey_original_loop_token = None self.staticdata.profiler.end_tracing() debug_stop('jit-tracing') def _handle_guard_failure(self, key): - original_greenkey = key.original_greenkey - # notice that here we just put the greenkey - # use -1 to mark that we will have to give up - # because we cannot reconstruct the beginning of the proper loop - self.current_merge_points = [(original_greenkey, -1)] + self.current_merge_points = [] self.resumekey = key self.seen_loop_header_for_jdindex = -1 try: self.prepare_resume_from_failure(key.guard_opnum) + if self.resumekey_original_loop_token is None: # very rare case + raise SwitchToBlackhole(ABORT_BRIDGE) self.interpret() except GenerateMergePoint, gmp: return self.designate_target_loop(gmp) @@ -1728,7 +1750,7 @@ num_green_args = self.jitdriver_sd.num_green_args for j in range(len(self.current_merge_points)-1, -1, -1): original_boxes, start = self.current_merge_points[j] - assert len(original_boxes) == len(live_arg_boxes) or start < 0 + assert len(original_boxes) == len(live_arg_boxes) for i in range(num_green_args): box1 = original_boxes[i] box2 = live_arg_boxes[i] @@ -1737,10 +1759,6 @@ break else: # Found! Compile it as a loop. - if start < 0: - # we cannot reconstruct the beginning of the proper loop - raise SwitchToBlackhole(ABORT_BRIDGE) - # raises in case it works -- which is the common case self.compile(original_boxes, live_arg_boxes, start) # creation of the loop was cancelled! @@ -1794,10 +1812,15 @@ raise NotImplementedError(opname[opnum]) def get_compiled_merge_points(self, greenkey): + """Get the list of looptokens corresponding to the greenkey. + Turns the (internal) list of weakrefs into regular refs. + """ cell = self.jitdriver_sd.warmstate.jit_cell_at_key(greenkey) - if cell.compiled_merge_points is None: - cell.compiled_merge_points = [] - return cell.compiled_merge_points + return cell.get_compiled_merge_points() + + def set_compiled_merge_points(self, greenkey, looptokens): + cell = self.jitdriver_sd.warmstate.jit_cell_at_key(greenkey) + cell.set_compiled_merge_points(looptokens) def compile(self, original_boxes, live_arg_boxes, start): num_green_args = self.jitdriver_sd.num_green_args @@ -1805,9 +1828,9 @@ greenkey = original_boxes[:num_green_args] old_loop_tokens = self.get_compiled_merge_points(greenkey) self.history.record(rop.JUMP, live_arg_boxes[num_green_args:], None) - loop_token = compile.compile_new_loop(self, old_loop_tokens, - greenkey, start) + loop_token = compile.compile_new_loop(self, old_loop_tokens, start) if loop_token is not None: # raise if it *worked* correctly + self.set_compiled_merge_points(greenkey, old_loop_tokens) raise GenerateMergePoint(live_arg_boxes, loop_token) self.history.operations.pop() # remove the JUMP @@ -2306,6 +2329,8 @@ else: raise AssertionError("bad argcode") position += 1 + elif argtype == "jitcode_position": + value = position else: raise AssertionError("bad argtype: %r" % (argtype,)) args += (value,) @@ -2350,3 +2375,15 @@ argtypes = unrolling_iterable(unboundmethod.argtypes) handler.func_name = 'handler_' + name return handler + +def put_back_list_of_boxes3(frame, position, newvalue): + code = frame.bytecode + length1 = ord(code[position]) + position2 = position + 1 + length1 + length2 = ord(code[position2]) + position3 = position2 + 1 + length2 + length3 = ord(code[position3]) + assert len(newvalue) == length1 + length2 + length3 + frame._put_back_list_of_boxes(newvalue, 0, position) + frame._put_back_list_of_boxes(newvalue, length1, position2) + frame._put_back_list_of_boxes(newvalue, length1 + length2, position3) Modified: pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/resoperation.py Wed Nov 24 14:21:00 2010 @@ -143,6 +143,16 @@ def can_raise(self): return rop._CANRAISE_FIRST <= self.getopnum() <= rop._CANRAISE_LAST + def is_malloc(self): + # a slightly different meaning from can_malloc + return rop._MALLOC_FIRST <= self.getopnum() <= rop._MALLOC_LAST + + def can_malloc(self): + return self.is_call() or self.is_malloc() + + def is_call(self): + return rop._CALL_FIRST <= self.getopnum() <= rop._CALL_LAST + def is_ovf(self): return rop._OVF_FIRST <= self.getopnum() <= rop._OVF_LAST @@ -441,9 +451,13 @@ 'GETARRAYITEM_RAW/2d', 'GETFIELD_GC/1d', 'GETFIELD_RAW/1d', + '_MALLOC_FIRST', 'NEW/0d', 'NEW_WITH_VTABLE/1', 'NEW_ARRAY/1d', + 'NEWSTR/1', + 'NEWUNICODE/1', + '_MALLOC_LAST', 'FORCE_TOKEN/0', 'VIRTUAL_REF/2', # removed before it's passed to the backend '_NOSIDEEFFECT_LAST', # ----- end of no_side_effect operations ----- @@ -452,10 +466,8 @@ 'SETARRAYITEM_RAW/3d', 'SETFIELD_GC/2d', 'SETFIELD_RAW/2d', - 'NEWSTR/1', 'STRSETITEM/3', 'UNICODESETITEM/3', - 'NEWUNICODE/1', #'RUNTIMENEW/1', # ootype operation 'COND_CALL_GC_WB/2d', # [objptr, newvalue] (for the write barrier) 'DEBUG_MERGE_POINT/2', # debugging only @@ -465,6 +477,7 @@ 'COPYUNICODECONTENT/5', '_CANRAISE_FIRST', # ----- start of can_raise operations ----- + '_CALL_FIRST', 'CALL/*d', 'CALL_ASSEMBLER/*d', # call already compiled assembler 'CALL_MAY_FORCE/*d', @@ -473,6 +486,7 @@ #'OOSEND_PURE', # ootype operation 'CALL_PURE/*d', # removed before it's passed to the backend # CALL_PURE(result, func, arg_1,..,arg_n) + '_CALL_LAST', '_CANRAISE_LAST', # ----- end of can_raise operations ----- '_OVF_FIRST', # ----- start of is_ovf operations ----- Modified: pypy/branch/fast-forward/pypy/jit/metainterp/resume.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/resume.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/resume.py Wed Nov 24 14:21:00 2010 @@ -65,12 +65,21 @@ snapshot = Snapshot(snapshot, boxes) storage.rd_snapshot = snapshot -class Numbering(object): - __slots__ = ('prev', 'nums') - - def __init__(self, prev, nums): - self.prev = prev - self.nums = nums +# +# The following is equivalent to the RPython-level declaration: +# +# class Numbering: __slots__ = ['prev', 'nums'] +# +# except that it is more compact in translated programs, because the +# array 'nums' is inlined in the single NUMBERING object. This is +# important because this is often the biggest single consumer of memory +# in a pypy-c-jit. +# +NUMBERINGP = lltype.Ptr(lltype.GcForwardReference()) +NUMBERING = lltype.GcStruct('Numbering', + ('prev', NUMBERINGP), + ('nums', lltype.Array(rffi.SHORT))) +NUMBERINGP.TO.become(NUMBERING) TAGMASK = 3 @@ -162,7 +171,7 @@ def number(self, values, snapshot): if snapshot is None: - return None, {}, 0 + return lltype.nullptr(NUMBERING), {}, 0 if snapshot in self.numberings: numb, liveboxes, v = self.numberings[snapshot] return numb, liveboxes.copy(), v @@ -171,7 +180,7 @@ n = len(liveboxes)-v boxes = snapshot.boxes length = len(boxes) - nums = [UNASSIGNED] * length + numb = lltype.malloc(NUMBERING, length) for i in range(length): box = boxes[i] value = values.get(box, None) @@ -190,9 +199,9 @@ tagged = tag(n, TAGBOX) n += 1 liveboxes[box] = tagged - nums[i] = tagged + numb.nums[i] = tagged # - numb = Numbering(numb1, nums) + numb.prev = numb1 self.numberings[snapshot] = numb, liveboxes, v return numb, liveboxes.copy(), v @@ -297,7 +306,7 @@ # compute the numbering storage = self.storage # make sure that nobody attached resume data to this guard yet - assert storage.rd_numb is None + assert not storage.rd_numb numb, liveboxes_from_env, v = self.memo.number(values, storage.rd_snapshot) self.liveboxes_from_env = liveboxes_from_env @@ -722,34 +731,36 @@ self.boxes_f = boxes_f self._prepare_next_section(info) - def consume_virtualizable_boxes(self, vinfo, nums): + def consume_virtualizable_boxes(self, vinfo, numb): # we have to ignore the initial part of 'nums' (containing vrefs), # find the virtualizable from nums[-1], and use it to know how many # boxes of which type we have to return. This does not write # anything into the virtualizable. - virtualizablebox = self.decode_ref(nums[-1]) + index = len(numb.nums) - 1 + virtualizablebox = self.decode_ref(numb.nums[index]) virtualizable = vinfo.unwrap_virtualizable_box(virtualizablebox) - return vinfo.load_list_of_boxes(virtualizable, self, nums) + return vinfo.load_list_of_boxes(virtualizable, self, numb) - def consume_virtualref_boxes(self, nums, end): + def consume_virtualref_boxes(self, numb, end): # Returns a list of boxes, assumed to be all BoxPtrs. # We leave up to the caller to call vrefinfo.continue_tracing(). assert (end & 1) == 0 - return [self.decode_ref(nums[i]) for i in range(end)] + return [self.decode_ref(numb.nums[i]) for i in range(end)] def consume_vref_and_vable_boxes(self, vinfo, ginfo): - nums = self.cur_numb.nums - self.cur_numb = self.cur_numb.prev + numb = self.cur_numb + self.cur_numb = numb.prev if vinfo is not None: - virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, nums) - end = len(nums) - len(virtualizable_boxes) + virtualizable_boxes = self.consume_virtualizable_boxes(vinfo, numb) + end = len(numb.nums) - len(virtualizable_boxes) elif ginfo is not None: - virtualizable_boxes = [self.decode_ref(nums[-1])] - end = len(nums) - 1 + index = len(numb.nums) - 1 + virtualizable_boxes = [self.decode_ref(numb.nums[index])] + end = len(numb.nums) - 1 else: virtualizable_boxes = None - end = len(nums) - virtualref_boxes = self.consume_virtualref_boxes(nums, end) + end = len(numb.nums) + virtualref_boxes = self.consume_virtualref_boxes(numb, end) return virtualizable_boxes, virtualref_boxes def allocate_with_vtable(self, known_class): @@ -977,23 +988,24 @@ info = blackholeinterp.get_current_position_info() self._prepare_next_section(info) - def consume_virtualref_info(self, vrefinfo, nums, end): + def consume_virtualref_info(self, vrefinfo, numb, end): # we have to decode a list of references containing pairs # [..., virtual, vref, ...] stopping at 'end' assert (end & 1) == 0 for i in range(0, end, 2): - virtual = self.decode_ref(nums[i]) - vref = self.decode_ref(nums[i+1]) + virtual = self.decode_ref(numb.nums[i]) + vref = self.decode_ref(numb.nums[i+1]) # For each pair, we store the virtual inside the vref. vrefinfo.continue_tracing(vref, virtual) - def consume_vable_info(self, vinfo, nums): + def consume_vable_info(self, vinfo, numb): # we have to ignore the initial part of 'nums' (containing vrefs), # find the virtualizable from nums[-1], load all other values # from the CPU stack, and copy them into the virtualizable if vinfo is None: - return len(nums) - virtualizable = self.decode_ref(nums[-1]) + return len(numb.nums) + index = len(numb.nums) - 1 + virtualizable = self.decode_ref(numb.nums[index]) virtualizable = vinfo.cast_gcref_to_vtype(virtualizable) if self.resume_after_guard_not_forced == 1: # in the middle of handle_async_forcing() @@ -1005,7 +1017,7 @@ # is and stays 0. Note the call to reset_vable_token() in # warmstate.py. assert not virtualizable.vable_token - return vinfo.write_from_resume_data_partial(virtualizable, self, nums) + return vinfo.write_from_resume_data_partial(virtualizable, self, numb) def load_value_of_type(self, TYPE, tagged): from pypy.jit.metainterp.warmstate import specialize_value @@ -1022,12 +1034,12 @@ load_value_of_type._annspecialcase_ = 'specialize:arg(1)' def consume_vref_and_vable(self, vrefinfo, vinfo, ginfo): - nums = self.cur_numb.nums - self.cur_numb = self.cur_numb.prev + numb = self.cur_numb + self.cur_numb = numb.prev if self.resume_after_guard_not_forced != 2: - end_vref = self.consume_vable_info(vinfo, nums) + end_vref = self.consume_vable_info(vinfo, numb) if ginfo is not None: end_vref -= 1 - self.consume_virtualref_info(vrefinfo, nums, end_vref) + self.consume_virtualref_info(vrefinfo, numb, end_vref) def allocate_with_vtable(self, known_class): from pypy.jit.metainterp.executor import exec_new_with_vtable @@ -1180,8 +1192,9 @@ 'at', compute_unique_id(frameinfo)) frameinfo = frameinfo.prev numb = storage.rd_numb - while numb is not None: - debug_print('\tnumb', str([untag(i) for i in numb.nums]), + while numb: + debug_print('\tnumb', str([untag(numb.nums[i]) + for i in range(len(numb.nums))]), 'at', compute_unique_id(numb)) numb = numb.prev for const in storage.rd_consts: Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_basic.py Wed Nov 24 14:21:00 2010 @@ -19,7 +19,11 @@ from pypy.jit.metainterp import simple_optimize class FakeJitCell: - compiled_merge_points = None + __compiled_merge_points = [] + def get_compiled_merge_points(self): + return self.__compiled_merge_points[:] + def set_compiled_merge_points(self, lst): + self.__compiled_merge_points = lst class FakeWarmRunnerState: def attach_unoptimized_bridge_from_interp(self, greenkey, newloop): @@ -35,7 +39,6 @@ optimize_bridge = staticmethod(simple_optimize.optimize_bridge) trace_limit = sys.maxint - debug_level = 2 func._jit_unroll_safe_ = True rtyper = support.annotate(func, values, type_system=type_system) Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_compile.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_compile.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_compile.py Wed Nov 24 14:21:00 2010 @@ -42,7 +42,6 @@ class FakeState: optimize_loop = staticmethod(optimize.optimize_loop) - debug_level = 0 class FakeGlobalData: loopnumbering = 0 @@ -54,6 +53,7 @@ stats = Stats() profiler = jitprof.EmptyProfiler() + warmrunnerdesc = None def log(self, msg, event_kind=None): pass @@ -85,7 +85,7 @@ metainterp.history.inputargs = loop.inputargs[:] # loop_tokens = [] - loop_token = compile_new_loop(metainterp, loop_tokens, [], 0) + loop_token = compile_new_loop(metainterp, loop_tokens, 0) assert loop_tokens == [loop_token] assert loop_token.number == 1 assert staticdata.globaldata.loopnumbering == 2 @@ -101,7 +101,7 @@ metainterp.history.operations = loop.operations[:] metainterp.history.inputargs = loop.inputargs[:] # - loop_token_2 = compile_new_loop(metainterp, loop_tokens, [], 0) + loop_token_2 = compile_new_loop(metainterp, loop_tokens, 0) assert loop_token_2 is loop_token assert loop_tokens == [loop_token] assert len(cpu.seen) == 0 @@ -207,14 +207,12 @@ class ExitFrameWithExceptionRef(Exception): pass FakeMetaInterpSD.cpu = cpu - class FakeJitDriverSD: - pass cpu.set_future_value_int(0, -156) cpu.set_future_value_int(1, -178) cpu.set_future_value_int(2, -190) fail_descr = cpu.execute_token(loop_token) try: - fail_descr.handle_fail(FakeMetaInterpSD(), FakeJitDriverSD()) + fail_descr.handle_fail(FakeMetaInterpSD(), None) except FakeMetaInterpSD.ExitFrameWithExceptionRef, e: assert lltype.cast_opaque_ptr(lltype.Ptr(EXC), e.args[1]) == llexc else: Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_exception.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_exception.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_exception.py Wed Nov 24 14:21:00 2010 @@ -1,6 +1,6 @@ import py, sys from pypy.jit.metainterp.test.test_basic import LLJitMixin, OOJitMixin -from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE +from pypy.rlib.jit import JitDriver, OPTIMIZER_SIMPLE, dont_look_inside from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask from pypy.jit.codewriter.policy import StopAtXPolicy @@ -588,6 +588,33 @@ res = self.interp_operations(f, [99]) assert res == 21 + def test_bug_exc1_noexc_exc2(self): + myjitdriver = JitDriver(greens=[], reds=['i']) + @dont_look_inside + def rescall(i): + if i < 10: + raise KeyError + if i < 20: + return None + raise ValueError + def f(i): + while i < 30: + myjitdriver.can_enter_jit(i=i) + myjitdriver.jit_merge_point(i=i) + try: + rescall(i) + except KeyError: + assert i < 10 + except ValueError: + assert i >= 20 + else: + assert 10 <= i < 20 + i += 1 + return i + res = self.meta_interp(f, [0], inline=True) + assert res == 30 + + class MyError(Exception): def __init__(self, n): self.n = n Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_logger.py Wed Nov 24 14:21:00 2010 @@ -14,11 +14,20 @@ def capturing(func, *args, **kwds): log_stream = StringIO() - debug._stderr = log_stream + class MyDebugLog: + def debug_print(self, *args): + for arg in args: + print >> log_stream, arg, + print >> log_stream + def debug_start(self, *args): + pass + def debug_stop(self, *args): + pass try: + debug._log = MyDebugLog() func(*args, **kwds) finally: - debug._stderr = sys.stderr + debug._log = None return log_stream.getvalue() class Logger(logger.Logger): @@ -112,7 +121,8 @@ equaloplists(loop.operations, oloop.operations) def test_jump(self): - namespace = {'target': LoopToken(3)} + namespace = {'target': LoopToken()} + namespace['target'].number = 3 inp = ''' [i0] jump(i0, descr=target) Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_optimizeopt.py Wed Nov 24 14:21:00 2010 @@ -41,7 +41,7 @@ b1 = BoxInt() opt = optimizeopt.Optimizer(FakeMetaInterpStaticData(LLtypeMixin.cpu), None) - fdescr = ResumeGuardDescr(None, None) + fdescr = ResumeGuardDescr() op = ResOperation(rop.GUARD_TRUE, ['dummy'], None, descr=fdescr) # setup rd data fi0 = resume.FrameInfo(None, "code0", 11) @@ -51,12 +51,12 @@ # opt.store_final_boxes_in_guard(op) if op.getfailargs() == [b0, b1]: - assert fdescr.rd_numb.nums == [tag(1, TAGBOX)] - assert fdescr.rd_numb.prev.nums == [tag(0, TAGBOX)] + assert list(fdescr.rd_numb.nums) == [tag(1, TAGBOX)] + assert list(fdescr.rd_numb.prev.nums) == [tag(0, TAGBOX)] else: assert op.getfailargs() == [b1, b0] - assert fdescr.rd_numb.nums == [tag(0, TAGBOX)] - assert fdescr.rd_numb.prev.nums == [tag(1, TAGBOX)] + assert list(fdescr.rd_numb.nums) == [tag(0, TAGBOX)] + assert list(fdescr.rd_numb.prev.nums) == [tag(1, TAGBOX)] assert fdescr.rd_virtuals is None assert fdescr.rd_consts == [] @@ -789,8 +789,12 @@ i3 = call(i2, descr=nonwritedescr) jump(i1) # the exception is considered lost when we loop back """ + # note that 'guard_no_exception' at the very start must be kept + # around: bridges may start with one. (In case of loops we could + # remove it, but we probably don't care.) expected = """ [i] + guard_no_exception() [] i1 = int_add(i, 3) i2 = call(i1, descr=nonwritedescr) guard_no_exception() [i1, i2] Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_pyjitpl.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_pyjitpl.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_pyjitpl.py Wed Nov 24 14:21:00 2010 @@ -17,6 +17,7 @@ portal.setup(None) class FakeStaticData: cpu = None + warmrunnerdesc = None metainterp = pyjitpl.MetaInterp(FakeStaticData(), None) metainterp.framestack = [] @@ -53,6 +54,7 @@ def test_remove_consts_and_duplicates(): class FakeStaticData: cpu = None + warmrunnerdesc = None def is_another_box_like(box, referencebox): assert box is not referencebox assert isinstance(box, referencebox.clonebox().__class__) Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_recursive.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_recursive.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_recursive.py Wed Nov 24 14:21:00 2010 @@ -1142,6 +1142,19 @@ res = self.meta_interp(main, [], inline=True, trace_limit=tlimit) assert ''.join(res.chars) == 'ABCDEFGHIabcdefghijJ' * 5 + def test_no_duplicates_bug(self): + driver = JitDriver(greens = ['codeno'], reds = ['i'], + get_printable_location = lambda codeno: str(codeno)) + def portal(codeno, i): + while i > 0: + driver.can_enter_jit(codeno=codeno, i=i) + driver.jit_merge_point(codeno=codeno, i=i) + if codeno > 0: + break + portal(i, i) + i -= 1 + self.meta_interp(portal, [0, 10], inline=True) + class TestLLtype(RecursiveTests, LLJitMixin): pass Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resoperation.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resoperation.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resoperation.py Wed Nov 24 14:21:00 2010 @@ -61,3 +61,10 @@ assert op.getarglist() == ['a', 'b'] assert op.result == 'c' assert op.getdescr() is mydescr + +def test_can_malloc(): + mydescr = AbstractDescr() + assert rop.ResOperation(rop.rop.NEW, [], 'b').can_malloc() + call = rop.ResOperation(rop.rop.CALL, ['a', 'b'], 'c', descr=mydescr) + assert call.can_malloc() + assert not rop.ResOperation(rop.rop.INT_ADD, ['a', 'b'], 'c').can_malloc() Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_resume.py Wed Nov 24 14:21:00 2010 @@ -142,6 +142,13 @@ assert bh.written_f == expected_f +def Numbering(prev, nums): + numb = lltype.malloc(NUMBERING, len(nums)) + numb.prev = prev or lltype.nullptr(NUMBERING) + for i in range(len(nums)): + numb.nums[i] = nums[i] + return numb + def test_simple_read(): #b1, b2, b3 = [BoxInt(), BoxPtr(), BoxInt()] c1, c2, c3 = [ConstInt(111), ConstInt(222), ConstInt(333)] @@ -391,15 +398,15 @@ assert fi1.pc == 3 def test_Numbering_create(): - l = [1, 2] + l = [rffi.r_short(1), rffi.r_short(2)] numb = Numbering(None, l) - assert numb.prev is None - assert numb.nums is l + assert not numb.prev + assert list(numb.nums) == l - l1 = ['b3'] + l1 = [rffi.r_short(3)] numb1 = Numbering(numb, l1) - assert numb1.prev is numb - assert numb1.nums is l1 + assert numb1.prev == numb + assert list(numb1.nums) == l1 def test_capture_resumedata(): b1, b2, b3 = [BoxInt(), BoxPtr(), BoxInt()] @@ -765,11 +772,12 @@ assert liveboxes == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b3: tag(2, TAGBOX)} - assert numb.nums == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), - tag(1, TAGINT)] - assert numb.prev.nums == [tag(0, TAGBOX), tag(1, TAGINT), tag(1, TAGBOX), - tag(0, TAGBOX), tag(2, TAGINT)] - assert numb.prev.prev is None + assert list(numb.nums) == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), + tag(1, TAGINT)] + assert list(numb.prev.nums) == [tag(0, TAGBOX), tag(1, TAGINT), + tag(1, TAGBOX), + tag(0, TAGBOX), tag(2, TAGINT)] + assert not numb.prev.prev numb2, liveboxes2, v = memo.number({}, snap2) assert v == 0 @@ -777,9 +785,9 @@ assert liveboxes2 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b3: tag(2, TAGBOX)} assert liveboxes2 is not liveboxes - assert numb2.nums == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb2.prev is numb.prev + assert list(numb2.nums) == [tag(3, TAGINT), tag(2, TAGBOX), tag(0, TAGBOX), + tag(3, TAGINT)] + assert numb2.prev == numb.prev env3 = [c3, b3, b1, c3] snap3 = Snapshot(snap, env3) @@ -800,9 +808,9 @@ assert v == 0 assert liveboxes3 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX)} - assert numb3.nums == [tag(3, TAGINT), tag(4, TAGINT), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb3.prev is numb.prev + assert list(numb3.nums) == [tag(3, TAGINT), tag(4, TAGINT), tag(0, TAGBOX), + tag(3, TAGINT)] + assert numb3.prev == numb.prev # virtual env4 = [c3, b4, b1, c3] @@ -813,9 +821,9 @@ assert liveboxes4 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b4: tag(0, TAGVIRTUAL)} - assert numb4.nums == [tag(3, TAGINT), tag(0, TAGVIRTUAL), tag(0, TAGBOX), - tag(3, TAGINT)] - assert numb4.prev is numb.prev + assert list(numb4.nums) == [tag(3, TAGINT), tag(0, TAGVIRTUAL), + tag(0, TAGBOX), tag(3, TAGINT)] + assert numb4.prev == numb.prev env5 = [b1, b4, b5] snap5 = Snapshot(snap4, env5) @@ -826,9 +834,9 @@ assert liveboxes5 == {b1: tag(0, TAGBOX), b2: tag(1, TAGBOX), b4: tag(0, TAGVIRTUAL), b5: tag(1, TAGVIRTUAL)} - assert numb5.nums == [tag(0, TAGBOX), tag(0, TAGVIRTUAL), - tag(1, TAGVIRTUAL)] - assert numb5.prev is numb4 + assert list(numb5.nums) == [tag(0, TAGBOX), tag(0, TAGVIRTUAL), + tag(1, TAGVIRTUAL)] + assert numb5.prev == numb4 def test_ResumeDataLoopMemo_number_boxes(): memo = ResumeDataLoopMemo(FakeMetaInterpStaticData()) Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmspot.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmspot.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmspot.py Wed Nov 24 14:21:00 2010 @@ -132,91 +132,6 @@ assert state.optimize_loop is optimize.optimize_loop assert state.optimize_bridge is optimize.optimize_bridge - def test_static_debug_level(self, capfd): - py.test.skip("debug_level is being deprecated") - from pypy.rlib.jit import DEBUG_PROFILE, DEBUG_OFF, DEBUG_STEPS - from pypy.jit.metainterp.jitprof import EmptyProfiler, Profiler - - myjitdriver = JitDriver(greens = [], reds = ['n']) - def f(n): - while n > 0: - myjitdriver.can_enter_jit(n=n) - myjitdriver.jit_merge_point(n=n) - n -= 1 - return n - - capfd.readouterr() - self.meta_interp(f, [10], debug_level=DEBUG_OFF, - ProfilerClass=Profiler) - out, err = capfd.readouterr() - assert not 'ENTER' in err - assert not 'LEAVE' in err - assert not "Running asm" in err - self.meta_interp(f, [10], debug_level=DEBUG_PROFILE, - ProfilerClass=Profiler) - out, err = capfd.readouterr() - assert not 'ENTER' in err - assert not 'LEAVE' in err - assert not 'compiled new' in err - assert "Running asm" in err - - self.meta_interp(f, [10], debug_level=DEBUG_STEPS, - ProfilerClass=Profiler) - out, err = capfd.readouterr() - assert 'ENTER' in err - assert 'LEAVE' in err - assert "Running asm" in err - - self.meta_interp(f, [10], debug_level=DEBUG_STEPS, - ProfilerClass=EmptyProfiler) - out, err = capfd.readouterr() - assert 'ENTER' in err - assert 'LEAVE' in err - assert not "Running asm" in err - - def test_set_param_debug(self): - py.test.skip("debug_level is being deprecated") - from pypy.rlib.jit import DEBUG_PROFILE, DEBUG_OFF, DEBUG_STEPS - from pypy.jit.metainterp.jitprof import EmptyProfiler, Profiler - - myjitdriver = JitDriver(greens = [], reds = ['n']) - def f(n): - while n > 0: - myjitdriver.can_enter_jit(n=n) - myjitdriver.jit_merge_point(n=n) - n -= 1 - return n - - def main(n, debug): - myjitdriver.set_param("debug", debug) - print f(n) - - outerr = py.io.StdCaptureFD() - self.meta_interp(main, [10, DEBUG_OFF], debug_level=DEBUG_STEPS, - ProfilerClass=Profiler) - out, errf = outerr.done() - err = errf.read() - assert not 'ENTER' in err - assert not 'LEAVE' in err - assert not "Running asm" in err - outerr = py.io.StdCaptureFD() - self.meta_interp(main, [10, DEBUG_PROFILE], debug_level=DEBUG_STEPS, - ProfilerClass=Profiler) - out, errf = outerr.done() - err = errf.read() - assert not 'ENTER' in err - assert not 'LEAVE' in err - assert not 'compiled new' in err - assert "Running asm" in err - outerr = py.io.StdCaptureFD() - self.meta_interp(main, [10, DEBUG_STEPS], debug_level=DEBUG_OFF, - ProfilerClass=Profiler) - out, errf = outerr.done() - err = errf.read() - assert 'ENTER' in err - assert 'LEAVE' in err - assert "Running asm" in err - def test_unwanted_loops(self): mydriver = JitDriver(reds = ['n', 'total', 'm'], greens = []) @@ -378,23 +293,30 @@ exc_vtable = lltype.malloc(OBJECT_VTABLE, immortal=True) cls.exc_vtable = exc_vtable - class FakeFailDescr(object): + class FakeLoopToken: def __init__(self, no): self.no = no + self.generation = 0 + + class FakeFailDescr(object): + def __init__(self, looptoken): + assert isinstance(looptoken, FakeLoopToken) + self.looptoken = looptoken def handle_fail(self, metainterp_sd, jitdrivers_sd): - if self.no == 0: + no = self.looptoken.no + if no == 0: raise metainterp_sd.warmrunnerdesc.DoneWithThisFrameInt(3) - if self.no == 1: + if no == 1: raise metainterp_sd.warmrunnerdesc.ContinueRunningNormally( [0], [], [], [1], [], []) - if self.no == 3: + if no == 3: exc = lltype.malloc(OBJECT) exc.typeptr = exc_vtable raise metainterp_sd.warmrunnerdesc.ExitFrameWithExceptionRef( metainterp_sd.cpu, lltype.cast_opaque_ptr(llmemory.GCREF, exc)) - return self.no + return self.looptoken class FakeDescr: def as_vtable_size_descr(self): @@ -419,11 +341,11 @@ sizeof = nodescr def get_fail_descr_from_number(self, no): - return FakeFailDescr(no) + return FakeFailDescr(FakeLoopToken(no)) def execute_token(self, token): - assert token == 2 - return FakeFailDescr(1) + assert token.no == 2 + return FakeFailDescr(FakeLoopToken(1)) driver = JitDriver(reds = ['red'], greens = ['green']) Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmstate.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmstate.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_warmstate.py Wed Nov 24 14:21:00 2010 @@ -99,6 +99,8 @@ lltype.Float], lltype.Void)) class FakeWarmRunnerDesc: rtyper = FakeRTyper() + cpu = None + memory_manager = None class FakeJitDriverSD: _get_jitcell_at_ptr = llhelper(GETTER, getter) _set_jitcell_at_ptr = llhelper(SETTER, setter) @@ -126,6 +128,7 @@ future_values[j] = "float", value class FakeWarmRunnerDesc: cpu = FakeCPU() + memory_manager = None class FakeJitDriverSD: _red_args_types = ["int", "float"] virtualizable_info = None @@ -154,16 +157,20 @@ _get_jitcell_at_ptr = None state = WarmEnterState(None, FakeJitDriverSD()) get_jitcell = state.make_jitcell_getter() + class FakeLoopToken(object): + pass + looptoken = FakeLoopToken() state.attach_unoptimized_bridge_from_interp([ConstInt(5), ConstFloat(2.25)], - "entry loop token") + looptoken) cell1 = get_jitcell(True, 5, 2.25) assert cell1.counter < 0 - assert cell1.entry_loop_token == "entry loop token" + assert cell1.get_entry_loop_token() is looptoken def test_make_jitdriver_callbacks_1(): class FakeWarmRunnerDesc: cpu = None + memory_manager = None class FakeJitDriverSD: _green_args_spec = [lltype.Signed, lltype.Float] _get_printable_location_ptr = None @@ -189,6 +196,7 @@ class FakeWarmRunnerDesc: rtyper = None cpu = None + memory_manager = None class FakeJitDriverSD: _green_args_spec = [lltype.Signed, lltype.Float] _get_printable_location_ptr = llhelper(GET_LOCATION, get_location) @@ -211,6 +219,7 @@ class FakeWarmRunnerDesc: rtyper = None cpu = None + memory_manager = None class FakeJitDriverSD: _green_args_spec = [lltype.Signed, lltype.Float] _get_printable_location_ptr = None @@ -233,6 +242,7 @@ class FakeWarmRunnerDesc: rtyper = None cpu = None + memory_manager = None class FakeJitDriverSD: _green_args_spec = [lltype.Signed, lltype.Float] _get_printable_location_ptr = None Modified: pypy/branch/fast-forward/pypy/jit/metainterp/test/test_ztranslation.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/test/test_ztranslation.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/test/test_ztranslation.py Wed Nov 24 14:21:00 2010 @@ -79,7 +79,7 @@ res = ll_meta_interp(main, [40, 5], CPUClass=self.CPUClass, type_system=self.type_system) assert res == main(40, 5) - res = rpython_ll_meta_interp(main, [40, 5], loops=2, + res = rpython_ll_meta_interp(main, [40, 5], CPUClass=self.CPUClass, type_system=self.type_system, optimizer=OPTIMIZER_FULL, @@ -120,7 +120,7 @@ res = ll_meta_interp(main, [40], CPUClass=self.CPUClass, type_system=self.type_system) assert res == main(40) - res = rpython_ll_meta_interp(main, [40], loops=2, CPUClass=self.CPUClass, + res = rpython_ll_meta_interp(main, [40], CPUClass=self.CPUClass, type_system=self.type_system, optimizer=OPTIMIZER_FULL, ProfilerClass=Profiler) Modified: pypy/branch/fast-forward/pypy/jit/metainterp/virtualizable.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/virtualizable.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/virtualizable.py Wed Nov 24 14:21:00 2010 @@ -100,48 +100,48 @@ i = i + 1 assert len(boxes) == i + 1 # - def write_from_resume_data_partial(virtualizable, reader, nums): + def write_from_resume_data_partial(virtualizable, reader, numb): # Load values from the reader (see resume.py) described by # the list of numbers 'nums', and write them in their proper # place in the 'virtualizable'. This works from the end of # the list and returns the index in 'nums' of the start of # the virtualizable data found, allowing the caller to do # further processing with the start of the list. - i = len(nums) - 1 + i = len(numb.nums) - 1 assert i >= 0 for ARRAYITEMTYPE, fieldname in unroll_array_fields_rev: lst = getattr(virtualizable, fieldname) for j in range(getlength(lst)-1, -1, -1): i -= 1 assert i >= 0 - x = reader.load_value_of_type(ARRAYITEMTYPE, nums[i]) + x = reader.load_value_of_type(ARRAYITEMTYPE, numb.nums[i]) setarrayitem(lst, j, x) for FIELDTYPE, fieldname in unroll_static_fields_rev: i -= 1 assert i >= 0 - x = reader.load_value_of_type(FIELDTYPE, nums[i]) + x = reader.load_value_of_type(FIELDTYPE, numb.nums[i]) setattr(virtualizable, fieldname, x) return i # - def load_list_of_boxes(virtualizable, reader, nums): + def load_list_of_boxes(virtualizable, reader, numb): # Uses 'virtualizable' only to know the length of the arrays; # does not write anything into it. The returned list is in # the format expected of virtualizable_boxes, so it ends in # the virtualizable itself. - i = len(nums) - 1 + i = len(numb.nums) - 1 assert i >= 0 - boxes = [reader.decode_box_of_type(self.VTYPEPTR, nums[i])] + boxes = [reader.decode_box_of_type(self.VTYPEPTR, numb.nums[i])] for ARRAYITEMTYPE, fieldname in unroll_array_fields_rev: lst = getattr(virtualizable, fieldname) for j in range(getlength(lst)-1, -1, -1): i -= 1 assert i >= 0 - box = reader.decode_box_of_type(ARRAYITEMTYPE, nums[i]) + box = reader.decode_box_of_type(ARRAYITEMTYPE,numb.nums[i]) boxes.append(box) for FIELDTYPE, fieldname in unroll_static_fields_rev: i -= 1 assert i >= 0 - box = reader.decode_box_of_type(FIELDTYPE, nums[i]) + box = reader.decode_box_of_type(FIELDTYPE, numb.nums[i]) boxes.append(box) boxes.reverse() return boxes Modified: pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/warmspot.py Wed Nov 24 14:21:00 2010 @@ -12,11 +12,12 @@ from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.rarithmetic import r_uint, intmask from pypy.rlib.debug import debug_print, fatalerror +from pypy.rlib.debug import debug_start, debug_stop from pypy.rpython.lltypesystem.lloperation import llop from pypy.translator.simplify import get_funcobj, get_functype from pypy.translator.unsimplify import call_final_function -from pypy.jit.metainterp import history, pyjitpl, gc +from pypy.jit.metainterp import history, pyjitpl, gc, memmgr from pypy.jit.metainterp.pyjitpl import MetaInterpStaticData, MetaInterp from pypy.jit.metainterp.typesystem import LLTypeHelper, OOTypeHelper from pypy.jit.metainterp.jitprof import Profiler, EmptyProfiler @@ -24,21 +25,17 @@ from pypy.jit.metainterp.jitdriver import JitDriverStaticData from pypy.jit.codewriter import support, codewriter from pypy.jit.codewriter.policy import JitPolicy -from pypy.rlib.jit import DEBUG_STEPS, DEBUG_DETAILED, DEBUG_OFF, DEBUG_PROFILE # ____________________________________________________________ # Bootstrapping -def apply_jit(translator, backend_name="auto", debug_level=DEBUG_STEPS, - inline=False, - **kwds): +def apply_jit(translator, backend_name="auto", inline=False, **kwds): if 'CPUClass' not in kwds: from pypy.jit.backend.detect_cpu import getcpuclass kwds['CPUClass'] = getcpuclass(backend_name) - if debug_level > DEBUG_OFF: - ProfilerClass = Profiler - else: - ProfilerClass = EmptyProfiler + ProfilerClass = Profiler + # Always use Profiler here, which should have a very low impact. + # Otherwise you can try with ProfilerClass = EmptyProfiler. warmrunnerdesc = WarmRunnerDesc(translator, translate_support_code=True, listops=True, @@ -47,7 +44,6 @@ **kwds) for jd in warmrunnerdesc.jitdrivers_sd: jd.warmstate.set_param_inlining(inline) - jd.warmstate.set_param_debug(debug_level) warmrunnerdesc.finish() translator.warmrunnerdesc = warmrunnerdesc # for later debugging @@ -66,7 +62,7 @@ def jittify_and_run(interp, graph, args, repeat=1, backendopt=False, trace_limit=sys.maxint, - debug_level=DEBUG_STEPS, inline=False, **kwds): + inline=False, loop_longevity=0, **kwds): from pypy.config.config import ConfigError translator = interp.typer.annotator.translator try: @@ -83,7 +79,7 @@ jd.warmstate.set_param_trace_eagerness(2) # for tests jd.warmstate.set_param_trace_limit(trace_limit) jd.warmstate.set_param_inlining(inline) - jd.warmstate.set_param_debug(debug_level) + jd.warmstate.set_param_loop_longevity(loop_longevity) warmrunnerdesc.finish() res = interp.eval_graph(graph, args) if not kwds.get('translate_support_code', False): @@ -98,8 +94,7 @@ repeat -= 1 return res -def rpython_ll_meta_interp(function, args, backendopt=True, - loops='not used right now', **kwds): +def rpython_ll_meta_interp(function, args, backendopt=True, **kwds): return ll_meta_interp(function, args, backendopt=backendopt, translate_support_code=True, **kwds) @@ -152,6 +147,7 @@ optimizer=None, ProfilerClass=EmptyProfiler, **kwds): pyjitpl._warmrunnerdesc = self # this is a global for debugging only! self.set_translator(translator) + self.memory_manager = memmgr.MemoryManager() self.build_cpu(CPUClass, **kwds) self.find_portals() self.codewriter = codewriter.CodeWriter(self.cpu, self.jitdrivers_sd) @@ -714,7 +710,7 @@ loop_token = fail_descr.handle_fail(self.metainterp_sd, jd) except JitException, e: return handle_jitexception(e) - fail_descr = self.cpu.execute_token(loop_token) + fail_descr = self.execute_token(loop_token) jd._assembler_call_helper = assembler_call_helper # for debugging jd._assembler_helper_ptr = self.helper_func( @@ -808,3 +804,10 @@ py.test.skip("rewrite_force_virtual: port it to ootype") all_graphs = self.translator.graphs vrefinfo.replace_force_virtual_with_call(all_graphs) + + # ____________________________________________________________ + + def execute_token(self, loop_token): + fail_descr = self.cpu.execute_token(loop_token) + self.memory_manager.keep_loop_alive(loop_token) + return fail_descr Modified: pypy/branch/fast-forward/pypy/jit/metainterp/warmstate.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/metainterp/warmstate.py (original) +++ pypy/branch/fast-forward/pypy/jit/metainterp/warmstate.py Wed Nov 24 14:21:00 2010 @@ -1,4 +1,4 @@ -import sys +import sys, weakref from pypy.rpython.lltypesystem import lltype, llmemory, rstr, rffi from pypy.rpython.ootypesystem import ootype from pypy.rpython.annlowlevel import hlstr, llstr, cast_base_ptr_to_instance @@ -9,7 +9,6 @@ from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.jit import (PARAMETERS, OPTIMIZER_SIMPLE, OPTIMIZER_FULL, OPTIMIZER_NO_PERFECTSPEC) -from pypy.rlib.jit import DEBUG_PROFILE from pypy.rlib.jit import BaseJitCell from pypy.rlib.debug import debug_start, debug_stop, debug_print from pypy.jit.metainterp import history @@ -150,9 +149,34 @@ # counter == -1: there is an entry bridge for this cell # counter == -2: tracing is currently going on for this cell counter = 0 - compiled_merge_points = None + compiled_merge_points_wref = None # list of weakrefs to LoopToken dont_trace_here = False - entry_loop_token = None + wref_entry_loop_token = None # (possibly) one weakref to LoopToken + + def get_compiled_merge_points(self): + result = [] + if self.compiled_merge_points_wref is not None: + for wref in self.compiled_merge_points_wref: + looptoken = wref() + if looptoken is not None: + result.append(looptoken) + return result + + def set_compiled_merge_points(self, looptokens): + self.compiled_merge_points_wref = [self._makeref(token) + for token in looptokens] + + def get_entry_loop_token(self): + if self.wref_entry_loop_token is not None: + return self.wref_entry_loop_token() + return None + + def set_entry_loop_token(self, looptoken): + self.wref_entry_loop_token = self._makeref(looptoken) + + def _makeref(self, looptoken): + assert looptoken is not None + return weakref.ref(looptoken) # ____________________________________________________________ @@ -165,6 +189,8 @@ "NOT_RPYTHON" self.warmrunnerdesc = warmrunnerdesc self.jitdriver_sd = jitdriver_sd + if warmrunnerdesc is not None: # for tests + self.cpu = warmrunnerdesc.cpu try: self.profiler = warmrunnerdesc.metainterp_sd.profiler except AttributeError: # for tests @@ -176,7 +202,7 @@ meth(default_value) def set_param_threshold(self, threshold): - if threshold < 0: + if threshold <= 0: self.increment_threshold = 0 # never reach the THRESHOLD_LIMIT return if threshold < 2: @@ -209,10 +235,11 @@ else: raise ValueError("unknown optimizer") - def set_param_debug(self, value): - self.debug_level = value - if self.profiler is not None: - self.profiler.set_printing(value >= DEBUG_PROFILE) + def set_param_loop_longevity(self, value): + # note: it's a global parameter, not a per-jitdriver one + if (self.warmrunnerdesc is not None and + self.warmrunnerdesc.memory_manager is not None): # all for tests + self.warmrunnerdesc.memory_manager.set_max_age(value) def disable_noninlinable_function(self, greenkey): cell = self.jit_cell_at_key(greenkey) @@ -225,12 +252,15 @@ def attach_unoptimized_bridge_from_interp(self, greenkey, entry_loop_token): cell = self.jit_cell_at_key(greenkey) - cell.counter = -1 - old_token = cell.entry_loop_token - cell.entry_loop_token = entry_loop_token + old_token = cell.get_entry_loop_token() + cell.set_entry_loop_token(entry_loop_token) + cell.counter = -1 # valid entry bridge attached if old_token is not None: - cpu = self.warmrunnerdesc.cpu - cpu.redirect_call_assembler(old_token, entry_loop_token) + self.cpu.redirect_call_assembler(old_token, entry_loop_token) + # entry_loop_token is also kept alive by any loop that used + # to point to old_token. Actually freeing old_token early + # is a pointless optimization (it is tiny). + old_token.record_jump_to(entry_loop_token) # ---------- @@ -239,7 +269,8 @@ if hasattr(self, 'maybe_compile_and_run'): return self.maybe_compile_and_run - metainterp_sd = self.warmrunnerdesc.metainterp_sd + warmrunnerdesc = self.warmrunnerdesc + metainterp_sd = warmrunnerdesc.metainterp_sd jitdriver_sd = self.jitdriver_sd vinfo = jitdriver_sd.virtualizable_info index_of_virtualizable = jitdriver_sd.index_of_virtualizable @@ -297,23 +328,27 @@ assert cell.counter == -1 if not confirm_enter_jit(*args): return + loop_token = cell.get_entry_loop_token() + if loop_token is None: # it was a weakref that has been freed + cell.counter = 0 + return # machine code was already compiled for these greenargs # get the assembler and fill in the boxes set_future_values(*args[num_green_args:]) - loop_token = cell.entry_loop_token # ---------- execute assembler ---------- while True: # until interrupted by an exception metainterp_sd.profiler.start_running() debug_start("jit-running") - fail_descr = metainterp_sd.cpu.execute_token(loop_token) + fail_descr = warmrunnerdesc.execute_token(loop_token) debug_stop("jit-running") metainterp_sd.profiler.end_running() + loop_token = None # for test_memmgr if vinfo is not None: vinfo.reset_vable_token(virtualizable) loop_token = fail_descr.handle_fail(metainterp_sd, jitdriver_sd) - + maybe_compile_and_run._dont_inline_ = True self.maybe_compile_and_run = maybe_compile_and_run return maybe_compile_and_run @@ -459,7 +494,7 @@ warmrunnerdesc = self.warmrunnerdesc jitdriver_sd = self.jitdriver_sd - cpu = warmrunnerdesc.cpu + cpu = self.cpu vinfo = jitdriver_sd.virtualizable_info red_args_types = unrolling_iterable(jitdriver_sd._red_args_types) # @@ -508,10 +543,11 @@ if hasattr(self, 'get_location_str'): return # + warmrunnerdesc = self.warmrunnerdesc unwrap_greenkey = self.make_unwrap_greenkey() jit_getter = self.make_jitcell_getter() jd = self.jitdriver_sd - cpu = self.warmrunnerdesc.cpu + cpu = self.cpu def can_inline_greenargs(*greenargs): if can_never_inline(*greenargs): @@ -529,11 +565,16 @@ def get_assembler_token(greenkey, redboxes): # 'redboxes' is only used to know the types of red arguments cell = self.jit_cell_at_key(greenkey) - if cell.entry_loop_token is None: + entry_loop_token = cell.get_entry_loop_token() + if entry_loop_token is None: from pypy.jit.metainterp.compile import compile_tmp_callback - cell.entry_loop_token = compile_tmp_callback(cpu, jd, greenkey, - redboxes) - return cell.entry_loop_token + if cell.counter == -1: # used to be a valid entry bridge, + cell.counter = 0 # but was freed in the meantime. + memmgr = warmrunnerdesc.memory_manager + entry_loop_token = compile_tmp_callback(cpu, jd, greenkey, + redboxes, memmgr) + cell.set_entry_loop_token(entry_loop_token) + return entry_loop_token self.get_assembler_token = get_assembler_token # Modified: pypy/branch/fast-forward/pypy/jit/tool/jitoutput.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/tool/jitoutput.py (original) +++ pypy/branch/fast-forward/pypy/jit/tool/jitoutput.py Wed Nov 24 14:21:00 2010 @@ -27,6 +27,10 @@ (('nvirtuals',), '^nvirtuals:\s+(\d+)$'), (('nvholes',), '^nvholes:\s+(\d+)$'), (('nvreused',), '^nvreused:\s+(\d+)$'), + (('total_compiled_loops',), '^Total # of loops:\s+(\d+)$'), + (('total_compiled_bridges',), '^Total # of bridges:\s+(\d+)$'), + (('total_freed_loops',), '^Freed # of loops:\s+(\d+)$'), + (('total_freed_bridges',), '^Freed # of bridges:\s+(\d+)$'), ] class Ops(object): Modified: pypy/branch/fast-forward/pypy/jit/tool/test/test_jitoutput.py ============================================================================== --- pypy/branch/fast-forward/pypy/jit/tool/test/test_jitoutput.py (original) +++ pypy/branch/fast-forward/pypy/jit/tool/test/test_jitoutput.py Wed Nov 24 14:21:00 2010 @@ -1,10 +1,11 @@ import py from pypy.jit.metainterp.warmspot import ll_meta_interp -from pypy.rlib.jit import JitDriver, DEBUG_PROFILE +from pypy.rlib.jit import JitDriver from pypy.jit.backend.llgraph import runner from pypy.jit.metainterp.jitprof import Profiler, JITPROF_LINES from pypy.jit.tool.jitoutput import parse_prof +from pypy.tool.logparser import parse_log, extract_category def test_really_run(): """ This test checks whether output of jitprof did not change. @@ -21,13 +22,15 @@ cap = py.io.StdCaptureFD() try: ll_meta_interp(f, [10], CPUClass=runner.LLtypeCPU, type_system='lltype', - ProfilerClass=Profiler, debug_level=DEBUG_PROFILE) + ProfilerClass=Profiler) finally: out, err = cap.reset() - err = "\n".join(err.splitlines()[-JITPROF_LINES:]) - print err - assert err.count("\n") == JITPROF_LINES - 1 - info = parse_prof(err) + + log = parse_log(err.splitlines(True)) + err_sections = list(extract_category(log, 'jit-summary')) + [err1] = err_sections # there should be exactly one jit-summary + assert err1.count("\n") == JITPROF_LINES + info = parse_prof(err1) # assert did not crash # asserts below are a bit delicate, possibly they might be deleted assert info.tracing_no == 1 @@ -60,6 +63,10 @@ nvirtuals: 13 nvholes: 14 nvreused: 15 +Total # of loops: 100 +Total # of bridges: 300 +Freed # of loops: 99 +Freed # of bridges: 299 ''' def test_parse(): Modified: pypy/branch/fast-forward/pypy/module/__pypy__/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__pypy__/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/__pypy__/__init__.py Wed Nov 24 14:21:00 2010 @@ -11,6 +11,10 @@ 'internal_repr' : 'interp_magic.internal_repr', 'bytebuffer' : 'bytebuffer.bytebuffer', 'identity_dict' : 'interp_identitydict.W_IdentityDict', + 'debug_start' : 'interp_debug.debug_start', + 'debug_print' : 'interp_debug.debug_print', + 'debug_stop' : 'interp_debug.debug_stop', + 'debug_print_once' : 'interp_debug.debug_print_once', } def setup_after_space_initialization(self): @@ -23,6 +27,9 @@ 'interp_magic.method_cache_counter') self.extra_interpdef('reset_method_cache_counter', 'interp_magic.reset_method_cache_counter') + if self.space.config.objspace.std.withmapdict: + self.extra_interpdef('mapdict_cache_counter', + 'interp_magic.mapdict_cache_counter') PYC_MAGIC = get_pyc_magic(self.space) self.extra_interpdef('PYC_MAGIC', 'space.wrap(%d)' % PYC_MAGIC) # Modified: pypy/branch/fast-forward/pypy/module/__pypy__/interp_magic.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/__pypy__/interp_magic.py (original) +++ pypy/branch/fast-forward/pypy/module/__pypy__/interp_magic.py Wed Nov 24 14:21:00 2010 @@ -2,6 +2,7 @@ from pypy.interpreter.gateway import ObjSpace from pypy.rlib.objectmodel import we_are_translated from pypy.objspace.std.typeobject import MethodCache +from pypy.objspace.std.mapdict import IndexCache def internal_repr(space, w_object): return space.wrap('%r' % (w_object,)) @@ -36,4 +37,17 @@ cache = space.fromcache(MethodCache) cache.misses = {} cache.hits = {} - + if space.config.objspace.std.withmapdict: + cache = space.fromcache(IndexCache) + cache.misses = {} + cache.hits = {} + +def mapdict_cache_counter(space, name): + """Return a tuple (index_cache_hits, index_cache_misses) for lookups + in the mapdict cache with the given attribute name.""" + assert space.config.objspace.std.withmethodcachecounter + assert space.config.objspace.std.withmapdict + cache = space.fromcache(IndexCache) + return space.newtuple([space.newint(cache.hits.get(name, 0)), + space.newint(cache.misses.get(name, 0))]) +mapdict_cache_counter.unwrap_spec = [ObjSpace, str] Modified: pypy/branch/fast-forward/pypy/module/_pickle_support/maker.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_pickle_support/maker.py (original) +++ pypy/branch/fast-forward/pypy/module/_pickle_support/maker.py Wed Nov 24 14:21:00 2010 @@ -67,11 +67,12 @@ return space.wrap(tb) traceback_new.unwrap_spec = [ObjSpace] -def generator_new(space, frame, running): +def generator_new(space, w_frame, running): + frame = space.interp_w(PyFrame, w_frame, can_be_None=True) new_generator = GeneratorIterator(frame) new_generator.running = running return space.wrap(new_generator) -generator_new.unwrap_spec = [ObjSpace, PyFrame, int] +generator_new.unwrap_spec = [ObjSpace, W_Root, int] def xrangeiter_new(space, current, remaining, step): from pypy.module.__builtin__.functional import W_XRangeIterator Modified: pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py (original) +++ pypy/branch/fast-forward/pypy/module/_rawffi/test/test__rawffi.py Wed Nov 24 14:21:00 2010 @@ -297,6 +297,7 @@ assert _rawffi.charp2string(res[0]) is None arg1.free() arg2.free() + a.free() def test_returning_unicode(self): import _rawffi Modified: pypy/branch/fast-forward/pypy/module/_rawffi/test/test_nested.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_rawffi/test/test_nested.py (original) +++ pypy/branch/fast-forward/pypy/module/_rawffi/test/test_nested.py Wed Nov 24 14:21:00 2010 @@ -107,7 +107,6 @@ assert S.fieldoffset('x') == 0 assert S.fieldoffset('ar') == A5alignment s = S() - s = S() s.x = 'G' raises(TypeError, 's.ar') assert s.fieldaddress('ar') == s.buffer + S.fieldoffset('ar') Modified: pypy/branch/fast-forward/pypy/module/array/interp_array.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/array/interp_array.py (original) +++ pypy/branch/fast-forward/pypy/module/array/interp_array.py Wed Nov 24 14:21:00 2010 @@ -28,7 +28,7 @@ typecode = typecode[0] if space.is_w(w_cls, space.gettypeobject(W_ArrayBase.typedef)): - if len(w_args.keywords_w) > 0: + if w_args.keywords: # XXX this might be forbidden fishing msg = 'array.array() does not take keyword arguments' raise OperationError(space.w_TypeError, space.wrap(msg)) Modified: pypy/branch/fast-forward/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/api.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/api.py Wed Nov 24 14:21:00 2010 @@ -941,8 +941,10 @@ from pypy.rlib import rdynload try: ll_libname = rffi.str2charp(path) - dll = rdynload.dlopen(ll_libname) - lltype.free(ll_libname, flavor='raw') + try: + dll = rdynload.dlopen(ll_libname) + finally: + lltype.free(ll_libname, flavor='raw') except rdynload.DLOpenError, e: raise operationerrfmt( space.w_ImportError, Modified: pypy/branch/fast-forward/pypy/module/cpyext/cdatetime.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/cdatetime.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/cdatetime.py Wed Nov 24 14:21:00 2010 @@ -4,7 +4,7 @@ from pypy.module.cpyext.api import ( cpython_api, CANNOT_FAIL, cpython_struct, PyObjectFields) from pypy.module.cpyext.import_ import PyImport_Import -from pypy.module.cpyext.typeobject import PyTypeObjectPtr +from pypy.module.cpyext.typeobject import PyTypeObjectPtr, render_immortal from pypy.module.cpyext.state import State from pypy.interpreter.error import OperationError from pypy.tool.sourcetools import func_renamer @@ -22,25 +22,34 @@ @cpython_api([], lltype.Ptr(PyDateTime_CAPI), error=lltype.nullptr(PyDateTime_CAPI)) def _PyDateTime_Import(space): - datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw') + datetimeAPI = lltype.malloc(PyDateTime_CAPI, flavor='raw', + track_allocation=False) if not we_are_translated(): datetimeAPI_dealloc(space) space.fromcache(State).datetimeAPI = datetimeAPI w_datetime = PyImport_Import(space, space.wrap("datetime")) + w_type = space.getattr(w_datetime, space.wrap("date")) datetimeAPI.c_DateType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DateType, w_type) + w_type = space.getattr(w_datetime, space.wrap("datetime")) datetimeAPI.c_DateTimeType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DateTimeType, w_type) + w_type = space.getattr(w_datetime, space.wrap("time")) datetimeAPI.c_TimeType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_TimeType, w_type) + w_type = space.getattr(w_datetime, space.wrap("timedelta")) datetimeAPI.c_DeltaType = rffi.cast( PyTypeObjectPtr, make_ref(space, w_type)) + render_immortal(datetimeAPI.c_DeltaType, w_type) return datetimeAPI Modified: pypy/branch/fast-forward/pypy/module/cpyext/presetup.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/presetup.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/presetup.py Wed Nov 24 14:21:00 2010 @@ -21,6 +21,8 @@ from pypy.conftest import gettestobjspace from pypy.module.cpyext.api import build_bridge +from pypy.module.imp.importing import get_so_extension + usemodules = ['cpyext', 'thread'] if sys.platform == 'win32': usemodules.append('_winreg') # necessary in distutils @@ -35,6 +37,7 @@ def patch_distutils(): sysconfig.get_python_inc = get_python_inc + sysconfig.get_config_vars()['SO'] = get_so_extension(space) patch_distutils() Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_borrow.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/test/test_borrow.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_borrow.py Wed Nov 24 14:21:00 2010 @@ -31,6 +31,7 @@ g = PyTuple_GetItem(t, 0); // borrows reference again printf("Refcnt4: %i\\n", f->ob_refcnt); printf("COMPARE: %i\\n", f == g); + fflush(stdout); Py_DECREF(t); Py_RETURN_TRUE; """), Modified: pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/test/test_cpyext.py Wed Nov 24 14:21:00 2010 @@ -51,7 +51,7 @@ PyUnicode_GetDefaultEncoding.restype = ctypes.c_char_p assert PyUnicode_GetDefaultEncoding() == 'ascii' -def compile_module(modname, **kwds): +def compile_module(space, modname, **kwds): """ Build an extension module and return the filename of the resulting native code file. @@ -74,10 +74,8 @@ [], eci, outputfilename=str(dirname/modname), standalone=False) - if sys.platform == 'win32': - pydname = soname.new(purebasename=modname, ext='.pyd') - else: - pydname = soname.new(purebasename=modname, ext='.so') + from pypy.module.imp.importing import get_so_extension + pydname = soname.new(purebasename=modname, ext=get_so_extension(space)) soname.rename(pydname) return str(pydname) @@ -162,7 +160,7 @@ kwds["link_files"] = [str(api_library + '.so')] if sys.platform == 'linux2': kwds["compile_extra"]=["-Werror=implicit-function-declaration"] - return compile_module(name, **kwds) + return compile_module(self.space, name, **kwds) def import_module(self, name, init=None, body='', load_it=True, filename=None): Modified: pypy/branch/fast-forward/pypy/module/cpyext/typeobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/cpyext/typeobject.py (original) +++ pypy/branch/fast-forward/pypy/module/cpyext/typeobject.py Wed Nov 24 14:21:00 2010 @@ -182,10 +182,10 @@ subtype = rffi.cast(PyTypeObjectPtr, make_ref(space, w_subtype)) try: - obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds) + w_obj = generic_cpy_call(space, tp_new, subtype, w_args, w_kwds) finally: Py_DecRef(space, w_subtype) - return obj + return w_obj @specialize.memo() def get_new_method_def(space): @@ -193,10 +193,14 @@ if state.new_method_def: return state.new_method_def from pypy.module.cpyext.modsupport import PyMethodDef - ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True) + ptr = lltype.malloc(PyMethodDef, flavor="raw", zero=True, + immortal=True) ptr.c_ml_name = rffi.str2charp("__new__") + lltype.render_immortal(ptr.c_ml_name) rffi.setintfield(ptr, 'c_ml_flags', METH_VARARGS | METH_KEYWORDS) - ptr.c_ml_doc = rffi.str2charp("T.__new__(S, ...) -> a new object with type S, a subtype of T") + ptr.c_ml_doc = rffi.str2charp( + "T.__new__(S, ...) -> a new object with type S, a subtype of T") + lltype.render_immortal(ptr.c_ml_doc) state.new_method_def = ptr return ptr @@ -429,6 +433,12 @@ finish_type_1(space, pto) finish_type_2(space, pto, w_type) + if space.type(w_type).is_cpytype(): + # XXX Types with a C metatype are never freed, try to see why... + render_immortal(pto, w_type) + lltype.render_immortal(pto) + lltype.render_immortal(pto.c_tp_name) + pto.c_tp_basicsize = rffi.sizeof(typedescr.basestruct) if pto.c_tp_base: if pto.c_tp_base.c_tp_basicsize > pto.c_tp_basicsize: @@ -534,12 +544,25 @@ w_obj.ready() finish_type_2(space, py_type, w_obj) + render_immortal(py_type, w_obj) state = space.fromcache(RefcountState) state.non_heaptypes_w.append(w_obj) return w_obj +def render_immortal(py_type, w_obj): + lltype.render_immortal(py_type.c_tp_bases) + lltype.render_immortal(py_type.c_tp_mro) + + assert isinstance(w_obj, W_TypeObject) + if w_obj.is_cpytype(): + lltype.render_immortal(py_type.c_tp_dict) + else: + lltype.render_immortal(py_type.c_tp_name) + if not w_obj.is_cpytype() and w_obj.is_heaptype(): + lltype.render_immortal(py_type) + def finish_type_1(space, pto): """ Sets up tp_bases, necessary before creating the interpreter type. Modified: pypy/branch/fast-forward/pypy/module/gc/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/gc/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/gc/__init__.py Wed Nov 24 14:21:00 2010 @@ -29,6 +29,7 @@ 'get_referents': 'referents.get_referents', 'get_referrers': 'referents.get_referrers', '_dump_rpy_heap': 'referents._dump_rpy_heap', + 'get_typeids_z': 'referents.get_typeids_z', 'GcRef': 'referents.W_GcRef', }) MixedModule.__init__(self, space, w_name) Modified: pypy/branch/fast-forward/pypy/module/gc/app_referents.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/gc/app_referents.py (original) +++ pypy/branch/fast-forward/pypy/module/gc/app_referents.py Wed Nov 24 14:21:00 2010 @@ -14,11 +14,25 @@ and [addr1]..[addrn] are addresses of other objects that this object points to. The full dump is a list of such objects, with a marker [0][0][0][-1] inserted after all GC roots, before all non-roots. + + If the argument is a filename and the 'zlib' module is available, + we also write a 'typeids.txt' in the same directory, if none exists. """ if isinstance(file, str): f = open(file, 'wb') gc._dump_rpy_heap(f.fileno()) f.close() + try: + import zlib, os + except ImportError: + pass + else: + filename2 = os.path.join(os.path.dirname(file), 'typeids.txt') + if not os.path.exists(filename2): + data = zlib.decompress(gc.get_typeids_z()) + f = open(filename2, 'wb') + f.write(data) + f.close() else: if isinstance(file, int): fd = file Modified: pypy/branch/fast-forward/pypy/module/gc/interp_gc.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/gc/interp_gc.py (original) +++ pypy/branch/fast-forward/pypy/module/gc/interp_gc.py Wed Nov 24 14:21:00 2010 @@ -10,6 +10,10 @@ from pypy.objspace.std.typeobject import MethodCache cache = space.fromcache(MethodCache) cache.clear() + if space.config.objspace.std.withmapdict: + from pypy.objspace.std.mapdict import IndexCache + cache = space.fromcache(IndexCache) + cache.clear() rgc.collect() return space.wrap(0) Modified: pypy/branch/fast-forward/pypy/module/gc/referents.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/gc/referents.py (original) +++ pypy/branch/fast-forward/pypy/module/gc/referents.py Wed Nov 24 14:21:00 2010 @@ -177,3 +177,9 @@ if not ok: raise missing_operation(space) _dump_rpy_heap.unwrap_spec = [ObjSpace, int] + +def get_typeids_z(space): + a = rgc.get_typeids_z() + s = ''.join([a[i] for i in range(len(a))]) + return space.wrap(s) +get_typeids_z.unwrap_spec = [ObjSpace] Modified: pypy/branch/fast-forward/pypy/module/gc/test/test_gc.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/gc/test/test_gc.py (original) +++ pypy/branch/fast-forward/pypy/module/gc/test/test_gc.py Wed Nov 24 14:21:00 2010 @@ -117,6 +117,33 @@ pass C().f() # Fill the method cache rlist.append(weakref.ref(C)) + for i in range(10): + f() + gc.collect() # the classes C should all go away here + # the last class won't go in mapdict, as long as the code object of f + # is around + rlist.pop() + for r in rlist: + assert r() is None + +class AppTestGcMapDictIndexCache(AppTestGcMethodCache): + def setup_class(cls): + cls.space = gettestobjspace(**{"objspace.std.withmethodcache": True, + "objspace.std.withmapdict": True}) + + + def test_clear_index_cache(self): + import gc, weakref + rlist = [] + def f(): + class C(object): + def f(self): + pass + c = C() + c.x = 1 + getattr(c, "x") # fill the index cache without using the local cache + getattr(c, "x") + rlist.append(weakref.ref(C)) for i in range(5): f() gc.collect() # the classes C should all go away here Modified: pypy/branch/fast-forward/pypy/module/imp/importing.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/imp/importing.py (original) +++ pypy/branch/fast-forward/pypy/module/imp/importing.py Wed Nov 24 14:21:00 2010 @@ -13,7 +13,7 @@ from pypy.rlib import streamio, jit from pypy.rlib.streamio import StreamErrors from pypy.rlib.rarithmetic import intmask -from pypy.rlib.objectmodel import we_are_translated +from pypy.rlib.objectmodel import we_are_translated, specialize SEARCH_ERROR = 0 PY_SOURCE = 1 @@ -26,10 +26,26 @@ # PY_CODERESOURCE = 8 IMP_HOOK = 9 -if sys.platform.startswith('win'): - so_extension = ".pyd" +if sys.platform == 'win32': + SO = ".pyd" else: - so_extension = ".so" + SO = ".so" +DEFAULT_SOABI = 'pypy-14' + + at specialize.memo() +def get_so_extension(space): + if space.config.objspace.soabi is not None: + soabi = space.config.objspace.soabi + else: + soabi = DEFAULT_SOABI + + if not soabi: + return SO + + if not space.config.translating: + soabi += 'i' + + return '.' + soabi + SO def find_modtype(space, filepart): """Check which kind of module to import for the given filepart, @@ -54,6 +70,7 @@ return PY_COMPILED, ".pyc", "rb" if space.config.objspace.usemodules.cpyext: + so_extension = get_so_extension(space) pydfile = filepart + so_extension if os.path.exists(pydfile) and case_ok(pydfile): return C_EXTENSION, so_extension, "rb" Modified: pypy/branch/fast-forward/pypy/module/imp/interp_imp.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/imp/interp_imp.py (original) +++ pypy/branch/fast-forward/pypy/module/imp/interp_imp.py Wed Nov 24 14:21:00 2010 @@ -8,10 +8,16 @@ def get_suffixes(space): w = space.wrap - return space.newlist([ + suffixes_w = [] + if space.config.objspace.usemodules.cpyext: + suffixes_w.append( + space.newtuple([w(importing.get_so_extension(space)), + w('rb'), w(importing.C_EXTENSION)])) + suffixes_w.extend([ space.newtuple([w('.py'), w('U'), w(importing.PY_SOURCE)]), space.newtuple([w('.pyc'), w('rb'), w(importing.PY_COMPILED)]), ]) + return space.newlist(suffixes_w) def get_magic(space): x = importing.get_pyc_magic(space) Modified: pypy/branch/fast-forward/pypy/module/imp/test/test_app.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/imp/test/test_app.py (original) +++ pypy/branch/fast-forward/pypy/module/imp/test/test_app.py Wed Nov 24 14:21:00 2010 @@ -47,6 +47,9 @@ elif mode == self.imp.PY_COMPILED: assert suffix in ('.pyc', '.pyo') assert type == 'rb' + elif mode == self.imp.C_EXTENSION: + assert suffix.endswith(('.pyd', '.so')) + assert type == 'rb' def test_obscure_functions(self): Modified: pypy/branch/fast-forward/pypy/module/imp/test/test_import.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/imp/test/test_import.py (original) +++ pypy/branch/fast-forward/pypy/module/imp/test/test_import.py Wed Nov 24 14:21:00 2010 @@ -478,6 +478,17 @@ except ImportError: pass +class TestAbi: + def test_abi_tag(self): + space1 = gettestobjspace(soabi='TEST') + space2 = gettestobjspace(soabi='') + if sys.platform == 'win32': + assert importing.get_so_extension(space1) == '.TESTi.pyd' + assert importing.get_so_extension(space2) == '.pyd' + else: + assert importing.get_so_extension(space1) == '.TESTi.so' + assert importing.get_so_extension(space2) == '.so' + def _getlong(data): x = marshal.dumps(data) return x[-4:] Modified: pypy/branch/fast-forward/pypy/module/pypyjit/__init__.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pypyjit/__init__.py (original) +++ pypy/branch/fast-forward/pypy/module/pypyjit/__init__.py Wed Nov 24 14:21:00 2010 @@ -15,6 +15,5 @@ # add the 'defaults' attribute from pypy.rlib.jit import PARAMETERS space = self.space - # XXX this is not really the default compiled into a pypy-c-jit XXX w_obj = space.wrap(PARAMETERS) space.setattr(space.wrap(self), space.wrap('defaults'), w_obj) Modified: pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/fast-forward/pypy/module/pypyjit/test/test_pypy_c.py Wed Nov 24 14:21:00 2010 @@ -377,10 +377,75 @@ ([1000], 49500), ([10000], 495000), ([100000], 4950000)) - assert len(self.loops) == 2 + assert len(self.loops) == 3 op, = self.get_by_bytecode("CALL_FUNCTION_KW") # XXX a bit too many guards, but better than before - assert len(op.get_opnames("guard")) <= 10 + assert len(op.get_opnames("guard")) <= 12 + + def test_stararg_virtual(self): + self.run_source(''' + d = {} + + def g(*args): + return len(args) + def h(a, b, c): + return c + + def main(x): + s = 0 + for i in range(x): + l = [i, x, 2] + s += g(*l) + s += h(*l) + s += g(i, x, 2) + for i in range(x): + l = [x, 2] + s += g(i, *l) + s += h(i, *l) + return s + ''', 100000, ([100], 1300), + ([1000], 13000), + ([10000], 130000), + ([100000], 1300000)) + assert len(self.loops) == 2 + ops = self.get_by_bytecode("CALL_FUNCTION_VAR") + assert len(ops) == 4 + for op in ops: + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 + + ops = self.get_by_bytecode("CALL_FUNCTION") + for op in ops: + assert len(op.get_opnames("new")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 + + def test_stararg(self): + self.run_source(''' + d = {} + + def g(*args): + return args[-1] + def h(*args): + return len(args) + + def main(x): + s = 0 + l = [] + i = 0 + while i < x: + l.append(1) + s += g(*l) + i = h(*l) + return s + ''', 100000, ([100], 100), + ([1000], 1000), + ([2000], 2000), + ([4000], 4000)) + assert len(self.loops) == 1 + ops = self.get_by_bytecode("CALL_FUNCTION_VAR") + for op in ops: + assert len(op.get_opnames("new_with_vtable")) == 0 + assert len(op.get_opnames("call_may_force")) == 0 def test_virtual_instance(self): self.run_source(''' Modified: pypy/branch/fast-forward/pypy/module/rctime/interp_time.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/rctime/interp_time.py (original) +++ pypy/branch/fast-forward/pypy/module/rctime/interp_time.py Wed Nov 24 14:21:00 2010 @@ -152,7 +152,7 @@ CLOCKS_PER_SEC = cConfig.CLOCKS_PER_SEC clock_t = cConfig.clock_t tm = cConfig.tm -glob_buf = lltype.malloc(tm, flavor='raw', zero=True) +glob_buf = lltype.malloc(tm, flavor='raw', zero=True, immortal=True) if cConfig.has_gettimeofday: c_gettimeofday = external('gettimeofday', [rffi.VOIDP, rffi.VOIDP], rffi.INT) Modified: pypy/branch/fast-forward/pypy/module/sys/state.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/sys/state.py (original) +++ pypy/branch/fast-forward/pypy/module/sys/state.py Wed Nov 24 14:21:00 2010 @@ -33,6 +33,8 @@ raise OSError(errno.ENOTDIR, path) +platform = sys.platform + def getinitialpath(prefix): from pypy.module.sys.version import CPYTHON_VERSION dirname = '%d.%d.%d' % (CPYTHON_VERSION[0], @@ -51,6 +53,15 @@ importlist.append(lib_pypy) importlist.append(python_std_lib_modified) importlist.append(python_std_lib) + # + # List here the extra platform-specific paths. + if platform != 'win32': + importlist.append(os.path.join(python_std_lib, 'plat-'+platform)) + if platform == 'darwin': + platmac = os.path.join(python_std_lib, 'plat-mac') + importlist.append(platmac) + importlist.append(os.path.join(platmac, 'lib-scriptpackages')) + # return importlist def pypy_initial_path(space, srcdir): Modified: pypy/branch/fast-forward/pypy/module/sys/test/test_initialpath.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/sys/test/test_initialpath.py (original) +++ pypy/branch/fast-forward/pypy/module/sys/test/test_initialpath.py Wed Nov 24 14:21:00 2010 @@ -15,4 +15,5 @@ def test_stdlib_in_prefix(tmpdir): dirs = build_hierarchy(tmpdir) path = getinitialpath(str(tmpdir)) - assert path == map(str, dirs) + # we get at least 'dirs', and maybe more (e.g. plat-linux2) + assert path[:len(dirs)] == map(str, dirs) Modified: pypy/branch/fast-forward/pypy/objspace/flow/model.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/flow/model.py (original) +++ pypy/branch/fast-forward/pypy/objspace/flow/model.py Wed Nov 24 14:21:00 2010 @@ -355,7 +355,7 @@ return "%r = %s(%s)" % (self.result, self.opname, ", ".join(map(repr, self.args))) -class Atom: +class Atom(object): def __init__(self, name): self.__name__ = name # make save_global happy def __repr__(self): Modified: pypy/branch/fast-forward/pypy/objspace/std/callmethod.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/callmethod.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/callmethod.py Wed Nov 24 14:21:00 2010 @@ -13,6 +13,9 @@ from pypy.interpreter import function from pypy.objspace.descroperation import object_getattribute from pypy.rlib import jit, rstack # for resume points +from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict, \ + LOOKUP_METHOD_mapdict_fill_cache_method + # This module exports two extra methods for StdObjSpaceFrame implementing # the LOOKUP_METHOD and CALL_METHOD opcodes in an efficient way, as well @@ -30,6 +33,13 @@ # space = f.space w_obj = f.popvalue() + + if space.config.objspace.std.withmapdict and not jit.we_are_jitted(): + # mapdict has an extra-fast version of this function + from pypy.objspace.std.mapdict import LOOKUP_METHOD_mapdict + if LOOKUP_METHOD_mapdict(f, nameindex, w_obj): + return + w_name = f.getname_w(nameindex) w_value = None @@ -50,6 +60,11 @@ # nothing in the instance f.pushvalue(w_descr) f.pushvalue(w_obj) + if (space.config.objspace.std.withmapdict and + not jit.we_are_jitted()): + # let mapdict cache stuff + LOOKUP_METHOD_mapdict_fill_cache_method( + f.getcode(), nameindex, w_obj, w_type, w_descr) return if w_value is None: w_value = space.getattr(w_obj, w_name) Modified: pypy/branch/fast-forward/pypy/objspace/std/mapdict.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/mapdict.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/mapdict.py Wed Nov 24 14:21:00 2010 @@ -1,4 +1,5 @@ from pypy.rlib import jit, objectmodel, debug +from pypy.rlib.rarithmetic import intmask, r_uint from pypy.interpreter.baseobjspace import W_Root from pypy.objspace.std.dictmultiobject import W_DictMultiObject @@ -15,24 +16,70 @@ # we want to propagate knowledge that the result cannot be negative class AbstractAttribute(object): - _immutable_fields_ = ['w_cls'] + _immutable_fields_ = ['terminator'] cache_attrs = None _size_estimate = 0 - def __init__(self, space, w_cls): + def __init__(self, space, terminator): self.space = space - self.w_cls = w_cls + assert isinstance(terminator, Terminator) + self.terminator = terminator def read(self, obj, selector): - raise NotImplementedError("abstract base class") + index = self.index(selector) + if index < 0: + return self.terminator._read_terminator(obj, selector) + return obj._mapdict_read_storage(index) def write(self, obj, selector, w_value): - raise NotImplementedError("abstract base class") + index = self.index(selector) + if index < 0: + return self.terminator._write_terminator(obj, selector, w_value) + obj._mapdict_write_storage(index, w_value) + return True def delete(self, obj, selector): return None def index(self, selector): + if (self.space.config.objspace.std.withmethodcache and + not jit.we_are_jitted()): + return self._index_cache(selector) + else: + return self._index(selector) + + @jit.dont_look_inside + def _index_cache(self, selector): + space = self.space + cache = space.fromcache(IndexCache) + SHIFT2 = r_uint.BITS - space.config.objspace.std.methodcachesizeexp + SHIFT1 = SHIFT2 - 5 + attrs_as_int = objectmodel.current_object_addr_as_int(self) + # ^^^Note: see comment in typeobject.py for + # _pure_lookup_where_with_method_cache() + hash_selector = objectmodel.compute_hash(selector) + product = intmask(attrs_as_int * hash_selector) + index_hash = (r_uint(product) ^ (r_uint(product) << SHIFT1)) >> SHIFT2 + # ^^^Note2: same comment too + cached_attr = cache.attrs[index_hash] + if cached_attr is self: + cached_selector = cache.selectors[index_hash] + if cached_selector == selector: + index = cache.indices[index_hash] + if space.config.objspace.std.withmethodcachecounter: + name = selector[0] + cache.hits[name] = cache.hits.get(name, 0) + 1 + return index + index = self._index(selector) + cache.attrs[index_hash] = self + cache.selectors[index_hash] = selector + cache.indices[index_hash] = index + if space.config.objspace.std.withmethodcachecounter: + name = selector[0] + cache.misses[name] = cache.misses.get(name, 0) + 1 + return index + + def _index(self, selector): return -1 def copy(self, obj): @@ -42,7 +89,7 @@ raise NotImplementedError("abstract base class") def get_terminator(self): - raise NotImplementedError("abstract base class") + return self.terminator def set_terminator(self, obj, terminator): raise NotImplementedError("abstract base class") @@ -95,15 +142,20 @@ raise NotImplementedError("abstract base class") def __repr__(self): - return "<%s w_cls=%s>" % (self.__class__.__name__, self.w_cls) + return "<%s>" % (self.__class__.__name__,) class Terminator(AbstractAttribute): + _immutable_fields_ = ['w_cls'] - def read(self, obj, selector): + def __init__(self, space, w_cls): + AbstractAttribute.__init__(self, space, self) + self.w_cls = w_cls + + def _read_terminator(self, obj, selector): return None - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): obj._get_mapdict_map().add_attr(obj, selector, w_value) return True @@ -116,9 +168,6 @@ def length(self): return 0 - def get_terminator(self): - return self - def set_terminator(self, obj, terminator): result = Object() result.space = self.space @@ -128,6 +177,9 @@ def remove_dict_entries(self, obj): return self.copy(obj) + def __repr__(self): + return "<%s w_cls=%s>" % (self.__class__.__name__, self.w_cls) + class DictTerminator(Terminator): _immutable_fields_ = ['devolved_dict_terminator'] def __init__(self, space, w_cls): @@ -142,27 +194,27 @@ class NoDictTerminator(Terminator): - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): if selector[1] == DICT: return False - return Terminator.write(self, obj, selector, w_value) + return Terminator._write_terminator(self, obj, selector, w_value) class DevolvedDictTerminator(Terminator): - def read(self, obj, selector): + def _read_terminator(self, obj, selector): if selector[1] == DICT: w_dict = obj.getdict() space = self.space return space.finditem_str(w_dict, selector[0]) - return Terminator.read(self, obj, selector) + return Terminator._read_terminator(self, obj, selector) - def write(self, obj, selector, w_value): + def _write_terminator(self, obj, selector, w_value): if selector[1] == DICT: w_dict = obj.getdict() space = self.space space.setitem_str(w_dict, selector[0], w_value) return True - return Terminator.write(self, obj, selector, w_value) + return Terminator._write_terminator(self, obj, selector, w_value) def delete(self, obj, selector): from pypy.interpreter.error import OperationError @@ -189,7 +241,7 @@ class PlainAttribute(AbstractAttribute): _immutable_fields_ = ['selector', 'position', 'back'] def __init__(self, selector, back): - AbstractAttribute.__init__(self, back.space, back.w_cls) + AbstractAttribute.__init__(self, back.space, back.terminator) self.selector = selector self.position = back.length() self.back = back @@ -199,17 +251,6 @@ w_value = self.read(obj, self.selector) new_obj._get_mapdict_map().add_attr(new_obj, self.selector, w_value) - def read(self, obj, selector): - if selector == self.selector: - return obj._mapdict_read_storage(self.position) - return self.back.read(obj, selector) - - def write(self, obj, selector, w_value): - if selector == self.selector: - obj._mapdict_write_storage(self.position, w_value) - return True - return self.back.write(obj, selector, w_value) - def delete(self, obj, selector): if selector == self.selector: # ok, attribute is deleted @@ -219,10 +260,10 @@ self._copy_attr(obj, new_obj) return new_obj - def index(self, selector): + def _index(self, selector): if selector == self.selector: return self.position - return self.back.index(selector) + return self.back._index(selector) def copy(self, obj): new_obj = self.back.copy(obj) @@ -232,9 +273,6 @@ def length(self): return self.position + 1 - def get_terminator(self): - return self.back.get_terminator() - def set_terminator(self, obj, terminator): new_obj = self.back.set_terminator(obj, terminator) self._copy_attr(obj, new_obj) @@ -268,6 +306,24 @@ # RPython reasons w_obj._set_mapdict_storage_and_map(new_obj.storage, new_obj.map) +class IndexCache(object): + def __init__(self, space): + assert space.config.objspace.std.withmethodcache + SIZE = 1 << space.config.objspace.std.methodcachesizeexp + self.attrs = [None] * SIZE + self._empty_selector = (None, INVALID) + self.selectors = [self._empty_selector] * SIZE + self.indices = [0] * SIZE + if space.config.objspace.std.withmethodcachecounter: + self.hits = {} + self.misses = {} + + def clear(self): + for i in range(len(self.attrs)): + self.attrs[i] = None + for i in range(len(self.selectors)): + self.selectors[i] = self._empty_selector + # ____________________________________________________________ # object implementation @@ -328,7 +384,7 @@ assert flag def getclass(self, space): - return self._get_mapdict_map().w_cls + return self._get_mapdict_map().terminator.w_cls def setclass(self, space, w_cls): new_obj = self._get_mapdict_map().set_terminator(self, w_cls.terminator) @@ -373,6 +429,7 @@ self.storage = make_sure_not_resized([None] * map.size_estimate()) def _mapdict_read_storage(self, index): + assert index >= 0 return self.storage[index] def _mapdict_write_storage(self, index, value): self.storage[index] = value @@ -440,6 +497,7 @@ return rerased.unerase_fixedsizelist(erased, W_Root) def _mapdict_read_storage(self, index): + assert index >= 0 for i in rangenmin1: if index == i: erased = getattr(self, "_value%s" % i) @@ -604,30 +662,54 @@ map = None version_tag = None index = 0 + w_method = None # for callmethod success_counter = 0 failure_counter = 0 + def is_valid_for_obj(self, w_obj): + map = w_obj._get_mapdict_map() + return self.is_valid_for_map(map) + + def is_valid_for_map(self, map): + if map is self.map: + version_tag = map.terminator.w_cls.version_tag() + if version_tag is self.version_tag: + # everything matches, it's incredibly fast + if map.space.config.objspace.std.withmethodcachecounter: + self.success_counter += 1 + return True + return False + INVALID_CACHE_ENTRY = CacheEntry() INVALID_CACHE_ENTRY.map = objectmodel.instantiate(AbstractAttribute) # different from any real map ^^^ -INVALID_CACHE_ENTRY.map.w_cls = None +INVALID_CACHE_ENTRY.map.terminator = None + def init_mapdict_cache(pycode): num_entries = len(pycode.co_names_w) pycode._mapdict_caches = [INVALID_CACHE_ENTRY] * num_entries +def _fill_cache(pycode, nameindex, map, version_tag, index, w_method=None): + entry = pycode._mapdict_caches[nameindex] + if entry is INVALID_CACHE_ENTRY: + entry = CacheEntry() + pycode._mapdict_caches[nameindex] = entry + entry.map = map + entry.version_tag = version_tag + entry.index = index + entry.w_method = w_method + if pycode.space.config.objspace.std.withmethodcachecounter: + entry.failure_counter += 1 + def LOAD_ATTR_caching(pycode, w_obj, nameindex): # this whole mess is to make the interpreter quite a bit faster; it's not # used if we_are_jitted(). entry = pycode._mapdict_caches[nameindex] map = w_obj._get_mapdict_map() - if map is entry.map: - version_tag = map.w_cls.version_tag() - if version_tag is entry.version_tag: - # everything matches, it's incredibly fast - if pycode.space.config.objspace.std.withmethodcachecounter: - entry.success_counter += 1 - return w_obj._mapdict_read_storage(entry.index) + if entry.is_valid_for_map(map) and entry.w_method is None: + # everything matches, it's incredibly fast + return w_obj._mapdict_read_storage(entry.index) return LOAD_ATTR_slowpath(pycode, w_obj, nameindex, map) LOAD_ATTR_caching._always_inline_ = True @@ -635,7 +717,7 @@ space = pycode.space w_name = pycode.co_names_w[nameindex] if map is not None: - w_type = map.w_cls + w_type = map.terminator.w_cls w_descr = w_type.getattribute_if_not_from_object() if w_descr is not None: return space._handle_getattribute(w_descr, w_obj, w_name) @@ -655,17 +737,30 @@ if selector[1] != INVALID: index = map.index(selector) if index >= 0: - entry = pycode._mapdict_caches[nameindex] - if entry is INVALID_CACHE_ENTRY: - entry = CacheEntry() - pycode._mapdict_caches[nameindex] = entry - entry.map = map - entry.version_tag = version_tag - entry.index = index - if space.config.objspace.std.withmethodcachecounter: - entry.failure_counter += 1 + _fill_cache(pycode, nameindex, map, version_tag, index) return w_obj._mapdict_read_storage(index) if space.config.objspace.std.withmethodcachecounter: INVALID_CACHE_ENTRY.failure_counter += 1 return space.getattr(w_obj, w_name) LOAD_ATTR_slowpath._dont_inline_ = True + +def LOOKUP_METHOD_mapdict(f, nameindex, w_obj): + space = f.space + pycode = f.getcode() + entry = pycode._mapdict_caches[nameindex] + if entry.is_valid_for_obj(w_obj): + w_method = entry.w_method + if w_method is not None: + f.pushvalue(w_method) + f.pushvalue(w_obj) + return True + return False + +def LOOKUP_METHOD_mapdict_fill_cache_method(pycode, nameindex, w_obj, w_type, w_method): + version_tag = w_type.version_tag() + if version_tag is None: + return + map = w_obj._get_mapdict_map() + if map is None: + return + _fill_cache(pycode, nameindex, map, version_tag, -1, w_method) Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_dictmultiobject.py Wed Nov 24 14:21:00 2010 @@ -757,6 +757,7 @@ withdictmeasurement = False withsmalldicts = False withcelldict = False + withmethodcache = False class opcodes: CALL_LIKELY_BUILTIN = False Modified: pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/test/test_mapdict.py Wed Nov 24 14:21:00 2010 @@ -24,13 +24,13 @@ hasdict = False def test_plain_attribute(): - space = " " w_cls = "class" aa = PlainAttribute(("b", DICT), PlainAttribute(("a", DICT), Terminator(space, w_cls))) assert aa.space is space - assert aa.w_cls is w_cls + assert aa.terminator.w_cls is w_cls + assert aa.get_terminator() is aa.terminator obj = Object() obj.map, obj.storage = aa, [10, 20] @@ -604,7 +604,8 @@ from pypy.interpreter import gateway cls.space = gettestobjspace( **{"objspace.std.withmapdict": True, - "objspace.std.withmethodcachecounter": True}) + "objspace.std.withmethodcachecounter": True, + "objspace.opcodes.CALL_METHOD": True}) # def check(space, w_func, name): w_code = space.getattr(w_func, space.wrap('func_code')) @@ -785,6 +786,135 @@ res = self.check(f, 'x') assert res == (0, 0, 1) + def test_call_method_uses_cache(self): + # bit sucky + global C + + class C(object): + def m(*args): + return args + C.sm = staticmethod(C.m.im_func) + C.cm = classmethod(C.m.im_func) + + exec """if 1: + + def f(): + c = C() + res = c.m(1) + assert res == (c, 1) + return 42 + + def g(): + c = C() + res = c.sm(1) + assert res == (1, ) + return 42 + + def h(): + c = C() + res = c.cm(1) + assert res == (C, 1) + return 42 + """ + res = self.check(f, 'm') + assert res == (1, 0, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + res = self.check(f, 'm') + assert res == (0, 1, 0) + + # static methods are not cached + res = self.check(g, 'sm') + assert res == (0, 0, 0) + res = self.check(g, 'sm') + assert res == (0, 0, 0) + + # neither are class methods + res = self.check(h, 'cm') + assert res == (0, 0, 0) + res = self.check(h, 'cm') + assert res == (0, 0, 0) + + def test_mix_cache_bug(self): + # bit sucky + global C + + class C(object): + def m(*args): + return args + + exec """if 1: + + def f(): + c = C() + res = c.m(1) + assert res == (c, 1) + bm = c.m + res = bm(1) + assert res == (c, 1) + return 42 + + """ + res = self.check(f, 'm') + assert res == (1, 1, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + res = self.check(f, 'm') + assert res == (0, 2, 1) + +class AppTestGlobalCaching(AppTestWithMapDict): + def setup_class(cls): + cls.space = gettestobjspace( + **{"objspace.std.withmethodcachecounter": True, + "objspace.std.withmapdict": True, + "objspace.opcodes.CALL_METHOD": True}) + + def test_mix_classes(self): + import __pypy__ + class A(object): + def f(self): + return 42 + class B(object): + def f(self): + return 43 + class C(object): + def f(self): + return 44 + l = [A(), B(), C()] * 10 + __pypy__.reset_method_cache_counter() + # 'exec' to make sure that a.f() is compiled with CALL_METHOD + exec """for i, a in enumerate(l): + assert a.f() == 42 + i % 3 +""" + cache_counter = __pypy__.mapdict_cache_counter("f") + assert cache_counter[0] >= 15 + assert cache_counter[1] >= 3 # should be (27, 3) + assert sum(cache_counter) == 30 + + def test_mix_classes_attribute(self): + import __pypy__ + class A(object): + def __init__(self): + self.x = 42 + class B(object): + def __init__(self): + self.x = 43 + class C(object): + def __init__(self): + self.x = 44 + l = [A(), B(), C()] * 10 + __pypy__.reset_method_cache_counter() + for i, a in enumerate(l): + assert a.x == 42 + i % 3 + cache_counter = __pypy__.mapdict_cache_counter("x") + assert cache_counter[0] >= 15 + assert cache_counter[1] >= 3 # should be (27, 3) + assert sum(cache_counter) == 30 + class TestDictSubclassShortcutBug(object): def setup_class(cls): cls.space = gettestobjspace( Modified: pypy/branch/fast-forward/pypy/rlib/clibffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/clibffi.py (original) +++ pypy/branch/fast-forward/pypy/rlib/clibffi.py Wed Nov 24 14:21:00 2010 @@ -438,7 +438,8 @@ flags=FUNCFLAG_CDECL): AbstractFuncPtr.__init__(self, "callback", argtypes, restype, flags) self.ll_closure = closureHeap.alloc() - self.ll_userdata = lltype.malloc(USERDATA_P.TO, flavor='raw') + self.ll_userdata = lltype.malloc(USERDATA_P.TO, flavor='raw', + track_allocation=False) self.ll_userdata.callback = rffi.llhelper(CALLBACK_TP, func) self.ll_userdata.addarg = additional_arg res = c_ffi_prep_closure(self.ll_closure, self.ll_cif, @@ -453,7 +454,7 @@ closureHeap.free(self.ll_closure) self.ll_closure = lltype.nullptr(FFI_CLOSUREP.TO) if self.ll_userdata: - lltype.free(self.ll_userdata, flavor='raw') + lltype.free(self.ll_userdata, flavor='raw', track_allocation=False) self.ll_userdata = lltype.nullptr(USERDATA_P.TO) class RawFuncPtr(AbstractFuncPtr): Modified: pypy/branch/fast-forward/pypy/rlib/debug.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/debug.py (original) +++ pypy/branch/fast-forward/pypy/rlib/debug.py Wed Nov 24 14:21:00 2010 @@ -53,13 +53,11 @@ _log = None # patched from tests to be an object of class DebugLog # or compatible -_stderr = sys.stderr # alternatively, this is patched from tests - # (redirects debug_print(), but not debug_start/stop) def debug_print(*args): for arg in args: - print >> _stderr, arg, - print >> _stderr + print >> sys.stderr, arg, + print >> sys.stderr if _log is not None: _log.debug_print(*args) @@ -87,13 +85,15 @@ _stop_colors = "" def debug_start(category): - print >> sys.stderr, '%s[%s] {%s%s' % (_start_colors_1, time.clock(), + c = int(time.clock() * 100) + print >> sys.stderr, '%s[%x] {%s%s' % (_start_colors_1, c, category, _stop_colors) if _log is not None: _log.debug_start(category) def debug_stop(category): - print >> sys.stderr, '%s[%s] %s}%s' % (_start_colors_2, time.clock(), + c = int(time.clock() * 100) + print >> sys.stderr, '%s[%x] %s}%s' % (_start_colors_2, c, category, _stop_colors) if _log is not None: _log.debug_stop(category) Modified: pypy/branch/fast-forward/pypy/rlib/jit.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/jit.py (original) +++ pypy/branch/fast-forward/pypy/rlib/jit.py Wed Nov 24 14:21:00 2010 @@ -260,17 +260,12 @@ OPTIMIZER_NO_PERFECTSPEC = 1 OPTIMIZER_FULL = 2 -DEBUG_OFF = 0 -DEBUG_PROFILE = 1 -DEBUG_STEPS = 2 -DEBUG_DETAILED = 3 - PARAMETERS = {'threshold': 1000, 'trace_eagerness': 200, 'trace_limit': 10000, 'inlining': False, 'optimizer': OPTIMIZER_FULL, - 'debug' : DEBUG_STEPS, + 'loop_longevity': 1000, } unroll_parameters = unrolling_iterable(PARAMETERS.keys()) Modified: pypy/branch/fast-forward/pypy/rlib/rgc.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rgc.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rgc.py Wed Nov 24 14:21:00 2010 @@ -379,6 +379,11 @@ "NOT_RPYTHON" raise NotImplementedError +def get_typeids_z(): + "NOT_RPYTHON" + raise NotImplementedError + +ARRAY_OF_CHAR = lltype.Array(lltype.Char) NULL_GCREF = lltype.nullptr(llmemory.GCREF.TO) class _GcRef(object): @@ -530,3 +535,12 @@ vlist = hop.inputargs(lltype.Signed) hop.exception_is_here() return hop.genop('gc_dump_rpy_heap', vlist, resulttype = hop.r_result) + +class Entry(ExtRegistryEntry): + _about_ = get_typeids_z + def compute_result_annotation(self): + from pypy.annotation.model import SomePtr + return SomePtr(lltype.Ptr(ARRAY_OF_CHAR)) + def specialize_call(self, hop): + hop.exception_is_here() + return hop.genop('gc_typeids_z', [], resulttype = hop.r_result) Modified: pypy/branch/fast-forward/pypy/rpython/annlowlevel.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/annlowlevel.py (original) +++ pypy/branch/fast-forward/pypy/rpython/annlowlevel.py Wed Nov 24 14:21:00 2010 @@ -136,7 +136,7 @@ return funcdesc.cachedgraph(TYPE, alt_name=valid_identifier(alt_name)) -class MixLevelHelperAnnotator: +class MixLevelHelperAnnotator(object): def __init__(self, rtyper): self.rtyper = rtyper Modified: pypy/branch/fast-forward/pypy/rpython/llinterp.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/llinterp.py (original) +++ pypy/branch/fast-forward/pypy/rpython/llinterp.py Wed Nov 24 14:21:00 2010 @@ -912,6 +912,9 @@ def op_gc_dump_rpy_heap(self): raise NotImplementedError("gc_dump_rpy_heap") + def op_gc_typeids_z(self): + raise NotImplementedError("gc_typeids_z") + def op_do_malloc_fixedsize_clear(self): raise NotImplementedError("do_malloc_fixedsize_clear") Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py (original) +++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/llarena.py Wed Nov 24 14:21:00 2010 @@ -440,7 +440,8 @@ [rffi.INT], rffi.INT, sandboxsafe=True, _nowrapper=True) - _dev_zero = rffi.str2charp_immortal('/dev/zero') # prebuilt + _dev_zero = rffi.str2charp('/dev/zero') # prebuilt + lltype.render_immortal(_dev_zero) def clear_large_memory_chunk(baseaddr, size): # on some Unixy platforms, reading from /dev/zero is the fastest way Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/llmemory.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/lltypesystem/llmemory.py (original) +++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/llmemory.py Wed Nov 24 14:21:00 2010 @@ -766,8 +766,10 @@ return '<%s>' % (self,) def __str__(self): return 'gctransformed_wref(%s)' % (self._ptr,) - def _normalizedcontainer(self): - return self._ptr._obj + def _normalizedcontainer(self, check=True): + return self._ptr._getobj(check=check)._normalizedcontainer(check=check) + def _was_freed(self): + return self._ptr._was_freed() # ____________________________________________________________ Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/lloperation.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/lltypesystem/lloperation.py (original) +++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/lloperation.py Wed Nov 24 14:21:00 2010 @@ -478,6 +478,7 @@ 'gc_get_rpy_type_index': LLOp(), 'gc_is_rpy_instance' : LLOp(), 'gc_dump_rpy_heap' : LLOp(), + 'gc_typeids_z' : LLOp(), # ------- JIT & GC interaction, only for some GCs ---------- Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/lltype.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/lltypesystem/lltype.py (original) +++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/lltype.py Wed Nov 24 14:21:00 2010 @@ -1875,6 +1875,13 @@ leakfinder.remember_free(p._obj0) p._obj0._free() +def render_immortal(p, track_allocation=True): + T = typeOf(p) + if not isinstance(T, Ptr) or p._togckind() != 'raw': + raise TypeError, "free(): only for pointers to non-gc containers" + if track_allocation: + leakfinder.remember_free(p._obj0) + def _make_scoped_allocator(T): class ScopedAlloc: def __init__(self, n=None, zero=False): Modified: pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py (original) +++ pypy/branch/fast-forward/pypy/rpython/lltypesystem/rffi.py Wed Nov 24 14:21:00 2010 @@ -609,15 +609,6 @@ return array str2charp._annenforceargs_ = [strtype] - def str2charp_immortal(s): - "NOT_RPYTHON" - array = lltype.malloc(TYPEP.TO, len(s) + 1, flavor='raw', - immortal=True) - for i in range(len(s)): - array[i] = s[i] - array[len(s)] = lastchar - return array - def free_charp(cp): lltype.free(cp, flavor='raw') @@ -736,19 +727,19 @@ l = [cp[i] for i in range(size)] return emptystr.join(l) - return (str2charp, str2charp_immortal, free_charp, charp2str, + return (str2charp, free_charp, charp2str, get_nonmovingbuffer, free_nonmovingbuffer, alloc_buffer, str_from_buffer, keep_buffer_alive_until_here, charp2strn, charpsize2str, ) -(str2charp, str2charp_immortal, free_charp, charp2str, +(str2charp, free_charp, charp2str, get_nonmovingbuffer, free_nonmovingbuffer, alloc_buffer, str_from_buffer, keep_buffer_alive_until_here, charp2strn, charpsize2str, ) = make_string_mappings(str) -(unicode2wcharp, unicode2wcharp_immortal, free_wcharp, wcharp2unicode, +(unicode2wcharp, free_wcharp, wcharp2unicode, get_nonmoving_unicodebuffer, free_nonmoving_unicodebuffer, alloc_unicodebuffer, unicode_from_buffer, keep_unicodebuffer_alive_until_here, wcharp2unicoden, wcharpsize2unicode, Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/base.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gc/base.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gc/base.py Wed Nov 24 14:21:00 2010 @@ -5,7 +5,6 @@ from pypy.rpython.memory.support import get_address_stack, get_address_deque from pypy.rpython.memory.support import AddressDict from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage -from pypy.rlib.rarithmetic import r_uint TYPEID_MAP = lltype.GcStruct('TYPEID_MAP', ('count', lltype.Signed), ('size', lltype.Signed), @@ -19,6 +18,7 @@ malloc_zero_filled = False prebuilt_gc_objects_are_static_roots = True object_minimal_size = 0 + gcflag_extra = 0 # or a real GC flag that is always 0 when not collecting def __init__(self, config, chunk_size=DEFAULT_CHUNK_SIZE, translated_to_c=True): @@ -410,42 +410,6 @@ GCClass = getattr(module, classname) return GCClass, GCClass.TRANSLATION_PARAMS -def _read_float_and_factor_from_env(varname): - import os - value = os.environ.get(varname) - if value: - if len(value) > 1 and value[-1] in 'bB': - value = value[:-1] - realvalue = value[:-1] - if value[-1] in 'kK': - factor = 1024 - elif value[-1] in 'mM': - factor = 1024*1024 - elif value[-1] in 'gG': - factor = 1024*1024*1024 - else: - factor = 1 - realvalue = value - try: - return (float(realvalue), factor) - except ValueError: - pass - return (0.0, 0) - -def read_from_env(varname): - value, factor = _read_float_and_factor_from_env(varname) - return int(value * factor) - -def read_uint_from_env(varname): - value, factor = _read_float_and_factor_from_env(varname) - return r_uint(value * factor) - -def read_float_from_env(varname): - value, factor = _read_float_and_factor_from_env(varname) - if factor != 1: - return 0.0 - return value - def _convert_callback_formats(callback): callback = getattr(callback, 'im_func', callback) if callback not in _converted_callback_formats: Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/generation.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gc/generation.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gc/generation.py Wed Nov 24 14:21:00 2010 @@ -2,7 +2,7 @@ from pypy.rpython.memory.gc.semispace import SemiSpaceGC from pypy.rpython.memory.gc.semispace import GCFLAG_EXTERNAL, GCFLAG_FORWARDED from pypy.rpython.memory.gc.semispace import GC_HASH_TAKEN_ADDR -from pypy.rpython.memory.gc.base import read_from_env +from pypy.rpython.memory.gc import env from pypy.rpython.lltypesystem.llmemory import NULL, raw_malloc_usage from pypy.rpython.lltypesystem import lltype, llmemory, llarena from pypy.rlib.objectmodel import free_non_gc_object @@ -93,7 +93,7 @@ if self.auto_nursery_size: newsize = nursery_size_from_env() if newsize <= 0: - newsize = estimate_best_nursery_size() + newsize = env.estimate_best_nursery_size() if newsize > 0: self.set_nursery_size(newsize) @@ -633,139 +633,5 @@ # ____________________________________________________________ -import os - def nursery_size_from_env(): - return read_from_env('PYPY_GENERATIONGC_NURSERY') - -def best_nursery_size_for_L2cache(L2cache): - # Heuristically, the best nursery size to choose is about half - # of the L2 cache. XXX benchmark some more. - return L2cache // 2 - - -if sys.platform == 'linux2': - def estimate_best_nursery_size(): - """Try to estimate the best nursery size at run-time, depending - on the machine we are running on. - """ - debug_start("gc-L2cache") - L2cache = sys.maxint - try: - fd = os.open('/proc/cpuinfo', os.O_RDONLY, 0644) - try: - data = [] - while True: - buf = os.read(fd, 4096) - if not buf: - break - data.append(buf) - finally: - os.close(fd) - except OSError: - pass - else: - data = ''.join(data) - linepos = 0 - while True: - start = findend(data, '\ncache size', linepos) - if start < 0: - break # done - linepos = findend(data, '\n', start) - if linepos < 0: - break # no end-of-line?? - # *** data[start:linepos] == " : 2048 KB\n" - start = skipspace(data, start) - if data[start] != ':': - continue - # *** data[start:linepos] == ": 2048 KB\n" - start = skipspace(data, start + 1) - # *** data[start:linepos] == "2048 KB\n" - end = start - while '0' <= data[end] <= '9': - end += 1 - # *** data[start:end] == "2048" - if start == end: - continue - number = int(data[start:end]) - # *** data[end:linepos] == " KB\n" - end = skipspace(data, end) - if data[end] not in ('K', 'k'): # assume kilobytes for now - continue - number = number * 1024 - # for now we look for the smallest of the L2 caches of the CPUs - if number < L2cache: - L2cache = number - - debug_print("L2cache =", L2cache) - debug_stop("gc-L2cache") - - if L2cache < sys.maxint: - return best_nursery_size_for_L2cache(L2cache) - else: - # Print a top-level warning even in non-debug builds - llop.debug_print(lltype.Void, - "Warning: cannot find your CPU L2 cache size in /proc/cpuinfo") - return -1 - - def findend(data, pattern, pos): - pos = data.find(pattern, pos) - if pos < 0: - return -1 - return pos + len(pattern) - - def skipspace(data, pos): - while data[pos] in (' ', '\t'): - pos += 1 - return pos - -elif sys.platform == 'darwin': - from pypy.rpython.lltypesystem import rffi - - sysctlbyname = rffi.llexternal('sysctlbyname', - [rffi.CCHARP, rffi.VOIDP, rffi.SIZE_TP, - rffi.VOIDP, rffi.SIZE_T], - rffi.INT, - sandboxsafe=True) - - def estimate_best_nursery_size(): - """Try to estimate the best nursery size at run-time, depending - on the machine we are running on. - """ - debug_start("gc-L2cache") - L2cache = 0 - l2cache_p = lltype.malloc(rffi.LONGLONGP.TO, 1, flavor='raw') - try: - len_p = lltype.malloc(rffi.SIZE_TP.TO, 1, flavor='raw') - try: - size = rffi.sizeof(rffi.LONGLONG) - l2cache_p[0] = rffi.cast(rffi.LONGLONG, 0) - len_p[0] = rffi.cast(rffi.SIZE_T, size) - # XXX a hack for llhelper not being robust-enough - result = sysctlbyname("hw.l2cachesize", - rffi.cast(rffi.VOIDP, l2cache_p), - len_p, - lltype.nullptr(rffi.VOIDP.TO), - rffi.cast(rffi.SIZE_T, 0)) - if (rffi.cast(lltype.Signed, result) == 0 and - rffi.cast(lltype.Signed, len_p[0]) == size): - L2cache = rffi.cast(lltype.Signed, l2cache_p[0]) - if rffi.cast(rffi.LONGLONG, L2cache) != l2cache_p[0]: - L2cache = 0 # overflow! - finally: - lltype.free(len_p, flavor='raw') - finally: - lltype.free(l2cache_p, flavor='raw') - debug_print("L2cache =", L2cache) - debug_stop("gc-L2cache") - if L2cache > 0: - return best_nursery_size_for_L2cache(L2cache) - else: - # Print a top-level warning even in non-debug builds - llop.debug_print(lltype.Void, - "Warning: cannot find your CPU L2 cache size with sysctl()") - return -1 - -else: - def estimate_best_nursery_size(): - return -1 # XXX implement me for other platforms + return env.read_from_env('PYPY_GENERATIONGC_NURSERY') Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/inspector.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gc/inspector.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gc/inspector.py Wed Nov 24 14:21:00 2010 @@ -4,7 +4,7 @@ from pypy.rpython.lltypesystem import lltype, llmemory, rffi from pypy.rlib.objectmodel import free_non_gc_object from pypy.rpython.module.ll_os import underscore_on_windows -from pypy.rlib import rposix +from pypy.rlib import rposix, rgc from pypy.rpython.memory.support import AddressDict, get_address_stack @@ -107,15 +107,18 @@ def __init__(self, gc, fd): self.gc = gc + self.gcflag = gc.gcflag_extra self.fd = rffi.cast(rffi.INT, fd) self.writebuffer = lltype.malloc(rffi.LONGP.TO, self.BUFSIZE, flavor='raw') self.buf_count = 0 - self.seen = AddressDict() + if self.gcflag == 0: + self.seen = AddressDict() self.pending = AddressStack() def delete(self): - self.seen.delete() + if self.gcflag == 0: + self.seen.delete() self.pending.delete() lltype.free(self.writebuffer, flavor='raw') free_non_gc_object(self) @@ -140,6 +143,8 @@ self.flush() write._always_inline_ = True + # ---------- + def write_marker(self): self.write(0) self.write(0) @@ -161,9 +166,15 @@ self.add(obj) def add(self, obj): - if not self.seen.contains(obj): - self.seen.setitem(obj, obj) - self.pending.append(obj) + if self.gcflag == 0: + if not self.seen.contains(obj): + self.seen.setitem(obj, obj) + self.pending.append(obj) + else: + hdr = self.gc.header(obj) + if (hdr.tid & self.gcflag) == 0: + hdr.tid |= self.gcflag + self.pending.append(obj) def add_roots(self): self.gc.enumerate_all_roots(_hd_add_root, self) @@ -177,13 +188,53 @@ while pending.non_empty(): self.writeobj(pending.pop()) + # ---------- + # A simplified copy of the above, to make sure we walk again all the + # objects to clear the 'gcflag'. + + def unwriteobj(self, obj): + gc = self.gc + gc.trace(obj, self._unwriteref, None) + + def _unwriteref(self, pointer, _): + obj = pointer.address[0] + self.unadd(obj) + + def unadd(self, obj): + assert self.gcflag != 0 + hdr = self.gc.header(obj) + if (hdr.tid & self.gcflag) != 0: + hdr.tid &= ~self.gcflag + self.pending.append(obj) + + def clear_gcflag_again(self): + self.gc.enumerate_all_roots(_hd_unadd_root, self) + pendingroots = self.pending + self.pending = AddressStack() + self.unwalk(pendingroots) + pendingroots.delete() + + def unwalk(self, pending): + while pending.non_empty(): + self.unwriteobj(pending.pop()) + def _hd_add_root(obj, heap_dumper): heap_dumper.add(obj) +def _hd_unadd_root(obj, heap_dumper): + heap_dumper.unadd(obj) + def dump_rpy_heap(gc, fd): heapdumper = HeapDumper(gc, fd) heapdumper.add_roots() heapdumper.walk(heapdumper.pending) heapdumper.flush() + if heapdumper.gcflag != 0: + heapdumper.clear_gcflag_again() + heapdumper.unwalk(heapdumper.pending) heapdumper.delete() return True + +def get_typeids_z(gc): + srcaddress = gc.root_walker.gcdata.typeids_z + return llmemory.cast_adr_to_ptr(srcaddress, lltype.Ptr(rgc.ARRAY_OF_CHAR)) Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/markcompact.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gc/markcompact.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gc/markcompact.py Wed Nov 24 14:21:00 2010 @@ -1,5 +1,6 @@ from pypy.rpython.lltypesystem import lltype, llmemory, llarena, llgroup -from pypy.rpython.memory.gc.base import MovingGCBase, read_from_env +from pypy.rpython.memory.gc.base import MovingGCBase +from pypy.rpython.memory.gc import env from pypy.rlib.debug import ll_assert, have_debug_prints from pypy.rlib.debug import debug_print, debug_start, debug_stop from pypy.rpython.memory.support import get_address_stack, get_address_deque @@ -110,10 +111,10 @@ return next def setup(self): - envsize = read_from_env('PYPY_MARKCOMPACTGC_MAX') + envsize = env.read_from_env('PYPY_MARKCOMPACTGC_MAX') if envsize >= 4096: self.space_size = envsize & ~4095 - mincollect = read_from_env('PYPY_MARKCOMPACTGC_MIN') + mincollect = env.read_from_env('PYPY_MARKCOMPACTGC_MIN') if mincollect >= 4096: self.min_next_collect_after = mincollect Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gc/minimark.py Wed Nov 24 14:21:00 2010 @@ -1,9 +1,48 @@ +""" MiniMark GC. + +Environment variables can be used to fine-tune the following parameters: + + PYPY_GC_NURSERY The nursery size. Defaults to half the size of + the L2 cache. Try values like '1.2MB'. + + PYPY_GC_MAJOR_COLLECT Major collection memory factor. Default is '1.82', + which means trigger a major collection when the + memory consumed equals 1.82 times the memory + really used at the end of the previous major + collection. + + PYPY_GC_GROWTH Major collection threshold's max growth rate. + Default is '1.4'. Useful to collect more often + than normally on sudden memory growth, e.g. when + there is a temporary peak in memory usage. + + PYPY_GC_MAX The max heap size. If coming near this limit, it + will first collect more often, then raise an + RPython MemoryError, and if that is not enough, + crash the program with a fatal error. Try values + like '1.6GB'. + + PYPY_GC_MAX_DELTA The major collection threshold will never be set + to more than PYPY_GC_MAX_DELTA the amount really + used after a collection. Defaults to 1/8th of the + total RAM size (which is constrained to be at most + 2/3/4GB on 32-bit systems). Try values like '200MB'. + + PYPY_GC_MIN Don't collect while the memory size is below this + limit. Useful to avoid spending all the time in + the GC in very small programs. Defaults to 8 + times the nursery. +""" +# XXX Should find a way to bound the major collection threshold by the +# XXX total addressable size. Maybe by keeping some minimarkpage arenas +# XXX pre-reserved, enough for a few nursery collections? What about +# XXX raw-malloced memory? import sys from pypy.rpython.lltypesystem import lltype, llmemory, llarena, llgroup from pypy.rpython.lltypesystem.lloperation import llop from pypy.rpython.lltypesystem.llmemory import raw_malloc_usage from pypy.rpython.memory.gc.base import GCBase, MovingGCBase -from pypy.rpython.memory.gc import minimarkpage, base, generation +from pypy.rpython.memory.gc import minimarkpage, env from pypy.rlib.rarithmetic import ovfcheck, LONG_BIT, intmask, r_uint from pypy.rlib.rarithmetic import LONG_BIT_SHIFT from pypy.rlib.debug import ll_assert, debug_print, debug_start, debug_stop @@ -63,6 +102,7 @@ needs_write_barrier = True prebuilt_gc_objects_are_static_roots = False malloc_zero_filled = True # xxx experiment with False + gcflag_extra = GCFLAG_FINALIZATION_ORDERING # All objects start with a HDR, i.e. with a field 'tid' which contains # a word. This word is divided in two halves: the lower half contains @@ -87,13 +127,8 @@ TRANSLATION_PARAMS = { # Automatically adjust the size of the nursery and the - # 'major_collection_threshold' from the environment. For - # 'nursery_size' it will look it up in the env var - # PYPY_GC_NURSERY and fall back to half the size of - # the L2 cache. For 'major_collection_threshold' it will look - # it up in the env var PYPY_GC_MAJOR_COLLECT. It also sets - # 'max_heap_size' to PYPY_GC_MAX. Finally, PYPY_GC_MIN sets - # the minimal value of 'next_major_collection_threshold'. + # 'major_collection_threshold' from the environment. + # See docstring at the start of the file. "read_from_env": True, # The size of the nursery. Note that this is only used as a @@ -121,6 +156,13 @@ # we trigger the next major collection. "major_collection_threshold": 1.82, + # Threshold to avoid that the total heap size grows by a factor of + # major_collection_threshold at every collection: it can only + # grow at most by the following factor from one collection to the + # next. Used e.g. when there is a sudden, temporary peak in memory + # usage; this avoids that the upper bound grows too fast. + "growth_rate_max": 1.4, + # The number of array indices that are mapped to a single bit in # write_barrier_from_array(). Must be a power of two. The default # value of 128 means that card pages are 512 bytes (1024 on 64-bits) @@ -146,6 +188,7 @@ arena_size=64*WORD, small_request_threshold=5*WORD, major_collection_threshold=2.5, + growth_rate_max=2.5, # for tests card_page_indices=0, large_object=8*WORD, large_object_gcptrs=10*WORD, @@ -157,10 +200,12 @@ self.nursery_size = nursery_size self.small_request_threshold = small_request_threshold self.major_collection_threshold = major_collection_threshold + self.growth_rate_max = growth_rate_max self.num_major_collects = 0 self.min_heap_size = 0.0 self.max_heap_size = 0.0 self.max_heap_size_already_raised = False + self.max_delta = float(r_uint(-1)) # self.card_page_indices = card_page_indices if self.card_page_indices > 0: @@ -245,32 +290,42 @@ # # From there on, the GC is fully initialized and the code # below can use it - newsize = base.read_from_env('PYPY_GC_NURSERY') + newsize = env.read_from_env('PYPY_GC_NURSERY') # PYPY_GC_NURSERY=1 forces a minor collect for every malloc. # Useful to debug external factors, like trackgcroot or the # handling of the write barrier. self.debug_always_do_minor_collect = newsize == 1 if newsize <= 0: - newsize = generation.estimate_best_nursery_size() + newsize = env.estimate_best_nursery_size() if newsize <= 0: newsize = defaultsize newsize = max(newsize, minsize) # - major_coll = base.read_float_from_env('PYPY_GC_MAJOR_COLLECT') - if major_coll >= 1.0: + major_coll = env.read_float_from_env('PYPY_GC_MAJOR_COLLECT') + if major_coll > 1.0: self.major_collection_threshold = major_coll # - min_heap_size = base.read_uint_from_env('PYPY_GC_MIN') + growth = env.read_float_from_env('PYPY_GC_GROWTH') + if growth > 1.0: + self.growth_rate_max = growth + # + min_heap_size = env.read_uint_from_env('PYPY_GC_MIN') if min_heap_size > 0: self.min_heap_size = float(min_heap_size) else: # defaults to 8 times the nursery self.min_heap_size = newsize * 8 # - max_heap_size = base.read_uint_from_env('PYPY_GC_MAX') + max_heap_size = env.read_uint_from_env('PYPY_GC_MAX') if max_heap_size > 0: self.max_heap_size = float(max_heap_size) # + max_delta = env.read_uint_from_env('PYPY_GC_MAX_DELTA') + if max_delta > 0: + self.max_delta = float(max_delta) + else: + self.max_delta = 0.125 * env.get_total_memory() + # self.minor_collection() # to empty the nursery llarena.arena_free(self.nursery) self.nursery_size = newsize @@ -295,11 +350,19 @@ # initialize the threshold self.min_heap_size = max(self.min_heap_size, self.nursery_size * self.major_collection_threshold) + self.next_major_collection_threshold = self.min_heap_size self.set_major_threshold_from(0.0) debug_stop("gc-set-nursery-size") - def set_major_threshold_from(self, threshold): + + def set_major_threshold_from(self, threshold, reserving_size=0): # Set the next_major_collection_threshold. + threshold_max = (self.next_major_collection_threshold * + self.growth_rate_max) + if threshold > threshold_max: + threshold = threshold_max + # + threshold += reserving_size if threshold < self.min_heap_size: threshold = self.min_heap_size # @@ -922,7 +985,7 @@ # # Now all live nursery objects should be out. Update the # young weakrefs' targets. - if self.young_objects_with_weakrefs.length() > 0: + if self.young_objects_with_weakrefs.non_empty(): self.invalidate_young_weakrefs() # # Clear this mapping. @@ -1016,7 +1079,7 @@ obj = oldlist.pop() # # Add the flag GCFLAG_NO_YOUNG_PTRS. All live objects should have - # this flag after a nursery collection. + # this flag set after a nursery collection. self.header(obj).tid |= GCFLAG_NO_YOUNG_PTRS # # Trace the 'obj' to replace pointers to nursery with pointers @@ -1079,7 +1142,7 @@ # nursery are kept unchanged in this step. llmemory.raw_memcopy(obj - size_gc_header, newhdr, totalsize) # - # Set the old object's tid to -1 (containing all flags) and + # Set the old object's tid to -42 (containing all flags) and # replace the old object's content with the target address. # A bit of no-ops to convince llarena that we are changing # the layout, in non-translated versions. @@ -1196,10 +1259,13 @@ # # Set the threshold for the next major collection to be when we # have allocated 'major_collection_threshold' times more than + # we currently have -- but no more than 'max_delta' more than # we currently have. + total_memory_used = float(self.get_total_memory_used()) bounded = self.set_major_threshold_from( - (self.get_total_memory_used() * self.major_collection_threshold) - + reserving_size) + min(total_memory_used * self.major_collection_threshold, + total_memory_used + self.max_delta), + reserving_size) # # Max heap size: gives an upper bound on the threshold. If we # already have at least this much allocated, raise MemoryError. @@ -1344,7 +1410,7 @@ if self.is_valid_gc_object(obj): if self.is_in_nursery(obj): # - # The object not a tagged pointer, and is it still in the + # The object is not a tagged pointer, and it is still in the # nursery. Find or allocate a "shadow" object, which is # where the object will be moved by the next minor # collection Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/semispace.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gc/semispace.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gc/semispace.py Wed Nov 24 14:21:00 2010 @@ -42,6 +42,7 @@ inline_simple_malloc_varsize = True malloc_zero_filled = True first_unused_gcflag = first_gcflag << 5 + gcflag_extra = GCFLAG_FINALIZATION_ORDERING HDR = lltype.Struct('header', ('tid', lltype.Signed)) # XXX or rffi.INT? typeid_is_in_field = 'tid' Modified: pypy/branch/fast-forward/pypy/rpython/memory/gc/test/test_minimark.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gc/test/test_minimark.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gc/test/test_minimark.py Wed Nov 24 14:21:00 2010 @@ -28,3 +28,42 @@ assert gc.card_marking_bytes_for_length(P+P+1) == 3 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P) == 8 assert gc.card_marking_bytes_for_length(P+P+P+P+P+P+P+P+1) == 9 + +def test_set_major_threshold(): + gc = MiniMarkGC(None, major_collection_threshold=2.0, + growth_rate_max=1.5) + gc.min_heap_size = 100.0 + gc.max_heap_size = 300.0 + gc.next_major_collection_threshold = 0.0 + # first, we don't grow past min_heap_size + for i in range(5): + gc.set_major_threshold_from(100.0) + assert gc.next_major_collection_threshold == 100.0 + # then we grow a lot + b = gc.set_major_threshold_from(100 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 150.0 + b = gc.set_major_threshold_from(150 * 2.0) + assert b is False + assert gc.next_major_collection_threshold == 225.0 + b = gc.set_major_threshold_from(225 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 # max reached + b = gc.set_major_threshold_from(300 * 2.0) + assert b is True + assert gc.next_major_collection_threshold == 300.0 + # then we shrink instantly + b = gc.set_major_threshold_from(100.0) + assert b is False + assert gc.next_major_collection_threshold == 100.0 + # then we grow a bit + b = gc.set_major_threshold_from(100 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 125.0 + b = gc.set_major_threshold_from(125 * 1.25) + assert b is False + assert gc.next_major_collection_threshold == 156.25 + # check that we cannot shrink below min_heap_size + b = gc.set_major_threshold_from(42.7) + assert b is False + assert gc.next_major_collection_threshold == 100.0 Modified: pypy/branch/fast-forward/pypy/rpython/memory/gctransform/framework.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/gctransform/framework.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/gctransform/framework.py Wed Nov 24 14:21:00 2010 @@ -172,6 +172,7 @@ gcdata.static_root_nongcend = a_random_address # patched in finish() gcdata.static_root_end = a_random_address # patched in finish() gcdata.max_type_id = 13 # patched in finish() + gcdata.typeids_z = a_random_address # patched in finish() self.gcdata = gcdata self.malloc_fnptr_cache = {} @@ -212,6 +213,9 @@ data_classdef.generalize_attr( 'max_type_id', annmodel.SomeInteger()) + data_classdef.generalize_attr( + 'typeids_z', + annmodel.SomeAddress()) annhelper = annlowlevel.MixLevelHelperAnnotator(self.translator.rtyper) @@ -415,6 +419,11 @@ [s_gc, annmodel.SomeInteger()], annmodel.s_Bool, minimal_transform=False) + self.get_typeids_z_ptr = getfn(inspector.get_typeids_z, + [s_gc], + annmodel.SomePtr( + lltype.Ptr(rgc.ARRAY_OF_CHAR)), + minimal_transform=False) self.set_max_heap_size_ptr = getfn(GCClass.set_max_heap_size.im_func, [s_gc, @@ -572,7 +581,14 @@ newgcdependencies = [] newgcdependencies.append(ll_static_roots_inside) ll_instance.inst_max_type_id = len(group.members) - self.write_typeid_list() + typeids_z = self.write_typeid_list() + ll_typeids_z = lltype.malloc(rgc.ARRAY_OF_CHAR, + len(typeids_z), + immortal=True) + for i in range(len(typeids_z)): + ll_typeids_z[i] = typeids_z[i] + ll_instance.inst_typeids_z = llmemory.cast_ptr_to_adr(ll_typeids_z) + newgcdependencies.append(ll_typeids_z) return newgcdependencies def get_finish_tables(self): @@ -599,6 +615,11 @@ for index in range(len(self.layoutbuilder.type_info_group.members)): f.write("member%-4d %s\n" % (index, all_ids.get(index, '?'))) f.close() + try: + import zlib + return zlib.compress(udir.join("typeids.txt").read(), 9) + except ImportError: + return '' def transform_graph(self, graph): func = getattr(graph, 'func', None) @@ -988,6 +1009,13 @@ resultvar=hop.spaceop.result) self.pop_roots(hop, livevars) + def gct_gc_typeids_z(self, hop): + livevars = self.push_roots(hop) + hop.genop("direct_call", + [self.get_typeids_z_ptr, self.c_const_gc], + resultvar=hop.spaceop.result) + self.pop_roots(hop, livevars) + def gct_malloc_nonmovable_varsize(self, hop): TYPE = hop.spaceop.result.concretetype if self.gcdata.gc.can_malloc_nonmovable(): Modified: pypy/branch/fast-forward/pypy/rpython/memory/support.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/support.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/support.py Wed Nov 24 14:21:00 2010 @@ -112,7 +112,7 @@ cur = next free_non_gc_object(self) - def length(self): + def _length_estimate(self): chunk = self.chunk count = self.used_in_last_chunk while chunk: @@ -135,7 +135,7 @@ foreach._annspecialcase_ = 'specialize:arg(1)' def stack2dict(self): - result = AddressDict(self.length()) + result = AddressDict(self._length_estimate()) self.foreach(_add_in_dict, result) return result Modified: pypy/branch/fast-forward/pypy/rpython/memory/test/test_gc.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/memory/test/test_gc.py (original) +++ pypy/branch/fast-forward/pypy/rpython/memory/test/test_gc.py Wed Nov 24 14:21:00 2010 @@ -278,6 +278,105 @@ res = self.interpret(f, []) assert res + def test_bug_1(self): + import weakref + class B(object): + pass + def g(): + b = B() + llop.gc__collect(lltype.Void) # force 'b' to be old + ref = weakref.ref(B()) + b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = (ref() is None) + return result + res = self.interpret(f, []) + assert res + + def test_cycle_with_weakref_and_del(self): + import weakref, gc + class A(object): + count = 0 + a = A() + class B(object): + def __del__(self): + # when __del__ is called, the weakref to c should be dead + if self.ref() is None: + a.count += 10 # ok + else: + a.count = 666 # not ok + class C(object): + pass + def g(): + c = C() + c.b = B() + ref = weakref.ref(c) + c.b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = a.count + (ref() is None) + return result + res = self.interpret(f, []) + assert res == 11 + + def test_weakref_to_object_with_finalizer_ordering(self): + import weakref, gc + class A(object): + count = 0 + a = A() + class B(object): + def __del__(self): + # when __del__ is called, the weakref to myself is still valid + # in RPython (at least with most GCs; this test might be + # skipped for specific GCs) + if self.ref() is self: + a.count += 10 # ok + else: + a.count = 666 # not ok + def g(): + b = B() + ref = weakref.ref(b) + b.ref = ref + return ref + def f(): + ref = g() + llop.gc__collect(lltype.Void) + llop.gc__collect(lltype.Void) + result = a.count + (ref() is None) + return result + res = self.interpret(f, []) + assert res == 11 + + def test_weakref_bug_1(self): + import weakref + class A(object): + pass + class B(object): + def __del__(self): + self.wref().x += 1 + def g(a): + b = B() + b.wref = weakref.ref(a) + # the only way to reach this weakref is via B, which is an + # object with finalizer (but the weakref itself points to + # a, which does not go away but will move during the next + # gc.collect) + def f(): + a = A() + a.x = 10 + g(a) + llop.gc__collect(lltype.Void) + return a.x + res = self.interpret(f, []) + assert res == 11 + def test_id(self): class A(object): pass @@ -635,6 +734,9 @@ class TestMarkSweepGC(GCTest): from pypy.rpython.memory.gc.marksweep import MarkSweepGC as GCClass + def test_weakref_to_object_with_finalizer_ordering(self): + py.test.skip("Does not work") + class TestSemiSpaceGC(GCTest, snippet.SemiSpaceGCTests): from pypy.rpython.memory.gc.semispace import SemiSpaceGC as GCClass GC_CAN_MOVE = True Modified: pypy/branch/fast-forward/pypy/rpython/rbuiltin.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/rbuiltin.py (original) +++ pypy/branch/fast-forward/pypy/rpython/rbuiltin.py Wed Nov 24 14:21:00 2010 @@ -386,6 +386,14 @@ hop.exception_cannot_occur() hop.genop('free', vlist) +def rtype_render_immortal(hop, i_track_allocation=None): + vlist = [hop.inputarg(hop.args_r[0], arg=0)] + v_track_allocation = parse_kwds(hop, + (i_track_allocation, None)) + hop.exception_cannot_occur() + if i_track_allocation is None or v_track_allocation.value: + hop.genop('track_alloc_stop', vlist) + def rtype_const_result(hop): hop.exception_cannot_occur() return hop.inputconst(hop.r_result.lowleveltype, hop.s_result.const) @@ -523,6 +531,7 @@ BUILTIN_TYPER[lltype.malloc] = rtype_malloc BUILTIN_TYPER[lltype.free] = rtype_free +BUILTIN_TYPER[lltype.render_immortal] = rtype_render_immortal BUILTIN_TYPER[lltype.cast_primitive] = rtype_cast_primitive BUILTIN_TYPER[lltype.cast_pointer] = rtype_cast_pointer BUILTIN_TYPER[lltype.cast_opaque_ptr] = rtype_cast_opaque_ptr Modified: pypy/branch/fast-forward/pypy/rpython/test/test_rclass.py ============================================================================== --- pypy/branch/fast-forward/pypy/rpython/test/test_rclass.py (original) +++ pypy/branch/fast-forward/pypy/rpython/test/test_rclass.py Wed Nov 24 14:21:00 2010 @@ -917,6 +917,7 @@ assert destrptr is not None def test_del_inheritance(self): + from pypy.rlib import rgc class State: pass s = State() @@ -937,6 +938,7 @@ A() B() C() + rgc.collect() return s.a_dels * 10 + s.b_dels res = f() assert res == 42 @@ -1067,6 +1069,7 @@ assert meth.finalizer def test_del_inheritance(self): + from pypy.rlib import rgc class State: pass s = State() @@ -1087,6 +1090,7 @@ A() B() C() + rgc.collect() return s.a_dels * 10 + s.b_dels res = f() assert res == 42 Modified: pypy/branch/fast-forward/pypy/tool/logparser.py ============================================================================== --- pypy/branch/fast-forward/pypy/tool/logparser.py (original) +++ pypy/branch/fast-forward/pypy/tool/logparser.py Wed Nov 24 14:21:00 2010 @@ -12,14 +12,6 @@ from pypy.tool import progressbar def parse_log_file(filename, verbose=True): - r_start = re.compile(r"\[([0-9a-fA-F]+)\] \{([\w-]+)$") - r_stop = re.compile(r"\[([0-9a-fA-F]+)\] ([\w-]+)\}$") - lasttime = 0 - log = DebugLog() - time_decrase = False - performance_log = True - nested = 0 - # f = open(filename, 'r') if f.read(2) == 'BZ': f.close() @@ -30,19 +22,33 @@ lines = f.readlines() f.close() # - if sys.stdout.isatty(): + return parse_log(lines, verbose=verbose) + +def parse_log(lines, verbose=False): + color = "(?:\x1b.*?m)" + r_start = re.compile(color + r"\[([0-9a-fA-F]+)\] \{([\w-]+)" + color + "$") + r_stop = re.compile(color + r"\[([0-9a-fA-F]+)\] ([\w-]+)\}" + color + "$") + lasttime = 0 + log = DebugLog() + time_decrase = False + performance_log = True + nested = 0 + # + if verbose and sys.stdout.isatty(): progress = progressbar.ProgressBar(color='green') + counter = 0 + else: + progress = None single_percent = len(lines) / 100 if verbose: - vnext = single_percent + vnext = 0 else: - vnext = len(lines) - counter = 0 + vnext = -1 for i, line in enumerate(lines): if i == vnext: - counter += 1 - if sys.stdout.isatty(): + if progress is not None: progress.render(counter) + counter += 1 vnext += single_percent else: sys.stderr.write('%d%%..' % int(100.0*i/len(lines))) Modified: pypy/branch/fast-forward/pypy/tool/release/force-builds.py ============================================================================== --- pypy/branch/fast-forward/pypy/tool/release/force-builds.py (original) +++ pypy/branch/fast-forward/pypy/tool/release/force-builds.py Wed Nov 24 14:21:00 2010 @@ -20,11 +20,12 @@ 'own-linux-x86-32', 'own-linux-x86-64', # 'own-macosx-x86-32', - 'pypy-c-app-level-linux-x86-32', - 'pypy-c-app-level-linux-x86-64', +# 'pypy-c-app-level-linux-x86-32', +# 'pypy-c-app-level-linux-x86-64', 'pypy-c-stackless-app-level-linux-x86-32', 'pypy-c-app-level-win-x86-32', 'pypy-c-jit-linux-x86-32', + 'pypy-c-jit-linux-x86-64', # 'pypy-c-jit-macosx-x86-32', 'pypy-c-jit-win-x86-32', ] Modified: pypy/branch/fast-forward/pypy/tool/release/make_release.py ============================================================================== --- pypy/branch/fast-forward/pypy/tool/release/make_release.py (original) +++ pypy/branch/fast-forward/pypy/tool/release/make_release.py Wed Nov 24 14:21:00 2010 @@ -4,7 +4,7 @@ into release packages. Note: you must run apropriate buildbots first and make sure there are no failures. Use force-builds.py from the same directory. -Usage: make_release.py release/ +Usage: make_release.py release/ release_version """ import autopath @@ -30,7 +30,8 @@ else: xml = override_xml dom = minidom.parseString(xml) - refs = [node.getAttribute('href') for node in dom.getElementsByTagName('a')] + refs = [node.getAttribute('href') for node in dom.getElementsByTagName('a') + if 'pypy' in node.getAttribute('href')] # all refs are of form: pypy-{type}-{revision}-{platform}.tar.bz2 r = re.compile('pypy-c-([\w\d]+)-(\d+)-([\w\d]+).tar.bz2$') d = {} @@ -76,7 +77,7 @@ t.add('pypy-%s' % release) alltars.append(name) t.close() - shutil.rmtree(str(tmpdir.join('pypy-1.3'))) + shutil.rmtree(str(tmpdir.join('pypy-' + release))) for name in alltars: print "Uploading %s" % name os.system('scp %s codespeak.net:/www/pypy.org/htdocs/download' % name) @@ -84,8 +85,8 @@ os.chdir(olddir) if __name__ == '__main__': - if len(sys.argv) != 2: + if len(sys.argv) != 3: print __doc__ sys.exit(1) - main(sys.argv[1], release='1.3') + main(sys.argv[1], release=sys.argv[2]) Modified: pypy/branch/fast-forward/pypy/tool/release/package.py ============================================================================== --- pypy/branch/fast-forward/pypy/tool/release/package.py (original) +++ pypy/branch/fast-forward/pypy/tool/release/package.py Wed Nov 24 14:21:00 2010 @@ -1,8 +1,12 @@ #!/usr/bin/env python """ A sample script that packages PyPy, provided that it's already built. -Usage: +It uses 'pypy/translator/goal/pypy-c' and parts of the rest of the working +copy. Usage: -package.py pypydir [name-of-archive] [name-of-pypy-c] + package.py root-pypy-dir [name-of-archive] [name-of-pypy-c] + +Usually you would do: package.py ../../.. pypy-VER-PLATFORM. +The output is found in the directory /tmp/usession-YOURNAME/build/. """ import autopath @@ -32,7 +36,7 @@ class PyPyCNotFound(Exception): pass -def package(basedir, name='pypy-nightly', rename_pypy_c='pypy-c', +def package(basedir, name='pypy-nightly', rename_pypy_c='pypy', copy_to_dir = None, override_pypy_c = None): basedir = py.path.local(basedir) if sys.platform == 'win32': @@ -64,6 +68,10 @@ headers = includedir.listdir('*.h') + includedir.listdir('*.inl') for n in headers: shutil.copy(str(n), str(pypydir.join('include'))) + # + spdir = pypydir.ensure('site-packages', dir=True) + shutil.copy(str(basedir.join('site-packages', 'README')), str(spdir)) + # pypydir.ensure('bin', dir=True) archive_pypy_c = pypydir.join('bin', rename_pypy_c) shutil.copy(str(pypy_c), str(archive_pypy_c)) Modified: pypy/branch/fast-forward/pypy/tool/release/test/test_package.py ============================================================================== --- pypy/branch/fast-forward/pypy/tool/release/test/test_package.py (original) +++ pypy/branch/fast-forward/pypy/tool/release/test/test_package.py Wed Nov 24 14:21:00 2010 @@ -18,7 +18,7 @@ prefix = builddir.join(test) cpyver = '%d.%d.%d' % CPYTHON_VERSION[:3] assert prefix.join('lib-python', cpyver, 'test').check() - assert prefix.join('bin', 'pypy-c').check() + assert prefix.join('bin', 'pypy').check() assert prefix.join('lib_pypy', 'syslog.py').check() assert not prefix.join('lib_pypy', 'py').check() assert not prefix.join('lib_pypy', 'ctypes_configure').check() Modified: pypy/branch/fast-forward/pypy/tool/terminal.py ============================================================================== --- pypy/branch/fast-forward/pypy/tool/terminal.py (original) +++ pypy/branch/fast-forward/pypy/tool/terminal.py Wed Nov 24 14:21:00 2010 @@ -62,9 +62,10 @@ for control in CONTROLS: # Set the control escape sequence setattr(MODULE, control, curses.tigetstr(CONTROLS[control]) or '') - for value in VALUES: - # Set terminal related values - setattr(MODULE, value, curses.tigetnum(VALUES[value])) + if hasattr(curses, 'tigetnum'): + for value in VALUES: + # Set terminal related values + setattr(MODULE, value, curses.tigetnum(VALUES[value])) def render(text): """Helper function to apply controls easily @@ -74,7 +75,16 @@ return text % MODULE.__dict__ try: - import curses + if '__pypy__' in sys.builtin_module_names: + # this is not really the whole curses, but our _minimal_curses it's + # better than nothing + import _minimal_curses as curses + # a bit of a hack: we have tigetstr but not tigetnum, so we call + # default() to have default values, then setup() will overwrite the + # ones it can + default() + else: + import curses setup() except Exception, e: # There is a failure; set all attributes to default Modified: pypy/branch/fast-forward/pypy/translator/c/funcgen.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/funcgen.py (original) +++ pypy/branch/fast-forward/pypy/translator/c/funcgen.py Wed Nov 24 14:21:00 2010 @@ -1,3 +1,4 @@ +import sys from pypy.translator.c.support import USESLOTS # set to False if necessary while refactoring from pypy.translator.c.support import cdecl from pypy.translator.c.support import llvalue_from_constant, gen_assignments @@ -757,6 +758,16 @@ format.append('%s') argv.append('(%s) ? "True" : "False"' % self.expr(arg)) continue + elif T == SignedLongLong: + if sys.platform == 'win32': + format.append('%I64d') + else: + format.append('%lld') + elif T == UnsignedLongLong: + if sys.platform == 'win32': + format.append('%I64u') + else: + format.append('%llu') else: raise Exception("don't know how to debug_print %r" % (T,)) argv.append(self.expr(arg)) @@ -765,17 +776,20 @@ "if (PYPY_HAVE_DEBUG_PRINTS) { fprintf(PYPY_DEBUG_FILE, %s); %s}" % (', '.join(argv), free_line)) + def _op_debug(self, opname, arg): + if isinstance(arg, Constant): + string_literal = c_string_constant(''.join(arg.value.chars)) + return "%s(%s);" % (opname, string_literal) + else: + x = "%s(RPyString_AsCharP(%s));\n" % (opname, self.expr(arg)) + x += "RPyString_FreeCache();" + return x + def OP_DEBUG_START(self, op): - arg = op.args[0] - assert isinstance(arg, Constant) - return "PYPY_DEBUG_START(%s);" % ( - c_string_constant(''.join(arg.value.chars)),) + return self._op_debug('PYPY_DEBUG_START', op.args[0]) def OP_DEBUG_STOP(self, op): - arg = op.args[0] - assert isinstance(arg, Constant) - return "PYPY_DEBUG_STOP(%s);" % ( - c_string_constant(''.join(arg.value.chars)),) + return self._op_debug('PYPY_DEBUG_STOP', op.args[0]) def OP_DEBUG_ASSERT(self, op): return 'RPyAssert(%s, %s);' % (self.expr(op.args[0]), Modified: pypy/branch/fast-forward/pypy/translator/c/gcc/test/test_trackgcroot.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/gcc/test/test_trackgcroot.py (original) +++ pypy/branch/fast-forward/pypy/translator/c/gcc/test/test_trackgcroot.py Wed Nov 24 14:21:00 2010 @@ -98,14 +98,13 @@ """ lines = source.splitlines(True) parts = list(DarwinAssemblerParser().find_functions(iter(lines))) - assert len(parts) == 7 + assert len(parts) == 6 assert parts[0] == (False, lines[:3]) assert parts[1] == (True, lines[3:7]) assert parts[2] == (True, lines[7:11]) - assert parts[3] == (True, lines[11:13]) - assert parts[4] == (False, lines[13:18]) - assert parts[5] == (True, lines[18:20]) - assert parts[6] == (False, lines[20:]) + assert parts[3] == (True, lines[11:18]) + assert parts[4] == (True, lines[18:20]) + assert parts[5] == (False, lines[20:]) def test_computegcmaptable(): tests = [] Modified: pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py (original) +++ pypy/branch/fast-forward/pypy/translator/c/gcc/trackgcroot.py Wed Nov 24 14:21:00 2010 @@ -1405,6 +1405,7 @@ 'const_data' ] r_sectionstart = re.compile(r"\t\.("+'|'.join(OTHERSECTIONS)+").*$") + sections_doesnt_end_function = {'cstring': True, 'const': True} def find_functions(self, iterlines): functionlines = [] @@ -1412,20 +1413,20 @@ in_function = False for n, line in enumerate(iterlines): if self.r_textstart.match(line): - assert not in_text, "unexpected repeated .text start: %d" % n in_text = True elif self.r_sectionstart.match(line): - if in_function: + sectionname = self.r_sectionstart.match(line).group(1) + if (in_function and + sectionname not in self.sections_doesnt_end_function): yield in_function, functionlines functionlines = [] + in_function = False in_text = False - in_function = False elif in_text and self.FunctionGcRootTracker.r_functionstart.match(line): yield in_function, functionlines functionlines = [] in_function = True functionlines.append(line) - if functionlines: yield in_function, functionlines @@ -1442,23 +1443,6 @@ format = "mingw32" FunctionGcRootTracker = Mingw32FunctionGcRootTracker - def find_functions(self, iterlines): - functionlines = [] - in_text = False - in_function = False - for n, line in enumerate(iterlines): - if self.r_textstart.match(line): - in_text = True - elif self.r_sectionstart.match(line): - in_text = False - elif in_text and self.FunctionGcRootTracker.r_functionstart.match(line): - yield in_function, functionlines - functionlines = [] - in_function = True - functionlines.append(line) - if functionlines: - yield in_function, functionlines - class MsvcAssemblerParser(AssemblerParser): format = "msvc" FunctionGcRootTracker = MsvcFunctionGcRootTracker Modified: pypy/branch/fast-forward/pypy/translator/c/node.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/node.py (original) +++ pypy/branch/fast-forward/pypy/translator/c/node.py Wed Nov 24 14:21:00 2010 @@ -714,7 +714,11 @@ s = ''.join([self.obj.getitem(i) for i in range(len(self.obj.items))]) else: s = ''.join(self.obj.items) - yield '\t%s%s' % (length, c_char_array_constant(s)) + array_constant = c_char_array_constant(s) + if array_constant.startswith('{') and barebonearray(T): + assert array_constant.endswith('}') + array_constant = array_constant[1:-1].strip() + yield '\t%s%s' % (length, array_constant) yield '}' else: barebone = barebonearray(T) Modified: pypy/branch/fast-forward/pypy/translator/c/src/asm_gcc_x86.h ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/src/asm_gcc_x86.h (original) +++ pypy/branch/fast-forward/pypy/translator/c/src/asm_gcc_x86.h Wed Nov 24 14:21:00 2010 @@ -2,6 +2,8 @@ * It replaces some complex macros with native assembler instructions. */ +#if 0 /* --- disabled: does not give any speed-up --- */ + #undef OP_INT_ADD_OVF #define OP_INT_ADD_OVF(x,y,r) \ asm volatile( \ @@ -50,6 +52,13 @@ : "0"(x), "g"(y) /* inputs */ \ : "cc", "memory") /* clobber */ +extern void op_int_overflowed(void) + asm ("_op_int_overflowed") + __attribute__((used)); + +#endif /* 0 */ + + /* Pentium only! */ #define READ_TIMESTAMP(val) \ asm volatile("rdtsc" : "=A" (val)) @@ -62,19 +71,15 @@ // I don't know how important it is, comment talks about time warps -/* prototypes */ - -extern void op_int_overflowed(void) - asm ("_op_int_overflowed") - __attribute__((used)); - /* implementations */ #ifndef PYPY_NOT_MAIN_FILE +# if 0 /* disabled */ void op_int_overflowed(void) { FAIL_OVF("integer operation"); } +# endif #endif Modified: pypy/branch/fast-forward/pypy/translator/c/src/debug_alloc.h ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/src/debug_alloc.h (original) +++ pypy/branch/fast-forward/pypy/translator/c/src/debug_alloc.h Wed Nov 24 14:21:00 2010 @@ -1,5 +1,5 @@ /**************************************************************/ - /*** tracking raw mallocs and frees for debugging ***/ +/*** tracking raw mallocs and frees for debugging ***/ #ifndef RPY_ASSERT Modified: pypy/branch/fast-forward/pypy/translator/c/src/debug_print.h ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/src/debug_print.h (original) +++ pypy/branch/fast-forward/pypy/translator/c/src/debug_print.h Wed Nov 24 14:21:00 2010 @@ -10,6 +10,7 @@ but not any nested debug_print :fname full logging prefix:fname conditional logging + prefix1,prefix2:fname conditional logging with multiple selections Conditional logging means that it only includes the debug_start/debug_stop sections whose name match 'prefix'. Other sections are ignored, including @@ -70,6 +71,12 @@ static void pypy_debug_open(void) { char *filename = getenv("PYPYLOG"); + if (filename) +#ifndef MS_WINDOWS + unsetenv("PYPYLOG"); /* don't pass it to subprocesses */ +#else + putenv("PYPYLOG="); /* don't pass it to subprocesses */ +#endif if (filename && filename[0]) { char *colon = strchr(filename, ':'); @@ -139,12 +146,22 @@ #endif -static bool_t startswith(const char *str, const char *substr) +static bool_t startswithoneof(const char *str, const char *substr) { - while (*substr) - if (*str++ != *substr++) - return 0; - return 1; + const char *p = str; + for (; *substr; substr++) + { + if (*substr != ',') + { + if (p && *p++ != *substr) + p = NULL; /* mismatch */ + } + else if (p != NULL) + return 1; /* match */ + else + p = str; /* mismatched, retry with the next */ + } + return p != NULL; } #if defined(_MSC_VER) || defined(__MINGW32__) @@ -175,7 +192,7 @@ if (!debug_profile) { /* non-profiling version */ - if (!debug_prefix || !startswith(category, debug_prefix)) + if (!debug_prefix || !startswithoneof(category, debug_prefix)) { /* wrong section name, or no PYPYLOG at all, skip it */ return; Modified: pypy/branch/fast-forward/pypy/translator/c/src/g_include.h ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/src/g_include.h (original) +++ pypy/branch/fast-forward/pypy/translator/c/src/g_include.h Wed Nov 24 14:21:00 2010 @@ -39,10 +39,13 @@ #include "src/instrument.h" /* optional assembler bits */ -// disabled: does not give any speed-up -//#if defined(__GNUC__) && defined(__i386__) -//# include "src/asm_gcc_x86.h" -//#endif +#if defined(__GNUC__) && defined(__i386__) +# include "src/asm_gcc_x86.h" +#endif + +#if defined(__GNUC__) && defined(__amd64__) +# include "src/asm_gcc_x86_64.h" +#endif #if defined(__GNUC__) && defined(__ppc__) # include "src/asm_ppc.h" Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_lltyped.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/test/test_lltyped.py (original) +++ pypy/branch/fast-forward/pypy/translator/c/test/test_lltyped.py Wed Nov 24 14:21:00 2010 @@ -1,6 +1,7 @@ import py from pypy.rpython.lltypesystem.lltype import * from pypy.translator.c.test import test_typed +from pypy.tool.sourcetools import func_with_new_name class TestLowLevelType(test_typed.CompilationTestCase): @@ -655,6 +656,45 @@ fn = self.getcompiled(llf) fn() + def test_prebuilt_raw_arrays(self): + from pypy.rpython.lltypesystem import rffi, ll2ctypes + # + def make_test_function(cast, haslength, length): + a = malloc(A, length, flavor='raw', immortal=True) + # two cases: a zero-terminated array if length == 6 or 1030, + # a non-zero-terminated array if length == 557 or 1031 + for i in range(length): + a[i] = cast(256 - 5 + i) + def llf(): + for i in range(length): + if a[i] != cast(256 - 5 + i): + return False + if haslength and len(a) != length: + return False + return True + return func_with_new_name(llf, repr((A, haslength, length))) + # + testfns = [] + records = [] + for OF, cast in [(Void, lambda n: None), + (Char, lambda n: chr(n & 0xFF)), + (Signed, lambda n: n)]: + for A, haslength in [(rffi.CArray(OF), False), + (Array(OF), True)]: + for length in [0, 6, 557, 1030, 1031]: + testfns.append(make_test_function(cast, haslength, length)) + records.append((A, haslength, length)) + def llf(): + i = 0 + for fn in testfns: + if not fn(): + return i # returns the index of the failing function + i += 1 + return -42 + fn = self.getcompiled(llf) + res = fn() + assert res == -42, "failing function: %r" % (records[res],) + def test_prebuilt_ll2ctypes_array(self): from pypy.rpython.lltypesystem import rffi, ll2ctypes A = rffi.CArray(Char) @@ -841,3 +881,17 @@ assert res == -98765432 res = fn(1) assert res == -9999999 + + def test_render_immortal(self): + A = FixedSizeArray(Signed, 1) + a1 = malloc(A, flavor='raw') + render_immortal(a1) + a1[0] = 42 + def llf(): + a2 = malloc(A, flavor='raw') + render_immortal(a2) + a2[0] = 3 + return a1[0] + a2[0] + fn = self.getcompiled(llf) + assert fn() == 45 + Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py (original) +++ pypy/branch/fast-forward/pypy/translator/c/test/test_newgc.py Wed Nov 24 14:21:00 2010 @@ -1064,14 +1064,16 @@ def test_get_rpy_type_index(self): self.run("get_rpy_type_index") - filename_dump = str(udir.join('test_dump_rpy_heap')) + filename1_dump = str(udir.join('test_dump_rpy_heap.1')) + filename2_dump = str(udir.join('test_dump_rpy_heap.2')) def define_dump_rpy_heap(self): U = lltype.GcForwardReference() U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)), ('x', lltype.Signed))) S = lltype.GcStruct('S', ('u', lltype.Ptr(U))) A = lltype.GcArray(lltype.Ptr(S)) - filename = self.filename_dump + filename1 = self.filename1_dump + filename2 = self.filename2_dump def fn(): s = lltype.malloc(S) @@ -1081,20 +1083,67 @@ a = lltype.malloc(A, 1000) s2 = lltype.malloc(S) # - fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666) - rgc.dump_rpy_heap(fd) + fd1 = os.open(filename1, os.O_WRONLY | os.O_CREAT, 0666) + fd2 = os.open(filename2, os.O_WRONLY | os.O_CREAT, 0666) + rgc.dump_rpy_heap(fd1) + rgc.dump_rpy_heap(fd2) # try twice in a row keepalive_until_here(s2) keepalive_until_here(s) keepalive_until_here(a) - os.close(fd) + os.close(fd1) + os.close(fd2) return 0 return fn def test_dump_rpy_heap(self): self.run("dump_rpy_heap") - assert os.path.exists(self.filename_dump) - assert os.path.getsize(self.filename_dump) > 64 + for fn in [self.filename1_dump, self.filename2_dump]: + assert os.path.exists(fn) + assert os.path.getsize(fn) > 64 + f = open(self.filename1_dump) + data1 = f.read() + f.close() + f = open(self.filename2_dump) + data2 = f.read() + f.close() + assert data1 == data2 + + filename_dump_typeids_z = str(udir.join('test_typeids_z')) + def define_write_typeids_z(self): + U = lltype.GcForwardReference() + U.become(lltype.GcStruct('U', ('next', lltype.Ptr(U)), + ('x', lltype.Signed))) + S = lltype.GcStruct('S', ('u', lltype.Ptr(U))) + A = lltype.GcArray(lltype.Ptr(S)) + filename = self.filename_dump_typeids_z + + def fn(): + s = lltype.malloc(S) + s.u = lltype.malloc(U) + s.u.next = lltype.malloc(U) + s.u.next.next = lltype.malloc(U) + a = lltype.malloc(A, 1000) + s2 = lltype.malloc(S) + # + p = rgc.get_typeids_z() + s = ''.join([p[i] for i in range(len(p))]) + fd = os.open(filename, os.O_WRONLY | os.O_CREAT, 0666) + os.write(fd, s) + os.close(fd) + return 0 + + return fn + + def test_write_typeids_z(self): + self.run("write_typeids_z") + f = open(self.filename_dump_typeids_z) + data_z = f.read() + f.close() + import zlib + data = zlib.decompress(data_z) + assert data.startswith('member0') + assert 'GcArray of * GcStruct S {' in data class TestSemiSpaceGC(TestUsingFramework, snippet.SemiSpaceGCTestDefines): gcpolicy = "semispace" Modified: pypy/branch/fast-forward/pypy/translator/c/test/test_standalone.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/c/test/test_standalone.py (original) +++ pypy/branch/fast-forward/pypy/translator/c/test/test_standalone.py Wed Nov 24 14:21:00 2010 @@ -272,7 +272,7 @@ x = "got:" debug_start ("mycat") if have_debug_prints(): x += "b" - debug_print ("foo", 2, "bar", 3) + debug_print ("foo", r_longlong(2), "bar", 3) debug_start ("cat2") if have_debug_prints(): x += "c" debug_print ("baz") @@ -368,12 +368,27 @@ assert not err assert path.check(file=1) data = path.read() - assert 'toplevel' in path.read() - assert 'mycat' not in path.read() - assert 'foo 2 bar 3' not in path.read() + assert 'toplevel' in data + assert 'mycat' not in data + assert 'foo 2 bar 3' not in data assert 'cat2' in data assert 'baz' in data assert 'bok' not in data + # check with PYPYLOG=myc,cat2:somefilename (includes mycat and cat2) + path = udir.join('test_debug_xxx_myc_cat2.log') + out, err = cbuilder.cmdexec("", err=True, + env={'PYPYLOG': 'myc,cat2:%s' % path}) + assert out.strip() == 'got:bcda.' + assert not err + assert path.check(file=1) + data = path.read() + assert 'toplevel' in data + assert '{mycat' in data + assert 'mycat}' in data + assert 'foo 2 bar 3' in data + assert 'cat2' in data + assert 'baz' in data + assert 'bok' in data # # finally, check compiling with logging disabled from pypy.config.pypyoption import get_pypy_config @@ -388,6 +403,20 @@ assert not err assert path.check(file=0) + def test_debug_print_start_stop_nonconst(self): + def entry_point(argv): + debug_start(argv[1]) + debug_print(argv[2]) + debug_stop(argv[1]) + return 0 + t, cbuilder = self.compile(entry_point) + out, err = cbuilder.cmdexec("foo bar", err=True, env={'PYPYLOG': ':-'}) + lines = err.splitlines() + assert '{foo' in lines[0] + assert 'bar' == lines[1] + assert 'foo}' in lines[2] + + def test_fatal_error(self): def g(x): if x == 1: Modified: pypy/branch/fast-forward/pypy/translator/driver.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/driver.py (original) +++ pypy/branch/fast-forward/pypy/translator/driver.py Wed Nov 24 14:21:00 2010 @@ -11,7 +11,7 @@ from pypy.annotation import policy as annpolicy import optparse from pypy.tool.udir import udir -from pypy.rlib.jit import DEBUG_OFF, DEBUG_DETAILED, DEBUG_PROFILE, DEBUG_STEPS +from pypy.tool.debug_print import debug_start, debug_print, debug_stop from pypy.rlib.entrypoint import secondary_entrypoints import py @@ -37,13 +37,6 @@ 'c': 'lltype', } -JIT_DEBUG = { - 'off' : DEBUG_OFF, - 'profile' : DEBUG_PROFILE, - 'steps' : DEBUG_STEPS, - 'detailed' : DEBUG_DETAILED, -} - def backend_to_typesystem(backend): return _BACKEND_TO_TYPESYSTEM.get(backend, 'ootype') @@ -283,6 +276,8 @@ return else: self.log.info("%s..." % title) + debug_start('translation-task') + debug_print('starting', goal) self.timer.start_event(goal) try: instrument = False @@ -300,11 +295,13 @@ assert False, 'we should not get here' finally: try: + debug_stop('translation-task') self.timer.end_event(goal) except (KeyboardInterrupt, SystemExit): raise except: pass + #import gc; gc.dump_rpy_heap('rpyheap-after-%s.dump' % goal) return res def task_annotate(self): @@ -399,7 +396,6 @@ # from pypy.jit.metainterp.warmspot import apply_jit apply_jit(self.translator, policy=self.jitpolicy, - debug_level=JIT_DEBUG[self.config.translation.jit_debug], backend_name=self.config.translation.jit_backend, inline=True) # self.log.info("the JIT compiler was generated") @@ -417,7 +413,6 @@ # from pypy.jit.metainterp.warmspot import apply_jit apply_jit(self.translator, policy=self.jitpolicy, - debug_level=JIT_DEBUG[self.config.translation.jit_debug], backend_name='cli', inline=True) #XXX # self.log.info("the JIT compiler was generated") Modified: pypy/branch/fast-forward/pypy/translator/goal/app_main.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/goal/app_main.py (original) +++ pypy/branch/fast-forward/pypy/translator/goal/app_main.py Wed Nov 24 14:21:00 2010 @@ -438,6 +438,9 @@ try: if run_command: # handle the "-c" command + # Put '' on sys.path + sys.path.insert(0, '') + def run_it(): exec cmd in mainmodule.__dict__ success = run_toplevel(run_it) Modified: pypy/branch/fast-forward/pypy/translator/goal/targetpypystandalone.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/goal/targetpypystandalone.py (original) +++ pypy/branch/fast-forward/pypy/translator/goal/targetpypystandalone.py Wed Nov 24 14:21:00 2010 @@ -28,9 +28,14 @@ w_call_finish_gateway = space.wrap(gateway.interp2app(call_finish)) w_call_startup_gateway = space.wrap(gateway.interp2app(call_startup)) w_os = setup_nanos(space) + withjit = space.config.objspace.usemodules.pypyjit def entry_point(argv): space.timer.start("Entrypoint") + if withjit: + from pypy.jit.backend.hlinfo import highleveljitinfo + highleveljitinfo.sys_executable = argv[0] + #debug("entry point starting") #for arg in argv: # debug(" argv -> " + arg) @@ -150,6 +155,9 @@ if config.objspace.allworkingmodules: from pypy.config.pypyoption import enable_allworkingmodules enable_allworkingmodules(config) + if config.objspace.translationmodules: + from pypy.config.pypyoption import enable_translationmodules + enable_translationmodules(config) if config.translation.type_system == 'ootype': config.objspace.usemodules.suggest(rbench=True) Modified: pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/branch/fast-forward/pypy/translator/goal/test2/test_app_main.py Wed Nov 24 14:21:00 2010 @@ -537,6 +537,9 @@ assert repr(str(tmpdir.join('otherpath'))) in data assert "''" not in data + data = self.run('-c "import sys; print sys.path"') + assert data.startswith("[''") + class AppTestAppMain: @@ -572,7 +575,8 @@ newpath = app_main.get_library_path('/tmp/pypy-c') # stdlib not found assert newpath == sys.path newpath = app_main.get_library_path(self.fake_exe) - assert newpath == self.expected_path + # we get at least 'expected_path', and maybe more (e.g.plat-linux2) + assert newpath[:len(self.expected_path)] == self.expected_path finally: sys.path.pop() @@ -585,7 +589,9 @@ app_main.os = os pypy_c = os.path.join(self.trunkdir, 'pypy', 'translator', 'goal', 'pypy-c') newpath = app_main.get_library_path(pypy_c) - assert len(newpath) == 3 + # we get at least lib_pypy, lib-python/modified-X.Y.Z, + # lib-python/X.Y.Z, and maybe more (e.g. plat-linux2) + assert len(newpath) >= 3 for p in newpath: assert p.startswith(self.trunkdir) finally: Modified: pypy/branch/fast-forward/pypy/translator/platform/linux.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/platform/linux.py (original) +++ pypy/branch/fast-forward/pypy/translator/platform/linux.py Wed Nov 24 14:21:00 2010 @@ -27,6 +27,8 @@ class Linux(BaseLinux): + shared_only = () # it seems that on 32-bit linux, compiling with -fPIC + # gives assembler that asmgcc is not happy about. def library_dirs_for_libffi_a(self): # places where we need to look for libffi.a return self.library_dirs_for_libffi() + ['/usr/lib'] Modified: pypy/branch/fast-forward/pypy/translator/platform/posix.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/platform/posix.py (original) +++ pypy/branch/fast-forward/pypy/translator/platform/posix.py Wed Nov 24 14:21:00 2010 @@ -104,12 +104,10 @@ else: target_name = exe_name.basename - cflags = self.cflags - if sys.maxint > 2147483647: # XXX XXX XXX sort this out - if shared: - cflags = self.cflags + self.shared_only - else: - cflags = self.cflags + self.standalone_only + if shared: + cflags = self.cflags + self.shared_only + else: + cflags = self.cflags + self.standalone_only m = GnuMakefile(path) m.exe_name = exe_name Modified: pypy/branch/fast-forward/pypy/translator/tool/reftracker.py ============================================================================== --- pypy/branch/fast-forward/pypy/translator/tool/reftracker.py (original) +++ pypy/branch/fast-forward/pypy/translator/tool/reftracker.py Wed Nov 24 14:21:00 2010 @@ -3,7 +3,7 @@ Usage: call track(obj). """ -import autopath, sys, os +import autopath, sys, os, types import gc from pypy.translator.tool.graphpage import GraphPage, DotGen from pypy.tool.uid import uid @@ -39,7 +39,7 @@ if o2 is None: continue addedge(objectlist[i], o2) - id2typename[uid(o2)] = type(o2).__name__ + id2typename[uid(o2)] = self.shortrepr(o2) del o2 for o2 in self.get_referrers(objectlist[i]): if o2 is None: @@ -47,7 +47,7 @@ if type(o2) is list and o2 and o2[0] is MARKER: continue addedge(o2, objectlist[i]) - id2typename[uid(o2)] = type(o2).__name__ + id2typename[uid(o2)] = self.shortrepr(o2) del o2 for ids, label in edges.items(): @@ -82,13 +82,23 @@ return self.newpage(objectlist) def formatobject(self, o): + header = self.shortrepr(o, compact=False) + secondline = repr(o.__class__) + return header, secondline, repr(o) + + def shortrepr(self, o, compact=True): + t = type(o) + if t is types.FrameType: + if compact: + return 'frame %r' % (o.f_code.co_name,) + else: + return 'frame %r' % (o.f_code,) s = repr(o) if len(s) > 50: - linktext = s s = s[:20] + ' ... ' + s[-20:] - else: - linktext = '' - return type(o).__name__, s, linktext + if s.startswith('<') and s.endswith('>'): + s = s[1:-1] + return s def edgelabel(self, o1, o2): return '' From arigo at codespeak.net Wed Nov 24 14:27:11 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 14:27:11 +0100 (CET) Subject: [pypy-svn] r79459 - in pypy/branch/rlist-jit/pypy: interpreter module/__builtin__ module/cpyext objspace/std Message-ID: <20101124132711.40BAD5080B@codespeak.net> Author: arigo Date: Wed Nov 24 14:27:09 2010 New Revision: 79459 Modified: pypy/branch/rlist-jit/pypy/interpreter/baseobjspace.py pypy/branch/rlist-jit/pypy/interpreter/function.py pypy/branch/rlist-jit/pypy/module/__builtin__/interp_classobj.py pypy/branch/rlist-jit/pypy/module/cpyext/typeobject.py pypy/branch/rlist-jit/pypy/objspace/std/objspace.py pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py pypy/branch/rlist-jit/pypy/objspace/std/typetype.py Log: Revert (again) the introduction of space.fixedunpack(). Will try yet another approach. Modified: pypy/branch/rlist-jit/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/rlist-jit/pypy/interpreter/baseobjspace.py Wed Nov 24 14:27:09 2010 @@ -772,13 +772,6 @@ fixedview_unroll = fixedview - def fixedunpack(self, w_iterable, expected_length=-1): - """ A fixed list with the content of w_iterable. - For most cases, use fixedview() instead. - """ - return make_sure_not_resized(self.unpackiterable(w_iterable, - expected_length)[:]) - def listview(self, w_iterable, expected_length=-1): """ A non-fixed view of w_iterable. Don't modify the result """ Modified: pypy/branch/rlist-jit/pypy/interpreter/function.py ============================================================================== --- pypy/branch/rlist-jit/pypy/interpreter/function.py (original) +++ pypy/branch/rlist-jit/pypy/interpreter/function.py Wed Nov 24 14:27:09 2010 @@ -198,7 +198,7 @@ else: name = None if not space.is_w(w_argdefs, space.w_None): - defs_w = space.fixedunpack(w_argdefs) + defs_w = space.fixedview(w_argdefs) else: defs_w = [] nfreevars = 0 @@ -323,7 +323,7 @@ if space.is_w(w_func_dict, space.w_None): w_func_dict = None self.w_func_dict = w_func_dict - self.defs_w = space.fixedunpack(w_defs_w) + self.defs_w = space.fixedview(w_defs_w) self.w_module = w_module def fget_func_defaults(space, self): @@ -338,7 +338,7 @@ return if not space.is_true(space.isinstance(w_defaults, space.w_tuple)): raise OperationError( space.w_TypeError, space.wrap("func_defaults must be set to a tuple object or None") ) - self.defs_w = space.fixedunpack(w_defaults) + self.defs_w = space.fixedview(w_defaults) def fdel_func_defaults(space, self): self.defs_w = [] Modified: pypy/branch/rlist-jit/pypy/module/__builtin__/interp_classobj.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/__builtin__/interp_classobj.py (original) +++ pypy/branch/rlist-jit/pypy/module/__builtin__/interp_classobj.py Wed Nov 24 14:27:09 2010 @@ -40,7 +40,7 @@ # XXX missing: lengthy and obscure logic about "__module__" - bases_w = space.fixedunpack(w_bases) + bases_w = space.fixedview(w_bases) for w_base in bases_w: if not isinstance(w_base, W_ClassObject): w_metaclass = space.type(w_base) @@ -92,7 +92,7 @@ raise OperationError( space.w_TypeError, space.wrap("__bases__ must be a tuple object")) - bases_w = space.fixedunpack(w_bases) + bases_w = space.fixedview(w_bases) for w_base in bases_w: if not isinstance(w_base, W_ClassObject): raise OperationError(space.w_TypeError, Modified: pypy/branch/rlist-jit/pypy/module/cpyext/typeobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/cpyext/typeobject.py (original) +++ pypy/branch/rlist-jit/pypy/module/cpyext/typeobject.py Wed Nov 24 14:27:09 2010 @@ -262,7 +262,7 @@ class W_PyCTypeObject(W_TypeObject): def __init__(self, space, pto): - bases_w = space.fixedunpack(from_ref(space, pto.c_tp_bases)) + bases_w = space.fixedview(from_ref(space, pto.c_tp_bases)) dict_w = {} add_operators(space, dict_w, pto) Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/objspace.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Wed Nov 24 14:27:09 2010 @@ -378,17 +378,6 @@ def fixedview_unroll(self, w_obj, expected_length=-1): return self.fixedview(w_obj, expected_length, unroll=True) - def fixedunpack(self, w_obj, expected_length=-1): - if isinstance(w_obj, W_TupleObject): - t = w_obj.wrappeditems[:] - elif isinstance(w_obj, W_ListObject): - t = w_obj.wrappeditems[:] - else: - return ObjSpace.fixedunpack(self, w_obj, expected_length) - if expected_length != -1 and len(t) != expected_length: - raise self._wrap_expected_length(expected_length, len(t)) - return t - def listview(self, w_obj, expected_length=-1): if isinstance(w_obj, W_ListObject): t = w_obj.wrappeditems Modified: pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/typeobject.py Wed Nov 24 14:27:09 2010 @@ -652,7 +652,7 @@ if w_mro_func is not None and not space.is_w(w_where, space.w_type): w_mro_meth = space.get(w_mro_func, w_self) w_mro = space.call_function(w_mro_meth) - mro_w = space.fixedunpack(w_mro) + mro_w = space.fixedview(w_mro) w_self.mro_w = validate_custom_mro(space, mro_w) return # done w_self.mro_w = w_self.compute_default_mro()[:] Modified: pypy/branch/rlist-jit/pypy/objspace/std/typetype.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/typetype.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/typetype.py Wed Nov 24 14:27:09 2010 @@ -12,7 +12,7 @@ w_typetype = _precheck_for_new(space, w_typetype) - bases_w = space.fixedunpack(w_bases) + bases_w = space.fixedview(w_bases) w_winner = w_typetype for base in bases_w: @@ -115,7 +115,7 @@ "can only assign tuple to %s.__bases__, not %s", w_type.name, space.type(w_value).getname(space, '?')) - newbases_w = space.fixedunpack(w_value) + newbases_w = space.fixedview(w_value) if len(newbases_w) == 0: raise operationerrfmt(space.w_TypeError, "can only assign non-empty tuple to %s.__bases__, not ()", From arigo at codespeak.net Wed Nov 24 14:41:50 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 14:41:50 +0100 (CET) Subject: [pypy-svn] r79460 - in pypy/branch/rlist-jit/pypy: objspace/std rlib Message-ID: <20101124134150.B96DB282BEB@codespeak.net> Author: arigo Date: Wed Nov 24 14:41:48 2010 New Revision: 79460 Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py pypy/branch/rlist-jit/pypy/rlib/debug.py Log: Tentative: introduce and use _list_annotated_as_modifiable_again(), which is a bit of a hack but at a single place. Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/objspace.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Wed Nov 24 14:41:48 2010 @@ -9,6 +9,7 @@ from pypy.objspace.descroperation import DescrOperation, raiseattrerror from pypy.rlib.objectmodel import instantiate, r_dict, specialize from pypy.rlib.debug import make_sure_not_resized, list_not_modified_any_more +from pypy.rlib.debug import _list_annotated_as_modifiable_again from pypy.rlib.rarithmetic import base_int from pypy.rlib.objectmodel import we_are_translated from pypy.rlib.jit import hint @@ -360,17 +361,16 @@ """ Fast paths """ if isinstance(w_obj, W_TupleObject): - t = w_obj.wrappeditems + t = _list_annotated_as_modifiable_again(w_obj.wrappeditems) elif isinstance(w_obj, W_ListObject): t = list_not_modified_any_more(w_obj.wrappeditems[:]) else: if unroll: - r = ObjSpace.unpackiterable_unroll( - self, w_obj, expected_length) + return make_sure_not_resized(ObjSpace.unpackiterable_unroll( + self, w_obj, expected_length)[:]) else: - r = ObjSpace.unpackiterable( - self, w_obj, expected_length) - return list_not_modified_any_more(r[:]) + return make_sure_not_resized(ObjSpace.unpackiterable( + self, w_obj, expected_length)[:]) if expected_length != -1 and len(t) != expected_length: raise self._wrap_expected_length(expected_length, len(t)) return t Modified: pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py Wed Nov 24 14:41:48 2010 @@ -13,6 +13,9 @@ def __init__(w_self, wrappeditems): w_self.wrappeditems = wrappeditems # a list of wrapped values + # Note that to make annotation happy with respect to + # _immutable_fields_, wrappeditems must be a list known to + # be never mutated. Use space.newtuple() if you are unsure. def __repr__(w_self): """ representation for debugging purposes """ Modified: pypy/branch/rlist-jit/pypy/rlib/debug.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rlib/debug.py (original) +++ pypy/branch/rlist-jit/pypy/rlib/debug.py Wed Nov 24 14:41:48 2010 @@ -254,6 +254,35 @@ return hop.inputarg(hop.args_r[0], arg=0) +def _list_annotated_as_modifiable_again(arg): + """ Returns an annotator-time copy of the list 'arg' which is + *not* flagged as 'don't mutate me'. Actually implemented as just + returning 'arg'. This is useful for cases like space.fixedview() + where the returned list can escape to a lot of places that don't + expect a list flagged 'don't mutate me'. Of course it relies on + the assumption that the code will not actually go ahead and mutate + the returned list, in practice. + """ + return arg + +class Entry(ExtRegistryEntry): + _about_ = _list_annotated_as_modifiable_again + + def compute_result_annotation(self, s_arg): + from pypy.annotation.model import SomeList + assert isinstance(s_arg, SomeList) + if self.bookkeeper.annotator.translator.config.translation.list_comprehension_operations: + s_arg = s_arg.listdef.offspring() + else: + from pypy.annotation.annrpython import log + log.WARNING('_list_annotated_as_modified_again called, but has no effect since list_comprehension is off') + return s_arg + + def specialize_call(self, hop): + hop.exception_cannot_occur() + return hop.inputarg(hop.args_r[0], arg=0) + + class IntegerCanBeNegative(Exception): pass From antocuni at codespeak.net Wed Nov 24 15:06:46 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 24 Nov 2010 15:06:46 +0100 (CET) Subject: [pypy-svn] r79461 - in pypy/trunk/pypy/jit/tool: . test Message-ID: <20101124140646.2EC225080C@codespeak.net> Author: antocuni Date: Wed Nov 24 15:06:44 2010 New Revision: 79461 Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric pypy/trunk/pypy/jit/tool/log2gnumeric.py pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py Log: add a sheet containing the number of loops and bridges alive. Draw it on the "Memory Usage" graph. Add a "Loops" graph Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric ============================================================================== Binary files. No diff available. Modified: pypy/trunk/pypy/jit/tool/log2gnumeric.py ============================================================================== --- pypy/trunk/pypy/jit/tool/log2gnumeric.py (original) +++ pypy/trunk/pypy/jit/tool/log2gnumeric.py Wed Nov 24 15:06:44 2010 @@ -28,7 +28,8 @@ xml = gzip.open('log-template.gnumeric').read() xml = replace_sheet(xml, 'translation-task', tasks_rows(time0, data)) xml = replace_sheet(xml, 'gc-collect', gc_collect_rows(time0, data)) - xml = replace_sheet(xml, 'memusage', memusage_rows(logname + '.memusage', maxtime)) + xml = replace_sheet(xml, 'loops', loops_rows(time0, data)) + xml = replace_sheet(xml, 'vmrss', memusage_rows(logname + '.vmrss', maxtime)) # out = gzip.open(outname, 'wb') out.write(xml) @@ -115,6 +116,31 @@ clock = int(a, 16) - time0 yield clock, 1, b + +def loops_rows(time0, data): + s = r""" +\[([0-9a-f]+)\] \{jit-mem-looptoken-(alloc|free) +(.*?)\[ +""" + # + r = re.compile(s.replace('\n', '')) + yield 'clock', 'total', 'loops', 'bridges' + loops = 0 + bridges = 0 + for clock, action, text in r.findall(data): + clock = int(clock, 16) - time0 + if text.startswith('allocating Loop #'): + loops += 1 + elif text.startswith('allocating Bridge #'): + bridges += 1 + elif text.startswith('freeing Loop #'): + match = re.match('freeing Loop # .* with ([0-9]*) attached bridges', text) + loops -=1 + bridges -= int(match.group(1)) + total = loops+bridges + yield clock, loops+bridges, loops, bridges + + def memusage_rows(filename, maxtime): try: lines = open(filename).readlines() Modified: pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py ============================================================================== --- pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py (original) +++ pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py Wed Nov 24 15:06:44 2010 @@ -73,4 +73,27 @@ assert rows[1] == (0, 100) assert rows[2] == (1000, 200) assert rows[3] == (2000, 300) - + +def test_loops_rows(): + log = """\ +[1000] {jit-mem-looptoken-alloc +allocating Loop # 0 +[1001] jit-mem-looptoken-alloc} +[2000] {jit-mem-looptoken-alloc +allocating Loop # 1 +[2001] jit-mem-looptoken-alloc} +[3000] {jit-mem-looptoken-alloc +allocating Bridge # 1 of Loop # 0 +[3001] jit-mem-looptoken-alloc} +[4000] {jit-mem-looptoken-free +freeing Loop # 0 with 1 attached bridges +[4001] +""" + log = log.replace('\n', '') + rows = list(log2gnumeric.loops_rows(0x1000, log)) + assert len(rows) == 5 + assert rows[0] == ('clock', 'total', 'loops', 'bridges') + assert rows[1] == ( 0x0, 1, 1, 0) + assert rows[2] == ( 0x1000, 2, 2, 0) + assert rows[3] == ( 0x2000, 3, 2, 1) + assert rows[4] == ( 0x3000, 1, 1, 0) From arigo at codespeak.net Wed Nov 24 15:10:48 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 15:10:48 +0100 (CET) Subject: [pypy-svn] r79462 - pypy/trunk/pypy/jit/metainterp/test Message-ID: <20101124141048.85C39282BEF@codespeak.net> Author: arigo Date: Wed Nov 24 15:10:47 2010 New Revision: 79462 Modified: pypy/trunk/pypy/jit/metainterp/test/test_memmgr.py Log: Bah. Modified: pypy/trunk/pypy/jit/metainterp/test/test_memmgr.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_memmgr.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_memmgr.py Wed Nov 24 15:10:47 2010 @@ -1,9 +1,10 @@ -import sys, py +import sys if len(sys.argv) >= 4 and sys.argv[1] == '--sub': sys.path[:] = eval(sys.argv[2]) # hacks for test_integration from pypy.conftest import option option.__dict__.update(eval(sys.argv[3])) +import py from pypy.jit.metainterp.memmgr import MemoryManager from pypy.jit.metainterp.test.test_basic import LLJitMixin from pypy.rlib.jit import JitDriver, dont_look_inside From arigo at codespeak.net Wed Nov 24 15:15:24 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 15:15:24 +0100 (CET) Subject: [pypy-svn] r79463 - pypy/branch/rlist-jit/pypy/rpython Message-ID: <20101124141524.189435080E@codespeak.net> Author: arigo Date: Wed Nov 24 15:15:22 2010 New Revision: 79463 Modified: pypy/branch/rlist-jit/pypy/rpython/rlist.py Log: Small fixes and improvements. Modified: pypy/branch/rlist-jit/pypy/rpython/rlist.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rpython/rlist.py (original) +++ pypy/branch/rlist-jit/pypy/rpython/rlist.py Wed Nov 24 15:15:22 2010 @@ -9,7 +9,7 @@ from pypy.rpython import robject from pypy.rlib.objectmodel import malloc_zero_filled from pypy.rlib.debug import ll_assert -from pypy.rlib.rarithmetic import ovfcheck, widen, r_uint +from pypy.rlib.rarithmetic import ovfcheck, widen, r_uint, intmask from pypy.rpython.annlowlevel import ADTInterface from pypy.rlib import rgc @@ -696,12 +696,12 @@ length = l.ll_length() # common case: 0 <= index < length if r_uint(index) >= r_uint(length): # Failed, so either (-length <= index < 0), or we have to raise - # IndexError. Using r_uint, the condition can be rewritten as - # (-length-1 < index), which is (~length < index). - if r_uint(~length) < r_uint(index): - index += length - else: + # IndexError. First add 'length' to get the final index, then + # check that we now have (0 <= index < length). + index = r_uint(index) + r_uint(length) + if index >= r_uint(length): raise IndexError + index = intmask(index) else: # We don't want checking, but still want to support index < 0. # Only call ll_length() if needed. @@ -736,14 +736,14 @@ if func is dum_checkidx: length = l.ll_length() if r_uint(index) >= r_uint(length): # see comments in ll_getitem(). - if r_uint(~length) < r_uint(index): - index += length - else: + index = r_uint(index) + r_uint(length) + if index >= r_uint(length): raise IndexError + index = intmask(index) else: if index < 0: index += l.ll_length() - ll_assert(index >= 0, "negative list getitem index out of bound") + ll_assert(index >= 0, "negative list setitem index out of bound") l.ll_setitem_fast(index, newitem) # no oopspec -- the function is inlined by the JIT @@ -770,18 +770,19 @@ ll_delitem_nonneg.oopspec = 'list.delitem(l, index)' def ll_delitem(func, l, i): - length = l.ll_length() - if i < 0: - i += length if func is dum_checkidx: - if r_uint(i) >= r_uint(length): - raise IndexError + length = l.ll_length() + if r_uint(index) >= r_uint(length): # see comments in ll_getitem(). + index = r_uint(index) + r_uint(length) + if index >= r_uint(length): + raise IndexError + index = intmask(index) else: - ll_assert(i >= 0, "negative list delitem index out of bound") - ll_assert(i < length, "list delitem index out of bound") + if index < 0: + index += l.ll_length() + ll_assert(index >= 0, "negative list delitem index out of bound") ll_delitem_nonneg(dum_nocheck, l, i) -ll_delitem.oopspec = 'list.delitem(l, i)' - +# no oopspec -- the function is inlined by the JIT def ll_extend(l1, l2): len1 = l1.ll_length() From arigo at codespeak.net Wed Nov 24 15:15:39 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 15:15:39 +0100 (CET) Subject: [pypy-svn] r79464 - in pypy/branch/rlist-jit/pypy: interpreter module/exceptions objspace/std Message-ID: <20101124141539.97F60282BF1@codespeak.net> Author: arigo Date: Wed Nov 24 15:15:37 2010 New Revision: 79464 Modified: pypy/branch/rlist-jit/pypy/interpreter/baseobjspace.py pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Log: Revert these files to their original status too. Modified: pypy/branch/rlist-jit/pypy/interpreter/baseobjspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/interpreter/baseobjspace.py (original) +++ pypy/branch/rlist-jit/pypy/interpreter/baseobjspace.py Wed Nov 24 15:15:37 2010 @@ -8,7 +8,7 @@ from pypy.tool.cache import Cache from pypy.tool.uid import HUGEVAL_BYTES from pypy.rlib.objectmodel import we_are_translated -from pypy.rlib.debug import make_sure_not_resized, list_not_modified_any_more +from pypy.rlib.debug import make_sure_not_resized from pypy.rlib.timer import DummyTimer, Timer from pypy.rlib.rarithmetic import r_uint from pypy.rlib import jit @@ -764,10 +764,9 @@ 'unpackiterable_unroll')) def fixedview(self, w_iterable, expected_length=-1): - """ A fixed list view of w_iterable. The result is supposed to be - used temporarily: it is a list with the annotation flag 'do not mutate'. + """ A fixed list view of w_iterable. Don't modify the result """ - return list_not_modified_any_more(self.unpackiterable(w_iterable, + return make_sure_not_resized(self.unpackiterable(w_iterable, expected_length)[:]) fixedview_unroll = fixedview Modified: pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py (original) +++ pypy/branch/rlist-jit/pypy/module/exceptions/interp_exceptions.py Wed Nov 24 15:15:37 2010 @@ -77,7 +77,6 @@ from pypy.interpreter.gateway import interp2app, Arguments from pypy.interpreter.error import OperationError from pypy.rlib import rwin32 -from pypy.rlib.debug import list_not_modified_any_more def readwrite_attrproperty_w(name, cls): def fget(space, obj): @@ -93,15 +92,11 @@ and will be deprecated at some point. """ w_dict = None - _empty_args_w = [] + args_w = [] def __init__(self, space): self.space = space self.w_message = space.w_None - self.args_w = list_not_modified_any_more(W_BaseException._empty_args_w) - # Note that 'self.args_w' is annotated as a list-that-is-not-modified, - # which cannot easily be mixed together with a general list annotation. - # That's why we use 'list_not_modified_any_more()'. def descr_init(self, space, args_w): self.args_w = args_w @@ -332,7 +327,7 @@ self.w_strerror = args_w[1] if len(args_w) == 3: self.w_filename = args_w[2] - self.args_w = list_not_modified_any_more([args_w[0], args_w[1]]) + self.args_w = [args_w[0], args_w[1]] descr_init.unwrap_spec = ['self', ObjSpace, 'args_w'] # since we rebind args_w, we need special reduce, grump Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/objspace.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Wed Nov 24 15:15:37 2010 @@ -363,7 +363,7 @@ if isinstance(w_obj, W_TupleObject): t = _list_annotated_as_modifiable_again(w_obj.wrappeditems) elif isinstance(w_obj, W_ListObject): - t = list_not_modified_any_more(w_obj.wrappeditems[:]) + t = w_obj.wrappeditems[:] else: if unroll: return make_sure_not_resized(ObjSpace.unpackiterable_unroll( From antocuni at codespeak.net Wed Nov 24 16:08:47 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 24 Nov 2010 16:08:47 +0100 (CET) Subject: [pypy-svn] r79465 - pypy/trunk/pypy/jit/tool Message-ID: <20101124150847.0979D282BD4@codespeak.net> Author: antocuni Date: Wed Nov 24 16:08:46 2010 New Revision: 79465 Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric Log: tweak the colors of the graph Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric ============================================================================== Binary files. No diff available. From arigo at codespeak.net Wed Nov 24 16:13:11 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 16:13:11 +0100 (CET) Subject: [pypy-svn] r79466 - pypy/trunk/pypy/doc/config Message-ID: <20101124151311.ED6F8282BDC@codespeak.net> Author: arigo Date: Wed Nov 24 16:13:10 2010 New Revision: 79466 Removed: pypy/trunk/pypy/doc/config/translation.jit_debug.txt Log: Kill; option is gone. From arigo at codespeak.net Wed Nov 24 16:14:55 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 16:14:55 +0100 (CET) Subject: [pypy-svn] r79467 - pypy/trunk/pypy/config/test Message-ID: <20101124151455.C891A282BE3@codespeak.net> Author: arigo Date: Wed Nov 24 16:14:53 2010 New Revision: 79467 Modified: pypy/trunk/pypy/config/test/test_support.py Log: Remove tabs. Modified: pypy/trunk/pypy/config/test/test_support.py ============================================================================== --- pypy/trunk/pypy/config/test/test_support.py (original) +++ pypy/trunk/pypy/config/test/test_support.py Wed Nov 24 16:14:53 2010 @@ -4,32 +4,32 @@ import os, sys, py cpuinfo = """ -processor : 0 +processor\t: 0 -processor : 1 -vendor_id : GenuineIntel -cpu family : 6 -model : 37 -model name : Intel(R) Core(TM) i7 CPU L 620 @ 2.00GHz -stepping : 2 - -processor : 2 -vendor_id : GenuineIntel -cpu family : 6 -model : 37 -model name : Intel(R) Core(TM) i7 CPU L 620 @ 2.00GHz -stepping : 2 - -processor : 3 -vendor_id : GenuineIntel -cpu family : 6 -model : 37 -model name : Intel(R) Core(TM) i7 CPU L 620 @ 2.00GHz -stepping : 2 -cpu MHz : 1199.000 -cache size : 4096 KB -physical id : 0 -siblings : 4 +processor\t: 1 +vendor_id\t: GenuineIntel +cpu family\t: 6 +model\t\t: 37 +model name\t: Intel(R) Core(TM) i7 CPU L 620 @ 2.00GHz +stepping\t: 2 + +processor\t: 2 +vendor_id\t: GenuineIntel +cpu family\t: 6 +model\t\t: 37 +model name\t: Intel(R) Core(TM) i7 CPU L 620 @ 2.00GHz +stepping\t: 2 + +processor\t: 3 +vendor_id\t: GenuineIntel +cpu family\t: 6 +model\t\t: 37 +model name\t: Intel(R) Core(TM) i7 CPU L 620 @ 2.00GHz +stepping\t: 2 +cpu MHz\t\t: 1199.000 +cache size\t: 4096 KB +physical id\t: 0 +siblings\t: 4 """ class FakeEnviron: From arigo at codespeak.net Wed Nov 24 16:17:43 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 16:17:43 +0100 (CET) Subject: [pypy-svn] r79468 - pypy/trunk/pypy/tool Message-ID: <20101124151743.6E147282BEF@codespeak.net> Author: arigo Date: Wed Nov 24 16:17:42 2010 New Revision: 79468 Modified: pypy/trunk/pypy/tool/logparser.py Log: Baaaah. Modified: pypy/trunk/pypy/tool/logparser.py ============================================================================== --- pypy/trunk/pypy/tool/logparser.py (original) +++ pypy/trunk/pypy/tool/logparser.py Wed Nov 24 16:17:42 2010 @@ -25,7 +25,7 @@ return parse_log(lines, verbose=verbose) def parse_log(lines, verbose=False): - color = "(?:\x1b.*?m)" + color = "(?:\x1b.*?m)?" r_start = re.compile(color + r"\[([0-9a-fA-F]+)\] \{([\w-]+)" + color + "$") r_stop = re.compile(color + r"\[([0-9a-fA-F]+)\] ([\w-]+)\}" + color + "$") lasttime = 0 From arigo at codespeak.net Wed Nov 24 16:22:58 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 16:22:58 +0100 (CET) Subject: [pypy-svn] r79469 - pypy/trunk/pypy/jit/backend Message-ID: <20101124152258.3BC87282BEF@codespeak.net> Author: arigo Date: Wed Nov 24 16:22:56 2010 New Revision: 79469 Modified: pypy/trunk/pypy/jit/backend/conftest.py Log: Revert r79395. It's not pointless, it breaks e.g. test_random.py. Modified: pypy/trunk/pypy/jit/backend/conftest.py ============================================================================== --- pypy/trunk/pypy/jit/backend/conftest.py (original) +++ pypy/trunk/pypy/jit/backend/conftest.py Wed Nov 24 16:22:56 2010 @@ -4,6 +4,8 @@ """ import py, random +option = py.test.config.option + def pytest_addoption(parser): group = parser.getgroup('random test options') group.addoption('--random-seed', action="store", type="int", From afa at codespeak.net Wed Nov 24 16:24:39 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Wed, 24 Nov 2010 16:24:39 +0100 (CET) Subject: [pypy-svn] r79470 - pypy/branch/fast-forward/pypy/objspace/std Message-ID: <20101124152439.6A4D4282BF4@codespeak.net> Author: afa Date: Wed Nov 24 16:24:37 2010 New Revision: 79470 Modified: pypy/branch/fast-forward/pypy/objspace/std/sliceobject.py Log: Add a repr() to W_SliceObject; for debugging Modified: pypy/branch/fast-forward/pypy/objspace/std/sliceobject.py ============================================================================== --- pypy/branch/fast-forward/pypy/objspace/std/sliceobject.py (original) +++ pypy/branch/fast-forward/pypy/objspace/std/sliceobject.py Wed Nov 24 16:24:37 2010 @@ -6,7 +6,6 @@ from pypy.objspace.std.register_all import register_all from pypy.objspace.std.slicetype import _Eval_SliceIndex - class W_SliceObject(W_Object): from pypy.objspace.std.slicetype import slice_typedef as typedef _immutable_ = True @@ -80,6 +79,10 @@ slicelength = (stop - start - 1) / step + 1 return start, stop, step, slicelength + def __repr__(self): + return "" % ( + self.w_start, self.w_stop, self.w_step) + registerimplementation(W_SliceObject) From arigo at codespeak.net Wed Nov 24 16:25:58 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 16:25:58 +0100 (CET) Subject: [pypy-svn] r79471 - pypy/trunk/pypy/jit/backend/x86/test Message-ID: <20101124152558.CA6B9282BF5@codespeak.net> Author: arigo Date: Wed Nov 24 16:25:57 2010 New Revision: 79471 Modified: pypy/trunk/pypy/jit/backend/x86/test/test_runner.py Log: Fix test. Modified: pypy/trunk/pypy/jit/backend/x86/test/test_runner.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/test/test_runner.py (original) +++ pypy/trunk/pypy/jit/backend/x86/test/test_runner.py Wed Nov 24 16:25:57 2010 @@ -521,8 +521,7 @@ self.cpu.finish_once() finally: debug._log = None - assert dlog[1] == ('jit-backend-counts', [('debug_print', '0:10')]) - + assert ('jit-backend-counts', [('debug_print', '0:10')]) in dlog def test_debugger_checksum(self): loop = """ From arigo at codespeak.net Wed Nov 24 16:40:21 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 16:40:21 +0100 (CET) Subject: [pypy-svn] r79472 - pypy/trunk/pypy/jit/backend/x86/test Message-ID: <20101124154021.81FBF36C221@codespeak.net> Author: arigo Date: Wed Nov 24 16:40:19 2010 New Revision: 79472 Modified: pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py Log: Revert this part of r79402. It looks like it was checked in together with the other files by mistake. Modified: pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py (original) +++ pypy/trunk/pypy/jit/backend/x86/test/test_ztranslation.py Wed Nov 24 16:40:19 2010 @@ -87,13 +87,8 @@ return int(res) # def main(i, j): - res = f(i, j) + libffi_stuff(i, j) - fd = os.open('/tmp/x', os.O_WRONLY | os.O_CREAT, 0666) - rgc.dump_rpy_heap(fd) - os.close(fd) - return res - #expected = main(40, -49) - expected = 3 + return f(i, j) + libffi_stuff(i, j) + expected = main(40, -49) res = self.meta_interp(main, [40, -49]) assert res == expected From arigo at codespeak.net Wed Nov 24 16:43:19 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 16:43:19 +0100 (CET) Subject: [pypy-svn] r79473 - in pypy/branch/rlist-jit/pypy: annotation objspace/std Message-ID: <20101124154319.80B7B5080E@codespeak.net> Author: arigo Date: Wed Nov 24 16:43:17 2010 New Revision: 79473 Modified: pypy/branch/rlist-jit/pypy/annotation/policy.py pypy/branch/rlist-jit/pypy/annotation/specialize.py pypy/branch/rlist-jit/pypy/objspace/std/objspace.py pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py Log: More reverts. Now the idea is that lists with the 'must_not_mutate' flag should not be present in general code, so space.newtuple() expects a list *without* this flag. Modified: pypy/branch/rlist-jit/pypy/annotation/policy.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/policy.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/policy.py Wed Nov 24 16:43:17 2010 @@ -1,7 +1,6 @@ # base annotation policy for specialization from pypy.annotation.specialize import default_specialize as default from pypy.annotation.specialize import specialize_argvalue, specialize_argtype, specialize_arglistitemtype -from pypy.annotation.specialize import specialize_list_not_modified_any_more from pypy.annotation.specialize import memo # for some reason, model must be imported first, # or we create a cycle. @@ -76,7 +75,6 @@ specialize__arg = staticmethod(specialize_argvalue) # specialize:arg(N) specialize__argtype = staticmethod(specialize_argtype) # specialize:argtype(N) specialize__arglistitemtype = staticmethod(specialize_arglistitemtype) - specialize__list_not_modified_any_more = staticmethod(specialize_list_not_modified_any_more) def specialize__ll(pol, *args): from pypy.rpython.annlowlevel import LowLevelAnnotatorPolicy Modified: pypy/branch/rlist-jit/pypy/annotation/specialize.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/specialize.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/specialize.py Wed Nov 24 16:43:17 2010 @@ -369,8 +369,3 @@ else: key = s.listdef.listitem.s_value.knowntype return maybe_star_args(funcdesc, key, args_s) - -def specialize_list_not_modified_any_more(funcdesc, args_s, i): - s = args_s[i] - key = (s.knowntype is list and s.listdef.listitem.must_not_mutate) - return maybe_star_args(funcdesc, key, args_s) Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/objspace.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Wed Nov 24 16:43:17 2010 @@ -266,8 +266,7 @@ def newtuple(self, list_w): assert isinstance(list_w, list) - return W_TupleObject(list_not_modified_any_more(list_w)) - newtuple._annspecialcase_ = 'specialize:list_not_modified_any_more(1)' + return W_TupleObject(list_not_modified_any_more(list_w) def newlist(self, list_w): return W_ListObject(list_w) Modified: pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py Wed Nov 24 16:43:17 2010 @@ -15,7 +15,8 @@ w_self.wrappeditems = wrappeditems # a list of wrapped values # Note that to make annotation happy with respect to # _immutable_fields_, wrappeditems must be a list known to - # be never mutated. Use space.newtuple() if you are unsure. + # be never mutated. Use space.newtuple() instead of directly + # calling this constructor. def __repr__(w_self): """ representation for debugging purposes """ @@ -61,7 +62,7 @@ def getslice__Tuple_ANY_ANY(space, w_tuple, w_start, w_stop): length = len(w_tuple.wrappeditems) start, stop = normalize_simple_slice(space, length, w_start, w_stop) - return W_TupleObject(w_tuple.wrappeditems[start:stop]) + return space.newtuple(w_tuple.wrappeditems[start:stop]) def contains__Tuple_ANY(space, w_tuple, w_obj): for w_item in w_tuple.wrappeditems: @@ -76,7 +77,7 @@ def add__Tuple_Tuple(space, w_tuple1, w_tuple2): items1 = w_tuple1.wrappeditems items2 = w_tuple2.wrappeditems - return W_TupleObject(items1 + items2) + return space.newtuple(items1 + items2) def mul_tuple_times(space, w_tuple, w_times): try: @@ -88,7 +89,7 @@ if times == 1 and space.type(w_tuple) == space.w_tuple: return w_tuple items = w_tuple.wrappeditems - return W_TupleObject(items * times) + return space.newtuple(items * times) def mul__Tuple_ANY(space, w_tuple, w_times): return mul_tuple_times(space, w_tuple, w_times) From arigo at codespeak.net Wed Nov 24 16:43:58 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 16:43:58 +0100 (CET) Subject: [pypy-svn] r79474 - pypy/branch/rlist-jit/pypy/objspace/std Message-ID: <20101124154358.0CD12282BD4@codespeak.net> Author: arigo Date: Wed Nov 24 16:43:57 2010 New Revision: 79474 Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Log: We can make sure that the list returned by fixedview() is never resized. Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/objspace.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Wed Nov 24 16:43:57 2010 @@ -372,7 +372,7 @@ self, w_obj, expected_length)[:]) if expected_length != -1 and len(t) != expected_length: raise self._wrap_expected_length(expected_length, len(t)) - return t + return make_sure_not_resized(t) def fixedview_unroll(self, w_obj, expected_length=-1): return self.fixedview(w_obj, expected_length, unroll=True) From arigo at codespeak.net Wed Nov 24 16:44:10 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 16:44:10 +0100 (CET) Subject: [pypy-svn] r79475 - pypy/branch/rlist-jit/pypy/annotation Message-ID: <20101124154410.D604A282BF6@codespeak.net> Author: arigo Date: Wed Nov 24 16:44:09 2010 New Revision: 79475 Modified: pypy/branch/rlist-jit/pypy/annotation/listdef.py Log: Improve the repr. Modified: pypy/branch/rlist-jit/pypy/annotation/listdef.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/listdef.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/listdef.py Wed Nov 24 16:44:09 2010 @@ -189,9 +189,11 @@ self.listitem.generalize(s_value) def __repr__(self): - return '<[%r]%s%s>' % (self.listitem.s_value, + return '<[%r]%s%s%s%s>' % (self.listitem.s_value, self.listitem.mutated and 'm' or '', - self.listitem.resized and 'r' or '') + self.listitem.resized and 'r' or '', + self.listitem.must_not_mutate and '!M' or '', + self.listitem.must_not_resize and '!R' or '') def mutate(self): self.listitem.mutate() From arigo at codespeak.net Wed Nov 24 16:51:47 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 16:51:47 +0100 (CET) Subject: [pypy-svn] r79476 - pypy/branch/rlist-jit/pypy/objspace/std Message-ID: <20101124155147.A44095080F@codespeak.net> Author: arigo Date: Wed Nov 24 16:51:46 2010 New Revision: 79476 Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Log: Bah, SyntaxError. Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/objspace.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Wed Nov 24 16:51:46 2010 @@ -266,7 +266,7 @@ def newtuple(self, list_w): assert isinstance(list_w, list) - return W_TupleObject(list_not_modified_any_more(list_w) + return W_TupleObject(list_not_modified_any_more(list_w)) def newlist(self, list_w): return W_ListObject(list_w) From antocuni at codespeak.net Wed Nov 24 17:05:36 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 24 Nov 2010 17:05:36 +0100 (CET) Subject: [pypy-svn] r79477 - pypy/trunk/pypy/jit/tool Message-ID: <20101124160536.3BC20282BF8@codespeak.net> Author: antocuni Date: Wed Nov 24 17:05:34 2010 New Revision: 79477 Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric Log: cosmetic changes to the graph Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric ============================================================================== Binary files. No diff available. From antocuni at codespeak.net Wed Nov 24 17:18:38 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 24 Nov 2010 17:18:38 +0100 (CET) Subject: [pypy-svn] r79478 - in pypy/trunk/pypy/jit/tool: . test Message-ID: <20101124161838.310C150810@codespeak.net> Author: antocuni Date: Wed Nov 24 17:18:36 2010 New Revision: 79478 Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric pypy/trunk/pypy/jit/tool/log2gnumeric.py pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py Log: add a series showing the cpython vmrss (scaled to take the same time as the pypy one). While at it, rename every occurence of "memusage" to "vmrss" Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric ============================================================================== Binary files. No diff available. Modified: pypy/trunk/pypy/jit/tool/log2gnumeric.py ============================================================================== --- pypy/trunk/pypy/jit/tool/log2gnumeric.py (original) +++ pypy/trunk/pypy/jit/tool/log2gnumeric.py Wed Nov 24 17:18:36 2010 @@ -29,7 +29,8 @@ xml = replace_sheet(xml, 'translation-task', tasks_rows(time0, data)) xml = replace_sheet(xml, 'gc-collect', gc_collect_rows(time0, data)) xml = replace_sheet(xml, 'loops', loops_rows(time0, data)) - xml = replace_sheet(xml, 'vmrss', memusage_rows(logname + '.vmrss', maxtime)) + xml = replace_sheet(xml, 'vmrss', vmrss_rows(logname + '.vmrss', maxtime)) + xml = replace_sheet(xml, 'cpython-vmrss', vmrss_rows('cpython.vmrss', maxtime)) # out = gzip.open(outname, 'wb') out.write(xml) @@ -141,16 +142,16 @@ yield clock, loops+bridges, loops, bridges -def memusage_rows(filename, maxtime): +def vmrss_rows(filename, maxtime): try: lines = open(filename).readlines() except IOError: - print 'Warning: cannot find file %s, skipping the memusage sheet' + print 'Warning: cannot find file %s, skipping this sheet' lines = [] - for row in memusage_rows_impl(lines, maxtime): + for row in vmrss_rows_impl(lines, maxtime): yield row -def memusage_rows_impl(lines, maxtime): +def vmrss_rows_impl(lines, maxtime): yield 'inferred clock', 'VmRSS' numlines = len(lines) for i, line in enumerate(lines): Modified: pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py ============================================================================== --- pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py (original) +++ pypy/trunk/pypy/jit/tool/test/test_log2gnumeric.py Wed Nov 24 17:18:36 2010 @@ -65,9 +65,9 @@ assert rows[2] == (0x8000-0x1000, 1, 'rtype_lltype') -def test_memusage_rows(): +def test_vmrss_rows(): lines = ['100', '200', '300'] - rows = list(log2gnumeric.memusage_rows_impl(lines, 2000)) + rows = list(log2gnumeric.vmrss_rows_impl(lines, 2000)) assert len(rows) == 4 assert rows[0] == ('inferred clock', 'VmRSS') assert rows[1] == (0, 100) From arigo at codespeak.net Wed Nov 24 17:31:11 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 17:31:11 +0100 (CET) Subject: [pypy-svn] r79479 - in pypy/branch/rlist-jit/pypy: annotation annotation/test interpreter module/_sre objspace/std rlib rlib/rsre Message-ID: <20101124163111.C4763282BEF@codespeak.net> Author: arigo Date: Wed Nov 24 17:31:09 2010 New Revision: 79479 Modified: pypy/branch/rlist-jit/pypy/annotation/description.py pypy/branch/rlist-jit/pypy/annotation/listdef.py pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py pypy/branch/rlist-jit/pypy/interpreter/pycode.py pypy/branch/rlist-jit/pypy/module/_sre/interp_sre.py pypy/branch/rlist-jit/pypy/objspace/std/objspace.py pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py pypy/branch/rlist-jit/pypy/rlib/debug.py pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py Log: Simplify yet another time the approach. Modified: pypy/branch/rlist-jit/pypy/annotation/description.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/description.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/description.py Wed Nov 24 17:31:09 2010 @@ -649,7 +649,7 @@ if search in cdesc.classdict['_immutable_fields_'].value: s_result.listdef.never_resize() s_copy = s_result.listdef.offspring() - s_copy.listdef.never_mutate() + s_copy.listdef.mark_as_immutable() return s_copy cdesc = cdesc.basedesc return s_result # common case Modified: pypy/branch/rlist-jit/pypy/annotation/listdef.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/listdef.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/listdef.py Wed Nov 24 17:31:09 2010 @@ -14,7 +14,7 @@ resized = False # True for lists resized after creation range_step = None # the step -- only for lists only created by a range() dont_change_any_more = False # set to True when too late for changes - must_not_mutate = False # list_not_modified_any_more() + immutable = False # for getattr out of _immutable_fields_ = ['attr[*]'] must_not_resize = False # make_sure_not_resized() # what to do if range_step is different in merge. @@ -38,8 +38,7 @@ if not self.mutated: if self.dont_change_any_more: raise TooLateForChange - if self.must_not_mutate: - raise ListChangeUnallowed("mutating list") + self.immutable = False self.mutated = True def resize(self): @@ -71,10 +70,7 @@ # things more general self, other = other, self - if other.must_not_mutate: - if self.mutated: - raise ListChangeUnallowed("list merge with a mutated") - self.must_not_mutate = True + self.immutable &= other.immutable if other.must_not_resize: if self.resized: raise ListChangeUnallowed("list merge with a resized") @@ -192,7 +188,7 @@ return '<[%r]%s%s%s%s>' % (self.listitem.s_value, self.listitem.mutated and 'm' or '', self.listitem.resized and 'r' or '', - self.listitem.must_not_mutate and '!M' or '', + self.listitem.immutable and 'I' or '', self.listitem.must_not_resize and '!R' or '') def mutate(self): @@ -207,11 +203,16 @@ raise ListChangeUnallowed("list already resized") self.listitem.must_not_resize = True - def never_mutate(self): + def mark_as_immutable(self): + # Sets the 'immutable' flag. Note that unlike "never resized", + # the immutable flag is only a hint. It is cleared again e.g. + # when we merge with a "normal" list that doesn't have it. It + # is thus expected to live only shortly, mostly for the case + # of writing 'x.list[n]'. self.never_resize() if self.listitem.mutated: raise ListChangeUnallowed("list already mutated") - self.listitem.must_not_mutate = True + self.listitem.immutable = True MOST_GENERAL_LISTDEF = ListDef(None, SomeObject()) Modified: pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py Wed Nov 24 17:31:09 2010 @@ -3375,24 +3375,6 @@ a.build_types(fn, []) # assert did not raise ListChangeUnallowed - def test_list_not_modified_any_more(self): - from pypy.rlib.debug import list_not_modified_any_more - - def pycode(consts): - return list_not_modified_any_more(consts) - def build1(): - return pycode(consts=[1]) - def build2(): - return pycode(consts=[0]) - def fn(): - build1() - build2() - - a = self.RPythonAnnotator() - a.translator.config.translation.list_comprehension_operations = True - a.build_types(fn, []) - # assert did not raise ListChangeUnallowed - def test_return_immutable_list(self): class A: _immutable_fields_ = 'lst[*]' @@ -3405,7 +3387,7 @@ a = self.RPythonAnnotator() s = a.build_types(f, [int]) - assert s.listdef.listitem.must_not_mutate + assert s.listdef.listitem.immutable def test_immutable_list_is_actually_resized(self): class A: @@ -3420,6 +3402,39 @@ a = self.RPythonAnnotator() py.test.raises(ListChangeUnallowed, a.build_types, f, [int]) + def test_can_merge_immutable_list_with_regular_list(self): + class A: + _immutable_fields_ = 'lst[*]' + def f(n): + a = A() + l1 = [n, 0] + l1[1] = n+1 + a.lst = l1 + if n > 0: + lst = a.lst + else: + lst = [] + return lst + + a = self.RPythonAnnotator() + s = a.build_types(f, [int]) + assert not s.listdef.listitem.immutable + + def f(n): + a = A() + l1 = [n, 0] + l1[1] = n+1 + a.lst = l1 + if n > 0: + lst = [] + else: + lst = a.lst + return lst + + a = self.RPythonAnnotator() + s = a.build_types(f, [int]) + assert not s.listdef.listitem.immutable + def g(n): return [0,1,2,n] Modified: pypy/branch/rlist-jit/pypy/interpreter/pycode.py ============================================================================== --- pypy/branch/rlist-jit/pypy/interpreter/pycode.py (original) +++ pypy/branch/rlist-jit/pypy/interpreter/pycode.py Wed Nov 24 17:31:09 2010 @@ -15,7 +15,7 @@ CO_OPTIMIZED, CO_NEWLOCALS, CO_VARARGS, CO_VARKEYWORDS, CO_NESTED, CO_GENERATOR, CO_CONTAINSGLOBALS) from pypy.rlib.rarithmetic import intmask -from pypy.rlib.debug import make_sure_not_resized, list_not_modified_any_more +from pypy.rlib.debug import make_sure_not_resized from pypy.rlib import jit from pypy.rlib.objectmodel import compute_hash from pypy.tool.stdlib_opcode import opcodedesc, HAVE_ARGUMENT @@ -69,7 +69,7 @@ self.co_stacksize = stacksize self.co_flags = flags self.co_code = code - self.co_consts_w = list_not_modified_any_more(consts) + self.co_consts_w = consts self.co_names_w = [space.new_interned_str(aname) for aname in names] self.co_varnames = varnames self.co_freevars = freevars Modified: pypy/branch/rlist-jit/pypy/module/_sre/interp_sre.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/_sre/interp_sre.py (original) +++ pypy/branch/rlist-jit/pypy/module/_sre/interp_sre.py Wed Nov 24 17:31:09 2010 @@ -6,7 +6,6 @@ from pypy.interpreter.gateway import interp2app, ObjSpace, W_Root from pypy.interpreter.error import OperationError from pypy.rlib.rarithmetic import intmask -from pypy.rlib.debug import list_not_modified_any_more from pypy.tool.pairtype import extendabletype @@ -112,18 +111,17 @@ space = self.space if pos < 0: pos = 0 if endpos < pos: endpos = pos - pattern = list_not_modified_any_more(self.code) if space.is_true(space.isinstance(w_string, space.w_unicode)): unicodestr = space.unicode_w(w_string) if pos > len(unicodestr): pos = len(unicodestr) if endpos > len(unicodestr): endpos = len(unicodestr) - return rsre_core.UnicodeMatchContext(pattern, unicodestr, + return rsre_core.UnicodeMatchContext(self.code, unicodestr, pos, endpos, self.flags) else: str = space.bufferstr_w(w_string) if pos > len(str): pos = len(str) if endpos > len(str): endpos = len(str) - return rsre_core.StrMatchContext(pattern, str, + return rsre_core.StrMatchContext(self.code, str, pos, endpos, self.flags) def getmatch(self, ctx, found): Modified: pypy/branch/rlist-jit/pypy/objspace/std/objspace.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/objspace.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/objspace.py Wed Nov 24 17:31:09 2010 @@ -8,8 +8,7 @@ transparent, callmethod, proxyobject) from pypy.objspace.descroperation import DescrOperation, raiseattrerror from pypy.rlib.objectmodel import instantiate, r_dict, specialize -from pypy.rlib.debug import make_sure_not_resized, list_not_modified_any_more -from pypy.rlib.debug import _list_annotated_as_modifiable_again +from pypy.rlib.debug import make_sure_not_resized from pypy.rlib.rarithmetic import base_int from pypy.rlib.objectmodel import we_are_translated from pypy.rlib.jit import hint @@ -266,7 +265,8 @@ def newtuple(self, list_w): assert isinstance(list_w, list) - return W_TupleObject(list_not_modified_any_more(list_w)) + make_sure_not_resized(list_w) + return W_TupleObject(list_w) def newlist(self, list_w): return W_ListObject(list_w) @@ -360,7 +360,7 @@ """ Fast paths """ if isinstance(w_obj, W_TupleObject): - t = _list_annotated_as_modifiable_again(w_obj.wrappeditems) + t = w_obj.wrappeditems elif isinstance(w_obj, W_ListObject): t = w_obj.wrappeditems[:] else: Modified: pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py ============================================================================== --- pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py (original) +++ pypy/branch/rlist-jit/pypy/objspace/std/tupleobject.py Wed Nov 24 17:31:09 2010 @@ -6,17 +6,15 @@ from pypy.rlib.rarithmetic import intmask from pypy.objspace.std.sliceobject import W_SliceObject, normalize_simple_slice from pypy.interpreter import gateway +from pypy.rlib.debug import make_sure_not_resized class W_TupleObject(W_Object): from pypy.objspace.std.tupletype import tuple_typedef as typedef _immutable_fields_ = ['wrappeditems[*]'] def __init__(w_self, wrappeditems): + make_sure_not_resized(wrappeditems) w_self.wrappeditems = wrappeditems # a list of wrapped values - # Note that to make annotation happy with respect to - # _immutable_fields_, wrappeditems must be a list known to - # be never mutated. Use space.newtuple() instead of directly - # calling this constructor. def __repr__(w_self): """ representation for debugging purposes """ @@ -57,12 +55,12 @@ for i in range(slicelength): subitems[i] = items[start] start += step - return space.newtuple(subitems) + return W_TupleObject(subitems) def getslice__Tuple_ANY_ANY(space, w_tuple, w_start, w_stop): length = len(w_tuple.wrappeditems) start, stop = normalize_simple_slice(space, length, w_start, w_stop) - return space.newtuple(w_tuple.wrappeditems[start:stop]) + return W_TupleObject(w_tuple.wrappeditems[start:stop]) def contains__Tuple_ANY(space, w_tuple, w_obj): for w_item in w_tuple.wrappeditems: @@ -77,7 +75,7 @@ def add__Tuple_Tuple(space, w_tuple1, w_tuple2): items1 = w_tuple1.wrappeditems items2 = w_tuple2.wrappeditems - return space.newtuple(items1 + items2) + return W_TupleObject(items1 + items2) def mul_tuple_times(space, w_tuple, w_times): try: @@ -89,7 +87,7 @@ if times == 1 and space.type(w_tuple) == space.w_tuple: return w_tuple items = w_tuple.wrappeditems - return space.newtuple(items * times) + return W_TupleObject(items * times) def mul__Tuple_ANY(space, w_tuple, w_times): return mul_tuple_times(space, w_tuple, w_times) Modified: pypy/branch/rlist-jit/pypy/rlib/debug.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rlib/debug.py (original) +++ pypy/branch/rlist-jit/pypy/rlib/debug.py Wed Nov 24 17:31:09 2010 @@ -226,62 +226,6 @@ hop.exception_cannot_occur() return hop.inputarg(hop.args_r[0], arg=0) -def list_not_modified_any_more(arg): - """ Returns an annotator-time copy of the list 'arg' which is - flagged as 'don't mutate me'. Actually implemented as just - returning 'arg'. This is useful for debugging only. - """ - return arg - -class Entry(ExtRegistryEntry): - _about_ = list_not_modified_any_more - - def compute_result_annotation(self, s_arg): - from pypy.annotation.model import SomeList - assert isinstance(s_arg, SomeList) - # the logic behind it is that we try not to propagate - # make_sure_not_resized, when list comprehension is not on - if self.bookkeeper.annotator.translator.config.translation.list_comprehension_operations: - s_arg = s_arg.listdef.offspring() - s_arg.listdef.never_mutate() - else: - from pypy.annotation.annrpython import log - log.WARNING('list_not_modified_any_more called, but has no effect since list_comprehension is off') - return s_arg - - def specialize_call(self, hop): - hop.exception_cannot_occur() - return hop.inputarg(hop.args_r[0], arg=0) - - -def _list_annotated_as_modifiable_again(arg): - """ Returns an annotator-time copy of the list 'arg' which is - *not* flagged as 'don't mutate me'. Actually implemented as just - returning 'arg'. This is useful for cases like space.fixedview() - where the returned list can escape to a lot of places that don't - expect a list flagged 'don't mutate me'. Of course it relies on - the assumption that the code will not actually go ahead and mutate - the returned list, in practice. - """ - return arg - -class Entry(ExtRegistryEntry): - _about_ = _list_annotated_as_modifiable_again - - def compute_result_annotation(self, s_arg): - from pypy.annotation.model import SomeList - assert isinstance(s_arg, SomeList) - if self.bookkeeper.annotator.translator.config.translation.list_comprehension_operations: - s_arg = s_arg.listdef.offspring() - else: - from pypy.annotation.annrpython import log - log.WARNING('_list_annotated_as_modified_again called, but has no effect since list_comprehension is off') - return s_arg - - def specialize_call(self, hop): - hop.exception_cannot_occur() - return hop.inputarg(hop.args_r[0], arg=0) - class IntegerCanBeNegative(Exception): pass Modified: pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py (original) +++ pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py Wed Nov 24 17:31:09 2010 @@ -1,5 +1,5 @@ import sys -from pypy.rlib.debug import check_nonneg, list_not_modified_any_more +from pypy.rlib.debug import check_nonneg, make_sure_not_modified from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.rsre import rsre_char from pypy.tool.sourcetools import func_with_new_name @@ -471,6 +471,7 @@ while True: op = ctx.pat(ppos) ppos += 1 + make_sure_not_modified(ctx.pattern) #jit.jit_debug("sre_match", op, ppos, ptr) # @@ -946,7 +947,6 @@ def match(pattern, string, start=0, end=sys.maxint, flags=0): start, end = _adjust(start, end, len(string)) - pattern = list_not_modified_any_more(pattern) ctx = StrMatchContext(pattern, string, start, end, flags) if match_context(ctx): return ctx @@ -955,7 +955,6 @@ def search(pattern, string, start=0, end=sys.maxint, flags=0): start, end = _adjust(start, end, len(string)) - pattern = list_not_modified_any_more(pattern) ctx = StrMatchContext(pattern, string, start, end, flags) if search_context(ctx): return ctx From antocuni at codespeak.net Wed Nov 24 17:39:04 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Wed, 24 Nov 2010 17:39:04 +0100 (CET) Subject: [pypy-svn] r79480 - pypy/trunk/pypy/jit/tool Message-ID: <20101124163904.C308C50810@codespeak.net> Author: antocuni Date: Wed Nov 24 17:39:03 2010 New Revision: 79480 Added: pypy/trunk/pypy/jit/tool/cpython.vmrss Log: add the vmrss file produced by armin's memusage.py when running ./translate.py -Ojit at rev 79456 Added: pypy/trunk/pypy/jit/tool/cpython.vmrss ============================================================================== --- (empty file) +++ pypy/trunk/pypy/jit/tool/cpython.vmrss Wed Nov 24 17:39:03 2010 @@ -0,0 +1,4101 @@ +124 +16600 +20620 +20640 +22036 +94084 +94084 +94108 +94108 +94108 +94108 +94108 +94120 +94164 +94160 +94160 +94160 +94160 +94160 +94216 +110644 +123144 +135236 +143680 +148500 +153104 +157088 +160640 +164760 +167992 +163108 +163232 +163232 +163436 +163436 +163436 +163444 +163444 +163444 +163448 +163448 +163448 +163448 +163448 +163448 +167060 +170948 +174432 +177212 +177216 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176508 +176520 +176520 +176520 +176520 +176520 +176520 +176520 +176520 +176520 +176540 +176544 +176544 +176544 +176544 +176544 +176544 +176544 +176544 +176544 +176544 +176544 +176544 +179120 +187120 +189380 +191052 +192156 +193320 +194900 +195860 +198516 +201484 +202600 +202600 +202600 +202600 +202832 +202832 +202836 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +202840 +207784 +212136 +216320 +220508 +224696 +228876 +245088 +245088 +247844 +252032 +256212 +260400 +264592 +268776 +272776 +275060 +279244 +283428 +287616 +291032 +293900 +298080 +302272 +304364 +308548 +310644 +312740 +316924 +319016 +323208 +325296 +327392 +331572 +333668 +335760 +337856 +354328 +356424 +358520 +362700 +364792 +366892 +368984 +371080 +373168 +375260 +377356 +379448 +381540 +383636 +383636 +385732 +387820 +390032 +391160 +392292 +394552 +396816 +397092 +399072 +401340 +403600 +405860 +408008 +408148 +412640 +414900 +417164 +419420 +421680 +423944 +426204 +428464 +430724 +432768 +434980 +436476 +437932 +440332 +441984 +442740 +445152 +447688 +449148 +449960 +452436 +454712 +454896 +457180 +458888 +459688 +462040 +463480 +464408 +466812 +467244 +469224 +471096 +471684 +474136 +474328 +476508 +478872 +481356 +483472 +483780 +486072 +488480 +489668 +490888 +493420 +495704 +496836 +498116 +500520 +502756 +503668 +505400 +507760 +509296 +510204 +512764 +514708 +515508 +517372 +519764 +520648 +522188 +524596 +525524 +527004 +529412 +534224 +536632 +538080 +539216 +541588 +542560 +543384 +543384 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +518804 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519024 +519028 +519028 +519028 +519028 +519028 +519028 +519028 +519028 +519028 +519028 +519028 +519028 +519032 +519032 +519032 +519032 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519036 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519048 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519052 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519056 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519060 +519064 +519064 +519064 +519064 +519064 +519064 +519064 +519064 +519064 +519064 +519064 +519064 +519064 +519064 +519064 +519064 +519064 +519064 +519064 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +519068 +520592 +520592 +520904 +522740 +522740 +523212 +525256 +525256 +525316 +526552 +526552 +526560 +528508 +528508 +528508 +530040 +530040 +530040 +532684 +532684 +532684 +534948 +534948 +534948 +538028 +538028 +538028 +541404 +541448 +541448 +543784 +543784 +543784 +545184 +545476 +545768 +545832 +545832 +545832 +546016 +546016 +546128 +546184 +546184 +546184 +546184 +546184 +546184 +546184 +546184 +546184 +546184 +546184 +546184 +546184 +546184 +546184 +546184 +546184 +546184 +546228 +546228 +546228 +546228 +546228 +546228 +546228 +546228 +546228 +546228 +546228 +546228 +546228 +546228 +546228 +546708 +546708 +546708 +547988 +550420 +550420 +550420 +552896 +555796 +555796 +555796 +559136 +560280 +560280 +560996 +562504 +563772 +564672 +564672 +565268 +567936 +568884 +569028 +569236 +569292 +569292 +570236 +572960 +573980 +573980 +574508 +577404 +579188 +579188 +579508 +582836 +584468 +584468 +585544 +591292 +591292 +591292 +595868 +597588 +597588 +598404 +602772 +603964 +603964 +605488 +609740 +610468 +610468 +611884 +616440 +617108 +617108 +618156 +622276 +623784 +623784 +624128 +624376 +625544 +626736 +627112 +627112 +627116 +627656 +628836 +628836 +628836 +629160 +630180 +630488 +630488 +630492 +631288 +632144 +632144 +632144 +632172 +632688 +633220 +633220 +633220 +633284 +633756 +633916 +633916 +633916 +634012 +634608 +634608 +634608 +634624 +634732 +635144 +635144 +635144 +635196 +635680 +635868 +635868 +635944 +638440 +639964 +639980 +639980 +640056 +641052 +642064 +642064 +642064 +642248 +643080 +643832 +643832 +643832 +644116 +646500 +647424 +648236 +649032 +649156 +649156 +649256 +651556 +652056 +652504 +652860 +653168 +653440 +653876 +654096 +654304 +654304 +654304 +654756 +655648 +657064 +657064 +657064 +657064 +657488 +657756 +657756 +657756 +658112 +658736 +658736 +658736 +658796 +659304 +659376 +659376 +659376 +659848 +661000 +661172 +661172 +661236 +662240 +663240 +663508 +663508 +663660 +664324 +665512 +665764 +665764 +665764 +666448 +667692 +668112 +668112 +668112 +668624 +669508 +670388 +670536 +670536 +670536 +670564 +671288 +672268 +672268 +672268 +672676 +674504 +675072 +675072 +675072 +675156 +675896 +676700 +676700 +676700 +676820 +677988 +678628 +678628 +678628 +678776 +679744 +680400 +680400 +680400 +705092 +705156 +705156 +705156 +705156 +705156 +705156 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705160 +705264 +705264 +705264 +705400 +705476 +706168 +706292 +706292 +706292 +706504 +706568 +706980 +707012 +707012 +707012 +707196 +707280 +707904 +707904 +707904 +707924 +708112 +708176 +708676 +708676 +708676 +708696 +708892 +708984 +709588 +709588 +709588 +709612 +709804 +709848 +710300 +710300 +710300 +710676 +710520 +710604 +711156 +711156 +711156 +711336 +711352 +711576 +712080 +712080 +712080 +712300 +712408 +712500 +712648 +712648 +712648 +712648 +713060 +713300 +713716 +713716 +713716 +714072 +714196 +714568 +714568 +714568 +714596 +714956 +715112 +715808 +715808 +715808 +717504 +717628 +717660 +717660 +717660 +718620 +719048 +719424 +719424 +719424 +719480 +719924 +720612 +720612 +720612 +720620 +722584 +722848 +722848 +722848 +723060 +724108 +724604 +724604 +724604 +725108 +726168 +726348 +726348 +726348 +727216 +728204 +728204 +728204 +728324 +729396 +730152 +730152 +730152 +730396 +736796 +736800 +736800 +736800 +736800 +736800 +736800 +736800 +736800 +737136 +738296 +738400 +738400 +738400 +739092 +740128 +740128 +740128 +740140 +741092 +741980 +741980 +741980 +742060 +743060 +743796 +743796 +743796 +743836 +744440 +745348 +745348 +745348 +745400 +746108 +746848 +746952 +746952 +746952 +747496 +748608 +748744 +749084 +749084 +749084 +749172 +750172 +751468 +751592 +751592 +751592 +751688 +751928 +752152 +752308 +759152 +760152 +760152 +760152 +756356 +754816 +756356 +756356 +756356 +756688 +756688 +756820 +757152 +757200 +757400 +757432 +757432 +757432 +757632 +757956 +758404 +758844 +759480 +760064 +760552 +760552 +760552 +760560 +761180 +761632 +762288 +762800 +763700 +764504 +764716 +764716 +764716 +764940 +765388 +765936 +767748 +767056 +767300 +767484 +767868 +768316 +768316 +768316 +768316 +768700 +768828 +768700 +769340 +769260 +771008 +771552 +771652 +771716 +772580 +772708 +772740 +772740 +772740 +772292 +772740 +772944 +773188 +773700 +774084 +774084 +774404 +774020 +774532 +774020 +774596 +774340 +774468 +774468 +774468 +774724 +774792 +774980 +775368 +775816 +775816 +776264 +777480 +778292 +778408 +778440 +778440 +778440 +778440 +778440 +778440 +778440 +778440 +778440 +778440 +778440 +778632 +778696 +778952 +779016 +779016 +779016 +779016 +780812 +780812 +780812 +781068 +781580 +781772 +781712 +781868 +782092 +782420 +782796 +782796 +782796 +782796 +782668 +783128 +783436 +783820 +784076 +784332 +784908 +785164 +785500 +786188 +786188 +786188 +786188 +786188 +786188 +786188 +786412 +786896 +787084 +787404 +787532 +787724 +787568 +788108 +788428 +788748 +788752 +789520 +789520 +789520 +788880 +789440 +789452 +789516 +790092 +790284 +790604 +790860 +791052 +791372 +791628 +792012 +792012 +792396 +792780 +792780 +792780 +792780 +792780 +793228 +793228 +793868 +793868 +793996 +793996 +794636 +794636 +794636 +795084 +795084 +795468 +795532 +795980 +795980 +796300 +796364 +796364 +796364 +796364 +796748 +797132 +797132 +797516 +797644 +797644 +798380 +798604 +798860 +798924 +799372 +799564 +799756 +799756 +799756 +799756 +799816 +800292 +800792 +801312 +801816 +802364 +802880 +803456 +803660 +803660 +803660 +803812 +804388 +804960 +805516 +806084 +806668 +807324 +807980 +807980 +807980 +807980 +808416 +809084 +809784 +810492 +811160 +812900 +813476 +813876 +813876 +813876 +813876 +814508 +815152 +815864 +816556 +817260 +817916 +818708 +819272 +819352 +819352 +819352 +819352 +819884 +820308 +820888 +821012 +821012 +821204 +821588 +821588 +821588 +821972 +822228 +822164 +822932 +823252 +823252 +823252 +823252 +823252 +823256 +823612 +823920 +823936 +824140 +824168 +824220 +824280 +824400 +824664 +825776 +825776 +825776 +825776 +832168 +832168 +832168 +832168 +832808 +833492 +833492 +833492 +833492 +833700 +834308 +834428 +834428 +834428 +834780 +836216 +836216 +836216 +836836 +839308 +839308 +839308 +839308 +839416 +840344 +840344 +840344 +840344 +841724 +841724 +841724 +841724 +843800 +843800 +843800 +843800 +845692 +846072 +846072 +846072 +846096 +846736 +847096 +847156 +847156 +847156 +847160 +847180 +847360 +847360 +847360 +847360 +847416 +849248 +849248 +849248 +849248 +849716 +849716 +849716 +849716 +849740 +851168 +851168 +851168 +851168 +854544 +854544 +854544 +854544 +854564 +855332 +855332 +855332 +855332 +857092 +857092 +857092 +857092 +858000 +859388 +859388 +859388 +859388 +861584 +861584 +861584 +861584 +861584 +863464 +863464 +863464 +863464 +864304 +866500 +866500 +866500 +866500 +868952 +868952 +868952 +868952 +869740 +872108 +872108 +872108 +872108 +873208 +875564 +875564 +875564 +875564 +878568 +878568 +878568 +878568 +878576 +880780 +880780 +880780 +880780 +884048 +884048 +884048 +884048 +884048 +884048 +886516 +886516 +886516 +886516 +886516 +886536 +888112 +888112 +888112 +888112 +888112 +888112 +888656 +888984 +888984 +888984 +888984 +888984 +888984 +889076 +890288 +890288 +890288 +890288 +890288 +890288 +890404 +892900 +892900 +892900 +892900 +892900 +892900 +892900 +892900 +895760 +895760 +895760 +895760 +895760 +895760 +895920 +897624 +897624 +897624 +897624 +897624 +897624 +897624 +897624 +897628 +898024 +898024 +898024 +898024 +898024 +898060 +900024 +900024 +900024 +900024 +900024 +900024 +900240 +901436 +901436 +901436 +901436 +901436 +901556 +903116 +903116 +903116 +903116 +903116 +903128 +905084 +905084 +905084 +905084 +905084 +905084 +905096 +906832 +906832 +906832 +906832 +906832 +906832 +908916 +908916 +908916 +908916 +908916 +908916 +908916 +910720 +910720 +910720 +910720 +910720 +910720 +911780 +912072 +912072 +912072 +912072 +914472 +914472 +914472 +914472 +914472 +917120 +917120 +917120 +917120 +917120 +919056 +919056 +919056 +919056 +919056 +920316 +920316 +920316 +920316 +920316 +920892 +920892 +920892 +920892 +920892 +922996 +922996 +922996 +922996 +922996 +925564 +925564 +925564 +925564 +925564 +927780 +927780 +927780 +927780 +928016 +928936 +929048 +930864 +930864 +930864 +930864 +932980 +933304 +933304 +933304 +933304 +933540 +934292 +935452 +935528 +935528 +935528 +935528 +935528 +935528 +935528 +935528 +935600 +936112 +936112 +936208 +936208 +936208 +936208 +936308 +936308 +936316 +936368 +936368 +936368 +936368 +936368 +936368 +936368 +936368 +936368 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +936376 +937404 +937404 +937404 +937404 +939968 +939968 +939968 +939968 +939968 +939968 +939968 +939968 +939968 +939968 +939968 +939968 +939968 +940516 +940516 +940516 +940516 +947168 +947168 +947168 +947168 +951948 +951948 +951948 +951948 +953488 +956916 +956916 +956916 +956916 +952296 +955376 +953420 +953260 +953900 +953516 +953900 +955052 +953516 +953516 +954284 +953324 +953516 +956208 +956208 +956208 +956208 +954668 +948988 +948988 +948988 +948988 +948988 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +945908 +947704 +948068 +948068 +948068 +948068 +948068 +948552 +949024 +949024 +949024 +949024 +949024 +949944 +950492 +950616 +950616 +950616 +950616 +951416 +952000 +952564 +952564 +952564 +952564 +952620 +953296 +953952 +954332 +954332 +954332 +954332 +954532 +955532 +956216 +956216 +956216 +956216 +956216 +956584 +957396 +957704 +957704 +957704 +957704 +957740 +958620 +959444 +959444 +959444 +959444 +959444 +960224 +963784 +963868 +963868 +963868 +963868 +963872 +964684 +972452 +972452 +972452 +972452 +972452 +972452 +973220 +974548 +974548 +974548 +974548 +974548 +975540 +977704 +978792 +978792 +978792 +978792 +978792 +981296 +982700 +983180 +983180 +983180 +983180 +983368 +985060 +987512 +987688 +987688 +987688 +987688 +988180 +990724 +993084 +993124 +993124 +993124 +993124 +993604 +995444 +996640 +996640 +996640 +996640 +996640 +997892 +999160 +1001452 +1001452 +1001452 +1001452 +1001452 +1002264 +1004120 +1004916 +1004916 +1004916 +1004916 +1004916 +1006648 +1010244 +1010548 +1010548 +1010548 +1010548 +1010572 +1011860 +1013748 +1017264 +1017264 +1017264 +1017264 +1017264 +1019096 +1021044 +1021044 +1021044 +1021044 +1021044 +1021536 +1023636 +1024964 +1024964 +1024964 +1024964 +1024964 +1025592 +1027672 +1029384 +1029384 +1029384 +1029384 +1029384 +1030056 +1032244 +1033756 +1033756 +1033756 +1033756 +1033756 +1034384 +1035856 +1036588 +1038428 +1038428 +1038428 +1038428 +1038428 +1039288 +1041088 +1042508 +1042508 +1042508 +1042508 +1042508 +1043544 +1045280 +1046580 +1046580 +1046580 +1046580 +1046580 +1047040 +1048560 +1049872 +1050576 +1050576 +1050576 +1050576 +1050576 +1052016 +1056044 +1057360 +1057360 +1057360 +1057360 +1057360 +1058336 +1059900 +1061244 +1061704 +1061704 +1061704 +1061704 +1061704 +1063148 +1063972 +1065404 +1067064 +1067536 +1067536 +1067536 +1067536 +1067536 +1069284 +1070524 +1072340 +1072836 +1072836 +1072836 +1072836 +1072836 +1073584 +1074592 +1076404 +1076832 +1076832 +1076832 +1076832 +1076832 +1078124 +1079640 +1080220 +1080644 +1080736 +1080736 +1080736 +1080736 +1080924 +1082868 +1083368 +1084412 +1084412 +1084412 +1084412 +1084412 +1085216 +1087068 +1087960 +1087960 +1087960 +1087960 +1087960 +1089788 +1093132 +1095064 +1095064 +1095064 +1095064 +1095064 +1095140 +1097092 +1098948 +1098948 +1098948 +1098948 +1098948 +1098980 +1100812 +1102032 +1102032 +1102032 +1102032 +1102032 +1105464 +1116944 +1119248 +1120364 +1120364 +1120364 +1120364 +1120364 +1121568 +1122908 +1123680 +1124604 +1125076 +1125076 +1125076 +1125076 +1125076 +1126292 +1128160 +1129952 +1129952 +1129952 +1129952 +1129952 +1130496 +1131884 +1133032 +1134204 +1135460 +1135636 +1135636 +1135636 +1135636 +1135636 +1136764 +1138048 +1139412 +1140764 +1141164 +1141188 +1142440 +1142440 +1142440 +1142440 +1142440 +1142440 +1143980 +1145876 +1146576 +1146576 +1146576 +1146576 +1146576 +1146576 +1147680 +1148328 +1148960 +1148960 +1148960 +1148960 +1148960 +1149004 +1150700 +1152228 +1153364 +1153364 +1153520 +1153784 +1154588 +1154680 +1154712 +1154728 +1154784 +1154992 +1155356 +1155620 +1155856 +1156044 +1156420 +1157392 +1158760 +1158980 +1158988 +1159000 +1162724 +1162740 +1162788 +1163112 +1163188 +1163188 +1163188 +1163188 +1163188 +1163384 +1165668 +1166648 +1166652 +1166664 +1166676 +1166688 +1166692 +1166696 +1166700 +1166700 +1172848 +1172852 +1174888 +1176824 +1176836 +1176852 +1176860 +1176876 +1176880 +1176888 +1176892 +1176900 +1176912 +1176944 +1177248 +1177712 +1178172 +1178536 +1178656 +1178780 +1178920 +1179044 +1179188 +1179384 +1180296 +1180300 +1180300 +1180300 +1180300 +1180300 +1180300 +1180372 +1180380 +1180468 +1180524 +1180524 +1180524 +1180524 +1180524 +1180576 +1180580 +1180644 +1180684 +1180684 +1180684 +1180684 +1180684 +1180684 +1180724 +1180756 +1180852 +1180904 +1180904 +1180904 +1180904 +1180904 +1180904 +1181096 +1181400 +1181744 +1181744 +1181744 +1181744 +1181744 +1181744 +1181936 +1181936 +1181936 +1181936 +1181936 +1181936 +1181936 +1181936 +1181936 +1181972 +1182004 +1182004 +1182004 +1182004 +1182004 +1182004 +1182004 +1182004 +1182004 +1182128 +1182156 +1182156 +1182156 +1182156 +1182156 +1182156 +1182156 +1182156 +1182180 +1182180 +1182180 +1182180 +1182180 +1182180 +1182180 +1182368 +1182516 +1182516 +1182516 +1182516 +1182516 +1182516 +1182516 +1182516 +1182516 +1182516 +1182516 +1182516 +1182516 +1182516 +1182516 +1182516 +1182516 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183576 +1183596 +1183772 +1183772 +1183772 +1183772 +1183772 +1183772 +1183772 +1183772 +1183772 +1183776 +1183776 +1183776 +1183776 +1183776 +1183776 +1183776 +1183776 +1183776 +1183952 +1184000 +1184000 +1184000 +1184000 +1184000 +1184000 +1184000 +1184200 +1184832 +1184832 +1184832 +1184832 +1184832 +1184832 +1184928 +1184928 +1184940 +1184940 +1184940 +1184940 +1184940 +1184940 +1184940 +1184940 +1185144 +1185144 +1185144 +1185144 +1185144 +1185144 +1185144 +1185144 +1185144 +1185144 +1185144 +1185144 +1185144 +1185144 +1185196 +1185196 +1185196 +1185196 +1185196 +1185196 +1185196 +1185196 +1185196 +1185268 +1185268 +1185268 +1185268 +1185268 +1185268 +1185268 +1185268 +1186444 +1186776 +1186776 +1186776 +1186776 +1186776 +1187664 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188072 +1188168 +1188168 +1188168 +1188168 +1188168 +1188168 +1188168 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189240 +1189292 +1189292 +1189292 +1189292 +1189292 +1189292 +1189292 +1189532 +1189532 +1189532 +1189532 +1189532 +1189532 +1189532 +1189532 +1189532 +1189532 +1189532 +1189532 +1189532 +1189532 +1189532 +1189532 +1189704 +1189708 +1189708 +1189708 +1189708 +1189708 +1190160 +1190160 +1190160 +1190160 +1190160 +1190160 +1190160 +1190160 +1190160 +1191100 +1191100 +1191100 +1191100 +1191100 +1191316 +1191316 +1191316 +1191316 +1191316 +1191316 +1191316 +1191316 +1191316 +1191316 +1191316 +1191316 +1191316 +1191316 +1191316 +1191316 +1191316 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191748 +1191772 +1191772 +1191772 +1191772 +1191772 +1191772 +1191772 +1191772 +1192964 +1192964 +1192964 +1192964 +1192964 +1192964 +1192964 +1193060 +1193060 +1193060 +1193060 +1193060 +1193060 +1193060 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193072 +1193076 +1193076 +1193076 +1193076 +1193076 +1193076 +1193124 +1193124 +1193360 +1194108 +1194108 +1194108 +1194108 +1194108 +1193380 +1193460 +1193460 +1193460 +1193460 +1193460 +1193460 +1193460 +1193460 +1193792 +1193792 +1193792 +1193792 +1193792 +1193792 +1194000 +1194000 +1194000 +1194000 +1194000 +1194000 +1194000 +1194000 +1194000 +1194000 +1194000 +1194000 +1194000 +1194000 +1194000 +1194000 +1194000 +1194048 +1194048 +1194048 +1194048 +1194048 +1194048 +1194048 +1194048 +1194048 +1194048 +1194048 +1194048 +1194048 +1194048 +1194048 +1194048 +1194328 +1194328 +1194328 +1194328 +1194328 +1194328 +1194328 +1194328 +1194328 +1194328 +1194360 +1194360 +1194360 +1194360 +1194360 +1194360 +1194360 +1194508 +1194508 +1194508 +1194508 +1194508 +1194508 +1194512 +1194668 +1194668 +1194668 +1194668 +1194668 +1194668 +1194668 +1194668 +1194668 +1194668 +1194668 +1194668 +1194668 +1194668 +1194668 +1194668 +1194912 +1194912 +1194912 +1194912 +1194912 +1194912 +1194912 +1194912 +1194912 +1194912 +1194912 +1194912 +1194912 +1194912 +1194912 +1194912 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196596 +1196660 +1196764 +1196764 +1196764 +1196764 +1196764 +1196764 +1196764 +1196764 +1196764 +1196764 +1196764 +1196764 +1196764 +1196764 +1196764 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1196948 +1197236 +1197236 +1197236 +1197236 +1197236 +1197236 +1197236 +1197236 +1197236 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197288 +1197376 +1197384 +1197596 +1197596 +1197596 +1197596 +1197596 +1197596 +1197596 +1197596 +1197588 +1197588 +1197588 +1197588 +1197588 +1197588 +1197588 +1197588 +1197564 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197460 +1197492 +1197508 +1197516 +1197516 +1197516 +1197516 +1197516 +1197516 From arigo at codespeak.net Wed Nov 24 17:45:30 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 17:45:30 +0100 (CET) Subject: [pypy-svn] r79481 - pypy/branch/rlist-jit/pypy/rlib/rsre Message-ID: <20101124164530.2F94D282BFA@codespeak.net> Author: arigo Date: Wed Nov 24 17:45:28 2010 New Revision: 79481 Modified: pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py Log: This hint is no longer needed now. (Hopefully the effect we get is the same.) Modified: pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py (original) +++ pypy/branch/rlist-jit/pypy/rlib/rsre/rsre_core.py Wed Nov 24 17:45:28 2010 @@ -1,5 +1,5 @@ import sys -from pypy.rlib.debug import check_nonneg, make_sure_not_modified +from pypy.rlib.debug import check_nonneg from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.rsre import rsre_char from pypy.tool.sourcetools import func_with_new_name @@ -471,7 +471,6 @@ while True: op = ctx.pat(ppos) ppos += 1 - make_sure_not_modified(ctx.pattern) #jit.jit_debug("sre_match", op, ppos, ptr) # From arigo at codespeak.net Wed Nov 24 18:13:52 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 18:13:52 +0100 (CET) Subject: [pypy-svn] r79482 - in pypy/branch/rlist-jit/pypy: annotation annotation/test rpython Message-ID: <20101124171352.D06485080C@codespeak.net> Author: arigo Date: Wed Nov 24 18:13:51 2010 New Revision: 79482 Modified: pypy/branch/rlist-jit/pypy/annotation/listdef.py pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py pypy/branch/rlist-jit/pypy/rpython/rlist.py Log: Fixes. Modified: pypy/branch/rlist-jit/pypy/annotation/listdef.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/listdef.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/listdef.py Wed Nov 24 18:13:51 2010 @@ -210,9 +210,10 @@ # is thus expected to live only shortly, mostly for the case # of writing 'x.list[n]'. self.never_resize() - if self.listitem.mutated: - raise ListChangeUnallowed("list already mutated") - self.listitem.immutable = True + if not self.listitem.mutated: + self.listitem.immutable = True + #else: it's fine, don't set immutable=True at all (see + # test_can_merge_immutable_list_with_regular_list) MOST_GENERAL_LISTDEF = ListDef(None, SomeObject()) Modified: pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py ============================================================================== --- pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py (original) +++ pypy/branch/rlist-jit/pypy/annotation/test/test_annrpython.py Wed Nov 24 18:13:51 2010 @@ -3405,20 +3405,23 @@ def test_can_merge_immutable_list_with_regular_list(self): class A: _immutable_fields_ = 'lst[*]' + def foo(lst): + pass + def f(n): a = A() l1 = [n, 0] l1[1] = n+1 a.lst = l1 if n > 0: - lst = a.lst + foo(a.lst) else: - lst = [] - return lst + lst = [0] + lst[0] = n + foo(lst) a = self.RPythonAnnotator() - s = a.build_types(f, [int]) - assert not s.listdef.listitem.immutable + a.build_types(f, [int]) def f(n): a = A() @@ -3426,14 +3429,14 @@ l1[1] = n+1 a.lst = l1 if n > 0: - lst = [] + lst = [0] + lst[0] = n + foo(lst) else: - lst = a.lst - return lst + foo(a.lst) a = self.RPythonAnnotator() - s = a.build_types(f, [int]) - assert not s.listdef.listitem.immutable + a.build_types(f, [int]) def g(n): Modified: pypy/branch/rlist-jit/pypy/rpython/rlist.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rpython/rlist.py (original) +++ pypy/branch/rlist-jit/pypy/rpython/rlist.py Wed Nov 24 18:13:51 2010 @@ -769,7 +769,7 @@ l._ll_resize_le(newlength) ll_delitem_nonneg.oopspec = 'list.delitem(l, index)' -def ll_delitem(func, l, i): +def ll_delitem(func, l, index): if func is dum_checkidx: length = l.ll_length() if r_uint(index) >= r_uint(length): # see comments in ll_getitem(). @@ -781,7 +781,7 @@ if index < 0: index += l.ll_length() ll_assert(index >= 0, "negative list delitem index out of bound") - ll_delitem_nonneg(dum_nocheck, l, i) + ll_delitem_nonneg(dum_nocheck, l, index) # no oopspec -- the function is inlined by the JIT def ll_extend(l1, l2): From arigo at codespeak.net Wed Nov 24 18:39:47 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 18:39:47 +0100 (CET) Subject: [pypy-svn] r79483 - pypy/trunk/pypy/jit/backend/x86 Message-ID: <20101124173947.62D6F282B9D@codespeak.net> Author: arigo Date: Wed Nov 24 18:39:45 2010 New Revision: 79483 Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py pypy/trunk/pypy/jit/backend/x86/runner.py Log: Silence some warnings. Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/assembler.py (original) +++ pypy/trunk/pypy/jit/backend/x86/assembler.py Wed Nov 24 18:39:45 2010 @@ -187,6 +187,7 @@ self.setup_failure_recovery() self._debug = False self.debug_counter_descr = cpu.fielddescrof(DEBUG_COUNTER, 'i') + self.fail_boxes_count = 0 def leave_jitted_hook(self): ptrs = self.fail_boxes_ptr.ar Modified: pypy/trunk/pypy/jit/backend/x86/runner.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/runner.py (original) +++ pypy/trunk/pypy/jit/backend/x86/runner.py Wed Nov 24 18:39:45 2010 @@ -168,17 +168,14 @@ CPU = CPU386 # silence warnings - history.LoopToken._x86_param_depth = 0 history.LoopToken._x86_arglocs = (None, None) history.LoopToken._x86_frame_depth = 0 history.LoopToken._x86_bootstrap_code = 0 history.LoopToken._x86_direct_bootstrap_code = 0 -history.LoopToken._x86_failure_recovery_bytecode = 0 history.LoopToken._x86_loop_code = 0 -history.LoopToken._x86_current_depths = (0, 0) - -compile._DoneWithThisFrameDescr._x86_current_depths = (0, 0) -compile._DoneWithThisFrameDescr._x86_failure_recovery_bytecode = 0 -compile._DoneWithThisFrameDescr._x86_adr_jump_offset = 0 - +history.LoopToken._x86_debug_checksum = 0 +compile.AbstractFailDescr._x86_current_depths = (0, 0) +compile.AbstractFailDescr._x86_failure_recovery_bytecode = 0 +compile.AbstractFailDescr._x86_adr_jump_offset = 0 +compile.AbstractFailDescr._x86_adr_recovery_stub = 0 From arigo at codespeak.net Wed Nov 24 18:48:12 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 18:48:12 +0100 (CET) Subject: [pypy-svn] r79484 - pypy/trunk/pypy/jit/backend/x86 Message-ID: <20101124174812.43D8D282B9E@codespeak.net> Author: arigo Date: Wed Nov 24 18:48:10 2010 New Revision: 79484 Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py pypy/trunk/pypy/jit/backend/x86/regalloc.py Log: Caching of tons of identical 2-tuples. Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/assembler.py (original) +++ pypy/trunk/pypy/jit/backend/x86/assembler.py Wed Nov 24 18:48:10 2010 @@ -188,6 +188,7 @@ self._debug = False self.debug_counter_descr = cpu.fielddescrof(DEBUG_COUNTER, 'i') self.fail_boxes_count = 0 + self._current_depths_cache = (0, 0) def leave_jitted_hook(self): ptrs = self.fail_boxes_ptr.ar Modified: pypy/trunk/pypy/jit/backend/x86/regalloc.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/regalloc.py (original) +++ pypy/trunk/pypy/jit/backend/x86/regalloc.py Wed Nov 24 18:48:10 2010 @@ -320,11 +320,22 @@ def locs_for_fail(self, guard_op): return [self.loc(v) for v in guard_op.getfailargs()] + def get_current_depth(self): + # return (self.fm.frame_depth, self.param_depth), but trying to share + # the resulting tuple among several calls + arg0 = self.fm.frame_depth + arg1 = self.param_depth + result = self.assembler._current_depths_cache + if result[0] != arg0 or result[1] != arg1: + result = (arg0, arg1) + self.assembler._current_depths_cache = result + return result + def perform_with_guard(self, op, guard_op, arglocs, result_loc): faillocs = self.locs_for_fail(guard_op) self.rm.position += 1 self.xrm.position += 1 - current_depths = (self.fm.frame_depth, self.param_depth) + current_depths = self.get_current_depth() self.assembler.regalloc_perform_with_guard(op, guard_op, faillocs, arglocs, result_loc, current_depths) @@ -340,7 +351,7 @@ arglocs)) else: self.assembler.dump('%s(%s)' % (guard_op, arglocs)) - current_depths = (self.fm.frame_depth, self.param_depth) + current_depths = self.get_current_depth() self.assembler.regalloc_perform_guard(guard_op, faillocs, arglocs, result_loc, current_depths) From arigo at codespeak.net Wed Nov 24 18:53:14 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Wed, 24 Nov 2010 18:53:14 +0100 (CET) Subject: [pypy-svn] r79485 - pypy/branch/rlist-jit/pypy/module/pypyjit/test Message-ID: <20101124175314.85880282B9E@codespeak.net> Author: arigo Date: Wed Nov 24 18:53:13 2010 New Revision: 79485 Modified: pypy/branch/rlist-jit/pypy/module/pypyjit/test/test_pypy_c.py Log: Good, at least this test improves. Modified: pypy/branch/rlist-jit/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/rlist-jit/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/rlist-jit/pypy/module/pypyjit/test/test_pypy_c.py Wed Nov 24 18:53:13 2010 @@ -608,11 +608,9 @@ i = t2[3] del t2 return i - ''', 100, ([], 100)) + ''', 40, ([], 100)) bytecode, = self.get_by_bytecode('BINARY_SUBSCR') - assert len(bytecode.get_opnames('new_array')) == 1 - # XXX I would like here to say that it's 0, but unfortunately - # call that can raise is not exchanged into getarrayitem_gc + assert len(bytecode.get_opnames('new_array')) == 0 def test_overflow_checking(self): startvalue = sys.maxint - 2147483647 From dstromberg at codespeak.net Thu Nov 25 01:00:33 2010 From: dstromberg at codespeak.net (dstromberg at codespeak.net) Date: Thu, 25 Nov 2010 01:00:33 +0100 (CET) Subject: [pypy-svn] r79486 - pypy/branch/gdbm/attic Message-ID: <20101125000033.B1E9436C221@codespeak.net> Author: dstromberg Date: Thu Nov 25 01:00:30 2010 New Revision: 79486 Removed: pypy/branch/gdbm/attic/ Log: Removing, by consensus on pypy-dev From dan at codespeak.net Thu Nov 25 04:36:15 2010 From: dan at codespeak.net (dan at codespeak.net) Date: Thu, 25 Nov 2010 04:36:15 +0100 (CET) Subject: [pypy-svn] r79487 - pypy/branch/psycopg2compatibility/pypy/module/cpyext Message-ID: <20101125033615.490A55080B@codespeak.net> Author: dan Date: Thu Nov 25 04:36:11 2010 New Revision: 79487 Modified: pypy/branch/psycopg2compatibility/pypy/module/cpyext/api.py pypy/branch/psycopg2compatibility/pypy/module/cpyext/pystate.py Log: Passing tests again, should translate as well, testing that soon. Modified: pypy/branch/psycopg2compatibility/pypy/module/cpyext/api.py ============================================================================== --- pypy/branch/psycopg2compatibility/pypy/module/cpyext/api.py (original) +++ pypy/branch/psycopg2compatibility/pypy/module/cpyext/api.py Thu Nov 25 04:36:11 2010 @@ -37,10 +37,7 @@ DEBUG_WRAPPER = True -if not we_are_translated() and DEBUG_WRAPPER: - from functools import wraps -else: - def wraps(f): return lambda f: f +from functools import wraps from pypy.tool.call_logger import CallLogger, SkipArgument from pypy.rlib.objectmodel import specialize @@ -49,55 +46,46 @@ def __init__(self): self.indentation = 0 - @specialize.argtype(2) - def format_arg(self, space, arg, argtype): - from pypy.rpython.lltypesystem.rffi import charp2str, unicode_from_buffer - from pypy.rpython.lltypesystem.lltype import direct_arrayitems - from pypy.module.cpyext.pyobject import from_ref, InvalidPointerException - from pypy.objspace.std.model import UnwrapError - from pypy.module.cpyext.object import PyObject_Repr - - if isinstance(arg, ObjSpace): - raise SkipArgument() - - if is_PyObject(argtype): - return space.str_w(PyObject_Repr(space, arg)) - - return repr(arg) - def log(self, logstr, depth=0): debug_print((' ' * depth) + logstr) - def log_cpyext_call(self, f): - if not we_are_translated() and DEBUG_WRAPPER: + def log_cpyext_call(self, space, f): + if DEBUG_WRAPPER: + from pypy.module.cpyext.pyobject import make_ref, from_ref + from pypy.module.cpyext.object import PyObject_Repr + from pypy.module.cpyext.pyobject import Py_DecRef api_function = f.api_func - name = f.__name__ argtypes = api_function.argtypes result_type = api_function.restype - def format_args(space, args): - argstrs = [] - for i in unrolling_iterable(range(len(argtypes))): - try: - argtype = argtypes[i] - arg = args[i] - argstrs.append(self.format_arg(space, arg, argtype)) - except SkipArgument, e: continue - return ', '.join(argstrs) - + argwrapped = [name.startswith("w_") for name in api_function.argnames] + assert len(api_function.argtypes) == len(api_function.argnames) + argtypes_enum_ui = unrolling_iterable(enumerate(zip(api_function.argtypes, + [name.startswith("w_") for name in api_function.argnames]), start=1)) + name = f.__name__ # XXX: above line was overwriting name "name" @wraps(f) def wrapped(*args): - space = args[0] - argstr = format_args(space, args[1:]) + argstrs = [] + for i, (argtype, is_wrapped) in argtypes_enum_ui: + arg = args[i] + if is_PyObject(argtype) and is_wrapped: + if arg is not None: + pyrepr = PyObject_Repr(space, arg) + argstrs.append(space.str_w(pyrepr)) + else: + argstrs.append(repr(arg)) + else: + argstrs.append(repr(arg)) + argstr = ', '.join(argstrs) + self.log("%s(%s)" % (name, argstr), depth=self.indentation) self.indentation += 1 result = f(*args) self.indentation -= 1 - try: - resultformat = self.format_arg(result, space, result_type) - except SkipArgument: - resultformat = '' - self.log("%s(%s)->%s" % (name, argstr, resultformat), depth=self.indentation) + + result_format = repr(result) # TODO: format nicer! + self.log("%s(%s)->%s" % (name, argstr, result_format), depth=self.indentation) return result + return wrapped else: return f @@ -540,7 +528,7 @@ [name.startswith("w_") for name in names]))) fatal_value = callable.api_func.restype._defl() - logged_callable = log_call(callable) + logged_callable = log_call(space, callable) @specialize.ll() @wraps(callable) @@ -567,7 +555,10 @@ boxed_args += (arg_conv, ) state = space.fromcache(State) try: - result = logged_callable(space, *boxed_args) + if not we_are_translated() and DEBUG_WRAPPER: + result = logged_callable(space, *boxed_args) + else: + result = callable(space, *boxed_args) except OperationError, e: failed = True state.set_exception(e) Modified: pypy/branch/psycopg2compatibility/pypy/module/cpyext/pystate.py ============================================================================== --- pypy/branch/psycopg2compatibility/pypy/module/cpyext/pystate.py (original) +++ pypy/branch/psycopg2compatibility/pypy/module/cpyext/pystate.py Thu Nov 25 04:36:11 2010 @@ -2,7 +2,6 @@ cpython_struct from pypy.rpython.lltypesystem import rffi, lltype - PyInterpreterState = lltype.Ptr(cpython_struct("PyInterpreterState", ())) PyThreadState = lltype.Ptr(cpython_struct("PyThreadState", [('interp', PyInterpreterState)])) @@ -38,14 +37,21 @@ # XXX: might be generally useful def encapsulator(T, flavor='raw'): class MemoryCapsule(object): - def __init__(self): - self.memory = lltype.malloc(T, flavor=flavor) + def __init__(self, alloc=True): + if alloc: + self.memory = lltype.malloc(T, flavor=flavor) + else: + self.memory = lltype.nullptr(T) def __del__(self): - lltype.free(self.memory, flavor=flavor) + if self.memory: + lltype.free(self.memory, flavor=flavor) return MemoryCapsule ThreadStateCapsule = encapsulator(PyThreadState.TO) +from pypy.interpreter.executioncontext import ExecutionContext +ExecutionContext.cpyext_threadstate = ThreadStateCapsule(alloc=False) + class InterpreterState(object): def __init__(self, space): self.interpreter_state = lltype.malloc(PyInterpreterState.TO, flavor='raw', immortal=True) @@ -61,11 +67,7 @@ return self._get_thread_state(ec).memory def _get_thread_state(self, ec): - try: - ts = ec.cpyext_threadstate - if not ts: - ec.cpyext_threadstate = self.new_thread_state() - except AttributeError, e: + if ec.cpyext_threadstate.memory == lltype.nullptr(PyThreadState.TO): ec.cpyext_threadstate = self.new_thread_state() return ec.cpyext_threadstate From fijall at gmail.com Thu Nov 25 07:40:16 2010 From: fijall at gmail.com (Maciej Fijalkowski) Date: Thu, 25 Nov 2010 08:40:16 +0200 Subject: [pypy-svn] r79480 - pypy/trunk/pypy/jit/tool In-Reply-To: <20101124163904.C308C50810@codespeak.net> References: <20101124163904.C308C50810@codespeak.net> Message-ID: Is pypy repository really a place for such files? Maybe we should keep it somewhere else? On Wed, Nov 24, 2010 at 6:39 PM, wrote: > Author: antocuni > Date: Wed Nov 24 17:39:03 2010 > New Revision: 79480 > > Added: > ? pypy/trunk/pypy/jit/tool/cpython.vmrss > Log: > add the vmrss file produced by armin's memusage.py when running ./translate.py -Ojit at rev 79456 > > > > Added: pypy/trunk/pypy/jit/tool/cpython.vmrss > ============================================================================== > --- (empty file) > +++ pypy/trunk/pypy/jit/tool/cpython.vmrss ? ? ?Wed Nov 24 17:39:03 2010 > @@ -0,0 +1,4101 @@ > +124 > +16600 > +20620 > +20640 > +22036 > +94084 > +94084 > +94108 > +94108 > +94108 > +94108 > +94108 > +94120 > +94164 > +94160 > +94160 > +94160 > +94160 > +94160 > +94216 > +110644 > +123144 > +135236 > +143680 > +148500 > +153104 > +157088 > +160640 > +164760 > +167992 > +163108 > +163232 > +163232 > +163436 > +163436 > +163436 > +163444 > +163444 > +163444 > +163448 > +163448 > +163448 > +163448 > +163448 > +163448 > +167060 > +170948 > +174432 > +177212 > +177216 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176508 > +176520 > +176520 > +176520 > +176520 > +176520 > +176520 > +176520 > +176520 > +176520 > +176540 > +176544 > +176544 > +176544 > +176544 > +176544 > +176544 > +176544 > +176544 > +176544 > +176544 > +176544 > +176544 > +179120 > +187120 > +189380 > +191052 > +192156 > +193320 > +194900 > +195860 > +198516 > +201484 > +202600 > +202600 > +202600 > +202600 > +202832 > +202832 > +202836 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +202840 > +207784 > +212136 > +216320 > +220508 > +224696 > +228876 > +245088 > +245088 > +247844 > +252032 > +256212 > +260400 > +264592 > +268776 > +272776 > +275060 > +279244 > +283428 > +287616 > +291032 > +293900 > +298080 > +302272 > +304364 > +308548 > +310644 > +312740 > +316924 > +319016 > +323208 > +325296 > +327392 > +331572 > +333668 > +335760 > +337856 > +354328 > +356424 > +358520 > +362700 > +364792 > +366892 > +368984 > +371080 > +373168 > +375260 > +377356 > +379448 > +381540 > +383636 > +383636 > +385732 > +387820 > +390032 > +391160 > +392292 > +394552 > +396816 > +397092 > +399072 > +401340 > +403600 > +405860 > +408008 > +408148 > +412640 > +414900 > +417164 > +419420 > +421680 > +423944 > +426204 > +428464 > +430724 > +432768 > +434980 > +436476 > +437932 > +440332 > +441984 > +442740 > +445152 > +447688 > +449148 > +449960 > +452436 > +454712 > +454896 > +457180 > +458888 > +459688 > +462040 > +463480 > +464408 > +466812 > +467244 > +469224 > +471096 > +471684 > +474136 > +474328 > +476508 > +478872 > +481356 > +483472 > +483780 > +486072 > +488480 > +489668 > +490888 > +493420 > +495704 > +496836 > +498116 > +500520 > +502756 > +503668 > +505400 > +507760 > +509296 > +510204 > +512764 > +514708 > +515508 > +517372 > +519764 > +520648 > +522188 > +524596 > +525524 > +527004 > +529412 > +534224 > +536632 > +538080 > +539216 > +541588 > +542560 > +543384 > +543384 > +518804 > +518804 > +518804 > +518804 > +518804 > +518804 > +518804 > +518804 > +518804 > +518804 > +518804 > +518804 > +518804 > +518804 > +518804 > +518804 > +518804 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519024 > +519028 > +519028 > +519028 > +519028 > +519028 > +519028 > +519028 > +519028 > +519028 > +519028 > +519028 > +519028 > +519032 > +519032 > +519032 > +519032 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519036 > +519048 > +519048 > +519048 > +519048 > +519048 > +519048 > +519048 > +519048 > +519048 > +519048 > +519048 > +519048 > +519048 > +519048 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519052 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519056 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519060 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519064 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +519068 > +520592 > +520592 > +520904 > +522740 > +522740 > +523212 > +525256 > +525256 > +525316 > +526552 > +526552 > +526560 > +528508 > +528508 > +528508 > +530040 > +530040 > +530040 > +532684 > +532684 > +532684 > +534948 > +534948 > +534948 > +538028 > +538028 > +538028 > +541404 > +541448 > +541448 > +543784 > +543784 > +543784 > +545184 > +545476 > +545768 > +545832 > +545832 > +545832 > +546016 > +546016 > +546128 > +546184 > +546184 > +546184 > +546184 > +546184 > +546184 > +546184 > +546184 > +546184 > +546184 > +546184 > +546184 > +546184 > +546184 > +546184 > +546184 > +546184 > +546184 > +546228 > +546228 > +546228 > +546228 > +546228 > +546228 > +546228 > +546228 > +546228 > +546228 > +546228 > +546228 > +546228 > +546228 > +546228 > +546708 > +546708 > +546708 > +547988 > +550420 > +550420 > +550420 > +552896 > +555796 > +555796 > +555796 > +559136 > +560280 > +560280 > +560996 > +562504 > +563772 > +564672 > +564672 > +565268 > +567936 > +568884 > +569028 > +569236 > +569292 > +569292 > +570236 > +572960 > +573980 > +573980 > +574508 > +577404 > +579188 > +579188 > +579508 > +582836 > +584468 > +584468 > +585544 > +591292 > +591292 > +591292 > +595868 > +597588 > +597588 > +598404 > +602772 > +603964 > +603964 > +605488 > +609740 > +610468 > +610468 > +611884 > +616440 > +617108 > +617108 > +618156 > +622276 > +623784 > +623784 > +624128 > +624376 > +625544 > +626736 > +627112 > +627112 > +627116 > +627656 > +628836 > +628836 > +628836 > +629160 > +630180 > +630488 > +630488 > +630492 > +631288 > +632144 > +632144 > +632144 > +632172 > +632688 > +633220 > +633220 > +633220 > +633284 > +633756 > +633916 > +633916 > +633916 > +634012 > +634608 > +634608 > +634608 > +634624 > +634732 > +635144 > +635144 > +635144 > +635196 > +635680 > +635868 > +635868 > +635944 > +638440 > +639964 > +639980 > +639980 > +640056 > +641052 > +642064 > +642064 > +642064 > +642248 > +643080 > +643832 > +643832 > +643832 > +644116 > +646500 > +647424 > +648236 > +649032 > +649156 > +649156 > +649256 > +651556 > +652056 > +652504 > +652860 > +653168 > +653440 > +653876 > +654096 > +654304 > +654304 > +654304 > +654756 > +655648 > +657064 > +657064 > +657064 > +657064 > +657488 > +657756 > +657756 > +657756 > +658112 > +658736 > +658736 > +658736 > +658796 > +659304 > +659376 > +659376 > +659376 > +659848 > +661000 > +661172 > +661172 > +661236 > +662240 > +663240 > +663508 > +663508 > +663660 > +664324 > +665512 > +665764 > +665764 > +665764 > +666448 > +667692 > +668112 > +668112 > +668112 > +668624 > +669508 > +670388 > +670536 > +670536 > +670536 > +670564 > +671288 > +672268 > +672268 > +672268 > +672676 > +674504 > +675072 > +675072 > +675072 > +675156 > +675896 > +676700 > +676700 > +676700 > +676820 > +677988 > +678628 > +678628 > +678628 > +678776 > +679744 > +680400 > +680400 > +680400 > +705092 > +705156 > +705156 > +705156 > +705156 > +705156 > +705156 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705160 > +705264 > +705264 > +705264 > +705400 > +705476 > +706168 > +706292 > +706292 > +706292 > +706504 > +706568 > +706980 > +707012 > +707012 > +707012 > +707196 > +707280 > +707904 > +707904 > +707904 > +707924 > +708112 > +708176 > +708676 > +708676 > +708676 > +708696 > +708892 > +708984 > +709588 > +709588 > +709588 > +709612 > +709804 > +709848 > +710300 > +710300 > +710300 > +710676 > +710520 > +710604 > +711156 > +711156 > +711156 > +711336 > +711352 > +711576 > +712080 > +712080 > +712080 > +712300 > +712408 > +712500 > +712648 > +712648 > +712648 > +712648 > +713060 > +713300 > +713716 > +713716 > +713716 > +714072 > +714196 > +714568 > +714568 > +714568 > +714596 > +714956 > +715112 > +715808 > +715808 > +715808 > +717504 > +717628 > +717660 > +717660 > +717660 > +718620 > +719048 > +719424 > +719424 > +719424 > +719480 > +719924 > +720612 > +720612 > +720612 > +720620 > +722584 > +722848 > +722848 > +722848 > +723060 > +724108 > +724604 > +724604 > +724604 > +725108 > +726168 > +726348 > +726348 > +726348 > +727216 > +728204 > +728204 > +728204 > +728324 > +729396 > +730152 > +730152 > +730152 > +730396 > +736796 > +736800 > +736800 > +736800 > +736800 > +736800 > +736800 > +736800 > +736800 > +737136 > +738296 > +738400 > +738400 > +738400 > +739092 > +740128 > +740128 > +740128 > +740140 > +741092 > +741980 > +741980 > +741980 > +742060 > +743060 > +743796 > +743796 > +743796 > +743836 > +744440 > +745348 > +745348 > +745348 > +745400 > +746108 > +746848 > +746952 > +746952 > +746952 > +747496 > +748608 > +748744 > +749084 > +749084 > +749084 > +749172 > +750172 > +751468 > +751592 > +751592 > +751592 > +751688 > +751928 > +752152 > +752308 > +759152 > +760152 > +760152 > +760152 > +756356 > +754816 > +756356 > +756356 > +756356 > +756688 > +756688 > +756820 > +757152 > +757200 > +757400 > +757432 > +757432 > +757432 > +757632 > +757956 > +758404 > +758844 > +759480 > +760064 > +760552 > +760552 > +760552 > +760560 > +761180 > +761632 > +762288 > +762800 > +763700 > +764504 > +764716 > +764716 > +764716 > +764940 > +765388 > +765936 > +767748 > +767056 > +767300 > +767484 > +767868 > +768316 > +768316 > +768316 > +768316 > +768700 > +768828 > +768700 > +769340 > +769260 > +771008 > +771552 > +771652 > +771716 > +772580 > +772708 > +772740 > +772740 > +772740 > +772292 > +772740 > +772944 > +773188 > +773700 > +774084 > +774084 > +774404 > +774020 > +774532 > +774020 > +774596 > +774340 > +774468 > +774468 > +774468 > +774724 > +774792 > +774980 > +775368 > +775816 > +775816 > +776264 > +777480 > +778292 > +778408 > +778440 > +778440 > +778440 > +778440 > +778440 > +778440 > +778440 > +778440 > +778440 > +778440 > +778440 > +778632 > +778696 > +778952 > +779016 > +779016 > +779016 > +779016 > +780812 > +780812 > +780812 > +781068 > +781580 > +781772 > +781712 > +781868 > +782092 > +782420 > +782796 > +782796 > +782796 > +782796 > +782668 > +783128 > +783436 > +783820 > +784076 > +784332 > +784908 > +785164 > +785500 > +786188 > +786188 > +786188 > +786188 > +786188 > +786188 > +786188 > +786412 > +786896 > +787084 > +787404 > +787532 > +787724 > +787568 > +788108 > +788428 > +788748 > +788752 > +789520 > +789520 > +789520 > +788880 > +789440 > +789452 > +789516 > +790092 > +790284 > +790604 > +790860 > +791052 > +791372 > +791628 > +792012 > +792012 > +792396 > +792780 > +792780 > +792780 > +792780 > +792780 > +793228 > +793228 > +793868 > +793868 > +793996 > +793996 > +794636 > +794636 > +794636 > +795084 > +795084 > +795468 > +795532 > +795980 > +795980 > +796300 > +796364 > +796364 > +796364 > +796364 > +796748 > +797132 > +797132 > +797516 > +797644 > +797644 > +798380 > +798604 > +798860 > +798924 > +799372 > +799564 > +799756 > +799756 > +799756 > +799756 > +799816 > +800292 > +800792 > +801312 > +801816 > +802364 > +802880 > +803456 > +803660 > +803660 > +803660 > +803812 > +804388 > +804960 > +805516 > +806084 > +806668 > +807324 > +807980 > +807980 > +807980 > +807980 > +808416 > +809084 > +809784 > +810492 > +811160 > +812900 > +813476 > +813876 > +813876 > +813876 > +813876 > +814508 > +815152 > +815864 > +816556 > +817260 > +817916 > +818708 > +819272 > +819352 > +819352 > +819352 > +819352 > +819884 > +820308 > +820888 > +821012 > +821012 > +821204 > +821588 > +821588 > +821588 > +821972 > +822228 > +822164 > +822932 > +823252 > +823252 > +823252 > +823252 > +823252 > +823256 > +823612 > +823920 > +823936 > +824140 > +824168 > +824220 > +824280 > +824400 > +824664 > +825776 > +825776 > +825776 > +825776 > +832168 > +832168 > +832168 > +832168 > +832808 > +833492 > +833492 > +833492 > +833492 > +833700 > +834308 > +834428 > +834428 > +834428 > +834780 > +836216 > +836216 > +836216 > +836836 > +839308 > +839308 > +839308 > +839308 > +839416 > +840344 > +840344 > +840344 > +840344 > +841724 > +841724 > +841724 > +841724 > +843800 > +843800 > +843800 > +843800 > +845692 > +846072 > +846072 > +846072 > +846096 > +846736 > +847096 > +847156 > +847156 > +847156 > +847160 > +847180 > +847360 > +847360 > +847360 > +847360 > +847416 > +849248 > +849248 > +849248 > +849248 > +849716 > +849716 > +849716 > +849716 > +849740 > +851168 > +851168 > +851168 > +851168 > +854544 > +854544 > +854544 > +854544 > +854564 > +855332 > +855332 > +855332 > +855332 > +857092 > +857092 > +857092 > +857092 > +858000 > +859388 > +859388 > +859388 > +859388 > +861584 > +861584 > +861584 > +861584 > +861584 > +863464 > +863464 > +863464 > +863464 > +864304 > +866500 > +866500 > +866500 > +866500 > +868952 > +868952 > +868952 > +868952 > +869740 > +872108 > +872108 > +872108 > +872108 > +873208 > +875564 > +875564 > +875564 > +875564 > +878568 > +878568 > +878568 > +878568 > +878576 > +880780 > +880780 > +880780 > +880780 > +884048 > +884048 > +884048 > +884048 > +884048 > +884048 > +886516 > +886516 > +886516 > +886516 > +886516 > +886536 > +888112 > +888112 > +888112 > +888112 > +888112 > +888112 > +888656 > +888984 > +888984 > +888984 > +888984 > +888984 > +888984 > +889076 > +890288 > +890288 > +890288 > +890288 > +890288 > +890288 > +890404 > +892900 > +892900 > +892900 > +892900 > +892900 > +892900 > +892900 > +892900 > +895760 > +895760 > +895760 > +895760 > +895760 > +895760 > +895920 > +897624 > +897624 > +897624 > +897624 > +897624 > +897624 > +897624 > +897624 > +897628 > +898024 > +898024 > +898024 > +898024 > +898024 > +898060 > +900024 > +900024 > +900024 > +900024 > +900024 > +900024 > +900240 > +901436 > +901436 > +901436 > +901436 > +901436 > +901556 > +903116 > +903116 > +903116 > +903116 > +903116 > +903128 > +905084 > +905084 > +905084 > +905084 > +905084 > +905084 > +905096 > +906832 > +906832 > +906832 > +906832 > +906832 > +906832 > +908916 > +908916 > +908916 > +908916 > +908916 > +908916 > +908916 > +910720 > +910720 > +910720 > +910720 > +910720 > +910720 > +911780 > +912072 > +912072 > +912072 > +912072 > +914472 > +914472 > +914472 > +914472 > +914472 > +917120 > +917120 > +917120 > +917120 > +917120 > +919056 > +919056 > +919056 > +919056 > +919056 > +920316 > +920316 > +920316 > +920316 > +920316 > +920892 > +920892 > +920892 > +920892 > +920892 > +922996 > +922996 > +922996 > +922996 > +922996 > +925564 > +925564 > +925564 > +925564 > +925564 > +927780 > +927780 > +927780 > +927780 > +928016 > +928936 > +929048 > +930864 > +930864 > +930864 > +930864 > +932980 > +933304 > +933304 > +933304 > +933304 > +933540 > +934292 > +935452 > +935528 > +935528 > +935528 > +935528 > +935528 > +935528 > +935528 > +935528 > +935600 > +936112 > +936112 > +936208 > +936208 > +936208 > +936208 > +936308 > +936308 > +936316 > +936368 > +936368 > +936368 > +936368 > +936368 > +936368 > +936368 > +936368 > +936368 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +936376 > +937404 > +937404 > +937404 > +937404 > +939968 > +939968 > +939968 > +939968 > +939968 > +939968 > +939968 > +939968 > +939968 > +939968 > +939968 > +939968 > +939968 > +940516 > +940516 > +940516 > +940516 > +947168 > +947168 > +947168 > +947168 > +951948 > +951948 > +951948 > +951948 > +953488 > +956916 > +956916 > +956916 > +956916 > +952296 > +955376 > +953420 > +953260 > +953900 > +953516 > +953900 > +955052 > +953516 > +953516 > +954284 > +953324 > +953516 > +956208 > +956208 > +956208 > +956208 > +954668 > +948988 > +948988 > +948988 > +948988 > +948988 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +945908 > +947704 > +948068 > +948068 > +948068 > +948068 > +948068 > +948552 > +949024 > +949024 > +949024 > +949024 > +949024 > +949944 > +950492 > +950616 > +950616 > +950616 > +950616 > +951416 > +952000 > +952564 > +952564 > +952564 > +952564 > +952620 > +953296 > +953952 > +954332 > +954332 > +954332 > +954332 > +954532 > +955532 > +956216 > +956216 > +956216 > +956216 > +956216 > +956584 > +957396 > +957704 > +957704 > +957704 > +957704 > +957740 > +958620 > +959444 > +959444 > +959444 > +959444 > +959444 > +960224 > +963784 > +963868 > +963868 > +963868 > +963868 > +963872 > +964684 > +972452 > +972452 > +972452 > +972452 > +972452 > +972452 > +973220 > +974548 > +974548 > +974548 > +974548 > +974548 > +975540 > +977704 > +978792 > +978792 > +978792 > +978792 > +978792 > +981296 > +982700 > +983180 > +983180 > +983180 > +983180 > +983368 > +985060 > +987512 > +987688 > +987688 > +987688 > +987688 > +988180 > +990724 > +993084 > +993124 > +993124 > +993124 > +993124 > +993604 > +995444 > +996640 > +996640 > +996640 > +996640 > +996640 > +997892 > +999160 > +1001452 > +1001452 > +1001452 > +1001452 > +1001452 > +1002264 > +1004120 > +1004916 > +1004916 > +1004916 > +1004916 > +1004916 > +1006648 > +1010244 > +1010548 > +1010548 > +1010548 > +1010548 > +1010572 > +1011860 > +1013748 > +1017264 > +1017264 > +1017264 > +1017264 > +1017264 > +1019096 > +1021044 > +1021044 > +1021044 > +1021044 > +1021044 > +1021536 > +1023636 > +1024964 > +1024964 > +1024964 > +1024964 > +1024964 > +1025592 > +1027672 > +1029384 > +1029384 > +1029384 > +1029384 > +1029384 > +1030056 > +1032244 > +1033756 > +1033756 > +1033756 > +1033756 > +1033756 > +1034384 > +1035856 > +1036588 > +1038428 > +1038428 > +1038428 > +1038428 > +1038428 > +1039288 > +1041088 > +1042508 > +1042508 > +1042508 > +1042508 > +1042508 > +1043544 > +1045280 > +1046580 > +1046580 > +1046580 > +1046580 > +1046580 > +1047040 > +1048560 > +1049872 > +1050576 > +1050576 > +1050576 > +1050576 > +1050576 > +1052016 > +1056044 > +1057360 > +1057360 > +1057360 > +1057360 > +1057360 > +1058336 > +1059900 > +1061244 > +1061704 > +1061704 > +1061704 > +1061704 > +1061704 > +1063148 > +1063972 > +1065404 > +1067064 > +1067536 > +1067536 > +1067536 > +1067536 > +1067536 > +1069284 > +1070524 > +1072340 > +1072836 > +1072836 > +1072836 > +1072836 > +1072836 > +1073584 > +1074592 > +1076404 > +1076832 > +1076832 > +1076832 > +1076832 > +1076832 > +1078124 > +1079640 > +1080220 > +1080644 > +1080736 > +1080736 > +1080736 > +1080736 > +1080924 > +1082868 > +1083368 > +1084412 > +1084412 > +1084412 > +1084412 > +1084412 > +1085216 > +1087068 > +1087960 > +1087960 > +1087960 > +1087960 > +1087960 > +1089788 > +1093132 > +1095064 > +1095064 > +1095064 > +1095064 > +1095064 > +1095140 > +1097092 > +1098948 > +1098948 > +1098948 > +1098948 > +1098948 > +1098980 > +1100812 > +1102032 > +1102032 > +1102032 > +1102032 > +1102032 > +1105464 > +1116944 > +1119248 > +1120364 > +1120364 > +1120364 > +1120364 > +1120364 > +1121568 > +1122908 > +1123680 > +1124604 > +1125076 > +1125076 > +1125076 > +1125076 > +1125076 > +1126292 > +1128160 > +1129952 > +1129952 > +1129952 > +1129952 > +1129952 > +1130496 > +1131884 > +1133032 > +1134204 > +1135460 > +1135636 > +1135636 > +1135636 > +1135636 > +1135636 > +1136764 > +1138048 > +1139412 > +1140764 > +1141164 > +1141188 > +1142440 > +1142440 > +1142440 > +1142440 > +1142440 > +1142440 > +1143980 > +1145876 > +1146576 > +1146576 > +1146576 > +1146576 > +1146576 > +1146576 > +1147680 > +1148328 > +1148960 > +1148960 > +1148960 > +1148960 > +1148960 > +1149004 > +1150700 > +1152228 > +1153364 > +1153364 > +1153520 > +1153784 > +1154588 > +1154680 > +1154712 > +1154728 > +1154784 > +1154992 > +1155356 > +1155620 > +1155856 > +1156044 > +1156420 > +1157392 > +1158760 > +1158980 > +1158988 > +1159000 > +1162724 > +1162740 > +1162788 > +1163112 > +1163188 > +1163188 > +1163188 > +1163188 > +1163188 > +1163384 > +1165668 > +1166648 > +1166652 > +1166664 > +1166676 > +1166688 > +1166692 > +1166696 > +1166700 > +1166700 > +1172848 > +1172852 > +1174888 > +1176824 > +1176836 > +1176852 > +1176860 > +1176876 > +1176880 > +1176888 > +1176892 > +1176900 > +1176912 > +1176944 > +1177248 > +1177712 > +1178172 > +1178536 > +1178656 > +1178780 > +1178920 > +1179044 > +1179188 > +1179384 > +1180296 > +1180300 > +1180300 > +1180300 > +1180300 > +1180300 > +1180300 > +1180372 > +1180380 > +1180468 > +1180524 > +1180524 > +1180524 > +1180524 > +1180524 > +1180576 > +1180580 > +1180644 > +1180684 > +1180684 > +1180684 > +1180684 > +1180684 > +1180684 > +1180724 > +1180756 > +1180852 > +1180904 > +1180904 > +1180904 > +1180904 > +1180904 > +1180904 > +1181096 > +1181400 > +1181744 > +1181744 > +1181744 > +1181744 > +1181744 > +1181744 > +1181936 > +1181936 > +1181936 > +1181936 > +1181936 > +1181936 > +1181936 > +1181936 > +1181936 > +1181972 > +1182004 > +1182004 > +1182004 > +1182004 > +1182004 > +1182004 > +1182004 > +1182004 > +1182004 > +1182128 > +1182156 > +1182156 > +1182156 > +1182156 > +1182156 > +1182156 > +1182156 > +1182156 > +1182180 > +1182180 > +1182180 > +1182180 > +1182180 > +1182180 > +1182180 > +1182368 > +1182516 > +1182516 > +1182516 > +1182516 > +1182516 > +1182516 > +1182516 > +1182516 > +1182516 > +1182516 > +1182516 > +1182516 > +1182516 > +1182516 > +1182516 > +1182516 > +1182516 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183576 > +1183596 > +1183772 > +1183772 > +1183772 > +1183772 > +1183772 > +1183772 > +1183772 > +1183772 > +1183772 > +1183776 > +1183776 > +1183776 > +1183776 > +1183776 > +1183776 > +1183776 > +1183776 > +1183776 > +1183952 > +1184000 > +1184000 > +1184000 > +1184000 > +1184000 > +1184000 > +1184000 > +1184200 > +1184832 > +1184832 > +1184832 > +1184832 > +1184832 > +1184832 > +1184928 > +1184928 > +1184940 > +1184940 > +1184940 > +1184940 > +1184940 > +1184940 > +1184940 > +1184940 > +1185144 > +1185144 > +1185144 > +1185144 > +1185144 > +1185144 > +1185144 > +1185144 > +1185144 > +1185144 > +1185144 > +1185144 > +1185144 > +1185144 > +1185196 > +1185196 > +1185196 > +1185196 > +1185196 > +1185196 > +1185196 > +1185196 > +1185196 > +1185268 > +1185268 > +1185268 > +1185268 > +1185268 > +1185268 > +1185268 > +1185268 > +1186444 > +1186776 > +1186776 > +1186776 > +1186776 > +1186776 > +1187664 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188072 > +1188168 > +1188168 > +1188168 > +1188168 > +1188168 > +1188168 > +1188168 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189240 > +1189292 > +1189292 > +1189292 > +1189292 > +1189292 > +1189292 > +1189292 > +1189532 > +1189532 > +1189532 > +1189532 > +1189532 > +1189532 > +1189532 > +1189532 > +1189532 > +1189532 > +1189532 > +1189532 > +1189532 > +1189532 > +1189532 > +1189532 > +1189704 > +1189708 > +1189708 > +1189708 > +1189708 > +1189708 > +1190160 > +1190160 > +1190160 > +1190160 > +1190160 > +1190160 > +1190160 > +1190160 > +1190160 > +1191100 > +1191100 > +1191100 > +1191100 > +1191100 > +1191316 > +1191316 > +1191316 > +1191316 > +1191316 > +1191316 > +1191316 > +1191316 > +1191316 > +1191316 > +1191316 > +1191316 > +1191316 > +1191316 > +1191316 > +1191316 > +1191316 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191748 > +1191772 > +1191772 > +1191772 > +1191772 > +1191772 > +1191772 > +1191772 > +1191772 > +1192964 > +1192964 > +1192964 > +1192964 > +1192964 > +1192964 > +1192964 > +1193060 > +1193060 > +1193060 > +1193060 > +1193060 > +1193060 > +1193060 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193072 > +1193076 > +1193076 > +1193076 > +1193076 > +1193076 > +1193076 > +1193124 > +1193124 > +1193360 > +1194108 > +1194108 > +1194108 > +1194108 > +1194108 > +1193380 > +1193460 > +1193460 > +1193460 > +1193460 > +1193460 > +1193460 > +1193460 > +1193460 > +1193792 > +1193792 > +1193792 > +1193792 > +1193792 > +1193792 > +1194000 > +1194000 > +1194000 > +1194000 > +1194000 > +1194000 > +1194000 > +1194000 > +1194000 > +1194000 > +1194000 > +1194000 > +1194000 > +1194000 > +1194000 > +1194000 > +1194000 > +1194048 > +1194048 > +1194048 > +1194048 > +1194048 > +1194048 > +1194048 > +1194048 > +1194048 > +1194048 > +1194048 > +1194048 > +1194048 > +1194048 > +1194048 > +1194048 > +1194328 > +1194328 > +1194328 > +1194328 > +1194328 > +1194328 > +1194328 > +1194328 > +1194328 > +1194328 > +1194360 > +1194360 > +1194360 > +1194360 > +1194360 > +1194360 > +1194360 > +1194508 > +1194508 > +1194508 > +1194508 > +1194508 > +1194508 > +1194512 > +1194668 > +1194668 > +1194668 > +1194668 > +1194668 > +1194668 > +1194668 > +1194668 > +1194668 > +1194668 > +1194668 > +1194668 > +1194668 > +1194668 > +1194668 > +1194668 > +1194912 > +1194912 > +1194912 > +1194912 > +1194912 > +1194912 > +1194912 > +1194912 > +1194912 > +1194912 > +1194912 > +1194912 > +1194912 > +1194912 > +1194912 > +1194912 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196596 > +1196660 > +1196764 > +1196764 > +1196764 > +1196764 > +1196764 > +1196764 > +1196764 > +1196764 > +1196764 > +1196764 > +1196764 > +1196764 > +1196764 > +1196764 > +1196764 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1196948 > +1197236 > +1197236 > +1197236 > +1197236 > +1197236 > +1197236 > +1197236 > +1197236 > +1197236 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197288 > +1197376 > +1197384 > +1197596 > +1197596 > +1197596 > +1197596 > +1197596 > +1197596 > +1197596 > +1197596 > +1197588 > +1197588 > +1197588 > +1197588 > +1197588 > +1197588 > +1197588 > +1197588 > +1197564 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197460 > +1197492 > +1197508 > +1197516 > +1197516 > +1197516 > +1197516 > +1197516 > +1197516 > _______________________________________________ > pypy-svn mailing list > pypy-svn at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-svn > From arigo at codespeak.net Thu Nov 25 09:55:18 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 09:55:18 +0100 (CET) Subject: [pypy-svn] r79488 - pypy/trunk/pypy/translator/c/src Message-ID: <20101125085518.2170D5080C@codespeak.net> Author: arigo Date: Thu Nov 25 09:55:16 2010 New Revision: 79488 Modified: pypy/trunk/pypy/translator/c/src/mem.h Log: Fix for Boehm. Modified: pypy/trunk/pypy/translator/c/src/mem.h ============================================================================== --- pypy/trunk/pypy/translator/c/src/mem.h (original) +++ pypy/trunk/pypy/translator/c/src/mem.h Thu Nov 25 09:55:16 2010 @@ -233,4 +233,4 @@ #define OP_GC_GET_RPY_MEMORY_USAGE(x, r) r = -1 #define OP_GC_GET_RPY_TYPE_INDEX(x, r) r = -1 #define OP_GC_IS_RPY_INSTANCE(x, r) r = 0 -#define OP_GC_DUMP_RPY_HEAP(r) r = 0 +#define OP_GC_DUMP_RPY_HEAP(fd, r) r = 0 From arigo at codespeak.net Thu Nov 25 09:57:25 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 09:57:25 +0100 (CET) Subject: [pypy-svn] r79490 - in pypy/trunk/pypy/jit/backend/llsupport: . test Message-ID: <20101125085725.32EEC282B90@codespeak.net> Author: arigo Date: Thu Nov 25 09:57:23 2010 New Revision: 79490 Added: pypy/trunk/pypy/jit/backend/llsupport/codebuf.py pypy/trunk/pypy/jit/backend/llsupport/test/test_codebuf.py Log: Initial work. Added: pypy/trunk/pypy/jit/backend/llsupport/codebuf.py ============================================================================== --- (empty file) +++ pypy/trunk/pypy/jit/backend/llsupport/codebuf.py Thu Nov 25 09:57:23 2010 @@ -0,0 +1,132 @@ +import sys +from pypy.rlib.rarithmetic import intmask, r_uint +from pypy.rlib.objectmodel import we_are_translated +from pypy.rlib import rmmap +from pypy.rpython.lltypesystem import lltype, llmemory, rffi + + +class AsmMemoryManager(object): + LARGE_ALLOC_SIZE = 1024 * 1024 # 1MB + MIN_FRAGMENT = 64 + NUM_INDICES = 32 # good for all sizes between 64 bytes and ~490 KB + + def __init__(self, large_alloc_size = LARGE_ALLOC_SIZE, + min_fragment = MIN_FRAGMENT, + num_indices = NUM_INDICES): + self.total_memory_allocated = r_uint(0) + self.large_alloc_size = large_alloc_size + self.min_fragment = min_fragment + self.num_indices = num_indices + self.free_blocks = {} # map {start: stop} + self.free_blocks_end = {} # map {stop: start} + self.blocks_by_size = [[] for i in range(self.num_indices)] + self._allocated = [] + + def malloc(self, minsize, maxsize): + """Allocate executable memory, between minsize and maxsize bytes, + and return a pair (start, stop). Does not perform any rounding + of minsize and maxsize. + """ + result = self._allocate_block(minsize) + (start, stop) = result + smaller_stop = start + maxsize + if smaller_stop + self.min_fragment <= stop: + result = (start, smaller_stop) + self._add_free_block(smaller_stop, stop) + return result # pair (start, stop) + + def free(self, start, stop): + """Free a block (start, stop) returned by a previous malloc().""" + self._add_free_block(start, stop) + + def _allocate_large_block(self, minsize): + # Compute 'size' from 'minsize': it must be rounded up to + # 'large_alloc_size'. Additionally, we use the following line + # to limit how many mmap() requests the OS will see in total: + minsize = max(minsize, intmask(self.total_memory_allocated >> 4)) + size = minsize + self.large_alloc_size - 1 + size = (size // self.large_alloc_size) * self.large_alloc_size + data = rmmap.alloc(size) + if not we_are_translated(): + self._allocated.append((data, size)) + if sys.maxint > 2147483647: + # Hack to make sure that mcs are not within 32-bits of one + # another for testing purposes + rmmap.hint.pos += 0x80000000 - size + self.total_memory_allocated += size + data = rffi.cast(lltype.Signed, data) + return self._add_free_block(data, data + size) + + def _get_index(self, length): + i = 0 + while length > self.min_fragment: + length = (length * 3) >> 2 + i += 1 + if i == self.num_indices - 1: + break + return i + + def _add_free_block(self, start, stop): + # Merge with the block on the left + if start in self.free_blocks_end: + left_start = self.free_blocks_end[start] + self._del_free_block(left_start, start) + start = left_start + # Merge with the block on the right + if stop in self.free_blocks: + right_stop = self.free_blocks[stop] + self._del_free_block(stop, right_stop) + stop = right_stop + # Add it to the dicts + self.free_blocks[start] = stop + self.free_blocks_end[stop] = start + i = self._get_index(stop - start) + self.blocks_by_size[i].append(start) + return start + + def _del_free_block(self, start, stop): + del self.free_blocks[start] + del self.free_blocks_end[stop] + i = self._get_index(stop - start) + self.blocks_by_size[i].remove(start) + + def _allocate_block(self, length): + # First look in the group of index i0 if there is a block that is + # big enough. Following an idea found in the Linux malloc.c, we + # prefer the oldest entries rather than the newest one, to let + # them have enough time to coalesce into bigger blocks. It makes + # a big difference on the purely random test (30% of total usage). + i0 = self._get_index(length) + bbs = self.blocks_by_size[i0] + for j in range(len(bbs)): + start = bbs[j] + stop = self.free_blocks[start] + if start + length <= stop: + del bbs[j] + break # found a block big enough + else: + # Then look in the larger groups + i = i0 + 1 + while i < self.num_indices: + if len(self.blocks_by_size[i]) > 0: + # any block found in a larger group is big enough + start = self.blocks_by_size[i].pop(0) + stop = self.free_blocks[start] + break + i += 1 + else: + # Exhausted the memory. Allocate the resulting block. + start = self._allocate_large_block(length) + stop = self.free_blocks[start] + i = self._get_index(stop - start) + assert self.blocks_by_size[i][-1] == start + self.blocks_by_size[i].pop() + # + del self.free_blocks[start] + del self.free_blocks_end[stop] + return (start, stop) + + def _delete(self): + "NOT_RPYTHON" + for data, size in self._allocated: + rmmap.free(data, size) Added: pypy/trunk/pypy/jit/backend/llsupport/test/test_codebuf.py ============================================================================== --- (empty file) +++ pypy/trunk/pypy/jit/backend/llsupport/test/test_codebuf.py Thu Nov 25 09:57:23 2010 @@ -0,0 +1,149 @@ +import random +from pypy.jit.backend.llsupport.codebuf import AsmMemoryManager + + +def test_get_index(): + memmgr = AsmMemoryManager(min_fragment=8, + num_indices=5) + index = 0 + for length in range(100): + assert memmgr._get_index(length) == index + if length in [8, 11, 15, 21]: + index += 1 + +def test_get_index_default_values(): + memmgr = AsmMemoryManager() + jumps = [64, 86, 115, 154, 206, 275, 367, 490, 654, 873, 1165, + 1554, 2073, 2765, 3687, 4917, 6557, 8743, 11658, 15545, + 20727, 27637, 36850, 49134, 65513, 87351, 116469, 155293, + 207058, 276078, 368105] + for i, jump in enumerate(jumps): + assert memmgr._get_index(jump) == i + assert memmgr._get_index(jump + 1) == i + 1 + +def test_add_free_block(): + memmgr = AsmMemoryManager(min_fragment=8, + num_indices=5) + memmgr._add_free_block(10, 18) + assert memmgr.free_blocks == {10: 18} + assert memmgr.free_blocks_end == {18: 10} + assert memmgr.blocks_by_size == [[10], [], [], [], []] + memmgr._add_free_block(20, 30) + assert memmgr.free_blocks == {10: 18, 20: 30} + assert memmgr.free_blocks_end == {18: 10, 30: 20} + assert memmgr.blocks_by_size == [[10], [20], [], [], []] + memmgr._add_free_block(18, 20) # merge both left and right + assert memmgr.free_blocks == {10: 30} + assert memmgr.free_blocks_end == {30: 10} + assert memmgr.blocks_by_size == [[], [], [], [10], []] + +def test_allocate_block(): + memmgr = AsmMemoryManager(min_fragment=8, + num_indices=5) + memmgr._add_free_block(10, 18) + memmgr._add_free_block(20, 30) + (start, stop) = memmgr._allocate_block(4) + assert (start, stop) == (10, 18) + assert memmgr.free_blocks == {20: 30} + assert memmgr.free_blocks_end == {30: 20} + assert memmgr.blocks_by_size == [[], [20], [], [], []] + (start, stop) = memmgr._allocate_block(4) + assert (start, stop) == (20, 30) + assert memmgr.free_blocks == {} + assert memmgr.free_blocks_end == {} + assert memmgr.blocks_by_size == [[], [], [], [], []] + +def test_malloc_without_fragment(): + memmgr = AsmMemoryManager(min_fragment=8, + num_indices=5) + memmgr._add_free_block(10, 18) + memmgr._add_free_block(20, 30) + for minsize in range(1, 11): + for maxsize in range(minsize, 14): + (start, stop) = memmgr.malloc(minsize, maxsize) + if minsize <= 8: + assert (start, stop) == (10, 18) + else: + assert (start, stop) == (20, 30) + memmgr._add_free_block(start, stop) + memmgr._add_free_block(40, 49) + (start, stop) = memmgr.malloc(10, 10) + assert (start, stop) == (20, 30) + +def test_malloc_with_fragment(): + for reqsize in range(1, 33): + memmgr = AsmMemoryManager(min_fragment=8, + num_indices=5) + memmgr._add_free_block(12, 44) + (start, stop) = memmgr.malloc(reqsize, reqsize) + if reqsize + 8 <= 32: + assert (start, stop) == (12, 12 + reqsize) + assert memmgr.free_blocks == {stop: 44} + assert memmgr.free_blocks_end == {44: stop} + assert [stop] in memmgr.blocks_by_size + else: + assert (start, stop) == (12, 44) + assert memmgr.free_blocks == {} + assert memmgr.free_blocks_end == {} + assert memmgr.blocks_by_size == [[], [], [], [], []] + + +class TestAsmMemoryManager: + + def setup_method(self, _): + self.memmgr = AsmMemoryManager(min_fragment=8, + num_indices=10, + large_alloc_size=8192) + + def teardown_method(self, _): + self.memmgr._delete() + + def test_malloc_simple(self): + for i in range(100): + while self.memmgr.total_memory_allocated < 16384: + reqsize = random.randrange(1, 200) + (start, stop) = self.memmgr.malloc(reqsize, reqsize) + assert reqsize <= stop - start < reqsize + 8 + assert self.memmgr.total_memory_allocated in [8192, 16384] + self.teardown_method(None) + self.setup_method(None) + + def test_random(self): + got = [] + real_use = 0 + prev_total = 0 + iterations_without_allocating_more = 0 + while True: + # + if got and (random.random() < 0.4 or len(got) == 1000): + # free + start, stop = got.pop(random.randrange(0, len(got))) + self.memmgr.free(start, stop) + real_use -= (stop - start) + assert real_use >= 0 + # + else: + # allocate + reqsize = random.randrange(1, 200) + if random.random() < 0.5: + reqmaxsize = reqsize + else: + reqmaxsize = reqsize + random.randrange(0, 200) + (start, stop) = self.memmgr.malloc(reqsize, reqmaxsize) + assert reqsize <= stop - start < reqmaxsize + 8 + for otherstart, otherstop in got: # no overlap + assert otherstop <= start or stop <= otherstart + got.append((start, stop)) + real_use += (stop - start) + if self.memmgr.total_memory_allocated == prev_total: + iterations_without_allocating_more += 1 + if iterations_without_allocating_more == 40000: + break # ok + else: + new_total = self.memmgr.total_memory_allocated + iterations_without_allocating_more = 0 + print real_use, new_total + # We seem to never see a printed value greater + # than 131072. Be reasonable and allow up to 147456. + assert new_total <= 147456 + prev_total = new_total From fijal at codespeak.net Thu Nov 25 10:03:28 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Thu, 25 Nov 2010 10:03:28 +0100 (CET) Subject: [pypy-svn] r79491 - pypy/release/1.4.x Message-ID: <20101125090328.416E9282B90@codespeak.net> Author: fijal Date: Thu Nov 25 10:03:26 2010 New Revision: 79491 Removed: pypy/release/1.4.x/ Log: Remove release branch (will be recreated from trunk) From fijal at codespeak.net Thu Nov 25 10:04:08 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Thu, 25 Nov 2010 10:04:08 +0100 (CET) Subject: [pypy-svn] r79492 - pypy/release/1.4.x Message-ID: <20101125090408.8AC02282B90@codespeak.net> Author: fijal Date: Thu Nov 25 10:04:07 2010 New Revision: 79492 Added: pypy/release/1.4.x/ (props changed) - copied from r79491, pypy/trunk/ Log: Create a release branch (again) From arigo at codespeak.net Thu Nov 25 10:15:42 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 10:15:42 +0100 (CET) Subject: [pypy-svn] r79493 - in pypy/trunk/pypy/jit/backend/llsupport: . test Message-ID: <20101125091542.5301D282B9D@codespeak.net> Author: arigo Date: Thu Nov 25 10:15:40 2010 New Revision: 79493 Removed: pypy/trunk/pypy/jit/backend/llsupport/codebuf.py pypy/trunk/pypy/jit/backend/llsupport/test/test_codebuf.py Log: Bah. This goes to the jit-free-asm branch. From arigo at codespeak.net Thu Nov 25 10:21:10 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 10:21:10 +0100 (CET) Subject: [pypy-svn] r79495 - pypy/branch/jit-free-asm Message-ID: <20101125092110.8E664282B9D@codespeak.net> Author: arigo Date: Thu Nov 25 10:21:09 2010 New Revision: 79495 Added: pypy/branch/jit-free-asm/ - copied from r79494, pypy/trunk/ Log: A branch in which to really finish freeing old loops -- which means, also freeing the assembler. (Create the branch again.) From arigo at codespeak.net Thu Nov 25 10:21:38 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 10:21:38 +0100 (CET) Subject: [pypy-svn] r79496 - in pypy/branch/jit-free-asm/pypy/jit/backend/llsupport: . test Message-ID: <20101125092138.5F918282B9E@codespeak.net> Author: arigo Date: Thu Nov 25 10:21:36 2010 New Revision: 79496 Added: pypy/branch/jit-free-asm/pypy/jit/backend/llsupport/codebuf.py pypy/branch/jit-free-asm/pypy/jit/backend/llsupport/test/test_codebuf.py Log: Initial work. Added: pypy/branch/jit-free-asm/pypy/jit/backend/llsupport/codebuf.py ============================================================================== --- (empty file) +++ pypy/branch/jit-free-asm/pypy/jit/backend/llsupport/codebuf.py Thu Nov 25 10:21:36 2010 @@ -0,0 +1,132 @@ +import sys +from pypy.rlib.rarithmetic import intmask, r_uint +from pypy.rlib.objectmodel import we_are_translated +from pypy.rlib import rmmap +from pypy.rpython.lltypesystem import lltype, llmemory, rffi + + +class AsmMemoryManager(object): + LARGE_ALLOC_SIZE = 1024 * 1024 # 1MB + MIN_FRAGMENT = 64 + NUM_INDICES = 32 # good for all sizes between 64 bytes and ~490 KB + + def __init__(self, large_alloc_size = LARGE_ALLOC_SIZE, + min_fragment = MIN_FRAGMENT, + num_indices = NUM_INDICES): + self.total_memory_allocated = r_uint(0) + self.large_alloc_size = large_alloc_size + self.min_fragment = min_fragment + self.num_indices = num_indices + self.free_blocks = {} # map {start: stop} + self.free_blocks_end = {} # map {stop: start} + self.blocks_by_size = [[] for i in range(self.num_indices)] + self._allocated = [] + + def malloc(self, minsize, maxsize): + """Allocate executable memory, between minsize and maxsize bytes, + and return a pair (start, stop). Does not perform any rounding + of minsize and maxsize. + """ + result = self._allocate_block(minsize) + (start, stop) = result + smaller_stop = start + maxsize + if smaller_stop + self.min_fragment <= stop: + result = (start, smaller_stop) + self._add_free_block(smaller_stop, stop) + return result # pair (start, stop) + + def free(self, start, stop): + """Free a block (start, stop) returned by a previous malloc().""" + self._add_free_block(start, stop) + + def _allocate_large_block(self, minsize): + # Compute 'size' from 'minsize': it must be rounded up to + # 'large_alloc_size'. Additionally, we use the following line + # to limit how many mmap() requests the OS will see in total: + minsize = max(minsize, intmask(self.total_memory_allocated >> 4)) + size = minsize + self.large_alloc_size - 1 + size = (size // self.large_alloc_size) * self.large_alloc_size + data = rmmap.alloc(size) + if not we_are_translated(): + self._allocated.append((data, size)) + if sys.maxint > 2147483647: + # Hack to make sure that mcs are not within 32-bits of one + # another for testing purposes + rmmap.hint.pos += 0x80000000 - size + self.total_memory_allocated += size + data = rffi.cast(lltype.Signed, data) + return self._add_free_block(data, data + size) + + def _get_index(self, length): + i = 0 + while length > self.min_fragment: + length = (length * 3) >> 2 + i += 1 + if i == self.num_indices - 1: + break + return i + + def _add_free_block(self, start, stop): + # Merge with the block on the left + if start in self.free_blocks_end: + left_start = self.free_blocks_end[start] + self._del_free_block(left_start, start) + start = left_start + # Merge with the block on the right + if stop in self.free_blocks: + right_stop = self.free_blocks[stop] + self._del_free_block(stop, right_stop) + stop = right_stop + # Add it to the dicts + self.free_blocks[start] = stop + self.free_blocks_end[stop] = start + i = self._get_index(stop - start) + self.blocks_by_size[i].append(start) + return start + + def _del_free_block(self, start, stop): + del self.free_blocks[start] + del self.free_blocks_end[stop] + i = self._get_index(stop - start) + self.blocks_by_size[i].remove(start) + + def _allocate_block(self, length): + # First look in the group of index i0 if there is a block that is + # big enough. Following an idea found in the Linux malloc.c, we + # prefer the oldest entries rather than the newest one, to let + # them have enough time to coalesce into bigger blocks. It makes + # a big difference on the purely random test (30% of total usage). + i0 = self._get_index(length) + bbs = self.blocks_by_size[i0] + for j in range(len(bbs)): + start = bbs[j] + stop = self.free_blocks[start] + if start + length <= stop: + del bbs[j] + break # found a block big enough + else: + # Then look in the larger groups + i = i0 + 1 + while i < self.num_indices: + if len(self.blocks_by_size[i]) > 0: + # any block found in a larger group is big enough + start = self.blocks_by_size[i].pop(0) + stop = self.free_blocks[start] + break + i += 1 + else: + # Exhausted the memory. Allocate the resulting block. + start = self._allocate_large_block(length) + stop = self.free_blocks[start] + i = self._get_index(stop - start) + assert self.blocks_by_size[i][-1] == start + self.blocks_by_size[i].pop() + # + del self.free_blocks[start] + del self.free_blocks_end[stop] + return (start, stop) + + def _delete(self): + "NOT_RPYTHON" + for data, size in self._allocated: + rmmap.free(data, size) Added: pypy/branch/jit-free-asm/pypy/jit/backend/llsupport/test/test_codebuf.py ============================================================================== --- (empty file) +++ pypy/branch/jit-free-asm/pypy/jit/backend/llsupport/test/test_codebuf.py Thu Nov 25 10:21:36 2010 @@ -0,0 +1,149 @@ +import random +from pypy.jit.backend.llsupport.codebuf import AsmMemoryManager + + +def test_get_index(): + memmgr = AsmMemoryManager(min_fragment=8, + num_indices=5) + index = 0 + for length in range(100): + assert memmgr._get_index(length) == index + if length in [8, 11, 15, 21]: + index += 1 + +def test_get_index_default_values(): + memmgr = AsmMemoryManager() + jumps = [64, 86, 115, 154, 206, 275, 367, 490, 654, 873, 1165, + 1554, 2073, 2765, 3687, 4917, 6557, 8743, 11658, 15545, + 20727, 27637, 36850, 49134, 65513, 87351, 116469, 155293, + 207058, 276078, 368105] + for i, jump in enumerate(jumps): + assert memmgr._get_index(jump) == i + assert memmgr._get_index(jump + 1) == i + 1 + +def test_add_free_block(): + memmgr = AsmMemoryManager(min_fragment=8, + num_indices=5) + memmgr._add_free_block(10, 18) + assert memmgr.free_blocks == {10: 18} + assert memmgr.free_blocks_end == {18: 10} + assert memmgr.blocks_by_size == [[10], [], [], [], []] + memmgr._add_free_block(20, 30) + assert memmgr.free_blocks == {10: 18, 20: 30} + assert memmgr.free_blocks_end == {18: 10, 30: 20} + assert memmgr.blocks_by_size == [[10], [20], [], [], []] + memmgr._add_free_block(18, 20) # merge both left and right + assert memmgr.free_blocks == {10: 30} + assert memmgr.free_blocks_end == {30: 10} + assert memmgr.blocks_by_size == [[], [], [], [10], []] + +def test_allocate_block(): + memmgr = AsmMemoryManager(min_fragment=8, + num_indices=5) + memmgr._add_free_block(10, 18) + memmgr._add_free_block(20, 30) + (start, stop) = memmgr._allocate_block(4) + assert (start, stop) == (10, 18) + assert memmgr.free_blocks == {20: 30} + assert memmgr.free_blocks_end == {30: 20} + assert memmgr.blocks_by_size == [[], [20], [], [], []] + (start, stop) = memmgr._allocate_block(4) + assert (start, stop) == (20, 30) + assert memmgr.free_blocks == {} + assert memmgr.free_blocks_end == {} + assert memmgr.blocks_by_size == [[], [], [], [], []] + +def test_malloc_without_fragment(): + memmgr = AsmMemoryManager(min_fragment=8, + num_indices=5) + memmgr._add_free_block(10, 18) + memmgr._add_free_block(20, 30) + for minsize in range(1, 11): + for maxsize in range(minsize, 14): + (start, stop) = memmgr.malloc(minsize, maxsize) + if minsize <= 8: + assert (start, stop) == (10, 18) + else: + assert (start, stop) == (20, 30) + memmgr._add_free_block(start, stop) + memmgr._add_free_block(40, 49) + (start, stop) = memmgr.malloc(10, 10) + assert (start, stop) == (20, 30) + +def test_malloc_with_fragment(): + for reqsize in range(1, 33): + memmgr = AsmMemoryManager(min_fragment=8, + num_indices=5) + memmgr._add_free_block(12, 44) + (start, stop) = memmgr.malloc(reqsize, reqsize) + if reqsize + 8 <= 32: + assert (start, stop) == (12, 12 + reqsize) + assert memmgr.free_blocks == {stop: 44} + assert memmgr.free_blocks_end == {44: stop} + assert [stop] in memmgr.blocks_by_size + else: + assert (start, stop) == (12, 44) + assert memmgr.free_blocks == {} + assert memmgr.free_blocks_end == {} + assert memmgr.blocks_by_size == [[], [], [], [], []] + + +class TestAsmMemoryManager: + + def setup_method(self, _): + self.memmgr = AsmMemoryManager(min_fragment=8, + num_indices=10, + large_alloc_size=8192) + + def teardown_method(self, _): + self.memmgr._delete() + + def test_malloc_simple(self): + for i in range(100): + while self.memmgr.total_memory_allocated < 16384: + reqsize = random.randrange(1, 200) + (start, stop) = self.memmgr.malloc(reqsize, reqsize) + assert reqsize <= stop - start < reqsize + 8 + assert self.memmgr.total_memory_allocated in [8192, 16384] + self.teardown_method(None) + self.setup_method(None) + + def test_random(self): + got = [] + real_use = 0 + prev_total = 0 + iterations_without_allocating_more = 0 + while True: + # + if got and (random.random() < 0.4 or len(got) == 1000): + # free + start, stop = got.pop(random.randrange(0, len(got))) + self.memmgr.free(start, stop) + real_use -= (stop - start) + assert real_use >= 0 + # + else: + # allocate + reqsize = random.randrange(1, 200) + if random.random() < 0.5: + reqmaxsize = reqsize + else: + reqmaxsize = reqsize + random.randrange(0, 200) + (start, stop) = self.memmgr.malloc(reqsize, reqmaxsize) + assert reqsize <= stop - start < reqmaxsize + 8 + for otherstart, otherstop in got: # no overlap + assert otherstop <= start or stop <= otherstart + got.append((start, stop)) + real_use += (stop - start) + if self.memmgr.total_memory_allocated == prev_total: + iterations_without_allocating_more += 1 + if iterations_without_allocating_more == 40000: + break # ok + else: + new_total = self.memmgr.total_memory_allocated + iterations_without_allocating_more = 0 + print real_use, new_total + # We seem to never see a printed value greater + # than 131072. Be reasonable and allow up to 147456. + assert new_total <= 147456 + prev_total = new_total From fijal at codespeak.net Thu Nov 25 10:31:55 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Thu, 25 Nov 2010 10:31:55 +0100 (CET) Subject: [pypy-svn] r79497 - in pypy/release/1.4.x/pypy/jit/backend/llsupport: . test Message-ID: <20101125093155.44476282BE3@codespeak.net> Author: fijal Date: Thu Nov 25 10:31:53 2010 New Revision: 79497 Removed: pypy/release/1.4.x/pypy/jit/backend/llsupport/codebuf.py pypy/release/1.4.x/pypy/jit/backend/llsupport/test/test_codebuf.py Log: Revert an accidental change From fijal at codespeak.net Thu Nov 25 10:45:00 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Thu, 25 Nov 2010 10:45:00 +0100 (CET) Subject: [pypy-svn] r79498 - pypy/trunk/pypy/jit/backend/x86 Message-ID: <20101125094500.9C3EF282BEA@codespeak.net> Author: fijal Date: Thu Nov 25 10:44:58 2010 New Revision: 79498 Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py Log: Change a hack to use official interface Modified: pypy/trunk/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/trunk/pypy/jit/backend/x86/assembler.py (original) +++ pypy/trunk/pypy/jit/backend/x86/assembler.py Thu Nov 25 10:44:58 2010 @@ -31,7 +31,8 @@ from pypy.jit.backend.x86 import rx86, regloc, codebuf from pypy.jit.metainterp.resoperation import rop, ResOperation from pypy.jit.backend.x86.support import values_array -from pypy.rlib.debug import debug_print, debug_start, debug_stop +from pypy.rlib.debug import debug_print, debug_start, debug_stop,\ + have_debug_prints from pypy.rlib import rgc from pypy.jit.backend.x86.jump import remap_frame_layout from pypy.jit.metainterp.history import ConstInt, BoxInt @@ -228,11 +229,9 @@ self._build_float_constants() if hasattr(gc_ll_descr, 'get_malloc_fixedsize_slowpath_addr'): self._build_malloc_fixedsize_slowpath() - s = os.environ.get('PYPYLOG') - if s: - if s.find(':') != -1: - s = s.split(':')[-1] - self.set_debug(True) + debug_start('jit-backend-counts') + self.set_debug(have_debug_prints()) + debug_stop('jit-backend-counts') # Intialize here instead of __init__ to prevent # pending_guard_tokens from being considered a prebuilt object, # which sometimes causes memory leaks since the prebuilt list is From arigo at codespeak.net Thu Nov 25 10:45:24 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 10:45:24 +0100 (CET) Subject: [pypy-svn] r79499 - in pypy/trunk/pypy: jit/metainterp rlib rpython/test Message-ID: <20101125094524.AD11A282BEB@codespeak.net> Author: arigo Date: Thu Nov 25 10:45:18 2010 New Revision: 79499 Modified: pypy/trunk/pypy/jit/metainterp/history.py pypy/trunk/pypy/jit/metainterp/memmgr.py pypy/trunk/pypy/rlib/rarithmetic.py pypy/trunk/pypy/rpython/test/test_rint.py Log: Fix on 64-bits: move 'r_int64' to the general rarithmetic module and use it. The issue is that r_longlong is r_int on 64-bits, and using r_int explicitly is not well supported by the llinterp. Modified: pypy/trunk/pypy/jit/metainterp/history.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/history.py (original) +++ pypy/trunk/pypy/jit/metainterp/history.py Thu Nov 25 10:45:18 2010 @@ -4,7 +4,7 @@ from pypy.rpython.ootypesystem import ootype from pypy.rlib.objectmodel import we_are_translated, r_dict, Symbolic from pypy.rlib.objectmodel import compute_hash, compute_unique_id -from pypy.rlib.rarithmetic import intmask, r_longlong +from pypy.rlib.rarithmetic import intmask, r_int64 from pypy.tool.uid import uid from pypy.conftest import option @@ -731,7 +731,7 @@ # specnodes = ... # and more data specified by the backend when the loop is compiled number = -1 - generation = r_longlong(0) + generation = r_int64(0) # one purpose of LoopToken is to keep alive the CompiledLoopToken # returned by the backend. When the LoopToken goes away, the # CompiledLoopToken has its __del__ called, which frees the assembler Modified: pypy/trunk/pypy/jit/metainterp/memmgr.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/memmgr.py (original) +++ pypy/trunk/pypy/jit/metainterp/memmgr.py Thu Nov 25 10:45:18 2010 @@ -1,5 +1,5 @@ import math -from pypy.rlib.rarithmetic import r_longlong +from pypy.rlib.rarithmetic import r_int64 from pypy.rlib.debug import debug_start, debug_print, debug_stop from pypy.rlib.objectmodel import we_are_translated @@ -24,7 +24,7 @@ def __init__(self): self.check_frequency = -1 - # NB. use of r_longlong to be extremely far on the safe side: + # NB. use of r_int64 to be extremely far on the safe side: # this is increasing by one after each loop or bridge is # compiled, and it must not overflow. If the backend implements # complete freeing in cpu.free_loop_and_bridges(), then it may @@ -32,13 +32,13 @@ # enough. But in this day and age, you'd still never have the # patience of waiting for a slowly-increasing 64-bit number to # overflow :-) - self.current_generation = r_longlong(1) - self.next_check = r_longlong(-1) + self.current_generation = r_int64(1) + self.next_check = r_int64(-1) self.alive_loops = {} def set_max_age(self, max_age, check_frequency=0): if max_age <= 0: - self.next_check = r_longlong(-1) + self.next_check = r_int64(-1) else: self.max_age = max_age if check_frequency <= 0: Modified: pypy/trunk/pypy/rlib/rarithmetic.py ============================================================================== --- pypy/trunk/pypy/rlib/rarithmetic.py (original) +++ pypy/trunk/pypy/rlib/rarithmetic.py Thu Nov 25 10:45:18 2010 @@ -389,6 +389,11 @@ r_longlong = build_int('r_longlong', True, 64) r_ulonglong = build_int('r_ulonglong', False, 64) +if r_longlong is not r_int: + r_int64 = r_longlong +else: + r_int64 = int + # float as string -> sign, beforept, afterpt, exponent Modified: pypy/trunk/pypy/rpython/test/test_rint.py ============================================================================== --- pypy/trunk/pypy/rpython/test/test_rint.py (original) +++ pypy/trunk/pypy/rpython/test/test_rint.py Thu Nov 25 10:45:18 2010 @@ -4,14 +4,9 @@ from pypy.annotation import model as annmodel from pypy.rpython.test import snippet from pypy.rlib.rarithmetic import r_int, r_uint, r_longlong, r_ulonglong -from pypy.rlib.rarithmetic import ovfcheck +from pypy.rlib.rarithmetic import ovfcheck, r_int64 from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin -if r_longlong is not r_int: - int64 = r_longlong -else: - int64 = int - class TestSnippet(object): @@ -110,10 +105,10 @@ def f(i): return str(i) - res = self.interpret(f, [int64(0)]) + res = self.interpret(f, [r_int64(0)]) assert self.ll_to_string(res) == '0' - res = self.interpret(f, [int64(413974738222117)]) + res = self.interpret(f, [r_int64(413974738222117)]) assert self.ll_to_string(res) == '413974738222117' def test_unsigned(self): @@ -135,7 +130,7 @@ f._annspecialcase_ = "specialize:argtype(0)" def g(n): if n > 0: - return f(int64(0)) + return f(r_int64(0)) else: return f(0) res = self.interpret(g, [0]) @@ -147,7 +142,7 @@ def test_downcast_int(self): def f(i): return int(i) - res = self.interpret(f, [int64(0)]) + res = self.interpret(f, [r_int64(0)]) assert res == 0 def test_isinstance_vs_int_types(self): @@ -157,7 +152,7 @@ return [None] if isinstance(x, str): return x - if isinstance(x, int64): + if isinstance(x, r_int64): return int(x) return "XXX" wrap._annspecialcase_ = 'specialize:argtype(0)' @@ -165,7 +160,7 @@ space = FakeSpace() def wrap(x): return space.wrap(x) - res = self.interpret(wrap, [int64(0)]) + res = self.interpret(wrap, [r_int64(0)]) assert res == 0 def test_truediv(self): @@ -178,25 +173,25 @@ def test_float_conversion(self): def f(ii): return float(ii) - res = self.interpret(f, [int64(100000000)]) + res = self.interpret(f, [r_int64(100000000)]) assert type(res) is float assert res == 100000000. - res = self.interpret(f, [int64(1234567890123456789)]) + res = self.interpret(f, [r_int64(1234567890123456789)]) assert type(res) is float assert self.float_eq(res, 1.2345678901234568e+18) def test_float_conversion_implicit(self): def f(ii): return 1.0 + ii - res = self.interpret(f, [int64(100000000)]) + res = self.interpret(f, [r_int64(100000000)]) assert type(res) is float assert res == 100000001. - res = self.interpret(f, [int64(1234567890123456789)]) + res = self.interpret(f, [r_int64(1234567890123456789)]) assert type(res) is float assert self.float_eq(res, 1.2345678901234568e+18) def test_rarithmetic(self): - inttypes = [int, r_uint, int64, r_ulonglong] + inttypes = [int, r_uint, r_int64, r_ulonglong] for inttype in inttypes: c = inttype() def f(): @@ -231,16 +226,16 @@ res = self.interpret(f, [int(-1<<(r_int.BITS-1))]) assert res == 0 - res = self.interpret(f, [int64(-1)]) + res = self.interpret(f, [r_int64(-1)]) assert res == 1 - res = self.interpret(f, [int64(-1)<<(r_longlong.BITS-1)]) + res = self.interpret(f, [r_int64(-1)<<(r_longlong.BITS-1)]) assert res == 0 div_mod_iteration_count = 1000 def test_div_mod(self): import random - for inttype in (int, int64): + for inttype in (int, r_int64): def d(x, y): return x/y @@ -303,7 +298,7 @@ except ZeroDivisionError: return 84 - for inttype in (int, int64): + for inttype in (int, r_int64): args = [( 5, 2), (-5, 2), ( 5,-2), (-5,-2), ( 6, 2), (-6, 2), ( 6,-2), (-6,-2), From david at codespeak.net Thu Nov 25 13:03:19 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 25 Nov 2010 13:03:19 +0100 (CET) Subject: [pypy-svn] r79503 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . helper Message-ID: <20101125120319.CF7265080E@codespeak.net> Author: david Date: Thu Nov 25 13:03:18 2010 New Revision: 79503 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Log: Patch stack pointer from bridges to allow extending the frame while compiling a bridge Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Thu Nov 25 13:03:18 2010 @@ -35,26 +35,26 @@ def setup_failure_recovery(self): @rgc.no_collect - def failure_recovery_func(mem_loc, frame_loc): + def failure_recovery_func(mem_loc, frame_pointer, stack_pointer): """mem_loc is a structure in memory describing where the values for the failargs are stored. frame loc is the address of the frame pointer for the frame to be decoded frame """ - return self.decode_registers_and_descr(mem_loc, frame_loc) + return self.decode_registers_and_descr(mem_loc, frame_pointer, stack_pointer) self.failure_recovery_func = failure_recovery_func @rgc.no_collect - def decode_registers_and_descr(self, mem_loc, frame_loc): + def decode_registers_and_descr(self, mem_loc, frame_loc, regs_loc): """Decode locations encoded in memory at mem_loc and write the values to the failboxes. Values for spilled vars and registers are stored on stack at frame_loc """ enc = rffi.cast(rffi.CCHARP, mem_loc) - frame_depth = self.decode32(enc, 0) - stack = rffi.cast(rffi.CCHARP, frame_loc - (frame_depth)*WORD) - regs = rffi.cast(rffi.CCHARP, frame_loc - (frame_depth + len(r.all_regs))*WORD) - i = 3 + frame_depth = frame_loc - (regs_loc + len(r.all_regs)*WORD) + stack = rffi.cast(rffi.CCHARP, frame_loc - frame_depth) + regs = rffi.cast(rffi.CCHARP, regs_loc) + i = -1 fail_index = -1 if self.debug: import pdb; pdb.set_trace() @@ -77,7 +77,7 @@ i += 4 elif res == '\xFC': # stack location stack_loc = self.decode32(enc, i+1) - value = self.decode32(stack, (frame_depth - stack_loc)*WORD) + value = self.decode32(stack, frame_depth - stack_loc*WORD) i += 4 else: # an int in a reg location reg = ord(enc[i]) @@ -96,11 +96,9 @@ self.fail_boxes_count = fail_index return descr - def decode_inputargs_and_frame_depth(self, enc, inputargs, regalloc): + def decode_inputargs(self, enc, inputargs, regalloc): locs = [] - # first word contains frame depth - frame_depth = self.decode32(enc, 0) + 1 - j = 4 + j = 0 for i in range(len(inputargs)): res = enc[j] if res == '\xFF': @@ -123,7 +121,7 @@ loc = r.all_regs[ord(res)] j += 1 locs.append(loc) - return frame_depth, locs + return locs def decode32(self, mem, index): highval = ord(mem[index+3]) @@ -142,13 +140,13 @@ def _gen_exit_path(self): self.setup_failure_recovery() - functype = lltype.Ptr(lltype.FuncType([lltype.Signed, lltype.Signed], lltype.Signed)) + functype = lltype.Ptr(lltype.FuncType([lltype.Signed, lltype.Signed, lltype.Signed], lltype.Signed)) decode_registers_addr = llhelper(functype, self.failure_recovery_func) self.mc.PUSH([reg.value for reg in r.all_regs]) # registers r0 .. r10 - self.mc.MOV_rr(r.r0.value, r.lr.value) # move mem block address, to r0 to pass as - # parameter to next procedure call - self.mc.MOV_rr(r.r1.value, r.fp.value) # pass the current frame pointer as second param + self.mc.MOV_rr(r.r0.value, r.lr.value) # move mem block address, to r0 to pass as + self.mc.MOV_rr(r.r1.value, r.fp.value) # pass the current frame pointer as second param + self.mc.MOV_rr(r.r2.value, r.sp.value) # pass the current stack pointer as third param self.mc.BL(rffi.cast(lltype.Signed, decode_registers_addr)) self.mc.MOV_rr(r.ip.value, r.r0.value) @@ -170,16 +168,15 @@ """ descr = op.getdescr() + if op.getopnum() != rop.FINISH: + descr._arm_frame_depth = regalloc.frame_manager.frame_depth reg = r.lr # XXX free this memory # XXX allocate correct amount of memory - mem = lltype.malloc(rffi.CArray(lltype.Char), len(args)*6+9, + mem = lltype.malloc(rffi.CArray(lltype.Char), len(args)*6+5, flavor='raw', track_allocation=False) - # Note, the actual frame depth is one less than the value stored in - # regalloc.frame_manager.frame_depth - self.encode32(mem, 0, regalloc.frame_manager.frame_depth - 1) i = 0 - j = 4 + j = 0 while(i < len(args)): if args[i]: loc = regalloc.loc(args[i]) @@ -328,9 +325,9 @@ if regalloc.frame_manager.frame_depth == 1: return n = (regalloc.frame_manager.frame_depth-1)*WORD - self._adjust_sp(n, regalloc, cb) + self._adjust_sp(n, regalloc, cb, base_reg=r.fp) - def _adjust_sp(self, n, regalloc, cb=None, fcond=c.AL): + def _adjust_sp(self, n, regalloc, cb=None, fcond=c.AL, base_reg=r.sp): if cb is None: cb = self.mc if n < 0: @@ -343,7 +340,7 @@ op = cb.ADD_ri else: op = cb.SUB_ri - op(r.sp.value, r.sp.value, n) + op(r.sp.value, base_reg.value, n) else: b = TempBox() reg = regalloc.force_allocate_reg(b) @@ -352,7 +349,7 @@ op = cb.ADD_rr else: op = cb.SUB_rr - op(r.sp.value, r.sp.value, reg.value, cond=fcond) + op(r.sp.value, base_reg.value, reg.value, cond=fcond) regalloc.possibly_free_var(b) def _walk_operations(self, operations, regalloc): @@ -386,12 +383,16 @@ regalloc = ARMRegisterManager(longevity, assembler=self, frame_manager=ARMFrameManager()) bridge_head = self.mc.curraddr() - frame_depth, locs = self.decode_inputargs_and_frame_depth(enc, inputargs, regalloc) + frame_depth = faildescr._arm_frame_depth + locs = self.decode_inputargs(enc, inputargs, regalloc) regalloc.update_bindings(locs, frame_depth, inputargs) + sp_patch_location = self._prepare_sp_patch_location() print 'Bridge', inputargs, operations self._walk_operations(operations, regalloc) + self._patch_sp_offset(sp_patch_location, regalloc) + print 'Done building bridges' self.patch_trace(faildescr, bridge_head, regalloc) print 'Done patching trace' Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Thu Nov 25 13:03:18 2010 @@ -11,9 +11,8 @@ self.mc.CMP_ri(reg.value, 0) self.mc.MOV_ri(res.value, 1, true_cond) self.mc.MOV_ri(res.value, 0, false_cond) - regalloc.possibly_free_vars_for_op(op) - if op.result: - regalloc.possibly_free_var(op.result) + regalloc.possibly_free_var(a0) + regalloc.possibly_free_var(op.result) return fcond return f @@ -42,9 +41,7 @@ l1 = regalloc.make_sure_var_in_reg(arg1, [arg0], imm_fine=False) res = regalloc.force_allocate_reg(op.result, [arg0, arg1]) rr_op(res.value, l0.value, l1.value) - regalloc.possibly_free_vars_for_op(op) - if op.result: - regalloc.possibly_free_var(op.result) + regalloc.possibly_free_vars([arg0, arg1, op.result]) return fcond return f @@ -53,7 +50,7 @@ assert fcond is not None a0 = op.getarg(0) a1 = op.getarg(1) - arg1 = regalloc.make_sure_var_in_reg(a0, selected_reg=r.r0, imm_fine=False) + arg1 = regalloc.make_sure_var_in_reg(a0, [a1], selected_reg=r.r0, imm_fine=False) arg2 = regalloc.make_sure_var_in_reg(a1, [a0], selected_reg=r.r1, imm_fine=False) assert arg1 == r.r0 assert arg2 == r.r1 @@ -61,7 +58,8 @@ getattr(self.mc, opname)(fcond) regalloc.after_call(op.result) - regalloc.possibly_free_vars_for_op(op) + regalloc.possibly_free_var(a0) + regalloc.possibly_free_var(a1) if op.result: regalloc.possibly_free_var(op.result) return fcond @@ -93,8 +91,6 @@ inv = c.get_opposite_of(condition) self.mc.MOV_ri(res.value, 1, cond=condition) self.mc.MOV_ri(res.value, 0, cond=inv) - regalloc.possibly_free_vars_for_op(op) - if op.result: - regalloc.possibly_free_var(op.result) + regalloc.possibly_free_vars([arg0, arg1, op.result]) return fcond return f Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Thu Nov 25 13:03:18 2010 @@ -1,4 +1,6 @@ from pypy.jit.backend.arm.assembler import AssemblerARM +from pypy.jit.backend.arm.arch import WORD +from pypy.jit.backend.arm.registers import all_regs from pypy.jit.backend.llsupport.llmodel import AbstractLLCPU from pypy.rpython.llinterp import LLInterpreter from pypy.rpython.lltypesystem import lltype, rffi, llmemory @@ -71,9 +73,12 @@ faildescr = self.get_fail_descr_from_number(fail_index) rffi.cast(TP, addr_of_force_index)[0] = -1 # start of "no gc operation!" block + addr_end_of_frame = (addr_of_force_index - + (faildescr._arm_frame_depth+len(all_regs))*WORD) fail_index_2 = self.assembler.failure_recovery_func( faildescr._failure_recovery_code, - addr_of_force_index) + addr_of_force_index, + addr_end_of_frame) self.assembler.leave_jitted_hook() # end of "no gc operation!" block #assert fail_index == fail_index_2 From arigo at codespeak.net Thu Nov 25 13:09:13 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 13:09:13 +0100 (CET) Subject: [pypy-svn] r79505 - pypy/trunk/pypy/module/itertools Message-ID: <20101125120913.0F3BA282BE3@codespeak.net> Author: arigo Date: Thu Nov 25 13:09:12 2010 New Revision: 79505 Modified: pypy/trunk/pypy/module/itertools/interp_itertools.py Log: JIT-friendly rewrites: where it is possible, move the loops to subfunctions to prevent the whole function from being seen by the jit. Modified: pypy/trunk/pypy/module/itertools/interp_itertools.py ============================================================================== --- pypy/trunk/pypy/module/itertools/interp_itertools.py (original) +++ pypy/trunk/pypy/module/itertools/interp_itertools.py Thu Nov 25 13:09:12 2010 @@ -391,7 +391,8 @@ self.iterators_w = iterators_w self.current_iterator = 0 self.num_iterators = len(iterators_w) - self.started = False + if self.num_iterators > 0: + self.w_it = iterators_w[0] def iter_w(self): return self.space.wrap(self) @@ -399,26 +400,23 @@ def next_w(self): if self.current_iterator >= self.num_iterators: raise OperationError(self.space.w_StopIteration, self.space.w_None) - if not self.started: - self.current_iterator = 0 - self.w_it = self.iterators_w[self.current_iterator] - self.started = True + try: + return self.space.next(self.w_it) + except OperationError, e: + return self._handle_error(e) + def _handle_error(self, e): while True: + if not e.match(self.space, self.space.w_StopIteration): + raise e + self.current_iterator += 1 + if self.current_iterator >= self.num_iterators: + raise e + self.w_it = self.iterators_w[self.current_iterator] try: - w_obj = self.space.next(self.w_it) + return self.space.next(self.w_it) except OperationError, e: - if e.match(self.space, self.space.w_StopIteration): - self.current_iterator += 1 - if self.current_iterator >= self.num_iterators: - raise OperationError(self.space.w_StopIteration, self.space.w_None) - else: - self.w_it = self.iterators_w[self.current_iterator] - else: - raise - else: - break - return w_obj + pass # loop back to the start of _handle_error(e) def W_Chain___new__(space, w_subtype, args_w): return space.wrap(W_Chain(space, args_w)) @@ -446,8 +444,10 @@ def __init__(self, space, w_fun, args_w): self.space = space - self.identity_fun = (self.space.is_w(w_fun, space.w_None)) - self.w_fun = w_fun + if self.space.is_w(w_fun, space.w_None): + self.w_fun = None + else: + self.w_fun = w_fun iterators_w = [] i = 0 @@ -470,12 +470,26 @@ return self.space.wrap(self) def next_w(self): - w_objects = self.space.newtuple([self.space.next(w_it) for w_it in self.iterators_w]) - if self.identity_fun: + # common case: 1 or 2 arguments + iterators_w = self.iterators_w + length = len(iterators_w) + if length == 1: + objects = [self.space.next(iterators_w[0])] + elif length == 2: + objects = [self.space.next(iterators_w[0]), + self.space.next(iterators_w[1])] + else: + objects = self._get_objects() + w_objects = self.space.newtuple(objects) + if self.w_fun is None: return w_objects else: return self.space.call(self.w_fun, w_objects) + def _get_objects(self): + # the loop is out of the way of the JIT + return [self.space.next(w_elem) for w_elem in self.iterators_w] + def W_IMap___new__(space, w_subtype, w_fun, args_w): if len(args_w) == 0: @@ -769,15 +783,7 @@ raise OperationError(self.space.w_StopIteration, self.space.w_None) if not self.new_group: - # Consume unwanted input until we reach the next group - try: - while True: - self.group_next(self.index) - - except StopIteration: - pass - if self.exhausted: - raise OperationError(self.space.w_StopIteration, self.space.w_None) + self._consume_unwanted_input() if not self.started: self.started = True @@ -799,6 +805,16 @@ w_iterator = self.space.wrap(W_GroupByIterator(self.space, self.index, self)) return self.space.newtuple([self.w_key, w_iterator]) + def _consume_unwanted_input(self): + # Consume unwanted input until we reach the next group + try: + while True: + self.group_next(self.index) + except StopIteration: + pass + if self.exhausted: + raise OperationError(self.space.w_StopIteration, self.space.w_None) + def group_next(self, group_index): if group_index < self.index: raise StopIteration From arigo at codespeak.net Thu Nov 25 13:10:41 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 13:10:41 +0100 (CET) Subject: [pypy-svn] r79506 - pypy/trunk/pypy/module/pypyjit Message-ID: <20101125121041.6F97D50812@codespeak.net> Author: arigo Date: Thu Nov 25 13:10:39 2010 New Revision: 79506 Modified: pypy/trunk/pypy/module/pypyjit/policy.py Log: Add itertools to look_inside_pypy_module(). Modified: pypy/trunk/pypy/module/pypyjit/policy.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/policy.py (original) +++ pypy/trunk/pypy/module/pypyjit/policy.py Thu Nov 25 13:10:39 2010 @@ -12,7 +12,7 @@ if '.' in modname: modname, _ = modname.split('.', 1) if modname in ['pypyjit', 'signal', 'micronumpy', 'math', 'exceptions', - 'imp', 'sys', 'array', '_ffi']: + 'imp', 'sys', 'array', '_ffi', 'itertools']: return True return False From arigo at codespeak.net Thu Nov 25 13:23:04 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 13:23:04 +0100 (CET) Subject: [pypy-svn] r79507 - in pypy/release/1.4.x/pypy: jit/metainterp rlib rpython/test Message-ID: <20101125122304.D2F31282BD6@codespeak.net> Author: arigo Date: Thu Nov 25 13:22:59 2010 New Revision: 79507 Modified: pypy/release/1.4.x/pypy/jit/metainterp/history.py pypy/release/1.4.x/pypy/jit/metainterp/memmgr.py pypy/release/1.4.x/pypy/rlib/rarithmetic.py pypy/release/1.4.x/pypy/rpython/test/test_rint.py Log: Merge r79499: test fix only. Modified: pypy/release/1.4.x/pypy/jit/metainterp/history.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/metainterp/history.py (original) +++ pypy/release/1.4.x/pypy/jit/metainterp/history.py Thu Nov 25 13:22:59 2010 @@ -4,7 +4,7 @@ from pypy.rpython.ootypesystem import ootype from pypy.rlib.objectmodel import we_are_translated, r_dict, Symbolic from pypy.rlib.objectmodel import compute_hash, compute_unique_id -from pypy.rlib.rarithmetic import intmask, r_longlong +from pypy.rlib.rarithmetic import intmask, r_int64 from pypy.tool.uid import uid from pypy.conftest import option @@ -731,7 +731,7 @@ # specnodes = ... # and more data specified by the backend when the loop is compiled number = -1 - generation = r_longlong(0) + generation = r_int64(0) # one purpose of LoopToken is to keep alive the CompiledLoopToken # returned by the backend. When the LoopToken goes away, the # CompiledLoopToken has its __del__ called, which frees the assembler Modified: pypy/release/1.4.x/pypy/jit/metainterp/memmgr.py ============================================================================== --- pypy/release/1.4.x/pypy/jit/metainterp/memmgr.py (original) +++ pypy/release/1.4.x/pypy/jit/metainterp/memmgr.py Thu Nov 25 13:22:59 2010 @@ -1,5 +1,5 @@ import math -from pypy.rlib.rarithmetic import r_longlong +from pypy.rlib.rarithmetic import r_int64 from pypy.rlib.debug import debug_start, debug_print, debug_stop from pypy.rlib.objectmodel import we_are_translated @@ -24,7 +24,7 @@ def __init__(self): self.check_frequency = -1 - # NB. use of r_longlong to be extremely far on the safe side: + # NB. use of r_int64 to be extremely far on the safe side: # this is increasing by one after each loop or bridge is # compiled, and it must not overflow. If the backend implements # complete freeing in cpu.free_loop_and_bridges(), then it may @@ -32,13 +32,13 @@ # enough. But in this day and age, you'd still never have the # patience of waiting for a slowly-increasing 64-bit number to # overflow :-) - self.current_generation = r_longlong(1) - self.next_check = r_longlong(-1) + self.current_generation = r_int64(1) + self.next_check = r_int64(-1) self.alive_loops = {} def set_max_age(self, max_age, check_frequency=0): if max_age <= 0: - self.next_check = r_longlong(-1) + self.next_check = r_int64(-1) else: self.max_age = max_age if check_frequency <= 0: Modified: pypy/release/1.4.x/pypy/rlib/rarithmetic.py ============================================================================== --- pypy/release/1.4.x/pypy/rlib/rarithmetic.py (original) +++ pypy/release/1.4.x/pypy/rlib/rarithmetic.py Thu Nov 25 13:22:59 2010 @@ -389,6 +389,11 @@ r_longlong = build_int('r_longlong', True, 64) r_ulonglong = build_int('r_ulonglong', False, 64) +if r_longlong is not r_int: + r_int64 = r_longlong +else: + r_int64 = int + # float as string -> sign, beforept, afterpt, exponent Modified: pypy/release/1.4.x/pypy/rpython/test/test_rint.py ============================================================================== --- pypy/release/1.4.x/pypy/rpython/test/test_rint.py (original) +++ pypy/release/1.4.x/pypy/rpython/test/test_rint.py Thu Nov 25 13:22:59 2010 @@ -4,14 +4,9 @@ from pypy.annotation import model as annmodel from pypy.rpython.test import snippet from pypy.rlib.rarithmetic import r_int, r_uint, r_longlong, r_ulonglong -from pypy.rlib.rarithmetic import ovfcheck +from pypy.rlib.rarithmetic import ovfcheck, r_int64 from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin -if r_longlong is not r_int: - int64 = r_longlong -else: - int64 = int - class TestSnippet(object): @@ -110,10 +105,10 @@ def f(i): return str(i) - res = self.interpret(f, [int64(0)]) + res = self.interpret(f, [r_int64(0)]) assert self.ll_to_string(res) == '0' - res = self.interpret(f, [int64(413974738222117)]) + res = self.interpret(f, [r_int64(413974738222117)]) assert self.ll_to_string(res) == '413974738222117' def test_unsigned(self): @@ -135,7 +130,7 @@ f._annspecialcase_ = "specialize:argtype(0)" def g(n): if n > 0: - return f(int64(0)) + return f(r_int64(0)) else: return f(0) res = self.interpret(g, [0]) @@ -147,7 +142,7 @@ def test_downcast_int(self): def f(i): return int(i) - res = self.interpret(f, [int64(0)]) + res = self.interpret(f, [r_int64(0)]) assert res == 0 def test_isinstance_vs_int_types(self): @@ -157,7 +152,7 @@ return [None] if isinstance(x, str): return x - if isinstance(x, int64): + if isinstance(x, r_int64): return int(x) return "XXX" wrap._annspecialcase_ = 'specialize:argtype(0)' @@ -165,7 +160,7 @@ space = FakeSpace() def wrap(x): return space.wrap(x) - res = self.interpret(wrap, [int64(0)]) + res = self.interpret(wrap, [r_int64(0)]) assert res == 0 def test_truediv(self): @@ -178,25 +173,25 @@ def test_float_conversion(self): def f(ii): return float(ii) - res = self.interpret(f, [int64(100000000)]) + res = self.interpret(f, [r_int64(100000000)]) assert type(res) is float assert res == 100000000. - res = self.interpret(f, [int64(1234567890123456789)]) + res = self.interpret(f, [r_int64(1234567890123456789)]) assert type(res) is float assert self.float_eq(res, 1.2345678901234568e+18) def test_float_conversion_implicit(self): def f(ii): return 1.0 + ii - res = self.interpret(f, [int64(100000000)]) + res = self.interpret(f, [r_int64(100000000)]) assert type(res) is float assert res == 100000001. - res = self.interpret(f, [int64(1234567890123456789)]) + res = self.interpret(f, [r_int64(1234567890123456789)]) assert type(res) is float assert self.float_eq(res, 1.2345678901234568e+18) def test_rarithmetic(self): - inttypes = [int, r_uint, int64, r_ulonglong] + inttypes = [int, r_uint, r_int64, r_ulonglong] for inttype in inttypes: c = inttype() def f(): @@ -231,16 +226,16 @@ res = self.interpret(f, [int(-1<<(r_int.BITS-1))]) assert res == 0 - res = self.interpret(f, [int64(-1)]) + res = self.interpret(f, [r_int64(-1)]) assert res == 1 - res = self.interpret(f, [int64(-1)<<(r_longlong.BITS-1)]) + res = self.interpret(f, [r_int64(-1)<<(r_longlong.BITS-1)]) assert res == 0 div_mod_iteration_count = 1000 def test_div_mod(self): import random - for inttype in (int, int64): + for inttype in (int, r_int64): def d(x, y): return x/y @@ -303,7 +298,7 @@ except ZeroDivisionError: return 84 - for inttype in (int, int64): + for inttype in (int, r_int64): args = [( 5, 2), (-5, 2), ( 5,-2), (-5,-2), ( 6, 2), (-6, 2), ( 6,-2), (-6,-2), From arigo at codespeak.net Thu Nov 25 13:25:43 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 13:25:43 +0100 (CET) Subject: [pypy-svn] r79508 - pypy/trunk/pypy/module/pypyjit Message-ID: <20101125122543.B51DC50813@codespeak.net> Author: arigo Date: Thu Nov 25 13:25:42 2010 New Revision: 79508 Modified: pypy/trunk/pypy/module/pypyjit/policy.py Log: Add operator too. Modified: pypy/trunk/pypy/module/pypyjit/policy.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/policy.py (original) +++ pypy/trunk/pypy/module/pypyjit/policy.py Thu Nov 25 13:25:42 2010 @@ -12,7 +12,7 @@ if '.' in modname: modname, _ = modname.split('.', 1) if modname in ['pypyjit', 'signal', 'micronumpy', 'math', 'exceptions', - 'imp', 'sys', 'array', '_ffi', 'itertools']: + 'imp', 'sys', 'array', '_ffi', 'itertools', 'operator']: return True return False From arigo at codespeak.net Thu Nov 25 13:30:31 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 13:30:31 +0100 (CET) Subject: [pypy-svn] r79509 - pypy/trunk/pypy/module/pypyjit Message-ID: <20101125123031.31271282BE3@codespeak.net> Author: arigo Date: Thu Nov 25 13:30:30 2010 New Revision: 79509 Modified: pypy/trunk/pypy/module/pypyjit/policy.py Log: Kill pointless comment. Modified: pypy/trunk/pypy/module/pypyjit/policy.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/policy.py (original) +++ pypy/trunk/pypy/module/pypyjit/policy.py Thu Nov 25 13:30:30 2010 @@ -17,8 +17,6 @@ return False def look_inside_function(self, func): - # this function should never actually return True directly - # but instead call the base implementation mod = func.__module__ or '?' if mod.startswith('pypy.objspace.'): From afa at codespeak.net Thu Nov 25 13:48:03 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 25 Nov 2010 13:48:03 +0100 (CET) Subject: [pypy-svn] r79510 - in pypy/trunk/pypy/interpreter: . test Message-ID: <20101125124803.9080350818@codespeak.net> Author: afa Date: Thu Nov 25 13:48:02 2010 New Revision: 79510 Modified: pypy/trunk/pypy/interpreter/gateway.py pypy/trunk/pypy/interpreter/test/test_gateway.py Log: Check that @py.test.mark.* can really be used with applevel tests. Previously only @py.test.mark.xfail used to work, because the transformation always failed :-/ Modified: pypy/trunk/pypy/interpreter/gateway.py ============================================================================== --- pypy/trunk/pypy/interpreter/gateway.py (original) +++ pypy/trunk/pypy/interpreter/gateway.py Thu Nov 25 13:48:02 2010 @@ -1083,7 +1083,7 @@ # these decorators are known to return the same function # object, we may ignore them assert '\n' in source - source = source[source.find('\n') + 1:] + source = source[source.find('\n') + 1:].lstrip() assert source.startswith("def "), "can only transform functions" source = source[4:] p = source.find('(') Modified: pypy/trunk/pypy/interpreter/test/test_gateway.py ============================================================================== --- pypy/trunk/pypy/interpreter/test/test_gateway.py (original) +++ pypy/trunk/pypy/interpreter/test/test_gateway.py Thu Nov 25 13:48:02 2010 @@ -578,6 +578,11 @@ w_res = space.call_args(w_g, args) assert space.eq_w(w_res, space.wrap((-1, 0))) +class AppTestPyTestMark: + @py.test.mark.unlikely_to_exist + def test_anything(self): + pass + class TestPassThroughArguments: From arigo at codespeak.net Thu Nov 25 13:48:38 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 13:48:38 +0100 (CET) Subject: [pypy-svn] r79511 - in pypy/trunk/pypy: jit/codewriter jit/metainterp/test module/pypyjit Message-ID: <20101125124838.BAE5C282BEA@codespeak.net> Author: arigo Date: Thu Nov 25 13:48:36 2010 New Revision: 79511 Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py pypy/trunk/pypy/jit/codewriter/support.py pypy/trunk/pypy/jit/metainterp/test/test_basic.py pypy/trunk/pypy/module/pypyjit/policy.py Log: "Support" for gc_id in the JIT. Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/jtransform.py (original) +++ pypy/trunk/pypy/jit/codewriter/jtransform.py Thu Nov 25 13:48:36 2010 @@ -398,6 +398,7 @@ rewrite_op_int_lshift_ovf = _do_builtin_call rewrite_op_int_abs = _do_builtin_call rewrite_op_gc_identityhash = _do_builtin_call + rewrite_op_gc_id = _do_builtin_call # ---------- # getfield/setfield/mallocs etc. Modified: pypy/trunk/pypy/jit/codewriter/support.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/support.py (original) +++ pypy/trunk/pypy/jit/codewriter/support.py Thu Nov 25 13:48:36 2010 @@ -168,6 +168,9 @@ def _ll_1_gc_identityhash(x): return lltype.identityhash(x) +def _ll_1_gc_id(ptr): + return llop.gc_id(lltype.Signed, ptr) + def _ll_1_jit_force_virtual(inst): return llop.jit_force_virtual(lltype.typeOf(inst), inst) @@ -548,6 +551,9 @@ def get_identityhash_oopspec(op): return 'gc_identityhash', op.args +def get_gcid_oopspec(op): + return 'gc_id', op.args + RENAMED_ADT_NAME = { 'list': { @@ -578,6 +584,8 @@ return get_oostring_oopspec(op) elif op.opname == 'gc_identityhash': return get_identityhash_oopspec(op) + elif op.opname == 'gc_id': + return get_gcid_oopspec(op) else: raise ValueError(op.opname) Modified: pypy/trunk/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_basic.py Thu Nov 25 13:48:36 2010 @@ -1701,6 +1701,29 @@ res = self.meta_interp(f, [5, 2]) assert 4 < res < 14 + def test_compute_identity_hash(self): + from pypy.rlib.objectmodel import compute_identity_hash + class A(object): + pass + def f(): + a = A() + return compute_identity_hash(a) == compute_identity_hash(a) + res = self.interp_operations(f, []) + assert res + # a "did not crash" kind of test + + def test_compute_unique_id(self): + from pypy.rlib.objectmodel import compute_unique_id + class A(object): + pass + def f(): + a1 = A() + a2 = A() + return (compute_unique_id(a1) == compute_unique_id(a1) and + compute_unique_id(a1) != compute_unique_id(a2)) + res = self.interp_operations(f, []) + assert res + class TestOOtype(BasicTests, OOJitMixin): Modified: pypy/trunk/pypy/module/pypyjit/policy.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/policy.py (original) +++ pypy/trunk/pypy/module/pypyjit/policy.py Thu Nov 25 13:48:36 2010 @@ -19,10 +19,6 @@ def look_inside_function(self, func): mod = func.__module__ or '?' - if mod.startswith('pypy.objspace.'): - # gc_id operation - if func.__name__ == 'id__ANY': - return False if mod == 'pypy.rlib.rbigint' or mod == 'pypy.rlib.rlocale': return False if '_geninterp_' in func.func_globals: # skip all geninterped stuff From afa at codespeak.net Thu Nov 25 14:05:52 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 25 Nov 2010 14:05:52 +0100 (CET) Subject: [pypy-svn] r79512 - in pypy/trunk/pypy: . module/cpyext/test Message-ID: <20101125130552.E8A945081A@codespeak.net> Author: afa Date: Thu Nov 25 14:05:50 2010 New Revision: 79512 Modified: pypy/trunk/pypy/conftest.py pypy/trunk/pypy/module/cpyext/test/test_typeobject.py Log: Don't search for memory leaks when the test function is marked with "dont_track_allocations". Use it in the cpyext module, because I really don't know how to properly deallocate memory there. Modified: pypy/trunk/pypy/conftest.py ============================================================================== --- pypy/trunk/pypy/conftest.py (original) +++ pypy/trunk/pypy/conftest.py Thu Nov 25 14:05:50 2010 @@ -336,13 +336,15 @@ self.runtest_finish() def runtest_open(self): - leakfinder.start_tracking_allocations() + if not getattr(self.obj, 'dont_track_allocations', False): + leakfinder.start_tracking_allocations() def runtest_perform(self): super(PyPyTestFunction, self).runtest() def runtest_close(self): - if leakfinder.TRACK_ALLOCATIONS: + if (not getattr(self.obj, 'dont_track_allocations', False) + and leakfinder.TRACK_ALLOCATIONS): self._pypytest_leaks = leakfinder.stop_tracking_allocations(False) else: # stop_tracking_allocations() already called self._pypytest_leaks = None Modified: pypy/trunk/pypy/module/cpyext/test/test_typeobject.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/test/test_typeobject.py (original) +++ pypy/trunk/pypy/module/cpyext/test/test_typeobject.py Thu Nov 25 14:05:50 2010 @@ -206,6 +206,7 @@ del obj.x assert obj.z == prop + @py.test.mark.dont_track_allocations def test_tp_dict(self): foo = self.import_module("foo") module = self.import_extension('test', [ From david at codespeak.net Thu Nov 25 14:16:41 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 25 Nov 2010 14:16:41 +0100 (CET) Subject: [pypy-svn] r79513 - in pypy/branch/arm-backend/pypy/jit/backend/arm: . helper Message-ID: <20101125131641.CB225282BEB@codespeak.net> Author: david Date: Thu Nov 25 14:16:40 2010 New Revision: 79513 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Some register allocation fixes Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Thu Nov 25 14:16:40 2010 @@ -27,17 +27,17 @@ imm_a0 = self._check_imm_arg(arg0, imm_size, allow_zero=allow_zero) imm_a1 = self._check_imm_arg(arg1, imm_size, allow_zero=allow_zero) if commutative and imm_a0: - l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=imm_a0) + l0 = regalloc.make_sure_var_in_reg(arg0, [arg1], imm_fine=imm_a0) l1 = regalloc.make_sure_var_in_reg(arg1, [arg0]) res = regalloc.force_allocate_reg(op.result, [arg0, arg1]) ri_op(res.value, l1.value, imm=l0.getint(), cond=fcond) elif imm_a1: - l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) + l0 = regalloc.make_sure_var_in_reg(arg0, [arg1], imm_fine=False) l1 = regalloc.make_sure_var_in_reg(arg1, [arg0], imm_fine=True) res = regalloc.force_allocate_reg(op.result, [arg0, arg1]) ri_op(res.value, l0.value, imm=l1.getint(), cond=fcond) else: - l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) + l0 = regalloc.make_sure_var_in_reg(arg0, [arg1], imm_fine=False) l1 = regalloc.make_sure_var_in_reg(arg1, [arg0], imm_fine=False) res = regalloc.force_allocate_reg(op.result, [arg0, arg1]) rr_op(res.value, l0.value, l1.value) @@ -50,8 +50,8 @@ assert fcond is not None a0 = op.getarg(0) a1 = op.getarg(1) - arg1 = regalloc.make_sure_var_in_reg(a0, [a1], selected_reg=r.r0, imm_fine=False) - arg2 = regalloc.make_sure_var_in_reg(a1, [a0], selected_reg=r.r1, imm_fine=False) + arg1 = regalloc.make_sure_var_in_reg(a0, selected_reg=r.r0, imm_fine=False) + arg2 = regalloc.make_sure_var_in_reg(a1, selected_reg=r.r1, imm_fine=False) assert arg1 == r.r0 assert arg2 == r.r1 regalloc.before_call() @@ -78,13 +78,13 @@ imm_a0 = self._check_imm_arg(arg0) imm_a1 = self._check_imm_arg(arg1) if imm_a1 and not imm_a0: - l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) - l1 = regalloc.make_sure_var_in_reg(arg1, [l0]) + l0 = regalloc.make_sure_var_in_reg(arg0, [arg1], imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(arg1, [arg0]) res = regalloc.force_allocate_reg(op.result) self.mc.CMP_ri(l0.value, imm=l1.getint(), cond=fcond) else: - l0 = regalloc.make_sure_var_in_reg(arg0, imm_fine=False) - l1 = regalloc.make_sure_var_in_reg(arg1, [l0], imm_fine=False) + l0 = regalloc.make_sure_var_in_reg(arg0, [arg1], imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(arg1, [arg0], imm_fine=False) res = regalloc.force_allocate_reg(op.result) self.mc.CMP_rr(l0.value, l1.value, cond=fcond) Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Thu Nov 25 14:16:40 2010 @@ -49,9 +49,10 @@ l1 = regalloc.make_sure_var_in_reg(a1, forbidden_vars=[a0], imm_fine=False) res = regalloc.force_allocate_reg(op.result, forbidden_vars=[a0, a1]) self.mc.ADD_rr(res.value, l0.value, l1.value, s=1) - regalloc.possibly_free_vars_for_op(op) - if op.result: - regalloc.possibly_free_var(op.result) + + regalloc.possibly_free_var(a0) + regalloc.possibly_free_var(a1) + regalloc.possibly_free_var(op.result) return fcond def emit_op_int_sub(self, op, regalloc, fcond): @@ -81,9 +82,9 @@ else: self.mc.SUB_rr(res.value, l0.value, l1.value, s=1) - regalloc.possibly_free_vars_for_op(op) - if op.result: - regalloc.possibly_free_var(op.result) + regalloc.possibly_free_var(a0) + regalloc.possibly_free_var(a1) + regalloc.possibly_free_var(op.result) return fcond def emit_op_int_mul(self, op, regalloc, fcond): @@ -291,21 +292,20 @@ if n_args > 4: assert n > 0 self._adjust_sp(-n, regalloc, fcond=fcond) - regalloc.possibly_free_vars(locs) + regalloc.possibly_free_vars(args) return fcond def emit_op_same_as(self, op, regalloc, fcond): resloc = regalloc.force_allocate_reg(op.result) arg = op.getarg(0) - imm_arg = isinstance(arg, ConstInt) and (arg.getint() <= 0xFF or -1 * arg.getint() <= 0xFF) - argloc = regalloc.make_sure_var_in_reg(arg, imm_fine=imm_arg) + imm_arg = self._check_imm_arg(arg) + argloc = regalloc.make_sure_var_in_reg(arg, [op.result], imm_fine=imm_arg) if argloc.is_imm(): self.mc.MOV_ri(resloc.value, argloc.getint()) else: self.mc.MOV_rr(resloc.value, argloc.value) regalloc.possibly_free_vars_for_op(op) - if op.result: - regalloc.possibly_free_var(op.result) + regalloc.possibly_free_var(op.result) return fcond class FieldOpAssembler(object): @@ -488,8 +488,7 @@ res = regalloc.force_allocate_reg(op.result) regalloc.possibly_free_vars_for_op(op) regalloc.possibly_free_var(t) - if op.result: - regalloc.possibly_free_var(op.result) + regalloc.possibly_free_var(op.result) basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.STR, self.cpu.translate_support_code) @@ -531,8 +530,7 @@ l0 = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) regalloc.possibly_free_vars_for_op(op) res = regalloc.force_allocate_reg(op.result) - if op.result: - regalloc.possibly_free_var(op.result) + regalloc.possibly_free_var(op.result) basesize, itemsize, ofs_length = symbolic.get_array_token(rstr.UNICODE, self.cpu.translate_support_code) l1 = regalloc.make_sure_var_in_reg(ConstInt(ofs_length)) @@ -693,7 +691,7 @@ offset = self.mc.currpos() - jmp_pos pmc = ARMv7InMemoryBuilder(jmp_location, WORD) pmc.ADD_ri(r.pc.value, r.pc.value, offset - PC_OFFSET) - t = TempBox() + l0 = regalloc.force_allocate_reg(t) self.mc.LDR_ri(l0.value, r.fp.value) self.mc.CMP_ri(l0.value, 0) From antocuni at codespeak.net Thu Nov 25 14:23:22 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 25 Nov 2010 14:23:22 +0100 (CET) Subject: [pypy-svn] r79514 - pypy/trunk/pypy/jit/tool Message-ID: <20101125132322.8985E50818@codespeak.net> Author: antocuni Date: Thu Nov 25 14:23:20 2010 New Revision: 79514 Modified: pypy/trunk/pypy/jit/tool/log2gnumeric.py Log: nicely pass the cpython vmrss file as an option Modified: pypy/trunk/pypy/jit/tool/log2gnumeric.py ============================================================================== --- pypy/trunk/pypy/jit/tool/log2gnumeric.py (original) +++ pypy/trunk/pypy/jit/tool/log2gnumeric.py Thu Nov 25 14:23:20 2010 @@ -14,9 +14,10 @@ import re, sys import gzip +import optparse -def main(): +def main(logname, options): logname = sys.argv[1] outname = logname + '.gnumeric' data = open(logname).read() @@ -30,7 +31,7 @@ xml = replace_sheet(xml, 'gc-collect', gc_collect_rows(time0, data)) xml = replace_sheet(xml, 'loops', loops_rows(time0, data)) xml = replace_sheet(xml, 'vmrss', vmrss_rows(logname + '.vmrss', maxtime)) - xml = replace_sheet(xml, 'cpython-vmrss', vmrss_rows('cpython.vmrss', maxtime)) + xml = replace_sheet(xml, 'cpython-vmrss', vmrss_rows(options.cpython_vmrss, maxtime)) # out = gzip.open(outname, 'wb') out.write(xml) @@ -143,11 +144,12 @@ def vmrss_rows(filename, maxtime): - try: - lines = open(filename).readlines() - except IOError: - print 'Warning: cannot find file %s, skipping this sheet' - lines = [] + lines = [] + if options.cpython_vmrss: + try: + lines = open(filename).readlines() + except IOError: + print 'Warning: cannot find file %s, skipping this sheet' for row in vmrss_rows_impl(lines, maxtime): yield row @@ -161,4 +163,11 @@ if __name__ == '__main__': - main() + parser = optparse.OptionParser(usage="%prog logfile [options]") + parser.add_option('-c', '--cpython-vmrss', dest='cpython_vmrss', default=None, metavar='FILE', type=str, + help='the .vmrss file produced by CPython') + options, args = parser.parse_args() + if len(args) != 1: + parser.print_help() + sys.exit(2) + main(args[0], options) From david at codespeak.net Thu Nov 25 14:47:13 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 25 Nov 2010 14:47:13 +0100 (CET) Subject: [pypy-svn] r79515 - pypy/branch/arm-backend/pypy/jit/backend/arm/helper Message-ID: <20101125134713.0CF78282BF2@codespeak.net> Author: david Date: Thu Nov 25 14:47:11 2010 New Revision: 79515 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Log: Another regalloc fix Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/helper/assembler.py Thu Nov 25 14:47:11 2010 @@ -68,6 +68,7 @@ def gen_emit_cmp_op(condition, inverse=False): def f(self, op, regalloc, fcond): assert fcond is not None + args = op.getarglist() if not inverse: arg0 = op.getarg(0) arg1 = op.getarg(1) @@ -78,13 +79,13 @@ imm_a0 = self._check_imm_arg(arg0) imm_a1 = self._check_imm_arg(arg1) if imm_a1 and not imm_a0: - l0 = regalloc.make_sure_var_in_reg(arg0, [arg1], imm_fine=False) - l1 = regalloc.make_sure_var_in_reg(arg1, [arg0]) + l0 = regalloc.make_sure_var_in_reg(arg0, args, imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(arg1, args) res = regalloc.force_allocate_reg(op.result) self.mc.CMP_ri(l0.value, imm=l1.getint(), cond=fcond) else: - l0 = regalloc.make_sure_var_in_reg(arg0, [arg1], imm_fine=False) - l1 = regalloc.make_sure_var_in_reg(arg1, [arg0], imm_fine=False) + l0 = regalloc.make_sure_var_in_reg(arg0, args, imm_fine=False) + l1 = regalloc.make_sure_var_in_reg(arg1, args, imm_fine=False) res = regalloc.force_allocate_reg(op.result) self.mc.CMP_rr(l0.value, l1.value, cond=fcond) From antocuni at codespeak.net Thu Nov 25 15:06:52 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 25 Nov 2010 15:06:52 +0100 (CET) Subject: [pypy-svn] r79516 - in pypy/trunk/pypy/rlib: . test Message-ID: <20101125140652.6BE0950819@codespeak.net> Author: antocuni Date: Thu Nov 25 15:06:50 2010 New Revision: 79516 Modified: pypy/trunk/pypy/rlib/libffi.py pypy/trunk/pypy/rlib/test/test_libffi.py Log: check that the number of arguments if correct Modified: pypy/trunk/pypy/rlib/libffi.py ============================================================================== --- pypy/trunk/pypy/rlib/libffi.py (original) +++ pypy/trunk/pypy/rlib/libffi.py Thu Nov 25 15:06:50 2010 @@ -178,6 +178,9 @@ # the optimizer will fail to recognize the pattern and won't turn it # into a fast CALL. Note that "arg = arg.next" is optimized away, # assuming that archain is completely virtual. + if argchain.numargs != len(self.argtypes): + raise TypeError, 'Wrong number of arguments: %d expected, got %d' %\ + (argchain.numargs, len(self.argtypes)) ll_args = self._prepare() i = 0 arg = argchain.first Modified: pypy/trunk/pypy/rlib/test/test_libffi.py ============================================================================== --- pypy/trunk/pypy/rlib/test/test_libffi.py (original) +++ pypy/trunk/pypy/rlib/test/test_libffi.py Thu Nov 25 15:06:50 2010 @@ -262,3 +262,9 @@ # res = self.call(get_dummy, [], rffi.LONG) assert res == initval+1 + + def test_wrong_number_of_arguments(self): + libfoo = self.get_libfoo() + func = (libfoo, 'sum_xy', [types.sint, types.double], types.sint) + py.test.raises(TypeError, "self.call(func, [38], rffi.LONG)") # one less + py.test.raises(TypeError, "self.call(func, [38, 12.3, 42], rffi.LONG)") # one more From antocuni at codespeak.net Thu Nov 25 15:36:56 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Thu, 25 Nov 2010 15:36:56 +0100 (CET) Subject: [pypy-svn] r79517 - pypy/trunk/pypy/jit/tool Message-ID: <20101125143656.1E0B5282B9E@codespeak.net> Author: antocuni Date: Thu Nov 25 15:36:55 2010 New Revision: 79517 Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric pypy/trunk/pypy/jit/tool/log2gnumeric.py Log: use "giga-ticks" instead of "ticks" Modified: pypy/trunk/pypy/jit/tool/log-template.gnumeric ============================================================================== Binary files. No diff available. Modified: pypy/trunk/pypy/jit/tool/log2gnumeric.py ============================================================================== --- pypy/trunk/pypy/jit/tool/log2gnumeric.py (original) +++ pypy/trunk/pypy/jit/tool/log2gnumeric.py Thu Nov 25 15:36:55 2010 @@ -81,10 +81,15 @@ # functions to extract various data from the logs # ======================================================================== +CLOCK_FACTOR = 1 +def read_clock(x): + timestamp = int(x, 16) + return timestamp / CLOCK_FACTOR + def get_clock_range(data): s = r"\[([0-9a-f]+)\] " r = re.compile(s) - clocks = [int(x, 16) for x in r.findall(data)] + clocks = [read_clock(x) for x in r.findall(data)] return min(clocks), max(clocks) def gc_collect_rows(time0, data): @@ -103,7 +108,7 @@ r = re.compile(s.replace('\n', '')) yield 'clock', 'gc-before', 'gc-after' for a,b,c,d,e,f in r.findall(data): - clock = int(f, 16) - time0 + clock = read_clock(f) - time0 yield clock, int(a)+int(b), int(c)+int(d) def tasks_rows(time0, data): @@ -115,7 +120,7 @@ r = re.compile(s.replace('\n', '')) yield 'clock', None, 'task' for a,b in r.findall(data): - clock = int(a, 16) - time0 + clock = read_clock(a) - time0 yield clock, 1, b @@ -129,8 +134,9 @@ yield 'clock', 'total', 'loops', 'bridges' loops = 0 bridges = 0 + fake_total = 0 for clock, action, text in r.findall(data): - clock = int(clock, 16) - time0 + clock = read_clock(clock) - time0 if text.startswith('allocating Loop #'): loops += 1 elif text.startswith('allocating Bridge #'): @@ -163,6 +169,7 @@ if __name__ == '__main__': + CLOCK_FACTOR = 1000000000.0 # report GigaTicks instead of Ticks parser = optparse.OptionParser(usage="%prog logfile [options]") parser.add_option('-c', '--cpython-vmrss', dest='cpython_vmrss', default=None, metavar='FILE', type=str, help='the .vmrss file produced by CPython') From david at codespeak.net Thu Nov 25 15:43:09 2010 From: david at codespeak.net (david at codespeak.net) Date: Thu, 25 Nov 2010 15:43:09 +0100 (CET) Subject: [pypy-svn] r79518 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101125144309.E7BEE282BE3@codespeak.net> Author: david Date: Thu Nov 25 15:43:03 2010 New Revision: 79518 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Log: Add ptr_eq and ptr_ne operations Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Thu Nov 25 15:43:03 2010 @@ -404,12 +404,13 @@ self.mc._dump_trace(name) def _check_imm_arg(self, arg, size=0xFF, allow_zero=True): - if allow_zero: - lower_bound = arg.getint() >= 0 - else: - lower_bound = arg.getint() > 0 - #XXX check ranges for different operations - return isinstance(arg, ConstInt) and arg.getint() <= size and lower_bound + if isinstance(arg, ConstInt): + if allow_zero: + lower_bound = arg.getint() >= 0 + else: + lower_bound = arg.getint() > 0 + return arg.getint() <= size and lower_bound + return False def _ensure_result_bit_extension(self, resloc, size, signed, regalloc): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Thu Nov 25 15:43:03 2010 @@ -142,6 +142,9 @@ emit_op_int_add_ovf = emit_op_int_add emit_op_int_sub_ovf = emit_op_int_sub + emit_op_ptr_eq = emit_op_int_eq + emit_op_ptr_ne = emit_op_int_ne + class UnaryIntOpAssembler(object): From arigo at codespeak.net Thu Nov 25 15:43:28 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 15:43:28 +0100 (CET) Subject: [pypy-svn] r79519 - in pypy/release/1.4.x/pypy/module: itertools pypyjit Message-ID: <20101125144328.E57EC5080C@codespeak.net> Author: arigo Date: Thu Nov 25 15:43:27 2010 New Revision: 79519 Modified: pypy/release/1.4.x/pypy/module/itertools/interp_itertools.py pypy/release/1.4.x/pypy/module/pypyjit/policy.py Log: Merge r79505, r79506, r79508 from trunk: add 'operator' and 'itertools' to the jit. Modified: pypy/release/1.4.x/pypy/module/itertools/interp_itertools.py ============================================================================== --- pypy/release/1.4.x/pypy/module/itertools/interp_itertools.py (original) +++ pypy/release/1.4.x/pypy/module/itertools/interp_itertools.py Thu Nov 25 15:43:27 2010 @@ -391,7 +391,8 @@ self.iterators_w = iterators_w self.current_iterator = 0 self.num_iterators = len(iterators_w) - self.started = False + if self.num_iterators > 0: + self.w_it = iterators_w[0] def iter_w(self): return self.space.wrap(self) @@ -399,26 +400,23 @@ def next_w(self): if self.current_iterator >= self.num_iterators: raise OperationError(self.space.w_StopIteration, self.space.w_None) - if not self.started: - self.current_iterator = 0 - self.w_it = self.iterators_w[self.current_iterator] - self.started = True + try: + return self.space.next(self.w_it) + except OperationError, e: + return self._handle_error(e) + def _handle_error(self, e): while True: + if not e.match(self.space, self.space.w_StopIteration): + raise e + self.current_iterator += 1 + if self.current_iterator >= self.num_iterators: + raise e + self.w_it = self.iterators_w[self.current_iterator] try: - w_obj = self.space.next(self.w_it) + return self.space.next(self.w_it) except OperationError, e: - if e.match(self.space, self.space.w_StopIteration): - self.current_iterator += 1 - if self.current_iterator >= self.num_iterators: - raise OperationError(self.space.w_StopIteration, self.space.w_None) - else: - self.w_it = self.iterators_w[self.current_iterator] - else: - raise - else: - break - return w_obj + pass # loop back to the start of _handle_error(e) def W_Chain___new__(space, w_subtype, args_w): return space.wrap(W_Chain(space, args_w)) @@ -446,8 +444,10 @@ def __init__(self, space, w_fun, args_w): self.space = space - self.identity_fun = (self.space.is_w(w_fun, space.w_None)) - self.w_fun = w_fun + if self.space.is_w(w_fun, space.w_None): + self.w_fun = None + else: + self.w_fun = w_fun iterators_w = [] i = 0 @@ -470,12 +470,26 @@ return self.space.wrap(self) def next_w(self): - w_objects = self.space.newtuple([self.space.next(w_it) for w_it in self.iterators_w]) - if self.identity_fun: + # common case: 1 or 2 arguments + iterators_w = self.iterators_w + length = len(iterators_w) + if length == 1: + objects = [self.space.next(iterators_w[0])] + elif length == 2: + objects = [self.space.next(iterators_w[0]), + self.space.next(iterators_w[1])] + else: + objects = self._get_objects() + w_objects = self.space.newtuple(objects) + if self.w_fun is None: return w_objects else: return self.space.call(self.w_fun, w_objects) + def _get_objects(self): + # the loop is out of the way of the JIT + return [self.space.next(w_elem) for w_elem in self.iterators_w] + def W_IMap___new__(space, w_subtype, w_fun, args_w): if len(args_w) == 0: @@ -769,15 +783,7 @@ raise OperationError(self.space.w_StopIteration, self.space.w_None) if not self.new_group: - # Consume unwanted input until we reach the next group - try: - while True: - self.group_next(self.index) - - except StopIteration: - pass - if self.exhausted: - raise OperationError(self.space.w_StopIteration, self.space.w_None) + self._consume_unwanted_input() if not self.started: self.started = True @@ -799,6 +805,16 @@ w_iterator = self.space.wrap(W_GroupByIterator(self.space, self.index, self)) return self.space.newtuple([self.w_key, w_iterator]) + def _consume_unwanted_input(self): + # Consume unwanted input until we reach the next group + try: + while True: + self.group_next(self.index) + except StopIteration: + pass + if self.exhausted: + raise OperationError(self.space.w_StopIteration, self.space.w_None) + def group_next(self, group_index): if group_index < self.index: raise StopIteration Modified: pypy/release/1.4.x/pypy/module/pypyjit/policy.py ============================================================================== --- pypy/release/1.4.x/pypy/module/pypyjit/policy.py (original) +++ pypy/release/1.4.x/pypy/module/pypyjit/policy.py Thu Nov 25 15:43:27 2010 @@ -12,7 +12,7 @@ if '.' in modname: modname, _ = modname.split('.', 1) if modname in ['pypyjit', 'signal', 'micronumpy', 'math', 'exceptions', - 'imp', 'sys', 'array', '_ffi']: + 'imp', 'sys', 'array', '_ffi', 'itertools', 'operator']: return True return False From arigo at codespeak.net Thu Nov 25 15:51:19 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Thu, 25 Nov 2010 15:51:19 +0100 (CET) Subject: [pypy-svn] r79520 - pypy/trunk/pypy/config Message-ID: <20101125145119.8C3D4282B9D@codespeak.net> Author: arigo Date: Thu Nov 25 15:51:15 2010 New Revision: 79520 Modified: pypy/trunk/pypy/config/pypyoption.py Log: Disabling geninterp seems to work (again; see r41262). Re-enable the option, which is '--no-objspace-geninterp'. Modified: pypy/trunk/pypy/config/pypyoption.py ============================================================================== --- pypy/trunk/pypy/config/pypyoption.py (original) +++ pypy/trunk/pypy/config/pypyoption.py Thu Nov 25 15:51:15 2010 @@ -161,7 +161,6 @@ suggests=[("objspace.allworkingmodules", False)]), BoolOption("geninterp", "specify whether geninterp should be used", - cmdline=None, default=True), BoolOption("logbytecodes", From cfbolz at codespeak.net Thu Nov 25 17:23:26 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 25 Nov 2010 17:23:26 +0100 (CET) Subject: [pypy-svn] r79521 - in pypy/branch/reflex-support/pypy: module/cppyy rlib Message-ID: <20101125162326.87D08282BFD@codespeak.net> Author: cfbolz Date: Thu Nov 25 17:23:22 2010 New Revision: 79521 Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py pypy/branch/reflex-support/pypy/module/cppyy/executor.py pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py pypy/branch/reflex-support/pypy/rlib/libffi.py Log: Support for much faster calls in cppyy: use rlib.libffi to do the calls, which should enable the JIT to optimize them into direct calls without any allocation. This is so far restricted to some argument/return types, but can be extended. Modified: pypy/branch/reflex-support/pypy/module/cppyy/converter.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/converter.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/converter.py Thu Nov 25 17:23:22 2010 @@ -3,7 +3,7 @@ from pypy.interpreter.buffer import Buffer from pypy.rpython.lltypesystem import rffi, lltype from pypy.rlib.rarithmetic import r_singlefloat -from pypy.rlib import jit +from pypy.rlib import jit, libffi from pypy.module._rawffi.interp_rawffi import unpack_simple_shape from pypy.module._rawffi.array import W_Array @@ -20,6 +20,8 @@ class TypeConverter(object): + libffitype = libffi.types.NULL + def __init__(self, space, array_size): pass @@ -35,6 +37,10 @@ def convert_argument(self, space, w_obj): self._is_abstract() + def convert_argument_libffi(self, space, w_obj, argchain): + from pypy.module.cppyy.interp_cppyy import FastCallNotPossible + raise FastCallNotPossible + def from_memory(self, space, w_obj, offset): self._is_abstract() @@ -55,6 +61,8 @@ class VoidConverter(TypeConverter): + libffitype = libffi.types.void + def __init__(self, space, name): self.name = name @@ -91,6 +99,8 @@ fieldptr[0] = '\x00' class CharConverter(TypeConverter): + libffitype = libffi.types.schar + def _from_space(self, space, w_value): # allow int to pass to char and make sure that str is of length 1 if space.isinstance_w(w_value, space.w_int): @@ -122,12 +132,20 @@ fieldptr[0] = self._from_space(space, w_value) class LongConverter(TypeConverter): + libffitype = libffi.types.slong + + def _unwrap_object(self, space, w_obj): + return space.c_int_w(w_obj) + def convert_argument(self, space, w_obj): - arg = space.c_int_w(w_obj) + arg = self._unwrap_object(space, w_obj) x = lltype.malloc(rffi.LONGP.TO, 1, flavor='raw') x[0] = arg return rffi.cast(rffi.VOIDP, x) + def convert_argument_libffi(self, space, w_obj, argchain): + argchain.arg(self._unwrap_object(space, w_obj)) + def from_memory(self, space, w_obj, offset): fieldptr = self._get_fieldptr(space, w_obj, offset) longptr = rffi.cast(rffi.LONGP, fieldptr) @@ -139,6 +157,8 @@ longptr[0] = space.c_int_w(w_value) class ShortConverter(LongConverter): + libffitype = libffi.types.sshort + def from_memory(self, space, w_obj, offset): fieldptr = self._get_fieldptr(space, w_obj, offset) shortptr = rffi.cast(rffi.SHORTP, fieldptr) @@ -150,11 +170,13 @@ shortptr[0] = rffi.cast(rffi.SHORT, space.c_int_w(w_value)) class FloatConverter(TypeConverter): + def _unwrap_object(self, space, w_obj): + return r_singlefloat(space.float_w(w_obj)) + def convert_argument(self, space, w_obj): - arg = space.float_w(w_obj) x = lltype.malloc(rffi.FLOATP.TO, 1, flavor='raw') - x[0] = r_singlefloat(arg) - return rffi.cast(rffi.VOIDP, x) + x[0] = self._unwrap_object(space, w_obj) + return rffi.cast(rffi.VOIDP, x) def from_memory(self, space, w_obj, offset): fieldptr = self._get_fieldptr(space, w_obj, offset) @@ -167,11 +189,18 @@ floatptr[0] = r_singlefloat(space.float_w(w_value)) class DoubleConverter(TypeConverter): + libffitype = libffi.types.double + + def _unwrap_object(self, space, w_obj): + return space.float_w(w_obj) + def convert_argument(self, space, w_obj): - arg = space.float_w(w_obj) x = lltype.malloc(rffi.DOUBLEP.TO, 1, flavor='raw') - x[0] = arg - return rffi.cast(rffi.VOIDP, x) + x[0] = self._unwrap_object(space, w_obj) + return rffi.cast(rffi.VOIDP, x) + + def convert_argument_libffi(self, space, w_obj, argchain): + argchain.arg(self._unwrap_object(space, w_obj)) def from_memory(self, space, w_obj, offset): fieldptr = self._get_fieldptr(space, w_obj, offset) Modified: pypy/branch/reflex-support/pypy/module/cppyy/executor.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/executor.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/executor.py Thu Nov 25 17:23:22 2010 @@ -1,4 +1,5 @@ from pypy.rpython.lltypesystem import rffi, lltype +from pypy.rlib import libffi from pypy.module.cppyy import helper, capi @@ -7,19 +8,29 @@ class FunctionExecutor(object): _immutable_ = True + libffitype = libffi.types.NULL + def __init__(self, space, cpptype): pass def execute(self, space, func, cppthis, num_args, args): raise NotImplementedError + def execute_libffi(self, space, libffifunc, argchain): + from pypy.module.cppyy.interp_cppyy import FastCallNotPossible + raise FastCallNotPossible + class VoidExecutor(FunctionExecutor): _immutable_ = True + def execute(self, space, func, cppthis, num_args, args): capi.c_call_v(func.cpptype.handle, func.method_index, cppthis, num_args, args) return space.w_None + def execute_libffi(self, space, libffifunc, argchain): + libffifunc.call(argchain, lltype.Void) + return space.w_None class BoolExecutor(FunctionExecutor): _immutable_ = True @@ -29,34 +40,48 @@ class CharExecutor(FunctionExecutor): _immutable_ = True + def execute(self, space, func, cppthis, num_args, args): result = capi.c_call_c(func.cpptype.handle, func.method_index, cppthis, num_args, args) return space.wrap(result) class ShortExecutor(FunctionExecutor): _immutable_ = True + libffitype = libffi.types.sshort + def execute(self, space, func, cppthis, num_args, args): result = capi.c_call_h(func.cpptype.handle, func.method_index, cppthis, num_args, args) return space.wrap(result) class LongExecutor(FunctionExecutor): _immutable_ = True + libffitype = libffi.types.slong + def execute(self, space, func, cppthis, num_args, args): result = capi.c_call_l(func.cpptype.handle, func.method_index, cppthis, num_args, args) return space.wrap(result) + def execute_libffi(self, space, libffifunc, argchain): + return space.wrap(libffifunc.call(argchain, lltype.Signed)) + class FloatExecutor(FunctionExecutor): _immutable_ = True + def execute(self, space, func, cppthis, num_args, args): result = capi.c_call_f(func.cpptype.handle, func.method_index, cppthis, num_args, args) return space.wrap(result) class DoubleExecutor(FunctionExecutor): _immutable_ = True + libffitype = libffi.types.double + def execute(self, space, func, cppthis, num_args, args): result = capi.c_call_d(func.cpptype.handle, func.method_index, cppthis, num_args, args) return space.wrap(result) + def execute_libffi(self, space, libffifunc, argchain): + return space.wrap(libffifunc.call(argchain, rffi.DOUBLE)) + class CStringExecutor(FunctionExecutor): _immutable_ = True def execute(self, space, func, cppthis, num_args, args): Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Thu Nov 25 17:23:22 2010 @@ -7,18 +7,18 @@ from pypy.rpython.lltypesystem import rffi, lltype -from pypy.rlib.libffi import CDLL +from pypy.rlib import libffi from pypy.rlib import jit, debug from pypy.module.cppyy import converter, executor -class HackCallNotPossible(Exception): +class FastCallNotPossible(Exception): pass NULL_VOIDP = lltype.nullptr(rffi.VOIDP.TO) def load_lib(space, name): - cdll = CDLL(name) + cdll = libffi.CDLL(name) return W_CPPLibrary(space, cdll) load_lib.unwrap_spec = [ObjSpace, str] @@ -72,18 +72,21 @@ self.arg_types = arg_types self.executor = executor.get_executor(self.space, result_type) self.arg_converters = None - # - self.hack_call = arg_types == ['int'] and result_type == 'int' - # + methgetter = get_methptr_getter(self.cpptype.handle, + self.method_index) + self.methgetter = methgetter + self._libffifunc_cache = {} def call(self, cppthis, args_w): if self.executor is None: raise OperationError(self.space.w_TypeError, self.space.wrap("return type not handled")) - if self.hack_call: + if self.methgetter and cppthis: # only for methods try: - return self.do_hack_call(cppthis, args_w) - except HackCallNotPossible: + print "trying fast call" + return self.do_fast_call(cppthis, args_w) + except FastCallNotPossible: + print "failed" pass args = self.prepare_arguments(args_w) @@ -92,24 +95,44 @@ finally: self.free_arguments(args) - INT_2_INT_FNPTR = lltype.Ptr(lltype.FuncType([rffi.VOIDP, rffi.INT], - rffi.INT)) - def do_hack_call(self, cppthis, args_w): - # hack: only for methods 'int m(int)' + @jit.unroll_safe + def do_fast_call(self, cppthis, args_w): space = self.space - if len(args_w) != 1: + # XXX factor out + if len(args_w) != len(self.arg_types): raise OperationError(space.w_TypeError, space.wrap("wrong number of args")) - arg = space.c_int_w(args_w[0]) - methgetter = get_methptr_getter(self.cpptype.handle, - self.method_index) - if not methgetter: - raise HackCallNotPossible - funcptr = methgetter(cppthis) - - funcptr = rffi.cast(self.INT_2_INT_FNPTR, funcptr) - funcptr = jit.hint(funcptr, promote=True) - result = funcptr(cppthis, arg) - return space.wrap(rffi.cast(lltype.Signed, result)) + if self.arg_converters is None: + self._build_converters() + funcptr = jit.hint(self.methgetter, promote=True)(cppthis) + libffi_func = self._get_libffi_func(jit.hint(funcptr, promote=True)) + if not libffi_func: + raise FastCallNotPossible + + argchain = libffi.ArgChain() + argchain.arg(cppthis) + for i in range(len(args_w)): + conv = self.arg_converters[i] + w_arg = args_w[i] + conv.convert_argument_libffi(space, w_arg, argchain) + return self.executor.execute_libffi(space, libffi_func, argchain) + + @jit.purefunction + def _get_libffi_func(self, funcptr): + key = rffi.cast(rffi.LONG, funcptr) + if key in self._libffifunc_cache: + return self._libffifunc_cache[key] + argtypes_libffi = [conv.libffitype for conv in self.arg_converters + if conv.libffitype] + if (len(argtypes_libffi) == len(self.arg_converters) and + self.executor.libffitype): + # add c++ this to the arguments + libffifunc = libffi.Func("XXX", + [libffi.types.pointer] + argtypes_libffi, + self.executor.libffitype, funcptr) + else: + libffifunc = None + self._libffifunc_cache[key] = libffifunc + return libffifunc def _build_converters(self): self.arg_converters = [converter.get_converter(self.space, arg_type) @@ -127,7 +150,7 @@ conv = self.arg_converters[i] w_arg = args_w[i] try: - arg = conv.convert_argument(space, w_arg) + arg = conv.convert_argument(space, w_arg) except: # fun :-( for j in range(i): @@ -148,6 +171,9 @@ return "CPPFunction(%s, %s, %r, %s)" % ( self.cpptype, self.method_index, self.executor, self.arg_types) + def _freeze_(self): + assert 0, "you should never have a pre-built instance of this!" + class CPPFunction(CPPMethod): def call(self, cppthis, args_w): Modified: pypy/branch/reflex-support/pypy/rlib/libffi.py ============================================================================== --- pypy/branch/reflex-support/pypy/rlib/libffi.py (original) +++ pypy/branch/reflex-support/pypy/rlib/libffi.py Thu Nov 25 17:23:22 2010 @@ -64,6 +64,8 @@ ## elif ffi_type is types.uint64: return 'u' raise KeyError + NULL = lltype.nullptr(clibffi.FFI_TYPE_P.TO) + types._import() @specialize.arg(0) @@ -149,7 +151,7 @@ _immutable_fields_ = ['funcsym'] argtypes = [] - restype = lltype.nullptr(clibffi.FFI_TYPE_P.TO) + restype = types.NULL funcsym = lltype.nullptr(rffi.VOIDP.TO) def __init__(self, name, argtypes, restype, funcsym, flags=FUNCFLAG_CDECL, @@ -178,6 +180,7 @@ # the optimizer will fail to recognize the pattern and won't turn it # into a fast CALL. Note that "arg = arg.next" is optimized away, # assuming that archain is completely virtual. + self = jit.hint(self, promote=True) ll_args = self._prepare() i = 0 arg = argchain.first From cfbolz at codespeak.net Thu Nov 25 17:31:33 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Thu, 25 Nov 2010 17:31:33 +0100 (CET) Subject: [pypy-svn] r79522 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101125163133.8F60D50810@codespeak.net> Author: cfbolz Date: Thu Nov 25 17:31:31 2010 New Revision: 79522 Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Log: remove debug prints Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Thu Nov 25 17:31:31 2010 @@ -83,10 +83,8 @@ if self.methgetter and cppthis: # only for methods try: - print "trying fast call" return self.do_fast_call(cppthis, args_w) except FastCallNotPossible: - print "failed" pass args = self.prepare_arguments(args_w) From afa at codespeak.net Thu Nov 25 19:31:51 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 25 Nov 2010 19:31:51 +0100 (CET) Subject: [pypy-svn] r79527 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101125183151.C12F2282B9E@codespeak.net> Author: afa Date: Thu Nov 25 19:31:49 2010 New Revision: 79527 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py Log: Correctly retrieve the objects to keep alive when storing a c_char_p in an array. This is needed by code like: array = (c_char_p * 5)() array[0] = 'foo bar' print array[0] import gc; gc.collect() print array[0] # used to print garbage This fixes all CPython tests in ctypes/test/test_objects.py. Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/array.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/array.py Thu Nov 25 19:31:49 2010 @@ -172,9 +172,10 @@ self._slice_setitem(index, value) return index = self._fix_index(index) - if ensure_objects(value) is not None: - store_reference(self, index, value._objects) - arg = self._type_._CData_value(value) + cobj = self._type_.from_param(value) + if ensure_objects(cobj) is not None: + store_reference(self, index, cobj._objects) + arg = cobj._get_buffer_value() if self._type_._fficompositesize is None: self._buffer[index] = arg # something more sophisticated, cannot set field directly @@ -193,7 +194,7 @@ return self._length_ def _get_buffer_for_param(self): - return CArgObject(self._buffer.byptr()) + return CArgObject(self, self._buffer.byptr()) def _get_buffer_value(self): return self._buffer.buffer Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py Thu Nov 25 19:31:49 2010 @@ -92,13 +92,17 @@ """ simple wrapper around buffer, just for the case of freeing it afterwards """ - def __init__(self, buffer): + def __init__(self, obj, buffer): + self._obj = obj self._buffer = buffer def __del__(self): self._buffer.free() self._buffer = None + def __repr__(self): + return repr(self._obj) + class _CData(object): """ The most basic object for all ctypes types """ @@ -128,7 +132,7 @@ return buffer(self._buffer) def _get_b_base(self): - return self._objects + return self._base _b_base_ = property(_get_b_base) _b_needsfree_ = False Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py Thu Nov 25 19:31:49 2010 @@ -134,8 +134,8 @@ ConvMode.errors) #self._objects = value array = _rawffi.Array('c')(len(value)+1, value) + self._objects = CArgObject(value, array) value = array.buffer - self._objects = {'0': CArgObject(array)} elif value is None: value = 0 self._buffer[0] = value @@ -156,8 +156,8 @@ ConvMode.errors) #self._objects = value array = _rawffi.Array('u')(len(value)+1, value) + self._objects = CArgObject(value, array) value = array.buffer - self._objects = {'0': CArgObject(array)} elif value is None: value = 0 self._buffer[0] = value @@ -175,8 +175,8 @@ def _setvalue(self, value): if isinstance(value, str): array = _rawffi.Array('c')(len(value)+1, value) + self._objects = CArgObject(value, array) value = array.buffer - self._objects = {'0': CArgObject(array)} elif value is None: value = 0 self._buffer[0] = value From afa at codespeak.net Thu Nov 25 19:49:41 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 25 Nov 2010 19:49:41 +0100 (CET) Subject: [pypy-svn] r79528 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101125184941.B40505080F@codespeak.net> Author: afa Date: Thu Nov 25 19:49:37 2010 New Revision: 79528 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py Log: Fix repr() for subclasses of simple types, and a hack to make the tests pass: CArgObject looks more like the object itself. Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py Thu Nov 25 19:49:37 2010 @@ -103,6 +103,9 @@ def __repr__(self): return repr(self._obj) + def __eq__(self, other): + return self._obj == other + class _CData(object): """ The most basic object for all ctypes types """ Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py Thu Nov 25 19:49:37 2010 @@ -316,7 +316,11 @@ return self.value def __repr__(self): - return "%s(%r)" % (type(self).__name__, self.value) + if type(self).__bases__[0] is _SimpleCData: + return "%s(%r)" % (type(self).__name__, self.value) + else: + return "<%s object at 0x%x>" % (type(self).__name__, + id(self)) def __nonzero__(self): return self._buffer[0] not in (0, '\x00') From afa at codespeak.net Thu Nov 25 19:57:51 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Thu, 25 Nov 2010 19:57:51 +0100 (CET) Subject: [pypy-svn] r79529 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101125185751.0C3F4282BEB@codespeak.net> Author: afa Date: Thu Nov 25 19:57:50 2010 New Revision: 79529 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py Log: Fix the order of fields for inherited Structures Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py Thu Nov 25 19:57:50 2010 @@ -41,9 +41,10 @@ tp._make_final() import ctypes all_fields = [] - for cls in inspect.getmro(superclass): - all_fields += getattr(cls, '_fields_', []) - all_fields += _fields_ + for cls in reversed(inspect.getmro(superclass)): + # The first field comes from the most base class + all_fields.extend(getattr(cls, '_fields_', [])) + all_fields.extend(_fields_) names = [f[0] for f in all_fields] rawfields = [(f[0], f[1]._ffishape) for f in all_fields] From afa at codespeak.net Fri Nov 26 00:15:09 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 26 Nov 2010 00:15:09 +0100 (CET) Subject: [pypy-svn] r79530 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101125231509.A93D9282B9D@codespeak.net> Author: afa Date: Fri Nov 26 00:15:06 2010 New Revision: 79530 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py Log: a result type of c_int returns an int value, but a subclass of c_int is returned as is. Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/primitive.py Fri Nov 26 00:15:06 2010 @@ -271,7 +271,9 @@ def _CData_output(self, resbuffer, base=None, index=-1): output = super(SimpleType, self)._CData_output(resbuffer, base, index) - return output.value + if self.__bases__[0] is _SimpleCData: + return output.value + return output def _sizeofinstances(self): return _rawffi.sizeof(self._type_) From afa at codespeak.net Fri Nov 26 00:36:32 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 26 Nov 2010 00:36:32 +0100 (CET) Subject: [pypy-svn] r79531 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101125233632.E00EE282B9D@codespeak.net> Author: afa Date: Fri Nov 26 00:36:31 2010 New Revision: 79531 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/function.py Log: Implement functptr.errcheck: called after the function call, can modify the result or raise an exception. Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/function.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/function.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/function.py Fri Nov 26 00:36:31 2010 @@ -35,6 +35,7 @@ _argtypes_ = None _restype_ = None + _errcheck_ = None _flags_ = 0 _ffiargshape = 'P' _ffishape = 'P' @@ -72,6 +73,19 @@ del self._restype_ restype = property(_getrestype, _setrestype, _delrestype) + def _geterrcheck(self): + return getattr(self, '_errcheck_', None) + def _seterrcheck(self, errcheck): + if not callable(errcheck): + raise TypeError("The errcheck attribute must be callable") + self._errcheck_ = errcheck + def _delerrcheck(self): + try: + del self._errcheck_ + except AttributeError: + pass + errcheck = property(_geterrcheck, _seterrcheck, _delerrcheck) + def _ffishapes(self, args, restype): argtypes = [arg._ffiargshape for arg in args] if restype is not None: @@ -168,7 +182,21 @@ restype = self._restype_ funcptr = self._getfuncptr(argtypes, restype, thisarg) resbuffer = funcptr(*[arg._buffer for _, arg in argsandobjs]) - return self._build_result(restype, resbuffer, argtypes, argsandobjs) + result = self._build_result(restype, resbuffer, argtypes, argsandobjs) + + # The 'errcheck' protocol + if self._errcheck_: + v = self._errcheck_(result, self, args) + # If the errcheck funtion failed, let it throw + # If the errcheck function returned callargs unchanged, + # continue normal processing. + # If the errcheck function returned something else, + # use that as result. + if v is not args: + result = v + + return result + def _getfuncptr(self, argtypes, restype, thisarg=None): if self._ptr is not None and argtypes is self._argtypes_: From afa at codespeak.net Fri Nov 26 00:46:02 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 26 Nov 2010 00:46:02 +0100 (CET) Subject: [pypy-svn] r79532 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101125234602.32B5036C220@codespeak.net> Author: afa Date: Fri Nov 26 00:46:00 2010 New Revision: 79532 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/function.py Log: Callbacks may be called with more arguments than declared. Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/function.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/function.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/function.py Fri Nov 26 00:46:00 2010 @@ -154,6 +154,7 @@ def __call__(self, *args): if self.callable is not None: + args = args[:len(self._argtypes_)] try: res = self.callable(*args) except: From antocuni at codespeak.net Fri Nov 26 10:58:33 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 26 Nov 2010 10:58:33 +0100 (CET) Subject: [pypy-svn] r79536 - in pypy/trunk/pypy/translator: c/gcc platform Message-ID: <20101126095833.84F91282B90@codespeak.net> Author: antocuni Date: Fri Nov 26 10:58:31 2010 New Revision: 79536 Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py pypy/trunk/pypy/translator/platform/darwin.py Log: (mvt) fix os/x 64 bit translation Modified: pypy/trunk/pypy/translator/c/gcc/trackgcroot.py ============================================================================== --- pypy/trunk/pypy/translator/c/gcc/trackgcroot.py (original) +++ pypy/trunk/pypy/translator/c/gcc/trackgcroot.py Fri Nov 26 10:58:31 2010 @@ -1106,7 +1106,7 @@ format = 'darwin64' function_names_prefix = '_' - LABEL = ElfFunctionGcRootTracker32.LABEL + LABEL = ElfFunctionGcRootTracker64.LABEL r_jmptable_item = re.compile(r"\t.(?:long|quad)\t"+LABEL+"(-\"?[A-Za-z0-9$]+\"?)?\s*$") r_functionstart = re.compile(r"_(\w+):\s*$") Modified: pypy/trunk/pypy/translator/platform/darwin.py ============================================================================== --- pypy/trunk/pypy/translator/platform/darwin.py (original) +++ pypy/trunk/pypy/translator/platform/darwin.py Fri Nov 26 10:58:31 2010 @@ -12,12 +12,14 @@ so_ext = 'so' + default_cc = 'gcc' + def __init__(self, cc=None): if cc is None: try: cc = os.environ['CC'] except KeyError: - cc = 'gcc' + cc = default_cc self.cc = cc def _args_for_shared(self, args): @@ -85,3 +87,4 @@ link_flags = ('-arch', 'x86_64', '-mmacosx-version-min=10.4') cflags = ('-arch', 'x86_64', '-O3', '-fomit-frame-pointer', '-mmacosx-version-min=10.4') + default_cc = 'gcc-4.0' From antocuni at codespeak.net Fri Nov 26 11:17:19 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 26 Nov 2010 11:17:19 +0100 (CET) Subject: [pypy-svn] r79541 - pypy/trunk/pypy/translator/platform Message-ID: <20101126101719.96FDA282B90@codespeak.net> Author: antocuni Date: Fri Nov 26 11:17:18 2010 New Revision: 79541 Modified: pypy/trunk/pypy/translator/platform/darwin.py Log: (mvt) hopefully *really* fix translation :-) Modified: pypy/trunk/pypy/translator/platform/darwin.py ============================================================================== --- pypy/trunk/pypy/translator/platform/darwin.py (original) +++ pypy/trunk/pypy/translator/platform/darwin.py Fri Nov 26 11:17:18 2010 @@ -19,7 +19,7 @@ try: cc = os.environ['CC'] except KeyError: - cc = default_cc + cc = self.default_cc self.cc = cc def _args_for_shared(self, args): From arigo at codespeak.net Fri Nov 26 13:18:17 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 26 Nov 2010 13:18:17 +0100 (CET) Subject: [pypy-svn] r79545 - pypy/trunk/pypy/doc Message-ID: <20101126121817.6126D282B90@codespeak.net> Author: arigo Date: Fri Nov 26 13:18:15 2010 New Revision: 79545 Modified: pypy/trunk/pypy/doc/cpython_differences.txt Log: Update the list of modules. Modified: pypy/trunk/pypy/doc/cpython_differences.txt ============================================================================== --- pypy/trunk/pypy/doc/cpython_differences.txt (original) +++ pypy/trunk/pypy/doc/cpython_differences.txt Fri Nov 26 13:18:15 2010 @@ -20,16 +20,21 @@ __builtin__ `__pypy__`_ + _ast + _bisect _codecs _lsprof `_minimal_curses`_ _random `_rawffi`_ + _ssl _socket _sre _weakref + array bz2 cStringIO + cpyext crypt errno exceptions @@ -72,7 +77,7 @@ * Supported by being rewritten in pure Python (possibly using ``ctypes``): see the `lib_pypy/`_ directory. Examples of modules that we - support this way: ``ctypes``, ``array``, ``cPickle``, + support this way: ``ctypes``, ``cPickle``, ``cStringIO``, ``cmath``, ``dbm`` (?), ``datetime``, ``binascii``... Note that some modules are both in there and in the list above; by default, the built-in module is used (but can be disabled From arigo at codespeak.net Fri Nov 26 13:18:29 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 26 Nov 2010 13:18:29 +0100 (CET) Subject: [pypy-svn] r79546 - pypy/extradoc/pypy.org/source Message-ID: <20101126121829.4E7D5282B90@codespeak.net> Author: arigo Date: Fri Nov 26 13:18:27 2010 New Revision: 79546 Modified: pypy/extradoc/pypy.org/source/compat.txt Log: Update the list of modules. Modified: pypy/extradoc/pypy.org/source/compat.txt ============================================================================== --- pypy/extradoc/pypy.org/source/compat.txt (original) +++ pypy/extradoc/pypy.org/source/compat.txt Fri Nov 26 13:18:27 2010 @@ -25,11 +25,11 @@ Standard library modules supported by PyPy, in alphabetical order: -* ``__builtin__ __pypy__ _codecs _lsprof _minimal_curses _random _rawffi _socket _sre _weakref bz2 cStringIO crypt errno exceptions fcntl gc itertools marshal math md5 mmap operator parser posix pyexpat select sha signal struct symbol sys termios thread time token unicodedata zipimport zlib`` +* ``__builtin__ __pypy__ _ast _bisect _codecs _lsprof _minimal_curses _random _rawffi _ssl _socket _sre _weakref array bz2 cStringIO cpyext crypt errno exceptions fcntl gc itertools marshal math md5 mmap operator parser posix pyexpat select sha signal struct symbol sys termios thread time token unicodedata zipimport zlib`` Supported, but written in pure-python: -* ``array binascii cPickle cmath collections ctypes datetime functools grp md5 pwd pyexpat sha sqlite3 syslog`` +* ``binascii cPickle cmath collections ctypes datetime functools grp pwd sqlite3 syslog`` All modules that are pure python in CPython of course work. From antocuni at codespeak.net Fri Nov 26 13:19:13 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 26 Nov 2010 13:19:13 +0100 (CET) Subject: [pypy-svn] r79547 - pypy/extradoc/pypy.org/source Message-ID: <20101126121913.DE00A282B9D@codespeak.net> Author: antocuni Date: Fri Nov 26 13:19:12 2010 New Revision: 79547 Modified: pypy/extradoc/pypy.org/source/compat.txt Log: rewrite this paragraph. We *do* support gc.get_referrers nowadays Modified: pypy/extradoc/pypy.org/source/compat.txt ============================================================================== --- pypy/extradoc/pypy.org/source/compat.txt (original) +++ pypy/extradoc/pypy.org/source/compat.txt Fri Nov 26 13:19:12 2010 @@ -72,11 +72,11 @@ with open("filename", "w") as f: f.write("stuff") -* We don't support certain attributes that were decided to be - implementation-dependent. For example, ``gc.get_referrers`` does not exist. - Others may have different behavior; for example, ``gc.enable`` and - ``gc.disable`` are supported, but they don't enable and disable the GC, but - instead just enable and disable the running of finalizers. +* For the same reason, some functions and attributes of the ``gc`` module + behave in a slightly different way: for example, ``gc.enable`` and + ``gc.disable`` are supported, but instead of enabling and disabling the GC, + they just enable and disable the execution of finalizers. Also, + ``gc.garbage`` always returns an empty list. * You can't attach a ``__del__`` method to a class after its creation. From arigo at codespeak.net Fri Nov 26 13:21:31 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 26 Nov 2010 13:21:31 +0100 (CET) Subject: [pypy-svn] r79548 - pypy/trunk/pypy/doc Message-ID: <20101126122131.79CC4282B90@codespeak.net> Author: arigo Date: Fri Nov 26 13:21:30 2010 New Revision: 79548 Modified: pypy/trunk/pypy/doc/cpython_differences.txt Log: Update for minimark. Modified: pypy/trunk/pypy/doc/cpython_differences.txt ============================================================================== --- pypy/trunk/pypy/doc/cpython_differences.txt (original) +++ pypy/trunk/pypy/doc/cpython_differences.txt Fri Nov 26 13:21:30 2010 @@ -134,12 +134,10 @@ .. __: http://morepypy.blogspot.com/2008/02/python-finalizers-semantics-part-1.html .. __: http://morepypy.blogspot.com/2008/02/python-finalizers-semantics-part-2.html -The built-in function ``id()`` returns numbers that are not addresses -for most of PyPy's garbage collectors. -This is most visible in the default repr: a typical PyPy object can -pretend to be located ``at 0x00000009``. This is just its ``id()``, not -its real address (because an object can move around in some GCs). Calling -``id`` a lot can lead to performance problem. +Using the default GC called ``minimark``, the built-in function ``id()`` +works like it does in CPython. With other GCs it returns numbers that +are not real addresses (because an object can move around several times) +and calling it a lot can lead to performance problem. Note that if you have a long chain of objects, each with a reference to the next one, and each with a __del__, PyPy's GC will perform badly. On From afa at codespeak.net Fri Nov 26 13:25:10 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 26 Nov 2010 13:25:10 +0100 (CET) Subject: [pypy-svn] r79549 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101126122510.0F971282B90@codespeak.net> Author: afa Date: Fri Nov 26 13:25:09 2010 New Revision: 79549 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py Log: Support full slice objects in ctypes arrays (with step != 1) Note that unbounded arrays like POINTER(c_int) don't normalize the index, i.e. x[-1] cannot be the last item of the array! This fixes 15 failures in the test suite. Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/array.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/array.py Fri Nov 26 13:25:09 2010 @@ -102,33 +102,43 @@ return _CDataMeta.from_param(self, value) def array_get_slice_params(self, index): - if index.step is not None: - raise TypeError("3 arg slices not supported (for no reason)") - check_bounds = hasattr(self, '_length_') - if index.start is not None: - start = index.start - if check_bounds and start < 0: - start = 0 + if hasattr(self, '_length_'): + start, stop, step = index.indices(self._length_) else: - start = 0 - if index.stop is not None: + step = index.step + if step is None: + step = 1 + start = index.start stop = index.stop - if check_bounds and stop > self._length_: - stop = self._length_ - else: - stop = self._length_ - return start, stop + if start is None: + if step > 0: + start = 0 + else: + raise ValueError("slice start is required for step < 0") + if stop is None: + raise ValueError("slice stop is required") + + return start, stop, step def array_slice_setitem(self, index, value): - start, stop = self._get_slice_params(index) - if stop - start != len(value): + start, stop, step = self._get_slice_params(index) + + if ((step < 0 and stop >= start) or + (step > 0 and start >= stop)): + slicelength = 0 + elif step < 0: + slicelength = (stop - start + 1) / step + 1 + else: + slicelength = (stop - start - 1) / step + 1; + + if slicelength != len(value): raise ValueError("Can only assign slices of the same length") - for i in range(start, stop): - self[i] = value[i - start] + for i, j in enumerate(range(start, stop, step)): + self[j] = value[j] def array_slice_getitem(self, index): - start, stop = self._get_slice_params(index) - l = [self[i] for i in range(start, stop)] + start, stop, step = self._get_slice_params(index) + l = [self[i] for i in range(start, stop, step)] letter = getattr(self._type_, '_type_', None) if letter == 'c': return "".join(l) From arigo at codespeak.net Fri Nov 26 13:33:59 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 26 Nov 2010 13:33:59 +0100 (CET) Subject: [pypy-svn] r79550 - pypy/trunk/pypy/doc Message-ID: <20101126123359.326B4282B9D@codespeak.net> Author: arigo Date: Fri Nov 26 13:33:56 2010 New Revision: 79550 Modified: pypy/trunk/pypy/doc/cpython_differences.txt Log: A link to cpyext. Modified: pypy/trunk/pypy/doc/cpython_differences.txt ============================================================================== --- pypy/trunk/pypy/doc/cpython_differences.txt (original) +++ pypy/trunk/pypy/doc/cpython_differences.txt Fri Nov 26 13:33:56 2010 @@ -34,7 +34,7 @@ array bz2 cStringIO - cpyext + `cpyext`_ crypt errno exceptions @@ -85,11 +85,13 @@ The extension modules (i.e. modules written in C, in the standard CPython) that are neither mentioned above nor in `lib_pypy/`_ are not available in PyPy. +(You may have a chance to use them anyway with `cpyext`_.) .. the nonstandard modules are listed below... .. _`__pypy__`: __pypy__-module.html .. _`_rawffi`: ctypes-implementation.html .. _`_minimal_curses`: config/objspace.usemodules._minimal_curses.html +.. _`cpyext`: http://morepypy.blogspot.com/2010/04/using-cpython-extension-modules-with.html .. _Stackless: stackless.html From arigo at codespeak.net Fri Nov 26 13:45:59 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 26 Nov 2010 13:45:59 +0100 (CET) Subject: [pypy-svn] r79551 - pypy/trunk/pypy/module/pypyjit/test Message-ID: <20101126124559.BA508282B9E@codespeak.net> Author: arigo Date: Fri Nov 26 13:45:58 2010 New Revision: 79551 Modified: pypy/trunk/pypy/module/pypyjit/test/test_policy.py Log: Fix the test: now id__ANY should be seen. Modified: pypy/trunk/pypy/module/pypyjit/test/test_policy.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/test/test_policy.py (original) +++ pypy/trunk/pypy/module/pypyjit/test/test_policy.py Fri Nov 26 13:45:58 2010 @@ -4,7 +4,7 @@ def test_id_any(): from pypy.objspace.std.default import id__ANY - assert not pypypolicy.look_inside_function(id__ANY) + assert pypypolicy.look_inside_function(id__ANY) def test_bigint(): from pypy.rlib.rbigint import rbigint From arigo at codespeak.net Fri Nov 26 13:47:18 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 26 Nov 2010 13:47:18 +0100 (CET) Subject: [pypy-svn] r79553 - pypy/trunk/pypy/rpython/lltypesystem Message-ID: <20101126124718.7F9D6282BD6@codespeak.net> Author: arigo Date: Fri Nov 26 13:47:17 2010 New Revision: 79553 Modified: pypy/trunk/pypy/rpython/lltypesystem/ll2ctypes.py Log: Add the _getid() method to ll2ctypes._parentable_mixin. This lets us return a more reasonable id for ll2ctypes objects. Fixes pypy.jit.backend.x86.test.test_basic.test_compute_unique_id. Modified: pypy/trunk/pypy/rpython/lltypesystem/ll2ctypes.py ============================================================================== --- pypy/trunk/pypy/rpython/lltypesystem/ll2ctypes.py (original) +++ pypy/trunk/pypy/rpython/lltypesystem/ll2ctypes.py Fri Nov 26 13:47:17 2010 @@ -442,6 +442,9 @@ "not allocated from RPython at all") self._storage = None + def _getid(self): + return self._addressof_storage() + def __eq__(self, other): if isinstance(other, _llgcopaque): addressof_other = other.intval From afa at codespeak.net Fri Nov 26 13:49:16 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 26 Nov 2010 13:49:16 +0100 (CET) Subject: [pypy-svn] r79554 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101126124916.B2CE3282BE3@codespeak.net> Author: afa Date: Fri Nov 26 13:49:15 2010 New Revision: 79554 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/function.py Log: Raise TypeError when func.restype is invalid, instead of AttributeError caused by the line after Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/function.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/function.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/function.py Fri Nov 26 13:49:15 2010 @@ -89,6 +89,8 @@ def _ffishapes(self, args, restype): argtypes = [arg._ffiargshape for arg in args] if restype is not None: + if not isinstance(restype, _CDataMeta): + raise TypeError("invalid result type for callback function") restype = restype._ffiargshape else: restype = 'O' # void From cfbolz at codespeak.net Fri Nov 26 14:19:12 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Fri, 26 Nov 2010 14:19:12 +0100 (CET) Subject: [pypy-svn] r79556 - pypy/extradoc/talk/pepm2011 Message-ID: <20101126131912.BD6BE282BE3@codespeak.net> Author: cfbolz Date: Fri Nov 26 14:19:09 2010 New Revision: 79556 Modified: pypy/extradoc/talk/pepm2011/paper.pdf pypy/extradoc/talk/pepm2011/paper.ps Log: fixes that the printers wanted Modified: pypy/extradoc/talk/pepm2011/paper.pdf ============================================================================== Binary files. No diff available. Modified: pypy/extradoc/talk/pepm2011/paper.ps ============================================================================== --- pypy/extradoc/talk/pepm2011/paper.ps (original) +++ pypy/extradoc/talk/pepm2011/paper.ps Fri Nov 26 14:19:09 2010 @@ -1,20 +1,20 @@ %!PS-Adobe-2.0 %%Creator: dvips(k) 5.98 Copyright 2009 Radical Eye Software %%Title: paper.dvi -%%CreationDate: Thu Nov 18 16:06:27 2010 +%%CreationDate: Fri Nov 26 14:12:07 2010 %%Pages: 10 %%PageOrder: Ascend -%%BoundingBox: 0 0 612 792 +%%BoundingBox: 0 0 596 842 %%DocumentFonts: Times-Bold Times-Roman CMMI8 CMMI6 SFSS0900 %%+ Times-BoldItalic Times-Italic CMSY7 BeraSansMono-Roman %%+ BeraSansMono-Bold CMMI9 CMR6 CMSY8 BeraSansMono-Oblique CMMI7 CMR5 %%+ CMR9 CMSY9 CMTT9 CMEX9 EUFM10 CMSY6 CMSY5 -%%DocumentPaperSizes: Letter +%%DocumentPaperSizes: a4 %%EndComments %DVIPSWebPage: (www.radicaleye.com) -%DVIPSCommandLine: dvips -P pdf -t letter -o paper.ps paper.dvi -%DVIPSParameters: dpi=8000 -%DVIPSSource: TeX output 2010.11.18:1606 +%DVIPSCommandLine: dvips paper.dvi +%DVIPSParameters: dpi=600 +%DVIPSSource: TeX output 2010.11.26:1411 %%BeginProcSet: tex.pro 0 0 %! /TeXDict 300 dict def TeXDict begin/N{def}def/B{bind def}N/S{exch}N/X{S @@ -61,26 +61,6 @@ rmoveto}B/y{3 2 roll p a}B/bos{/SS save N}B/eos{SS restore}B end %%EndProcSet -%%BeginProcSet: alt-rule.pro 0 0 -%! -% Patch by TVZ -% Makes dvips files draw rules with stroke rather than fill. -% Makes narrow rules more predictable at low resolutions -% after distilling to PDF. -% May have unknown consequences for very thick rules. -% Tested only with dvips 5.85(k). -TeXDict begin -/QV { - gsave newpath /ruleY X /ruleX X - Rx Ry gt - { ruleX ruleY Ry 2 div sub moveto Rx 0 rlineto Ry } - { ruleX Rx 2 div add ruleY moveto 0 Ry neg rlineto Rx } - ifelse - setlinewidth 0 setlinecap stroke grestore -} bind def -end - -%%EndProcSet %%BeginProcSet: 8r.enc 0 0 % File 8r.enc TeX Base 1 Encoding Revision 2.0 2002-10-30 % @@ -5847,915 +5827,764 @@ cleartomark {restore}if %%EndFont -TeXDict begin 40258431 52099146 1000 8000 8000 (paper.dvi) - at start /Fa 133[344 393 393 591 393 443 246 344 344 1[443 -443 443 639 246 393 246 246 443 443 246 393 443 393 443 -443 10[541 1[492 443 2[541 639 591 738 492 1[393 295 -639 639 541 541 639 591 541 541 7[443 1[443 2[443 443 -443 443 443 246 221 295 221 4[295 39[{TeXBase1Encoding ReEncodeFont}55 -885.568 /Times-Italic rf /Fb 252[407 3[{}1 553.48 /CMSY5 -rf /Fc 207[243 44[424 3[{}2 664.176 /CMSY6 rf /Fd 171[666 -84[{}1 996.264 /EUFM10 rf /Fe 244[484 484 6[427 427 469 -469{}6 996.264 /CMEX9 rf /Ff 136[523 1[523 523 523 523 -3[523 1[523 4[523 1[523 523 523 1[523 97[{}12 996.264 -/CMTT9 rf /Fg 145[512 4[398 398 512 512 6[683 3[683 27[796 -7[0 0 3[683 8[1024 7[1024 30[796 2[{}14 996.264 /CMSY9 -rf /Fh 133[455 540 2[540 569 398 404 401 1[569 512 569 -853 284 1[313 284 569 1[313 455 569 455 569 512 3[284 -1[284 7[569 10[768 2[697 7[796 2[284 16[398 398 6[284 -20[569 12[{}32 996.264 /CMR9 rf /Fi 198[377 377 377 377 -377 377 377 377 377 377 48[{}10 553.48 /CMR5 rf /Fj 136[640 -443 523 4[456 6[313 20[523 1[667 5[607 76[{}8 774.872 -/CMMI7 rf /Fk 134[466 466 1[466 466 466 466 466 1[466 -466 466 466 3[466 466 466 466 466 466 466 1[466 1[466 -21[466 6[466 7[466 8[466 466 1[466 466 466 2[466 466 -4[466 35[{TeXBase1Encoding ReEncodeFont}31 774.872 /BeraSansMono-Oblique -rf /Fl 198[277 277 277 277 277 277 277 277 277 277 48[{ -TeXBase1Encoding ReEncodeFont}10 553.48 /Times-Roman -rf /Fm 240[470 8[732 3[732 2[{}3 885.568 /CMSY8 rf /Fn -138[449 320 324 320 1[449 406 449 1[234 2[234 2[255 363 -449 43[406 406 406 406 406 1[406 406 406 35[467 12[{}21 -664.176 /CMR6 rf /Fo 136[734 497 589 1[477 2[514 496 -2[306 2[350 18[597 1[597 626 775 5[697 3[847 1[654 753 -6[796 512 796 284 284 15[1024 42[{}22 996.264 /CMMI9 -rf /Fp 134[466 466 466 466 466 466 466 466 1[466 466 -466 466 466 1[466 466 466 466 466 466 466 466 466 466 -1[466 21[466 2[466 3[466 3[466 466 2[466 8[466 466 1[466 -466 466 466 1[466 466 40[{TeXBase1Encoding ReEncodeFont}38 -774.872 /BeraSansMono-Roman rf /Fq 136[466 1[466 466 -466 466 1[466 1[466 1[466 2[466 466 1[466 466 466 466 -1[466 97[{TeXBase1Encoding ReEncodeFont}15 774.872 /BeraSansMono-Bold -rf /Fr 134[533 533 533 533 533 533 533 533 1[533 533 -533 533 533 533 533 533 533 533 533 533 533 533 533 533 -1[533 21[533 2[533 3[533 7[533 533 1[533 4[533 1[533 -533 533 533 45[{TeXBase1Encoding ReEncodeFont}36 885.568 -/BeraSansMono-Roman rf /Fs 133[442 498 498 719 1[554 -332 388 442 1[554 498 554 830 277 554 332 277 554 498 -332 442 554 442 554 498 1[498 10[665 1[719 1[609 775 -4[498 388 2[609 665 1[719 1[719 8[498 498 498 498 498 -498 498 498 2[249 332 41[554 3[{TeXBase1Encoding ReEncodeFont}46 -996.264 /Times-Bold rf /Ft 242[882 13[{}1 774.872 /CMSY7 -rf /Fu 175[431 2[689 7[473 19[387 4[194 4[258 39[{ -.167 SlantFont TeXBase1Encoding ReEncodeFont}6 774.872 -/Times-Roman rf /Fv 105[387 28[387 387 559 387 387 215 -301 258 387 387 387 387 603 215 387 1[215 387 387 258 -344 387 344 387 344 11[559 473 431 2[431 2[689 2[301 -6[517 1[559 7[387 387 387 387 387 387 387 387 387 387 -215 194 258 194 7[387 33[431 2[{TeXBase1Encoding ReEncodeFont}49 -774.872 /Times-Roman rf /Fw 7[443 96[886 443 1[393 393 -24[393 443 443 639 443 443 246 344 295 1[443 443 443 -689 246 443 246 246 443 443 295 393 443 393 443 393 1[443 -1[295 1[295 541 639 1[836 639 639 541 492 591 639 492 -639 639 787 541 639 344 295 639 639 492 541 639 591 591 -639 5[246 246 443 443 443 443 443 443 443 443 443 443 -246 221 295 221 2[295 295 295 1[738 29[246 3[492 492 -2[{TeXBase1Encoding ReEncodeFont}81 885.568 /Times-Roman -rf /Fx 134[442 442 665 442 498 277 388 388 1[498 498 -498 719 277 442 277 277 498 498 277 442 498 442 498 498 -10[609 1[554 498 609 1[609 719 2[554 1[442 332 4[719 -9[332 11[249 332 249 44[{TeXBase1Encoding ReEncodeFont}38 -996.264 /Times-Italic rf /Fy 134[442 1[665 1[554 277 -388 388 1[498 498 554 775 277 1[277 277 1[498 1[442 498 -442 498 498 12[609 554 7[665 3[719 2[719 665 67[{ -TeXBase1Encoding ReEncodeFont}25 996.264 /Times-BoldItalic -rf /Fz 198[332 332 332 332 332 332 332 332 332 332 48[{ -TeXBase1Encoding ReEncodeFont}10 664.176 /Times-Roman -rf /FA 136[879 609 677 405 474 541 1[677 609 677 1014 -338 677 405 338 677 609 405 541 677 541 677 609 9[1218 -2[812 1[879 2[947 2[812 1[609 474 2[744 812 879 879 812 -879 8[609 609 609 609 609 609 609 609 2[304 46[{ -TeXBase1Encoding ReEncodeFont}44 1217.66 /Times-Bold -rf /FB 133[445 1[472 2[529 370 392 350 1[529 512 529 -813 244 1[273 244 529 512 313 455 529 455 529 492 32[682 -17[284 341 16[549 28[{T1Encoding ReEncodeFont}25 996.264 -/SFSS0900 rf /FC 3[498 5[498 17[442 76[996 498 1[442 -442 24[442 498 498 719 498 498 277 388 332 498 498 498 -498 775 277 498 277 277 498 498 332 442 498 442 498 442 -1[498 1[332 1[332 609 2[940 719 719 609 554 665 1[554 -719 719 886 609 719 388 332 719 719 554 609 719 665 665 -719 5[277 277 498 498 498 498 498 498 498 498 498 498 -277 249 332 249 2[332 332 332 1[830 1[498 406 30[554 -554 2[{TeXBase1Encoding ReEncodeFont}83 996.264 /Times-Roman -rf /FD 156[371 364 448 14[612 5[562 5[515 70[{}6 664.176 -/CMMI6 rf /FE 156[408 403 500 97[{}3 885.568 /CMMI8 rf -/FF 133[541 2[879 1[609 338 474 405 2[609 609 947 338 -609 338 338 609 609 1[541 609 541 1[541 13[677 812 1[677 -2[1082 744 5[677 2[812 812 879 57[338 7[{TeXBase1Encoding ReEncodeFont} -29 1217.66 /Times-Roman rf /FG 134[996 2[996 1108 664 -1[885 2[996 1108 1660 554 2[554 1[996 1[885 1[885 1108 -996 12[1329 1[1439 1[1217 5[996 775 3[1329 3[1439 65[{ -TeXBase1Encoding ReEncodeFont}22 1992.53 /Times-Bold -rf end +TeXDict begin 39158280 55380996 1000 600 600 (paper.dvi) + at start /Fa 133[26 29 29 44 29 33 18 26 26 1[33 33 33 +48 18 29 18 18 33 33 18 29 33 29 33 33 10[41 1[37 33 +2[41 48 44 55 37 1[29 22 48 48 41 41 48 44 41 41 7[33 +1[33 2[33 33 33 33 33 18 17 22 17 4[22 39[{ +TeXBase1Encoding ReEncodeFont}55 66.4176 /Times-Italic +rf /Fb 252[31 3[{}1 41.511 /CMSY5 rf /Fc 207[18 44[32 +3[{}2 49.8132 /CMSY6 rf /Fd 171[50 84[{}1 74.7198 /EUFM10 +rf /Fe 244[36 36 6[32 32 35 35{}6 74.7198 /CMEX9 rf /Ff +136[39 1[39 39 39 39 3[39 1[39 4[39 1[39 39 39 1[39 97[{}12 +74.7198 /CMTT9 rf /Fg 145[38 4[30 30 38 38 6[51 3[51 +27[60 7[0 0 3[51 8[77 7[77 30[60 2[{}14 74.7198 /CMSY9 +rf /Fh 133[34 41 2[41 43 30 30 30 1[43 38 43 64 21 1[23 +21 43 1[23 34 43 34 43 38 3[21 1[21 7[43 10[58 2[52 7[60 +2[21 16[30 30 6[21 20[43 12[{}32 74.7198 /CMR9 rf /Fi +198[28 28 28 28 28 28 28 28 28 28 48[{}10 41.511 /CMR5 +rf /Fj 136[48 33 39 4[34 6[23 20[39 1[50 5[46 76[{}8 +58.1154 /CMMI7 rf /Fk 134[35 35 1[35 35 35 35 35 1[35 +35 35 35 3[35 35 35 35 35 35 35 1[35 1[35 21[35 6[35 +7[35 8[35 35 1[35 35 35 2[35 35 4[35 35[{TeXBase1Encoding ReEncodeFont} +31 58.1154 /BeraSansMono-Oblique rf /Fl 198[21 21 21 +21 21 21 21 21 21 21 48[{TeXBase1Encoding ReEncodeFont}10 +41.511 /Times-Roman rf /Fm 240[35 8[55 3[55 2[{}3 66.4176 +/CMSY8 rf /Fn 138[34 24 24 24 1[34 30 34 1[18 2[18 2[19 +27 34 43[30 30 30 30 30 1[30 30 30 35[35 12[{}21 49.8132 +/CMR6 rf /Fo 136[55 37 44 1[36 2[39 37 2[23 2[26 18[45 +1[45 47 58 5[52 3[64 1[49 57 6[60 38 60 21 21 15[77 42[{}22 +74.7198 /CMMI9 rf /Fp 134[35 35 35 35 35 35 35 35 1[35 +35 35 35 35 1[35 35 35 35 35 35 35 35 35 35 1[35 21[35 +2[35 3[35 3[35 35 2[35 8[35 35 1[35 35 35 35 1[35 35 +40[{TeXBase1Encoding ReEncodeFont}38 58.1154 /BeraSansMono-Roman +rf /Fq 136[35 1[35 35 35 35 1[35 1[35 1[35 2[35 35 1[35 +35 35 35 1[35 97[{TeXBase1Encoding ReEncodeFont}15 58.1154 +/BeraSansMono-Bold rf /Fr 134[40 40 40 40 40 40 40 40 +1[40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 40 1[40 +21[40 2[40 3[40 7[40 40 1[40 4[40 1[40 40 40 40 45[{ +TeXBase1Encoding ReEncodeFont}36 66.4176 /BeraSansMono-Roman +rf /Fs 133[33 37 37 54 1[42 25 29 33 1[42 37 42 62 21 +42 25 21 42 37 25 33 42 33 42 37 1[37 10[50 1[54 1[46 +58 4[37 29 2[46 50 1[54 1[54 8[37 37 37 37 37 37 37 37 +2[19 25 41[42 3[{TeXBase1Encoding ReEncodeFont}46 74.7198 +/Times-Bold rf /Ft 242[66 13[{}1 58.1154 /CMSY7 rf /Fu +175[32 2[52 7[36 19[29 4[15 4[19 39[{ +.167 SlantFont TeXBase1Encoding ReEncodeFont}6 58.1154 +/Times-Roman rf /Fv 105[29 28[29 29 42 29 29 16 23 19 +29 29 29 29 45 16 29 1[16 29 29 19 26 29 26 29 26 11[42 +36 32 2[32 2[52 2[23 6[39 1[42 7[29 29 29 29 29 29 29 +29 29 29 16 15 19 15 7[29 33[32 2[{TeXBase1Encoding ReEncodeFont}49 +58.1154 /Times-Roman rf /Fw 7[33 96[66 33 1[29 29 24[29 +33 33 48 33 33 18 26 22 1[33 33 33 52 18 33 18 18 33 +33 22 29 33 29 33 29 1[33 1[22 1[22 41 48 1[63 48 48 +41 37 44 48 37 48 48 59 41 48 26 22 48 48 37 41 48 44 +44 48 5[18 18 33 33 33 33 33 33 33 33 33 33 18 17 22 +17 2[22 22 22 1[55 29[18 3[37 37 2[{TeXBase1Encoding ReEncodeFont}81 +66.4176 /Times-Roman rf /Fx 134[33 33 50 33 37 21 29 +29 1[37 37 37 54 21 33 21 21 37 37 21 33 37 33 37 37 +10[46 1[42 37 46 1[46 54 2[42 1[33 25 4[54 9[25 11[19 +25 19 44[{TeXBase1Encoding ReEncodeFont}38 74.7198 /Times-Italic +rf /Fy 134[33 1[50 1[42 21 29 29 1[37 37 42 58 21 1[21 +21 1[37 1[33 37 33 37 37 12[46 42 7[50 3[54 2[54 50 67[{ +TeXBase1Encoding ReEncodeFont}25 74.7198 /Times-BoldItalic +rf /Fz 198[25 25 25 25 25 25 25 25 25 25 48[{ +TeXBase1Encoding ReEncodeFont}10 49.8132 /Times-Roman +rf /FA 136[66 46 51 30 36 41 1[51 46 51 76 25 51 30 25 +51 46 30 41 51 41 51 46 9[91 2[61 1[66 2[71 2[61 1[46 +36 2[56 61 66 66 61 66 8[46 46 46 46 46 46 46 46 2[23 +46[{TeXBase1Encoding ReEncodeFont}44 91.3242 /Times-Bold +rf /FB 133[33 1[35 2[40 28 29 26 1[40 38 40 61 18 1[20 +18 40 38 23 34 40 34 40 37 32[51 17[21 26 16[41 28[{ +T1Encoding ReEncodeFont}25 74.7198 /SFSS0900 rf /FC 3[37 +5[37 17[33 76[75 37 1[33 33 24[33 37 37 54 37 37 21 29 +25 37 37 37 37 58 21 37 21 21 37 37 25 33 37 33 37 33 +1[37 1[25 1[25 46 2[71 54 54 46 42 50 1[42 54 54 66 46 +54 29 25 54 54 42 46 54 50 50 54 5[21 21 37 37 37 37 +37 37 37 37 37 37 21 19 25 19 2[25 25 25 1[62 1[37 30 +30[42 42 2[{TeXBase1Encoding ReEncodeFont}83 74.7198 +/Times-Roman rf /FD 156[28 27 34 14[46 5[42 5[39 70[{}6 +49.8132 /CMMI6 rf /FE 156[31 30 37 97[{}3 66.4176 /CMMI8 +rf /FF 133[41 2[66 1[46 25 36 30 2[46 46 71 25 46 25 +25 46 46 1[41 46 41 1[41 13[51 61 1[51 2[81 56 5[51 2[61 +61 66 57[25 7[{TeXBase1Encoding ReEncodeFont}29 91.3242 +/Times-Roman rf /FG 134[75 2[75 83 50 1[66 2[75 83 124 +42 2[42 1[75 1[66 1[66 83 75 12[100 1[108 1[91 5[75 58 +3[100 3[108 65[{TeXBase1Encoding ReEncodeFont}22 149.44 +/Times-Bold rf end %%EndProlog %%BeginSetup -%%Feature: *Resolution 8000dpi +%%Feature: *Resolution 600dpi TeXDict begin -%%BeginPaperSize: Letter +%%BeginPaperSize: a4 /setpagedevice where -{ pop << /PageSize [612 792] >> setpagedevice } -{ /letter where { pop letter } if } +{ pop << /PageSize [595 842] >> setpagedevice } +{ /a4 where { pop a4 } if } ifelse %%EndPaperSize end %%EndSetup %%Page: 1 1 -TeXDict begin 1 0 bop Black Black Black Black Black 193 -2148 a FG(Allocation)696 b(Remo)-20 b(v)g(al)698 b(by)f(P)-20 -b(artial)698 b(Ev)-20 b(aluation)697 b(in)g(a)h(T)-147 -b(racing)696 b(JIT)4611 6133 y FF(Carl)306 b(Friedrich)f(Bolz)14081 -5693 y FE(a)16712 6133 y FF(Antonio)g(Cuni)23376 5693 -y FE(a)26007 6133 y FF(Maciej)f(Fija\007k)-12 b(o)-30 -b(wski)35400 5693 y FE(b)37934 6133 y FF(Michael)305 -b(Leuschel)46626 5693 y FE(a)17280 7572 y FF(Samuele)f(Pedroni)25566 -7132 y FE(c)28104 7572 y FF(Armin)i(Rigo)33956 7132 y -FE(a)12639 8809 y FD(a)13143 9232 y FC(Heinrich-Heine-Uni)-25 -b(v)-15 b(ersit\344t)251 b(D\374sseldorf,)d(STUPS)h(Group,)h(German)-15 -b(y)17766 10020 y FD(b)18185 10443 y FC(merlinux)249 -b(GmbH,)h(Hildesheim,)g(German)-15 b(y)19728 11127 y -FD(c)20155 11550 y FC(Open)250 b(End,)f(G\366tebor)-18 -b(g,)250 b(Sweden)5170 12878 y FB(cfb)28 b(olz at gmx.de)2113 -b(anto.cuni at gmail.com)g(\034jal at merlinux.eu)g -(leuschel at cs.uni-duesseldo)-28 b(rf.de)15552 13985 y(samuele.p)28 -b(edroni at gmail.com)2114 b(a)-28 b(rigo at tunes.o)g(rg)p -Black -2000 20378 a FA(Abstract)-2000 21928 y FC(The)248 -b(performance)h(of)f(man)-15 b(y)249 b(dynamic)g(language)h -(implementations)f(suf-)-2000 23035 y(fers)203 b(from)g(high)h -(allocation)h(rates)e(and)h(runtime)g(type)g(checks.)h(This)e(mak)-10 -b(es)-2000 24142 y(dynamic)243 b(languages)f(less)f(applicable)j(to)d -(purely)i(algorithmic)f(problems,)-2000 25249 y(despite)273 -b(their)f(gro)-25 b(wing)273 b(popularity)-65 b(.)273 -b(In)f(this)g(paper)h(we)g(present)f(a)h(simple)-2000 -26356 y(compiler)232 b(optimization)h(based)f(on)g(online)h(partial)f -(e)-25 b(v)g(aluation)233 b(to)f(remo)-15 b(v)g(e)-2000 -27463 y(object)259 b(allocations)h(and)f(runtime)f(type)h(checks)h(in)e -(the)h(conte)-15 b(xt)260 b(of)e(a)h(trac-)-2000 28570 -y(ing)268 b(JIT)-74 b(.)267 b(W)-80 b(e)269 b(e)-25 b(v)g(aluate)269 -b(the)f(optimization)h(using)f(a)g(Python)g(VM)g(and)g(\002nd)-2000 -29714 y(that)249 b(it)g(gi)-25 b(v)-15 b(es)250 b(good)f(results)g(for) -f(all)i(our)f(\(real-life\))f(benchmarks.)21561 29291 -y Fz(1)-2000 31445 y Fy(Categories)390 b(and)g(Subject)h(Descriptors) -996 b FC(D.3.4)391 b([)p Fx(Pr)-45 b(o)-10 b(gr)-15 b(amming)390 -b(Lan-)-2000 32552 y(gua)-10 b(g)g(es)p FC(]:)268 b(Processors\227code) -g(generation,)h(interpreters,)f(run-time)g(en)-40 b(vi-)-2000 -33659 y(ronments)-2000 35389 y Fy(General)250 b(T)-92 -b(erms)995 b FC(Languages,)250 b(Performance,)g(Experimentation)-2000 -37120 y Fy(K)-25 b(eyw)-15 b(ords)995 b FC(T)-35 b(racing)250 -b(JIT)-74 b(,)248 b(P)-15 b(artial)250 b(Ev)-25 b(aluation,)250 -b(Optimization)-2000 39626 y FA(1.)1218 b(Intr)-22 b(oduction)-2000 -41175 y FC(The)337 b(objecti)-25 b(v)-15 b(e)338 b(of)e(a)h -(just-in-time)g(\(JIT\))e(compiler)i(for)f(a)h(dynamic)h(lan-)-2000 -42282 y(guage)238 b(is)f(to)h(impro)-15 b(v)g(e)238 b(the)g(speed)f(of) -h(the)f(language)i(o)-15 b(v)g(er)238 b(an)g(implementa-)-2000 -43389 y(tion)261 b(of)g(the)g(language)i(that)e(uses)f(interpretation.) -i(The)f(\002rst)f(goal)i(of)f(a)g(JIT)-2000 44496 y(is)229 -b(therefore)g(to)g(remo)-15 b(v)g(e)230 b(the)g(interpretation)g(o)-15 -b(v)g(erhead,)231 b(i.e.)e(the)h(o)-15 b(v)g(erhead)-2000 -45603 y(of)231 b(bytecode)h(\(or)e(AST\))g(dispatch)h(and)h(the)f(o)-15 -b(v)g(erhead)232 b(of)e(the)h(interpreter')-55 b(s)-2000 -46710 y(data)364 b(structures,)f(such)h(as)f(operand)i(stack)f(etc.)g -(The)g(second)g(important)-2000 47817 y(problem)241 b(that)f(an)-15 -b(y)241 b(JIT)e(for)h(a)h(dynamic)g(language)g(needs)g(to)f(solv)-15 -b(e)241 b(is)e(ho)-25 b(w)-2000 48924 y(to)228 b(deal)g(with)g(the)f(o) --15 b(v)g(erhead)229 b(of)f(boxing)g(primiti)-25 b(v)-15 -b(e)228 b(types)f(and)h(of)g(type)g(dis-)-2000 50031 -y(patching.)221 b(Those)e(are)h(problems)f(that)h(are)f(usually)h(not)g -(present)f(or)g(at)h(least)-2000 51138 y(less)248 b(se)-25 -b(v)-15 b(ere)250 b(in)f(statically)h(typed)f(languages.)-672 -52245 y(Boxing)386 b(of)g(primiti)-25 b(v)-15 b(e)387 -b(types)f(is)f(necessary)i(because)g(dynamic)g(lan-)-2000 -53352 y(guages)310 b(need)h(to)f(be)g(able)h(to)f(handle)h(all)f -(objects,)g(e)-25 b(v)-15 b(en)311 b(inte)-15 b(gers,)310 -b(\003oats,)-2000 54459 y(booleans)220 b(etc.)g(in)g(the)f(same)h(w)-10 -b(ay)220 b(as)f(user)-20 b(-de\002ned)220 b(instances.)g(Thus)f(those) --2000 55566 y(primiti)-25 b(v)-15 b(e)265 b(types)g(are)g(usually)g -Fx(boxed)p FC(,)g(i.e.,)g(a)g(small)g(heap-structure)g(is)f(al-)-2000 -56673 y(located)356 b(for)f(them)g(that)h(contains)f(the)h(actual)g(v) --25 b(alue.)356 b(Boxing)f(primiti)-25 b(v)-15 b(e)-2000 -57780 y(types)261 b(can)h(be)g(v)-15 b(ery)262 b(costly)-65 -b(,)261 b(because)h(a)g(lot)f(of)g(common)h(operations,)g(par)-20 -b(-)-2000 58887 y(ticularly)335 b(all)g(arithmetic)h(operations,)f(ha) --20 b(v)-15 b(e)336 b(to)f(produce)g(ne)-25 b(w)336 b(box)-15 -b(es,)335 b(in)p Black -2000 60005 13284 37 v -2000 60688 -a Fz(1)-1502 61000 y Fw(This)305 b(research)h(is)f(partially)h -(supported)e(by)g(the)h(BMBF)g(funded)f(project)g(PyJIT)-2000 -61997 y(\(nr)-49 b(.)222 b(01QE0913B;)e(Eureka)h(Eurostars\).)p -Black Black -2000 66693 a Fv(Permission)257 b(to)e(mak)-8 -b(e)255 b(digital)i(or)e(hard)h(copies)g(of)f(all)g(or)g(part)h(of)f -(this)h(w)-8 b(ork)256 b(for)f(personal)i(or)-2000 67578 -y(classroom)217 b(use)f(is)f(granted)i(without)h(fee)d(pro)-12 -b(vided)217 b(that)g(copies)f(are)f(not)h(made)f(or)h(distrib)-15 -b(uted)-2000 68464 y(for)178 b(pro\002t)h(or)e(commercial)i(adv)-19 -b(antage)179 b(and)g(that)f(copies)i(bear)e(this)h(notice)g(and)f(the)h -(full)f(citation)-2000 69349 y(on)213 b(the)g(\002rst)g(page.)g(T)-62 -b(o)213 b(cop)-8 b(y)214 b(otherwise,)h(to)e(republish,)i(to)e(post)h -(on)f(serv)-12 b(ers)214 b(or)f(to)g(redistrib)-15 b(ute)-2000 -70235 y(to)195 b(lists,)g(requires)h(prior)f(speci\002c)g(permission)h -(and/or)g(a)d(fee.)-2000 71342 y Fu(PEPM'11,)789 b Fv(January)195 -b(24\22625,)h(2011,)f(Austin,)h(T)-54 b(e)-12 b(xas,)195 -b(USA.)-2000 72227 y(Cop)-8 b(yright)1598 72204 y(c)1329 -72227 y Ft(\015)194 b Fv(2011)i(A)-31 b(CM)193 b -(978-1-4503-0485-6/11/01.)116 b(.)g(.)g($10.00)p Black -Black Black 27224 20378 a FC(addition)294 b(to)h(the)f(actual)h -(computation)g(the)-15 b(y)295 b(do.)f(Because)h(the)f(box)-15 -b(es)294 b(are)27224 21485 y(allocated)217 b(on)f(the)g(heap,)g -(producing)h(man)-15 b(y)216 b(of)g(them)g(puts)f(pressure)g(on)h(the) -27224 22592 y(g)-5 b(arbage)250 b(collector)-55 b(.)28552 -23699 y(T)-80 b(ype)330 b(dispatching)g(is)e(the)i(process)e(of)h -(\002nding)h(the)f(concrete)i(imple-)27224 24806 y(mentation)223 -b(that)f(is)f(applicable)i(to)f(the)g(objects)g(at)g(hand)h(when)g -(performing)27224 25913 y(a)262 b(generic)h(operation)f(on)g(them.)h -(An)f(e)-15 b(xample)263 b(w)-10 b(ould)263 b(be)f(the)g(addition)h(of) -27224 27020 y(tw)-10 b(o)285 b(objects:)f(F)-15 b(or)285 -b(addition)g(the)f(types)h(of)f(the)h(concrete)h(objects)e(need)i(to) -27224 28127 y(be)264 b(check)-10 b(ed)266 b(and)f(the)g(suiting)f -(implementation)i(chosen.)f(T)-80 b(ype)265 b(dispatch-)27224 -29271 y(ing)229 b(is)f(a)h(v)-15 b(ery)230 b(common)f(operation)h(in)f -(modern)44029 28848 y Fz(2)44646 29271 y FC(dynamic)h(languages)g(be-) -27224 30378 y(cause)j(no)g(types)g(are)g(kno)-25 b(wn)234 -b(at)f(compile)h(time.)f(Therefore)g(all)g(operations)27224 -31485 y(need)250 b(it.)28552 32592 y(A)230 b(recently)h(popular)f -(approach)h(to)f(implementing)h(just-in-time)e(com-)27224 -33699 y(pilers)271 b(for)h(dynamic)h(languages)f(is)g(that)g(of)g(a)g -(tracing)g(JIT)-74 b(.)271 b(A)h(tracing)h(JIT)27224 -34806 y(w)-10 b(orks)364 b(by)h(observing)f(the)h(running)g(program)g -(and)g(recording)g(its)e(com-)27224 35913 y(monly)245 -b(e)-15 b(x)g(ecuted)246 b(parts)f(into)f Fx(linear)h(e)-20 -b(xecution)246 b(tr)-15 b(aces)p FC(.)245 b(Those)f(traces)h(are)27224 -37020 y(optimized)250 b(and)f(turned)h(into)f(machine)i(code.)28552 -38127 y(One)376 b(reason)g(for)f(the)h(popularity)g(of)f(tracing)h -(JITs)e(is)g(their)i(relati)-25 b(v)-15 b(e)27224 39234 -y(simplicity)-65 b(.)489 b(The)-15 b(y)489 b(can)h(often)f(be)h(added)g -(to)f(an)g(e)-15 b(xisting)489 b(interpreter)-40 b(,)27224 -40341 y(reusing)428 b(a)g(lot)g(of)g(the)g(interpreter')-55 -b(s)428 b(infrastructure.)g(The)-15 b(y)428 b(gi)-25 -b(v)-15 b(e)429 b(some)27224 41448 y(important)251 b(optimizations)g -(lik)-10 b(e)251 b(inlining)h(and)f(constant-folding)g(for)g(free.) -27224 42555 y(A)349 b(tracing)g(JIT)e(al)-10 b(w)g(ays)350 -b(produces)f(linear)g(pieces)g(of)f(code,)i(which)f(sim-)27224 -43662 y(pli\002es)296 b(man)-15 b(y)296 b(of)g(the)h(hard)f(algorithms) -g(in)g(a)g(compiler)-40 b(,)297 b(such)f(as)g(re)-15 -b(gister)27224 44769 y(allocation.)28552 45876 y(The)308 -b(use)g(of)g(a)h(tracing)f(JIT)f(can)i(remo)-15 b(v)g(e)309 -b(the)g(o)-15 b(v)g(erhead)309 b(of)f(bytecode)27224 -46983 y(dispatch)386 b(and)f(that)h(of)f(the)h(interpreter)f(data)h -(structures.)f(In)g(this)g(paper)27224 48090 y(we)442 -b(w)-10 b(ant)442 b(to)g(present)f(a)h(ne)-25 b(w)443 -b(optimization)f(that)g(can)g(be)g(added)h(to)e(a)27224 -49197 y(tracing)228 b(JIT)f(that)i(further)f(remo)-15 -b(v)g(es)228 b(some)g(of)g(the)h(o)-15 b(v)g(erhead)229 -b(more)f(closely)27224 50304 y(associated)444 b(to)g(dynamic)h -(languages,)f(such)g(as)g(boxing)g(o)-15 b(v)g(erhead)445 -b(and)27224 51410 y(type)355 b(dispatching.)h(Our)f(e)-15 -b(xperimental)357 b(platform)e(is)g(the)g(PyPy)g(project,)27224 -52517 y(which)283 b(is)g(an)g(en)-40 b(vironment)284 -b(for)e(implementing)i(dynamic)g(programming)27224 53624 -y(languages.)472 b(PyPy)f(and)h(tracing)f(JITs)f(are)h(described)h(in)f -(more)g(detail)27224 54731 y(in)386 b(Section)i(2.)e(Section)i(3)e -(analyzes)i(the)f(problem)g(to)g(be)g(solv)-15 b(ed)386 -b(more)27224 55838 y(closely)-65 b(.)28552 56945 y(The)297 -b(core)g(of)f(our)h(trace)g(optimization)h(technique)g(can)f(be)g(vie) --25 b(wed)298 b(as)27224 58052 y(partial)258 b(e)-25 -b(v)g(aluation:)259 b(the)g(partial)f(e)-25 b(v)g(aluation)259 -b(performs)e(a)i(form)e(of)h(escape)27224 59159 y(analysis)223 -b([4)o(])g(on)g(the)h(traces)f(and)g(mak)-10 b(es)224 -b(some)e(objects)i(that)f(are)g(allocated)27224 60266 -y(in)342 b(the)g(trace)h Fx(static,)e FC(which)i(means)f(that)h(the)-15 -b(y)342 b(do)g(not)g(occur)h(an)-15 b(y)342 b(more)27224 -61373 y(in)366 b(the)h(optimized)g(trace.)g(This)f(technique)i(is)d -(informally)i(described)g(in)27224 62480 y(Section)394 -b(4;)g(a)g(more)g(formal)g(description)g(is)g(gi)-25 -b(v)-15 b(en)394 b(in)g(Section)h(5.)f(The)27224 63587 -y(introduced)485 b(techniques)g(are)f(e)-25 b(v)g(aluated)486 -b(in)e(Section)g(6)g(using)g(PyPy')-55 b(s)27224 64694 -y(Python)249 b(interpreter)-55 b(.)28552 65801 y(The)249 -b(contrib)-20 b(utions)250 b(made)g(by)f(this)g(paper)g(are:)p -Black 27224 69407 V 27224 70090 a Fz(2)27722 70403 y -Fw(F)-13 b(or)197 b(languages)g(in)g(the)h(LISP)f(f)-9 -b(amily)-58 b(,)199 b(basic)f(arithmetic)g(operations)f(are)g -(typically)27224 71399 y(not)209 b(o)-13 b(v)g(erloaded;)208 -b(e)-22 b(v)-13 b(en)209 b(in)g(Smalltalk,)i(type)e(dispatching)h(is)g -(much)f(simpler)h(than)f(in)27224 72395 y(Python)221 -b(or)g(Ja)-18 b(v)c(aScript.)p Black Black Black eop -end +TeXDict begin 1 0 bop Black Black Black Black Black 14 +161 a FG(Allocation)51 b(Remo)o(v)o(al)f(by)j(P)o(artial)d(Ev)o +(aluation)g(in)i(a)g(T)-11 b(racing)51 b(JIT)346 460 +y FF(Carl)23 b(Friedrich)g(Bolz)1057 427 y FE(a)1253 +460 y FF(Antonio)g(Cuni)1754 427 y FE(a)1950 460 y FF(Maciej)g +(Fija\007k)o(o)n(wski)2656 427 y FE(b)2845 460 y FF(Michael)f(Leuschel) +3499 427 y FE(a)1296 568 y FF(Samuele)f(Pedroni)1918 +535 y FE(c)2108 568 y FF(Armin)i(Rigo)2547 535 y FE(a)948 +661 y FD(a)986 692 y FC(Heinrich-Heine-Uni)n(v)o(ersit\344t)c +(D\374sseldorf,)g(STUPS)e(Group,)i(German)o(y)1332 751 +y FD(b)1364 783 y FC(merlinux)g(GmbH,)g(Hildesheim,)g(German)o(y)1480 +835 y FD(c)1512 866 y FC(Open)g(End,)g(G\366tebor)o(g,)g(Sweden)388 +966 y FB(cfb)r(olz at gmx.de)161 b(anto.cuni at gmail.com)f +(\034jal at merlinux.eu)g(leuschel at cs.uni-duesseldo)n(rf.de)1166 +1049 y(samuele.p)r(edroni at gmail.com)i(a)n(rigo at tunes.o)n(rg)p +Black -150 1528 a FA(Abstract)-150 1645 y FC(The)19 b(performance)h(of) +f(man)o(y)g(dynamic)h(language)h(implementations)e(suf-)-150 +1728 y(fers)c(from)h(high)g(allocation)g(rates)f(and)h(runtime)g(type)g +(checks.)g(This)f(mak)o(es)-150 1811 y(dynamic)k(languages)h(less)f +(applicable)g(to)f(purely)h(algorithmic)g(problems,)-150 +1894 y(despite)i(their)f(gro)n(wing)i(popularity)-5 b(.)21 +b(In)g(this)g(paper)g(we)f(present)i(a)e(simple)-150 +1977 y(compiler)e(optimization)g(based)h(on)f(online)g(partial)f(e)n(v) +n(aluation)i(to)f(remo)o(v)o(e)-150 2060 y(object)i(allocations)g(and)g +(runtime)g(type)g(checks)h(in)f(the)f(conte)o(xt)h(of)g(a)g(trac-)-150 +2143 y(ing)h(JIT)-6 b(.)19 b(W)-6 b(e)20 b(e)n(v)n(aluate)i(the)e +(optimization)h(using)g(a)f(Python)h(VM)f(and)h(\002nd)-150 +2229 y(that)e(it)f(gi)n(v)o(es)h(good)i(results)d(for)h(all)f(our)i +(\(real-life\))e(benchmarks.)1617 2197 y Fz(1)-150 2358 +y Fy(Categories)30 b(and)g(Subject)e(Descriptors)76 b +FC(D.3.4)29 b([)p Fx(Pr)m(o)o(gr)o(amming)h(Lan-)-150 +2441 y(gua)o(g)o(es)p FC(]:)22 b(Processors\227code)f(generation,)h +(interpreters,)e(run-time)g(en)m(vi-)-150 2524 y(ronments)-150 +2654 y Fy(General)f(T)-7 b(erms)75 b FC(Languages,)20 +b(Performance,)f(Experimentation)-150 2784 y Fy(K)n(eyw)o(ords)76 +b FC(T)m(racing)19 b(JIT)-6 b(,)18 b(P)o(artial)g(Ev)n(aluation,)h +(Optimization)-150 2972 y FA(1.)91 b(Intr)n(oduction)-150 +3088 y FC(The)25 b(objecti)n(v)o(e)i(of)e(a)h(just-in-time)e(\(JIT\))h +(compiler)h(for)f(a)h(dynamic)g(lan-)-150 3171 y(guage)19 +b(is)f(to)g(impro)o(v)o(e)g(the)h(speed)f(of)g(the)h(language)g(o)o(v)o +(er)f(an)h(implementa-)-150 3254 y(tion)h(of)g(the)g(language)h(that)e +(uses)i(interpretation.)f(The)f(\002rst)g(goal)h(of)g(a)g(JIT)-150 +3337 y(is)d(therefore)h(to)f(remo)o(v)o(e)h(the)g(interpretation)f(o)o +(v)o(erhead,)i(i.e.)d(the)i(o)o(v)o(erhead)-150 3420 +y(of)g(bytecode)h(\(or)e(AST\))f(dispatch)j(and)f(the)f(o)o(v)o(erhead) +i(of)f(the)f(interpreter')l(s)-150 3503 y(data)28 b(structures,)f(such) +h(as)g(operand)h(stack)f(etc.)f(The)g(second)i(important)-150 +3586 y(problem)19 b(that)f(an)o(y)h(JIT)f(for)g(a)g(dynamic)h(language) +h(needs)f(to)f(solv)o(e)h(is)f(ho)n(w)-150 3669 y(to)f(deal)h(with)f +(the)g(o)o(v)o(erhead)i(of)e(boxing)i(primiti)n(v)o(e)e(types)g(and)h +(of)g(type)f(dis-)-150 3752 y(patching.)g(Those)h(are)e(problems)i +(that)e(are)h(usually)g(not)g(present)g(or)g(at)g(least)-150 +3835 y(less)i(se)n(v)o(ere)g(in)g(statically)g(typed)g(languages.)-50 +3918 y(Boxing)30 b(of)f(primiti)n(v)o(e)g(types)g(is)g(necessary)i +(because)f(dynamic)g(lan-)-150 4001 y(guages)25 b(need)f(to)f(be)h +(able)g(to)f(handle)i(all)e(objects,)g(e)n(v)o(en)i(inte)o(gers,)e +(\003oats,)-150 4084 y(booleans)18 b(etc.)e(in)h(the)g(same)g(w)o(ay)g +(as)g(user)o(-de\002ned)g(instances.)g(Thus)g(those)-150 +4167 y(primiti)n(v)o(e)j(types)g(are)h(usually)f Fx(boxed)p +FC(,)h(i.e.,)e(a)h(small)g(heap-structure)h(is)f(al-)-150 +4250 y(located)27 b(for)g(them)g(that)g(contains)h(the)f(actual)g(v)n +(alue.)g(Boxing)h(primiti)n(v)o(e)-150 4333 y(types)20 +b(can)h(be)f(v)o(ery)g(costly)-5 b(,)20 b(because)h(a)f(lot)f(of)h +(common)h(operations,)g(par)o(-)-150 4416 y(ticularly)k(all)g +(arithmetic)g(operations,)h(ha)o(v)o(e)g(to)f(produce)i(ne)n(w)e(box)o +(es,)h(in)p Black -150 4501 997 3 v -150 4552 a Fz(1)-113 +4575 y Fw(This)d(research)i(is)f(partially)i(supported)f(by)e(the)h +(BMBF)f(funded)i(project)g(PyJIT)-150 4650 y(\(nr)l(.)17 +b(01QE0913B;)h(Eureka)g(Eurostars\).)p Black Black -150 +5002 a Fv(Permission)h(to)h(mak)o(e)f(digital)h(or)f(hard)h(copies)e +(of)i(all)f(or)g(part)h(of)f(this)g(w)o(ork)h(for)g(personal)f(or)-150 +5068 y(classroom)d(use)f(is)h(granted)h(without)f(fee)g(pro)o(vided)h +(that)g(copies)e(are)h(not)h(made)f(or)g(distrib)o(uted)-150 +5135 y(for)e(pro\002t)g(or)g(commercial)g(adv)o(antage)e(and)h(that)h +(copies)f(bear)g(this)g(notice)h(and)f(the)g(full)h(citation)-150 +5201 y(on)i(the)g(\002rst)g(page.)f(T)-5 b(o)16 b(cop)o(y)h(otherwise,) +e(to)h(republish,)g(to)g(post)g(on)g(serv)o(ers)g(or)g(to)g(redistrib)o +(ute)-150 5268 y(to)f(lists,)e(requires)i(prior)h(speci\002c)e +(permission)g(and/or)h(a)g(fee.)-150 5351 y Fu(PEPM'11,)59 +b Fv(January)14 b(24\22625,)h(2011,)f(Austin,)g(T)l(e)o(xas,)f(USA.) +-150 5417 y(Cop)o(yright)120 5415 y(c)100 5417 y Ft(\015)h +Fv(2011)h(A)n(CM)f(978-1-4503-0485-6/11/01.)9 b(.)g(.)g($10.00)p +Black Black Black 2042 1528 a FC(addition)23 b(to)f(the)g(actual)h +(computation)g(the)o(y)g(do.)f(Because)h(the)g(box)o(es)g(are)2042 +1611 y(allocated)17 b(on)g(the)f(heap,)h(producing)h(man)o(y)f(of)g +(them)f(puts)h(pressure)g(on)g(the)2042 1694 y(garbage)j(collector)l(.) +2141 1777 y(T)-6 b(ype)26 b(dispatching)g(is)e(the)h(process)h(of)f +(\002nding)g(the)g(concrete)h(imple-)2042 1860 y(mentation)17 +b(that)g(is)f(applicable)i(to)f(the)g(objects)g(at)g(hand)h(when)f +(performing)2042 1943 y(a)j(generic)g(operation)h(on)f(them.)g(An)g(e)o +(xample)h(w)o(ould)f(be)h(the)f(addition)g(of)2042 2026 +y(tw)o(o)h(objects:)h(F)o(or)f(addition)h(the)g(types)g(of)g(the)f +(concrete)i(objects)f(need)g(to)2042 2110 y(be)e(check)o(ed)i(and)f +(the)f(suiting)g(implementation)h(chosen.)g(T)-6 b(ype)20 +b(dispatch-)2042 2195 y(ing)d(is)g(a)h(v)o(ery)g(common)g(operation)h +(in)e(modern)3301 2164 y Fz(2)3348 2195 y FC(dynamic)i(languages)g(be-) +2042 2278 y(cause)f(no)g(types)h(are)e(kno)n(wn)i(at)f(compile)g(time.) +f(Therefore)h(all)f(operations)2042 2361 y(need)j(it.)2141 +2444 y(A)e(recently)f(popular)i(approach)g(to)e(implementing)i +(just-in-time)e(com-)2042 2527 y(pilers)j(for)h(dynamic)g(languages)i +(is)d(that)g(of)h(a)f(tracing)h(JIT)-6 b(.)20 b(A)h(tracing)g(JIT)2042 +2610 y(w)o(orks)28 b(by)g(observing)h(the)f(running)g(program)h(and)f +(recording)h(its)e(com-)2042 2693 y(monly)19 b(e)o(x)o(ecuted)h(parts)e +(into)h Fx(linear)g(e)o(xecution)g(tr)o(aces)p FC(.)g(Those)f(traces)h +(are)2042 2777 y(optimized)g(and)h(turned)f(into)g(machine)h(code.)2141 +2860 y(One)29 b(reason)g(for)f(the)h(popularity)g(of)g(tracing)f(JITs)g +(is)g(their)g(relati)n(v)o(e)2042 2943 y(simplicity)-5 +b(.)36 b(The)o(y)h(can)h(often)f(be)g(added)h(to)f(an)g(e)o(xisting)g +(interpreter)m(,)2042 3026 y(reusing)c(a)f(lot)g(of)g(the)h +(interpreter')l(s)f(infrastructure.)g(The)o(y)h(gi)n(v)o(e)g(some)2042 +3109 y(important)19 b(optimizations)h(lik)o(e)f(inlining)g(and)h +(constant-folding)h(for)e(free.)2042 3192 y(A)26 b(tracing)h(JIT)e(al)o +(w)o(ays)i(produces)h(linear)f(pieces)f(of)h(code,)g(which)f(sim-)2042 +3275 y(pli\002es)c(man)o(y)h(of)f(the)h(hard)g(algorithms)g(in)f(a)g +(compiler)m(,)h(such)g(as)f(re)o(gister)2042 3358 y(allocation.)2141 +3441 y(The)i(use)f(of)h(a)f(tracing)h(JIT)e(can)i(remo)o(v)o(e)g(the)g +(o)o(v)o(erhead)g(of)g(bytecode)2042 3524 y(dispatch)30 +b(and)g(that)e(of)i(the)f(interpreter)g(data)g(structures.)g(In)g(this) +g(paper)2042 3607 y(we)k(w)o(ant)h(to)f(present)h(a)f(ne)n(w)h +(optimization)g(that)f(can)h(be)f(added)i(to)e(a)2042 +3690 y(tracing)17 b(JIT)g(that)g(further)h(remo)o(v)o(es)g(some)g(of)f +(the)g(o)o(v)o(erhead)i(more)f(closely)2042 3773 y(associated)34 +b(to)g(dynamic)g(languages,)h(such)f(as)g(boxing)h(o)o(v)o(erhead)f +(and)2042 3856 y(type)27 b(dispatching.)h(Our)f(e)o(xperimental)g +(platform)g(is)g(the)g(PyPy)f(project,)2042 3939 y(which)c(is)f(an)g +(en)m(vironment)i(for)f(implementing)g(dynamic)h(programming)2042 +4022 y(languages.)37 b(PyPy)e(and)h(tracing)g(JITs)f(are)g(described)i +(in)e(more)h(detail)2042 4105 y(in)29 b(Section)g(2.)g(Section)g(3)h +(analyzes)g(the)f(problem)h(to)f(be)h(solv)o(ed)g(more)2042 +4188 y(closely)-5 b(.)2141 4271 y(The)23 b(core)g(of)f(our)h(trace)f +(optimization)h(technique)h(can)f(be)g(vie)n(wed)g(as)2042 +4354 y(partial)c(e)n(v)n(aluation:)i(the)e(partial)h(e)n(v)n(aluation)g +(performs)h(a)e(form)h(of)f(escape)2042 4437 y(analysis)e([4])g(on)h +(the)f(traces)g(and)g(mak)o(es)h(some)f(objects)h(that)f(are)g +(allocated)2042 4520 y(in)26 b(the)g(trace)f Fx(static,)h +FC(which)g(means)g(that)g(the)o(y)g(do)h(not)f(occur)g(an)o(y)h(more) +2042 4603 y(in)g(the)h(optimized)h(trace.)e(This)g(technique)i(is)e +(informally)h(described)h(in)2042 4686 y(Section)g(4;)h(a)g(more)g +(formal)g(description)g(is)g(gi)n(v)o(en)g(in)g(Section)g(5.)f(The)2042 +4769 y(introduced)38 b(techniques)g(are)e(e)n(v)n(aluated)i(in)e +(Section)h(6)f(using)i(PyPy')l(s)2042 4852 y(Python)19 +b(interpreter)l(.)2141 4935 y(The)g(contrib)o(utions)h(made)f(by)h +(this)e(paper)i(are:)p Black 2042 5206 V 2042 5257 a +Fz(2)2079 5280 y Fw(F)o(or)15 b(languages)j(in)d(the)h(LISP)e(f)o +(amily)l(,)i(basic)g(arithmetic)i(operations)g(are)e(typically)2042 +5355 y(not)g(o)o(v)o(erloaded;)k(e)n(v)o(en)d(in)f(Smalltalk,)i(type)f +(dispatching)i(is)d(much)h(simpler)g(than)g(in)2042 5430 +y(Python)h(or)f(Ja)o(v)n(aScript.)p Black Black Black +eop end %%Page: 2 2 -TeXDict begin 2 1 bop Black Black Black -1696 886 a FC(1.)p -Black 388 w(A)211 b(description)g(of)f(a)h(practical,)h(ef)-25 -b(\002cient)211 b(and)h(ef)-25 b(fecti)g(v)-15 b(e)211 -b(algorithm)g(for)-561 1993 y(remo)-15 b(ving)250 b(object)g -(allocations)g(in)f(a)g(tracing)h(JIT)-74 b(.)p Black --1696 3542 a(2.)p Black 388 w(A)249 b(characterization)j(of)c(this)h -(algorithm)h(as)f(partial)g(e)-25 b(v)g(aluation.)p Black --1696 5092 a(3.)p Black 388 w(Performance)250 b(benchmarks)g(for)f -(this)f(algorithm.)-2000 7777 y FA(2.)1218 b(Backgr)-22 -b(ound)-2000 9327 y Fs(2.1)997 b(PyPy)-2000 10876 y FC(The)359 -b(w)-10 b(ork)358 b(described)i(in)e(this)g(paper)h(w)-10 -b(as)359 b(done)g(in)g(the)g(conte)-15 b(xt)359 b(of)g(the)-2000 -12021 y(PyPy)365 b(project)3235 11598 y Fz(3)3987 12021 -y FC([26].)g(PyPy)f(is)g(an)h(en)-40 b(vironment)366 -b(where)f(dynamic)h(lan-)-2000 13128 y(guages)374 b(can)f(be)h -(implemented)g(in)f(a)h(simple)f(yet)g(ef)-25 b(\002cient)374 -b(w)-10 b(ay)-65 b(.)374 b(When)-2000 14235 y(implementing)317 -b(a)e(language)i(with)f(PyPy)g(one)g(writes)f(an)h Fx(interpr)-37 -b(eter)338 b FC(for)-2000 15342 y(the)393 b(language)h(in)f -Fx(RPython)g FC([1].)f(RPython)h(\("restricted)g(Python"\))g(is)g(a) --2000 16449 y(subset)232 b(of)h(Python)g(chosen)h(in)f(such)g(a)g(w)-10 -b(ay)234 b(that)f(type)g(inference)h(becomes)-2000 17556 -y(possible.)371 b(The)g(language)h(interpreter)g(can)f(then)h(be)f -(compiled)h(\(\223trans-)-2000 18663 y(lated\224\))269 -b(with)f(PyPy')-55 b(s)267 b(tools)h(into)g(a)g(VM)g(on)g(C)f(le)-25 -b(v)-15 b(el.)269 b(During)f(translation)-2000 19770 -y(to)217 b(C,)f(man)-15 b(y)218 b(lo)-25 b(w-le)g(v)-15 -b(el)218 b(aspects)e(of)h(the)g(\002nal)g(VM,)g(such)g(as)g(object)g -(layout,)-2000 20876 y(g)-5 b(arbage)320 b(collection)f(and)g(memory)g -(model,)g(are)g(w)-10 b(o)-15 b(v)g(en)320 b(into)f(the)g(gener)-20 -b(-)-2000 21983 y(ated)246 b(code.)h(Therefore)f(the)g(interpreter)g -(itself)f(can)i(remain)f(at)g(a)g(relati)-25 b(v)-15 -b(ely)-2000 23090 y(high)249 b(le)-25 b(v)-15 b(el)250 -b(of)f(abstraction.)-672 24197 y(A)460 b(number)f(of)g(languages)h(ha) --20 b(v)-15 b(e)461 b(been)f(implemented)g(with)g(PyPy)-65 -b(.)-2000 25304 y(The)380 b(project)g(w)-10 b(as)380 -b(initiated)g(to)g(get)g(a)g(better)g(Python)g(implementation,)-2000 -26411 y(which)315 b(inspired)f(the)g(name)h(of)e(the)i(project)f(and)h -(is)e(still)g(the)i(main)f(focus)-2000 27518 y(of)445 -b(de)-25 b(v)-15 b(elopment.)446 b(In)f(addition)g(a)h(number)f(of)f -(other)i(languages)f(were)-2000 28625 y(implemented,)211 -b(among)f(them)g(a)g(Prolog)f(interpreter)h([7],)f(a)h(Smalltalk)g(VM) --2000 29732 y([6])249 b(and)g(a)h(GameBo)-10 b(y)250 -b(emulator)f([8].)-672 30839 y(The)289 b(feature)f(that)h(mak)-10 -b(es)289 b(PyPy)f(more)g(than)h(a)f(compiler)h(with)g(a)f(run-)-2000 -31946 y(time)339 b(system)f(is)g(its)g(support)g(for)g(automated)i(JIT) -e(compiler)h(generation)-2000 33053 y([5].)317 b(During)h(the)g -(translation)f(to)h(C,)e(PyPy')-55 b(s)317 b(tools)g(can)i(generate)f -(a)g(trac-)-2000 34160 y(ing)400 b(just-in-time)g(compiler)h(for)f(the) -g(language)i(that)e(the)h(interpreter)g(is)-2000 35267 -y(implementing.)343 b(This)d(process)h(is)g(mostly)g(automatic;)h(it)g -(only)f(needs)h(to)-2000 36374 y(be)i(guided)g(by)g(the)g(language)h -(implementer)g(using)e(a)h(small)f(number)h(of)-2000 -37481 y(source-code)274 b(hints)f(in)g(the)g(interpreter)-55 -b(.)274 b(Mostly-automatically)f(generat-)-2000 38588 -y(ing)211 b(a)f(JIT)g(compiler)h(has)f(man)-15 b(y)211 -b(adv)-25 b(antages)212 b(o)-15 b(v)g(er)211 b(writing)g(one)g -(manually)-65 b(,)-2000 39695 y(an)267 b(error)-20 b(-prone)266 -b(and)h(tedious)f(process.)g(By)g(construction,)h(the)g(generated)-2000 -40802 y(JIT)235 b(has)g(the)i(same)f(semantics)f(as)h(the)g -(interpreter)-55 b(.)237 b(Optimizations)f(can)h(be)-2000 -41909 y(shared)249 b(between)i(dif)-25 b(ferent)249 b(languages)h -(implemented)h(with)e(PyPy)-65 b(.)-672 43016 y(Moreo)-15 -b(v)g(er)-40 b(,)300 b(thanks)g(to)f(the)h(internal)f(design)h(of)f -(the)h(JIT)e(generator)-40 b(,)300 b(it)-2000 44123 y(is)243 -b(v)-15 b(ery)244 b(easy)g(to)f(add)h(ne)-25 b(w)244 -b Fx(bac)-20 b(k)-10 b(ends)245 b FC(for)e(producing)h(the)g(actual)h -(machine)-2000 45230 y(code.)227 b(Examples)f(of)g(JIT)e(back)-10 -b(ends)227 b(that)g(are)f(implemented)h(are)f(those)g(for)-2000 -46337 y(Intel)347 b(x86)g(and)g(x86-64)g(and)g(an)g(e)-15 -b(xperimental)348 b(one)f(for)f(the)h(CLI)e(.NET)-2000 -47444 y(V)-60 b(irtual)250 b(Machine)g([12].)-2000 49529 -y Fs(2.2)997 b(T)-74 b(racing)249 b(JIT)f(Compilers)-2000 -51078 y FC(T)-35 b(racing)306 b(JITs)f(are)h(a)h(recently)g(popular)f -(approach)h(to)f(write)h(just-in-time)-2000 52185 y(compilers)274 -b(for)g(dynamic)h(languages.)g(Their)f(origins)g(lie)g(in)g(the)h -(Dynamo)-2000 53292 y(project,)324 b(which)h(used)e(a)h(tracing)g -(approach)h(to)f(optimize)g(machine)h(code)-2000 54399 -y(using)249 b(e)-15 b(x)g(ecution)250 b(traces)f([2].)f(T)-35 -b(racing)249 b(JITs)f(ha)-20 b(v)-15 b(e)249 b(then)h(be)f(adapted)h -(to)f(be)-2000 55506 y(used)299 b(for)f(a)h(v)-15 b(ery)300 -b(light-weight)f(Ja)-20 b(v)-25 b(a)299 b(VM)g([15])g(and)g(afterw)-10 -b(ards)299 b(used)g(in)-2000 56613 y(se)-25 b(v)-15 b(eral)270 -b(implementations)h(of)f(dynamic)g(languages,)h(such)f(as)f(Ja)-20 -b(v)-25 b(aScript)-2000 57758 y([13],)249 b(Lua)1707 -57334 y Fz(4)2344 57758 y FC(and)g(no)-25 b(w)250 b(Python)f(\(and)h -(other)f(languages\))h(via)g(PyPy)-65 b(.)-672 58865 -y(The)270 b(core)g(idea)g(of)f(tracing)h(JITs)e(is)h(to)h(focus)f(the)h -(optimization)g(ef)-25 b(fort)-2000 59972 y(of)338 b(the)h(JIT)f -(compiler)h(on)g(the)g(commonly)g(e)-15 b(x)g(ecuted,)341 -b(i.e.,)e Fx(hot)357 b FC(paths)338 b(of)-2000 61079 -y(the)402 b(core)g(loops)g(of)g(the)g(program)g(and)g(to)g(just)f(use)h -(an)g(interpreter)g(for)-2000 62185 y(the)303 b(less)f(commonly)i(e)-15 -b(x)g(ecuted)304 b(parts.)e(VMs)g(that)h(use)g(a)g(tracing)g(JIT)f(are) --2000 63292 y(mostly)287 b(mix)-15 b(ed-mode)289 b(e)-15 -b(x)g(ecution)288 b(en)-40 b(vironments,)289 b(the)-15 -b(y)288 b(contain)g(both)g(an)-2000 64399 y(interpreter)370 -b(and)g(a)g(JIT)f(compiler)-55 b(.)370 b(By)g(def)-10 -b(ault)370 b(the)g(interpreter)g(is)f(used)-2000 65506 -y(to)h(e)-15 b(x)g(ecute)371 b(the)f(program,)g(doing)g(some)f -(light-weight)h(pro\002ling)g(at)g(the)-2000 66613 y(same)353 -b(time.)g(This)g(pro\002ling)f(is)h(used)g(to)g(identify)g(the)g(hot)g -(loops)g(of)g(the)-2000 67720 y(program.)271 b(If)e(a)i(hot)f(loop)g -(is)g(found)g(in)h(that)f(w)-10 b(ay)-65 b(,)271 b(the)g(interpreter)g -(enters)f(a)-2000 68827 y(special)318 b Fx(tr)-15 b(acing)318 -b(mode)p FC(.)g(In)f(this)g(tracing)h(mode,)g(the)g(interpreter)g -(tries)f(to)p Black -2000 70104 13284 37 v -2000 70788 -a Fz(3)-1502 71100 y Fr(http://pypy.org)-2000 72083 y -Fz(4)-1502 72395 y Fr(http://luajit.org/)p Black Black -Black 27224 886 a FC(record)225 b(all)g(operations)g(that)g(it)g(is)f -(e)-15 b(x)g(ecuting)226 b(while)g(running)f(one)g(iteration)27224 -1993 y(of)291 b(the)g(hot)g(loop.)h(This)e(history)g(of)h(e)-15 -b(x)g(ecuted)293 b(operations)f(of)e(one)i(loop)f(is)27224 -3099 y(called)307 b(a)g Fx(tr)-15 b(ace)p FC(.)307 b(Because)g(the)g -(trace)g(corresponds)f(to)h(one)g(iteration)g(of)f(a)27224 -4206 y(loop,)337 b(it)h(al)-10 b(w)g(ays)337 b(ends)h(with)f(a)h(jump)f -(to)h(its)e(o)-25 b(wn)338 b(be)-15 b(ginning.)339 b(The)e(trace)27224 -5313 y(also)406 b(contains)h(all)g(operations)f(that)h(are)g(performed) -g(in)f(functions)g(that)27224 6420 y(were)303 b(called)g(in)f(the)h -(loop,)f(thus)g(a)h(tracing)f(JIT)g(automatically)h(performs)27224 -7527 y(inlining.)329 b(This)g(trace)h(of)f(operations)h(subsequently)g -(forms)e(the)i(basis)f(of)27224 8634 y(the)372 b(generated)g(code.)h -(The)e(trace)h(is)f(\002rst)g(optimized,)h(and)g(then)g(turned)27224 -9741 y(into)205 b(machine)g(code.)h(Both)e(optimization)i(and)f -(machine)h(code)g(generation)27224 10848 y(are)225 b(simple,)f(because) -i(the)f(traces)g(are)g(linear)-55 b(.)225 b(This)f(linearity)h(mak)-10 -b(es)225 b(man)-15 b(y)27224 11955 y(optimizations)379 -b(a)g(lot)f(more)h(tractable,)g(and)h(the)e(inlining)h(that)g(happens) -27224 13062 y(gi)-25 b(v)-15 b(es)249 b(the)h(optimizations)f -(automatically)i(more)e(conte)-15 b(xt)250 b(to)f(w)-10 -b(ork)250 b(with.)28552 14169 y(Since)234 b(the)g(trace)g(corresponds)f -(to)h(one)g(concrete)g(e)-15 b(x)g(ecution)235 b(of)f(a)f(loop,)27224 -15276 y(the)358 b(code)g(generated)h(from)e(it)h(is)f(only)h(one)g -(possible)f(path)h(through)g(the)27224 16383 y(loop.)303 -b(T)-80 b(o)303 b(mak)-10 b(e)304 b(sure)e(that)i(the)f(trace)h -(maintains)f(the)g(correct)h(semantics,)27224 17490 y(it)374 -b(contains)h(a)g Fx(guar)-37 b(d)402 b FC(at)375 b(all)g(places)g -(where)g(the)g(e)-15 b(x)g(ecution)376 b(could)g(ha)-20 -b(v)-15 b(e)27224 18597 y(di)-25 b(v)-15 b(er)d(ged)245 -b(from)e(the)h(path.)h(Those)e(guards)h(check)h(the)f(assumptions)f -(under)27224 19704 y(which)369 b(e)-15 b(x)g(ecution)370 -b(can)g(stay)e(on)h(the)g(trace.)h(As)e(an)h(e)-15 b(xample,)370 -b(if)e(a)h(loop)27224 20811 y(contains)419 b(an)h(if-statement,)f(the)g -(trace)h(will)f(contain)h(the)f(e)-15 b(x)g(ecution)421 -b(of)27224 21918 y(one)307 b(of)g(the)g(paths)g(only)-65 -b(,)308 b(which)g(is)e(the)h(path)h(that)f(w)-10 b(as)307 -b(tak)-10 b(en)308 b(during)f(the)27224 23025 y(production)386 -b(of)f(the)h(trace.)g(The)g(trace)g(will)g(also)f(contain)i(a)e(guard)h -(that)27224 24132 y(checks)293 b(that)f(the)h(condition)g(of)f(the)h -(if-statement)f(is)f(the)i(same)g(as)f(during)27224 25239 -y(tracing,)249 b(because)i(if)d(it)h(isn')-18 b(t,)249 -b(the)g(rest)g(of)g(the)g(trace)h(w)-10 b(ould)250 b(not)f(be)h(v)-25 -b(alid.)28552 26346 y(When)391 b(generating)f(machine)h(code,)g(e)-25 -b(v)-15 b(ery)390 b(guard)g(is)f(turned)h(into)f(a)27224 -27453 y(quick)291 b(check)i(to)d(see)i(whether)f(the)h(assumption)f -(still)f(holds.)h(When)h(such)27224 28560 y(a)377 b(guard)g(is)f(hit)g -(during)h(the)g(e)-15 b(x)g(ecution)378 b(of)e(the)h(machine)h(code)f -(and)g(the)27224 29667 y(assumption)388 b(does)g(not)g(hold,)g(the)h(e) --15 b(x)g(ecution)389 b(of)f(the)g(machine)h(code)g(is)27224 -30773 y(stopped,)261 b(and)g(interpreter)g(continues)g(to)f(run)h(from) -f(that)h(point)g(on.)g(These)27224 31880 y(guards)249 -b(are)i(the)f(only)g(mechanism)g(to)g(stop)g(the)g(e)-15 -b(x)g(ecution)251 b(of)f(a)g(trace,)g(the)27224 32987 -y(loop)f(end)h(condition)g(also)f(tak)-10 b(es)249 b(the)h(form)e(of)h -(a)h(guard.)28552 34094 y(If)242 b(one)i(speci\002c)f(guard)h(f)-10 -b(ails)242 b(a)h(lot)g(\(i.e.,)g(more)h(than)f(some)g(threshold\),) -27224 35201 y(the)327 b(tracing)g(JIT)f(will)h(generate)h(a)f(ne)-25 -b(w)328 b(trace)f(that)g(starts)f(e)-15 b(xactly)328 -b(at)f(the)27224 36308 y(position)210 b(of)g(the)h(f)-10 -b(ailing)211 b(guard)g([14].)f(The)h(e)-15 b(xisting)210 -b(assembler)h(is)e(patched)27224 37415 y(to)469 b(jump)g(to)g(the)h(ne) --25 b(w)470 b(trace)f(when)h(the)g(guard)f(f)-10 b(ails.)469 -b(This)g(approach)27224 38522 y(guarantees)297 b(that)f(all)g(the)g -(hot)g(paths)g(in)g(the)g(program)g(will)g(e)-25 b(v)-15 -b(entually)297 b(be)27224 39629 y(traced)250 b(and)f(compiled)h(into)g -(ef)-25 b(\002cient)250 b(code.)27224 43171 y Fs(2.3)996 -b(Running)249 b(Example)27224 44721 y FC(F)-15 b(or)250 -b(the)h(purpose)g(of)f(this)h(paper)-40 b(,)251 b(we)g(are)g(going)g -(to)g(use)g(a)g(tin)-15 b(y)250 b(interpreter)27224 45828 -y(for)290 b(a)i(dynamic)g(language)g(with)g(a)f(v)-15 -b(ery)291 b(simple)g(object)h(model,)g(that)f(just)27224 -46935 y(supports)296 b(an)h(inte)-15 b(ger)298 b(and)f(a)g(\003oat)h -(type.)f(The)g(objects)g(support)g(only)g(tw)-10 b(o)27224 -48042 y(operations,)322 b Fr(add)p Black Black FC(,)g(which)g(adds)g -(tw)-10 b(o)322 b(objects)g(\(promoting)g(ints)f(to)h(\003oats)27224 -49149 y(in)233 b(a)g(mix)-15 b(ed)234 b(addition\))f(and)h -Fr(is)38184 48940 y(_)38717 49149 y(positive)p Black -Black 1 w FC(,)f(which)h(returns)e(whether)i(the)27224 -50256 y(number)259 b(is)f(greater)h(than)g(zero.)g(The)g -(implementation)h(of)e Fr(add)p Black Black 259 w FC(uses)g(clas-)27224 -51363 y(sical)c(Smalltalk-lik)-10 b(e)255 b(double-dispatching.)h(The)e -(classes)g(can)h(be)g(seen)g(in)27224 52470 y(Figure)249 -b(1)g(\(written)g(in)h(RPython\).)28552 53577 y(Using)405 -b(these)g(classes)f(to)h(implement)g(arithmetic)h(sho)-25 -b(ws)404 b(the)h(basic)27224 54684 y(problem)315 b(of)h(a)f(dynamic)h -(language)h(implementation.)g(All)e(the)h(numbers)27224 -55791 y(are)e(instances)g(of)g(either)g Fr(BoxedInteger)p -Black Black 315 w FC(or)g Fr(BoxedFloat)p Black Black -1 w FC(,)g(therefore)27224 56898 y(the)-15 b(y)296 b(consume)h(space)g -(on)f(the)g(heap.)h(Performing)f(man)-15 b(y)297 b(arithmetic)g(op-) -27224 58005 y(erations)319 b(produces)g(lots)f(of)h(g)-5 -b(arbage)320 b(quickly)-65 b(,)320 b(putting)f(pressure)f(on)h(the) -27224 59112 y(g)-5 b(arbage)310 b(collector)-55 b(.)310 -b(Using)f(double)g(dispatching)h(to)f(implement)h(the)f(nu-)27224 -60219 y(meric)192 b(to)-25 b(wer)193 b(needs)f(tw)-10 -b(o)193 b(method)g(calls)f(per)g(arithmetic)h(operation,)g(which)27224 -61326 y(is)248 b(costly)h(due)h(to)f(the)h(method)g(dispatch.)28552 -62433 y(Let)276 b(us)g(no)-25 b(w)277 b(consider)g(a)f(simple)g -(\223interpreter\224)i(function)e Fr(f)p Black Black -277 w FC(that)g(uses)27224 63540 y(the)213 b(object)g(model)g(\(see)g -(the)g(bottom)g(of)f(Figure)h(1\).)g(The)f(loop)h(in)g -Fr(f)p Black Black 213 w FC(iterates)27224 64646 y Fr(y)p -Black Black 334 w FC(times,)335 b(and)g(computes)g(something)f(in)h -(the)g(process.)f(Simply)g(running)27224 65753 y(this)277 -b(function)h(is)f(slo)-25 b(w)-65 b(,)278 b(because)h(there)f(are)h -(lots)e(of)g(virtual)h(method)h(calls)27224 66860 y(inside)264 -b(the)h(loop,)g(one)g(for)f(each)i Fr(is)39920 66651 -y(_)40453 66860 y(positive)p Black Black 265 w FC(and)f(e)-25 -b(v)-15 b(en)266 b(tw)-10 b(o)265 b(for)f(each)27224 -67967 y(call)446 b(to)g Fr(add)p Black Black 1 w FC(.)g(These)g(method) -g(calls)g(need)h(to)f(check)i(the)e(type)h(of)e(the)27224 -69074 y(in)-40 b(v)-20 b(olv)-15 b(ed)349 b(objects)f(repeatedly)i(and) -e(redundantly)-65 b(.)350 b(In)e(addition,)h(a)f(lot)g(of)27224 -70181 y(objects)224 b(are)g(created)h(when)g(e)-15 b(x)g(ecuting)225 -b(that)f(loop,)g(man)-15 b(y)225 b(of)e(these)h(objects)27224 -71288 y(are)341 b(short-li)-25 b(v)-15 b(ed.)342 b(The)f(actual)i -(computation)f(that)g(is)f(performed)g(by)h Fr(f)p Black -Black 341 w FC(is)27224 72395 y(simply)249 b(a)g(sequence)h(of)f -(\003oat)h(or)f(inte)-15 b(ger)249 b(additions.)p Black +TeXDict begin 2 1 bop Black Black Black -127 66 a FC(1.)p +Black 29 w(A)16 b(description)h(of)f(a)g(practical,)f(ef)n(\002cient)h +(and)h(ef)n(fecti)n(v)o(e)f(algorithm)g(for)-42 149 y(remo)o(ving)k +(object)f(allocations)h(in)e(a)h(tracing)g(JIT)-6 b(.)p +Black -127 266 a(2.)p Black 29 w(A)19 b(characterization)g(of)g(this)g +(algorithm)g(as)g(partial)g(e)n(v)n(aluation.)p Black +-127 382 a(3.)p Black 29 w(Performance)g(benchmarks)i(for)e(this)g +(algorithm.)-150 583 y FA(2.)91 b(Backgr)n(ound)-150 +699 y Fs(2.1)75 b(PyPy)-150 816 y FC(The)27 b(w)o(ork)h(described)g(in) +f(this)g(paper)h(w)o(as)f(done)h(in)f(the)g(conte)o(xt)h(of)f(the)-150 +902 y(PyPy)g(project)242 870 y Fz(3)299 902 y FC([26)q(].)g(PyPy)g(is)g +(an)h(en)m(vironment)h(where)f(dynamic)g(lan-)-150 985 +y(guages)h(can)g(be)g(implemented)g(in)f(a)g(simple)g(yet)h(ef)n +(\002cient)f(w)o(ay)-5 b(.)28 b(When)-150 1068 y(implementing)d(a)f +(language)h(with)f(PyPy)f(one)i(writes)e(an)h Fx(interpr)m(eter)i +FC(for)-150 1151 y(the)k(language)h(in)f Fx(RPython)f +FC([1)q(].)g(RPython)h(\("restricted)f(Python"\))i(is)e(a)-150 +1234 y(subset)18 b(of)g(Python)g(chosen)h(in)f(such)g(a)g(w)o(ay)g +(that)g(type)g(inference)g(becomes)-150 1317 y(possible.)28 +b(The)g(language)i(interpreter)e(can)h(then)f(be)g(compiled)h +(\(\223trans-)-150 1400 y(lated\224\))20 b(with)g(PyPy')l(s)g(tools)g +(into)h(a)f(VM)h(on)f(C)g(le)n(v)o(el.)g(During)h(translation)-150 +1483 y(to)16 b(C,)g(man)o(y)h(lo)n(w-le)n(v)o(el)g(aspects)g(of)f(the)h +(\002nal)f(VM,)g(such)i(as)e(object)h(layout,)-150 1566 +y(garbage)25 b(collection)g(and)f(memory)h(model,)g(are)f(w)o(o)o(v)o +(en)h(into)f(the)g(gener)o(-)-150 1649 y(ated)19 b(code.)g(Therefore)g +(the)g(interpreter)f(itself)g(can)h(remain)g(at)g(a)f(relati)n(v)o(ely) +-150 1732 y(high)i(le)n(v)o(el)e(of)h(abstraction.)-50 +1815 y(A)34 b(number)i(of)e(languages)j(ha)o(v)o(e)d(been)i +(implemented)g(with)e(PyPy)-5 b(.)-150 1898 y(The)29 +b(project)g(w)o(as)g(initiated)f(to)h(get)f(a)h(better)g(Python)g +(implementation,)-150 1981 y(which)24 b(inspired)g(the)g(name)h(of)e +(the)h(project)g(and)h(is)e(still)g(the)h(main)g(focus)-150 +2064 y(of)34 b(de)n(v)o(elopment.)h(In)e(addition)i(a)e(number)i(of)e +(other)h(languages)h(were)-150 2147 y(implemented,)17 +b(among)g(them)f(a)g(Prolog)g(interpreter)g([7],)f(a)h(Smalltalk)f(VM) +-150 2230 y([6])k(and)g(a)g(GameBo)o(y)h(emulator)f([8)q(].)-50 +2313 y(The)i(feature)h(that)g(mak)o(es)h(PyPy)e(more)h(than)h(a)f +(compiler)g(with)f(a)h(run-)-150 2396 y(time)j(system)h(is)f(its)g +(support)i(for)e(automated)i(JIT)e(compiler)h(generation)-150 +2479 y([5].)e(During)g(the)g(translation)g(to)g(C,)g(PyPy')l(s)f(tools) +h(can)h(generate)g(a)f(trac-)-150 2562 y(ing)31 b(just-in-time)e +(compiler)i(for)f(the)g(language)i(that)e(the)g(interpreter)h(is)-150 +2645 y(implementing.)26 b(This)g(process)h(is)e(mostly)h(automatic;)g +(it)f(only)i(needs)f(to)-150 2728 y(be)g(guided)h(by)g(the)f(language)h +(implementer)g(using)g(a)f(small)f(number)i(of)-150 2811 +y(source-code)22 b(hints)f(in)g(the)g(interpreter)l(.)f +(Mostly-automatically)i(generat-)-150 2894 y(ing)16 b(a)g(JIT)g +(compiler)g(has)g(man)o(y)h(adv)n(antages)h(o)o(v)o(er)e(writing)g(one) +h(manually)-5 b(,)-150 2977 y(an)21 b(error)o(-prone)f(and)h(tedious)g +(process.)g(By)f(construction,)h(the)f(generated)-150 +3060 y(JIT)d(has)i(the)f(same)g(semantics)g(as)g(the)g(interpreter)l(.) +f(Optimizations)i(can)f(be)-150 3143 y(shared)i(between)f(dif)n(ferent) +g(languages)i(implemented)f(with)e(PyPy)-5 b(.)-50 3226 +y(Moreo)o(v)o(er)m(,)23 b(thanks)h(to)e(the)h(internal)g(design)g(of)g +(the)g(JIT)f(generator)m(,)h(it)-150 3309 y(is)18 b(v)o(ery)h(easy)g +(to)f(add)i(ne)n(w)e Fx(bac)o(k)o(ends)j FC(for)d(producing)i(the)f +(actual)f(machine)-150 3392 y(code.)g(Examples)f(of)g(JIT)g(back)o +(ends)i(that)e(are)g(implemented)i(are)e(those)g(for)-150 +3475 y(Intel)26 b(x86)h(and)g(x86-64)h(and)e(an)h(e)o(xperimental)g +(one)g(for)f(the)g(CLI)g(.NET)-150 3558 y(V)l(irtual)18 +b(Machine)i([12)q(].)-150 3715 y Fs(2.2)75 b(T)-6 b(racing)19 +b(JIT)g(Compilers)-150 3831 y FC(T)m(racing)24 b(JITs)e(are)h(a)h +(recently)f(popular)h(approach)h(to)e(write)g(just-in-time)-150 +3914 y(compilers)e(for)g(dynamic)h(languages.)g(Their)e(origins)h(lie)g +(in)f(the)h(Dynamo)-150 3997 y(project,)j(which)h(used)h(a)e(tracing)h +(approach)h(to)e(optimize)h(machine)g(code)-150 4080 +y(using)20 b(e)o(x)o(ecution)f(traces)g([2)q(].)f(T)m(racing)h(JITs)g +(ha)o(v)o(e)g(then)g(be)g(adapted)h(to)f(be)-150 4163 +y(used)k(for)g(a)f(v)o(ery)i(light-weight)e(Ja)o(v)n(a)h(VM)g([15])g +(and)g(afterw)o(ards)g(used)g(in)-150 4246 y(se)n(v)o(eral)e +(implementations)g(of)g(dynamic)g(languages,)h(such)f(as)g(Ja)o(v)n +(aScript)-150 4332 y([13)q(],)d(Lua)128 4300 y Fz(4)176 +4332 y FC(and)h(no)n(w)h(Python)f(\(and)h(other)f(languages\))h(via)f +(PyPy)-5 b(.)-50 4415 y(The)20 b(core)h(idea)g(of)f(tracing)h(JITs)f +(is)g(to)g(focus)h(the)f(optimization)h(ef)n(fort)-150 +4498 y(of)26 b(the)f(JIT)g(compiler)h(on)g(the)g(commonly)h(e)o(x)o +(ecuted,)g(i.e.,)d Fx(hot)j FC(paths)f(of)-150 4581 y(the)k(core)h +(loops)g(of)g(the)f(program)h(and)g(to)g(just)f(use)g(an)h(interpreter) +f(for)-150 4664 y(the)23 b(less)g(commonly)h(e)o(x)o(ecuted)g(parts.)f +(VMs)g(that)g(use)g(a)g(tracing)g(JIT)f(are)-150 4747 +y(mostly)g(mix)o(ed-mode)h(e)o(x)o(ecution)g(en)m(vironments,)g(the)o +(y)f(contain)h(both)f(an)-150 4830 y(interpreter)28 b(and)h(a)e(JIT)h +(compiler)l(.)g(By)g(def)o(ault)g(the)g(interpreter)g(is)g(used)-150 +4913 y(to)g(e)o(x)o(ecute)h(the)f(program,)g(doing)h(some)f +(light-weight)g(pro\002ling)h(at)e(the)-150 4996 y(same)g(time.)f(This) +g(pro\002ling)h(is)f(used)i(to)f(identify)f(the)h(hot)g(loops)h(of)e +(the)-150 5079 y(program.)21 b(If)f(a)h(hot)f(loop)h(is)g(found)g(in)f +(that)h(w)o(ay)-5 b(,)21 b(the)f(interpreter)h(enters)f(a)-150 +5162 y(special)k Fx(tr)o(acing)h(mode)p FC(.)f(In)g(this)g(tracing)g +(mode,)h(the)f(interpreter)g(tries)f(to)p Black -150 +5258 997 3 v -150 5309 a Fz(3)-113 5333 y Fr(http://pypy.org)-150 +5406 y Fz(4)-113 5430 y Fr(http://luajit.org/)p Black +Black Black 2042 66 a FC(record)17 b(all)g(operations)h(that)f(it)g(is) +f(e)o(x)o(ecuting)i(while)f(running)i(one)e(iteration)2042 +149 y(of)22 b(the)g(hot)g(loop.)h(This)e(history)i(of)f(e)o(x)o(ecuted) +h(operations)g(of)f(one)h(loop)f(is)2042 232 y(called)h(a)g +Fx(tr)o(ace)p FC(.)g(Because)h(the)g(trace)f(corresponds)i(to)e(one)h +(iteration)f(of)g(a)2042 315 y(loop,)j(it)e(al)o(w)o(ays)j(ends)f(with) +f(a)h(jump)g(to)f(its)g(o)n(wn)h(be)o(ginning.)h(The)e(trace)2042 +399 y(also)31 b(contains)g(all)f(operations)i(that)f(are)f(performed)i +(in)f(functions)g(that)2042 482 y(were)23 b(called)g(in)g(the)f(loop,)i +(thus)f(a)g(tracing)g(JIT)f(automatically)i(performs)2042 +565 y(inlining.)h(This)f(trace)h(of)g(operations)h(subsequently)h +(forms)e(the)g(basis)g(of)2042 648 y(the)j(generated)h(code.)g(The)f +(trace)g(is)g(\002rst)f(optimized,)h(and)h(then)f(turned)2042 +731 y(into)15 b(machine)i(code.)f(Both)g(optimization)g(and)g(machine)h +(code)f(generation)2042 814 y(are)h(simple,)g(because)h(the)f(traces)g +(are)g(linear)l(.)g(This)g(linearity)f(mak)o(es)i(man)o(y)2042 +897 y(optimizations)29 b(a)g(lot)f(more)h(tractable,)f(and)h(the)g +(inlining)g(that)f(happens)2042 980 y(gi)n(v)o(es)19 +b(the)g(optimizations)h(automatically)f(more)g(conte)o(xt)h(to)f(w)o +(ork)g(with.)2141 1063 y(Since)f(the)g(trace)f(corresponds)j(to)e(one)g +(concrete)h(e)o(x)o(ecution)f(of)g(a)g(loop,)2042 1146 +y(the)27 b(code)h(generated)g(from)f(it)f(is)h(only)h(one)g(possible)f +(path)h(through)g(the)2042 1229 y(loop.)23 b(T)-6 b(o)23 +b(mak)o(e)h(sure)f(that)g(the)g(trace)f(maintains)i(the)f(correct)g +(semantics,)2042 1312 y(it)k(contains)j(a)e Fx(guar)m(d)k +FC(at)c(all)g(places)g(where)h(the)f(e)o(x)o(ecution)i(could)f(ha)o(v)o +(e)2042 1395 y(di)n(v)o(er)o(ged)19 b(from)g(the)f(path.)h(Those)g +(guards)g(check)h(the)e(assumptions)i(under)2042 1478 +y(which)28 b(e)o(x)o(ecution)h(can)f(stay)g(on)h(the)f(trace.)f(As)h +(an)g(e)o(xample,)g(if)g(a)f(loop)2042 1561 y(contains)32 +b(an)g(if-statement,)f(the)h(trace)g(will)e(contain)j(the)e(e)o(x)o +(ecution)i(of)2042 1644 y(one)24 b(of)f(the)g(paths)h(only)-5 +b(,)24 b(which)f(is)g(the)h(path)f(that)g(w)o(as)h(tak)o(en)g(during)g +(the)2042 1727 y(production)30 b(of)g(the)f(trace.)g(The)g(trace)g +(will)f(also)h(contain)h(a)f(guard)h(that)2042 1810 y(checks)23 +b(that)f(the)g(condition)h(of)g(the)f(if-statement)g(is)f(the)i(same)f +(as)g(during)2042 1893 y(tracing,)d(because)h(if)e(it)g(isn')o(t,)g +(the)h(rest)g(of)g(the)g(trace)g(w)o(ould)g(not)g(be)h(v)n(alid.)2141 +1976 y(When)30 b(generating)h(machine)f(code,)g(e)n(v)o(ery)g(guard)g +(is)g(turned)g(into)f(a)2042 2059 y(quick)23 b(check)g(to)f(see)g +(whether)g(the)g(assumption)i(still)d(holds.)h(When)g(such)2042 +2142 y(a)28 b(guard)i(is)e(hit)g(during)h(the)g(e)o(x)o(ecution)g(of)g +(the)f(machine)i(code)f(and)g(the)2042 2225 y(assumption)h(does)g(not)g +(hold,)g(the)f(e)o(x)o(ecution)h(of)g(the)f(machine)h(code)g(is)2042 +2308 y(stopped,)20 b(and)h(interpreter)f(continues)h(to)e(run)h(from)g +(that)g(point)g(on.)g(These)2042 2391 y(guards)g(are)f(the)g(only)g +(mechanism)h(to)f(stop)h(the)f(e)o(x)o(ecution)h(of)f(a)f(trace,)h(the) +2042 2474 y(loop)g(end)h(condition)g(also)f(tak)o(es)g(the)g(form)g(of) +g(a)g(guard.)2141 2557 y(If)f(one)h(speci\002c)g(guard)g(f)o(ails)f(a)h +(lot)f(\(i.e.,)f(more)i(than)g(some)g(threshold\),)2042 +2640 y(the)25 b(tracing)g(JIT)f(will)f(generate)j(a)f(ne)n(w)g(trace)g +(that)f(starts)g(e)o(xactly)i(at)e(the)2042 2723 y(position)16 +b(of)g(the)g(f)o(ailing)h(guard)g([14].)e(The)h(e)o(xisting)h +(assembler)f(is)g(patched)2042 2806 y(to)35 b(jump)h(to)f(the)h(ne)n(w) +f(trace)h(when)g(the)f(guard)i(f)o(ails.)d(This)h(approach)2042 +2889 y(guarantees)23 b(that)g(all)f(the)g(hot)h(paths)g(in)f(the)h +(program)g(will)e(e)n(v)o(entually)j(be)2042 2972 y(traced)19 +b(and)h(compiled)f(into)g(ef)n(\002cient)g(code.)2042 +3238 y Fs(2.3)75 b(Running)17 b(Example)2042 3354 y FC(F)o(or)h(the)h +(purpose)i(of)e(this)f(paper)m(,)i(we)f(are)g(going)h(to)f(use)g(a)g +(tin)o(y)g(interpreter)2042 3437 y(for)j(a)g(dynamic)h(language)g(with) +f(a)g(v)o(ery)h(simple)f(object)g(model,)g(that)g(just)2042 +3520 y(supports)h(an)g(inte)o(ger)g(and)g(a)f(\003oat)g(type.)h(The)f +(objects)h(support)h(only)f(tw)o(o)2042 3603 y(operations,)i +Fr(add)p Black Black FC(,)f(which)g(adds)h(tw)o(o)g(objects)g +(\(promoting)g(ints)f(to)g(\003oats)2042 3686 y(in)17 +b(a)h(mix)o(ed)g(addition\))g(and)h Fr(is)2864 3671 y(_)2904 +3686 y(positive)p Black Black FC(,)e(which)h(returns)g(whether)g(the) +2042 3769 y(number)i(is)g(greater)f(than)i(zero.)e(The)h +(implementation)g(of)g Fr(add)p Black Black 19 w FC(uses)g(clas-)2042 +3852 y(sical)f(Smalltalk-lik)o(e)g(double-dispatching.)i(The)e(classes) +h(can)g(be)f(seen)h(in)2042 3935 y(Figure)f(1)g(\(written)f(in)h +(RPython\).)2141 4018 y(Using)31 b(these)g(classes)g(to)f(implement)h +(arithmetic)g(sho)n(ws)g(the)g(basic)2042 4101 y(problem)24 +b(of)g(a)g(dynamic)h(language)g(implementation.)g(All)e(the)h(numbers) +2042 4184 y(are)g(instances)g(of)g(either)f Fr(BoxedInteger)p +Black Black 24 w FC(or)h Fr(BoxedFloat)p Black Black +-1 w FC(,)g(therefore)2042 4267 y(the)o(y)f(consume)g(space)h(on)e(the) +h(heap.)g(Performing)f(man)o(y)i(arithmetic)e(op-)2042 +4350 y(erations)i(produces)i(lots)e(of)g(garbage)h(quickly)-5 +b(,)25 b(putting)g(pressure)f(on)h(the)2042 4433 y(garbage)f(collector) +l(.)f(Using)h(double)g(dispatching)h(to)e(implement)h(the)g(nu-)2042 +4516 y(meric)14 b(to)n(wer)h(needs)h(tw)o(o)e(method)i(calls)e(per)h +(arithmetic)f(operation,)i(which)2042 4599 y(is)i(costly)h(due)h(to)f +(the)g(method)h(dispatch.)2141 4682 y(Let)h(us)g(no)n(w)g(consider)h(a) +f(simple)g(\223interpreter\224)g(function)h Fr(f)p Black +Black 21 w FC(that)f(uses)2042 4765 y(the)16 b(object)h(model)f(\(see)h +(the)f(bottom)h(of)f(Figure)g(1\).)g(The)g(loop)h(in)f +Fr(f)p Black Black 16 w FC(iterates)2042 4848 y Fr(y)p +Black Black 25 w FC(times,)25 b(and)g(computes)i(something)f(in)g(the)f +(process.)h(Simply)f(running)2042 4932 y(this)c(function)g(is)g(slo)n +(w)-5 b(,)21 b(because)h(there)g(are)f(lots)f(of)h(virtual)g(method)h +(calls)2042 5015 y(inside)e(the)g(loop,)h(one)f(for)g(each)h +Fr(is)2994 4999 y(_)3034 5015 y(positive)p Black Black +20 w FC(and)f(e)n(v)o(en)h(tw)o(o)f(for)g(each)2042 5098 +y(call)33 b(to)h Fr(add)p Black Black FC(.)f(These)h(method)g(calls)g +(need)g(to)g(check)g(the)g(type)g(of)g(the)2042 5181 +y(in)m(v)o(olv)o(ed)27 b(objects)g(repeatedly)g(and)g(redundantly)-5 +b(.)28 b(In)e(addition,)h(a)f(lot)g(of)2042 5264 y(objects)17 +b(are)g(created)h(when)f(e)o(x)o(ecuting)h(that)f(loop,)h(man)o(y)f(of) +g(these)h(objects)2042 5347 y(are)26 b(short-li)n(v)o(ed.)g(The)f +(actual)h(computation)i(that)d(is)h(performed)h(by)f +Fr(f)p Black Black 25 w FC(is)2042 5430 y(simply)19 b(a)g(sequence)h +(of)f(\003oat)g(or)g(inte)o(ger)g(additions.)p Black Black eop end %%Page: 3 3 -TeXDict begin 3 2 bop Black Black Black -2001 1315 a -Fq(c)o(l)o(a)o(s)o(s)463 b Fp(B)o(a)o(s)o(e\()n(o)o(b)o(j)o(e)o(c)o(t)o -(\))o(:)-606 2245 y Fq(p)o(a)o(s)o(s)-2001 4105 y(c)o(l)o(a)o(s)o(s)g -Fp(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)o(\()n(B)o(a)o(s)o(e)o -(\))o(:)-606 5034 y Fq(d)o(e)o(f)1253 4851 y Fp(_)o(_)2183 -5034 y(i)o(n)o(i)o(t)4043 4851 y(_)o(_)4973 5034 y(\()o(s)o(e)o(l)o(f)o -(,)g(i)o(n)o(t)o(v)o(a)o(l)o(\))o(:)788 5964 y(s)o(e)o(l)o(f.)n(i)o(n)o -(t)o(v)o(a)o(l)h(=)f(i)o(n)o(t)o(v)o(a)o(l)-606 7824 -y Fq(d)o(e)o(f)g Fp(a)o(d)o(d\()n(s)o(e)o(l)o(f)o(,)g(o)o(t)o(h)o(e)o -(r)o(\))o(:)788 8754 y Fq(r)o(e)o(t)o(u)o(r)o(n)h Fp(o)o(t)o(h)o(e)o(r) -o(.)n(a)o(d)o(d)8227 8571 y(_)o(_)9157 8754 y(i)o(n)o(t)o(\()o(s)o(e)o -(l)o(f)o(.)n(i)o(n)o(t)o(v)o(a)o(l)o(\))-606 10614 y -Fq(d)o(e)o(f)f Fp(a)o(d)o(d)2648 10431 y(_)o(_)3578 10614 -y(i)o(n)o(t)o(\()o(s)o(e)o(l)o(f)o(,)g(i)o(n)o(t)o(o)o(t)o(h)o(e)o(r)o -(\))o(:)788 11543 y Fq(r)o(e)o(t)o(u)o(r)o(n)h Fp(B)o(o)o(x)o(e)o(d)o -(I)o(n)o(t)o(e)o(g)o(e)o(r)n(\()o(i)o(n)o(t)o(o)o(t)o(h)o(e)o(r)g(+)f -(s)o(e)o(l)o(f)o(.)o(i)o(n)o(t)o(v)o(a)o(l)o(\))-606 -13403 y Fq(d)o(e)o(f)g Fp(a)o(d)o(d)2648 13220 y(_)o(_)3578 -13403 y(f)o(l)o(o)o(a)o(t)o(\()o(s)o(e)o(l)o(f)o(,)g(f)o(l)o(o)o(a)o(t) -o(o)o(t)o(h)o(e)o(r)o(\))o(:)788 14333 y(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l) -o(u)o(e)h(=)f(f)o(l)o(o)o(a)o(t)o(o)o(t)o(h)o(e)o(r)h(+)f(f)o(l)o(o)o -(a)o(t\()n(s)o(e)o(l)o(f)o(.)o(i)o(n)o(t)o(v)o(a)o(l)o(\))788 -15263 y Fq(r)o(e)o(t)o(u)o(r)o(n)h Fp(B)o(o)o(x)o(e)o(d)o(F)o(l)o(o)o -(a)o(t)o(\()n(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l)o(u)o(e)o(\))-606 -17123 y Fq(d)o(e)o(f)f Fp(i)o(s)2183 16940 y(_)2648 17123 -y(p)o(o)o(s)o(i)o(t)o(i)o(v)o(e)o(\()n(s)o(e)o(l)o(f)o(\))o(:)788 -18052 y Fq(r)o(e)o(t)o(u)o(r)o(n)h Fp(s)o(e)o(l)o(f)o(.)n(i)o(n)o(t)o -(v)o(a)o(l)g(>)g(0)-2001 19912 y Fq(c)o(l)o(a)o(s)o(s)f -Fp(B)o(o)o(x)o(e)o(d)o(F)o(l)o(o)o(a)o(t)o(\()o(B)o(a)o(s)o(e)n(\))o(:) --606 20842 y Fq(d)o(e)o(f)1253 20659 y Fp(_)o(_)2183 -20842 y(i)o(n)o(i)o(t)4043 20659 y(_)o(_)4973 20842 y(\()o(s)o(e)o(l)o -(f)o(,)g(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l)o(\))o(:)788 21772 -y(s)o(e)o(l)o(f.)n(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l)h(=)f(f)o(l)o(o)o(a)o -(t)o(v)o(a)o(l)-606 23631 y Fq(d)o(e)o(f)g Fp(a)o(d)o(d\()n(s)o(e)o(l)o -(f)o(,)g(o)o(t)o(h)o(e)o(r)o(\))o(:)788 24561 y Fq(r)o(e)o(t)o(u)o(r)o -(n)h Fp(o)o(t)o(h)o(e)o(r)o(.)n(a)o(d)o(d)8227 24378 -y(_)o(_)9157 24561 y(f)o(l)o(o)o(a)o(t)o(\()o(s)o(e)o(l)o(f)o(.)n(f)o -(l)o(o)o(a)o(t)o(v)o(a)o(l)o(\))-606 26421 y Fq(d)o(e)o(f)f -Fp(a)o(d)o(d)2648 26238 y(_)o(_)3578 26421 y(i)o(n)o(t)o(\()o(s)o(e)o -(l)o(f)o(,)g(i)o(n)o(t)o(o)o(t)o(h)o(e)o(r)o(\))o(:)788 -27351 y(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l)o(u)o(e)h(=)f(f)o(l)o(o)o(a)o(t)o -(\()o(i)o(n)o(t)o(o)o(t)o(h)o(e)o(r)o(\))h(+)f(s)o(e)o(l)o(f)o(.)o(f)o -(l)o(o)o(a)o(t)o(v)o(a)o(l)788 28281 y Fq(r)o(e)o(t)o(u)o(r)o(n)h -Fp(B)o(o)o(x)o(e)o(d)o(F)o(l)o(o)o(a)o(t)o(\()n(f)o(l)o(o)o(a)o(t)o(v)o -(a)o(l)o(u)o(e)o(\))-606 30140 y Fq(d)o(e)o(f)f Fp(a)o(d)o(d)2648 -29957 y(_)o(_)3578 30140 y(f)o(l)o(o)o(a)o(t)o(\()o(s)o(e)o(l)o(f)o(,)g -(f)o(l)o(o)o(a)o(t)o(o)o(t)o(h)o(e)o(r)o(\))o(:)788 31070 -y Fq(r)o(e)o(t)o(u)o(r)o(n)h Fp(B)o(o)o(x)o(e)o(d)o(F)o(l)o(o)o(a)o(t)o -(\()n(f)o(l)o(o)o(a)o(t)o(o)o(t)o(h)o(e)o(r)g(+)f(s)o(e)o(l)o(f)o(.)o -(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l)o(\))-606 32930 y Fq(d)o(e)o(f)g -Fp(i)o(s)2183 32747 y(_)2648 32930 y(p)o(o)o(s)o(i)o(t)o(i)o(v)o(e)o -(\()n(s)o(e)o(l)o(f)o(\))o(:)788 33860 y Fq(r)o(e)o(t)o(u)o(r)o(n)h -Fp(s)o(e)o(l)o(f)o(.)n(f)o(l)o(o)o(a)o(t)o(v)o(a)o(l)g(>)g(0)o(.)o(0) --2001 36649 y Fq(d)o(e)o(f)g Fp(f)o(\()o(y)n(\))o(:)-606 -37579 y(r)o(e)o(s)g(=)f(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)n -(\()o(0)o(\))-607 38509 y Fq(w)o(h)o(i)o(l)o(e)i Fp(y)o(.)n(i)o(s)4043 -38326 y(_)4508 38509 y(p)o(o)o(s)o(i)o(t)o(i)o(v)o(e)n(\()o(\))o(:)788 -39439 y(r)o(e)o(s)g(=)e(r)o(e)o(s)o(.)o(a)o(d)o(d)o(\()o(y)n(\))o(.)o -(a)o(d)o(d)o(\()o(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)n(\()n -(-)o(1)o(0)o(0)o(\))o(\))789 40369 y(y)h(=)g(y)n(.)o(a)o(d)o(d)o(\()n -(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)o(\()n(-)o(1)o(\))o(\)) --607 41299 y Fq(r)o(e)o(t)o(u)o(r)o(n)g Fp(r)o(e)o(s)p -Black Black -2000 42279 26568 37 v -2000 43407 a Fs(Figur)-18 -b(e)269 b(1.)499 b FC(An)269 b(\223Interpreter\224)h(for)e(a)h(T)-35 -b(in)-15 b(y)269 b(Dynamic)h(Language)h(Written)-2000 -44514 y(in)249 b(RPython)p Black -672 48119 a(If)336 -b(the)h(function)f(is)g(e)-15 b(x)g(ecuted)338 b(using)e(the)g(tracing) -h(JIT)-74 b(,)336 b(with)g Fr(y)p Black Black 336 w FC(being)-2000 -49226 y(a)383 b Fr(BoxedInteger)p Black Black 1 w FC(,)g(the)h -(produced)f(trace)h(looks)f(lik)-10 b(e)383 b(the)g(one)h(of)e(Fig-) --2000 50333 y(ure)268 b(2)f(\(lines)g(starting)h(with)g(a)f(hash)h -(\223#\224)g(are)g(comments\).)g(The)g(trace)g(cor)-20 -b(-)-2000 51440 y(responds)249 b(to)g(one)h(iteration)f(of)g(the)h -(while-loop)f(in)h Fr(f)p Black Black FC(.)-672 52547 -y(The)364 b(operations)g(in)f(the)h(trace)h(are)e(indented)i -(corresponding)f(to)g(the)-2000 53654 y(stack)323 b(le)-25 -b(v)-15 b(el)324 b(of)e(the)i(function)f(that)g(contains)g(the)h -(traced)f(operation.)h(The)-2000 54761 y(trace)358 b(is)e(in)h -(single-assignment)g(form,)f(meaning)i(that)g(each)g(v)-25 -b(ariable)358 b(is)-2000 55868 y(assigned)223 b(a)h(v)-25 -b(alue)224 b(e)-15 b(xactly)225 b(once.)f(The)f(ar)-18 -b(guments)224 b Fo(p)16757 55979 y Fn(0)17442 55868 y -FC(and)f Fo(p)19617 55979 y Fn(1)20302 55868 y FC(of)g(the)h(loop)-2000 -56975 y(correspond)267 b(to)f(the)g(li)-25 b(v)-15 b(e)267 -b(v)-25 b(ariables)267 b Fr(y)p Black Black 266 w FC(and)g -Fr(res)p Black Black 267 w FC(in)f(the)h(while-loop)g(of)f(the)-2000 -58081 y(original)249 b(function.)-672 59188 y(The)305 -b(operations)h(in)f(the)g(trace)h(correspond)f(to)g(the)g(operations)h -(in)f(the)-2000 60295 y(RPython)249 b(program)g(in)g(Figure)h(1:)p -Black -1419 62335 a Fm(\017)p Black -561 62433 a Fr(new)p -Black Black 249 w FC(creates)g(a)f(ne)-25 b(w)250 b(object.)p -Black -1419 63884 a Fm(\017)p Black -561 63982 a Fr(get)p -Black Black 249 w FC(reads)g(an)f(attrib)-20 b(ute)250 -b(of)f(an)g(object.)p Black -1419 65434 a Fm(\017)p Black --561 65532 a Fr(set)p Black Black 249 w FC(writes)g(to)g(an)h(attrib) --20 b(ute)250 b(of)f(an)g(object.)p Black -1419 66984 -a Fm(\017)p Black -561 67082 a Fr(guard)2104 66873 y(_)2637 -67082 y(class)p Black Black 224 w FC(is)222 b(a)g(precise)h(type)g -(check)h(and)f(precedes)g(an)g(\(inlined\))-561 68189 -y(method)250 b(call)g(and)f(is)g(follo)-25 b(wed)250 -b(by)f(the)g(trace)h(of)f(the)h(called)g(method.)p Black --1419 69640 a Fm(\017)p Black -561 69738 a Fr(int)1038 -69529 y(_)1571 69738 y(add)p Black Black 492 w FC(and)492 -b Fr(int)7191 69529 y(_)7724 69738 y(gt)p Black Black -492 w FC(are)f(inte)-15 b(ger)492 b(addition)g(and)g(comparison)-561 -70845 y(\(\223greater)250 b(than\224\),)g(respecti)-25 -b(v)-15 b(ely)-65 b(.)p Black -1419 72297 a Fm(\017)p -Black -561 72395 a Fr(guard)2104 72186 y(_)2637 72395 -y(true)p Black Black 250 w FC(checks)250 b(that)g(a)f(boolean)h(is)f -(true.)p Black Black Black 51577 1315 a Fl(1)p 0.3 0.3 0.3 -TeXcolorrgb -24631 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 464 w(a)o(r)o(g)o(u)o(m)o(e)o(n)o(t)o(s)p -Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -463 w(t)o(o)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 -TeXcolorrgb 464 w(t)o(h)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 463 w(t)o(r)o(a)o(c)o(e)p -Black 0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 -TeXcolorrgb Black 465 w Fj(p)39768 1426 y Fi(0)p 0.3 0.3 0.3 -TeXcolorrgb 40199 1315 a Fk(,)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 465 w Fj(p)41586 1426 y Fi(1)51577 2245 y Fl(2)p -0.3 0.3 0.3 TeXcolorrgb -24631 w Fk(#)p Black 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p -Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -463 w(f)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black +TeXDict begin 3 2 bop Black Black Black -150 99 a Fq(class)34 +b Fp(Base\()o(object\):)-45 168 y Fq(pass)-150 308 y(class)g +Fp(BoxedInteger)o(\()o(Base\):)-45 378 y Fq(def)94 364 +y Fp(__)164 378 y(init)304 364 y(__)373 378 y(\(self)o(,)h(intval)o +(\):)59 447 y(self.)o(intval)g(=)f(intval)-45 587 y Fq(def)g +Fp(add\()o(self,)h(other)o(\):)59 657 y Fq(return)f Fp(other.)o(add)617 +643 y(__)687 657 y(int)o(\(self.intval)o(\))-45 796 y +Fq(def)g Fp(add)199 782 y(__)269 796 y(int)o(\(self)o(,)h(intother)o +(\):)59 866 y Fq(return)f Fp(BoxedInteger)o(\(intother)f(+)i(self.)o +(intval\))-45 1005 y Fq(def)f Fp(add)199 991 y(__)269 +1005 y(float)o(\(self)o(,)h(floatother)n(\):)59 1075 +y(floatvalue)f(=)g(floatother)g(+)h(float)o(\(self)o(.intval\))59 +1145 y Fq(return)f Fp(BoxedFloat)o(\(floatvalue)o(\))-45 +1284 y Fq(def)g Fp(is)164 1270 y(_)199 1284 y(positive)o(\()o(self\):) +59 1354 y Fq(return)g Fp(self.intval)g(>)g(0)-150 1493 +y Fq(class)g Fp(BoxedFloat)o(\(Base)o(\):)-45 1563 y +Fq(def)94 1549 y Fp(__)164 1563 y(init)304 1549 y(__)373 +1563 y(\(self)o(,)h(floatval)o(\):)59 1633 y(self.)o(floatval)f(=)h +(floatval)-45 1772 y Fq(def)f Fp(add\()o(self,)h(other)o(\):)59 +1842 y Fq(return)f Fp(other.)o(add)617 1828 y(__)687 +1842 y(float)o(\(self.)o(floatval)o(\))-45 1982 y Fq(def)g +Fp(add)199 1968 y(__)269 1982 y(int)o(\(self)o(,)h(intother)o(\):)59 +2051 y(floatvalue)f(=)g(float\(intother)o(\))g(+)h(self.)o(floatval)59 +2121 y Fq(return)f Fp(BoxedFloat)o(\(floatvalue)o(\))-45 +2261 y Fq(def)g Fp(add)199 2247 y(__)269 2261 y(float)o(\(self)o(,)h +(floatother)n(\):)59 2330 y Fq(return)f Fp(BoxedFloat)o(\(floatother)f +(+)i(self.)o(floatval)o(\))-45 2470 y Fq(def)f Fp(is)164 +2456 y(_)199 2470 y(positive)o(\()o(self\):)59 2539 y +Fq(return)g Fp(self.floatval)f(>)i(0.0)-150 2749 y Fq(def)f +Fp(f\(y\):)-45 2818 y(res)g(=)h(BoxedInteger)n(\(0\))-45 +2888 y Fq(while)f Fp(y.)o(is)303 2874 y(_)338 2888 y(positive)o(\(\):) +59 2958 y(res)h(=)f(res.add)o(\(y\).add\()o(BoxedInteger)o(\(-100\)\)) +59 3028 y(y)h(=)g(y.)o(add\(BoxedInteger)n(\(-1\)\))-45 +3097 y Fq(return)f Fp(res)p Black Black -150 3171 1993 +3 v -150 3256 a Fs(Figur)o(e)19 b(1.)38 b FC(An)20 b +(\223Interpreter\224)h(for)f(a)h(T)m(in)o(y)f(Dynamic)h(Language)h +(Written)-150 3339 y(in)d(RPython)p Black -50 3609 a(If)25 +b(the)g(function)h(is)f(e)o(x)o(ecuted)i(using)f(the)g(tracing)f(JIT)-6 +b(,)25 b(with)g Fr(y)p Black Black 26 w FC(being)-150 +3692 y(a)k Fr(BoxedInteger)p Black Black FC(,)f(the)h(produced)i(trace) +e(looks)g(lik)o(e)g(the)g(one)h(of)f(Fig-)-150 3775 y(ure)20 +b(2)h(\(lines)f(starting)g(with)g(a)g(hash)h(\223#\224)g(are)f +(comments\).)h(The)f(trace)h(cor)o(-)-150 3858 y(responds)f(to)f(one)h +(iteration)e(of)h(the)g(while-loop)h(in)f Fr(f)p Black +Black FC(.)-50 3941 y(The)27 b(operations)i(in)e(the)h(trace)f(are)h +(indented)g(corresponding)i(to)d(the)-150 4024 y(stack)e(le)n(v)o(el)f +(of)h(the)f(function)h(that)g(contains)g(the)f(traced)h(operation.)g +(The)-150 4107 y(trace)i(is)g(in)f(single-assignment)j(form,)e(meaning) +h(that)e(each)i(v)n(ariable)g(is)-150 4190 y(assigned)18 +b(a)f(v)n(alue)h(e)o(xactly)f(once.)h(The)f(ar)o(guments)g +Fo(p)1257 4198 y Fn(0)1308 4190 y FC(and)h Fo(p)1472 +4198 y Fn(1)1523 4190 y FC(of)f(the)g(loop)-150 4273 +y(correspond)22 b(to)e(the)g(li)n(v)o(e)g(v)n(ariables)h +Fr(y)p Black Black 20 w FC(and)g Fr(res)p Black Black +20 w FC(in)f(the)g(while-loop)h(of)f(the)-150 4356 y(original)f +(function.)-50 4439 y(The)k(operations)h(in)f(the)g(trace)g(correspond) +i(to)e(the)g(operations)h(in)f(the)-150 4522 y(RPython)c(program)h(in)f +(Figure)g(1:)p Black -106 4675 a Fm(\017)p Black -42 +4682 a Fr(new)p Black Black 19 w FC(creates)g(a)g(ne)n(w)g(object.)p +Black -106 4792 a Fm(\017)p Black -42 4799 a Fr(get)p +Black Black 19 w FC(reads)g(an)g(attrib)o(ute)f(of)h(an)g(object.)p +Black -106 4908 a Fm(\017)p Black -42 4915 a Fr(set)p +Black Black 19 w FC(writes)f(to)h(an)g(attrib)o(ute)f(of)h(an)g +(object.)p Black -106 5024 a Fm(\017)p Black -42 5031 +a Fr(guard)158 5015 y(_)198 5031 y(class)p Black Black +16 w FC(is)e(a)g(precise)g(type)g(check)h(and)g(precedes)g(an)f +(\(inlined\))-42 5114 y(method)j(call)e(and)i(is)e(follo)n(wed)i(by)f +(the)g(trace)g(of)g(the)g(called)g(method.)p Black -106 +5223 a Fm(\017)p Black -42 5230 a Fr(int)78 5215 y(_)118 +5230 y(add)p Black Black 37 w FC(and)37 b Fr(int)539 +5215 y(_)579 5230 y(gt)p Black Black 37 w FC(are)g(inte)o(ger)g +(addition)h(and)g(comparison)-42 5313 y(\(\223greater)19 +b(than\224\),)g(respecti)n(v)o(ely)-5 b(.)p Black -106 +5423 a Fm(\017)p Black -42 5430 a Fr(guard)158 5414 y(_)198 +5430 y(true)p Black Black 18 w FC(checks)20 b(that)f(a)g(boolean)h(is)f +(true.)p Black Black Black 3868 99 a Fl(1)p 0.3 0.3 0.3 +TeXcolorrgb -1847 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 35 w(arguments)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -464 w(r)o(e)o(s)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(.)p -Black 0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)p Black -0.3 0.3 0.3 TeXcolorrgb -2 w(\()p Black 0.3 0.3 0.3 TeXcolorrgb --1 w(y)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(\))p Black -51577 3175 a Fl(3)-24631 b Fp(g)o(u)o(a)o(r)o(d)29548 -2992 y(_)30013 3175 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 -3286 y Fi(1)33690 3175 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)n(\))51577 4105 y Fl(4)p 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb -22772 w Fk(#)p Black 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p -Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p +33 w(to)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 35 w(the)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 34 w(trace)p Black 0.3 0.3 0.3 +TeXcolorrgb(:)p Black 0.3 0.3 0.3 TeXcolorrgb Black 34 +w Fj(p)2982 107 y Fi(0)p 0.3 0.3 0.3 TeXcolorrgb 3015 +99 a Fk(,)p Black 0.3 0.3 0.3 TeXcolorrgb Black 35 w +Fj(p)3119 107 y Fi(1)3868 168 y Fl(2)p 0.3 0.3 0.3 TeXcolorrgb +-1847 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 35 w(inside)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 34 w(f)p Black 0.3 0.3 0.3 +TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 TeXcolorrgb Black +0.3 0.3 0.3 TeXcolorrgb 35 w(res)p Black 0.3 0.3 0.3 +TeXcolorrgb(.)p Black 0.3 0.3 0.3 TeXcolorrgb(add)p Black +0.3 0.3 0.3 TeXcolorrgb -1 w(\()p Black 0.3 0.3 0.3 TeXcolorrgb(y)p +Black 0.3 0.3 0.3 TeXcolorrgb(\))p Black 3868 238 a Fl(3)-1847 +b Fp(guard)2217 224 y(_)2252 238 y(class)n(\()p Fj(p)2494 +246 y Fi(1)2527 238 y Fp(,)34 b(BoxedInteger)o(\))3868 +308 y Fl(4)p 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb -1708 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 35 w(inside)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 34 w(BoxedInteger)p +Black 0.3 0.3 0.3 TeXcolorrgb -1 w(.)p Black 0.3 0.3 0.3 +TeXcolorrgb -1 w(add)p Black 3868 378 a Fl(5)-1708 b +Fj(i)2204 386 y Fi(2)2272 378 y Fp(=)35 b(get)o(\()p +Fj(p)2515 386 y Fi(1)2548 378 y Fp(,)g(intval)o(\))3868 +447 y Fl(6)-1708 b Fp(guard)2356 433 y(_)2391 447 y(class)o(\()p +Fj(p)2634 455 y Fi(0)2666 447 y Fp(,)35 b(BoxedInteger)n(\))3868 +517 y Fl(7)p 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb -1568 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 34 w(inside)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 35 w(BoxedInteger)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black 0.3 0.3 0.3 -TeXcolorrgb -1 w(a)o(d)o(d)p Black 51577 5034 a Fl(5)-22771 -b Fj(i)29396 5145 y Fi(2)30293 5034 y Fp(=)463 b(g)o(e)o(t)o(\()p -Fj(p)33539 5145 y Fi(1)33970 5034 y Fp(,)h(i)o(n)o(t)o(v)o(a)o(l)o(\)) -51577 5964 y Fl(6)-22772 b Fp(g)o(u)o(a)o(r)o(d)31407 -5781 y(_)31872 5964 y(c)o(l)o(a)o(s)o(s)o(\()p Fj(p)35119 -6075 y Fi(0)35550 5964 y Fp(,)463 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)o(\))51577 6894 y Fl(7)p 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb -20912 w Fk(#)p Black 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p -Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p +TeXcolorrgb(add)3193 503 y(__)3263 517 y(int)p Black +3868 587 a Fl(8)-1568 b Fj(i)2344 595 y Fi(3)2411 587 +y Fp(=)35 b(get\()p Fj(p)2655 595 y Fi(0)2687 587 y Fp(,)g(intval)o(\)) +3868 657 y Fl(9)-1568 b Fj(i)2344 665 y Fi(4)2411 657 +y Fp(=)35 b(int)2586 643 y(_)2621 657 y(add)o(\()p Fj(i)2783 +665 y Fi(2)2816 657 y Fp(,)g Fj(i)2909 665 y Fi(3)2942 +657 y Fp(\))3868 726 y Fl(10)-1589 b Fj(p)2355 734 y +Fi(5)2422 726 y Fp(=)35 b(new\()o(BoxedInteger)o(\))3868 +796 y Fl(11)p 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb -1450 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 35 w(inside)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 34 w(BoxedInteger)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black 0.3 0.3 0.3 -TeXcolorrgb -1 w(a)o(d)o(d)42566 6711 y(_)o(_)43496 6894 -y(i)o(n)o(t)p Black 51577 7824 a Fl(8)-20911 b Fj(i)31256 -7935 y Fi(3)32153 7824 y Fp(=)463 b(g)o(e)o(t)o(\()p -Fj(p)35399 7935 y Fi(0)35830 7824 y Fp(,)h(i)o(n)o(t)o(v)o(a)o(l)o(\)) -51577 8754 y Fl(9)-20911 b Fj(i)31256 8865 y Fi(4)32153 -8754 y Fp(=)463 b(i)o(n)o(t)34477 8571 y(_)34942 8754 -y(a)o(d)o(d)o(\()p Fj(i)37116 8865 y Fi(2)37547 8754 -y Fp(,)i Fj(i)38791 8865 y Fi(3)39222 8754 y Fp(\))51577 -9684 y Fl(10)-21188 b Fj(p)31399 9795 y Fi(5)32296 9684 -y Fp(=)463 b(n)o(e)o(w)o(\()o(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o -(e)o(r)n(\))51577 10614 y Fl(11)p 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb -19329 w Fk(#)p Black 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p -Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -463 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p -Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black 0.3 0.3 0.3 -TeXcolorrgb 43030 10431 a(_)o(_)43960 10614 y(i)o(n)o(i)o(t)45820 -10431 y(_)o(_)p Black 51577 11543 a Fl(12)-19329 b Fp(s)o(e)o(t)o(\()p -Fj(p)35119 11654 y Fi(5)35550 11543 y Fp(,)464 b(i)o(n)o(t)o(v)o(a)o(l) -o(,)g Fj(i)40513 11654 y Fi(4)40945 11543 y Fp(\))51577 -12473 y Fl(13)51577 13403 y(14)p 0.3 0.3 0.3 TeXcolorrgb --24908 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 -TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 463 w(f)p Black -0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(\()n(-)o(1)o(0)o(0)o -(\))p Black 51577 14333 a Fl(15)-24907 b Fj(p)27680 14444 -y Fi(6)28576 14333 y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o(o)o(x)o(e)o(d)o(I) -o(n)o(t)o(e)o(g)o(e)o(r)o(\))51577 15263 y Fl(16)p 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -23049 w Fk(#)p -Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black -0.3 0.3 0.3 TeXcolorrgb 39311 15080 a(_)o(_)40241 15263 -y(i)o(n)o(i)o(t)42101 15080 y(_)o(_)p Black 51577 16193 -a Fl(17)-23049 b Fp(s)o(e)o(t)o(\()p Fj(p)31399 16304 -y Fi(6)31830 16193 y Fp(,)464 b(i)o(n)o(t)o(v)o(a)o(l)o(,)f(-)o(1)o(0)o -(0)o(\))51577 17123 y Fl(18)51577 18052 y(19)p 0.3 0.3 0.3 -TeXcolorrgb -24908 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p +TeXcolorrgb -14 x(__)3297 796 y(init)3437 782 y(__)p +Black 3868 866 a Fl(12)-1450 b Fp(set\()p Fj(p)2634 874 +y Fi(5)2666 866 y Fp(,)35 b(intval)o(,)g Fj(i)3038 874 +y Fi(4)3071 866 y Fp(\))3868 935 y Fl(13)3868 1005 y(14)p +0.3 0.3 0.3 TeXcolorrgb -1868 w Fk(#)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 35 w(inside)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -463 w(f)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black +34 w(f)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -464 w(.)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)p -Black 0.3 0.3 0.3 TeXcolorrgb -1 w(\()p Black 0.3 0.3 0.3 -TeXcolorrgb -1 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p -Black 0.3 0.3 0.3 TeXcolorrgb -2 w(\()n(-)o(1)o(0)o(0)o(\))o(\))p -Black 51577 18982 a Fl(20)-24908 b Fp(g)o(u)o(a)o(r)o(d)29548 -18799 y(_)30013 18982 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 -19093 y Fi(5)33690 18982 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o -(e)o(g)o(e)o(r)n(\))51577 19912 y Fl(21)p 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -23049 w Fk(#)p -Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black -0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)p Black 51577 -20842 a Fl(22)-23048 b Fj(i)29396 20953 y Fi(7)30293 -20842 y Fp(=)463 b(g)o(e)o(t)o(\()p Fj(p)33539 20953 -y Fi(5)33970 20842 y Fp(,)h(i)o(n)o(t)o(v)o(a)o(l)o(\))51577 -21772 y Fl(23)-23049 b Fp(g)o(u)o(a)o(r)o(d)31407 21589 -y(_)31872 21772 y(c)o(l)o(a)o(s)o(s)o(\()p Fj(p)35119 -21883 y Fi(6)35550 21772 y Fp(,)463 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o -(e)o(g)o(e)o(r)o(\))51577 22702 y Fl(24)p 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -21189 w Fk(#)p -Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black -0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)42566 22519 y(_)o(_)43496 -22702 y(i)o(n)o(t)p Black 51577 23631 a Fl(25)-21188 -b Fj(i)31256 23742 y Fi(8)32153 23631 y Fp(=)463 b(g)o(e)o(t)o(\()p -Fj(p)35399 23742 y Fi(6)35830 23631 y Fp(,)h(i)o(n)o(t)o(v)o(a)o(l)o -(\))51577 24561 y Fl(26)-21188 b Fj(i)31256 24672 y Fi(9)32153 -24561 y Fp(=)463 b(i)o(n)o(t)34477 24378 y(_)34942 24561 -y(a)o(d)o(d)o(\()p Fj(i)37116 24672 y Fi(7)37547 24561 -y Fp(,)i Fj(i)38791 24672 y Fi(8)39222 24561 y Fp(\))51577 -25491 y Fl(27)-21188 b Fj(p)31399 25602 y Fi(10)32672 -25491 y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)o(\))51577 26421 y Fl(28)p 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb -19329 w Fk(#)p Black 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p -Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -463 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p +35 w(BoxedInteger)p Black 0.3 0.3 0.3 TeXcolorrgb -1 +w(\(-100\))p Black 3868 1075 a Fl(15)-1868 b Fj(p)2076 +1083 y Fi(6)2143 1075 y Fp(=)35 b(new\()o(BoxedInteger)o(\))3868 +1145 y Fl(16)p 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb -1729 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 35 w(inside)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 34 w(BoxedInteger)p +Black 0.3 0.3 0.3 TeXcolorrgb -1 w(.)p Black 0.3 0.3 0.3 +TeXcolorrgb 2948 1131 a(__)3018 1145 y(init)3158 1131 +y(__)p Black 3868 1214 a Fl(17)-1729 b Fp(set\()p Fj(p)2355 +1222 y Fi(6)2387 1214 y Fp(,)35 b(intval)o(,)g(-100\))3868 +1284 y Fl(18)3868 1354 y(19)p 0.3 0.3 0.3 TeXcolorrgb +-1868 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 35 w(inside)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 34 w(f)p Black 0.3 0.3 0.3 +TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 TeXcolorrgb Black +0.3 0.3 0.3 TeXcolorrgb 35 w(.)p Black 0.3 0.3 0.3 TeXcolorrgb(add)p +Black 0.3 0.3 0.3 TeXcolorrgb(\()p Black 0.3 0.3 0.3 +TeXcolorrgb(BoxedInteger)p Black 0.3 0.3 0.3 TeXcolorrgb +-2 w(\(-100\)\))p Black 3868 1424 a Fl(20)-1868 b Fp(guard)2217 +1410 y(_)2252 1424 y(class)n(\()p Fj(p)2494 1432 y Fi(5)2527 +1424 y Fp(,)34 b(BoxedInteger)o(\))3868 1493 y Fl(21)p +0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +-1729 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 35 w(inside)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 34 w(BoxedInteger)p Black +0.3 0.3 0.3 TeXcolorrgb -1 w(.)p Black 0.3 0.3 0.3 TeXcolorrgb +-1 w(add)p Black 3868 1563 a Fl(22)-1729 b Fj(i)2204 +1571 y Fi(7)2272 1563 y Fp(=)35 b(get)o(\()p Fj(p)2515 +1571 y Fi(5)2548 1563 y Fp(,)g(intval)o(\))3868 1633 +y Fl(23)-1729 b Fp(guard)2356 1619 y(_)2391 1633 y(class)o(\()p +Fj(p)2634 1641 y Fi(6)2666 1633 y Fp(,)35 b(BoxedInteger)n(\))3868 +1703 y Fl(24)p 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb -1589 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 34 w(inside)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 35 w(BoxedInteger)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black 0.3 0.3 0.3 -TeXcolorrgb 43030 26238 a(_)o(_)43960 26421 y(i)o(n)o(i)o(t)45820 -26238 y(_)o(_)p Black 51577 27351 a Fl(29)-19329 b Fp(s)o(e)o(t)o(\()p -Fj(p)35119 27462 y Fi(10)35927 27351 y Fp(,)463 b(i)o(n)o(t)o(v)o(a)o -(l)o(,)i Fj(i)40890 27462 y Fi(9)41321 27351 y Fp(\))51577 -28281 y Fl(30)51577 29211 y(31)p 0.3 0.3 0.3 TeXcolorrgb --24908 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 -TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 463 w(f)p Black -0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(\()n(-)o(1)o(\))p -Black 51577 30140 a Fl(32)-24907 b Fj(p)27680 30251 y -Fi(11)28953 30140 y Fp(=)463 b(n)o(e)o(w\()n(B)o(o)o(x)o(e)o(d)o(I)o(n) -o(t)o(e)o(g)o(e)o(r)o(\))51577 31070 y Fl(33)p 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -23049 w Fk(#)p -Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black -0.3 0.3 0.3 TeXcolorrgb 39311 30887 a(_)o(_)40241 31070 -y(i)o(n)o(i)o(t)42101 30887 y(_)o(_)p Black 51577 32000 -a Fl(34)-23049 b Fp(s)o(e)o(t)o(\()p Fj(p)31399 32111 -y Fi(11)32207 32000 y Fp(,)464 b(i)o(n)o(t)o(v)o(a)o(l)o(,)f(-)o(1)o -(\))51577 32930 y Fl(35)51577 33860 y(36)p 0.3 0.3 0.3 -TeXcolorrgb -24908 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p -Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -463 w(f)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black +TeXcolorrgb(add)3193 1689 y(__)3263 1703 y(int)p Black +3868 1772 a Fl(25)-1589 b Fj(i)2344 1780 y Fi(8)2411 +1772 y Fp(=)35 b(get\()p Fj(p)2655 1780 y Fi(6)2687 1772 +y Fp(,)g(intval)o(\))3868 1842 y Fl(26)-1589 b Fj(i)2344 +1850 y Fi(9)2411 1842 y Fp(=)35 b(int)2586 1828 y(_)2621 +1842 y(add)o(\()p Fj(i)2783 1850 y Fi(7)2816 1842 y Fp(,)g +Fj(i)2909 1850 y Fi(8)2942 1842 y Fp(\))3868 1912 y Fl(27)-1589 +b Fj(p)2355 1920 y Fi(10)2450 1912 y Fp(=)35 b(new\(BoxedInteger)n(\)) +3868 1982 y Fl(28)p 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb -1450 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 35 w(inside)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 34 w(BoxedInteger)p +Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black 0.3 0.3 0.3 +TeXcolorrgb -14 x(__)3297 1982 y(init)3437 1968 y(__)p +Black 3868 2051 a Fl(29)-1450 b Fp(set\()p Fj(p)2634 +2059 y Fi(10)2694 2051 y Fp(,)35 b(intval)o(,)g Fj(i)3066 +2059 y Fi(9)3099 2051 y Fp(\))3868 2121 y Fl(30)3868 +2191 y(31)p 0.3 0.3 0.3 TeXcolorrgb -1868 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -464 w(y)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(.)p Black -0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)p Black 0.3 0.3 0.3 -TeXcolorrgb -1 w(\()p Black 0.3 0.3 0.3 TeXcolorrgb -2 -w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p Black -0.3 0.3 0.3 TeXcolorrgb -1 w(\()n(-)o(1)o(\))o(\))p Black -51577 34790 a Fl(37)-24908 b Fp(g)o(u)o(a)o(r)o(d)29548 -34607 y(_)30013 34790 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 -34901 y Fi(0)33690 34790 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o -(e)o(g)o(e)o(r)n(\))51577 35719 y Fl(38)p 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -23049 w Fk(#)p +35 w(inside)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 34 w(f)p Black 0.3 0.3 0.3 TeXcolorrgb -1 +w(:)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 35 w(BoxedInteger)p Black 0.3 0.3 0.3 TeXcolorrgb +-1 w(\(-1\))p Black 3868 2261 a Fl(32)-1868 b Fj(p)2076 +2269 y Fi(11)2171 2261 y Fp(=)35 b(new\(BoxedInteger)n(\))3868 +2330 y Fl(33)p 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb -1729 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 35 w(inside)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 34 w(BoxedInteger)p +Black 0.3 0.3 0.3 TeXcolorrgb -1 w(.)p Black 0.3 0.3 0.3 +TeXcolorrgb 2948 2316 a(__)3018 2330 y(init)3158 2316 +y(__)p Black 3868 2400 a Fl(34)-1729 b Fp(set\()p Fj(p)2355 +2408 y Fi(11)2416 2400 y Fp(,)34 b(intval,)g(-1\))3868 +2470 y Fl(35)3868 2539 y(36)p 0.3 0.3 0.3 TeXcolorrgb +-1868 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 35 w(inside)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 34 w(f)p Black 0.3 0.3 0.3 +TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 TeXcolorrgb Black +0.3 0.3 0.3 TeXcolorrgb 35 w(y)p Black 0.3 0.3 0.3 TeXcolorrgb(.)p +Black 0.3 0.3 0.3 TeXcolorrgb(add)p Black 0.3 0.3 0.3 +TeXcolorrgb(\()p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(BoxedInteger)p +Black 0.3 0.3 0.3 TeXcolorrgb -1 w(\(-1\)\))p Black 3868 +2609 a Fl(37)-1868 b Fp(guard)2217 2595 y(_)2252 2609 +y(class)n(\()p Fj(p)2494 2617 y Fi(0)2527 2609 y Fp(,)34 +b(BoxedInteger)o(\))3868 2679 y Fl(38)p 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb -1729 w Fk(#)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 35 w(inside)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black -0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)p Black 51577 -36649 a Fl(39)-23048 b Fj(i)29396 36760 y Fi(12)30670 -36649 y Fp(=)463 b(g)o(e)o(t)o(\()p Fj(p)33916 36760 -y Fi(0)34347 36649 y Fp(,)h(i)o(n)o(t)o(v)o(a)o(l)o(\))51577 -37579 y Fl(40)-23049 b Fp(g)o(u)o(a)o(r)o(d)31407 37396 -y(_)31872 37579 y(c)o(l)o(a)o(s)o(s)o(\()p Fj(p)35119 -37690 y Fi(11)35927 37579 y Fp(,)463 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o -(e)o(g)o(e)o(r)o(\))51577 38509 y Fl(41)p 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -21189 w Fk(#)p -Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black -0.3 0.3 0.3 TeXcolorrgb -1 w(a)o(d)o(d)42566 38326 y(_)o(_)43496 -38509 y(i)o(n)o(t)p Black 51577 39439 a Fl(42)-21188 -b Fj(i)31256 39550 y Fi(13)32529 39439 y Fp(=)464 b(g)o(e)o(t)o(\()p -Fj(p)35776 39550 y Fi(11)36584 39439 y Fp(,)f(i)o(n)o(t)o(v)o(a)o(l)o -(\))51577 40369 y Fl(43)-21188 b Fj(i)31256 40480 y Fi(14)32529 -40369 y Fp(=)463 b(i)o(n)o(t)34853 40186 y(_)35318 40369 -y(a)o(d)o(d\()o Fj(i)37492 40480 y Fi(12)38300 40369 -y Fp(,)i Fj(i)39544 40480 y Fi(13)40352 40369 y Fp(\))51577 -41299 y Fl(44)-21188 b Fj(p)31399 41410 y Fi(15)32672 -41299 y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)o(\))51577 42228 y Fl(45)p 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb -19329 w Fk(#)p Black 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p -Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -463 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)p +34 w(BoxedInteger)p Black 0.3 0.3 0.3 TeXcolorrgb -1 +w(.)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(add)p Black +3868 2749 a Fl(39)-1729 b Fj(i)2204 2757 y Fi(12)2300 +2749 y Fp(=)35 b(get\()o Fj(p)2543 2757 y Fi(0)2576 2749 +y Fp(,)g(intval)o(\))3868 2818 y Fl(40)-1729 b Fp(guard)2356 +2804 y(_)2391 2818 y(class)o(\()p Fj(p)2634 2826 y Fi(11)2694 +2818 y Fp(,)35 b(BoxedInteger)o(\))3868 2888 y Fl(41)p +0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +-1589 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 34 w(inside)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 35 w(BoxedInteger)p Black +0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black 0.3 0.3 0.3 TeXcolorrgb(add)3193 +2874 y(__)3263 2888 y(int)p Black 3868 2958 a Fl(42)-1589 +b Fj(i)2344 2966 y Fi(13)2440 2958 y Fp(=)34 b(get\()p +Fj(p)2683 2966 y Fi(11)2744 2958 y Fp(,)g(intval\))3868 +3028 y Fl(43)-1589 b Fj(i)2344 3036 y Fi(14)2440 3028 +y Fp(=)34 b(int)2614 3014 y(_)2649 3028 y(add\()o Fj(i)2811 +3036 y Fi(12)2873 3028 y Fp(,)g Fj(i)2965 3036 y Fi(13)3026 +3028 y Fp(\))3868 3097 y Fl(44)-1589 b Fj(p)2355 3105 +y Fi(15)2450 3097 y Fp(=)35 b(new\(BoxedInteger)n(\))3868 +3167 y Fl(45)p 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb -1450 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 35 w(inside)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 34 w(BoxedInteger)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black 0.3 0.3 0.3 -TeXcolorrgb 43030 42045 a(_)o(_)43960 42228 y(i)o(n)o(i)o(t)45820 -42045 y(_)o(_)p Black 51577 43158 a Fl(46)-19329 b Fp(s)o(e)o(t)o(\()p -Fj(p)35119 43269 y Fi(15)35927 43158 y Fp(,)463 b(i)o(n)o(t)o(v)o(a)o -(l)o(,)i Fj(i)40890 43269 y Fi(14)41698 43158 y Fp(\))51577 -44088 y Fl(47)51577 45018 y(48)p 0.3 0.3 0.3 TeXcolorrgb --24908 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 -TeXcolorrgb 464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 463 w(f)p Black -0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 464 w(y)p Black 0.3 0.3 0.3 -TeXcolorrgb -1 w(.)p Black 0.3 0.3 0.3 TeXcolorrgb -1 -w(i)o(s)34662 44835 y(_)35127 45018 y(p)o(o)o(s)o(i)o(t)o(i)o(v)o(e)p -Black 0.3 0.3 0.3 TeXcolorrgb -2 w(\()o(\))p Black 51577 -45948 a Fl(49)-24908 b Fp(g)o(u)o(a)o(r)o(d)29548 45765 -y(_)30013 45948 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 -46059 y Fi(15)34067 45948 y Fp(,)463 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o -(e)o(g)o(e)o(r)o(\))51577 46878 y Fl(50)p 0.3 0.3 0.3 -TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -23049 w Fk(#)p +TeXcolorrgb -14 x(__)3297 3167 y(init)3437 3153 y(__)p +Black 3868 3237 a Fl(46)-1450 b Fp(set\()p Fj(p)2634 +3245 y Fi(15)2694 3237 y Fp(,)35 b(intval)o(,)g Fj(i)3066 +3245 y Fi(14)3127 3237 y Fp(\))3868 3307 y Fl(47)3868 +3376 y(48)p 0.3 0.3 0.3 TeXcolorrgb -1868 w Fk(#)p Black +0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb +35 w(inside)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 34 w(f)p Black 0.3 0.3 0.3 TeXcolorrgb -1 +w(:)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 35 w(y)p Black 0.3 0.3 0.3 TeXcolorrgb(.)p +Black 0.3 0.3 0.3 TeXcolorrgb(is)2600 3362 y(_)2635 3376 +y(positive)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(\(\))p +Black 3868 3446 a Fl(49)-1868 b Fp(guard)2217 3432 y(_)2252 +3446 y(class)n(\()p Fj(p)2494 3454 y Fi(15)2555 3446 +y Fp(,)35 b(BoxedInteger)n(\))3868 3516 y Fl(50)p 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -1729 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 464 w(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)p Black 0.3 0.3 0.3 TeXcolorrgb -2 w(.)p Black -0.3 0.3 0.3 TeXcolorrgb -1 w(i)o(s)40241 46695 y(_)40706 -46878 y(p)o(o)o(s)o(i)o(t)o(i)o(v)o(e)p Black 51577 47808 -a Fl(51)-23048 b Fj(i)29396 47919 y Fi(16)30670 47808 -y Fp(=)463 b(g)o(e)o(t)o(\()p Fj(p)33916 47919 y Fi(15)34724 -47808 y Fp(,)g(i)o(n)o(t)o(v)o(a)o(l\))51577 48737 y -Fl(52)-23048 b Fj(i)29396 48848 y Fi(17)30670 48737 y -Fp(=)463 b(i)o(n)o(t)32994 48554 y(_)33459 48737 y(g)o(t)o(\()p -Fj(i)35168 48848 y Fi(16)35976 48737 y Fp(,)g(0)o(\))51577 -49667 y Fl(53)p 0.3 0.3 0.3 TeXcolorrgb -24908 w Fk(#)p +35 w(inside)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 34 w(BoxedInteger)p Black 0.3 0.3 0.3 TeXcolorrgb +-1 w(.)p Black 0.3 0.3 0.3 TeXcolorrgb -1 w(is)3018 3502 +y(_)3053 3516 y(positive)p Black 3868 3586 a Fl(51)-1729 +b Fj(i)2204 3594 y Fi(16)2300 3586 y Fp(=)35 b(get\()o +Fj(p)2543 3594 y Fi(15)2604 3586 y Fp(,)g(intval)o(\))3868 +3655 y Fl(52)-1729 b Fj(i)2204 3663 y Fi(17)2300 3655 +y Fp(=)35 b(int)2475 3641 y(_)2510 3655 y(gt)o(\()p Fj(i)2637 +3663 y Fi(16)2698 3655 y Fp(,)g(0\))3868 3725 y Fl(53)p +0.3 0.3 0.3 TeXcolorrgb -1868 w Fk(#)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 35 w(inside)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -464 w(i)o(n)o(s)o(i)o(d)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 463 w(f)p Black 51577 50597 -a Fl(54)-24908 b Fp(g)o(u)o(a)o(r)o(d)29548 50414 y(_)30013 -50597 y(t)o(r)o(u)o(e)n(\()p Fj(i)32651 50708 y Fi(17)33459 -50597 y Fp(\))51577 51527 y Fl(55)g Fp(j)o(u)o(m)o(p)o(\()o -Fj(p)30004 51638 y Fi(15)30812 51527 y Fp(,)465 b Fj(p)32199 -51638 y Fi(10)33008 51527 y Fp(\))p Black Black 27224 -52507 V 28388 53636 a Fs(Figur)-18 b(e)249 b(2.)499 b -FC(An)249 b(Unoptimized)i(T)-35 b(race)249 b(of)g(the)h(Example)g -(Interpreter)p Black 28552 56913 a(Method)475 b(calls)g(in)f(the)h -(trace)g(are)g(preceded)h(by)f(a)g Fr(guard)50592 56704 -y(_)51125 56913 y(class)p Black Black 27224 58020 a FC(operation,)279 -b(to)f(check)i(that)e(the)h(class)f(of)g(the)g(recei)-25 -b(v)-15 b(er)280 b(is)d(the)i(same)f(as)g(the)27224 59165 -y(one)218 b(that)f(w)-10 b(as)218 b(observ)-15 b(ed)218 -b(during)f(tracing.)41980 58741 y Fz(5)42586 59165 y -FC(These)g(guards)g(mak)-10 b(e)218 b(the)g(trace)27224 -60272 y(speci\002c)273 b(to)h(the)f(situation)g(where)h -Fr(y)p Black Black 274 w FC(is)e(really)i(a)f Fr(BoxedInteger)p -Black Black 2 w FC(.)g(When)27224 61379 y(the)473 b(trace)g(is)f -(turned)h(into)g(machine)h(code)g(and)f(afterw)-10 b(ards)473 -b(e)-15 b(x)g(ecuted)27224 62485 y(with)252 b Fr(BoxedFloat)p -Black Black 1 w FC(,)g(the)g(\002rst)f Fr(guard)41015 -62276 y(_)41548 62485 y(class)p Black Black 253 w FC(instruction)h -(will)g(f)-10 b(ail)252 b(and)27224 63592 y(e)-15 b(x)g(ecution)250 -b(will)g(continue)g(using)f(the)g(interpreter)-55 b(.)28552 -64699 y(The)328 b(trace)g(sho)-25 b(ws)327 b(the)g(inef)-25 -b(\002ciencies)329 b(of)e Fr(f)p Black Black 327 w FC(clearly)-65 -b(,)328 b(if)f(one)h(looks)f(at)27224 65806 y(the)k(number)g(of)f -Fr(new)p Black Black 1 w FC(,)g Fr(set)q(/get)p Black -Black 331 w FC(and)h Fr(guard)43982 65597 y(_)44515 65806 -y(class)p Black Black 332 w FC(operations.)g(The)27224 -66913 y(number)219 b(of)f Fr(guard)34199 66704 y(_)34732 -66913 y(class)p Black Black 219 w FC(operation)h(is)f(particularly)h -(problematic,)h(not)27224 68020 y(only)270 b(because)h(of)f(the)g(time) -g(it)g(tak)-10 b(es)270 b(to)g(run)g(them.)g(All)g(guards)g(also)g(ha) --20 b(v)-15 b(e)27224 69127 y(additional)310 b(information)f(attached)h -(that)f(mak)-10 b(es)310 b(it)e(possible)h(to)g(return)g(to)p -Black 27224 70403 13284 37 v 27224 71086 a Fz(5)27722 -71399 y Fr(guard)30387 71190 y(_)30920 71399 y(class)p -Black Black 358 w Fw(performs)357 b(a)g(precise)h(class)g(check,)g(not) -e(checking)h(for)g(sub-)27224 72395 y(classes.)p Black -Black Black eop end +34 w(f)p Black 3868 3795 a Fl(54)-1868 b Fp(guard)2217 +3781 y(_)2252 3795 y(true)n(\()p Fj(i)2448 3803 y Fi(17)2509 +3795 y Fp(\))3868 3865 y Fl(55)g Fp(jump)o(\()p Fj(p)2250 +3873 y Fi(15)2311 3865 y Fp(,)35 b Fj(p)2415 3873 y Fi(10)2476 +3865 y Fp(\))p Black Black 2042 3938 V 2129 4023 a Fs(Figur)o(e)18 +b(2.)38 b FC(An)18 b(Unoptimized)i(T)m(race)f(of)g(the)g(Example)h +(Interpreter)p Black 2141 4268 a(Method)37 b(calls)f(in)g(the)g(trace)f +(are)h(preceded)h(by)g(a)e Fr(guard)3794 4253 y(_)3834 +4268 y(class)p Black Black 2042 4352 a FC(operation,)22 +b(to)f(check)h(that)f(the)g(class)g(of)g(the)g(recei)n(v)o(er)h(is)e +(the)h(same)h(as)f(the)2042 4437 y(one)c(that)f(w)o(as)h(observ)o(ed)h +(during)f(tracing.)3148 4406 y Fz(5)3194 4437 y FC(These)g(guards)g +(mak)o(e)h(the)e(trace)2042 4520 y(speci\002c)21 b(to)f(the)h +(situation)g(where)g Fr(y)p Black Black 20 w FC(is)g(really)f(a)h +Fr(BoxedInteger)p Black Black FC(.)f(When)2042 4603 y(the)36 +b(trace)f(is)g(turned)i(into)f(machine)g(code)h(and)f(afterw)o(ards)g +(e)o(x)o(ecuted)2042 4686 y(with)19 b Fr(BoxedFloat)p +Black Black -1 w FC(,)g(the)g(\002rst)f Fr(guard)3076 +4671 y(_)3116 4686 y(class)p Black Black 19 w FC(instruction)i(will)e +(f)o(ail)g(and)2042 4769 y(e)o(x)o(ecution)i(will)e(continue)i(using)f +(the)g(interpreter)l(.)2141 4852 y(The)25 b(trace)g(sho)n(ws)g(the)g +(inef)n(\002ciencies)g(of)g Fr(f)p Black Black 25 w FC(clearly)-5 +b(,)24 b(if)g(one)i(looks)f(at)2042 4935 y(the)g(number)h(of)f +Fr(new)p Black Black FC(,)f Fr(set/get)p Black Black +25 w FC(and)i Fr(guard)3299 4920 y(_)3339 4935 y(class)p +Black Black 24 w FC(operations.)g(The)2042 5018 y(number)17 +b(of)g Fr(guard)2565 5003 y(_)2605 5018 y(class)p Black +Black 16 w FC(operation)h(is)e(particularly)h(problematic,)g(not)2042 +5102 y(only)k(because)h(of)e(the)h(time)f(it)f(tak)o(es)j(to)e(run)h +(them.)f(All)g(guards)h(also)g(ha)o(v)o(e)2042 5185 y(additional)j +(information)g(attached)g(that)f(mak)o(es)h(it)f(possible)h(to)g +(return)f(to)p Black 2042 5280 997 3 v 2042 5331 a Fz(5)2079 +5355 y Fr(guard)2279 5339 y(_)2319 5355 y(class)p Black +Black 27 w Fw(performs)k(a)h(precise)g(class)g(check,)h(not)e(checking) +j(for)d(sub-)2042 5430 y(classes.)p Black Black Black +eop end %%Page: 4 4 -TeXDict begin 4 3 bop Black Black Black Black Black 4217 -32334 a @beginspecial 0 @llx -1 @lly 212 @urx 484 @ury +TeXDict begin 4 3 bop Black Black Black Black Black 316 +2425 a @beginspecial 0 @llx -1 @lly 212 @urx 484 @ury 1272 @rwi @setspecial %%BeginDocument: figures/obj-lifetime.eps %!PS-Adobe-3.0 EPSF-3.0 @@ -7972,1595 +7801,1365 @@ %%EOF %%EndDocument - @endspecial -2000 33588 26568 37 v 3674 34717 a Fs(Figur)-18 -b(e)249 b(3.)499 b FC(Object)250 b(Lifetimes)e(in)h(a)h(T)-35 -b(race)p Black -2000 37832 a(the)400 b(interpreter)-40 -b(,)401 b(should)e(the)i(guard)f(f)-10 b(ail.)400 b(This)f(means)h -(that)g(too)g(man)-15 b(y)-2000 38939 y(guard)250 b(operations)f(also)g -(consume)h(a)f(lot)g(of)g(memory)-65 b(.)-672 40046 y(In)471 -b(the)g(rest)f(of)h(the)g(paper)h(we)f(will)g(see)g(ho)-25 -b(w)471 b(this)g(trace)g(can)h(be)-2000 41153 y(optimized)250 -b(using)f(partial)g(e)-25 b(v)g(aluation.)-2000 43714 -y FA(3.)1218 b(Object)305 b(Lifetimes)h(in)f(a)f(T)-90 -b(racing)304 b(JIT)-2000 45264 y FC(T)-80 b(o)197 b(understand)h(the)g -(problems)f(that)g(this)g(paper)h(is)f(trying)g(to)g(solv)-15 -b(e)197 b(in)h(more)-2000 46371 y(detail,)261 b(we)g(\002rst)e(need)j -(to)e(understand)h(v)-25 b(arious)260 b(cases)h(of)f(object)h -(lifetimes)-2000 47478 y(that)249 b(can)h(occur)g(in)f(a)h(tracing)f -(JIT)f(compiler)-55 b(.)-672 48585 y(Figure)357 b(3)g(sho)-25 -b(ws)357 b(a)g(trace)h(before)f(optimization,)h(together)f(with)h(the) --2000 49692 y(lifetime)326 b(of)f(v)-25 b(arious)326 -b(kinds)g(of)f(objects)h(created)h(in)f(the)g(trace.)g(It)f(is)g(e)-15 -b(x)g(e-)-2000 50799 y(cuted)254 b(from)g(top)g(to)f(bottom.)h(At)g -(the)g(bottom,)g(a)g(jump)g(is)f(used)h(to)f(e)-15 b(x)g(ecute)-2000 -51906 y(the)269 b(same)g(loop)g(another)g(time)g(\(for)f(clarity)-65 -b(,)270 b(the)f(\002gure)g(sho)-25 b(ws)268 b(tw)-10 -b(o)269 b(itera-)-2000 53013 y(tions)198 b(of)g(the)g(loop\).)g(The)h -(loop)f(is)f(e)-15 b(x)g(ecuted)200 b(until)e(one)h(of)f(the)g(guards)h -(in)f(the)-2000 54120 y(trace)246 b(f)-10 b(ails,)244 -b(and)i(the)f(e)-15 b(x)g(ecution)247 b(is)d(aborted)i(and)g -(interpretation)f(resumes.)-672 55227 y(Some)405 b(of)e(the)i -(operations)f(within)g(this)g(trace)g(are)g Fr(new)p -Black Black 405 w FC(operations,)-2000 56334 y(which)345 -b(each)f(create)h(a)f(ne)-25 b(w)345 b(instance)f(of)g(some)g(class.)f -(These)h(instances)-2000 57441 y(are)338 b(used)f(for)h(some)f(time,)h -(e.g.,)g(by)g(calling)g(methods)g(on)f(them)h(\(which)-2000 -58548 y(are)242 b(inlined)h(into)f(the)g(trace\),)h(reading)f(and)h -(writing)f(their)g(\002elds.)g(Some)g(of)-2000 59655 -y(these)321 b(instances)f Fx(escape)p FC(,)h(which)g(means)g(that)g -(the)-15 b(y)321 b(are)f(stored)h(in)f(some)-2000 60762 -y(globally)284 b(accessible)g(place)h(or)e(are)h(passed)g(into)f(a)h -(non-inlined)h(function)-2000 61869 y(via)249 b(a)h(residual)f(call.) --672 62976 y(T)-80 b(ogether)226 b(with)f(the)g Fr(new)p -Black Black 225 w FC(operations,)g(the)h(\002gure)f(sho)-25 -b(ws)224 b(the)h(lifetimes)-2000 64082 y(of)335 b(the)h(created)h -(objects.)e(The)h(objects)g(that)f(are)h(created)h(within)f(a)f(trace) --2000 65189 y(using)249 b Fr(new)p Black Black 249 w -FC(f)-10 b(all)249 b(into)h(one)f(of)g(se)-25 b(v)-15 -b(eral)250 b(cate)-15 b(gories:)p Black -1696 67082 a(1.)p -Black 388 w(Objects)294 b(that)f(li)-25 b(v)-15 b(e)294 -b(for)f(some)g(time,)g(and)h(are)f(then)h(just)f(not)g(used)g(an)-15 -b(y)-561 68189 y(more)249 b(afterw)-10 b(ards.)p Black --1696 69738 a(2.)p Black 388 w(Objects)250 b(that)f(li)-25 -b(v)-15 b(e)250 b(for)e(some)i(time)f(and)h(then)f(escape.)p -Black -1696 71288 a(3.)p Black 388 w(Objects)292 b(that)g(li)-25 -b(v)-15 b(e)293 b(for)e(some)h(time,)g(survi)-25 b(v)-15 -b(e)292 b(across)g(the)g(jump)g(to)g(the)-561 72395 y(be)-15 -b(ginning)250 b(of)f(the)h(loop,)f(and)h(are)f(then)h(not)f(used)g(an) --15 b(y)250 b(more.)p Black Black Black 27528 886 a(4.)p -Black 388 w(Objects)352 b(that)g(li)-25 b(v)-15 b(e)352 -b(for)f(some)g(time,)h(survi)-25 b(v)-15 b(e)351 b(across)h(the)f -(jump,)h(and)28663 1993 y(then)271 b(escape.)g(T)-80 -b(o)271 b(these)g(we)g(also)f(count)i(the)e(objects)h(that)g(li)-25 -b(v)-15 b(e)271 b(across)28663 3099 y(se)-25 b(v)-15 -b(eral)249 b(jumps)g(and)h(then)g(either)f(escape)h(or)f(stop)g(being)h -(used.)28552 4945 y(The)309 b(objects)g(that)g(are)h(allocated)g(in)f -(the)g(e)-15 b(xample)310 b(trace)g(in)f(Figure)g(2)27224 -6052 y(f)-10 b(all)378 b(into)g(cate)-15 b(gories)379 -b(1)g(and)f(3.)h(Objects)g(stored)f(in)g Fo(p)46971 6163 -y Fn(5)47432 6052 y FC(,)g Fo(p)48573 6163 y Fn(6)49035 -6052 y FC(,)g Fo(p)50176 6163 y Fn(11)51421 6052 y FC(are)h(in)27224 -7159 y(cate)-15 b(gory)250 b(1,)f(objects)h(in)f Fo(p)36494 -7270 y Fn(10)37361 7159 y FC(,)g Fo(p)38373 7270 y Fn(15)39489 -7159 y FC(are)h(in)f(cate)-15 b(gory)250 b(3.)28552 8266 -y(The)357 b(creation)h(of)e(objects)h(in)g(cate)-15 b(gory)358 -b(1)f(is)f(remo)-15 b(v)g(ed)358 b(by)f(the)g(opti-)27224 -9373 y(mization)310 b(described)h(in)f(Sections)g(4)g(and)g(5.)g -(Objects)h(in)f(the)g(other)g(cate-)27224 10518 y(gories)249 -b(are)g(partially)h(optimized)g(by)f(this)g(approach)h(as)f(well.)49054 -10095 y Fz(6)27224 13016 y FA(4.)1218 b(Allocation)305 -b(Remo)-12 b(v)g(al)304 b(in)h(T)-90 b(races)27224 14566 -y FC(The)284 b(main)g(insight)f(to)h(impro)-15 b(v)g(e)284 -b(the)g(code)h(sho)-25 b(wn)284 b(in)g(Section)g(2.3)g(is)f(that)27224 -15673 y(objects)258 b(in)g(cate)-15 b(gory)259 b(1)g(do)f(not)g(survi) --25 b(v)-15 b(e)258 b(v)-15 b(ery)259 b(long)f(\226)g(the)-15 -b(y)259 b(are)f(used)g(only)27224 16780 y(inside)405 -b(the)h(loop)g(and)g(there)g(is)f(no)h(other)f(outside)h(reference)h -(to)e(them.)27224 17887 y(The)448 b(optimizer)h(identi\002es)f(objects) -h(in)f(cate)-15 b(gory)450 b(1)e(and)h(remo)-15 b(v)g(es)448 -b(the)27224 18993 y(allocation)250 b(of)f(these)g(objects,)h(and)f(all) -h(operations)f(manipulating)i(them.)28552 20100 y(This)407 -b(is)f(a)i(process)f(that)g(is)g(usually)g(called)h Fx(escape)g -(analysis)f FC([18].)27224 21207 y(In)460 b(this)f(paper)h(we)h(will)f -(perform)g(escape)h(analysis)e(by)i(using)e(partial)27224 -22314 y(e)-25 b(v)g(aluation.)386 b(The)e(use)g(of)h(partial)f(e)-25 -b(v)g(aluation)386 b(is)e(a)h(bit)f(peculiar)h(in)f(that)27224 -23421 y(it)221 b(recei)-25 b(v)-15 b(es)222 b(no)g(static)f(input)g(ar) --18 b(guments)222 b(for)f(the)h(trace,)g(b)-20 b(ut)221 -b(it)g(is)g(only)h(used)27224 24528 y(to)344 b(optimize)i(operations)f -(within)g(the)g(trace.)g(This)f(section)h(will)g(gi)-25 -b(v)-15 b(e)345 b(an)27224 25635 y(informal)230 b(account)i(of)e(this)g -(process)h(by)f(e)-15 b(xamining)232 b(the)f(e)-15 b(xample)232 -b(trace)f(in)27224 26742 y(Figure)261 b(2.)g(The)g(\002nal)h(trace)f -(after)h(optimization)g(can)f(be)h(seen)f(in)g(Figure)h(4)27224 -27849 y(\(the)287 b(line)h(numbers)g(are)f(the)h(lines)f(of)h(the)f -(unoptimized)i(trace)f(where)g(the)27224 28956 y(operation)250 -b(originates\).)28552 30063 y(T)-80 b(o)266 b(optimize)i(the)e(trace,)h -(it)f(is)g(tra)-20 b(v)-15 b(ersed)267 b(from)f(be)-15 -b(ginning)267 b(to)f(end)h(and)27224 31170 y(an)342 b(output)g(trace)g -(is)f(produced.)h(Ev)-15 b(ery)342 b(operation)g(in)g(the)f(input)h -(trace)g(is)27224 32277 y(either)422 b(remo)-15 b(v)g(ed)422 -b(or)g(copied)h(into)e(the)i(output)f(trace.)g(Sometimes)g(ne)-25 -b(w)27224 33384 y(operations)380 b(need)g(to)g(be)g(produced)h(as)f -(well.)g(The)g(optimizer)g(can)g(only)27224 34491 y(remo)-15 -b(v)g(e)238 b(operations)f(that)h(manipulate)g(objects)g(that)f(ha)-20 -b(v)-15 b(e)238 b(been)g(allocated)27224 35598 y(within)255 -b(the)g(trace,)g(while)g(all)g(other)g(operations)f(are)h(copied)h(to)e -(the)h(output)27224 36705 y(trace)250 b(unchanged.)28552 -37812 y(Looking)221 b(at)g(the)g(e)-15 b(xample)222 b(trace)f(of)g -(Figure)f(2,)h(the)g(operations)g(in)g(lines)27224 38919 -y(1\2269)378 b(are)f(manipulating)i(objects)f(which)g(e)-15 -b(xisted)378 b(before)g(the)g(trace)g(and)27224 40026 -y(that)257 b(are)g(passed)g(in)g(as)g(ar)-18 b(guments:)258 -b(therefore)f(the)g(optimizer)h(just)e(copies)27224 41133 -y(them)249 b(into)h(the)f(output)h(trace.)28552 42240 -y(The)f(follo)-25 b(wing)250 b(operations)f(\(lines)g(10\22617\))h(are) -f(more)g(interesting:)51577 43786 y Fl(10)-22693 b Fj(p)29894 -43897 y Fi(5)30790 43786 y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o(o)o(x)o(e)o -(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)o(\))51577 44716 y Fl(12)-22694 -b Fp(s)o(e)o(t)o(\()o Fj(p)31753 44827 y Fi(5)32185 44716 -y Fp(,)463 b(i)o(n)o(t)o(v)o(a)o(l)o(,)i Fj(i)37148 44827 -y Fi(4)37579 44716 y Fp(\))51577 45645 y Fl(15)-22693 -b Fj(p)29894 45756 y Fi(6)30790 45645 y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o -(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o(r)o(\))51577 -46575 y Fl(17)-22694 b Fp(s)o(e)o(t)o(\()o Fj(p)31753 -46686 y Fi(6)32185 46575 y Fp(,)463 b(i)o(n)o(t)o(v)o(a)o(l)o(,)g(-)o -(1)o(0)o(0)o(\))p Black Black 28552 48299 a FC(When)322 -b(the)f(optimizer)h(encounters)g(a)f Fr(new)p Black Black -FC(,)g(it)g(remo)-15 b(v)g(es)322 b(it)e(optimisti-)27224 -49405 y(cally)-65 b(,)263 b(and)f(assumes)g(that)g(the)g(object)h(is)f -(in)g(cate)-15 b(gory)263 b(1.)f(If)f(later)i(the)f(opti-)27224 -50512 y(mizer)231 b(\002nds)g(that)g(the)g(object)h(escapes,)f(it)g -(will)g(be)h(allocated)g(at)f(that)g(point.)27224 51619 -y(The)370 b(optimizer)h(needs)g(to)f(k)-10 b(eep)371 -b(track)g(of)f(the)h(state)f(of)h(the)f(object)h(that)27224 -52726 y(the)385 b(operation)g(w)-10 b(ould)385 b(ha)-20 -b(v)-15 b(e)385 b(created.)h(This)e(is)f(done)j(with)e(the)h(help)g(of) -27224 53853 y(a)f Fx(static)f(object)33044 53430 y Fz(7)33433 -53853 y FC(.)g(The)h(static)g(object)h(describes)f(the)g(shape)g(of)g -(the)g(ob-)27224 54960 y(ject)272 b(that)g(w)-10 b(ould)273 -b(ha)-20 b(v)-15 b(e)272 b(been)h(allocated,)g(i.e.,)f(the)h(type)f(of) -f(the)i(object)f(and)27224 56067 y(where)f(the)h(v)-25 -b(alues)271 b(that)g(w)-10 b(ould)272 b(be)f(stored)g(in)f(the)i -(\002elds)e(of)h(the)g(allocated)27224 57174 y(object)250 -b(come)g(from.)28552 58281 y(In)315 b(the)h(snippet)f(abo)-15 -b(v)g(e,)317 b(the)e(tw)-10 b(o)316 b Fr(new)p Black -Black 316 w FC(operations)f(are)h(remo)-15 b(v)g(ed)316 -b(and)27224 59388 y(tw)-10 b(o)297 b(static)f(objects)h(are)g -(constructed.)h(The)f Fr(set)p Black Black 296 w FC(operations)g -(manipulate)27224 60495 y(static)251 b(objects,)g(therefore)h(the)-15 -b(y)252 b(can)f(be)h(remo)-15 b(v)g(ed)252 b(as)f(well;)h(their)f(ef) --25 b(fect)251 b(is)27224 61602 y(remembered)f(in)f(the)h(static)f -(objects.)28552 62709 y(After)299 b(the)h(operations)f(the)h(static)f -(object)h(associated)f(with)h Fo(p)50550 62820 y Fn(5)51310 -62709 y FC(w)-10 b(ould)27224 63816 y(store)418 b(the)i(kno)-25 -b(wledge)420 b(that)f(it)g(is)f(a)i Fr(BoxedInteger)p -Black Black 420 w FC(whose)f Fr(intval)p Black Black -27224 64923 a FC(\002eld)255 b(contains)h Fo(i)33176 -65034 y Fn(4)33637 64923 y FC(;)g(the)f(one)h(associated)g(with)f -Fo(p)44226 65034 y Fn(6)44943 64923 y FC(w)-10 b(ould)255 -b(store)g(that)h(it)f(is)f(a)27224 66030 y Fr(BoxedInteger)p -Black Black 250 w FC(whose)c Fr(intval)p Black Black -249 w FC(\002eld)g(contains)f(the)h(constant)f(-100.)p -Black 27224 67115 13284 37 v 27224 67799 a Fz(6)27722 -68111 y Fw(W)-71 b(e)202 b(also)g(started)h(to)f(w)-9 -b(ork)202 b(on)f(optimizing)h(objects)h(in)e(cate)-13 -b(gory)202 b(3,)g(which)g(will)h(be)27224 69108 y(the)221 -b(subject)h(of)g(a)f(later)h(paper)-49 b(.)27224 70090 -y Fz(7)27722 70403 y Fw(Here)338 b(\223static\224)h(is)g(meant)e(in)h -(the)g(sense)g(of)f(partial)i(e)-22 b(v)g(aluation,)337 -b(i.e.,)j(kno)-22 b(wn)336 b(at)27224 71399 y(partial)378 -b(e)-22 b(v)g(aluation)378 b(time,)h(not)e(in)h(the)g(sense)h(of)f -(\223static)h(allocation\224)g(or)e(\223static)27224 -72395 y(method\224.)p Black Black Black eop end + @endspecial -150 2519 1993 3 v 276 2604 a Fs(Figur)o(e)18 +b(3.)37 b FC(Object)19 b(Lifetimes)f(in)h(a)g(T)m(race)p +Black -150 2837 a(the)30 b(interpreter)m(,)g(should)h(the)g(guard)g(f)o +(ail.)e(This)h(means)h(that)f(too)h(man)o(y)-150 2920 +y(guard)20 b(operations)g(also)f(consume)h(a)f(lot)g(of)f(memory)-5 +b(.)-50 3003 y(In)35 b(the)h(rest)f(of)h(the)f(paper)h(we)g(will)e(see) +i(ho)n(w)g(this)f(trace)h(can)g(be)-150 3086 y(optimized)20 +b(using)f(partial)g(e)n(v)n(aluation.)-150 3279 y FA(3.)91 +b(Object)22 b(Lifetimes)h(in)g(a)g(T)-7 b(racing)21 b(JIT)-150 +3395 y FC(T)-6 b(o)15 b(understand)h(the)g(problems)f(that)g(this)g +(paper)h(is)e(trying)i(to)f(solv)o(e)g(in)g(more)-150 +3478 y(detail,)k(we)h(\002rst)f(need)h(to)g(understand)h(v)n(arious)g +(cases)f(of)g(object)g(lifetimes)-150 3561 y(that)f(can)g(occur)h(in)f +(a)f(tracing)i(JIT)e(compiler)l(.)-50 3644 y(Figure)26 +b(3)i(sho)n(ws)f(a)g(trace)g(before)h(optimization,)f(together)h(with)e +(the)-150 3727 y(lifetime)e(of)h(v)n(arious)g(kinds)g(of)g(objects)g +(created)g(in)g(the)g(trace.)f(It)g(is)g(e)o(x)o(e-)-150 +3810 y(cuted)c(from)f(top)h(to)f(bottom.)g(At)g(the)g(bottom,)h(a)f +(jump)h(is)e(used)i(to)g(e)o(x)o(ecute)-150 3893 y(the)g(same)h(loop)g +(another)g(time)f(\(for)h(clarity)-5 b(,)19 b(the)i(\002gure)f(sho)n +(ws)h(tw)o(o)g(itera-)-150 3976 y(tions)15 b(of)g(the)g(loop\).)h(The)f +(loop)g(is)g(e)o(x)o(ecuted)h(until)f(one)h(of)f(the)g(guards)h(in)f +(the)-150 4059 y(trace)k(f)o(ails,)f(and)h(the)f(e)o(x)o(ecution)i(is)e +(aborted)i(and)f(interpretation)g(resumes.)-50 4142 y(Some)30 +b(of)h(the)f(operations)i(within)e(this)g(trace)h(are)f +Fr(new)p Black Black 31 w FC(operations,)-150 4225 y(which)c(each)h +(create)f(a)g(ne)n(w)h(instance)f(of)g(some)h(class.)e(These)i +(instances)-150 4308 y(are)f(used)g(for)f(some)h(time,)f(e.g.,)g(by)h +(calling)g(methods)g(on)g(them)g(\(which)-150 4391 y(are)18 +b(inlined)h(into)f(the)h(trace\),)f(reading)h(and)g(writing)f(their)g +(\002elds.)g(Some)g(of)-150 4474 y(these)25 b(instances)g +Fx(escape)p FC(,)g(which)f(means)h(that)f(the)o(y)h(are)f(stored)h(in)f +(some)-150 4557 y(globally)e(accessible)g(place)g(or)g(are)f(passed)i +(into)e(a)h(non-inlined)g(function)-150 4640 y(via)d(a)g(residual)g +(call.)-50 4723 y(T)-6 b(ogether)17 b(with)g(the)g Fr(new)p +Black Black 17 w FC(operations,)h(the)f(\002gure)g(sho)n(ws)h(the)f +(lifetimes)-150 4806 y(of)25 b(the)h(created)g(objects.)f(The)h +(objects)f(that)h(are)f(created)h(within)f(a)g(trace)-150 +4889 y(using)20 b Fr(new)p Black Black 18 w FC(f)o(all)f(into)g(one)g +(of)g(se)n(v)o(eral)g(cate)o(gories:)p Black -127 5031 +a(1.)p Black 29 w(Objects)j(that)g(li)n(v)o(e)g(for)g(some)h(time,)f +(and)h(are)f(then)g(just)g(not)h(used)g(an)o(y)-42 5114 +y(more)c(afterw)o(ards.)p Black -127 5230 a(2.)p Black +29 w(Objects)g(that)g(li)n(v)o(e)f(for)h(some)h(time)e(and)i(then)f +(escape.)p Black -127 5347 a(3.)p Black 29 w(Objects)j(that)g(li)n(v)o +(e)g(for)g(some)h(time,)e(survi)n(v)o(e)i(across)g(the)f(jump)g(to)g +(the)-42 5430 y(be)o(ginning)e(of)f(the)g(loop,)g(and)h(are)f(then)g +(not)g(used)h(an)o(y)g(more.)p Black Black Black 2065 +66 a(4.)p Black 29 w(Objects)26 b(that)h(li)n(v)o(e)f(for)h(some)g +(time,)f(survi)n(v)o(e)h(across)g(the)g(jump,)f(and)2150 +149 y(then)21 b(escape.)g(T)-6 b(o)20 b(these)h(we)f(also)h(count)g +(the)g(objects)g(that)f(li)n(v)o(e)h(across)2150 232 +y(se)n(v)o(eral)e(jumps)g(and)h(then)f(either)g(escape)h(or)f(stop)g +(being)h(used.)2141 371 y(The)k(objects)g(that)f(are)g(allocated)h(in)f +(the)h(e)o(xample)g(trace)f(in)h(Figure)f(2)2042 454 +y(f)o(all)28 b(into)h(cate)o(gories)g(1)f(and)i(3.)e(Objects)h(stored)g +(in)f Fo(p)3523 462 y Fn(5)3557 454 y FC(,)g Fo(p)3643 +462 y Fn(6)3678 454 y FC(,)g Fo(p)3764 462 y Fn(11)3857 +454 y FC(are)g(in)2042 537 y(cate)o(gory)20 b(1,)e(objects)i(in)e +Fo(p)2737 545 y Fn(10)2802 537 y FC(,)g Fo(p)2878 545 +y Fn(15)2962 537 y FC(are)h(in)f(cate)o(gory)i(3.)2141 +620 y(The)27 b(creation)h(of)f(objects)g(in)g(cate)o(gory)h(1)f(is)g +(remo)o(v)o(ed)h(by)f(the)g(opti-)2042 703 y(mization)d(described)g(in) +f(Sections)h(4)g(and)g(5.)f(Objects)g(in)h(the)f(other)h(cate-)2042 +789 y(gories)19 b(are)g(partially)g(optimized)g(by)h(this)e(approach)j +(as)e(well.)3680 757 y Fz(6)2042 976 y FA(4.)91 b(Allocation)22 +b(Remo)o(v)o(al)g(in)h(T)-7 b(races)2042 1092 y FC(The)21 +b(main)h(insight)g(to)f(impro)o(v)o(e)h(the)g(code)g(sho)n(wn)h(in)e +(Section)g(2.3)h(is)f(that)2042 1175 y(objects)f(in)f(cate)o(gory)i(1)e +(do)i(not)e(survi)n(v)o(e)i(v)o(ery)f(long)g(\226)g(the)o(y)g(are)f +(used)h(only)2042 1258 y(inside)31 b(the)f(loop)i(and)f(there)g(is)f +(no)h(other)g(outside)g(reference)h(to)e(them.)2042 1341 +y(The)k(optimizer)g(identi\002es)f(objects)h(in)g(cate)o(gory)h(1)f +(and)g(remo)o(v)o(es)h(the)2042 1425 y(allocation)19 +b(of)g(these)g(objects,)g(and)h(all)e(operations)i(manipulating)g +(them.)2141 1508 y(This)31 b(is)f(a)h(process)g(that)g(is)f(usually)i +(called)f Fx(escape)h(analysis)f FC([18)q(].)2042 1591 +y(In)k(this)f(paper)h(we)g(will)f(perform)h(escape)h(analysis)f(by)g +(using)g(partial)2042 1674 y(e)n(v)n(aluation.)30 b(The)f(use)g(of)g +(partial)g(e)n(v)n(aluation)h(is)f(a)g(bit)g(peculiar)g(in)g(that)2042 +1757 y(it)16 b(recei)n(v)o(es)h(no)h(static)e(input)h(ar)o(guments)h +(for)e(the)h(trace,)g(b)o(ut)f(it)g(is)h(only)g(used)2042 +1840 y(to)26 b(optimize)g(operations)h(within)f(the)g(trace.)g(This)g +(section)g(will)f(gi)n(v)o(e)i(an)2042 1923 y(informal)17 +b(account)i(of)f(this)f(process)h(by)g(e)o(xamining)h(the)e(e)o(xample) +i(trace)e(in)2042 2006 y(Figure)i(2.)h(The)g(\002nal)f(trace)h(after)f +(optimization)i(can)f(be)g(seen)h(in)e(Figure)h(4)2042 +2089 y(\(the)i(line)f(numbers)i(are)f(the)g(lines)f(of)h(the)g +(unoptimized)h(trace)f(where)g(the)2042 2172 y(operation)e +(originates\).)2141 2255 y(T)-6 b(o)20 b(optimize)h(the)f(trace,)g(it)g +(is)f(tra)o(v)o(ersed)i(from)f(be)o(ginning)h(to)g(end)f(and)2042 +2338 y(an)26 b(output)h(trace)e(is)h(produced.)h(Ev)o(ery)f(operation)h +(in)f(the)f(input)i(trace)e(is)2042 2421 y(either)32 +b(remo)o(v)o(ed)g(or)g(copied)h(into)f(the)g(output)g(trace.)g +(Sometimes)g(ne)n(w)2042 2504 y(operations)e(need)f(to)g(be)g(produced) +h(as)f(well.)f(The)g(optimizer)h(can)g(only)2042 2587 +y(remo)o(v)o(e)18 b(operations)h(that)f(manipulate)h(objects)g(that)e +(ha)o(v)o(e)i(been)g(allocated)2042 2670 y(within)g(the)g(trace,)g +(while)g(all)g(other)h(operations)g(are)f(copied)i(to)e(the)g(output) +2042 2753 y(trace)g(unchanged.)2141 2836 y(Looking)f(at)f(the)g(e)o +(xample)g(trace)g(of)g(Figure)f(2,)h(the)g(operations)g(in)g(lines)2042 +2919 y(1\2269)29 b(are)g(manipulating)g(objects)g(which)g(e)o(xisted)g +(before)g(the)g(trace)f(and)2042 3002 y(that)19 b(are)h(passed)g(in)g +(as)f(ar)o(guments:)h(therefore)g(the)f(optimizer)h(just)f(copies)2042 +3085 y(them)g(into)g(the)g(output)g(trace.)2141 3168 +y(The)g(follo)n(wing)h(operations)g(\(lines)e(10\22617\))j(are)e(more)g +(interesting:)3868 3284 y Fl(10)-1702 b Fj(p)2242 3292 +y Fi(5)2309 3284 y Fp(=)35 b(new\()o(BoxedInteger)o(\))3868 +3354 y Fl(12)-1702 b Fp(set)o(\()p Fj(p)2381 3362 y Fi(5)2414 +3354 y Fp(,)35 b(intval)o(,)g Fj(i)2786 3362 y Fi(4)2818 +3354 y Fp(\))3868 3423 y Fl(15)-1702 b Fj(p)2242 3431 +y Fi(6)2309 3423 y Fp(=)35 b(new\()o(BoxedInteger)o(\))3868 +3493 y Fl(17)-1702 b Fp(set)o(\()p Fj(p)2381 3501 y Fi(6)2414 +3493 y Fp(,)35 b(intval)o(,)f(-100\))p Black Black 2141 +3622 a FC(When)25 b(the)f(optimizer)h(encounters)h(a)e +Fr(new)p Black Black FC(,)f(it)h(remo)o(v)o(es)h(it)f(optimisti-)2042 +3705 y(cally)-5 b(,)20 b(and)g(assumes)h(that)e(the)h(object)h(is)e(in) +h(cate)o(gory)h(1.)e(If)h(later)f(the)h(opti-)2042 3788 +y(mizer)d(\002nds)h(that)f(the)h(object)g(escapes,)g(it)f(will)f(be)i +(allocated)g(at)g(that)f(point.)2042 3871 y(The)28 b(optimizer)g(needs) +h(to)f(k)o(eep)h(track)f(of)g(the)g(state)g(of)g(the)g(object)g(that) +2042 3954 y(the)h(operation)h(w)o(ould)g(ha)o(v)o(e)f(created.)g(This)g +(is)g(done)h(with)e(the)i(help)f(of)2042 4039 y(a)g Fx(static)f(object) +2477 4007 y Fz(7)2507 4039 y FC(.)h(The)g(static)f(object)i(describes)g +(the)f(shape)h(of)f(the)g(ob-)2042 4122 y(ject)20 b(that)h(w)o(ould)g +(ha)o(v)o(e)g(been)g(allocated,)g(i.e.,)e(the)i(type)g(of)g(the)f +(object)h(and)2042 4205 y(where)g(the)f(v)n(alues)i(that)e(w)o(ould)h +(be)g(stored)g(in)g(the)f(\002elds)g(of)h(the)g(allocated)2042 +4288 y(object)e(come)h(from.)2141 4371 y(In)k(the)g(snippet)h(abo)o(v)o +(e,)f(the)g(tw)o(o)g Fr(new)p Black Black 24 w FC(operations)h(are)f +(remo)o(v)o(ed)g(and)2042 4454 y(tw)o(o)e(static)g(objects)h(are)g +(constructed.)g(The)g Fr(set)p Black Black 22 w FC(operations)g +(manipulate)2042 4537 y(static)18 b(objects,)i(therefore)f(the)o(y)g +(can)h(be)f(remo)o(v)o(ed)h(as)f(well;)g(their)f(ef)n(fect)h(is)2042 +4620 y(remembered)h(in)f(the)g(static)f(objects.)2141 +4703 y(After)k(the)h(operations)h(the)f(static)f(object)h(associated)g +(with)g Fo(p)3792 4711 y Fn(5)3848 4703 y FC(w)o(ould)2042 +4786 y(store)31 b(the)h(kno)n(wledge)i(that)d(it)g(is)g(a)h +Fr(BoxedInteger)p Black Black 31 w FC(whose)g Fr(intval)p +Black Black 2042 4869 a FC(\002eld)19 b(contains)h Fo(i)2488 +4877 y Fn(4)2523 4869 y FC(;)f(the)g(one)h(associated)g(with)f +Fo(p)3317 4877 y Fn(6)3371 4869 y FC(w)o(ould)h(store)f(that)g(it)g(is) +g(a)2042 4952 y Fr(BoxedInteger)p Black Black 18 w FC(whose)h +Fr(intval)p Black Black 18 w FC(\002eld)f(contains)h(the)f(constant)g +(-100.)p Black 2042 5034 997 3 v 2042 5085 a Fz(6)2079 +5108 y Fw(W)-5 b(e)15 b(also)h(started)h(to)f(w)o(ork)g(on)g +(optimizing)i(objects)f(in)f(cate)o(gory)i(3,)d(which)h(will)h(be)2042 +5183 y(the)h(subject)g(of)f(a)g(later)i(paper)l(.)2042 +5257 y Fz(7)2079 5280 y Fw(Here)27 b(\223static\224)h(is)e(meant)g(in)g +(the)h(sense)f(of)g(partial)i(e)n(v)n(aluation,)h(i.e.,)c(kno)n(wn)h +(at)2042 5355 y(partial)31 b(e)n(v)n(aluation)i(time,)c(not)g(in)g(the) +g(sense)g(of)g(\223static)i(allocation\224)i(or)c(\223static)2042 +5430 y(method\224.)p Black Black Black eop end %%Page: 5 5 -TeXDict begin 5 4 bop Black Black -672 886 a FC(The)287 -b(subsequent)g(operations)f(\(line)h(20\22626\))f(in)h(Figure)f(2,)h -(which)g(use)-2000 1993 y Fo(p)-1486 2104 y Fn(5)-776 -1993 y FC(and)250 b Fo(p)1426 2104 y Fn(6)1887 1993 y -FC(,)f(can)h(then)g(be)f(optimized)h(using)f(that)h(kno)-25 -b(wledge:)22353 3438 y Fl(20)-22694 b Fp(g)o(u)o(a)o(r)o(d)2538 -3255 y(_)3003 3438 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)6249 -3549 y Fi(5)6680 3438 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)o(\))22353 4368 y Fl(22)-22693 b Fj(i)527 -4479 y Fi(7)1423 4368 y Fp(=)464 b(g)o(e)o(t)o(\()p Fj(p)4670 -4479 y Fi(5)5101 4368 y Fp(,)g(i)o(n)o(t)o(v)o(a)o(l)o(\))22353 -5298 y Fl(23)-22694 b Fp(g)o(u)o(a)o(r)o(d)2538 5115 -y(_)3003 5298 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)6249 5409 -y Fi(6)6680 5298 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o -(e)o(r)o(\))22353 6227 y Fl(25)-22693 b Fj(i)527 6338 -y Fi(8)1423 6227 y Fp(=)464 b(g)o(e)o(t)o(\()p Fj(p)4670 -6338 y Fi(6)5101 6227 y Fp(,)g(i)o(n)o(t)o(v)o(a)o(l)o(\))22353 -7157 y Fl(26)-22693 b Fj(i)527 7268 y Fi(9)1423 7157 -y Fp(=)464 b(i)o(n)o(t)3748 6974 y(_)4213 7157 y(a)o(d)o(d)o(\()o -Fj(i)6386 7268 y Fi(7)6818 7157 y Fp(,)g Fj(i)8061 7268 -y Fi(8)8493 7157 y Fp(\))p Black Black -672 8780 a FC(The)336 -b Fr(guard)3878 8571 y(_)4411 8780 y(class)p Black Black -338 w FC(operations)g(can)h(be)f(remo)-15 b(v)g(ed,)337 -b(since)f(their)g(ar)-20 b(-)-2000 9887 y(guments)249 -b(are)g(static)g(objects)g(with)g(the)h(matching)f(type)h -Fr(BoxedInteger)p Black Black 1 w FC(.)-2000 10994 y(The)263 -b Fr(get)p Black Black 264 w FC(operations)g(can)h(be)f(remo)-15 -b(v)g(ed)265 b(as)e(well,)g(because)h(each)h(of)e(them)-2000 -12101 y(reads)288 b(a)g(\002eld)g(out)g(of)g(a)g(static)g(object.)g -(The)g(results)f(of)h(the)g(get)g(operation)-2000 13208 -y(are)227 b(replaced)h(with)e(what)h(the)g(static)g(object)g(stores)f -(in)g(these)h(\002elds:)f(all)h(the)-2000 14315 y(occurences)213 -b(of)g Fo(i)4030 14426 y Fn(7)4703 14315 y FC(and)g Fo(i)6704 -14426 y Fn(8)7378 14315 y FC(in)f(the)g(trace)h(are)g(just)f(replaced)h -(by)f Fo(i)20167 14426 y Fn(4)20841 14315 y FC(and)h(-100.)-2000 -15422 y(The)249 b(only)h(operation)g(copied)g(into)f(the)g(optimized)h -(trace)g(is)f(the)g(addition:)22353 16867 y Fl(26)-22693 -b Fj(i)527 16978 y Fi(9)1423 16867 y Fp(=)464 b(i)o(n)o(t)3748 -16684 y(_)4213 16867 y(a)o(d)o(d)o(\()o Fj(i)6386 16978 -y Fi(4)6818 16867 y Fp(,)e(-)o(1)o(0)o(0)o(\))p Black -Black -672 18489 a FC(The)204 b(rest)f(of)h(the)g(trace)g(from)g -(Figure)g(2)g(is)f(optimized)h(in)g(a)g(similar)f(v)-15 -b(ein.)-2000 19596 y(The)262 b(operations)g(in)f(lines)h(27\22635)g -(produce)g(tw)-10 b(o)262 b(more)g(static)g(objects)f(and)-2000 -20703 y(are)348 b(remo)-15 b(v)g(ed.)349 b(Those)e(in)h(line)g -(36\22639)h(are)f(just)f(copied)i(into)f(the)g(output)-2000 -21810 y(trace)274 b(because)g(the)-15 b(y)274 b(manipulate)h(objects)e -(that)h(are)f(allocated)i(before)f(the)-2000 22917 y(trace.)369 -b(Lines)e(40\22642)i(are)f(remo)-15 b(v)g(ed)369 b(because)g(the)-15 -b(y)368 b(operate)h(on)f(a)g(static)-2000 24024 y(object.)213 -b(Line)e(43)h(is)f(copied)i(into)f(the)g(output)g(trace.)g(Lines)g -(44\22646)g(produce)-2000 25131 y(a)330 b(ne)-25 b(w)331 -b(static)f(object)g(and)g(are)h(remo)-15 b(v)g(ed,)331 -b(lines)e(48\22651)i(manipulate)g(that)-2000 26238 y(static)272 -b(object)i(and)f(are)g(remo)-15 b(v)g(ed)273 b(as)f(well.)h(Lines)f -(52\22654)i(are)e(copied)i(into)-2000 27345 y(the)249 -b(output)h(trace.)-672 28452 y(The)g(last)f(operation)h(\(line)f(55\))h -(is)e(an)i(interesting)f(case.)h(It)f(is)g(the)h Fr(jump)p -Black Black -2000 29559 a FC(operation)387 b(that)h(passes)e(control)h -(back)g(to)g(the)g(be)-15 b(ginning)388 b(of)e(the)h(trace.)-2000 -30666 y(The)257 b(tw)-10 b(o)257 b(ar)-18 b(guments)258 -b(to)f(this)f(operation)i(at)f(this)f(point)h(are)h(static)f(objects.) --2000 31773 y(Ho)-25 b(we)g(v)-15 b(er)-40 b(,)223 b(because)g(the)-15 -b(y)222 b(are)g(passed)g(into)g(the)g(ne)-15 b(xt)222 -b(iteration)g(of)g(the)g(loop)-2000 32880 y(the)-15 b(y)320 -b(li)-25 b(v)-15 b(e)319 b(longer)g(than)h(the)f(trace)h(and)f -(therefore)h(cannot)f(remain)h(static.)-2000 33987 y(The)-15 -b(y)307 b(need)g(to)g(be)g(turned)g(into)f(dynamic)i(\(runtime\))e -(objects)h(before)g(the)-2000 35094 y(actual)238 b Fr(jump)p -Black Black 237 w FC(operation.)g(This)e(process)g(of)h(turning)f(a)i -(static)e(object)i(into)f(a)-2000 36201 y(dynamic)250 -b(one)g(is)e(called)i Fx(lifting)p FC(.)-672 37308 y(Lifting)427 -b(a)h(static)g(object)g(puts)f Fr(new)p Black Black 428 -w FC(and)h Fr(set)p Black Black 428 w FC(operations)g(into)g(the)-2000 -38415 y(output)373 b(trace.)g(Those)g(operations)g(produce)g(an)g -(object)h(at)f(runtime)g(that)-2000 39522 y(has)335 b(the)g(shape)g -(described)h(by)f(the)g(static)f(object.)i(This)e(process)h(is)f(a)h -(bit)-2000 40629 y(delicate,)350 b(because)f(the)g(static)f(objects)h -(could)g(form)f(an)g(arbitrary)h(graph)-2000 41736 y(structure.)249 -b(In)g(our)g(e)-15 b(xample)250 b(it)f(is)g(simple,)g(though:)22353 -43181 y Fl(44)-22693 b Fj(p)670 43292 y Fi(15)1943 43181 -y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o -(e)o(r)o(\))22353 44111 y Fl(46)-22694 b Fp(s)o(e)o(t)o(\()p -Fj(p)2530 44222 y Fi(15)3338 44111 y Fp(,)463 b(i)o(n)o(t)o(v)o(a)o(l)o -(,)i Fj(i)8301 44222 y Fi(14)9109 44111 y Fp(\))22353 -45041 y Fl(27)-22693 b Fj(p)670 45152 y Fi(10)1943 45041 -y Fp(=)464 b(n)o(e)o(w)o(\()n(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o -(e)o(r)o(\))22353 45970 y Fl(29)-22694 b Fp(s)o(e)o(t)o(\()p -Fj(p)2530 46081 y Fi(10)3338 45970 y Fp(,)463 b(i)o(n)o(t)o(v)o(a)o(l)o -(,)i Fj(i)8301 46081 y Fi(9)8732 45970 y Fp(\))22353 -46900 y Fl(55)-22694 b Fp(j)o(u)o(m)o(p)o(\()p Fj(p)2995 -47011 y Fi(15)3803 46900 y Fp(,)464 b Fj(p)5189 47011 -y Fi(10)5998 46900 y Fp(\))p Black Black -672 48523 a -FC(Observ)-15 b(e)416 b(ho)-25 b(w)416 b(the)f(operations)h(for)f -(creating)h(these)f(tw)-10 b(o)416 b(instances)-2000 -49630 y(ha)-20 b(v)-15 b(e)333 b(been)g(mo)-15 b(v)g(ed)332 -b(to)g(a)g(later)g(point)g(in)g(the)g(trace.)h(This)e(is)g(w)-10 -b(orthwhile)-2000 50737 y(e)-25 b(v)-15 b(en)381 b(though)f(the)g -(objects)g(ha)-20 b(v)-15 b(e)381 b(to)f(be)g(allocated)h(in)f(the)g -(end)g(because)-2000 51844 y(some)368 b Fr(get)p Black -Black 368 w FC(operations)h(and)g Fr(guard)11429 51635 -y(_)11962 51844 y(class)p Black Black 369 w FC(operations)f(on)h(the)f -(lifted)-2000 52951 y(static)249 b(objects)h(could)f(be)h(remo)-15 -b(v)g(ed.)-672 54058 y(More)311 b(generally)-65 b(,)312 -b(lifting)f(needs)g(to)g(occur)g(if)f(a)h(static)g(object)h(is)e(used) --2000 55165 y(in)281 b(an)-15 b(y)281 b(operation)h(apart)f(from)g -Fr(get)p Black Black FC(,)g Fr(set)p Black Black 1 w -FC(,)f(and)i Fr(guard)p Black Black FC(.)f(It)g(also)g(needs)g(to)-2000 -56272 y(occur)250 b(if)f Fr(set)p Black Black 249 w FC(is)f(used)i(to)f -(store)g(a)g(static)g(object)h(into)f(a)h(non-static)f(one.)-672 -57378 y(The)198 b(\002nal)g(optimized)h(trace)f(of)g(the)g(e)-15 -b(xample)199 b(can)g(be)f(seen)g(in)g(Figure)g(4.)-2000 -58485 y(The)343 b(optimized)h(trace)f(contains)h(only)f(tw)-10 -b(o)343 b(allocations,)h(instead)f(of)g(the)-2000 59592 -y(original)c(\002)-25 b(v)-15 b(e,)339 b(and)h(only)f(three)g -Fr(guard)12363 59383 y(_)12896 59592 y(class)p Black -Black 340 w FC(operations,)g(compared)-2000 60699 y(to)249 -b(the)h(original)f(se)-25 b(v)-15 b(en.)-2000 63097 y -FA(5.)1218 b(F)-30 b(ormal)304 b(Description)g(of)h(the)g(Algorithm) --2000 64646 y FC(In)203 b(this)f(section)h(we)h(w)-10 -b(ant)203 b(to)g(gi)-25 b(v)-15 b(e)204 b(a)f(formal)g(description)g -(of)g(the)g(semantics)-2000 65753 y(of)404 b(the)g(traces)g(and)g(of)g -(the)g(optimizer)g(and)h(lik)-10 b(en)404 b(the)g(optimization)h(to) --2000 66860 y(partial)429 b(e)-25 b(v)g(aluation.)430 -b(W)-80 b(e)429 b(focus)g(on)f(the)h(operations)g(for)f(manipulating) --2000 67967 y(heap)291 b(allocated)g(objects,)g(as)e(those)h(are)h(the) -f(only)g(ones)g(that)h(are)f(actually)-2000 69074 y(optimized.)197 -b(W)-80 b(e)197 b(also)e(consider)h(only)h(objects)f(with)g(tw)-10 -b(o)196 b(\002elds)g Fo(L)f FC(and)h Fo(R)203 b FC(in)-2000 -70181 y(this)190 b(section,)h(generalizing)h(to)e(arbitrary)h(man)-15 -b(y)191 b(\002elds)f(is)g(straightforw)-10 b(ard.)-2000 -71288 y(T)-35 b(races)328 b(are)g(lists)e(of)i(operations.)g(The)g -(operations)g(considered)g(here)h(are)-2000 72395 y Fr(new)p -Black Black FC(,)249 b Fr(get)p Black Black 1 w FC(,)g -Fr(set)p Black Black 249 w FC(and)h Fr(guard)8396 72186 -y(_)8929 72395 y(class)p Black Black 1 w FC(.)p Black -Black Black 51577 1315 a Fl(1)p 0.3 0.3 0.3 TeXcolorrgb --24631 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 -TeXcolorrgb 464 w(a)o(r)o(g)o(u)o(m)o(e)o(n)o(t)o(s)p +TeXDict begin 5 4 bop Black Black -50 66 a FC(The)21 +b(subsequent)j(operations)f(\(line)e(20\22626\))i(in)f(Figure)f(2,)h +(which)g(use)-150 149 y Fo(p)-111 157 y Fn(5)-58 149 +y FC(and)d Fo(p)107 157 y Fn(6)142 149 y FC(,)f(can)h(then)h(be)f +(optimized)g(using)h(that)f(kno)n(wledge:)1676 258 y +Fl(20)-1702 b Fp(guard)191 244 y(_)226 258 y(class)o(\()o +Fj(p)468 266 y Fi(5)501 258 y Fp(,)35 b(BoxedInteger)n(\))1676 +328 y Fl(22)-1702 b Fj(i)39 336 y Fi(7)107 328 y Fp(=)34 +b(get\()p Fj(p)350 336 y Fi(5)383 328 y Fp(,)g(intval\))1676 +397 y Fl(23)-1702 b Fp(guard)191 383 y(_)226 397 y(class)o(\()o +Fj(p)468 405 y Fi(6)501 397 y Fp(,)35 b(BoxedInteger)n(\))1676 +467 y Fl(25)-1702 b Fj(i)39 475 y Fi(8)107 467 y Fp(=)34 +b(get\()p Fj(p)350 475 y Fi(6)383 467 y Fp(,)g(intval\))1676 +537 y Fl(26)-1702 b Fj(i)39 545 y Fi(9)107 537 y Fp(=)34 +b(int)281 523 y(_)316 537 y(add\()p Fj(i)479 545 y Fi(7)511 +537 y Fp(,)h Fj(i)604 545 y Fi(8)637 537 y Fp(\))p Black +Black -50 658 a FC(The)25 b Fr(guard)291 643 y(_)331 +658 y(class)p Black Black 25 w FC(operations)h(can)g(be)g(remo)o(v)o +(ed,)g(since)g(their)f(ar)o(-)-150 742 y(guments)20 b(are)f(static)f +(objects)h(with)g(the)g(matching)h(type)f Fr(BoxedInteger)p +Black Black FC(.)-150 825 y(The)h Fr(get)p Black Black +20 w FC(operations)h(can)f(be)g(remo)o(v)o(ed)h(as)f(well,)f(because)j +(each)e(of)g(them)-150 908 y(reads)i(a)g(\002eld)f(out)i(of)e(a)h +(static)f(object.)h(The)g(results)g(of)g(the)f(get)h(operation)-150 +991 y(are)17 b(replaced)h(with)f(what)g(the)h(static)e(object)i(stores) +f(in)g(these)h(\002elds:)f(all)f(the)-150 1074 y(occurences)i(of)e +Fo(i)302 1082 y Fn(7)353 1074 y FC(and)h Fo(i)503 1082 +y Fn(8)553 1074 y FC(in)f(the)h(trace)f(are)g(just)g(replaced)h(by)f +Fo(i)1512 1082 y Fn(4)1563 1074 y FC(and)h(-100.)-150 +1157 y(The)i(only)g(operation)h(copied)g(into)f(the)g(optimized)h +(trace)f(is)f(the)h(addition:)1676 1265 y Fl(26)-1702 +b Fj(i)39 1273 y Fi(9)107 1265 y Fp(=)34 b(int)281 1251 +y(_)316 1265 y(add\()p Fj(i)479 1273 y Fi(4)511 1265 +y Fp(,)h(-100\))p Black Black -50 1387 a FC(The)15 b(rest)g(of)h(the)f +(trace)h(from)f(Figure)h(2)f(is)g(optimized)i(in)e(a)g(similar)g(v)o +(ein.)-150 1470 y(The)20 b(operations)h(in)e(lines)h(27\22635)i +(produce)f(tw)o(o)f(more)g(static)f(objects)i(and)-150 +1553 y(are)26 b(remo)o(v)o(ed.)h(Those)g(in)f(line)g(36\22639)i(are)f +(just)f(copied)h(into)f(the)h(output)-150 1636 y(trace)21 +b(because)h(the)o(y)f(manipulate)g(objects)g(that)g(are)g(allocated)g +(before)g(the)-150 1719 y(trace.)27 b(Lines)h(40\22642)h(are)f(remo)o +(v)o(ed)h(because)g(the)o(y)f(operate)h(on)f(a)g(static)-150 +1802 y(object.)16 b(Line)g(43)h(is)e(copied)i(into)g(the)f(output)h +(trace.)e(Lines)h(44\22646)i(produce)-150 1885 y(a)25 +b(ne)n(w)g(static)g(object)g(and)h(are)f(remo)o(v)o(ed,)g(lines)g +(48\22651)i(manipulate)e(that)-150 1968 y(static)20 b(object)h(and)h +(are)e(remo)o(v)o(ed)i(as)f(well.)e(Lines)i(52\22654)h(are)f(copied)g +(into)-150 2051 y(the)e(output)h(trace.)-50 2134 y(The)f(last)f +(operation)i(\(line)f(55\))g(is)f(an)i(interesting)f(case.)g(It)f(is)h +(the)g Fr(jump)p Black Black -150 2217 a FC(operation)30 +b(that)f(passes)h(control)g(back)g(to)f(the)g(be)o(ginning)i(of)e(the)h +(trace.)-150 2300 y(The)19 b(tw)o(o)h(ar)o(guments)g(to)g(this)f +(operation)h(at)g(this)f(point)h(are)f(static)g(objects.)-150 +2383 y(Ho)n(we)n(v)o(er)m(,)e(because)h(the)o(y)g(are)e(passed)i(into)f +(the)g(ne)o(xt)g(iteration)g(of)g(the)g(loop)-150 2466 +y(the)o(y)24 b(li)n(v)o(e)g(longer)h(than)g(the)f(trace)g(and)h +(therefore)g(cannot)g(remain)f(static.)-150 2549 y(The)o(y)f(need)h(to) +g(be)f(turned)h(into)f(dynamic)i(\(runtime\))e(objects)g(before)h(the) +-150 2632 y(actual)18 b Fr(jump)p Black Black 18 w FC(operation.)h +(This)e(process)i(of)f(turning)h(a)e(static)h(object)g(into)g(a)-150 +2715 y(dynamic)i(one)g(is)e(called)h Fx(lifting)p FC(.)-50 +2798 y(Lifting)31 b(a)h(static)g(object)h(puts)g Fr(new)p +Black Black 32 w FC(and)g Fr(set)p Black Black 32 w FC(operations)g +(into)f(the)-150 2881 y(output)d(trace.)f(Those)g(operations)i(produce) +f(an)g(object)f(at)g(runtime)g(that)-150 2964 y(has)e(the)f(shape)h +(described)h(by)f(the)f(static)g(object.)g(This)g(process)h(is)f(a)h +(bit)-150 3047 y(delicate,)g(because)i(the)e(static)g(objects)h(could)g +(form)f(an)h(arbitrary)f(graph)-150 3130 y(structure.)19 +b(In)g(our)g(e)o(xample)h(it)e(is)g(simple,)h(though:)1676 +3239 y Fl(44)-1702 b Fj(p)50 3247 y Fi(15)146 3239 y +Fp(=)34 b(new\(BoxedInteger)n(\))1676 3308 y Fl(46)-1702 +b Fp(set\()p Fj(p)190 3316 y Fi(15)250 3308 y Fp(,)35 +b(intval)o(,)g Fj(i)622 3316 y Fi(14)683 3308 y Fp(\))1676 +3378 y Fl(27)-1702 b Fj(p)50 3386 y Fi(10)146 3378 y +Fp(=)34 b(new\(BoxedInteger)n(\))1676 3448 y Fl(29)-1702 +b Fp(set\()p Fj(p)190 3456 y Fi(10)250 3448 y Fp(,)35 +b(intval)o(,)g Fj(i)622 3456 y Fi(9)655 3448 y Fp(\))1676 +3518 y Fl(55)-1702 b Fp(jump)o(\()p Fj(p)224 3526 y Fi(15)285 +3518 y Fp(,)35 b Fj(p)389 3526 y Fi(10)450 3518 y Fp(\))p +Black Black -50 3639 a FC(Observ)o(e)d(ho)n(w)g(the)f(operations)h(for) +g(creating)f(these)h(tw)o(o)g(instances)-150 3722 y(ha)o(v)o(e)25 +b(been)h(mo)o(v)o(ed)g(to)f(a)g(later)g(point)h(in)f(the)g(trace.)f +(This)h(is)g(w)o(orthwhile)-150 3805 y(e)n(v)o(en)30 +b(though)g(the)e(objects)i(ha)o(v)o(e)e(to)h(be)g(allocated)g(in)g(the) +g(end)g(because)-150 3888 y(some)f Fr(get)p Black Black +28 w FC(operations)h(and)f Fr(guard)857 3873 y(_)897 +3888 y(class)p Black Black 28 w FC(operations)h(on)f(the)g(lifted)-150 +3971 y(static)18 b(objects)i(could)f(be)h(remo)o(v)o(ed.)-50 +4054 y(More)k(generally)-5 b(,)24 b(lifting)f(needs)h(to)g(occur)g(if)f +(a)g(static)h(object)f(is)h(used)-150 4137 y(in)d(an)o(y)h(operation)g +(apart)g(from)f Fr(get)p Black Black FC(,)g Fr(set)p +Black Black FC(,)f(and)i Fr(guard)p Black Black FC(.)f(It)g(also)g +(needs)h(to)-150 4220 y(occur)e(if)e Fr(set)p Black Black +19 w FC(is)g(used)i(to)f(store)f(a)h(static)g(object)g(into)g(a)g +(non-static)g(one.)-50 4303 y(The)c(\002nal)f(optimized)i(trace)f(of)g +(the)g(e)o(xample)h(can)g(be)f(seen)h(in)f(Figure)f(4.)-150 +4386 y(The)26 b(optimized)g(trace)g(contains)h(only)g(tw)o(o)f +(allocations,)g(instead)g(of)g(the)-150 4469 y(original)g(\002)n(v)o +(e,)f(and)h(only)h(three)e Fr(guard)927 4454 y(_)967 +4469 y(class)p Black Black 26 w FC(operations,)h(compared)-150 +4552 y(to)19 b(the)g(original)g(se)n(v)o(en.)-150 4732 +y FA(5.)91 b(F)n(ormal)21 b(Description)h(of)h(the)f(Algorithm)-150 +4848 y FC(In)15 b(this)h(section)g(we)f(w)o(ant)h(to)f(gi)n(v)o(e)h(a)f +(formal)h(description)g(of)f(the)h(semantics)-150 4932 +y(of)31 b(the)f(traces)h(and)g(of)f(the)h(optimizer)g(and)g(lik)o(en)g +(the)f(optimization)h(to)-150 5015 y(partial)h(e)n(v)n(aluation.)h(W)-6 +b(e)32 b(focus)h(on)g(the)g(operations)g(for)f(manipulating)-150 +5098 y(heap)23 b(allocated)f(objects,)g(as)g(those)h(are)f(the)g(only)g +(ones)h(that)f(are)g(actually)-150 5181 y(optimized.)15 +b(W)-6 b(e)15 b(also)g(consider)h(only)f(objects)g(with)g(tw)o(o)g +(\002elds)g Fo(L)g FC(and)g Fo(R)g FC(in)-150 5264 y(this)f(section,)h +(generalizing)h(to)e(arbitrary)g(man)o(y)i(\002elds)e(is)g +(straightforw)o(ard.)-150 5347 y(T)m(races)25 b(are)g(lists)f(of)h +(operations.)g(The)g(operations)h(considered)g(here)f(are)-150 +5430 y Fr(new)p Black Black FC(,)18 b Fr(get)p Black +Black FC(,)h Fr(set)p Black Black 18 w FC(and)h Fr(guard)630 +5414 y(_)670 5430 y(class)p Black Black FC(.)p Black +Black Black 3868 99 a Fl(1)p 0.3 0.3 0.3 TeXcolorrgb +-1847 w Fk(#)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 +TeXcolorrgb 35 w(arguments)p Black 0.3 0.3 0.3 TeXcolorrgb +Black 0.3 0.3 0.3 TeXcolorrgb 33 w(to)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb 35 w(the)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 TeXcolorrgb -463 w(t)o(o)p Black 0.3 0.3 0.3 TeXcolorrgb Black 0.3 0.3 0.3 -TeXcolorrgb 464 w(t)o(h)o(e)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 0.3 0.3 0.3 TeXcolorrgb 463 w(t)o(r)o(a)o(c)o(e)p -Black 0.3 0.3 0.3 TeXcolorrgb -1 w(:)p Black 0.3 0.3 0.3 -TeXcolorrgb Black 465 w Fj(p)39768 1426 y Fi(0)p 0.3 0.3 0.3 -TeXcolorrgb 40199 1315 a Fk(,)p Black 0.3 0.3 0.3 TeXcolorrgb -Black 465 w Fj(p)41586 1426 y Fi(1)p 0.3 0.3 0.3 TeXcolorrgb -Black 51577 2245 a Fl(3)-24631 b Fp(g)o(u)o(a)o(r)o(d)29548 -2062 y(_)30013 2245 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 -2356 y Fi(1)33690 2245 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)n(\))51577 3175 y Fl(5)-24630 b Fj(i)27537 -3286 y Fi(2)28433 3175 y Fp(=)464 b(g)o(e)o(t)o(\()o -Fj(p)31679 3286 y Fi(1)32111 3175 y Fp(,)f(i)o(n)o(t)o(v)o(a)o(l)o(\)) -51577 4105 y Fl(6)-24631 b Fp(g)o(u)o(a)o(r)o(d)29548 -3922 y(_)30013 4105 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 -4216 y Fi(0)33690 4105 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)n(\))51577 5034 y Fl(8)-24630 b Fj(i)27537 -5145 y Fi(3)28433 5034 y Fp(=)464 b(g)o(e)o(t)o(\()o -Fj(p)31679 5145 y Fi(0)32111 5034 y Fp(,)f(i)o(n)o(t)o(v)o(a)o(l)o(\)) -51577 5964 y Fl(9)-24630 b Fj(i)27537 6075 y Fi(4)28433 -5964 y Fp(=)463 b(i)o(n)o(t)30757 5781 y(_)31222 5964 -y(a)o(d)o(d)o(\()p Fj(i)33396 6075 y Fi(2)33827 5964 -y Fp(,)i Fj(i)35071 6075 y Fi(3)35503 5964 y Fp(\))51577 -6894 y Fl(26)-24907 b Fj(i)27537 7005 y Fi(9)28433 6894 -y Fp(=)463 b(i)o(n)o(t)30757 6711 y(_)31222 6894 y(a)o(d)o(d)o(\()p -Fj(i)33396 7005 y Fi(4)33827 6894 y Fp(,)g(-)o(1)o(0)o(0)o(\))51577 -8754 y Fl(37)-24908 b Fp(g)o(u)o(a)o(r)o(d)29548 8571 -y(_)30013 8754 y(c)o(l)o(a)o(s)o(s)n(\()p Fj(p)33259 -8865 y Fi(0)33690 8754 y Fp(,)464 b(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o -(g)o(e)o(r)n(\))51577 9684 y Fl(39)-24907 b Fj(i)27537 -9795 y Fi(12)28810 9684 y Fp(=)463 b(g)o(e)o(t)o(\()p -Fj(p)32056 9795 y Fi(0)32487 9684 y Fp(,)h(i)o(n)o(t)o(v)o(a)o(l)o(\)) -51577 10614 y Fl(43)-24907 b Fj(i)27537 10725 y Fi(14)28810 -10614 y Fp(=)463 b(i)o(n)o(t)31134 10431 y(_)31599 10614 -y(a)o(d)o(d)o(\()p Fj(i)33773 10725 y Fi(12)34581 10614 -y Fp(,)g(-)o(1)o(\))51577 12473 y Fl(52)-24907 b Fj(i)27537 -12584 y Fi(17)28810 12473 y Fp(=)463 b(i)o(n)o(t)31134 -12290 y(_)31599 12473 y(g)o(t)o(\()p Fj(i)33308 12584 -y Fi(14)34116 12473 y Fp(,)h(0)o(\))51577 13403 y Fl(54)-24908 -b Fp(g)o(u)o(a)o(r)o(d)29548 13220 y(_)30013 13403 y(t)o(r)o(u)o(e)n -(\()p Fj(i)32651 13514 y Fi(17)33459 13403 y Fp(\))51577 -15263 y Fl(44)h Fj(p)27680 15374 y Fi(15)28953 15263 -y Fp(=)463 b(n)o(e)o(w\()n(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g)o(e)o -(r)o(\))51577 16193 y Fl(46)-24908 b Fp(s)o(e)o(t)o(\()o -Fj(p)29539 16304 y Fi(15)30347 16193 y Fp(,)464 b(i)o(n)o(t)o(v)o(a)o -(l)o(,)h Fj(i)35311 16304 y Fi(14)36119 16193 y Fp(\))51577 -17123 y Fl(27)-24907 b Fj(p)27680 17234 y Fi(10)28953 -17123 y Fp(=)463 b(n)o(e)o(w\()n(B)o(o)o(x)o(e)o(d)o(I)o(n)o(t)o(e)o(g) -o(e)o(r)o(\))51577 18052 y Fl(29)-24908 b Fp(s)o(e)o(t)o(\()o -Fj(p)29539 18163 y Fi(10)30347 18052 y Fp(,)464 b(i)o(n)o(t)o(v)o(a)o -(l)o(,)h Fj(i)35311 18163 y Fi(9)35742 18052 y Fp(\))51577 -19912 y Fl(55)-24908 b Fp(j)o(u)o(m)o(p)o(\()o Fj(p)30004 -20023 y Fi(15)30812 19912 y Fp(,)465 b Fj(p)32199 20023 -y Fi(10)33008 19912 y Fp(\))p Black Black 27224 20892 -26568 37 v 29901 22021 a Fs(Figur)-18 b(e)249 b(4.)499 -b FC(Resulting)248 b(T)-35 b(race)250 b(After)f(Allocation)h(Remo)-15 -b(v)-25 b(al)p Black 28552 25493 a(The)437 b(v)-25 b(alues)437 -b(of)g(all)g(v)-25 b(ariables)437 b(are)g(locations)g(\(i.e.,)g -(pointers\).)f(Lo-)27224 26600 y(cations)422 b(are)g(mapped)i(to)e -(objects,)g(which)h(are)f(represented)h(by)f(triples)27224 -27706 y Fh(\()p Fo(T)85 b(;)172 b(l)29066 27817 y Fn(1)29526 -27706 y Fo(;)g(l)30288 27817 y Fn(2)30748 27706 y Fh(\))234 -b FC(of)e(a)i(type)f Fo(T)142 b FC(,)234 b(and)f(tw)-10 -b(o)234 b(locations)f(that)g(represent)h(the)f(\002elds)g(of)27224 -28813 y(the)310 b(object.)h(When)f(a)g(ne)-25 b(w)311 -b(object)g(is)e(created,)i(the)f(\002elds)g(are)g(initialized)27224 -29920 y(to)243 b(null,)h(b)-20 b(ut)244 b(we)g(require)g(that)g(the)-15 -b(y)245 b(are)f(initialized)g(to)g(a)g(real)g(location)h(be-)27224 -31027 y(fore)252 b(being)h(read,)g(otherwise)g(the)g(trace)g(is)f -(malformed)g(\(this)g(condition)h(is)27224 32134 y(guaranteed)d(by)g -(ho)-25 b(w)249 b(the)h(traces)f(are)h(generated)g(in)f(PyPy\).)28552 -33241 y(W)-80 b(e)234 b(use)f(some)h(abbre)-25 b(viations)234 -b(when)f(dealing)i(with)e(object)h(triples.)f(T)-80 b(o)27224 -34348 y(read)279 b(the)g(type)g(of)f(an)h(object,)g Fh(t)-28 -b(yp)28 b(e)q(\(\()p Fo(T)85 b(;)172 b(l)42199 34459 -y Fn(1)42660 34348 y Fo(;)f(l)43421 34459 y Fn(2)43882 -34348 y Fh(\)\))341 b(=)g Fo(T)421 b FC(is)278 b(used.)g(Reading)27224 -35455 y(a)335 b(\002eld)h Fo(F)477 b FC(from)335 b(an)h(object)g(is)f -(written)h Fh(\()p Fo(T)85 b(;)172 b(l)43557 35566 y -Fn(1)44017 35455 y Fo(;)f(l)44778 35566 y Fn(2)45239 -35455 y Fh(\))45637 35566 y FD(F)46650 35455 y FC(which)336 -b(either)g(is)f Fo(l)53330 35566 y Fn(1)27224 36562 y -FC(if)314 b Fo(F)550 b Fh(=)408 b Fo(L)314 b FC(or)g -Fo(l)33016 36673 y Fn(2)33791 36562 y FC(if)g Fo(F)550 -b Fh(=)409 b Fo(R)7 b FC(.)314 b(T)-80 b(o)314 b(set)g(\002eld)g -Fo(F)456 b FC(to)314 b(a)h(ne)-25 b(w)315 b(location)g -Fo(l)17 b FC(,)313 b(we)27224 37669 y(use)h(the)g(notation)h -Fh(\()p Fo(T)85 b(;)172 b(l)35819 37780 y Fn(1)36280 -37669 y Fo(;)f(l)37041 37780 y Fn(2)37502 37669 y Fh(\)!)38184 -37780 y FD(F)38862 37669 y Fo(l)17 b FC(,)313 b(which)i(yields)f(a)h -(ne)-25 b(w)315 b(triple)f Fh(\()p Fo(T)85 b(;)172 b(l)17 -b(;)170 b(l)52932 37780 y Fn(2)53393 37669 y Fh(\))27224 -38776 y FC(if)249 b Fo(F)426 b Fh(=)285 b Fo(L)249 b -FC(or)g(a)g(ne)-25 b(w)250 b(triple)f Fh(\()p Fo(T)85 -b(;)172 b(l)39037 38887 y Fn(1)39498 38776 y Fo(;)f(l)17 -b Fh(\))248 b FC(if)h Fo(F)426 b Fh(=)285 b Fo(R)7 b -FC(.)28552 39883 y(Figure)292 b(5)g(sho)-25 b(ws)291 -b(the)h(operational)g(semantics)g(for)f(traces.)h(The)g(inter)-20 -b(-)27224 40990 y(preter)333 b(formalized)h(there)f(e)-15 -b(x)g(ecutes)335 b(one)e(operation)h(at)g(a)f(time.)g(Its)g(state)27224 -42097 y(is)294 b(represented)i(by)f(an)h(en)-40 b(vironment)296 -b Fo(E)352 b FC(and)296 b(a)f(heap)h Fo(H)79 b FC(,)295 -b(which)h(may)f(be)27224 43204 y(changed)377 b(by)f(the)g(e)-15 -b(x)g(ecution)377 b(of)e(an)h(operation.)h(The)f(en)-40 -b(vironment)376 b(is)f(a)27224 44311 y(partial)239 b(function)h(from)f -(v)-25 b(ariables)240 b(to)f(locations)h(and)f(the)h(heap)g(is)f(a)g -(partial)27224 45418 y(function)326 b(from)f(locations)g(to)h(objects.) -f(Note)h(that)g(a)g(v)-25 b(ariable)326 b(can)g(ne)-25 -b(v)-15 b(er)27224 46525 y(be)361 b(null)f(in)h(the)g(en)-40 -b(vironment,)361 b(otherwise)g(the)g(trace)g(w)-10 b(ould)361 -b(ha)-20 b(v)-15 b(e)362 b(been)27224 47632 y(malformed.)351 -b(The)h(en)-40 b(vironment)352 b(could)g(not)f(directly)h(map)f(v)-25 -b(ariables)352 b(to)27224 48739 y(objects,)256 b(because)i(se)-25 -b(v)-15 b(eral)257 b(v)-25 b(ariables)256 b(can)h(point)g(to)f(the)h -Fx(same)f FC(object,)i(be-)27224 49846 y(cause)249 b(of)g(aliasing.) -28552 50953 y(W)-80 b(e)388 b(use)f(the)h(follo)-25 b(wing)387 -b(notation)h(for)e(updating)i(partial)g(functions:)27224 -52060 y Fo(E)57 b Fh([)p Fo(v)397 b Fg(7!)361 b Fo(l)17 -b Fh(])288 b FC(denotes)i(the)g(en)-40 b(vironment)290 -b(which)g(is)f(just)f(lik)-10 b(e)290 b Fo(E)57 b FC(,)289 -b(b)-20 b(ut)290 b(maps)27224 53167 y Fo(v)285 b FC(to)249 -b Fo(l)17 b FC(.)28552 54274 y(The)344 b Fr(new)p Black -Black 343 w FC(operation)h(creates)f(a)f(ne)-25 b(w)345 -b(object)f Fh(\()p Fo(T)85 b(;)172 b Fh(n)-28 b(ull)p -Fo(;)171 b Fh(n)-28 b(ull\))344 b FC(on)f(the)27224 55380 -y(heap)382 b(under)g(a)g(fresh)f(location)i Fo(l)398 -b FC(and)382 b(adds)g(the)g(result)f(v)-25 b(ariable)383 -b(to)e(the)27224 56487 y(en)-40 b(vironment,)250 b(mapping)g(it)f(to)g -(the)h(ne)-25 b(w)250 b(location)g Fo(l)17 b FC(.)28552 -57594 y(The)232 b Fr(get)p Black Black 231 w FC(operation)g(reads)f(a)h -(\002eld)g Fo(F)373 b FC(out)231 b(of)g(an)h(object,)g(and)g(adds)f -(the)27224 58701 y(result)272 b(v)-25 b(ariable)273 b(to)g(the)g(en)-40 -b(vironment,)274 b(mapping)f(it)f(to)h(the)g(read)g(location.)27224 -59808 y(The)249 b(heap)h(is)f(unchanged.)28552 60915 -y(The)362 b Fr(set)p Black Black 361 w FC(operation)g(changes)h -(\002eld)e Fo(F)504 b FC(of)361 b(an)h(object)g(stored)f(at)g(the)27224 -62022 y(location)308 b(that)h(v)-25 b(ariable)308 b Fo(v)344 -b FC(maps)308 b(to.)g(The)g(ne)-25 b(w)308 b(v)-25 b(alue)309 -b(of)f(the)g(\002eld)g(is)f(the)27224 63129 y(location)250 -b(in)f(v)-25 b(ariable)250 b Fo(u)p FC(.)f(The)h(en)-40 -b(vironment)250 b(is)e(unchanged.)28552 64236 y(The)302 -b Fr(guard)33068 64027 y(_)33601 64236 y(class)p Black -Black 304 w FC(operation)h(is)e(used)i(to)f(check)h(whether)g(the)g -(ob-)27224 65343 y(ject)329 b(stored)f(at)g(the)h(location)g(that)g(v) --25 b(ariable)329 b Fo(v)364 b FC(maps)328 b(to)h(is)f(of)g(type)g -Fo(T)142 b FC(.)329 b(If)27224 66450 y(that)364 b(is)f(the)h(case,)g -(then)g(e)-15 b(x)g(ecution)365 b(continues)f(without)g(changing)h -(heap)27224 67557 y(and)249 b(en)-40 b(vironment.)251 -b(Otherwise,)e(e)-15 b(x)g(ecution)251 b(is)e(stopped.)27224 -69738 y Fs(5.1)996 b(Optimizing)249 b(T)-74 b(races)27224 -71288 y FC(T)-80 b(o)397 b(optimize)i(the)f(simple)f(traces)h(of)f(the) -h(last)g(section,)g(we)g(use)f(online)27224 72395 y(partial)406 -b(e)-25 b(v)g(aluation.)408 b(The)f(partial)g(e)-25 b(v)g(aluator)407 -b(optimizes)g(one)g(operation)p Black Black eop end +34 w(trace)p Black 0.3 0.3 0.3 TeXcolorrgb(:)p Black +0.3 0.3 0.3 TeXcolorrgb Black 34 w Fj(p)2982 107 y Fi(0)p +0.3 0.3 0.3 TeXcolorrgb 3015 99 a Fk(,)p Black 0.3 0.3 0.3 +TeXcolorrgb Black 35 w Fj(p)3119 107 y Fi(1)p 0.3 0.3 0.3 +TeXcolorrgb Black 3868 168 a Fl(3)-1847 b Fp(guard)2217 +154 y(_)2252 168 y(class)n(\()p Fj(p)2494 176 y Fi(1)2527 +168 y Fp(,)34 b(BoxedInteger)o(\))3868 238 y Fl(5)-1847 +b Fj(i)2065 246 y Fi(2)2132 238 y Fp(=)35 b(get\()p Fj(p)2376 +246 y Fi(1)2408 238 y Fp(,)g(intval)o(\))3868 308 y Fl(6)-1847 +b Fp(guard)2217 294 y(_)2252 308 y(class)n(\()p Fj(p)2494 +316 y Fi(0)2527 308 y Fp(,)34 b(BoxedInteger)o(\))3868 +378 y Fl(8)-1847 b Fj(i)2065 386 y Fi(3)2132 378 y Fp(=)35 +b(get\()p Fj(p)2376 386 y Fi(0)2408 378 y Fp(,)g(intval)o(\))3868 +447 y Fl(9)-1847 b Fj(i)2065 455 y Fi(4)2132 447 y Fp(=)35 +b(int)2307 433 y(_)2342 447 y(add)o(\()p Fj(i)2504 455 +y Fi(2)2537 447 y Fp(,)g Fj(i)2630 455 y Fi(3)2663 447 +y Fp(\))3868 517 y Fl(26)-1868 b Fj(i)2065 525 y Fi(9)2132 +517 y Fp(=)35 b(int)2307 503 y(_)2342 517 y(add)o(\()p +Fj(i)2504 525 y Fi(4)2537 517 y Fp(,)g(-100\))3868 657 +y Fl(37)-1868 b Fp(guard)2217 643 y(_)2252 657 y(class)n(\()p +Fj(p)2494 665 y Fi(0)2527 657 y Fp(,)34 b(BoxedInteger)o(\))3868 +726 y Fl(39)-1868 b Fj(i)2065 734 y Fi(12)2161 726 y +Fp(=)34 b(get\()p Fj(p)2404 734 y Fi(0)2437 726 y Fp(,)g(intval\))3868 +796 y Fl(43)-1868 b Fj(i)2065 804 y Fi(14)2161 796 y +Fp(=)34 b(int)2335 782 y(_)2370 796 y(add\()o Fj(i)2532 +804 y Fi(12)2594 796 y Fp(,)g(-1\))3868 935 y Fl(52)-1868 +b Fj(i)2065 943 y Fi(17)2161 935 y Fp(=)34 b(int)2335 +921 y(_)2370 935 y(gt\()p Fj(i)2498 943 y Fi(14)2559 +935 y Fp(,)g(0\))3868 1005 y Fl(54)-1868 b Fp(guard)2217 +991 y(_)2252 1005 y(true)n(\()p Fj(i)2448 1013 y Fi(17)2509 +1005 y Fp(\))3868 1145 y Fl(44)g Fj(p)2076 1153 y Fi(15)2171 +1145 y Fp(=)35 b(new\(BoxedInteger)n(\))3868 1214 y Fl(46)-1868 +b Fp(set)o(\()p Fj(p)2215 1222 y Fi(15)2276 1214 y Fp(,)35 +b(intval)o(,)g Fj(i)2648 1222 y Fi(14)2709 1214 y Fp(\))3868 +1284 y Fl(27)-1868 b Fj(p)2076 1292 y Fi(10)2171 1284 +y Fp(=)35 b(new\(BoxedInteger)n(\))3868 1354 y Fl(29)-1868 +b Fp(set)o(\()p Fj(p)2215 1362 y Fi(10)2276 1354 y Fp(,)35 +b(intval)o(,)g Fj(i)2648 1362 y Fi(9)2681 1354 y Fp(\))3868 +1493 y Fl(55)-1868 b Fp(jump)o(\()p Fj(p)2250 1501 y +Fi(15)2311 1493 y Fp(,)35 b Fj(p)2415 1501 y Fi(10)2476 +1493 y Fp(\))p Black Black 2042 1567 1993 3 v 2243 1652 +a Fs(Figur)o(e)18 b(4.)37 b FC(Resulting)19 b(T)m(race)g(After)g +(Allocation)g(Remo)o(v)n(al)p Black 2141 1912 a(The)33 +b(v)n(alues)h(of)f(all)g(v)n(ariables)g(are)g(locations)h(\(i.e.,)d +(pointers\).)i(Lo-)2042 1995 y(cations)f(are)g(mapped)h(to)f(objects,)g +(which)g(are)g(represented)h(by)g(triples)2042 2078 y +Fh(\()p Fo(T)6 b(;)13 b(l)2180 2086 y Fn(1)2214 2078 +y Fo(;)h(l)2272 2086 y Fn(2)2306 2078 y Fh(\))j FC(of)h(a)g(type)g +Fo(T)11 b FC(,)17 b(and)h(tw)o(o)g(locations)g(that)g(represent)g(the)g +(\002elds)f(of)2042 2161 y(the)23 b(object.)h(When)g(a)f(ne)n(w)h +(object)g(is)f(created,)g(the)h(\002elds)f(are)g(initialized)2042 +2244 y(to)18 b(null,)g(b)o(ut)h(we)f(require)h(that)f(the)o(y)h(are)g +(initialized)f(to)g(a)h(real)f(location)h(be-)2042 2327 +y(fore)g(being)h(read,)f(otherwise)h(the)f(trace)g(is)g(malformed)g +(\(this)g(condition)h(is)2042 2410 y(guaranteed)g(by)g(ho)n(w)f(the)g +(traces)g(are)g(generated)h(in)f(PyPy\).)2141 2493 y(W)-6 +b(e)18 b(use)g(some)g(abbre)n(viations)h(when)f(dealing)h(with)e +(object)h(triples.)f(T)-6 b(o)2042 2576 y(read)21 b(the)g(type)h(of)f +(an)h(object,)f Fh(t)n(yp)r(e)o(\(\()p Fo(T)6 b(;)13 +b(l)3165 2584 y Fn(1)3200 2576 y Fo(;)g(l)3257 2584 y +Fn(2)3291 2576 y Fh(\)\))25 b(=)h Fo(T)31 b FC(is)21 +b(used.)g(Reading)2042 2659 y(a)k(\002eld)g Fo(F)36 b +FC(from)25 b(an)h(object)g(is)f(written)g Fh(\()p Fo(T)6 +b(;)13 b(l)3267 2667 y Fn(1)3301 2659 y Fo(;)g(l)3358 +2667 y Fn(2)3393 2659 y Fh(\))3423 2667 y FD(F)3499 2659 +y FC(which)26 b(either)f(is)g Fo(l)4000 2667 y Fn(1)2042 +2742 y FC(if)e Fo(F)41 b Fh(=)31 b Fo(L)23 b FC(or)h +Fo(l)2476 2750 y Fn(2)2534 2742 y FC(if)g Fo(F)41 b Fh(=)30 +b Fo(R)q FC(.)23 b(T)-6 b(o)24 b(set)f(\002eld)h Fo(F)34 +b FC(to)24 b(a)f(ne)n(w)h(location)h Fo(l)q FC(,)e(we)2042 +2825 y(use)h(the)g(notation)g Fh(\()p Fo(T)6 b(;)13 b(l)2686 +2833 y Fn(1)2721 2825 y Fo(;)g(l)2778 2833 y Fn(2)2813 +2825 y Fh(\)!)2864 2833 y FD(F)2915 2825 y Fo(l)q FC(,)23 +b(which)h(yields)g(a)g(ne)n(w)g(triple)f Fh(\()p Fo(T)6 +b(;)14 b(l)q(;)f(l)3970 2833 y Fn(2)4004 2825 y Fh(\))2042 +2908 y FC(if)18 b Fo(F)32 b Fh(=)21 b Fo(L)e FC(or)g(a)g(ne)n(w)g +(triple)g Fh(\()p Fo(T)6 b(;)13 b(l)2928 2916 y Fn(1)2962 +2908 y Fo(;)g(l)q Fh(\))19 b FC(if)f Fo(F)33 b Fh(=)21 +b Fo(R)q FC(.)2141 2991 y(Figure)h(5)h(sho)n(ws)f(the)h(operational)g +(semantics)f(for)g(traces.)g(The)g(inter)o(-)2042 3074 +y(preter)j(formalized)h(there)f(e)o(x)o(ecutes)h(one)g(operation)g(at)f +(a)g(time.)g(Its)f(state)2042 3157 y(is)e(represented)h(by)g(an)g(en)m +(vironment)h Fo(E)h FC(and)e(a)g(heap)g Fo(H)6 b FC(,)21 +b(which)i(may)g(be)2042 3240 y(changed)30 b(by)f(the)f(e)o(x)o(ecution) +i(of)e(an)h(operation.)g(The)f(en)m(vironment)i(is)e(a)2042 +3323 y(partial)18 b(function)h(from)f(v)n(ariables)h(to)f(locations)g +(and)h(the)g(heap)g(is)e(a)h(partial)2042 3406 y(function)25 +b(from)g(locations)g(to)g(objects.)f(Note)h(that)f(a)h(v)n(ariable)g +(can)g(ne)n(v)o(er)2042 3489 y(be)i(null)h(in)f(the)g(en)m(vironment,)i +(otherwise)e(the)g(trace)h(w)o(ould)g(ha)o(v)o(e)f(been)2042 +3572 y(malformed.)g(The)f(en)m(vironment)i(could)g(not)e(directly)h +(map)g(v)n(ariables)g(to)2042 3655 y(objects,)19 b(because)i(se)n(v)o +(eral)f(v)n(ariables)g(can)g(point)g(to)f(the)h Fx(same)f +FC(object,)h(be-)2042 3738 y(cause)f(of)g(aliasing.)2141 +3821 y(W)-6 b(e)29 b(use)h(the)f(follo)n(wing)h(notation)g(for)f +(updating)i(partial)d(functions:)2042 3904 y Fo(E)t Fh([)p +Fo(v)i Fg(7!)d Fo(l)q Fh(])22 b FC(denotes)h(the)f(en)m(vironment)h +(which)g(is)e(just)h(lik)o(e)g Fo(E)t FC(,)f(b)o(ut)h(maps)2042 +3987 y Fo(v)f FC(to)e Fo(l)q FC(.)2141 4071 y(The)26 +b Fr(new)p Black Black 26 w FC(operation)h(creates)f(a)g(ne)n(w)g +(object)h Fh(\()p Fo(T)6 b(;)13 b Fh(n)n(ull)p Fo(;)g +Fh(n)n(ull\))26 b FC(on)g(the)2042 4154 y(heap)j(under)h(a)f(fresh)g +(location)g Fo(l)h FC(and)g(adds)f(the)g(result)g(v)n(ariable)g(to)g +(the)2042 4237 y(en)m(vironment,)20 b(mapping)g(it)e(to)h(the)g(ne)n(w) +g(location)h Fo(l)q FC(.)2141 4320 y(The)e Fr(get)p Black +Black 17 w FC(operation)h(reads)f(a)f(\002eld)g Fo(F)29 +b FC(out)17 b(of)h(an)g(object,)f(and)i(adds)f(the)2042 +4403 y(result)i(v)n(ariable)h(to)g(the)g(en)m(vironment,)h(mapping)g +(it)e(to)g(the)h(read)g(location.)2042 4486 y(The)e(heap)g(is)g +(unchanged.)2141 4569 y(The)28 b Fr(set)p Black Black +27 w FC(operation)g(changes)h(\002eld)e Fo(F)38 b FC(of)27 +b(an)h(object)f(stored)h(at)f(the)2042 4652 y(location)d(that)f(v)n +(ariable)h Fo(v)i FC(maps)e(to.)e(The)i(ne)n(w)f(v)n(alue)h(of)g(the)f +(\002eld)g(is)g(the)2042 4735 y(location)c(in)g(v)n(ariable)g +Fo(u)p FC(.)g(The)g(en)m(vironment)h(is)f(unchanged.)2141 +4818 y(The)k Fr(guard)2480 4802 y(_)2520 4818 y(class)p +Black Black 23 w FC(operation)h(is)e(used)i(to)f(check)g(whether)h(the) +f(ob-)2042 4901 y(ject)h(stored)i(at)e(the)h(location)g(that)g(v)n +(ariable)g Fo(v)j FC(maps)d(to)g(is)g(of)f(type)i Fo(T)11 +b FC(.)24 b(If)2042 4984 y(that)j(is)g(the)h(case,)f(then)h(e)o(x)o +(ecution)h(continues)f(without)g(changing)h(heap)2042 +5067 y(and)19 b(en)m(vironment.)h(Otherwise,)f(e)o(x)o(ecution)h(is)e +(stopped.)2042 5230 y Fs(5.1)75 b(Optimizing)18 b(T)-6 +b(races)2042 5347 y FC(T)g(o)30 b(optimize)g(the)g(simple)g(traces)g +(of)g(the)g(last)g(section,)g(we)g(use)h(online)2042 +5430 y(partial)f(e)n(v)n(aluation.)i(The)f(partial)f(e)n(v)n(aluator)i +(optimizes)f(one)g(operation)p Black Black eop end %%Page: 6 6 -TeXDict begin 6 5 bop Black Black Black Black Black 325 -1393 a Fx(ne)-15 b(w)14699 748 y Fo(l)187 b Fh(fresh)p -4107 1165 23818 43 v 4107 2258 a Fo(v)321 b Fh(=)284 -b Ff(new)q Fh(\()p Fo(T)142 b Fh(\))p Fo(;)172 b(E)57 -b(;)171 b(H)12257 1687 y Fn(run)12041 2258 y Fh(=)-171 -b Fg(\))286 b Fo(E)227 b Fh([)q Fo(v)320 b Fg(7!)285 -b Fo(l)17 b Fh(])170 b Fo(;)h(H)250 b Fh([)p Fo(l)301 -b Fg(7!)284 b Fh(\()p Fo(T)85 b(;)172 b Fh(n)-28 b(ull)p -Fo(;)171 b Fh(n)-28 b(ull\))q(])32192 1393 y Fx(guar)-37 -b(d)39514 747 y Fh(t)-28 b(yp)28 b(e)q(\()p Fo(H)79 b -Fh(\()p Fo(E)57 b Fh(\()p Fo(v)36 b Fh(\)\))286 b(=)e -Fo(T)p 36023 1165 15310 43 v 36023 2258 a Ff(guard)p -FC(_)q Ff(class)p Fh(\()p Fo(v)36 b(;)171 b(T)142 b Fh(\))p -Fo(;)172 b(E)57 b(;)171 b(H)47423 1687 y Fn(run)47207 -2258 y Fh(=)-171 b Fg(\))285 b Fo(E)57 b(;)171 b(H)325 -6106 y Fx(g)-10 b(et)p 5307 5878 21419 43 v 5307 6971 -a Fo(u)284 b Fh(=)h Ff(get)p Fh(\()p Fo(v)36 b(;)172 -b(F)142 b Fh(\))p Fo(;)171 b(E)57 b(;)171 b(H)14558 6401 -y Fn(run)14342 6971 y Fh(=)-171 b Fg(\))285 b Fo(E)17257 -6166 y Fe(\002)17683 6971 y Fo(u)g Fg(7!)f Fo(H)250 b -Fh(\()p Fo(E)228 b Fh(\()q Fo(v)36 b Fh(\)\))24069 7275 -y FD(F)24747 6166 y Fe(\003)25344 6971 y Fo(;)171 b(H)39514 -5461 y Fh(t)-28 b(yp)28 b(e)q(\()p Fo(H)79 b Fh(\()p -Fo(E)57 b Fh(\()p Fo(v)36 b Fh(\)\))286 b Fg(6)p Fh(=)e -Fo(T)p 36095 5878 15166 43 v 36095 6971 a Ff(guard)p -FC(_)p Ff(class)q Fh(\()p Fo(v)36 b(;)171 b(T)142 b Fh(\))p -Fo(;)172 b(E)57 b(;)171 b(H)47495 6401 y Fn(run)47279 -6971 y Fh(=)-171 b Fg(\))285 b(?)p Fo(;)171 b Fg(?)325 -10202 y Fx(set)p 3377 9974 25279 43 v 3377 11067 a Ff(set)f -Fh(\()q Fo(v)36 b(;)171 b(F)28 b(;)171 b(u)p Fh(\))g -Fo(;)g(E)57 b(;)172 b(H)11945 10497 y Fn(run)11729 11067 -y Fh(=)-171 b Fg(\))285 b Fo(E)57 b(;)172 b(H)249 b Fh([)q -Fo(E)228 b Fh(\()p Fo(v)36 b Fh(\))285 b Fg(7!)f Fh(\()p -Fo(H)250 b Fh(\()p Fo(E)228 b Fh(\()p Fo(v)36 b Fh(\))q(\)!)25099 -11178 y FD(F)25777 11067 y Fo(E)57 b Fh(\()p Fo(u)p Fh(\)\))q(])-2000 -16080 y Fx(Object)250 b(Domains:)2170 17723 y Fo(u;)171 -b(v)235 b Fg(2)199 b Fo(V)2809 b Fh(v)-57 b(ariables)250 -b(in)g(trace)3008 18830 y Fo(T)341 b Fg(2)199 b Fd(T)2740 -b Fh(run)-28 b(time)249 b(t)-28 b(yp)28 b(es)2951 19937 -y Fo(F)341 b Fg(2)199 b(f)p Fo(L;)171 b(R)7 b Fg(g)448 -b Fh(\014elds)250 b(of)327 b(ob)57 b(jects)3425 21044 -y Fo(l)215 b Fg(2)199 b Fo(L)2709 b Fh(lo)28 b(cations)250 -b(on)f(heap)20296 16272 y Fx(Semantic)i(V)-111 b(alues:)20512 -17723 y Fo(E)256 b Fg(2)199 b Fo(V)512 b(*)285 b(L)13458 -b Fh(En)-28 b(vironmen)g(t)20396 18830 y Fo(H)278 b Fg(2)199 -b Fo(L)284 b(*)h Fd(T)228 b Fg(\002)g Fh(\()p Fo(L)f -Fg([)g(f)p Fh(n)-28 b(ull)p Fg(g)p Fh(\))228 b Fg(\002)g -Fh(\()p Fo(L)g Fg([)f(f)p Fh(n)-28 b(ull)p Fg(g)p Fh(\))448 -b(Heap)p -2000 21855 55791 37 v 14148 22984 a Fs(Figur)-18 -b(e)250 b(5.)498 b FC(The)249 b(Operational)i(Semantics)e(of)g -(Simpli\002ed)h(T)-35 b(races)p Black -2000 26299 a(of)373 -b(a)g(trace)h(at)f(a)h(time.)f(Ev)-15 b(ery)373 b(operation)h(in)f(the) -h(unoptimized)g(trace)g(is)-2000 27406 y(replaced)313 -b(by)f(a)g(list)f(of)g(operations)h(in)g(the)g(optimized)g(trace.)h -(This)e(list)g(is)-2000 28513 y(empty)371 b(if)f(the)g(operation)h(can) -g(be)g(optimized)g(a)-15 b(w)-10 b(ay)-65 b(.)372 b(The)e(optimization) --2000 29620 y(rules)481 b(can)g(be)h(seen)f(in)g(Figure)g(6.)g(Lists)f -(are)h(written)g(using)g(angular)-2000 30727 y(brack)-10 -b(ets)262 b Fo(<)309 b(:::)h(>)p FC(,)263 b(list)e(concatenation)j(is)d -(e)-15 b(xpressed)262 b(using)g(tw)-10 b(o)262 b(colons:)-2000 -31834 y Fo(l)-1694 31945 y Fn(1)-949 31834 y Fh(::)285 -b Fo(l)210 31945 y Fn(2)671 31834 y FC(.)-672 32941 y(The)358 -b(state)h(of)e(the)i(optimizer)f(is)g(stored)g(in)g(an)g(en)-40 -b(vironment)359 b Fo(E)415 b FC(and)-2000 34048 y(a)309 -b Fx(static)g(heap)g Fo(S)57 b FC(.)308 b(Each)i(step)e(of)h(the)g -(optimizer)h(tak)-10 b(es)309 b(an)g(operation,)h(an)-2000 -35155 y(en)-40 b(vironment)305 b(and)f(a)g(static)f(heap)i(and)f -(produces)g(a)g(list)e(of)i(operations,)g(a)-2000 36262 -y(ne)-25 b(w)250 b(en)-40 b(vironment)250 b(and)g(a)f(ne)-25 -b(w)250 b(static)f(heap.)-672 37369 y(The)290 b(en)-40 -b(vironment)290 b(is)f(a)g(partial)h(function)g(from)e(v)-25 -b(ariables)290 b(in)f(the)h(un-)-2000 38476 y(optimized)208 -b(trace)f Fo(V)435 b FC(to)207 b(v)-25 b(ariables)207 -b(in)g(the)g(optimized)h(trace)f Fo(V)19691 38053 y Fc(\003)20377 -38476 y FC(\(which)h(are)-2000 39583 y(themselv)-15 b(es)223 -b(written)g(with)f(a)8560 39160 y Fc(\003)9262 39583 -y FC(for)g(clarity\).)h(The)g(reason)f(for)g(introducing)-2000 -40690 y(ne)-25 b(w)345 b(v)-25 b(ariables)345 b(in)f(the)g(optimized)h -(trace)g(is)f(that)g(se)-25 b(v)-15 b(eral)345 b(v)-25 -b(ariables)345 b(that)-2000 41797 y(appear)364 b(in)f(the)g -(unoptimized)h(trace)g(can)f(turn)g(into)g(the)g(same)g(v)-25 -b(ariables)-2000 42904 y(in)308 b(the)g(optimized)g(trace.)h(The)f(en) --40 b(vironment)309 b(of)e(the)h(optimizer)g(serv)-15 -b(es)308 b(a)-2000 44011 y(function)267 b(similar)g(to)g(that)g(of)f -(the)i(en)-40 b(vironment)268 b(in)f(the)g(semantics:)g(to)g(e)-15 -b(x-)-2000 45118 y(press)248 b(sharing.)-672 46225 y(The)207 -b(static)g(heap)g(is)f(a)h(partial)g(function)g(from)f -Fo(V)16248 45801 y Fc(\003)16934 46225 y FC(into)g(the)h(set)g(of)f -(static)-2000 47332 y(objects,)331 b(which)h(are)f(triples)f(of)h(a)g -(type)g(and)h(tw)-10 b(o)331 b(elements)g(of)g Fo(V)21958 -46908 y Fc(\003)22438 47332 y FC(.)g(The)-2000 48439 -y(object)309 b(referenced)h(by)f(a)f(v)-25 b(ariable)310 -b Fo(v)11336 48015 y Fc(\003)12124 48439 y FC(is)e(static,)h(if)f -Fo(v)17208 48015 y Fc(\003)17996 48439 y FC(is)g(in)g(the)h(domain) --2000 49546 y(of)293 b(the)g(static)g(heap)h Fo(S)57 -b FC(.)293 b(The)g(object)h Fo(S)57 b Fh(\()p Fo(v)12612 -49122 y Fc(\003)13091 49546 y Fh(\))293 b FC(describes)g(what)h(is)f -(statically)-2000 50652 y(kno)-25 b(wn)361 b(about)f(the)g(object,)h -(i.e.,)f(its)f(type)h(and)g(its)f(\002elds.)h(The)g(\002elds)f(of)-2000 -51759 y(objects)243 b(in)g(the)g(static)g(heap)h(are)f(also)g(elements) -g(of)g Fo(V)17001 51336 y Fc(\003)17724 51759 y FC(\(or)f(null,)h(for)g -(short)-2000 52866 y(periods)249 b(of)g(time\).)-672 -53973 y(When)331 b(the)g(optimizer)g(sees)e(a)i Fr(new)p -Black Black 330 w FC(operation,)g(it)f(optimistically)g(re-)-2000 -55080 y(mo)-15 b(v)g(es)288 b(it)f(and)h(assumes)e(that)i(the)g -(resulting)f(object)h(can)g(stay)f(static.)h(The)-2000 -56187 y(optimization)309 b(for)g(all)f(further)g(operations)h(is)f -(split)g(into)h(tw)-10 b(o)309 b(cases.)f(One)-2000 57294 -y(case)233 b(is)f(for)g(when)i(the)f(in)-40 b(v)-20 b(olv)-15 -b(ed)233 b(v)-25 b(ariables)234 b(are)e(in)h(the)g(static)g(heap,)g -(which)-2000 58401 y(means)211 b(that)g(the)h(operation)f(can)h(be)f -(performed)g(at)g(optimization)h(time)f(and)-2000 59508 -y(can)353 b(be)h(remo)-15 b(v)g(ed)353 b(from)g(the)g(trace.)g(These)g -(rules)f(mirror)g(the)h(e)-15 b(x)g(ecution)-2000 60615 -y(semantics)253 b(closely)-65 b(.)253 b(The)g(other)g(case)g(is)f(for)g -(when)i(not)e(enough)i(is)e(kno)-25 b(wn)-2000 61722 -y(about)250 b(the)f(v)-25 b(ariables,)250 b(and)g(the)f(operation)h -(has)f(to)g(be)h(residualized.)-672 62829 y(If)336 b(the)h(ar)-18 -b(gument)337 b Fo(v)373 b FC(of)336 b(a)h Fr(get)p Black -Black 337 w FC(operation)g(is)f(mapped)h(to)g(something)-2000 -63936 y(in)298 b(the)g(static)g(heap,)h(the)f Fr(get)p -Black Black 298 w FC(can)g(be)h(performed)f(at)g(optimization)h(time.) --2000 65043 y(Otherwise,)250 b(the)f Fr(get)p Black Black -250 w FC(operation)h(needs)f(to)g(be)h(residualized.)-672 -66150 y(If)327 b(the)g(\002rst)f(ar)-18 b(gument)327 -b Fo(v)363 b FC(to)327 b(a)g Fr(set)p Black Black 327 -w FC(operation)h(is)e(mapped)i(to)f(some-)-2000 67257 -y(thing)391 b(in)g(the)g(static)f(heap,)i(then)f(the)g -Fr(set)p Black Black 391 w FC(can)g(be)g(performed)g(at)g(opti-)-2000 -68364 y(mization)313 b(time)f(\(which)g(updates)g(the)g(static)g -(heap\).)g(Otherwise)h(the)f Fr(set)p Black Black -2000 -69471 a FC(operation)235 b(needs)f(to)g(be)h(residualized.)g(This)e -(needs)i(to)f(be)g(done)h(carefully)-65 b(,)-2000 70578 -y(because)222 b(the)f(ne)-25 b(w)221 b(v)-25 b(alue)222 -b(for)e(the)h(\002eld,)g(from)f(the)h(v)-25 b(ariable)221 -b Fo(u)p FC(,)g(could)h(itself)-2000 71685 y(be)250 b(static,)f(in)g -(which)h(case)f(it)g(needs)h(to)f(be)h(lifted)f(\002rst.)p -Black Black 28552 26299 a(If)214 b(a)i Fr(guard)32753 -26090 y(_)33286 26299 y(class)p Black Black 216 w FC(is)e(performed)h -(on)g(a)g(v)-25 b(ariable)216 b(that)f(is)g(in)f(the)i(static)27224 -27406 y(heap,)250 b(the)g(type)h(check)g(can)f(be)g(performed)g(at)g -(optimization)h(time,)f(which)27224 28513 y(means)255 -b(the)f(operation)i(can)f(be)g(remo)-15 b(v)g(ed)255 -b(if)f(the)h(types)f(match.)i(If)d(the)i(type)27224 29620 -y(check)345 b(f)-10 b(ails)343 b(statically)h(or)g(if)g(the)g(object)g -(is)g(not)g(in)g(the)g(static)g(heap,)g(the)27224 30727 -y Fr(guard)29889 30518 y(_)30422 30727 y(class)p Black -Black 333 w FC(is)332 b(residualized.)i(This)d(also)i(needs)f(to)h -(lift)f(the)h(v)-25 b(ariable)27224 31834 y(on)249 b(which)h(the)f -Fr(guard)35284 31625 y(_)35817 31834 y(class)p Black -Black 251 w FC(is)f(performed.)28552 32941 y(Lifting)300 -b(tak)-10 b(es)300 b(a)h(v)-25 b(ariable)301 b(and)g(turns)f(it)g(into) -g(a)h(dynamic)g(v)-25 b(ariable.)301 b(If)27224 34048 -y(the)316 b(v)-25 b(ariable)317 b(is)f(already)h(dynamic,)g(nothing)g -(needs)f(to)g(be)h(done.)g(If)e(it)h(is)27224 35155 y(in)346 -b(the)h(static)g(heap,)g(operations)g(are)g(emitted)g(that)f(construct) -h(an)g(object)27224 36262 y(with)336 b(the)f(shape)h(described)g -(there,)g(and)g(the)g(v)-25 b(ariable)337 b(is)e(remo)-15 -b(v)g(ed)336 b(from)27224 37369 y(the)249 b(static)g(heap.)28552 -38476 y(Lifting)328 b(a)g(static)g(object)h(needs)f(to)g(recursi)-25 -b(v)-15 b(ely)329 b(lift)e(its)g(\002elds.)h(Some)27224 -39583 y(care)417 b(needs)g(to)g(be)g(tak)-10 b(en)418 -b(when)f(lifting)f(a)h(static)g(object,)h(because)f(the)27224 -40690 y(structures)456 b(described)h(by)g(the)g(static)g(heap)h(can)f -(be)g(c)-15 b(yclic.)458 b(T)-80 b(o)457 b(mak)-10 b(e)27224 -41797 y(sure)286 b(that)g(the)h(same)f(static)g(object)h(is)f(not)g -(lifted)g(twice,)h(the)g Fr(liftfield)p Black Black 27224 -42904 a FC(operation)271 b(remo)-15 b(v)g(es)271 b(it)g(from)f(the)h -(static)g(heap)h Fx(befor)-37 b(e)271 b FC(recursi)-25 -b(v)-15 b(ely)271 b(lifting)27224 44011 y(its)248 b(\002elds.)28552 -45118 y(As)h(an)h(e)-15 b(xample)250 b(for)f(lifting,)g(consider)g(the) -g(static)h(heap)32088 46777 y Fg(f)p Fo(v)33133 46299 -y Fc(\003)33897 46777 y Fg(7!)284 b Fh(\()p Fo(T)36200 -46888 y Fn(1)36662 46777 y Fo(;)171 b(w)37878 46299 y -Fc(\003)38358 46777 y Fo(;)g(v)39346 46299 y Fc(\003)39825 -46777 y Fh(\))p Fo(;)h(w)41440 46299 y Fc(\003)42203 -46777 y Fg(7!)285 b Fh(\()p Fo(T)44507 46888 y Fn(2)44969 -46777 y Fo(;)171 b(u)46013 46299 y Fc(\003)46493 46777 -y Fo(;)g(u)47537 46299 y Fc(\003)48017 46777 y Fh(\))p -Fg(g)27224 48437 y FC(which)448 b(contains)g(tw)-10 b(o)448 -b(static)g(objects.)g(If)f Fo(v)43520 48014 y Fc(\003)44447 -48437 y FC(needs)h(to)g(be)g(lifted,)f(the)27224 49544 -y(follo)-25 b(wing)249 b(residual)h(operations)f(are)g(produced:)29438 -51134 y Fj(v)29909 50800 y Fb(\003)30835 51134 y Fp(=)464 -b(n)o(e)o(w)o(\()p Fj(T)34149 51245 y Fi(1)34580 51134 -y Fp(\))29438 52064 y Fj(w)30099 51730 y Fb(\003)31026 -52064 y Fp(=)f(n)o(e)o(w)o(\()p Fj(T)34339 52175 y Fi(2)34770 -52064 y Fp(\))29437 52994 y(s)o(e)o(t)o(\()o Fj(w)31958 -52660 y Fb(\003)32420 52994 y Fp(,)i Fj(L)p Fp(,)f Fj(u)35411 -52660 y Fb(\003)35874 52994 y Fp(\))29437 53924 y(s)o(e)o(t)o(\()o -Fj(w)31958 53590 y Fb(\003)32420 53924 y Fp(,)h Fj(R)s -Fp(,)g Fj(u)35475 53590 y Fb(\003)35937 53924 y Fp(\))29437 -54854 y(s)o(e)o(t)o(\()o Fj(v)31768 54520 y Fb(\003)32230 -54854 y Fp(,)g Fj(L)o Fp(,)g Fj(w)35359 54520 y Fb(\003)35821 -54854 y Fp(\))29437 55784 y(s)o(e)o(t)o(\()o Fj(v)31768 -55449 y Fb(\003)32230 55784 y Fp(,)g Fj(R)r Fp(,)g Fj(v)35232 -55449 y Fb(\003)35694 55784 y Fp(\))p Black Black 28552 -57551 a FC(After)330 b(the)g(lifting)g(the)g(static)g(heap)h(is)f(the)g -(empty)h(set,)e(because)i(both)27224 58658 y(static)223 -b(objects)g(were)h(lifted.)f(If)g(we)g(had)h(lifted)f -Fo(w)44349 58235 y Fc(\003)45052 58658 y FC(instead)g(of)g -Fo(v)49683 58235 y Fc(\003)50163 58658 y FC(,)g(then)g(the)27224 -59765 y(follo)-25 b(wing)249 b(operations)h(w)-10 b(ould)250 -b(ha)-20 b(v)-15 b(e)250 b(been)g(produced:)29438 61356 -y Fj(w)30099 61022 y Fb(\003)31026 61356 y Fp(=)463 b(n)o(e)o(w)o(\()p -Fj(T)34339 61467 y Fi(2)34770 61356 y Fp(\))29437 62285 -y(s)o(e)o(t)o(\()o Fj(w)31958 61951 y Fb(\003)32420 62285 -y Fp(,)i Fj(L)p Fp(,)f Fj(u)35411 61951 y Fb(\003)35874 -62285 y Fp(\))29437 63215 y(s)o(e)o(t)o(\()o Fj(w)31958 -62881 y Fb(\003)32420 63215 y Fp(,)h Fj(R)s Fp(,)g Fj(u)35475 -62881 y Fb(\003)35937 63215 y Fp(\))p Black Black 28552 -64983 a FC(In)249 b(this)g(case,)g(the)h(static)f(heap)h(afterw)-10 -b(ards)249 b(w)-10 b(ould)250 b(be:)36183 66642 y Fg(f)p -Fo(v)37228 66164 y Fc(\003)37993 66642 y Fg(7!)284 b -Fh(\()p Fo(T)40296 66753 y Fn(1)40758 66642 y Fo(;)171 -b(w)41974 66164 y Fc(\003)42453 66642 y Fo(;)g(v)43441 -66164 y Fc(\003)43921 66642 y Fh(\))p Fg(g)27224 68632 -y Fs(5.2)996 b(Analysis)249 b(of)g(the)g(Algorithm)27224 -70181 y FC(While)275 b(we)f(do)h(not)f(of)-25 b(fer)274 -b(a)h(formal)f(proof)g(of)g(it,)g(it)g(can)h(ar)-18 b(gue)275 -b(informally)27224 71288 y(that)309 b(the)g(algorithm)g(presented)g -(abo)-15 b(v)g(e)310 b(is)e(sound:)h(it)f(w)-10 b(orks)309 -b(by)g(delaying)27224 72395 y(\(and)323 b(often)g(completely)h(remo)-15 -b(ving\))324 b(some)f(operations.)g(The)g(algorithm)p -Black Black eop end +TeXDict begin 6 5 bop Black Black Black Black Black 24 +104 a Fx(ne)o(w)1102 56 y Fo(l)14 b Fh(fresh)p 308 87 +1787 4 v 308 169 a Fo(v)24 b Fh(=)d Ff(new)q Fh(\()p +Fo(T)11 b Fh(\))p Fo(;)h(E)t(;)h(H)919 127 y Fn(run)903 +169 y Fh(=)-13 b Fg(\))21 b Fo(E)c Fh([)p Fo(v)24 b Fg(7!)e +Fo(l)q Fh(])13 b Fo(;)g(H)18 b Fh([)q Fo(l)k Fg(7!)f +Fh(\()p Fo(T)6 b(;)13 b Fh(n)n(ull)p Fo(;)g Fh(n)n(ull\)])2414 +104 y Fx(guar)m(d)2964 56 y Fh(t)n(yp)r(e)o(\()p Fo(H)6 +b Fh(\()p Fo(E)t Fh(\()p Fo(v)s Fh(\)\))20 b(=)h Fo(T)p +2702 87 1149 4 v 2702 169 a Ff(guard)p FC(_)q Ff(class)q +Fh(\()p Fo(v)s(;)13 b(T)e Fh(\))p Fo(;)i(E)t(;)g(H)3557 +127 y Fn(run)3541 169 y Fh(=)-13 b Fg(\))21 b Fo(E)t(;)12 +b(H)24 458 y Fx(g)o(et)p 398 441 1607 4 v 398 523 a Fo(u)22 +b Fh(=)f Ff(get)p Fh(\()p Fo(v)s(;)13 b(F)e Fh(\))p Fo(;)i(E)t(;)g(H) +1092 480 y Fn(run)1076 523 y Fh(=)-13 b Fg(\))21 b Fo(E)1294 +462 y Fe(\002)1326 523 y Fo(u)h Fg(7!)f Fo(H)d Fh(\()p +Fo(E)f Fh(\()o Fo(v)s Fh(\)\))1805 546 y FD(F)1856 462 +y Fe(\003)1901 523 y Fo(;)c(H)2964 410 y Fh(t)n(yp)r(e)o(\()p +Fo(H)6 b Fh(\()p Fo(E)t Fh(\()p Fo(v)s Fh(\)\))20 b Fg(6)p +Fh(=)h Fo(T)p 2707 441 1138 4 v 2707 523 a Ff(guard)p +FC(_)r Ff(class)q Fh(\()p Fo(v)s(;)13 b(T)e Fh(\))p Fo(;)h(E)t(;)h(H) +3562 480 y Fn(run)3546 523 y Fh(=)-13 b Fg(\))21 b(?)p +Fo(;)13 b Fg(?)24 765 y Fx(set)p 253 748 1896 4 v 253 +830 a Ff(set)h Fh(\()p Fo(v)s(;)f(F)r(;)g(u)p Fh(\))g +Fo(;)g(E)t(;)g(H)896 787 y Fn(run)880 830 y Fh(=)-13 +b Fg(\))21 b Fo(E)t(;)13 b(H)18 b Fh([)p Fo(E)f Fh(\()p +Fo(v)s Fh(\))k Fg(7!)g Fh(\()p Fo(H)d Fh(\()p Fo(E)f +Fh(\()o Fo(v)s Fh(\)\)!)1882 838 y FD(F)1933 830 y Fo(E)t +Fh(\()p Fo(u)p Fh(\)\)])-150 1206 y Fx(Object)i(Domains:)163 +1329 y Fo(u;)13 b(v)18 b Fg(2)d Fo(V)211 b Fh(v)l(ariables)19 +b(in)g(trace)226 1412 y Fo(T)25 b Fg(2)15 b Fd(T)206 +b Fh(run)n(time)18 b(t)n(yp)r(es)221 1495 y Fo(F)26 b +Fg(2)15 b(f)q Fo(L;)e(R)q Fg(g)34 b Fh(\014elds)18 b(of)26 +b(ob)t(jects)257 1578 y Fo(l)16 b Fg(2)f Fo(L)204 b Fh(lo)r(cations)20 +b(on)f(heap)1522 1220 y Fx(Semantic)h(V)-8 b(alues:)1538 +1329 y Fo(E)19 b Fg(2)c Fo(V)38 b(*)22 b(L)1009 b Fh(En)n(vironmen)n(t) +1530 1412 y Fo(H)20 b Fg(2)15 b Fo(L)22 b(*)f Fd(T)c +Fg(\002)g Fh(\()p Fo(L)g Fg([)g(f)p Fh(n)n(ull)q Fg(g)p +Fh(\))g Fg(\002)g Fh(\()p Fo(L)g Fg([)g(f)p Fh(n)n(ull)p +Fg(g)p Fh(\))34 b(Heap)p -150 1639 4185 3 v 1061 1724 +a Fs(Figur)o(e)18 b(5.)38 b FC(The)18 b(Operational)i(Semantics)f(of)g +(Simpli\002ed)f(T)m(races)p Black -150 1972 a(of)28 b(a)g(trace)h(at)e +(a)i(time.)e(Ev)o(ery)h(operation)i(in)e(the)g(unoptimized)i(trace)e +(is)-150 2055 y(replaced)c(by)h(a)e(list)g(of)g(operations)i(in)f(the)f +(optimized)h(trace.)g(This)f(list)g(is)-150 2138 y(empty)29 +b(if)e(the)h(operation)h(can)f(be)h(optimized)f(a)o(w)o(ay)-5 +b(.)29 b(The)f(optimization)-150 2222 y(rules)36 b(can)h(be)g(seen)g +(in)f(Figure)g(6.)g(Lists)f(are)i(written)e(using)i(angular)-150 +2305 y(brack)o(ets)21 b Fo(<)i(:::)h(>)p FC(,)19 b(list)g +(concatenation)j(is)d(e)o(xpressed)i(using)g(tw)o(o)f(colons:)-150 +2388 y Fo(l)-127 2396 y Fn(1)-71 2388 y Fh(::)i Fo(l)16 +2396 y Fn(2)50 2388 y FC(.)-50 2471 y(The)27 b(state)f(of)i(the)f +(optimizer)g(is)g(stored)g(in)g(an)g(en)m(vironment)i +Fo(E)i FC(and)-150 2554 y(a)23 b Fx(static)g(heap)i Fo(S)t +FC(.)e(Each)g(step)h(of)f(the)h(optimizer)f(tak)o(es)h(an)g(operation,) +g(an)-150 2637 y(en)m(vironment)g(and)g(a)f(static)g(heap)g(and)h +(produces)h(a)d(list)h(of)g(operations,)g(a)-150 2720 +y(ne)n(w)c(en)m(vironment)i(and)e(a)g(ne)n(w)g(static)g(heap.)-50 +2803 y(The)j(en)m(vironment)h(is)e(a)h(partial)g(function)h(from)f(v)n +(ariables)g(in)g(the)g(un-)-150 2886 y(optimized)16 b(trace)g +Fo(V)32 b FC(to)16 b(v)n(ariables)g(in)g(the)g(optimized)g(trace)g +Fo(V)1477 2854 y Fc(\003)1528 2886 y FC(\(which)g(are)-150 +2969 y(themselv)o(es)i(written)e(with)g(a)642 2937 y +Fc(\003)695 2969 y FC(for)g(clarity\).)h(The)g(reason)g(for)g +(introducing)-150 3052 y(ne)n(w)26 b(v)n(ariables)h(in)f(the)g +(optimized)h(trace)f(is)f(that)h(se)n(v)o(eral)h(v)n(ariables)f(that) +-150 3135 y(appear)i(in)g(the)f(unoptimized)i(trace)e(can)h(turn)g +(into)f(the)h(same)g(v)n(ariables)-150 3218 y(in)23 b(the)h(optimized)g +(trace.)e(The)i(en)m(vironment)h(of)e(the)g(optimizer)h(serv)o(es)f(a) +-150 3301 y(function)e(similar)f(to)g(that)g(of)g(the)g(en)m(vironment) +i(in)e(the)h(semantics:)f(to)g(e)o(x-)-150 3384 y(press)f(sharing.)-50 +3467 y(The)c(static)h(heap)g(is)f(a)h(partial)f(function)i(from)f +Fo(V)1219 3435 y Fc(\003)1270 3467 y FC(into)g(the)g(set)f(of)h(static) +-150 3550 y(objects,)25 b(which)h(are)f(triples)f(of)h(a)g(type)h(and)f +(tw)o(o)h(elements)f(of)g Fo(V)1647 3518 y Fc(\003)1683 +3550 y FC(.)f(The)-150 3633 y(object)g(referenced)g(by)g(a)f(v)n +(ariable)h Fo(v)850 3601 y Fc(\003)909 3633 y FC(is)f(static,)g(if)g +Fo(v)1291 3601 y Fc(\003)1350 3633 y FC(is)g(in)g(the)g(domain)-150 +3716 y(of)f(the)h(static)e(heap)i Fo(S)t FC(.)f(The)g(object)h +Fo(S)t Fh(\()p Fo(v)946 3684 y Fc(\003)982 3716 y Fh(\))f +FC(describes)h(what)f(is)g(statically)-150 3799 y(kno)n(wn)28 +b(about)g(the)g(object,)f(i.e.,)f(its)h(type)g(and)h(its)f(\002elds.)f +(The)h(\002elds)g(of)-150 3882 y(objects)19 b(in)f(the)h(static)f(heap) +h(are)f(also)h(elements)g(of)f Fo(V)1275 3850 y Fc(\003)1329 +3882 y FC(\(or)h(null,)f(for)g(short)-150 3965 y(periods)i(of)f +(time\).)-50 4048 y(When)25 b(the)g(optimizer)g(sees)g(a)g +Fr(new)p Black Black 25 w FC(operation,)h(it)e(optimistically)h(re-) +-150 4131 y(mo)o(v)o(es)d(it)f(and)i(assumes)g(that)e(the)h(resulting)g +(object)g(can)g(stay)g(static.)f(The)-150 4214 y(optimization)j(for)f +(all)g(further)g(operations)i(is)e(split)g(into)g(tw)o(o)h(cases.)f +(One)-150 4297 y(case)18 b(is)f(for)h(when)g(the)g(in)m(v)o(olv)o(ed)g +(v)n(ariables)h(are)e(in)h(the)g(static)f(heap,)h(which)-150 +4380 y(means)f(that)f(the)g(operation)h(can)f(be)h(performed)g(at)e +(optimization)i(time)f(and)-150 4463 y(can)27 b(be)g(remo)o(v)o(ed)h +(from)f(the)f(trace.)h(These)f(rules)h(mirror)g(the)f(e)o(x)o(ecution) +-150 4546 y(semantics)20 b(closely)-5 b(.)19 b(The)g(other)h(case)f(is) +g(for)g(when)h(not)f(enough)i(is)e(kno)n(wn)-150 4629 +y(about)h(the)f(v)n(ariables,)g(and)g(the)g(operation)h(has)g(to)e(be)i +(residualized.)-50 4712 y(If)25 b(the)g(ar)o(gument)h +Fo(v)i FC(of)e(a)f Fr(get)p Black Black 26 w FC(operation)h(is)f +(mapped)i(to)e(something)-150 4795 y(in)d(the)h(static)f(heap,)h(the)g +Fr(get)p Black Black 22 w FC(can)g(be)g(performed)h(at)e(optimization)h +(time.)-150 4878 y(Otherwise,)18 b(the)h Fr(get)p Black +Black 19 w FC(operation)h(needs)g(to)f(be)g(residualized.)-50 +4961 y(If)24 b(the)h(\002rst)e(ar)o(gument)j Fo(v)h FC(to)e(a)g +Fr(set)p Black Black 24 w FC(operation)h(is)e(mapped)i(to)f(some-)-150 +5044 y(thing)30 b(in)f(the)h(static)f(heap,)h(then)g(the)g +Fr(set)p Black Black 29 w FC(can)g(be)g(performed)g(at)f(opti-)-150 +5127 y(mization)24 b(time)f(\(which)h(updates)h(the)e(static)g(heap\).) +h(Otherwise)g(the)g Fr(set)p Black Black -150 5210 a +FC(operation)19 b(needs)f(to)g(be)g(residualized.)g(This)g(needs)h(to)e +(be)h(done)h(carefully)-5 b(,)-150 5293 y(because)18 +b(the)f(ne)n(w)g(v)n(alue)h(for)e(the)h(\002eld,)f(from)h(the)g(v)n +(ariable)g Fo(u)p FC(,)g(could)g(itself)-150 5376 y(be)i(static,)f(in)h +(which)g(case)h(it)e(needs)i(to)f(be)g(lifted)f(\002rst.)p +Black Black 2141 1972 a(If)e(a)h Fr(guard)2457 1957 y(_)2497 +1972 y(class)p Black Black 15 w FC(is)f(performed)i(on)f(a)f(v)n +(ariable)h(that)f(is)g(in)g(the)h(static)2042 2055 y(heap,)i(the)g +(type)h(check)g(can)f(be)g(performed)h(at)f(optimization)g(time,)g +(which)2042 2138 y(means)h(the)f(operation)h(can)g(be)g(remo)o(v)o(ed)g +(if)f(the)g(types)h(match.)f(If)g(the)h(type)2042 2222 +y(check)27 b(f)o(ails)f(statically)f(or)h(if)g(the)g(object)g(is)g(not) +g(in)g(the)g(static)f(heap,)i(the)2042 2305 y Fr(guard)2242 +2289 y(_)2282 2305 y(class)p Black Black 25 w FC(is)d(residualized.)i +(This)f(also)g(needs)h(to)f(lift)f(the)i(v)n(ariable)2042 +2388 y(on)19 b(which)g(the)g Fr(guard)2646 2372 y(_)2686 +2388 y(class)p Black Black 19 w FC(is)f(performed.)2141 +2471 y(Lifting)k(tak)o(es)i(a)e(v)n(ariable)i(and)f(turns)g(it)f(into)h +(a)g(dynamic)h(v)n(ariable.)f(If)2042 2554 y(the)h(v)n(ariable)g(is)g +(already)g(dynamic,)h(nothing)g(needs)g(to)f(be)g(done.)h(If)e(it)g(is) +2042 2637 y(in)j(the)g(static)g(heap,)h(operations)g(are)f(emitted)g +(that)g(construct)h(an)g(object)2042 2720 y(with)e(the)g(shape)i +(described)f(there,)f(and)h(the)g(v)n(ariable)g(is)f(remo)o(v)o(ed)h +(from)2042 2803 y(the)19 b(static)f(heap.)2141 2886 y(Lifting)25 +b(a)f(static)h(object)g(needs)h(to)e(recursi)n(v)o(ely)i(lift)e(its)g +(\002elds.)g(Some)2042 2969 y(care)31 b(needs)i(to)e(be)h(tak)o(en)g +(when)g(lifting)f(a)h(static)f(object,)g(because)i(the)2042 +3052 y(structures)i(described)g(by)g(the)g(static)f(heap)h(can)g(be)g +(c)o(yclic.)f(T)-6 b(o)34 b(mak)o(e)2042 3135 y(sure)22 +b(that)f(the)h(same)g(static)f(object)h(is)g(not)g(lifted)f(twice,)g +(the)g Fr(liftfield)p Black Black 2042 3218 a FC(operation)g(remo)o(v)o +(es)g(it)f(from)h(the)f(static)h(heap)g Fx(befor)m(e)g +FC(recursi)n(v)o(ely)h(lifting)2042 3301 y(its)c(\002elds.)2141 +3384 y(As)h(an)g(e)o(xample)h(for)f(lifting,)f(consider)i(the)f(static) +f(heap)2407 3508 y Fg(f)p Fo(v)2485 3472 y Fc(\003)2542 +3508 y Fg(7!)j Fh(\()p Fo(T)2715 3516 y Fn(1)2750 3508 +y Fo(;)13 b(w)2841 3472 y Fc(\003)2877 3508 y Fo(;)g(v)2951 +3472 y Fc(\003)2987 3508 y Fh(\))p Fo(;)g(w)3108 3472 +y Fc(\003)3165 3508 y Fg(7!)21 b Fh(\()p Fo(T)3338 3516 +y Fn(2)3373 3508 y Fo(;)13 b(u)3451 3472 y Fc(\003)3487 +3508 y Fo(;)g(u)3565 3472 y Fc(\003)3601 3508 y Fh(\))p +Fg(g)2042 3633 y FC(which)34 b(contains)h(tw)o(o)e(static)h(objects.)g +(If)f Fo(v)3264 3601 y Fc(\003)3334 3633 y FC(needs)h(to)g(be)g +(lifted,)f(the)2042 3716 y(follo)n(wing)19 b(residual)g(operations)h +(are)f(produced:)2208 3835 y Fj(v)2243 3810 y Fb(\003)2313 +3835 y Fp(=)34 b(new\()p Fj(T)2561 3843 y Fi(1)2593 3835 +y Fp(\))2208 3905 y Fj(w)2258 3880 y Fb(\003)2327 3905 +y Fp(=)h(new)o(\()p Fj(T)2575 3913 y Fi(2)2608 3905 y +Fp(\))2208 3975 y(set)o(\()p Fj(w)2397 3949 y Fb(\003)2432 +3975 y Fp(,)f Fj(L)p Fp(,)h Fj(u)2656 3949 y Fb(\003)2691 +3975 y Fp(\))2208 4044 y(set)o(\()p Fj(w)2397 4019 y +Fb(\003)2432 4044 y Fp(,)f Fj(R)q Fp(,)g Fj(u)2660 4019 +y Fb(\003)2695 4044 y Fp(\))2208 4114 y(set)o(\()p Fj(v)2382 +4089 y Fb(\003)2417 4114 y Fp(,)h Fj(L)p Fp(,)f Fj(w)2652 +4089 y Fb(\003)2687 4114 y Fp(\))2208 4184 y(set)o(\()p +Fj(v)2382 4159 y Fb(\003)2417 4184 y Fp(,)h Fj(R)p Fp(,)g +Fj(v)2642 4159 y Fb(\003)2677 4184 y Fp(\))p Black Black +2141 4316 a FC(After)25 b(the)g(lifting)f(the)h(static)g(heap)h(is)e +(the)h(empty)h(set,)e(because)i(both)2042 4399 y(static)16 +b(objects)i(were)f(lifted.)f(If)g(we)h(had)h(lifted)e +Fo(w)3326 4368 y Fc(\003)3379 4399 y FC(instead)h(of)g +Fo(v)3726 4368 y Fc(\003)3762 4399 y FC(,)g(then)g(the)2042 +4482 y(follo)n(wing)i(operations)h(w)o(ould)g(ha)o(v)o(e)f(been)h +(produced:)2208 4602 y Fj(w)2258 4577 y Fb(\003)2327 +4602 y Fp(=)35 b(new)o(\()p Fj(T)2575 4610 y Fi(2)2608 +4602 y Fp(\))2208 4671 y(set)o(\()p Fj(w)2397 4646 y +Fb(\003)2432 4671 y Fp(,)f Fj(L)p Fp(,)h Fj(u)2656 4646 +y Fb(\003)2691 4671 y Fp(\))2208 4741 y(set)o(\()p Fj(w)2397 +4716 y Fb(\003)2432 4741 y Fp(,)f Fj(R)q Fp(,)g Fj(u)2660 +4716 y Fb(\003)2695 4741 y Fp(\))p Black Black 2141 4874 +a FC(In)19 b(this)g(case,)g(the)g(static)f(heap)i(afterw)o(ards)f(w)o +(ould)h(be:)2714 4998 y Fg(f)p Fo(v)2792 4962 y Fc(\003)2849 +4998 y Fg(7!)i Fh(\()p Fo(T)3023 5006 y Fn(1)3057 4998 +y Fo(;)13 b(w)3148 4962 y Fc(\003)3184 4998 y Fo(;)g(v)3258 +4962 y Fc(\003)3294 4998 y Fh(\))p Fg(g)2042 5147 y Fs(5.2)75 +b(Analysis)18 b(of)h(the)g(Algorithm)2042 5264 y FC(While)h(we)h(do)g +(not)g(of)n(fer)g(a)g(formal)f(proof)i(of)f(it,)e(it)h(can)i(ar)o(gue)f +(informally)2042 5347 y(that)i(the)g(algorithm)h(presented)h(abo)o(v)o +(e)f(is)f(sound:)h(it)f(w)o(orks)h(by)g(delaying)2042 +5430 y(\(and)h(often)g(completely)g(remo)o(ving\))g(some)g(operations.) +g(The)g(algorithm)p Black Black eop end %%Page: 7 7 -TeXDict begin 7 6 bop Black Black Black Black Black 7379 -8835 a Fx(ne)-15 b(w)26088 8191 y Fo(v)26621 7767 y Fc(\003)27271 -8191 y Fh(fresh)p 14598 8607 26305 43 v 14598 9863 a -Fo(v)321 b Fh(=)284 b Ff(new)q Fh(\()p Fo(T)142 b Fh(\))p -Fo(;)172 b(E)57 b(;)171 b(S)22526 9252 y Fn(opt)22288 -9863 y Fh(=)-171 b Fg(\))286 b(h)170 b(i)h Fo(;)g(E)228 -b Fh([)q Fo(v)320 b Fg(7!)284 b Fo(v)29739 9577 y Fc(\003)30219 -9863 y Fh(])171 b Fo(;)g(S)227 b Fh([)q Fo(v)32800 9577 -y Fc(\003)33564 9863 y Fg(7!)284 b Fh(\()p Fo(T)85 b(;)172 -b Fh(n)-28 b(ull)p Fo(;)171 b Fh(n)-28 b(ull\))q(])7379 -13548 y Fx(g)-10 b(et)24348 12903 y Fo(E)57 b Fh(\()p -Fo(v)36 b Fh(\))285 b Fg(2)f Fh(dom\()p Fo(S)57 b Fh(\))p -16923 13320 21654 43 v 16923 14576 a Fo(u)285 b Fh(=)g -Ff(get)p Fh(\()p Fo(v)36 b(;)171 b(F)142 b Fh(\))p Fo(;)172 -b(E)57 b(;)171 b(S)25952 13966 y Fn(opt)25714 14576 y -Fh(=)-171 b Fg(\))285 b(h)171 b(i)g Fo(;)g(E)228 b Fh([)q -Fo(u)284 b Fg(7!)h Fo(S)57 b Fh(\()p Fo(E)g Fh(\()p Fo(v)36 -b Fh(\)\))36307 14687 y FD(F)36985 14576 y Fh(])171 b -Fo(;)g(S)22174 17616 y(E)57 b Fh(\()p Fo(v)36 b Fh(\))399 -b Fo(=)-626 b Fg(2)284 b Fh(dom\()p Fo(S)57 b Fh(\))p -Fo(;)683 b(u)30534 17193 y Fc(\003)31185 17616 y Fh(fresh)p -14527 18034 26447 43 v 14527 19290 a Fo(u)285 b Fh(=)f -Ff(get)p Fh(\()p Fo(v)36 b(;)172 b(F)142 b Fh(\))p Fo(;)171 -b(E)57 b(;)171 b(S)23556 18679 y Fn(opt)23318 19290 y -Fh(=)-171 b Fg(\))285 b(h)p Fo(u)26239 19004 y Fc(\003)27004 -19290 y Fh(=)f Ff(get)q Fh(\()p Fo(E)57 b Fh(\()p Fo(v)36 -b Fh(\))p Fo(;)172 b(F)142 b Fh(\))p Fg(i)171 b Fo(;)g(E)228 -b Fh([)p Fo(u)285 b Fg(7!)f Fo(u)38901 19004 y Fc(\003)39381 -19290 y Fh(])171 b Fo(;)g(S)7379 22975 y Fx(set)24348 -22330 y Fo(E)57 b Fh(\()p Fo(v)36 b Fh(\))285 b Fg(2)f -Fh(dom\()p Fo(S)57 b Fh(\))p 14851 22747 25799 43 v 14851 -24003 a Ff(set)171 b Fh(\()p Fo(v)36 b(;)171 b(F)28 b(;)171 -b(u)p Fh(\))h Fo(;)f(E)57 b(;)171 b(S)23197 23393 y Fn(opt)22960 -24003 y Fh(=)-171 b Fg(\))285 b(h)170 b(i)h Fo(;)g(E)57 -b(;)172 b(S)227 b Fh([)p Fo(E)h Fh(\()p Fo(v)36 b Fh(\))285 -b Fg(7!)f Fh(\()p Fo(S)57 b Fh(\()p Fo(E)g Fh(\()p Fo(v)36 -b Fh(\)\)!)37092 24114 y FD(F)37771 24003 y Fo(E)57 b -Fh(\()p Fo(u)p Fh(\)\))q(])18860 27438 y Fo(E)g Fh(\()p -Fo(v)36 b Fh(\))399 b Fo(=)-625 b Fg(2)284 b Fh(dom)170 -b(\()q Fo(S)57 b Fh(\))170 b Fo(;)341 b Fh(\()p Fo(E)57 -b Fh(\()p Fo(v)36 b Fh(\))p Fo(;)172 b(S)57 b Fh(\))31293 -26868 y Fn(lift)30989 27438 y Fh(=)-171 b Fg(\))285 b -Fh(\(ops)p Fo(;)171 b(S)35944 27015 y Fc(0)36242 27438 -y Fh(\))p 15352 27855 24797 43 v 15352 29112 a Ff(set)f -Fh(\()q Fo(v)36 b(;)171 b(F)28 b(;)171 b(u)p Fh(\))g -Fo(;)g(E)57 b(;)172 b(S)23698 28501 y Fn(opt)23460 29112 -y Fh(=)-171 b Fg(\))285 b Fh(ops)f(::)i Fg(h)p Ff(set)171 -b Fh(\()p Fo(E)57 b Fh(\()p Fo(v)36 b Fh(\))p Fo(;)172 -b(F)28 b(;)171 b(E)57 b Fh(\()p Fo(u)p Fh(\)\))q Fg(i)171 -b Fo(;)g(E)57 b(;)171 b(S)39851 28826 y Fc(0)7379 32797 -y Fx(guar)-37 b(d)19794 32152 y Fo(E)57 b Fh(\()p Fo(v)36 -b Fh(\))285 b Fg(2)f Fh(dom\()p Fo(S)57 b Fh(\))p Fo(;)342 -b Fh(t)-28 b(yp)28 b(e)q(\()p Fo(S)57 b Fh(\()p Fo(E)g -Fh(\()p Fo(v)36 b Fh(\)\)\))285 b(=)g Fo(T)p 19543 32569 -16414 43 v 19543 33825 a Ff(guard)p FC(_)p Ff(class)q -Fh(\()p Fo(v)36 b(;)171 b(T)142 b Fh(\))p Fo(;)172 b(E)57 -b(;)171 b(S)30720 33214 y Fn(opt)30483 33825 y Fh(=)-171 -b Fg(\))285 b(h)171 b(i)f Fo(;)i(E)57 b(;)171 b(S)14249 -37260 y(E)57 b Fh(\()p Fo(v)36 b Fh(\))399 b Fo(=)-625 -b Fg(2)284 b Fh(dom\()p Fo(S)57 b Fh(\))227 b Fg(_)g -Fh(t)-28 b(yp)28 b(e)q(\()p Fo(S)57 b Fh(\()p Fo(E)g -Fh(\()p Fo(v)36 b Fh(\)\)\))285 b Fg(6)p Fh(=)g Fo(T)85 -b(;)342 b Fh(\()p Fo(E)57 b Fh(\()p Fo(v)36 b Fh(\))p -Fo(;)172 b(S)57 b Fh(\))35904 36690 y Fn(lift)35600 37260 -y Fh(=)-171 b Fg(\))285 b Fh(\(ops)p Fo(;)171 b(S)40555 -36837 y Fc(0)40852 37260 y Fh(\))p 13068 37677 29365 -43 v 13068 38933 a Ff(guard)p FC(_)p Ff(class)p Fh(\()p -Fo(v)36 b(;)172 b(T)142 b Fh(\))p Fo(;)172 b(E)57 b(;)171 -b(S)24245 38323 y Fn(opt)24007 38933 y Fh(=)-171 b Fg(\))286 -b Fh(ops)e(::)h Fg(h)p Ff(guard)p FC(_)p Ff(class)q Fh(\()p -Fo(E)228 b Fh(\()p Fo(v)36 b Fh(\))171 b Fo(;)g(T)142 -b Fh(\))p Fg(i)171 b Fo(;)h(E)57 b(;)171 b(S)42135 38647 -y Fc(0)7379 42619 y Fx(lifting)24912 41973 y Fo(v)25445 -41550 y Fc(\003)26323 41973 y Fo(=)-626 b Fg(2)284 b -Fh(dom\()p Fo(S)57 b Fh(\))p 24428 42391 6644 43 v 24428 -43659 a Fo(v)24961 43373 y Fc(\003)25441 43659 y Fo(;)171 -b(S)27166 43089 y Fn(lift)26863 43659 y Fh(=)-171 b Fg(\))285 -b(h)171 b(i)g Fo(;)g(S)19532 47082 y(v)20065 46658 y -Fc(\003)20829 47082 y Fg(2)285 b Fh(dom\()p Fo(S)57 b -Fh(\))p Fo(;)341 b Fh(\()p Fo(v)26766 46658 y Fc(\003)27246 -47082 y Fo(;)171 b(S)57 b Fh(\))29077 46512 y Fn(lift\014elds)29066 -47082 y Fh(=)-171 b(=)g(=)g Fg(\))286 b Fh(\(ops)p Fo(;)171 -b(S)35272 46658 y Fc(0)35570 47082 y Fh(\))p 17885 47499 -19731 43 v 17885 48767 a Fo(v)18418 48481 y Fc(\003)18897 -48767 y Fo(;)g(S)20623 48197 y Fn(lift)20319 48767 y -Fh(=)-171 b Fg(\))285 b(h)p Fo(v)23184 48481 y Fc(\003)23949 -48767 y Fh(=)f Ff(new)171 b Fh(\(t)-28 b(yp)28 b(e)172 -b(\()p Fo(S)227 b Fh(\()p Fo(v)31483 48481 y Fc(\003)31963 -48767 y Fh(\)\)\))p Fg(i)285 b Fh(::)g Fo(ops;)171 b(S)37318 -48481 y Fc(0)11321 51385 y Fe(\000)11790 52190 y Fo(S)227 -b Fh(\()p Fo(v)13574 51767 y Fc(\003)14054 52190 y Fh(\))14452 -52494 y FD(L)15240 52190 y Fo(;)171 b(S)284 b Fg(n)227 -b(f)p Fo(v)18389 51767 y Fc(\003)19154 52190 y Fg(7!)284 -b Fo(S)227 b Fh(\()p Fo(v)22246 51767 y Fc(\003)22726 -52190 y Fh(\))p Fg(g)23636 51385 y Fe(\001)24693 51620 -y Fn(lift)24390 52190 y Fh(=)-171 b Fg(\))285 b Fh(\(ops)28207 -52439 y FD(L)28824 52190 y Fo(;)171 b(S)29962 51767 y -Fc(0)30260 52190 y Fh(\))f Fo(;)31454 51385 y Fe(\000)31923 -52190 y Fo(S)227 b Fh(\()p Fo(v)33707 51767 y Fc(\003)34187 -52190 y Fh(\))34585 52494 y FD(R)35426 52190 y Fo(;)171 -b(S)36564 51767 y Fc(0)36862 51385 y Fe(\001)37919 51620 -y Fn(lift)37616 52190 y Fh(=)-171 b Fg(\))285 b Fh(\(ops)41433 -52439 y FD(R)42103 52190 y Fo(;)171 b(S)43241 51767 y -Fc(00)43781 52190 y Fh(\))p 11221 52707 33058 43 v 11221 -53975 a Fo(v)11754 53689 y Fc(\003)12234 53975 y Fo(;)g(S)13667 -53405 y Fn(lift\014elds)13656 53975 y Fh(=)-171 b(=)g(=)g -Fg(\))286 b Fh(ops)18326 54224 y FD(L)19228 53975 y Fh(::)f(ops)21566 -54224 y FD(R)22520 53975 y Fh(::)23374 53170 y Fe(\012)23857 -53975 y Ff(set)25597 53170 y Fe(\000)26066 53975 y Fo(v)26599 -53689 y Fc(\003)27079 53975 y Fo(;)171 b(L;)g(S)227 b -Fh(\()p Fo(v)30470 53689 y Fc(\003)30950 53975 y Fh(\))31348 -54280 y FD(L)31965 53170 y Fe(\001)32605 53975 y Fo(;)342 -b Ff(set)34971 53170 y Fe(\000)35440 53975 y Fo(v)35973 -53689 y Fc(\003)36453 53975 y Fo(;)171 b(R)7 b(;)171 -b(S)227 b Fh(\()p Fo(v)39929 53689 y Fc(\003)40409 53975 -y Fh(\))40807 54280 y FD(R)41477 53170 y Fe(\001\013)42601 -53975 y Fo(;)171 b(S)43739 53689 y Fc(00)-2000 57817 -y Fx(Object)250 b(Domains:)417 59460 y Fo(u;)172 b(v)235 -b Fg(2)199 b Fo(V)2809 b Fh(v)-57 b(ariables)250 b(in)f(trace)-542 -60567 y Fo(u)47 60143 y Fc(\003)527 60567 y Fo(;)171 -b(v)1515 60143 y Fc(\003)2194 60567 y Fg(2)199 b Fo(V)3901 -60143 y Fc(\003)6482 60567 y Fh(v)-57 b(ariables)250 -b(in)f(optimized)i(trace)1255 61674 y Fo(T)342 b Fg(2)199 -b Fd(T)2740 b Fh(run)-28 b(time)249 b(t)-28 b(yp)28 b(es)1199 -62781 y Fo(F)341 b Fg(2)199 b(f)p Fo(L;)171 b(R)7 b Fg(g)448 -b Fh(\014elds)249 b(of)327 b(ob)57 b(jects)20296 58009 -y Fx(Semantic)251 b(V)-111 b(alues:)20396 59460 y Fo(E)256 -b Fg(2)199 b Fo(V)512 b(*)285 b(V)25529 59036 y Fc(\003)40683 -59460 y Fh(En)-28 b(vironmen)g(t)20524 60567 y Fo(S)255 -b Fg(2)199 b Fo(V)23112 60143 y Fc(\003)23876 60567 y -Fo(*)284 b Fd(T)228 b Fg(\002)g Fh(\()p Fo(V)28325 60143 -y Fc(\003)29033 60567 y Fg([)f(f)p Fh(n)-28 b(ull)p Fg(g)p -Fh(\))227 b Fg(\002)h Fh(\()p Fo(V)35517 60143 y Fc(\003)36224 -60567 y Fg([)g(f)p Fh(n)-28 b(ull)p Fg(g)p Fh(\))448 -b(Static)250 b(Heap)p -2000 63592 55791 37 v 19858 64721 -a Fs(Figur)-18 b(e)249 b(6.)498 b FC(Optimization)251 +TeXDict begin 7 6 bop Black Black Black Black Black 553 +663 a Fx(ne)o(w)1957 614 y Fo(v)1997 583 y Fc(\003)2045 +614 y Fh(fresh)p 1095 646 1973 4 v 1095 740 a Fo(v)24 +b Fh(=)d Ff(new)q Fh(\()p Fo(T)11 b Fh(\))p Fo(;)h(E)t(;)h(S)1689 +694 y Fn(opt)1672 740 y Fh(=)-13 b Fg(\))21 b(h)12 b(i)h +Fo(;)g(E)k Fh([)p Fo(v)24 b Fg(7!)d Fo(v)2230 718 y Fc(\003)2266 +740 y Fh(])14 b Fo(;)f(S)k Fh([)p Fo(v)2460 718 y Fc(\003)2517 +740 y Fg(7!)k Fh(\()p Fo(T)6 b(;)14 b Fh(n)n(ull)p Fo(;)f +Fh(n)n(ull)o(\)])553 1016 y Fx(g)o(et)1826 968 y Fo(E)t +Fh(\()p Fo(v)s Fh(\))21 b Fg(2)g Fh(dom)q(\()p Fo(S)t +Fh(\))p 1269 999 1625 4 v 1269 1093 a Fo(u)h Fh(=)f Ff(get)q +Fh(\()p Fo(v)s(;)12 b(F)f Fh(\))p Fo(;)i(E)t(;)g(S)1946 +1047 y Fn(opt)1929 1093 y Fh(=)-13 b Fg(\))21 b(h)12 +b(i)h Fo(;)g(E)k Fh([)p Fo(u)22 b Fg(7!)f Fo(S)t Fh(\()p +Fo(E)t Fh(\()p Fo(v)s Fh(\)\))2724 1101 y FD(F)2774 1093 +y Fh(])13 b Fo(;)g(S)1663 1321 y(E)t Fh(\()p Fo(v)s Fh(\))29 +b Fo(=)-46 b Fg(2)21 b Fh(dom\()p Fo(S)t Fh(\))p Fo(;)52 +b(u)2290 1289 y Fc(\003)2339 1321 y Fh(fresh)p 1090 1353 +1984 4 v 1090 1447 a Fo(u)21 b Fh(=)g Ff(get)q Fh(\()p +Fo(v)s(;)13 b(F)e Fh(\))p Fo(;)h(E)t(;)h(S)1767 1401 +y Fn(opt)1749 1447 y Fh(=)-13 b Fg(\))21 b(h)p Fo(u)1968 +1425 y Fc(\003)2025 1447 y Fh(=)g Ff(get)q Fh(\()p Fo(E)t +Fh(\()p Fo(v)s Fh(\))p Fo(;)12 b(F)f Fh(\))p Fg(i)i Fo(;)g(E)j +Fh([)q Fo(u)21 b Fg(7!)g Fo(u)2917 1425 y Fc(\003)2954 +1447 y Fh(])13 b Fo(;)g(S)553 1723 y Fx(set)1826 1675 +y Fo(E)t Fh(\()p Fo(v)s Fh(\))21 b Fg(2)g Fh(dom)q(\()p +Fo(S)t Fh(\))p 1114 1706 1935 4 v 1114 1800 a Ff(set)13 +b Fh(\()p Fo(v)s(;)g(F)r(;)h(u)p Fh(\))e Fo(;)i(E)t(;)e(S)1740 +1754 y Fn(opt)1722 1800 y Fh(=)-13 b Fg(\))21 b(h)13 +b(i)f Fo(;)h(E)t(;)g(S)k Fh([)q Fo(E)f Fh(\()p Fo(v)s +Fh(\))21 b Fg(7!)g Fh(\()p Fo(S)t Fh(\()p Fo(E)t Fh(\()p +Fo(v)s Fh(\)\)!)2782 1808 y FD(F)2833 1800 y Fo(E)t Fh(\()p +Fo(u)p Fh(\)\))o(])1415 2058 y Fo(E)t Fh(\()p Fo(v)s +Fh(\))29 b Fo(=)-47 b Fg(2)22 b Fh(dom)13 b(\()p Fo(S)t +Fh(\))f Fo(;)26 b Fh(\()p Fo(E)t Fh(\()p Fo(v)s Fh(\))p +Fo(;)13 b(S)t Fh(\))2347 2015 y Fn(lift)2324 2058 y Fh(=)-13 +b Fg(\))21 b Fh(\(ops)p Fo(;)14 b(S)2696 2026 y Fc(0)2718 +2058 y Fh(\))p 1151 2089 1860 4 v 1151 2183 a Ff(set)g +Fh(\()p Fo(v)s(;)f(F)r(;)g(u)p Fh(\))g Fo(;)g(E)t(;)g(S)1777 +2138 y Fn(opt)1760 2183 y Fh(=)-13 b Fg(\))21 b Fh(ops)g(::)h +Fg(h)p Ff(set)14 b Fh(\()o Fo(E)t Fh(\()p Fo(v)s Fh(\))p +Fo(;)f(F)r(;)g(E)t Fh(\()p Fo(u)p Fh(\)\))p Fg(i)g Fo(;)g(E)t(;)g(S) +2989 2162 y Fc(0)553 2460 y Fx(guar)m(d)1485 2411 y Fo(E)t +Fh(\()p Fo(v)s Fh(\))20 b Fg(2)i Fh(dom\()p Fo(S)t Fh(\))p +Fo(;)k Fh(t)n(yp)r(e)o(\()p Fo(S)t Fh(\()p Fo(E)t Fh(\()p +Fo(v)s Fh(\)\)\))20 b(=)h Fo(T)p 1466 2443 1232 4 v 1466 +2537 a Ff(guard)p FC(_)q Ff(class)q Fh(\()p Fo(v)s(;)13 +b(T)e Fh(\))p Fo(;)i(E)t(;)g(S)2304 2491 y Fn(opt)2286 +2537 y Fh(=)-13 b Fg(\))21 b(h)13 b(i)g Fo(;)g(E)t(;)g(S)1069 +2794 y(E)t Fh(\()p Fo(v)s Fh(\))29 b Fo(=)-46 b Fg(2)21 +b Fh(dom\()p Fo(S)t Fh(\))c Fg(_)g Fh(t)n(yp)r(e\()p +Fo(S)t Fh(\()p Fo(E)t Fh(\()p Fo(v)s Fh(\)\)\))j Fg(6)p +Fh(=)h Fo(T)6 b(;)26 b Fh(\()p Fo(E)t Fh(\()p Fo(v)s +Fh(\))p Fo(;)13 b(S)t Fh(\))2693 2752 y Fn(lift)2670 +2794 y Fh(=)-13 b Fg(\))21 b Fh(\(ops)p Fo(;)13 b(S)3041 +2763 y Fc(0)3064 2794 y Fh(\))p 980 2826 2203 4 v 980 +2920 a Ff(guard)p FC(_)r Ff(class)q Fh(\()p Fo(v)s(;)g(T)e +Fh(\))p Fo(;)h(E)t(;)h(S)1818 2874 y Fn(opt)1801 2920 +y Fh(=)-13 b Fg(\))21 b Fh(ops)g(::)h Fg(h)p Ff(guard)p +FC(_)r Ff(class)q Fh(\()p Fo(E)16 b Fh(\()p Fo(v)s Fh(\))d +Fo(;)g(T)e Fh(\))p Fg(i)h Fo(;)h(E)t(;)g(S)3160 2899 +y Fc(0)553 3196 y Fx(lifting)1868 3148 y Fo(v)1908 3116 +y Fc(\003)1974 3148 y Fo(=)-46 b Fg(2)21 b Fh(dom\()p +Fo(S)t Fh(\))p 1832 3179 499 4 v 1832 3274 a Fo(v)1872 +3253 y Fc(\003)1908 3274 y Fo(;)13 b(S)2037 3232 y Fn(lift)2015 +3274 y Fh(=)-13 b Fg(\))21 b(h)12 b(i)h Fo(;)g(S)1465 +3531 y(v)1505 3499 y Fc(\003)1562 3531 y Fg(2)22 b Fh(dom\()p +Fo(S)t Fh(\))p Fo(;)k Fh(\()o Fo(v)2007 3499 y Fc(\003)2043 +3531 y Fo(;)14 b(S)t Fh(\))2181 3488 y Fn(lift\014elds)2180 +3531 y Fh(=)-13 b(=)g(=)g Fg(\))21 b Fh(\(ops)p Fo(;)13 +b(S)2645 3499 y Fc(0)2668 3531 y Fh(\))p 1341 3562 1480 +4 v 1341 3658 a Fo(v)1381 3636 y Fc(\003)1417 3658 y +Fo(;)g(S)1547 3615 y Fn(lift)1524 3658 y Fh(=)-13 b Fg(\))21 +b(h)p Fo(v)1739 3636 y Fc(\003)1796 3658 y Fh(=)g Ff(new)14 +b Fh(\(t)n(yp)r(e)e(\()o Fo(S)17 b Fh(\()p Fo(v)2361 +3636 y Fc(\003)2397 3658 y Fh(\)\)\))p Fg(i)k Fh(::)h +Fo(ops;)13 b(S)2799 3636 y Fc(0)849 3854 y Fe(\000)884 +3914 y Fo(S)k Fh(\()p Fo(v)1018 3883 y Fc(\003)1054 3914 +y Fh(\))1084 3937 y FD(L)1143 3914 y Fo(;)c(S)21 b Fg(n)d(f)p +Fo(v)1379 3883 y Fc(\003)1437 3914 y Fg(7!)j Fo(S)c Fh(\()o +Fo(v)1668 3883 y Fc(\003)1704 3914 y Fh(\))p Fg(g)1773 +3854 y Fe(\001)1852 3871 y Fn(lift)1829 3914 y Fh(=)-13 +b Fg(\))21 b Fh(\(ops)2115 3933 y FD(L)2162 3914 y Fo(;)13 +b(S)2247 3883 y Fc(0)2269 3914 y Fh(\))g Fo(;)2359 3854 +y Fe(\000)2394 3914 y Fo(S)k Fh(\()p Fo(v)2528 3883 y +Fc(\003)2564 3914 y Fh(\))2594 3937 y FD(R)2657 3914 +y Fo(;)c(S)2742 3883 y Fc(0)2765 3854 y Fe(\001)2844 +3871 y Fn(lift)2821 3914 y Fh(=)-13 b Fg(\))21 b Fh(\(ops)3107 +3933 y FD(R)3158 3914 y Fo(;)13 b(S)3243 3883 y Fc(00)3284 +3914 y Fh(\))p 842 3953 2480 4 v 842 4048 a Fo(v)882 +4027 y Fc(\003)918 4048 y Fo(;)g(S)1025 4005 y Fn(lift\014elds)1024 +4048 y Fh(=)-13 b(=)g(=)g Fg(\))21 b Fh(ops)1374 4067 +y FD(L)1442 4048 y Fh(::)h(ops)1617 4067 y FD(R)1689 +4048 y Fh(::)1753 3988 y Fe(\012)1789 4048 y Ff(set)1920 +3988 y Fe(\000)1955 4048 y Fo(v)1995 4027 y Fc(\003)2031 +4048 y Fo(;)13 b(L;)g(S)k Fh(\()p Fo(v)2285 4027 y Fc(\003)2321 +4048 y Fh(\))2351 4071 y FD(L)2397 3988 y Fe(\001)2445 +4048 y Fo(;)26 b Ff(set)2623 3988 y Fe(\000)2658 4048 +y Fo(v)2698 4027 y Fc(\003)2734 4048 y Fo(;)13 b(R)q(;)g(S)k +Fh(\()p Fo(v)2995 4027 y Fc(\003)3031 4048 y Fh(\))3061 +4071 y FD(R)3111 3988 y Fe(\001\013)3195 4048 y Fo(;)c(S)3280 +4027 y Fc(00)-150 4336 y Fx(Object)19 b(Domains:)31 4459 +y Fo(u;)14 b(v)k Fg(2)d Fo(V)210 b Fh(v)l(ariables)20 +b(in)e(trace)-41 4542 y Fo(u)3 4511 y Fc(\003)40 4542 +y Fo(;)13 b(v)114 4511 y Fc(\003)165 4542 y Fg(2)i Fo(V)293 +4511 y Fc(\003)486 4542 y Fh(v)l(ariables)20 b(in)e(optimized)i(trace) +94 4626 y Fo(T)26 b Fg(2)15 b Fd(T)205 b Fh(run)n(time)18 +b(t)n(yp)r(es)90 4709 y Fo(F)26 b Fg(2)15 b(f)p Fo(L;)e(R)q +Fg(g)34 b Fh(\014elds)19 b(of)25 b(ob)t(jects)1522 4351 +y Fx(Semantic)20 b(V)-8 b(alues:)1530 4459 y Fo(E)18 +b Fg(2)e Fo(V)38 b(*)21 b(V)1915 4428 y Fc(\003)3051 +4459 y Fh(En)n(vironmen)n(t)1539 4542 y Fo(S)e Fg(2)d +Fo(V)1733 4511 y Fc(\003)1791 4542 y Fo(*)21 b Fd(T)c +Fg(\002)g Fh(\()p Fo(V)2124 4511 y Fc(\003)2177 4542 +y Fg([)h(f)p Fh(n)n(ull)p Fg(g)p Fh(\))f Fg(\002)g Fh(\()p +Fo(V)2664 4511 y Fc(\003)2717 4542 y Fg([)g(f)p Fh(n)n(ull)p +Fg(g)p Fh(\))34 b(Static)19 b(Heap)p -150 4769 4185 3 +v 1489 4854 a Fs(Figur)o(e)f(6.)38 b FC(Optimization)19 b(Rules)p Black Black Black eop end %%Page: 8 8 -TeXDict begin 8 7 bop Black Black -2000 886 a FC(runs)279 -b(in)g(a)h(single)f(pass)f(o)-15 b(v)g(er)280 b(the)g(list)e(of)h -(operations.)h(W)-80 b(e)280 b(can)g(check)h(that)-2000 -1993 y(although)387 b(recursi)-25 b(v)-15 b(ely)386 b(lifting)g(a)h -(static)f(object)g(is)g(not)g(a)g(constant-time)-2000 -3099 y(operation,)266 b(the)g(algorithm)g(only)f(tak)-10 -b(es)266 b(a)f(total)h(time)g(linear)f(in)h(the)f(length)-2000 -4206 y(of)337 b(the)g(trace.)g(The)g(algorithm)g(itself)g(is)f(not)h -(particularly)g(comple)-15 b(x;)338 b(our)-2000 5313 -y(focus)343 b(is)g(rather)h(that)g Fx(in)f(the)h(conte)-20 -b(xt)345 b(of)e(tr)-15 b(acing)344 b(JITs)f FC(it)g(is)g(possible)g(to) --2000 6420 y(\002nd)249 b(a)h(simple)f(enough)h(algorithm)f(that)h -(performs)e(well.)-672 7527 y(Note)302 b(in)f(particular)h(that)g -(objects)f(in)g(cate)-15 b(gory)303 b(1)e(\(i.e.,)g(those)g(that)h(do) --2000 8634 y(not)240 b(escape\))h(are)g(completely)g(remo)-15 -b(v)g(ed;)241 b(moreo)-15 b(v)g(er)-40 b(,)241 b(objects)g(in)f(cate) --15 b(gory)-2000 9741 y(2)302 b(\(i.e.,)g(escaping\))h(are)f(still)g -(partially)g(optimized:)h(all)f(the)h(operations)f(in)-2000 -10848 y(between)329 b(the)f(creation)h(of)e(the)i(object)f(and)g(the)g -(point)g(where)h(it)e(escapes)-2000 11955 y(that)225 -b(in)-40 b(v)-20 b(olv)-15 b(e)226 b(the)g(object)g(are)f(remo)-15 -b(v)g(ed.)226 b(Objects)g(in)f(cate)-15 b(gory)226 b(3)f(and)h(4)f(are) --2000 13062 y(also)330 b(partially)g(optimized,)g(their)g(allocation)h -(is)e(delayed)i(till)e(the)h(end)g(of)-2000 14169 y(the)249 -b(trace.)-672 15276 y(The)362 b(optimization)h(is)e(particularly)i(ef) --25 b(fecti)g(v)-15 b(e)363 b(for)e(chains)h(of)g(opera-)-2000 -16383 y(tions.)355 b(F)-15 b(or)355 b(e)-15 b(xample,)357 -b(it)f(is)e(typical)j(for)e(an)g(interpreter)h(to)g(generate)g(se-) --2000 17490 y(quences)196 b(of)e(writes-follo)-25 b(wed-by-reads,)195 -b(where)h(one)f(interpreted)h(opcode)-2000 18597 y(writes)273 -b(to)g(some)h(object')-55 b(s)273 b(\002eld)h(and)f(the)h(ne)-15 -b(xt)274 b(interpreted)g(opcode)g(reads)-2000 19704 y(it)295 -b(back,)g(possibly)f(dispatching)i(on)e(the)i(type)f(of)f(the)h(object) -h(created)f(just)-2000 20811 y(before.)355 b(A)g(typical)h(e)-15 -b(xample)356 b(w)-10 b(ould)355 b(be)g(a)g(chain)h(of)e(arithmetic)i -(opera-)-2000 21918 y(tions.)-2000 24492 y FA(6.)1218 -b(Implementation)305 b(and)g(Ev)-12 b(aluation)-2000 -26042 y FC(The)227 b(allocation)g(remo)-15 b(v)-25 b(al)228 -b(techniques)f(described)g(in)g(this)f(paper)h(were)g(im-)-2000 -27149 y(plemented)240 b(in)f(the)h(optimizer)f(of)g(PyPy')-55 -b(s)239 b(tracing)h(JIT)-74 b(.)238 b(The)h(optimization)-2000 -28256 y(is)219 b(independent)i(of)f(which)g(interpreter)g(a)g(JIT)f(is) -g(generated)i(for)-55 b(.)220 b(There)g(are)-2000 29363 -y(some)202 b(practical)h(issues)e(be)-15 b(yond)203 b(the)g(techniques) -g(described)f(in)g(this)g(paper)-55 b(.)-2000 30470 y(The)222 -b(actual)h(implementation)g(needs)f(to)g(deal)h(with)f(more)g -(operations)g(than)-2000 31577 y(described)237 b(in)g(Section)h(5,)f -(e.g.,)g(to)g(also)g(support)f(static)h(arrays)f(in)h(addition)-2000 -32684 y(to)292 b(static)g(objects.)h(The)f(implementation)i(of)e(this)g -(optimization)h(is)e(about)-2000 33791 y(400)249 b(lines)g(of)g -(RPython)g(code.)-672 34898 y(A)423 b(further)f(complication)i(is)e -(that)h(most)f(interpreters)g(written)h(with)-2000 36005 -y(PyPy)464 b(use)g(heap-allocated)i(frame)e(objects)g(to)g(store)g -(local)g(v)-25 b(ariables.)-2000 37112 y(Those)306 b(se)-25 -b(v)-15 b(erely)307 b(hinder)f(the)h(ef)-25 b(fecti)g(v)-15 -b(eness)307 b(of)e(allocation)j(remo)-15 b(v)-25 b(al,)307 -b(be-)-2000 38219 y(cause)225 b(e)-25 b(v)-15 b(ery)226 -b(time)f(an)g(object)g(is)f(stored)h(into)g(a)g(local)g(v)-25 -b(ariable,)226 b(it)e(is)g(stored)-2000 39326 y(into)343 -b(the)g(frame-object,)h(which)f(mak)-10 b(es)343 b(it)g(escape.)h(W)-80 -b(e)344 b(implemented)g(a)-2000 40433 y(technique)392 -b(to)e(treat)g(such)h(frames)f(objects)g(in)g(a)h(special)g(w)-10 -b(ay)391 b(to)f(solv)-15 b(e)-2000 41540 y(this)280 b(problem.)g(This)f -(is)h(a)g(common)h(approach)g(in)f(VM)g(implementations)-2000 -42647 y([13,)449 b(22];)f(the)h(no)-15 b(v)g(elty)449 -b(of)g(our)f(approach)i(is)e(that)h(we)g(generalized)h(it)-2000 -43754 y(enough)250 b(to)f(be)h(usable)f(for)g(dif)-25 -b(ferent)249 b(interpreters.)-672 44860 y(T)-80 b(o)386 -b(e)-25 b(v)g(aluate)387 b(our)e(allocation)h(remo)-15 -b(v)-25 b(al)386 b(algorithm,)g(we)g(look)f(at)h(the)-2000 -45967 y(ef)-25 b(fecti)g(v)-15 b(eness)433 b(when)g(used)f(in)g(the)h -(generated)h(tracing)e(JIT)f(of)h(PyPy')-55 b(s)-2000 -47074 y(Python)488 b(interpreter)-55 b(.)489 b(This)f(interpreter)g(is) -f(a)i(full)f(implementation)h(of)-2000 48181 y(Python)579 -b(2.5)f(language)i(semantics)e(and)h(is)f(about)h(30,000)g(lines)f(of) --2000 49288 y(RPython)249 b(code.)-672 50395 y(The)476 -b(benchmarks)f(we)h(used)f(are)h(small-to-medium)f(Python)g(pro-)-2000 -51540 y(grams,)249 b(some)g(synthetic)h(benchmarks,)g(some)f(real)g -(applications.)21601 51117 y Fz(8)-672 52647 y FC(Some)410 -b(of)e(them)i(are)f(from)f(the)h(Computer)g(Language)h(Benchmark)-2000 -53791 y(Game)378 53368 y Fz(9)766 53791 y FC(:)250 b -Fs(fannkuch)p FC(,)e Fs(nbody)p FC(,)i Fs(meteor)-37 -b(-contest)p FC(,)249 b Fs(spectral-norm)p FC(.)-672 -54898 y(Furthermore)h(there)f(are)h(the)f(follo)-25 b(wing)250 -b(benchmarks:)p Black -1419 56470 a Fm(\017)p Black -561 -56568 a Fs(crypto_pyaes)p FC(:)g(An)f(AES)g(implementation.)p -Black -1419 58020 a Fm(\017)p Black -561 58118 a Fs(django)p -FC(:)519 b(The)h(templating)g(engine)h(of)e(the)h(Django)g(web)g -(frame-)-561 59263 y(w)-10 b(ork)1476 58839 y Fz(10)2196 -59263 y FC(.)p Black -1419 60714 a Fm(\017)p Black -561 -60812 a Fs(go)p FC(:)249 b(A)h(Monte-Carlo)e(Go)i(AI)9843 -60389 y Fz(11)10562 60812 y FC(.)p Black -1419 62264 -a Fm(\017)p Black -561 62362 a Fs(html5lib)p FC(:)f(An)g(HTML5)g -(parser)-55 b(.)p Black -2000 63529 13284 37 v -2000 -64212 a Fz(8)-1502 64525 y Fw(All)901 b(the)f(source)h(code)f(of)g(the) -h(benchmarks)f(can)g(be)g(found)g(at)-2000 65521 y Fr -(http://codespeak.net/svn/pypy/benchmarks/)p Fw(.)2428 -b(There)-2000 66517 y(is)766 b(also)g(a)g(website)h(that)e(monitors)h -(PyPy')-49 b(s)766 b(performance)g(nightly)f(at)-2000 -67513 y Fr(http://speed.pypy.org/)p Fw(.)-2000 68496 -y Fz(9)-1502 68809 y Fr(http://shootout.alioth.debian.org/)-2000 -69791 y Fz(10)-1170 70104 y Fr(http://www.djangoproject.com/)-2000 -71086 y Fz(11)-1170 71399 y Fr(http://shed-skin.blogspot.com/2009/07/) --2000 72395 y(disco-elegant-python-go-player.html)p Black -Black Black Black 27805 788 a Fm(\017)p Black 28663 886 -a Fs(py\003ate-fast)p FC(:)248 b(A)h(BZ2)g(decoder)-55 -b(.)p Black 27805 2337 a Fm(\017)p Black 28663 2435 a -Fs(raytrace-simple)p FC(:)250 b(A)f(ray)g(tracer)-55 -b(.)p Black 27805 3887 a Fm(\017)p Black 28663 3985 a -Fs(richards)p FC(:)249 b(The)h(Richards)e(benchmark.)p -Black 27805 5437 a Fm(\017)p Black 28663 5535 a Fs(spambay)-10 -b(es)p FC(:)248 b(A)i(Bayesian)f(spam)g(\002lter)42932 -5112 y Fz(12)43652 5535 y FC(.)p Black 27805 7024 a Fm(\017)p -Black 28663 7122 a Fs(telco)p FC(:)451 b(A)f(Python)h(v)-15 -b(ersion)451 b(of)f(the)h(T)-70 b(elco)451 b(decimal)h(benchmark)52821 -6699 y Fz(13)53542 7122 y FC(,)28663 8229 y(using)249 -b(a)g(pure)h(Python)f(decimal)h(\003oating)g(point)f(implementation.)p -Black 27805 9681 a Fm(\017)p Black 28663 9779 a Fs(twisted_names)p -FC(:)361 b(A)h(DNS)h(serv)-15 b(er)362 b(benchmark)h(using)f(the)h(T) --80 b(wisted)28663 10923 y(netw)-10 b(orking)250 b(frame)-25 -b(w)-10 b(ork)37775 10500 y Fz(14)38495 10923 y FC(.)28552 -13236 y(W)-80 b(e)322 b(e)-25 b(v)g(aluate)322 b(the)e(allocation)i -(remo)-15 b(v)-25 b(al)321 b(algorithm)g(along)g(tw)-10 -b(o)321 b(lines:)27224 14343 y(\002rst)361 b(we)h(w)-10 -b(ant)363 b(to)f(kno)-25 b(w)363 b(ho)-25 b(w)362 b(man)-15 -b(y)363 b(allocations)g(could)f(be)h(optimized)27224 -15450 y(a)-15 b(w)-10 b(ay)-65 b(.)230 b(On)g(the)f(other)g(hand,)h(we) -g(w)-10 b(ant)229 b(to)g(kno)-25 b(w)230 b(ho)-25 b(w)230 -b(much)g(the)f(run)g(times)27224 16557 y(of)249 b(the)g(benchmarks)h -(is)f(impro)-15 b(v)g(ed.)28552 17664 y(The)233 b(benchmarks)h(were)g -(run)f(on)h(an)f(otherwise)h(idle)f(Intel)g(Core2)g(Duo)27224 -18771 y(P8400)326 b(processor)f(with)h(2.26)g(GHz)h(and)f(3072)g(KB)f -(of)h(cache)h(on)f(a)g(ma-)27224 19878 y(chine)224 b(with)g(3GB)f(RAM)g -(running)h(Linux)f(2.6.35.)i(W)-80 b(e)224 b(compared)h(the)f(per)-20 -b(-)27224 20985 y(formance)333 b(of)f(v)-25 b(arious)332 -b(Python)h(implementations)g(on)f(the)h(benchmarks.)27224 -22092 y(As)452 b(a)h(baseline,)g(we)g(used)f(the)h(standard)f(Python)h -(implementation)h(in)27224 23236 y(C,)296 b(CPython)h(2.6.6)34211 -22813 y Fz(15)34931 23236 y FC(,)g(which)h(uses)f(a)g(bytecode-based)i -(interpreter)-55 b(.)297 b(Fur)-20 b(-)27224 24343 y(thermore)260 -b(we)g(compared)g(ag)-5 b(ainst)260 b(Psyco[25])f(1.6,)h(a)g -(\(hand-written\))g(e)-15 b(x-)27224 25450 y(tension)357 -b(module)g(to)g(CPython)f(which)i(is)e(a)h(just-in-time)f(compiler)h -(that)27224 26557 y(produces)343 b(machine)h(code)f(at)g(run-time.)g -(It)f(is)g(not)h(based)g(on)g(traces.)g(Fi-)27224 27664 -y(nally)-65 b(,)281 b(we)g(used)g(tw)-10 b(o)281 b(v)-15 -b(ersions)280 b(of)g(PyPy')-55 b(s)280 b(Python)h(interpreter)g(\(re) --25 b(vision)27224 28809 y(77823)323 b(of)g(SVN)h(trunk)35609 -28385 y Fz(16)36329 28809 y FC(\):)e(one)i(including)g(the)f(JIT)f(b) --20 b(ut)324 b(not)f(optimizing)27224 29916 y(the)309 -b(traces,)h(and)g(one)g(using)f(the)h(allocation)g(remo)-15 -b(v)-25 b(al)310 b(optimizations)g(\(as)27224 31022 y(well)249 -b(as)g(some)g(minor)h(other)f(optimizations,)h(such)f(as)g(constant)g -(folding\).)28552 32129 y(As)224 b(the)h(\002rst)e(step,)i(we)g -(counted)g(the)g(occurring)g(operations)g(in)f(all)g(gen-)27224 -33236 y(erated)262 b(traces)f(before)g(and)h(after)f(the)h -(optimization)g(phase)f(for)g(all)g(bench-)27224 34343 -y(marks.)393 b(The)g(resulting)f(numbers)h(can)h(be)f(seen)h(in)f -(Figure)g(7.)g(The)g(op-)27224 35450 y(timization)416 -b(remo)-15 b(v)g(es)417 b(between)g(4\045)f(and)g(90\045)g(and)h(of)f -(allocation)h(op-)27224 36557 y(erations)g(in)g(the)g(traces)g(of)g -(the)g(benchmarks.)h(All)f(benchmarks)h(tak)-10 b(en)27224 -37664 y(together)-40 b(,)332 b(the)f(optimization)h(remo)-15 -b(v)g(es)332 b(70\045)f(percent)h(of)f(allocation)h(op-)27224 -38771 y(erations.)e(The)g(numbers)f(look)i(similar)e(for)g(reading)i -(and)f(writing)g(of)g(at-)27224 39878 y(trib)-20 b(utes.)298 -b(There)i(are)f(e)-25 b(v)-15 b(en)300 b(more)f Fr(guard)p -Black Black 299 w FC(operations)g(that)g(are)h(remo)-15 -b(v)g(ed,)27224 40985 y(ho)-25 b(we)g(v)-15 b(er)356 -b(there)f(is)f(an)i(additional)g(optimization)f(that)g(remo)-15 -b(v)g(es)356 b(guards,)27224 42092 y(so)277 b(not)g(all)g(the)h(remo) --15 b(v)g(ed)278 b(guards)g(are)f(an)h(ef)-25 b(fect)278 -b(of)f(the)g(optimization)i(de-)27224 43199 y(scribed)192 -b(here)g(\(for)f(technical)i(reasons,)f(it)f(w)-10 b(ould)193 -b(be)f(v)-15 b(ery)192 b(hard)g(to)g(separate)27224 44306 -y(the)249 b(tw)-10 b(o)250 b(ef)-25 b(fects\).)28552 -45413 y(In)327 b(addition)h(to)f(the)g(count)h(of)e(operations)i(we)f -(also)g(performed)g(time)27224 46520 y(measurements.)242 -b(All)f(benchmarks)h(were)g(run)g(50)f(times)g(in)h(the)f(same)h(pro-) -27224 47627 y(cess,)255 b(to)h(gi)-25 b(v)-15 b(e)256 -b(the)g(JIT)f(time)h(to)g(produce)h(machine)f(code.)h(The)f(arithmetic) -27224 48734 y(mean)303 b(of)g(the)g(times)g(of)g(the)g(last)f(30)h -(runs)g(were)g(used)g(as)g(the)g(result.)g(The)27224 -49841 y(errors)262 b(were)i(computed)h(using)e(a)h(con\002dence)h -(interv)-25 b(al)264 b(with)f(a)h(95\045)f(con-)27224 -50948 y(\002dence)354 b(le)-25 b(v)-15 b(el)353 b([16].)g(The)g -(results)f(are)h(reported)g(in)g(Figure)g(8.)g(F)-15 -b(or)353 b(each)27224 52055 y(implementation)327 b(the)g(table)g(also)f -(reports)f(the)i(speedup)g(that)f(PyPy)g(with)27224 53162 -y(optimization)250 b(achie)-25 b(v)-15 b(es)250 b(o)-15 -b(v)g(er)250 b(it.)28552 54269 y(W)-40 b(ith)238 b(the)f(optimization)h -(turned)f(on,)g(PyPy')-55 b(s)236 b(Python)i(interpreter)f(out-)27224 -55376 y(performs)453 b(CPython)g(in)g(all)h(benchmarks)g(e)-15 -b(xcept)455 b(spambayes)f(\(which)27224 56483 y(hea)-20 -b(vily)376 b(relies)f(on)g(re)-15 b(gular)376 b(e)-15 -b(xpression)375 b(performance)h(and)g(thus)f(is)g(not)27224 -57590 y(helped)369 b(much)h(by)f(our)f(Python)h(JIT\))e(and)j(meteor) --20 b(-contest.)369 b(All)g(bench-)27224 58696 y(marks)306 -b(are)h(impro)-15 b(v)g(ed)307 b(by)g(the)f(allocation)i(remo)-15 -b(v)-25 b(al)307 b(optimization,)h(by)e(at)27224 59803 -y(least)249 b(20\045)g(and)h(by)f(as)g(much)h(as)f(a)g(f)-10 -b(actor)250 b(of)e(6.95.)28552 60910 y(Psyco)262 b(is)f(able)i(to)e -(outperform)h(PyPy')-55 b(s)262 b(JIT)e(in)i(\002)-25 -b(v)-15 b(e)262 b(out)g(of)g(14)g(bench-)27224 62017 -y(marks.)336 b(W)-80 b(e)337 b(hope)f(to)g(o)-15 b(v)g(ertak)-10 -b(e)337 b(Psyco)f(\(which)h(is)e(no)h(longer)h(being)f(ac-)27224 -63124 y(ti)-25 b(v)-15 b(ely)249 b(de)-25 b(v)-15 b(elopped\))251 -b(by)e(adding)h(some)g(further)e(optimizations.)p Black -27224 66219 V 27224 66902 a Fz(12)28054 67215 y Fr -(http://spambayes.sourceforge.net/)27224 68197 y Fz(13)28054 -68510 y Fr(http://speleotrove.com/decimal/telco.html)27224 -69492 y Fz(14)28054 69805 y Fr(http://twistedmatrix.com/)27224 -70788 y Fz(15)28054 71100 y Fr(http://python.org)27224 -72083 y Fz(16)28054 72395 y Fr(http://codespeak.net/svn/pypy/trunk)p +TeXDict begin 8 7 bop Black Black -150 66 a FC(runs)22 +b(in)f(a)g(single)g(pass)h(o)o(v)o(er)f(the)h(list)e(of)h(operations.)h +(W)-6 b(e)21 b(can)g(check)h(that)-150 149 y(although)30 +b(recursi)n(v)o(ely)g(lifting)f(a)g(static)g(object)g(is)g(not)h(a)f +(constant-time)-150 232 y(operation,)21 b(the)f(algorithm)g(only)h(tak) +o(es)g(a)f(total)g(time)f(linear)i(in)f(the)g(length)-150 +315 y(of)25 b(the)h(trace.)f(The)h(algorithm)f(itself)g(is)g(not)h +(particularly)g(comple)o(x;)g(our)-150 399 y(focus)h(is)e(rather)h +(that)g Fx(in)g(the)g(conte)o(xt)g(of)g(tr)o(acing)g(JITs)g +FC(it)f(is)h(possible)g(to)-150 482 y(\002nd)19 b(a)g(simple)g(enough)h +(algorithm)g(that)e(performs)i(well.)-50 565 y(Note)i(in)h(particular)g +(that)g(objects)g(in)g(cate)o(gory)g(1)g(\(i.e.,)f(those)h(that)g(do) +-150 648 y(not)18 b(escape\))i(are)e(completely)h(remo)o(v)o(ed;)g +(moreo)o(v)o(er)m(,)g(objects)f(in)g(cate)o(gory)-150 +731 y(2)23 b(\(i.e.,)f(escaping\))h(are)g(still)f(partially)h +(optimized:)g(all)g(the)f(operations)i(in)-150 814 y(between)i(the)e +(creation)i(of)e(the)h(object)h(and)f(the)g(point)g(where)g(it)f +(escapes)-150 897 y(that)17 b(in)m(v)o(olv)o(e)g(the)h(object)f(are)g +(remo)o(v)o(ed.)h(Objects)f(in)g(cate)o(gory)h(3)f(and)h(4)f(are)-150 +980 y(also)25 b(partially)g(optimized,)g(their)g(allocation)g(is)g +(delayed)h(till)e(the)h(end)g(of)-150 1063 y(the)19 b(trace.)-50 +1146 y(The)27 b(optimization)h(is)f(particularly)g(ef)n(fecti)n(v)o(e)h +(for)f(chains)h(of)g(opera-)-150 1229 y(tions.)f(F)o(or)f(e)o(xample,)h +(it)g(is)f(typical)h(for)g(an)g(interpreter)g(to)g(generate)g(se-)-150 +1312 y(quences)16 b(of)f(writes-follo)n(wed-by-reads,)h(where)f(one)g +(interpreted)h(opcode)-150 1395 y(writes)k(to)h(some)g(object')l(s)g +(\002eld)f(and)i(the)e(ne)o(xt)h(interpreted)h(opcode)g(reads)-150 +1478 y(it)g(back,)g(possibly)i(dispatching)f(on)g(the)g(type)f(of)h +(the)f(object)h(created)g(just)-150 1561 y(before.)k(A)g(typical)g(e)o +(xample)g(w)o(ould)h(be)f(a)g(chain)g(of)g(arithmetic)g(opera-)-150 +1644 y(tions.)-150 1837 y FA(6.)91 b(Implementation)22 +b(and)g(Ev)o(aluation)-150 1953 y FC(The)17 b(allocation)h(remo)o(v)n +(al)g(techniques)g(described)h(in)e(this)g(paper)h(were)f(im-)-150 +2036 y(plemented)i(in)f(the)g(optimizer)h(of)f(PyPy')l(s)g(tracing)g +(JIT)-6 b(.)18 b(The)g(optimization)-150 2119 y(is)e(independent)j(of)e +(which)g(interpreter)f(a)h(JIT)f(is)h(generated)g(for)l(.)g(There)f +(are)-150 2202 y(some)g(practical)f(issues)h(be)o(yond)h(the)e +(techniques)i(described)g(in)e(this)g(paper)l(.)-150 +2285 y(The)i(actual)g(implementation)h(needs)f(to)g(deal)g(with)g(more) +g(operations)h(than)-150 2368 y(described)h(in)f(Section)g(5,)g(e.g.,)f +(to)h(also)g(support)h(static)e(arrays)i(in)f(addition)-150 +2451 y(to)k(static)g(objects.)g(The)g(implementation)h(of)f(this)g +(optimization)h(is)f(about)-150 2534 y(400)e(lines)f(of)g(RPython)g +(code.)-50 2617 y(A)31 b(further)h(complication)h(is)f(that)f(most)h +(interpreters)g(written)g(with)-150 2700 y(PyPy)j(use)g(heap-allocated) +h(frame)g(objects)f(to)g(store)g(local)g(v)n(ariables.)-150 +2783 y(Those)24 b(se)n(v)o(erely)g(hinder)f(the)h(ef)n(fecti)n(v)o +(eness)g(of)f(allocation)h(remo)o(v)n(al,)f(be-)-150 +2866 y(cause)18 b(e)n(v)o(ery)g(time)e(an)i(object)f(is)g(stored)g +(into)g(a)g(local)h(v)n(ariable,)f(it)f(is)h(stored)-150 +2949 y(into)26 b(the)g(frame-object,)g(which)g(mak)o(es)h(it)f(escape.) +g(W)-6 b(e)25 b(implemented)i(a)-150 3032 y(technique)k(to)e(treat)g +(such)h(frames)g(objects)g(in)f(a)h(special)g(w)o(ay)g(to)f(solv)o(e) +-150 3115 y(this)21 b(problem.)h(This)f(is)f(a)i(common)g(approach)h +(in)e(VM)g(implementations)-150 3198 y([13)q(,)33 b(22)q(];)g(the)h(no) +o(v)o(elty)g(of)g(our)h(approach)g(is)e(that)h(we)g(generalized)h(it) +-150 3282 y(enough)21 b(to)d(be)i(usable)f(for)g(dif)n(ferent)g +(interpreters.)-50 3365 y(T)-6 b(o)29 b(e)n(v)n(aluate)h(our)f +(allocation)h(remo)o(v)n(al)f(algorithm,)h(we)f(look)g(at)g(the)-150 +3448 y(ef)n(fecti)n(v)o(eness)34 b(when)f(used)g(in)g(the)f(generated)i +(tracing)f(JIT)f(of)h(PyPy')l(s)-150 3531 y(Python)k(interpreter)l(.)g +(This)f(interpreter)h(is)f(a)h(full)g(implementation)g(of)-150 +3614 y(Python)44 b(2.5)g(language)h(semantics)f(and)g(is)g(about)g +(30,000)h(lines)e(of)-150 3697 y(RPython)19 b(code.)-50 +3780 y(The)35 b(benchmarks)j(we)e(used)g(are)g(small-to-medium)h +(Python)f(pro-)-150 3865 y(grams,)19 b(some)g(synthetic)h(benchmarks,)g +(some)g(real)e(applications.)1619 3834 y Fz(8)-50 3949 +y FC(Some)30 b(of)h(them)g(are)g(from)g(the)g(Computer)h(Language)g +(Benchmark)-150 4034 y(Game)28 4003 y Fz(9)57 4034 y +FC(:)19 b Fs(fannkuch)p FC(,)d Fs(nbody)p FC(,)i Fs(meteor)m(-contest)p +FC(,)i Fs(spectral-norm)p FC(.)-50 4117 y(Furthermore)f(there)g(are)g +(the)g(follo)n(wing)g(benchmarks:)p Black -106 4236 a +Fm(\017)p Black -42 4243 a Fs(crypto_pyaes)p FC(:)h(An)f(AES)f +(implementation.)p Black -106 4352 a Fm(\017)p Black +-42 4359 a Fs(django)p FC(:)39 b(The)g(templating)g(engine)i(of)e(the)g +(Django)h(web)f(frame-)-42 4445 y(w)o(ork)110 4413 y +Fz(10)165 4445 y FC(.)p Black -106 4554 a Fm(\017)p Black +-42 4561 a Fs(go)p FC(:)19 b(A)g(Monte-Carlo)g(Go)g(AI)738 +4529 y Fz(11)792 4561 y FC(.)p Black -106 4670 a Fm(\017)p +Black -42 4677 a Fs(html5lib)p FC(:)e(An)i(HTML5)g(parser)l(.)p +Black -150 4765 997 3 v -150 4816 a Fz(8)-113 4839 y +Fw(All)69 b(the)f(source)h(code)g(of)f(the)h(benchmarks)g(can)g(be)f +(found)h(at)-150 4914 y Fr(http://codespeak.net/svn/pypy/benchmarks/)p +Fw(.)181 b(There)-150 4989 y(is)58 b(also)g(a)g(website)h(that)g +(monitors)g(PyPy')l(s)f(performance)i(nightly)g(at)-150 +5064 y Fr(http://speed.pypy.org/)p Fw(.)-150 5137 y Fz(9)-113 +5161 y Fr(http://shootout.alioth.debian.org/)-150 5234 +y Fz(10)-88 5258 y Fr(http://www.djangoproject.com/)-150 +5331 y Fz(11)-88 5355 y Fr(http://shed-skin.blogspot.com/2009/07/)-150 +5430 y(disco-elegant-python-go-player.html)p Black Black +Black Black 2085 59 a Fm(\017)p Black 2150 66 a Fs(py\003ate-fast)p +FC(:)18 b(A)h(BZ2)f(decoder)l(.)p Black 2085 176 a Fm(\017)p +Black 2150 183 a Fs(raytrace-simple)p FC(:)h(A)g(ray)g(tracer)l(.)p +Black 2085 292 a Fm(\017)p Black 2150 299 a Fs(richards)p +FC(:)f(The)h(Richards)g(benchmark.)p Black 2085 408 a +Fm(\017)p Black 2150 415 a Fs(spambay)o(es)p FC(:)g(A)g(Bayesian)g +(spam)h(\002lter)3221 383 y Fz(12)3274 415 y FC(.)p Black +2085 527 a Fm(\017)p Black 2150 534 a Fs(telco)p FC(:)34 +b(A)f(Python)i(v)o(ersion)g(of)f(the)g(T)-5 b(elco)33 +b(decimal)i(benchmark)3960 502 y Fz(13)4016 534 y FC(,)2150 +617 y(using)19 b(a)g(pure)h(Python)f(decimal)g(\003oating)g(point)h +(implementation.)p Black 2085 726 a Fm(\017)p Black 2150 +733 a Fs(twisted_names)p FC(:)27 b(A)g(DNS)f(serv)o(er)i(benchmark)h +(using)f(the)g(T)-6 b(wisted)2150 819 y(netw)o(orking)20 +b(frame)n(w)o(ork)2832 788 y Fz(14)2887 819 y FC(.)2141 +993 y(W)-6 b(e)24 b(e)n(v)n(aluate)h(the)g(allocation)f(remo)o(v)n(al)h +(algorithm)g(along)g(tw)o(o)f(lines:)2042 1076 y(\002rst)i(we)h(w)o +(ant)h(to)f(kno)n(w)i(ho)n(w)f(man)o(y)g(allocations)g(could)g(be)f +(optimized)2042 1159 y(a)o(w)o(ay)-5 b(.)18 b(On)f(the)h(other)f(hand,) +h(we)g(w)o(ant)f(to)h(kno)n(w)g(ho)n(w)g(much)g(the)g(run)f(times)2042 +1242 y(of)i(the)g(benchmarks)h(is)f(impro)o(v)o(ed.)2141 +1325 y(The)f(benchmarks)i(were)d(run)h(on)h(an)f(otherwise)g(idle)f +(Intel)h(Core2)g(Duo)2042 1408 y(P8400)25 b(processor)h(with)e(2.26)h +(GHz)g(and)g(3072)h(KB)e(of)h(cache)g(on)h(a)e(ma-)2042 +1491 y(chine)17 b(with)g(3GB)g(RAM)g(running)h(Linux)f(2.6.35.)h(W)-6 +b(e)16 b(compared)i(the)g(per)o(-)2042 1574 y(formance)26 +b(of)f(v)n(arious)h(Python)g(implementations)g(on)f(the)h(benchmarks.) +2042 1657 y(As)34 b(a)g(baseline,)g(we)g(used)h(the)f(standard)h +(Python)g(implementation)g(in)2042 1743 y(C,)22 b(CPython)g(2.6.6)2565 +1711 y Fz(15)2620 1743 y FC(,)g(which)h(uses)g(a)f(bytecode-based)j +(interpreter)l(.)d(Fur)o(-)2042 1826 y(thermore)e(we)g(compared)h +(against)e(Psyco[25)q(])h(1.6,)f(a)h(\(hand-written\))g(e)o(x-)2042 +1909 y(tension)27 b(module)h(to)f(CPython)h(which)f(is)f(a)h +(just-in-time)g(compiler)g(that)2042 1992 y(produces)g(machine)g(code)g +(at)e(run-time.)h(It)g(is)f(not)h(based)h(on)f(traces.)g(Fi-)2042 +2075 y(nally)-5 b(,)21 b(we)g(used)h(tw)o(o)g(v)o(ersions)g(of)f(PyPy') +l(s)g(Python)g(interpreter)h(\(re)n(vision)2042 2161 +y(77823)k(of)e(SVN)g(trunk)2670 2129 y Fz(16)2725 2161 +y FC(\):)g(one)h(including)g(the)g(JIT)f(b)o(ut)g(not)g(optimizing)2042 +2244 y(the)f(traces,)g(and)h(one)h(using)f(the)f(allocation)h(remo)o(v) +n(al)g(optimizations)g(\(as)2042 2327 y(well)18 b(as)h(some)g(minor)h +(other)f(optimizations,)g(such)h(as)f(constant)g(folding\).)2141 +2410 y(As)e(the)g(\002rst)f(step,)h(we)g(counted)i(the)e(occurring)h +(operations)g(in)f(all)g(gen-)2042 2493 y(erated)j(traces)g(before)g +(and)h(after)e(the)h(optimization)g(phase)h(for)f(all)f(bench-)2042 +2576 y(marks.)30 b(The)f(resulting)h(numbers)h(can)f(be)g(seen)g(in)g +(Figure)g(7.)f(The)h(op-)2042 2659 y(timization)h(remo)o(v)o(es)h +(between)g(4\045)g(and)g(90\045)g(and)g(of)g(allocation)g(op-)2042 +2742 y(erations)g(in)f(the)h(traces)f(of)h(the)f(benchmarks.)i(All)e +(benchmarks)i(tak)o(en)2042 2825 y(together)m(,)25 b(the)g +(optimization)h(remo)o(v)o(es)g(70\045)f(percent)h(of)f(allocation)h +(op-)2042 2908 y(erations.)f(The)g(numbers)h(look)f(similar)g(for)f +(reading)i(and)g(writing)f(of)f(at-)2042 2991 y(trib)o(utes.)e(There)g +(are)h(e)n(v)o(en)g(more)g Fr(guard)p Black Black 23 +w FC(operations)g(that)g(are)f(remo)o(v)o(ed,)2042 3074 +y(ho)n(we)n(v)o(er)28 b(there)f(is)f(an)h(additional)h(optimization)f +(that)g(remo)o(v)o(es)g(guards,)2042 3157 y(so)21 b(not)g(all)g(the)g +(remo)o(v)o(ed)h(guards)g(are)f(an)g(ef)n(fect)g(of)g(the)g +(optimization)h(de-)2042 3240 y(scribed)15 b(here)g(\(for)f(technical)h +(reasons,)h(it)d(w)o(ould)j(be)f(v)o(ery)g(hard)g(to)g(separate)2042 +3323 y(the)k(tw)o(o)g(ef)n(fects\).)2141 3406 y(In)25 +b(addition)h(to)e(the)h(count)h(of)e(operations)i(we)f(also)g +(performed)g(time)2042 3489 y(measurements.)19 b(All)f(benchmarks)i +(were)e(run)h(50)g(times)f(in)g(the)g(same)h(pro-)2042 +3572 y(cess,)g(to)g(gi)n(v)o(e)h(the)g(JIT)f(time)g(to)g(produce)i +(machine)f(code.)g(The)f(arithmetic)2042 3655 y(mean)k(of)g(the)g +(times)g(of)g(the)g(last)f(30)i(runs)f(were)g(used)h(as)f(the)g +(result.)f(The)2042 3738 y(errors)e(were)g(computed)h(using)g(a)f +(con\002dence)h(interv)n(al)f(with)g(a)g(95\045)h(con-)2042 +3821 y(\002dence)27 b(le)n(v)o(el)g([16].)f(The)h(results)g(are)f +(reported)i(in)e(Figure)h(8.)f(F)o(or)g(each)2042 3904 +y(implementation)f(the)g(table)g(also)g(reports)g(the)f(speedup)j(that) +d(PyPy)g(with)2042 3987 y(optimization)19 b(achie)n(v)o(es)h(o)o(v)o +(er)f(it.)2141 4070 y(W)m(ith)f(the)g(optimization)g(turned)h(on,)f +(PyPy')l(s)f(Python)i(interpreter)f(out-)2042 4153 y(performs)35 +b(CPython)f(in)g(all)g(benchmarks)i(e)o(xcept)f(spambayes)h(\(which) +2042 4236 y(hea)o(vily)28 b(relies)g(on)h(re)o(gular)g(e)o(xpression)g +(performance)h(and)f(thus)g(is)f(not)2042 4319 y(helped)h(much)f(by)h +(our)f(Python)g(JIT\))f(and)i(meteor)o(-contest.)f(All)f(bench-)2042 +4402 y(marks)c(are)h(impro)o(v)o(ed)g(by)f(the)h(allocation)f(remo)o(v) +n(al)h(optimization,)g(by)f(at)2042 4485 y(least)18 b(20\045)i(and)g +(by)f(as)g(much)h(as)f(a)f(f)o(actor)i(of)f(6.95.)2141 +4568 y(Psyco)i(is)e(able)h(to)g(outperform)h(PyPy')l(s)e(JIT)g(in)h +(\002)n(v)o(e)g(out)g(of)g(14)g(bench-)2042 4651 y(marks.)25 +b(W)-6 b(e)25 b(hope)i(to)e(o)o(v)o(ertak)o(e)h(Psyco)g(\(which)g(is)f +(no)h(longer)g(being)g(ac-)2042 4734 y(ti)n(v)o(ely)19 +b(de)n(v)o(elopped\))i(by)e(adding)h(some)g(further)f(optimizations.)p +Black 2042 4967 V 2042 5018 a Fz(12)2104 5041 y Fr +(http://spambayes.sourceforge.net/)2042 5115 y Fz(13)2104 +5138 y Fr(http://speleotrove.com/decimal/telco.html)2042 +5212 y Fz(14)2104 5235 y Fr(http://twistedmatrix.com/)2042 +5309 y Fz(15)2104 5333 y Fr(http://python.org)2042 5406 +y Fz(16)2104 5430 y Fr(http://codespeak.net/svn/pypy/trunk)p Black Black Black eop end %%Page: 9 9 -TeXDict begin 9 8 bop Black Black Black Black Black 2853 -44 46086 45 v 2831 1041 45 997 v 9667 1041 V 9889 1041 -V 10575 742 a Fw(num)221 b(loops)p 14932 1041 V 2089 -w(ne)-22 b(w)1328 b(remo)-13 b(v)g(ed)p 22874 1041 V -2115 w(get/set)1329 b(remo)-13 b(v)g(ed)p 31702 1041 -V 1968 w(guard)1328 b(remo)-13 b(v)g(ed)p 40088 1041 -V 2090 w(all)222 b(ops)1328 b(remo)-13 b(v)g(ed)p 48916 -1041 V 2853 1085 46086 45 v 2831 2081 45 997 v 3517 1782 -a(crypto_p)k(yaes)p 9667 2081 V 9888 2081 V 5174 w(78)p -14932 2081 V 1771 w(3088)2776 b(50\045)p 22874 2081 V -2213 w(57148)g(25\045)p 31702 2081 V 2213 w(9055)h(95\045)p -40088 2081 V 1770 w(137189)f(80\045)p 48916 2081 V 2831 -3077 V 3517 2778 a(django)p 9667 3077 V 9888 3077 V 7476 -w(51)p 14932 3077 V 2213 w(673)h(54\045)p 22874 3077 -V 2213 w(19318)f(18\045)p 31702 3077 V 2213 w(3876)h(93\045)p -40088 3077 V 2213 w(55682)f(85\045)p 48916 3077 V 2831 -4074 V 3517 3775 a(f)-9 b(annkuch)p 9667 4074 V 9888 -4074 V 6600 w(43)p 14932 4074 V 2213 w(171)2777 b(49\045)p -22874 4074 V 3099 w(886)f(63\045)p 31702 4074 V 2213 -w(1159)h(81\045)p 40088 4074 V 2656 w(4935)f(45\045)p -48916 4074 V 2831 5070 V 3517 4771 a(go)p 9667 5070 V -9888 5070 V 8558 w(517)p 14932 5070 V 1328 w(12234)g(76\045)p -22874 5070 V 1770 w(200842)g(21\045)p 31702 5070 V 1771 -w(53138)g(90\045)p 40088 5070 V 1770 w(568542)g(84\045)p -48916 5070 V 2831 6066 V 3517 5767 a(html5lib)p 9667 -6066 V 9888 6066 V 6442 w(498)p 14932 6066 V 1328 w(14432)g(68\045)p -22874 6066 V 1770 w(503390)g(11\045)p 31702 6066 V 1771 -w(71592)g(94\045)p 40088 6066 V 1327 w(1405780)g(91\045)p -48916 6066 V 2831 7062 V 3517 6764 a(meteor)-18 b(-contest)p -9667 7062 V 9888 7062 V 4643 w(59)p 14932 7062 V 2213 -w(277)2777 b(36\045)p 22874 7062 V 2656 w(4402)f(31\045)p -31702 7062 V 2213 w(1078)h(83\045)p 40088 7062 V 2213 -w(12862)f(68\045)p 48916 7062 V 2831 8059 V 3517 7760 -a(nbody)p 9667 8059 V 9888 8059 V 7672 w(13)p 14932 8059 -V 2656 w(96)h(38\045)p 22874 8059 V 3099 w(443)f(69\045)p -31702 8059 V 2656 w(449)h(78\045)p 40088 8059 V 2656 -w(2107)f(38\045)p 48916 8059 V 2831 9055 V 3517 8756 -a(p)-9 b(y\003ate-f)g(ast)p 9667 9055 V 9888 9055 V 5479 -w(162)p 14932 9055 V 1771 w(2278)2776 b(55\045)p 22874 -9055 V 2213 w(39126)g(20\045)p 31702 9055 V 2213 w(8194)h(92\045)p -40088 9055 V 1770 w(112857)f(80\045)p 48916 9055 V 2831 -10051 V 3517 9752 a(raytrace-simple)p 9667 10051 V 9888 -10051 V 3937 w(120)p 14932 10051 V 1771 w(3118)g(59\045)p -22874 10051 V 2213 w(91982)g(15\045)p 31702 10051 V 1771 -w(13572)g(95\045)p 40088 10051 V 1770 w(247436)g(89\045)p -48916 10051 V 2831 11047 V 3517 10749 a(richards)p 9667 -11047 V 9888 11047 V 7035 w(87)p 14932 11047 V 2213 w(844)3220 -b(4\045)p 22874 11047 V 2213 w(49875)2776 b(22\045)p -31702 11047 V 2213 w(4130)h(91\045)p 40088 11047 V 1770 -w(133898)f(83\045)p 48916 11047 V 2831 12044 V 3517 11745 -a(spambayes)p 9667 12044 V 9888 12044 V 5559 w(314)p -14932 12044 V 1771 w(5608)g(79\045)p 22874 12044 V 1770 -w(117002)g(11\045)p 31702 12044 V 1771 w(25313)g(94\045)p -40088 12044 V 1770 w(324125)g(90\045)p 48916 12044 V -2831 13040 V 3517 12741 a(spectral-norm)p 9667 13040 -V 9888 13040 V 4969 w(38)p 14932 13040 V 2213 w(360)h(64\045)p -22874 13040 V 2656 w(5553)f(20\045)p 31702 13040 V 2213 -w(1122)h(92\045)p 40088 13040 V 2213 w(11878)f(77\045)p -48916 13040 V 2831 14036 V 3517 13737 a(telco)p 9667 -14036 V 9888 14036 V 8166 w(46)p 14932 14036 V 1771 w(1257)g(90\045)p -22874 14036 V 2213 w(37470)3219 b(3\045)p 31702 14036 -V 2213 w(6644)2777 b(99\045)p 40088 14036 V 2213 w(98590)f(97\045)p -48916 14036 V 2831 15033 V 3517 14734 a(twisted-names)p -9667 15033 V 9888 15033 V 4330 w(214)p 14932 15033 V -1771 w(5273)g(84\045)p 22874 15033 V 1770 w(100010)g(10\045)p -31702 15033 V 1771 w(23247)g(96\045)p 40088 15033 V 1770 -w(279667)g(92\045)p 48916 15033 V 2853 15077 46086 45 -v 2831 16073 45 997 v 3517 15774 a(total)p 9667 16073 -V 9888 16073 V 7427 w(2240)p 14932 16073 V 1328 w(49709)g(70\045)p -22874 16073 V 1328 w(1227447)f(14\045)p 31702 16073 V -1328 w(222569)h(93\045)p 40088 16073 V 1327 w(3395548)g(89\045)p -48916 16073 V 2853 16117 46086 45 v -2000 17372 55791 -37 v 10632 18501 a Fs(Figur)-18 b(e)249 b(7.)499 b FC(Number)249 -b(of)g(Operations)h(and)g(Percentage)g(Remo)-15 b(v)g(ed)250 -b(By)f(Optimization)p Black Black Black Black -790 20753 -53371 45 v -812 21749 45 997 v 6322 21749 V 6544 21749 -V 8773 21451 a Fw(CPython)221 b([ms])p 14360 21749 V -2589 w Fm(\002)p 17681 21749 V 17902 21749 V 4077 w Fw(Psyco)h([ms])p -25719 21749 V 2147 w Fm(\002)p 28597 21749 V 28818 21749 -V 1549 w Fw(PyPy)g(w/o)g(optim.)g([ms])p 37921 21749 -V 2146 w Fm(\002)p 40799 21749 V 41020 21749 V 1550 w -Fw(PyPy)f(w/)h(optim.)g([ms])p 49680 21749 V 2146 w Fm(\002)p -52558 21749 V -790 21794 53371 45 v -812 22790 45 997 -v -126 22491 a Fw(crypto_p)-9 b(yaes)p 6322 22790 V 6544 -22790 V 3529 w(2757.80)220 b Fm(\006)h Fw(0.98)p 14360 -22790 V 1328 w(10.33)p 17681 22790 V 17902 22790 V 3321 -w(67.90)g Fm(\006)g Fw(0.47)p 25719 22790 V 1328 w(0.25)p -28597 22790 V 28818 22790 V 3721 w(1652.00)g Fm(\006)g -Fw(4.00)p 37921 22790 V 1328 w(6.19)p 40799 22790 V 41020 -22790 V 3722 w(266.86)f Fm(\006)h Fw(5.94)p 49680 22790 -V 1329 w(1.00)p 52558 22790 V -812 23786 V -126 23487 -a(django)p 6322 23786 V 6544 23786 V 6273 w(993.19)g -Fm(\006)g Fw(0.50)p 14360 23786 V 1771 w(3.83)p 17681 -23786 V 17902 23786 V 2878 w(913.51)g Fm(\006)g Fw(4.22)p -25719 23786 V 1328 w(3.52)p 28597 23786 V 28818 23786 -V 4164 w(694.73)g Fm(\006)g Fw(2.86)p 37921 23786 V 1328 -w(2.68)p 40799 23786 V 41020 23786 V 3722 w(259.53)f -Fm(\006)h Fw(1.79)p 49680 23786 V 1329 w(1.00)p 52558 -23786 V -812 24783 V -126 24484 a(f)-9 b(annkuch)p 6322 -24783 V 6544 24783 V 4955 w(1987.22)220 b Fm(\006)h Fw(2.02)p -14360 24783 V 1771 w(4.26)p 17681 24783 V 17902 24783 -V 2878 w(944.44)g Fm(\006)g Fw(0.61)p 25719 24783 V 1328 -w(2.02)p 28597 24783 V 28818 24783 V 4164 w(566.99)g -Fm(\006)g Fw(1.06)p 37921 24783 V 1328 w(1.21)p 40799 -24783 V 41020 24783 V 3722 w(466.87)f Fm(\006)h Fw(1.85)p -49680 24783 V 1329 w(1.00)p 52558 24783 V -812 25779 -V -126 25480 a(go)p 6322 25779 V 6544 25779 V 7798 w(947.21)g -Fm(\006)g Fw(1.58)p 14360 25779 V 1771 w(3.00)p 17681 -25779 V 17902 25779 V 2878 w(445.96)g Fm(\006)g Fw(0.68)p -25719 25779 V 1328 w(1.41)p 28597 25779 V 28818 25779 -V 3279 w(2197.71)f Fm(\006)h Fw(25.21)p 37921 25779 V -1328 w(6.95)p 40799 25779 V 41020 25779 V 3722 w(316.15)f -Fm(\006)h Fw(9.33)p 49680 25779 V 1329 w(1.00)p 52558 -25779 V -812 26775 V -126 26476 a(html5lib)p 6322 26775 -V 6544 26775 V 4354 w(13987.12)f Fm(\006)h Fw(19.51)p -14360 26775 V 1771 w(1.39)p 17681 26775 V 17902 26775 -V 1550 w(17398.25)f Fm(\006)h Fw(36.50)p 25719 26775 -V 1328 w(1.72)p 28597 26775 V 28818 26775 V 2836 w(27194.45)f -Fm(\006)h Fw(46.62)p 37921 26775 V 1328 w(2.69)p 40799 -26775 V 41020 26775 V 2393 w(10092.19)f Fm(\006)i Fw(23.50)p -49680 26775 V 1328 w(1.00)p 52558 26775 V -812 27771 -V -126 27472 a(meteor)-18 b(-contest)p 6322 27771 V 6544 -27771 V 3440 w(346.98)221 b Fm(\006)g Fw(0.35)p 14360 -27771 V 1771 w(0.88)p 17681 27771 V 17902 27771 V 2878 -w(215.66)g Fm(\006)g Fw(0.23)p 25719 27771 V 1328 w(0.55)p -28597 27771 V 28818 27771 V 4164 w(433.04)g Fm(\006)g -Fw(1.45)p 37921 27771 V 1328 w(1.10)p 40799 27771 V 41020 -27771 V 3722 w(392.85)f Fm(\006)h Fw(0.87)p 49680 27771 -V 1329 w(1.00)p 52558 27771 V -812 28768 V -126 28469 -a(nbody_modi\002ed)p 6322 28768 V 6544 28768 V 2877 w(637.90)g -Fm(\006)g Fw(1.82)p 14360 28768 V 1771 w(6.14)p 17681 -28768 V 17902 28768 V 2878 w(256.78)g Fm(\006)g Fw(0.18)p -25719 28768 V 1328 w(2.47)p 28597 28768 V 28818 28768 -V 4164 w(135.55)g Fm(\006)g Fw(0.33)p 37921 28768 V 1328 -w(1.30)p 40799 28768 V 41020 28768 V 3722 w(103.93)f -Fm(\006)h Fw(0.25)p 49680 28768 V 1329 w(1.00)p 52558 -28768 V -812 29764 V -126 29465 a(p)-9 b(y\003ate-f)g(ast)p -6322 29764 V 6544 29764 V 4277 w(3169.35)220 b Fm(\006)h -Fw(1.89)p 14360 29764 V 1771 w(1.74)p 17681 29764 V 17902 -29764 V 2435 w(1278.16)g Fm(\006)g Fw(3.13)p 25719 29764 -V 1328 w(0.70)p 28597 29764 V 28818 29764 V 3721 w(3285.89)g -Fm(\006)g Fw(8.51)p 37921 29764 V 1328 w(1.80)p 40799 -29764 V 41020 29764 V 2836 w(1822.36)f Fm(\006)i Fw(11.52)p -49680 29764 V 1328 w(1.00)p 52558 29764 V -812 30760 -V -126 30461 a(raytrace-simple)p 6322 30760 V 6544 30760 -V 2292 w(2744.60)e Fm(\006)h Fw(51.72)p 14360 30760 V -1771 w(4.24)p 17681 30760 V 17902 30760 V 2435 w(1072.66)g -Fm(\006)g Fw(1.08)p 25719 30760 V 1328 w(1.66)p 28597 -30760 V 28818 30760 V 3279 w(2778.27)f Fm(\006)h Fw(15.13)p -37921 30760 V 1328 w(4.29)p 40799 30760 V 41020 30760 -V 3722 w(647.24)f Fm(\006)h Fw(5.44)p 49680 30760 V 1329 -w(1.00)p 52558 30760 V -812 31756 V -126 31458 a(richards)p -6322 31756 V 6544 31756 V 5832 w(354.06)g Fm(\006)g Fw(1.00)p -14360 31756 V 1771 w(4.01)p 17681 31756 V 17902 31756 -V 3321 w(63.48)g Fm(\006)g Fw(0.15)p 25719 31756 V 1328 -w(0.72)p 28597 31756 V 28818 31756 V 4164 w(383.93)g -Fm(\006)g Fw(3.28)p 37921 31756 V 1328 w(4.35)p 40799 -31756 V 41020 31756 V 4164 w(88.32)g Fm(\006)g Fw(0.91)p -49680 31756 V 1329 w(1.00)p 52558 31756 V -812 32753 -V -126 32454 a(spambayes)p 6322 32753 V 6544 32753 V -4799 w(299.16)g Fm(\006)g Fw(0.35)p 14360 32753 V 1771 -w(0.75)p 17681 32753 V 17902 32753 V 2878 w(338.68)g -Fm(\006)g Fw(3.14)p 25719 32753 V 1328 w(0.85)p 28597 -32753 V 28818 32753 V 3721 w(580.90)g Fm(\006)g Fw(24.68)p -37921 32753 V 1328 w(1.46)p 40799 32753 V 41020 32753 -V 3279 w(397.37)f Fm(\006)i Fw(10.60)p 49680 32753 V -1328 w(1.00)p 52558 32753 V -812 33749 V -126 33450 a(spectral-norm)p -6322 33749 V 6544 33749 V 3766 w(478.63)f Fm(\006)g Fw(0.80)p -14360 33749 V 1771 w(4.27)p 17681 33749 V 17902 33749 -V 2878 w(139.83)g Fm(\006)g Fw(1.54)p 25719 33749 V 1328 -w(1.25)p 28597 33749 V 28818 33749 V 4164 w(353.51)g -Fm(\006)g Fw(1.39)p 37921 33749 V 1328 w(3.15)p 40799 -33749 V 41020 33749 V 3722 w(112.10)f Fm(\006)h Fw(1.17)p -49680 33749 V 1329 w(1.00)p 52558 33749 V -812 34745 -V -126 34446 a(telco)p 6322 34745 V 6544 34745 V 6521 -w(1207.67)f Fm(\006)h Fw(2.03)p 14360 34745 V 1771 w(2.44)p -17681 34745 V 17902 34745 V 2878 w(730.00)g Fm(\006)g -Fw(2.66)p 25719 34745 V 1328 w(1.47)p 28597 34745 V 28818 -34745 V 3721 w(1296.08)g Fm(\006)g Fw(4.37)p 37921 34745 -V 1328 w(2.62)p 40799 34745 V 41020 34745 V 3722 w(495.23)f -Fm(\006)h Fw(2.14)p 49680 34745 V 1329 w(1.00)p 52558 -34745 V -812 35741 V -126 35443 a(twisted_names)p 6322 -35741 V 6544 35741 V 4308 w(9.58)g Fm(\006)g Fw(0.01)p -14360 35741 V 1771 w(1.34)p 17681 35741 V 17902 35741 -V 3321 w(10.43)g Fm(\006)g Fw(0.01)p 25719 35741 V 1328 -w(1.46)p 28597 35741 V 28818 35741 V 4607 w(17.99)g Fm(\006)g -Fw(0.27)p 37921 35741 V 1328 w(2.52)p 40799 35741 V 41020 -35741 V 4607 w(7.13)g Fm(\006)g Fw(0.09)p 49680 35741 -V 1329 w(1.00)p 52558 35741 V -790 35786 53371 45 v -2000 -37040 55791 37 v 6051 38169 a Fs(Figur)-18 b(e)250 b(8.)498 -b FC(Benchmark)250 b(T)-35 b(imes)248 b(in)i(Milliseconds,)e(T)-80 -b(ogether)250 b(W)-40 b(ith)250 b(F)-15 b(actor)250 b(Ov)-15 -b(er)249 b(PyPy)g(W)-40 b(ith)250 b(Optimizations)p Black --2000 41484 a FA(7.)1218 b(Related)305 b(W)-91 b(ork)-2000 -43034 y FC(There)373 b(e)-15 b(xists)373 b(a)g(lar)-18 -b(ge)374 b(number)f(of)g(w)-10 b(orks)373 b(on)g(escape)h(analysis,)f -(which)-2000 44141 y(is)438 b(a)h(program)f(analysis)h(that)g(tries)f -(to)g(\002nd)h(an)g(upper)f(bound)i(for)e(the)-2000 45248 -y(lifetime)480 b(of)g(objects)h(allocated)g(at)g(speci\002c)f(program)h -(points)f([4,)g(11,)-2000 46355 y(18,)348 b(24].)g(This)f(information)g -(can)i(then)f(be)g(used)g(to)f(decide)i(that)f(certain)-2000 -47462 y(objects)341 b(can)h(be)g(allocated)g(on)f(the)h(stack,)f -(because)h(their)g(lifetime)f(does)-2000 48569 y(not)269 -b(e)-15 b(xceed)271 b(that)f(of)f(the)g(stack)h(frame)f(it)g(is)g -(allocated)h(in.)g(The)f(dif)-25 b(ference)-2000 49676 -y(to)303 b(our)h(w)-10 b(ork)303 b(is)g(that)h(escape)g(analysis)f(is)g -(split)f(into)i(an)g(analysis)f(and)h(an)-2000 50783 -y(optimization)327 b(phase.)f(The)g(analysis)f(can)i(be)f(a)g(lot)g -(more)g(comple)-15 b(x)326 b(than)-2000 51890 y(our)388 -b(simple)g(one-pass)g(optimization.)h(Also,)f(stack-allocation)h -(reduces)-2000 52997 y(g)-5 b(arbage-collection)293 b(pressure)d(b)-20 -b(ut)291 b(does)g(not)g(optimize)g(a)-15 b(w)-10 b(ay)292 -b(the)f(actual)-2000 54104 y(accesses)283 b(to)f(the)h(stack-allocated) -h(object.)f(In)f(our)h(case,)g(an)g(object)g(is)f(not)-2000 -55211 y(needed)250 b(at)g(all)f(an)-15 b(y)250 b(more.)-672 -56318 y(Chang)219 b Fx(et)g(al.)f FC(describe)h(a)f(tracing)h(JIT)f -(for)g(Ja)-20 b(v)-25 b(aScript)219 b(running)f(on)h(top)-2000 -57425 y(of)276 b(a)g(JVM)g([10].)f(The)-15 b(y)277 b(mention)f(in)g -(passing)g(an)h(approach)g(to)f(allocation)-2000 58531 -y(remo)-15 b(v)-25 b(al)399 b(that)g(mo)-15 b(v)g(es)398 -b(the)h(allocation)g(of)f(an)h(object)g(of)f(type)g(1)h(out)f(of)-2000 -59638 y(the)422 b(loop)g(to)g(only)g(allocate)g(it)g(once,)h(instead)f -(of)f(e)-25 b(v)-15 b(ery)422 b(iteration.)h(No)-2000 -60745 y(details)196 b(are)g(gi)-25 b(v)-15 b(en)197 b(for)e(this)h -(optimization.)g(The)g(f)-10 b(act)197 b(that)f(the)g(object)g(is)g -(still)-2000 61852 y(allocated)238 b(and)g(needs)f(to)g(be)g(written)g -(to)g(means)h(that)f(only)g(the)g(allocations)-2000 62959 -y(are)377 b(optimized)g(a)-15 b(w)-10 b(ay)-65 b(,)377 -b(b)-20 b(ut)377 b(not)f(the)h(reads)f(out)g(of)h(and)f(writes)g(into)h -(the)-2000 64066 y(object.)-672 65173 y(SPUR,)282 b(a)g(tracing)h(JIT)e -(for)g(C#)h(seems)f(to)h(be)h(able)f(to)g(remo)-15 b(v)g(e)283 -b(alloca-)-2000 66280 y(tions)294 b(in)g(a)h(similar)e(w)-10 -b(ay)295 b(to)f(the)h(approach)g(described)g(here,)g(as)f(hinted)h(at) --2000 67387 y(in)212 b(the)f(technical)i(report)f([3].)f(Ho)-25 -b(we)g(v)-15 b(er)-40 b(,)213 b(no)f(details)g(for)f(the)h(approach)g -(and)-2000 68494 y(its)249 b(implementation)h(are)g(gi)-25 -b(v)-15 b(en.)-672 69601 y(Psyco)294 b([25])f(is)h(a)g(\(non-tracing\)) -g(JIT)e(for)h(Python)i(that)f(implements)g(a)-2000 70708 -y(more)262 b(ad-hoc)h(v)-15 b(ersion)263 b(of)f(the)g(allocation)h -(remo)-15 b(v)-25 b(al)263 b(described)g(here.)g(Our)-2000 -71815 y(static)292 b(objects)h(could)g(be)f(related)h(to)g(what)f(are)h -(called)g Fx(virtual)g FC(objects)g(in)p Black Black -27224 41484 a(Psyco.)328 b(Historically)-65 b(,)328 b(PyPy')-55 -b(s)328 b(JIT)e(can)j(be)f(seen)h(as)e(some)h(successor)g(of)27224 -42591 y(Psyco)278 b(for)g(a)h(general)g(conte)-15 b(xt)279 -b(\(one)g(of)f(the)h(authors)f(of)h(this)f(paper)h(is)e(the)27224 -43698 y(author)249 b(of)g(Psyco\).)28552 44805 y(The)493 -b(original)f(SELF)g(JIT)f(compiler)i([9])f(used)h(an)f(algorithm)h(for) -27224 45912 y(forw)-10 b(ard-propag)-5 b(ating)295 b(the)f(types)h(of)f -(v)-25 b(ariables)294 b(as)g(part)h(of)f(its)f(optimiza-)27224 -47019 y(tions.)206 b(This)f(mak)-10 b(es)207 b(it)f(possible)g(to)h -(remo)-15 b(v)g(e)207 b(all)g(type)f(checks)i(on)e(a)h(v)-25 -b(ariable)27224 48126 y(b)-20 b(ut)247 b(the)h(\002rst)f(one.)h(The)f -(optimization)i(does)e(not)h(deal)g(with)f(remo)-15 b(ving)248 -b(the)27224 49233 y(full)253 b(object,)h(if)e(it)h(is)g(short-li)-25 -b(v)-15 b(ed,)253 b(b)-20 b(ut)253 b(the)h(type)f(check)i(remo)-15 -b(v)-25 b(als)253 b(are)g(sim-)27224 50340 y(ilar)c(to)g(what)h(our)f -(optimization)h(achie)-25 b(v)-15 b(es.)28552 51447 y(P)g(artially)204 -b(kno)-25 b(wn)205 b(data)f(structures)g(are)g(b)-20 -b(uilt)204 b(directly)g(into)g(Prolog)g(\(via)27224 52554 -y(unbound)317 b(logic)h(v)-25 b(ariables\))317 b(and)g(thus)g(the)g -(treatment)h(of)e(partially)i(static)27224 53661 y(data)358 -b(structures)f(w)-10 b(as)358 b(part)g(of)g(partial)g(e)-25 -b(v)g(aluation)360 b(of)d(Prolog)h(programs)27224 54768 -y(from)239 b(the)i(early)f(stages)g([21].)g(One)h(ef)-25 -b(fect)240 b(of)g(unfolding)h(in)f(Prolog)g(is)f(that)27224 -55875 y(terms)350 b(that)h(are)g(constructed)h(and)f(immediately)h -(matched)g(ag)-5 b(ain,)351 b(com-)27224 56982 y(pletely)336 -b(disappear)f(in)h(the)f(residual)h(program.)f(This)g(is)f(similar)h -(to)g(what)27224 58089 y(our)417 b(optimization)h(does)f(for)g(an)h -(imperati)-25 b(v)-15 b(e)418 b(language.)h(In)e(functional)27224 -59196 y(programming)216 b(this)f(idea)h(w)-10 b(as)216 -b(introduced)g(as)g(constructor)f(specialisation)27224 -60303 y(by)249 b(Mogensen)h([23].)28552 61410 y(A)453 -b(related)g(optimization)g(is)f(also)g(that)h(of)f(deforestation)h -([17,)g(27])27224 62517 y(which)306 b(remo)-15 b(v)g(es)305 -b(intermediate)i(lists)d(or)h(trees)g(in)h(functional)g(languages.) -27224 63623 y(A)290 b(more)f(general)i(approach)g(is)e(boxing)h -(analysis)g([20])f(which)h(optimizes)27224 64730 y(pairs)437 -b(of)h(calls)g(to)g(box/unbox)h(in)f(a)g(functional)h(language.)g -(Similarly)-65 b(,)27224 65837 y("dynamic)464 b(typing")g([19])f(tries) -g(to)g(remo)-15 b(v)g(e)463 b(dynamic)i(type)e(coercions)27224 -66944 y(in)384 b(a)g(dynamically)h(typed)g(lambda-calculus.)g(All)f -(these)g(optimizations)27224 68051 y(w)-10 b(ork)433 -b(by)f(analyzing)i(the)f(program)g(before)g(e)-15 b(x)g(ecution,)434 -b(which)f(mak)-10 b(es)27224 69158 y(them)282 b(unsuitable)g(for)g -(dynamic)h(languages)g(lik)-10 b(e)282 b(Python,)g(where)h(almost)27224 -70265 y(nothing)249 b(can)h(be)g(inferred)f(purely)g(by)h(looking)f(at) -h(the)f(source)h(code.)p Black Black eop end +TeXDict begin 9 8 bop Black Black Black Black Black 214 +3 3457 4 v 212 78 4 75 v 725 78 V 742 78 V 793 56 a Fw(num)17 +b(loops)p 1120 78 V 157 w(ne)n(w)101 b(remo)o(v)o(ed)p +1716 78 V 160 w(get/set)h(remo)o(v)o(ed)p 2378 78 V 149 +w(guard)f(remo)o(v)o(ed)p 3007 78 V 158 w(all)18 b(ops)100 +b(remo)o(v)o(ed)p 3669 78 V 214 81 3457 4 v 212 156 4 +75 v 264 134 a(crypto_p)o(yaes)p 725 156 V 742 156 V +391 w(78)p 1120 156 V 134 w(3088)209 b(50\045)p 1716 +156 V 166 w(57148)h(25\045)p 2378 156 V 167 w(9055)f(95\045)p +3007 156 V 133 w(137189)h(80\045)p 3669 156 V 212 231 +V 264 208 a(django)p 725 231 V 742 231 V 562 w(51)p 1120 +231 V 167 w(673)f(54\045)p 1716 231 V 166 w(19318)h(18\045)p +2378 231 V 167 w(3876)f(93\045)p 3007 231 V 166 w(55682)h(85\045)p +3669 231 V 212 306 V 264 283 a(f)o(annkuch)p 725 306 +V 742 306 V 497 w(43)p 1120 306 V 167 w(171)f(49\045)p +1716 306 V 233 w(886)g(63\045)p 2378 306 V 167 w(1159)g(81\045)p +3007 306 V 200 w(4935)g(45\045)p 3669 306 V 212 380 V +264 358 a(go)p 725 380 V 742 380 V 642 w(517)p 1120 380 +V 100 w(12234)h(76\045)p 1716 380 V 133 w(200842)g(21\045)p +2378 380 V 133 w(53138)g(90\045)p 3007 380 V 133 w(568542)g(84\045)p +3669 380 V 212 455 V 264 433 a(html5lib)p 725 455 V 742 +455 V 485 w(498)p 1120 455 V 100 w(14432)g(68\045)p 1716 +455 V 133 w(503390)g(11\045)p 2378 455 V 133 w(71592)g(94\045)p +3007 455 V 100 w(1405780)g(91\045)p 3669 455 V 212 530 +V 264 507 a(meteor)o(-contest)p 725 530 V 742 530 V 351 +w(59)p 1120 530 V 167 w(277)f(36\045)p 1716 530 V 200 +w(4402)g(31\045)p 2378 530 V 167 w(1078)g(83\045)p 3007 +530 V 166 w(12862)h(68\045)p 3669 530 V 212 604 V 264 +582 a(nbody)p 725 604 V 742 604 V 576 w(13)p 1120 604 +V 200 w(96)f(38\045)p 1716 604 V 233 w(443)g(69\045)p +2378 604 V 200 w(449)g(78\045)p 3007 604 V 200 w(2107)g(38\045)p +3669 604 V 212 679 V 264 657 a(p)o(y\003ate-f)o(ast)p +725 679 V 742 679 V 414 w(162)p 1120 679 V 134 w(2278)g(55\045)p +1716 679 V 166 w(39126)h(20\045)p 2378 679 V 167 w(8194)f(92\045)p +3007 679 V 133 w(112857)h(80\045)p 3669 679 V 212 754 +V 264 731 a(raytrace-simple)p 725 754 V 742 754 V 299 +w(120)p 1120 754 V 134 w(3118)f(59\045)p 1716 754 V 166 +w(91982)h(15\045)p 2378 754 V 133 w(13572)g(95\045)p +3007 754 V 133 w(247436)g(89\045)p 3669 754 V 212 829 +V 264 806 a(richards)p 725 829 V 742 829 V 529 w(87)p +1120 829 V 167 w(844)242 b(4\045)p 1716 829 V 166 w(49875)210 +b(22\045)p 2378 829 V 167 w(4130)f(91\045)p 3007 829 +V 133 w(133898)h(83\045)p 3669 829 V 212 903 V 264 881 +a(spambayes)p 725 903 V 742 903 V 418 w(314)p 1120 903 +V 134 w(5608)f(79\045)p 1716 903 V 133 w(117002)h(11\045)p +2378 903 V 133 w(25313)g(94\045)p 3007 903 V 133 w(324125)g(90\045)p +3669 903 V 212 978 V 264 956 a(spectral-norm)p 725 978 +V 742 978 V 375 w(38)p 1120 978 V 167 w(360)f(64\045)p +1716 978 V 200 w(5553)g(20\045)p 2378 978 V 167 w(1122)g(92\045)p +3007 978 V 166 w(11878)h(77\045)p 3669 978 V 212 1053 +V 264 1030 a(telco)p 725 1053 V 742 1053 V 614 w(46)p +1120 1053 V 134 w(1257)f(90\045)p 1716 1053 V 166 w(37470)243 +b(3\045)p 2378 1053 V 167 w(6644)209 b(99\045)p 3007 +1053 V 166 w(98590)h(97\045)p 3669 1053 V 212 1127 V +264 1105 a(twisted-names)p 725 1127 V 742 1127 V 327 +w(214)p 1120 1127 V 134 w(5273)f(84\045)p 1716 1127 V +133 w(100010)h(10\045)p 2378 1127 V 133 w(23247)g(96\045)p +3007 1127 V 133 w(279667)g(92\045)p 3669 1127 V 214 1131 +3457 4 v 212 1205 4 75 v 264 1183 a(total)p 725 1205 +V 742 1205 V 559 w(2240)p 1120 1205 V 100 w(49709)g(70\045)p +1716 1205 V 100 w(1227447)g(14\045)p 2378 1205 V 100 +w(222569)g(93\045)p 3007 1205 V 100 w(3395548)g(89\045)p +3669 1205 V 214 1209 3457 4 v -150 1303 4185 3 v 797 +1388 a Fs(Figur)o(e)18 b(7.)38 b FC(Number)19 b(of)g(Operations)h(and)f +(Percentage)h(Remo)o(v)o(ed)g(By)f(Optimization)p Black +Black Black Black -59 1556 4003 4 v -61 1631 4 75 v 474 +1631 V 491 1631 V 658 1609 a Fw(CPython)f([ms])p 1077 +1631 V 194 w Fm(\002)p 1326 1631 V 1343 1631 V 306 w +Fw(Psyco)f([ms])p 1929 1631 V 161 w Fm(\002)p 2145 1631 +V 2161 1631 V 116 w Fw(PyPy)g(w/o)g(optim.)g([ms])p 2844 +1631 V 161 w Fm(\002)p 3060 1631 V 3077 1631 V 116 w +Fw(PyPy)g(w/)g(optim.)g([ms])p 3726 1631 V 161 w Fm(\002)p +3942 1631 V -59 1635 4003 4 v -61 1709 4 75 v -9 1687 +a Fw(crypto_p)o(yaes)p 474 1709 V 491 1709 V 268 w(2757.80)g +Fm(\006)g Fw(0.98)p 1077 1709 V 99 w(10.33)p 1326 1709 +V 1343 1709 V 250 w(67.90)g Fm(\006)g Fw(0.47)p 1929 +1709 V 99 w(0.25)p 2145 1709 V 2161 1709 V 280 w(1652.00)g +Fm(\006)g Fw(4.00)p 2844 1709 V 100 w(6.19)p 3060 1709 +V 3077 1709 V 279 w(266.86)g Fm(\006)g Fw(5.94)p 3726 +1709 V 99 w(1.00)p 3942 1709 V -61 1784 V -9 1762 a(django)p +474 1784 V 491 1784 V 472 w(993.19)g Fm(\006)g Fw(0.50)p +1077 1784 V 133 w(3.83)p 1326 1784 V 1343 1784 V 216 +w(913.51)g Fm(\006)g Fw(4.22)p 1929 1784 V 99 w(3.52)p +2145 1784 V 2161 1784 V 313 w(694.73)g Fm(\006)g Fw(2.86)p +2844 1784 V 100 w(2.68)p 3060 1784 V 3077 1784 V 279 +w(259.53)g Fm(\006)g Fw(1.79)p 3726 1784 V 99 w(1.00)p +3942 1784 V -61 1859 V -9 1836 a(f)o(annkuch)p 474 1859 +V 491 1859 V 374 w(1987.22)g Fm(\006)g Fw(2.02)p 1077 +1859 V 133 w(4.26)p 1326 1859 V 1343 1859 V 216 w(944.44)g +Fm(\006)g Fw(0.61)p 1929 1859 V 99 w(2.02)p 2145 1859 +V 2161 1859 V 313 w(566.99)g Fm(\006)g Fw(1.06)p 2844 +1859 V 100 w(1.21)p 3060 1859 V 3077 1859 V 279 w(466.87)g +Fm(\006)g Fw(1.85)p 3726 1859 V 99 w(1.00)p 3942 1859 +V -61 1933 V -9 1911 a(go)p 474 1933 V 491 1933 V 585 +w(947.21)g Fm(\006)g Fw(1.58)p 1077 1933 V 133 w(3.00)p +1326 1933 V 1343 1933 V 216 w(445.96)g Fm(\006)g Fw(0.68)p +1929 1933 V 99 w(1.41)p 2145 1933 V 2161 1933 V 247 w(2197.71)g +Fm(\006)f Fw(25.21)p 2844 1933 V 101 w(6.95)p 3060 1933 +V 3077 1933 V 279 w(316.15)h Fm(\006)g Fw(9.33)p 3726 +1933 V 99 w(1.00)p 3942 1933 V -61 2008 V -9 1986 a(html5lib)p +474 2008 V 491 2008 V 328 w(13987.12)h Fm(\006)e Fw(19.51)p +1077 2008 V 134 w(1.39)p 1326 2008 V 1343 2008 V 116 +w(17398.25)i Fm(\006)e Fw(36.50)p 1929 2008 V 100 w(1.72)p +2145 2008 V 2161 2008 V 213 w(27194.45)i Fm(\006)e Fw(46.62)p +2844 2008 V 101 w(2.69)p 3060 2008 V 3077 2008 V 179 +w(10092.19)i Fm(\006)e Fw(23.50)p 3726 2008 V 100 w(1.00)p +3942 2008 V -61 2083 V -9 2060 a(meteor)o(-contest)p +474 2083 V 491 2083 V 261 w(346.98)h Fm(\006)g Fw(0.35)p +1077 2083 V 133 w(0.88)p 1326 2083 V 1343 2083 V 216 +w(215.66)g Fm(\006)g Fw(0.23)p 1929 2083 V 99 w(0.55)p +2145 2083 V 2161 2083 V 313 w(433.04)g Fm(\006)g Fw(1.45)p +2844 2083 V 100 w(1.10)p 3060 2083 V 3077 2083 V 279 +w(392.85)g Fm(\006)g Fw(0.87)p 3726 2083 V 99 w(1.00)p +3942 2083 V -61 2158 V -9 2135 a(nbody_modi\002ed)p 474 +2158 V 491 2158 V 218 w(637.90)g Fm(\006)g Fw(1.82)p +1077 2158 V 133 w(6.14)p 1326 2158 V 1343 2158 V 216 +w(256.78)g Fm(\006)g Fw(0.18)p 1929 2158 V 99 w(2.47)p +2145 2158 V 2161 2158 V 313 w(135.55)g Fm(\006)g Fw(0.33)p +2844 2158 V 100 w(1.30)p 3060 2158 V 3077 2158 V 279 +w(103.93)g Fm(\006)g Fw(0.25)p 3726 2158 V 99 w(1.00)p +3942 2158 V -61 2232 V -9 2210 a(p)o(y\003ate-f)o(ast)p +474 2232 V 491 2232 V 324 w(3169.35)g Fm(\006)g Fw(1.89)p +1077 2232 V 133 w(1.74)p 1326 2232 V 1343 2232 V 183 +w(1278.16)g Fm(\006)g Fw(3.13)p 1929 2232 V 99 w(0.70)p +2145 2232 V 2161 2232 V 280 w(3285.89)g Fm(\006)g Fw(8.51)p +2844 2232 V 100 w(1.80)p 3060 2232 V 3077 2232 V 212 +w(1822.36)h Fm(\006)e Fw(11.52)p 3726 2232 V 100 w(1.00)p +3942 2232 V -61 2307 V -9 2285 a(raytrace-simple)p 474 +2307 V 491 2307 V 175 w(2744.60)i Fm(\006)e Fw(51.72)p +1077 2307 V 134 w(4.24)p 1326 2307 V 1343 2307 V 183 +w(1072.66)h Fm(\006)g Fw(1.08)p 1929 2307 V 99 w(1.66)p +2145 2307 V 2161 2307 V 247 w(2778.27)g Fm(\006)f Fw(15.13)p +2844 2307 V 101 w(4.29)p 3060 2307 V 3077 2307 V 279 +w(647.24)h Fm(\006)g Fw(5.44)p 3726 2307 V 99 w(1.00)p +3942 2307 V -61 2382 V -9 2359 a(richards)p 474 2382 +V 491 2382 V 439 w(354.06)g Fm(\006)g Fw(1.00)p 1077 +2382 V 133 w(4.01)p 1326 2382 V 1343 2382 V 249 w(63.48)g +Fm(\006)g Fw(0.15)p 1929 2382 V 99 w(0.72)p 2145 2382 +V 2161 2382 V 313 w(383.93)g Fm(\006)g Fw(3.28)p 2844 +2382 V 100 w(4.35)p 3060 2382 V 3077 2382 V 312 w(88.32)g +Fm(\006)g Fw(0.91)p 3726 2382 V 99 w(1.00)p 3942 2382 +V -61 2456 V -9 2434 a(spambayes)p 474 2456 V 491 2456 +V 361 w(299.16)g Fm(\006)g Fw(0.35)p 1077 2456 V 133 +w(0.75)p 1326 2456 V 1343 2456 V 216 w(338.68)g Fm(\006)g +Fw(3.14)p 1929 2456 V 99 w(0.85)p 2145 2456 V 2161 2456 +V 280 w(580.90)g Fm(\006)f Fw(24.68)p 2844 2456 V 101 +w(1.46)p 3060 2456 V 3077 2456 V 246 w(397.37)h Fm(\006)f +Fw(10.60)p 3726 2456 V 100 w(1.00)p 3942 2456 V -61 2531 +V -9 2509 a(spectral-norm)p 474 2531 V 491 2531 V 285 +w(478.63)h Fm(\006)g Fw(0.80)p 1077 2531 V 133 w(4.27)p +1326 2531 V 1343 2531 V 216 w(139.83)g Fm(\006)g Fw(1.54)p +1929 2531 V 99 w(1.25)p 2145 2531 V 2161 2531 V 313 w(353.51)g +Fm(\006)g Fw(1.39)p 2844 2531 V 100 w(3.15)p 3060 2531 +V 3077 2531 V 279 w(112.10)g Fm(\006)g Fw(1.17)p 3726 +2531 V 99 w(1.00)p 3942 2531 V -61 2606 V -9 2583 a(telco)p +474 2606 V 491 2606 V 491 w(1207.67)g Fm(\006)g Fw(2.03)p +1077 2606 V 133 w(2.44)p 1326 2606 V 1343 2606 V 216 +w(730.00)g Fm(\006)g Fw(2.66)p 1929 2606 V 99 w(1.47)p +2145 2606 V 2161 2606 V 280 w(1296.08)g Fm(\006)g Fw(4.37)p +2844 2606 V 100 w(2.62)p 3060 2606 V 3077 2606 V 279 +w(495.23)g Fm(\006)g Fw(2.14)p 3726 2606 V 99 w(1.00)p +3942 2606 V -61 2681 V -9 2658 a(twisted_names)p 474 +2681 V 491 2681 V 325 w(9.58)g Fm(\006)g Fw(0.01)p 1077 +2681 V 133 w(1.34)p 1326 2681 V 1343 2681 V 249 w(10.43)g +Fm(\006)g Fw(0.01)p 1929 2681 V 99 w(1.46)p 2145 2681 +V 2161 2681 V 346 w(17.99)g Fm(\006)g Fw(0.27)p 2844 +2681 V 100 w(2.52)p 3060 2681 V 3077 2681 V 345 w(7.13)g +Fm(\006)g Fw(0.09)p 3726 2681 V 99 w(1.00)p 3942 2681 +V -59 2684 4003 4 v -150 2778 4185 3 v 454 2863 a Fs(Figur)o(e)h(8.)37 +b FC(Benchmark)20 b(T)m(imes)f(in)g(Milliseconds,)g(T)-6 +b(ogether)20 b(W)m(ith)e(F)o(actor)h(Ov)o(er)f(PyPy)h(W)m(ith)f +(Optimizations)p Black -150 3111 a FA(7.)91 b(Related)22 +b(W)-7 b(ork)-150 3228 y FC(There)28 b(e)o(xists)g(a)g(lar)o(ge)g +(number)i(of)e(w)o(orks)h(on)f(escape)i(analysis,)e(which)-150 +3311 y(is)33 b(a)g(program)h(analysis)f(that)g(tries)g(to)g(\002nd)g +(an)h(upper)g(bound)g(for)f(the)-150 3394 y(lifetime)i(of)i(objects)f +(allocated)h(at)f(speci\002c)g(program)i(points)e([4,)g(11)q(,)-150 +3477 y(18)q(,)25 b(24)q(].)h(This)g(information)h(can)f(then)h(be)g +(used)g(to)f(decide)h(that)f(certain)-150 3560 y(objects)g(can)h(be)f +(allocated)g(on)g(the)g(stack,)g(because)h(their)f(lifetime)f(does)-150 +3643 y(not)c(e)o(xceed)g(that)f(of)h(the)f(stack)h(frame)g(it)e(is)h +(allocated)h(in.)f(The)h(dif)n(ference)-150 3726 y(to)i(our)g(w)o(ork)h +(is)e(that)h(escape)h(analysis)g(is)e(split)h(into)g(an)g(analysis)g +(and)h(an)-150 3809 y(optimization)h(phase.)g(The)g(analysis)g(can)g +(be)g(a)g(lot)f(more)h(comple)o(x)h(than)-150 3892 y(our)k(simple)f +(one-pass)i(optimization.)e(Also,)g(stack-allocation)h(reduces)-150 +3975 y(garbage-collection)23 b(pressure)g(b)o(ut)f(does)h(not)f +(optimize)g(a)o(w)o(ay)h(the)f(actual)-150 4058 y(accesses)g(to)g(the)f +(stack-allocated)h(object.)g(In)f(our)h(case,)g(an)f(object)h(is)f(not) +-150 4141 y(needed)f(at)f(all)f(an)o(y)i(more.)-50 4224 +y(Chang)d Fx(et)f(al.)h FC(describe)g(a)f(tracing)h(JIT)f(for)h(Ja)o(v) +n(aScript)f(running)i(on)f(top)-150 4307 y(of)k(a)g(JVM)g([10)q(].)f +(The)o(y)h(mention)h(in)e(passing)i(an)f(approach)i(to)e(allocation) +-150 4390 y(remo)o(v)n(al)31 b(that)f(mo)o(v)o(es)g(the)h(allocation)f +(of)g(an)h(object)f(of)g(type)h(1)f(out)g(of)-150 4473 +y(the)i(loop)g(to)g(only)h(allocate)f(it)f(once,)h(instead)g(of)g(e)n +(v)o(ery)h(iteration.)e(No)-150 4556 y(details)15 b(are)g(gi)n(v)o(en)g +(for)g(this)g(optimization.)g(The)g(f)o(act)g(that)g(the)g(object)g(is) +f(still)-150 4639 y(allocated)k(and)h(needs)g(to)f(be)g(written)g(to)g +(means)g(that)g(only)h(the)f(allocations)-150 4722 y(are)28 +b(optimized)i(a)o(w)o(ay)-5 b(,)28 b(b)o(ut)h(not)f(the)h(reads)g(out)f +(of)h(and)g(writes)f(into)g(the)-150 4805 y(object.)-50 +4888 y(SPUR,)19 b(a)j(tracing)f(JIT)g(for)g(C#)h(seems)f(to)h(be)f +(able)h(to)f(remo)o(v)o(e)h(alloca-)-150 4971 y(tions)g(in)h(a)f +(similar)f(w)o(ay)i(to)f(the)h(approach)h(described)f(here,)f(as)h +(hinted)g(at)-150 5054 y(in)16 b(the)g(technical)h(report)f([3].)g(Ho)n +(we)n(v)o(er)m(,)g(no)h(details)f(for)g(the)g(approach)i(and)-150 +5137 y(its)g(implementation)i(are)f(gi)n(v)o(en.)-50 +5220 y(Psyco)j([25)q(])g(is)g(a)g(\(non-tracing\))h(JIT)f(for)g(Python) +h(that)f(implements)g(a)-150 5303 y(more)e(ad-hoc)h(v)o(ersion)g(of)f +(the)g(allocation)g(remo)o(v)n(al)g(described)h(here.)f(Our)-150 +5386 y(static)i(objects)g(could)h(be)g(related)f(to)g(what)g(are)g +(called)h Fx(virtual)f FC(objects)g(in)p Black Black +2042 3111 a(Psyco.)j(Historically)-5 b(,)24 b(PyPy')l(s)g(JIT)g(can)i +(be)f(seen)g(as)g(some)g(successor)h(of)2042 3194 y(Psyco)21 +b(for)g(a)g(general)h(conte)o(xt)g(\(one)f(of)g(the)h(authors)g(of)f +(this)f(paper)i(is)f(the)2042 3277 y(author)e(of)g(Psyco\).)2141 +3360 y(The)38 b(original)f(SELF)e(JIT)i(compiler)g([9)q(])f(used)i(an)g +(algorithm)f(for)2042 3443 y(forw)o(ard-propagating)24 +b(the)e(types)h(of)f(v)n(ariables)h(as)g(part)f(of)g(its)g(optimiza-) +2042 3526 y(tions.)15 b(This)h(mak)o(es)g(it)f(possible)i(to)e(remo)o +(v)o(e)i(all)e(type)h(checks)h(on)f(a)g(v)n(ariable)2042 +3609 y(b)o(ut)i(the)h(\002rst)f(one.)h(The)g(optimization)g(does)h(not) +f(deal)g(with)f(remo)o(ving)i(the)2042 3692 y(full)f(object,)g(if)f(it) +h(is)g(short-li)n(v)o(ed,)g(b)o(ut)g(the)g(type)h(check)g(remo)o(v)n +(als)g(are)f(sim-)2042 3775 y(ilar)f(to)h(what)g(our)g(optimization)h +(achie)n(v)o(es.)2141 3859 y(P)o(artially)15 b(kno)n(wn)i(data)e +(structures)h(are)g(b)o(uilt)e(directly)i(into)g(Prolog)f(\(via)2042 +3942 y(unbound)26 b(logic)e(v)n(ariables\))h(and)f(thus)h(the)f +(treatment)g(of)g(partially)g(static)2042 4025 y(data)j(structures)g(w) +o(as)h(part)f(of)g(partial)g(e)n(v)n(aluation)h(of)f(Prolog)g(programs) +2042 4108 y(from)18 b(the)g(early)h(stages)f([21)q(].)f(One)i(ef)n +(fect)f(of)g(unfolding)i(in)e(Prolog)g(is)g(that)2042 +4191 y(terms)26 b(that)g(are)h(constructed)h(and)f(immediately)g +(matched)g(again,)g(com-)2042 4274 y(pletely)e(disappear)i(in)e(the)g +(residual)h(program.)g(This)f(is)g(similar)g(to)g(what)2042 +4357 y(our)32 b(optimization)g(does)g(for)f(an)h(imperati)n(v)o(e)g +(language.)h(In)e(functional)2042 4440 y(programming)18 +b(this)e(idea)g(w)o(as)h(introduced)h(as)e(constructor)i +(specialisation)2042 4523 y(by)h(Mogensen)i([23)q(].)2141 +4606 y(A)34 b(related)h(optimization)f(is)g(also)g(that)g(of)g +(deforestation)h([17)q(,)f(27])2042 4689 y(which)23 b(remo)o(v)o(es)h +(intermediate)f(lists)g(or)g(trees)g(in)g(functional)h(languages.)2042 +4772 y(A)d(more)i(general)f(approach)i(is)d(boxing)j(analysis)e([20)q +(])f(which)i(optimizes)2042 4855 y(pairs)33 b(of)g(calls)g(to)g +(box/unbox)i(in)e(a)g(functional)h(language.)g(Similarly)-5 +b(,)2042 4938 y("dynamic)36 b(typing")h([19])e(tries)f(to)h(remo)o(v)o +(e)h(dynamic)g(type)f(coercions)2042 5021 y(in)29 b(a)g(dynamically)h +(typed)g(lambda-calculus.)g(All)e(these)h(optimizations)2042 +5104 y(w)o(ork)k(by)g(analyzing)h(the)f(program)g(before)h(e)o(x)o +(ecution,)f(which)g(mak)o(es)2042 5187 y(them)21 b(unsuitable)i(for)e +(dynamic)h(languages)h(lik)o(e)f(Python,)f(where)h(almost)2042 +5270 y(nothing)e(can)f(be)g(inferred)h(purely)f(by)h(looking)g(at)e +(the)h(source)h(code.)p Black Black eop end %%Page: 10 10 -TeXDict begin 10 9 bop Black Black -2000 886 a FA(8.)1218 -b(Conclusion)304 b(and)h(Futur)-22 b(e)304 b(W)-91 b(ork)-2000 -2435 y FC(In)368 b(this)g(paper)-40 b(,)370 b(we)f(used)g(an)g -(approach)h(based)e(on)h(online)g(partial)g(e)-25 b(v)g(al-)-2000 -3542 y(uation)330 b(to)g(optimize)g(a)-15 b(w)-10 b(ay)331 -b(allocations)f(and)g(type)g(guards)f(in)h(the)g(traces)-2000 -4649 y(of)273 b(a)g(tracing)h(JIT)-74 b(.)272 b(In)h(this)g(conte)-15 -b(xt)274 b(a)f(simple)g(approach)i(based)e(on)h(partial)-2000 -5756 y(e)-25 b(v)g(aluation)229 b(gi)-25 b(v)-15 b(es)228 -b(good)g(results.)f(This)g(is)g(due)i(to)e(the)h(f)-10 -b(act)228 b(that)g(the)g(tracing)-2000 6863 y(JIT)243 -b(itself)h(is)g(responsible)g(for)g(all)h(control)g(issues,)e(which)i -(are)g(usually)g(the)-2000 7970 y(hardest)327 b(part)g(of)f(partial)h -(e)-25 b(v)g(aluation:)328 b(the)f(tracing)g(JIT)f(selects)g(the)h -(parts)-2000 9077 y(of)260 b(the)h(program)g(that)g(are)g(w)-10 -b(orthwhile)261 b(to)f(optimize,)i(and)f(e)-15 b(xtracts)261 -b(linear)-2000 10184 y(paths)250 b(through)h(them,)g(inlining)f -(functions)h(as)f(necessary)-65 b(.)251 b(What)g(is)f(left)g(to)-2000 -11291 y(optimize)g(are)f(only)h(those)f(linear)h(paths.)-672 -12398 y(W)-80 b(e)326 b(e)-15 b(xpect)326 b(a)f(similar)f(result)g(for) -g(other)h(optimizations)g(that)g(usually)-2000 13505 -y(require)236 b(a)g(comple)-15 b(x)236 b(analysis)g(phase)g(and)g(are)g -(thus)f(normally)h(too)f(slo)-25 b(w)236 b(to)-2000 14612 -y(use)295 b(at)g(runtime.)g(A)g(tracing)g(JIT)f(selects)h(interesting)f -(linear)i(paths)e(by)h(it-)-2000 15719 y(self;)215 b(therefore,)g(a)g -(nai)-25 b(v)-15 b(e)216 b(v)-15 b(ersion)215 b(of)g(man)-15 -b(y)216 b(optimizations)g(on)f(such)g(paths)-2000 16826 -y(should)207 b(gi)-25 b(v)-15 b(e)207 b(mostly)f(the)h(same)g(results.) -f(F)-15 b(or)206 b(e)-15 b(xample,)208 b(we)f(e)-15 b(xperimented)-2000 -17933 y(with)324 b(\(and)g(plan)g(to)f(write)h(about\))g(store-load)f -(propag)-5 b(ation)325 b(with)f(a)g(v)-15 b(ery)-2000 -19040 y(simple)249 b(alias)g(analysis.)-2000 21902 y -FA(Ackno)-12 b(wledgements)-2000 23452 y FC(The)261 b(authors)h(w)-10 -b(ould)262 b(lik)-10 b(e)261 b(to)h(thank)f(Stef)-10 -b(an)262 b(Hallerstede,)g(Da)-20 b(vid)262 b(Schnei-)-2000 -24559 y(der)351 b(and)h(Thomas)f(Stiehl)g(for)g(fruitful)f(discussions) -g(and)i(detailed)g(feed-)-2000 25666 y(back)284 b(during)g(the)f -(writing)h(of)f(the)h(paper)-55 b(.)284 b(W)-80 b(e)284 -b(thank)g(the)g(anon)-15 b(ymous)284 b(re-)-2000 26773 -y(vie)-25 b(wers)249 b(for)g(the)h(v)-25 b(aluable)250 -b(comments.)-2000 29635 y FA(Refer)-22 b(ences)p Black --1557 31074 a Fw([1])p Black 387 w(D.)202 b(Ancona,)g(M.)h(Ancona,)f -(A.)g(Cuni,)f(and)h(N.)g(D.)g(Matsakis.)278 b(RPython:)202 -b(a)f(step)-137 32070 y(to)-22 b(w)-9 b(ards)298 b(reconciling)g -(dynamically)g(and)f(statically)j(typed)d(OO)h(languages.)-137 -33066 y(In)239 b Fa(Pr)-40 b(oceedings)241 b(of)f(the)g(2007)f -(symposium)i(on)f(Dynamic)h(langua)-9 b(g)g(es)p Fw(,)240 -b(pages)-137 34063 y(53\22664,)220 b(Montreal,)j(Quebec,)f(Canada,)g -(2007.)f(A)-35 b(CM.)p Black -1557 35444 a([2])p Black -387 w(V)-114 b(.)377 b(Bala,)f(E.)h(Duesterw)-9 b(ald,)379 -b(and)d(S.)h(Banerjia.)815 b(Dynamo:)376 b(a)h(transparent)-137 -36440 y(dynamic)300 b(optimization)h(system.)574 b Fa(A)-27 -b(CM)301 b(SIGPLAN)f(Notices)p Fw(,)i(35\(5\):1\22612,)-137 -37436 y(2000.)p Black -1557 38817 a([3])p Black 387 w(M.)398 -b(Bebenita,)g(F)-71 b(.)398 b(Brandner)-35 b(,)397 b(M.)h(F)-13 -b(ahndrich,)398 b(F)-71 b(.)398 b(Logozzo,)g(W)-81 b(.)397 -b(Schulte,)-137 39814 y(N.)322 b(T)-31 b(illmann,)323 -b(and)e(H.)i(V)-98 b(enter)-49 b(.)640 b(SPUR:)323 b(a)f(trace-based)h -(JIT)f(compiler)f(for)-137 40810 y(CIL.)744 b(In)354 -b Fa(Pr)-40 b(oceedings)355 b(of)f(the)g(A)-27 b(CM)354 -b(international)h(confer)-33 b(ence)355 b(on)f(Ob-)-137 -41806 y(ject)199 b(oriented)h(pr)-40 b(o)-9 b(gr)c(amming)199 -b(systems)i(langua)-9 b(g)g(es)199 b(and)f(applications)p -Fw(,)i(pages)-137 42802 y(708\226725,)220 b(Reno/T)-71 -b(ahoe,)222 b(Ne)-22 b(v)g(ada,)222 b(USA,)h(2010.)d(A)-35 -b(CM.)p Black -1557 44184 a([4])p Black 387 w(B.)330 -b(Blanchet.)668 b(Escape)331 b(analysis)g(for)f(Ja)-18 -b(v)c(a:)331 b(Theory)f(and)g(practice.)668 b Fa(A)-27 -b(CM)-137 45180 y(T)-49 b(r)-13 b(ans.)222 b(Pr)-40 b(o)-9 -b(gr)c(am.)223 b(Lang)-13 b(.)221 b(Syst.)p Fw(,)i(25\(6\):713\226775,) -d(2003.)p Black -1557 46561 a([5])p Black 387 w(C.)257 -b(F)-71 b(.)258 b(Bolz,)g(A.)g(Cuni,)f(M.)h(Fija\007k)-9 -b(o)-22 b(wski,)259 b(and)e(A.)h(Rigo.)434 b(T)-31 b(racing)257 -b(the)h(meta-)-137 47557 y(le)-22 b(v)-13 b(el:)183 b(PyPy')-49 -b(s)183 b(tracing)g(JIT)g(compiler)-49 b(.)236 b(In)182 -b Fa(Pr)-40 b(oceedings)184 b(of)f(the)g(4th)f(workshop)-137 -48553 y(on)197 b(the)i(Implementation,)g(Compilation,)f(Optimization)h -(of)f(Object-Oriented)-137 49550 y(Langua)-9 b(g)g(es)342 -b(and)g(Pr)-40 b(o)-9 b(gr)c(amming)343 b(Systems)p Fw(,)h(pages)f -(18\22625,)f(Geno)-13 b(v)-22 b(a,)342 b(Italy)-58 b(,)-137 -50546 y(2009.)221 b(A)-35 b(CM.)p Black -1557 51927 a([6])p -Black 387 w(C.)180 b(F)-71 b(.)181 b(Bolz,)f(A.)h(K)-13 -b(uhn,)180 b(A.)h(Lienhard,)f(N.)h(Matsakis,)h(O.)f(Nierstrasz,)h(L.)f -(Reng-)-137 52923 y(gli,)f(A.)h(Rigo,)f(and)g(T)-66 b(.)181 -b(V)-98 b(erw)-9 b(aest.)231 b(Back)180 b(to)g(the)g(future)g(in)g(one) -g(week)g(\227)g(imple-)-137 53919 y(menting)272 b(a)h(Smalltalk)h(VM)f -(in)g(PyPy.)483 b(In)272 b Fa(Self-Sustaining)h(Systems)p -Fw(,)i(pages)-137 54916 y(123\226139.)220 b(2008.)p Black --1557 56297 a([7])p Black 387 w(C.)343 b(F)-71 b(.)344 -b(Bolz,)g(M.)g(Leuschel,)g(and)f(D.)h(Schneider)-49 b(.)710 -b(T)-71 b(o)-22 b(w)-9 b(ards)344 b(a)g(jitting)f(VM)-137 -57293 y(for)278 b(Prolog)h(e)-13 b(x)g(ecution.)502 b(In)279 -b Fa(Pr)-40 b(oceedings)280 b(of)f(the)g(12th)f(international)h(A)-27 -b(CM)-137 58289 y(SIGPLAN)292 b(symposium)i(on)e(Principles)i(and)d(pr) --13 b(actice)293 b(of)g(declar)-13 b(ative)293 b(pr)-40 -b(o-)-137 59286 y(gr)-13 b(amming)p Fw(,)222 b(pages)g(99\226108,)e -(Hagenber)-16 b(g,)222 b(Austria,)h(2010.)e(A)-35 b(CM.)p -Black -1557 60667 a([8])p Black 387 w(C.)338 b(Bruni)g(and)g(T)-66 -b(.)339 b(V)-98 b(erw)-9 b(aest.)695 b(PyGirl:)339 b(generating)f -(Whole-System)h(VMs)-137 61663 y(from)241 b(High-Le)-22 -b(v)-13 b(el)241 b(prototypes)g(using)h(PyPy.)383 b(In)241 -b(W)-81 b(.)241 b(Aalst,)j(J.)e(Mylopoulos,)-137 62659 -y(N.)380 b(M.)h(Sadeh,)f(M.)g(J.)h(Sha)-13 b(w)-58 b(,)381 -b(C.)e(Szyperski,)i(M.)f(Oriol,)h(and)e(B.)h(Me)-13 b(yer)-35 -b(,)-137 63655 y(editors,)394 b Fa(Objects,)h(Components,)f(Models)f -(and)g(P)-71 b(atterns)p Fw(,)395 b(v)-18 b(olume)393 -b(33)g(of)-137 64652 y Fa(Lectur)-33 b(e)364 b(Notes)g(in)e(Business)j -(Information)e(Pr)-40 b(ocessing)p Fw(,)365 b(pages)e(328\226347.)-137 -65648 y(Springer)221 b(Berlin)g(Heidelber)-16 b(g,)223 -b(2009.)318 b(10.1007/978-3-642-02571-6_19.)p Black -1557 -67029 a([9])p Black 387 w(C.)276 b(Chambers,)g(D.)g(Ung)l(ar)-35 -b(,)276 b(and)f(E.)i(Lee.)493 b(An)276 b(ef)-22 b(\002cient)277 -b(implementation)f(of)-137 68025 y(SELF)283 b(a)g(dynamically-typed)g -(object-oriented)g(language)f(based)h(on)g(proto-)-137 -69022 y(types.)319 b Fa(SIGPLAN)222 b(Not.)p Fw(,)g(24\(10\):49\22670,) -e(1989.)p Black -2000 70403 a([10])p Black 387 w(M.)230 -b(Chang,)g(M.)g(Bebenita,)g(A.)h(Y)-89 b(ermolo)-13 b(vich,)230 -b(A.)h(Gal,)g(and)e(M.)i(Franz.)346 b(Ef)-22 b(\002-)-137 -71399 y(cient)257 b(just-in-time)h(e)-13 b(x)g(ecution)256 -b(of)h(dynamically)g(typed)f(languages)h(via)g(code)-137 -72395 y(specialization)359 b(using)f(precise)h(runtime)f(type)g -(inference.)757 b(T)-62 b(echnical)358 b(Re-)p Black -Black 29087 886 a(port)c(ICS-TR-07-10,)g(Donald)g(Bren)h(School)f(of)h -(Information)f(and)g(Com-)29087 1882 y(puter)221 b(Science,)i(Uni)-22 -b(v)-13 b(ersity)221 b(of)g(California,)h(Irvine,)g(2007.)p -Black 27224 3210 a([11])p Black 387 w(J.)200 b(Choi,)g(M.)g(Gupta,)h -(M.)f(Serrano,)g(V)-114 b(.)201 b(C.)f(Sreedhar)-35 b(,)200 -b(and)f(S.)i(Midkif)-22 b(f.)272 b(Escape)29087 4206 -y(analysis)222 b(for)f(Ja)-18 b(v)c(a.)320 b Fa(SIGPLAN)222 -b(Not.)p Fw(,)g(34\(10\):1\22619,)e(1999.)p Black 27224 -5535 a([12])p Black 387 w(A.)273 b(Cuni.)483 b Fa(High)273 -b(performance)h(implementation)g(of)e(Python)h(for)g(CLI/.NET)29087 -6531 y(with)365 b(JIT)i(compiler)g(g)-9 b(ener)c(ation)366 -b(for)g(dynamic)h(langua)-9 b(g)g(es.)782 b Fw(PhD)366 -b(thesis,)29087 7527 y(Dipartimento)283 b(di)f(Informatica)h(e)g -(Scienze)h(dell'Informazione,)e(Uni)-22 b(v)-13 b(ersity)29087 -8524 y(of)221 b(Geno)-13 b(v)-22 b(a,)221 b(2010.)318 -b(T)-62 b(echnical)223 b(Report)d(DISI-TH-2010-05.)p -Black 27224 9852 a([13])p Black 387 w(A.)428 b(Gal,)g(B.)g(Eich,)g(M.)g -(Sha)-18 b(v)-13 b(er)-35 b(,)428 b(D.)g(Anderson,)g(B.)f(Kaplan,)i(G.) -f(Hoare,)29087 10848 y(D.)460 b(Mandelin,)g(B.)g(Zbarsk)-13 -b(y)-58 b(,)460 b(J.)h(Orendorf)-22 b(f,)459 b(M.)i(Bebenita,)f(M.)g -(Chang,)29087 11844 y(M.)176 b(Franz,)g(E.)g(Smith,)g(R.)g(Reitmaier) --35 b(,)176 b(and)f(M.)h(Haghighat.)220 b(T)-31 b(race-based)176 -b(just-)29087 12841 y(in-time)221 b(type)h(specialization)h(for)e -(dynamic)g(languages.)319 b(In)222 b Fa(PLDI)p Fw(,)g(2009.)p -Black 27224 14169 a([14])p Black 387 w(A.)372 b(Gal)h(and)f(M.)g -(Franz.)801 b(Incremental)373 b(dynamic)f(code)f(generation)h(with) -29087 15165 y(trace)267 b(trees.)467 b(T)-62 b(echnical)269 -b(Report)d(ICS-TR-06-16,)h(Donald)g(Bren)g(School)g(of)29087 -16162 y(Information)297 b(and)h(Computer)g(Science,)h(Uni)-22 -b(v)-13 b(ersity)299 b(of)f(California,)h(Irvine,)29087 -17158 y(No)-13 b(v)-58 b(.)221 b(2006.)p Black 27224 -18486 a([15])p Black 387 w(A.)330 b(Gal,)h(C.)f(W)-81 -b(.)329 b(Probst,)i(and)e(M.)i(Franz.)666 b(HotpathVM:)331 -b(an)e(ef)-22 b(fecti)g(v)-13 b(e)330 b(JIT)29087 19482 -y(compiler)411 b(for)g(resource-constrained)h(de)-22 -b(vices.)929 b(In)411 b Fa(Pr)-40 b(oceedings)412 b(of)g(the)29087 -20479 y(2nd)179 b(international)h(confer)-33 b(ence)181 -b(on)e(V)-66 b(irtual)181 b(e)-18 b(xecution)181 b(en)-35 -b(vir)-40 b(onments)p Fw(,)181 b(pages)29087 21475 y(144\226153,)220 -b(Otta)-13 b(w)k(a,)223 b(Ontario,)f(Canada,)g(2006.)f(A)-35 -b(CM.)p Black 27224 22803 a([16])p Black 387 w(A.)239 -b(Geor)-16 b(ges,)239 b(D.)g(Buytaert,)g(and)f(L.)h(Eeckhout.)373 -b(Statistically)241 b(rigorous)d(Ja)-18 b(v)c(a)29087 -23800 y(performance)221 b(e)-22 b(v)g(aluation.)319 b -Fa(SIGPLAN)221 b(Not.)p Fw(,)h(42\(10\):57\22676,)e(2007.)p -Black 27224 25128 a([17])p Black 387 w(A.)287 b(Gill,)h(J.)g(Launchb) --18 b(ury)-58 b(,)287 b(and)g(S.)h(L.)f(P)-98 b(.)287 -b(Jones.)530 b(A)287 b(short)g(cut)g(to)g(deforesta-)29087 -26124 y(tion.)603 b(In)310 b Fa(Pr)-40 b(oceedings)312 -b(of)e(the)h(confer)-33 b(ence)311 b(on)f(Functional)g(pr)-40 -b(o)-9 b(gr)c(amming)29087 27121 y(langua)k(g)g(es)209 -b(and)g(computer)i(ar)-33 b(c)-13 b(hitectur)-33 b(e)p -Fw(,)212 b(FPCA)e('93,)g(page)f(223\226232,)g(Ne)-22 -b(w)29087 28117 y(Y)-97 b(ork,)221 b(NY)-114 b(,)222 -b(USA,)h(1993.)e(A)-35 b(CM.)p Black 27224 29445 a([18])p -Black 387 w(B.)213 b(Goldber)-16 b(g)213 b(and)g(Y)-114 -b(.)214 b(G.)g(P)-13 b(ark.)301 b(Higher)214 b(order)f(escape)h -(analysis:)h(optimizing)29087 30441 y(stack)265 b(allocation)g(in)f -(functional)g(program)g(implementations.)458 b(In)264 -b Fa(Pr)-40 b(oceed-)29087 31438 y(ings)223 b(of)g(the)h(thir)-33 -b(d)224 b(Eur)-40 b(opean)223 b(symposium)i(on)d(pr)-40 -b(o)-9 b(gr)c(amming)224 b(on)f(ESOP)h('90)p Fw(,)29087 -32434 y(pages)320 b(152\226160,)f(Copenhagen,)h(Denmark,)h(1990.)e -(Springer)-18 b(-V)-98 b(erlag)321 b(Ne)-22 b(w)29087 -33430 y(Y)-97 b(ork,)221 b(Inc.)p Black 27224 34759 a([19])p -Black 387 w(F)-71 b(.)251 b(Henglein.)411 b(Dynamic)251 -b(typing:)f(syntax)g(and)g(proof)f(theory)-58 b(.)411 -b Fa(Sci.)251 b(Comput.)29087 35755 y(Pr)-40 b(o)-9 b(gr)c(am.)p -Fw(,)222 b(22:197\226230,)f(June)g(1994.)p Black 27224 -37083 a([20])p Black 387 w(J.)395 b(J\370r)-16 b(gensen.)875 -b Fa(A)394 b(Calculus)h(for)g(Boxing)g(Analysis)h(of)e(P)-71 -b(olymorphically)29087 38079 y(T)-66 b(yped)201 b(Langua)-9 -b(g)g(es)p Fw(.)274 b(Ph.D.)202 b(Thesis,)g(Uni)-22 b(v)-13 -b(ersity)201 b(of)f(Copenhapen,)g(1996.)273 b(TR)29087 -39076 y(96/28.)p Black 27224 40404 a([21])p Black 387 -w(J.)312 b(W)-81 b(.)312 b(Llo)-9 b(yd)311 b(and)g(J.)i(C.)e -(Shepherdson.)608 b(P)-13 b(artial)313 b(e)-22 b(v)g(aluation)311 -b(in)h(logic)f(pro-)29087 41400 y(gramming.)318 b Fa(J)-22 -b(.)222 b(Lo)-9 b(g)c(.)222 b(Pr)-40 b(o)-9 b(gr)c(am.)p -Fw(,)223 b(11\(3-4\):217\226242,)c(1991.)p Black 27224 -42729 a([22])p Black 387 w(E.)175 b(Miranda.)220 b(Conte)-13 -b(xt)174 b(management)i(in)f(V)-53 b(isualW)-71 b(orks)176 -b(5i.)220 b(T)-62 b(echnical)176 b(report,)29087 43725 -y(P)-13 b(arcPlace)222 b(Di)-22 b(vision,)223 b(CINCOM,)f(Inc.,)g -(1999.)p Black 27224 45053 a([23])p Black 387 w(T)-66 -b(.)413 b(Mogensen.)933 b(Constructor)412 b(specialization.)934 -b(In)413 b Fa(Pr)-40 b(oceedings)414 b(of)f(the)29087 -46050 y(1993)180 b(A)-27 b(CM)180 b(SIGPLAN)h(symposium)i(on)d(P)-71 -b(artial)182 b(e)-13 b(valuation)180 b(and)g(semantics-)29087 -47046 y(based)339 b(pr)-40 b(o)-9 b(gr)c(am)341 b(manipulation)p -Fw(,)f(pages)g(22\22632,)f(Copenhagen,)g(Denmark,)29087 -48042 y(1993.)220 b(A)-35 b(CM.)p Black 27224 49370 a([24])p -Black 387 w(Y)-114 b(.)225 b(G.)h(P)-13 b(ark)225 b(and)g(B.)g(Goldber) --16 b(g.)330 b(Escape)226 b(analysis)g(on)f(lists.)332 -b Fa(SIGPLAN)225 b(Not.)p Fw(,)29087 50367 y(27\(7\):116\226127,)219 -b(1992.)p Black 27224 51695 a([25])p Black 387 w(A.)403 -b(Rigo.)899 b(Representation-based)403 b(just-in-time)h(specialization) -g(and)f(the)29087 52691 y(Psyco)298 b(prototype)g(for)g(Python.)565 -b(In)298 b Fa(Pr)-40 b(oceedings)299 b(of)f(the)g(2004)g(A)-27 -b(CM)298 b(SIG-)29087 53688 y(PLAN)190 b(symposium)i(on)f(P)-71 -b(artial)191 b(e)-13 b(valuation)190 b(and)g(semantics-based)i(pr)-40 -b(o)-9 b(gr)c(am)29087 54684 y(manipulation)p Fw(,)221 -b(pages)h(15\22626,)e(V)-98 b(erona,)222 b(Italy)-58 -b(,)223 b(2004.)d(A)-35 b(CM.)p Black 27224 56012 a([26])p -Black 387 w(A.)349 b(Rigo)g(and)g(S.)h(Pedroni.)728 b(PyPy')-49 -b(s)350 b(approach)f(to)g(virtual)h(machine)f(con-)29087 -57008 y(struction.)620 b(In)315 b Fa(Companion)f(to)i(the)f(21st)h(A) --27 b(CM)315 b(SIGPLAN)h(confer)-33 b(ence)316 b(on)29087 -58005 y(Object-oriented)j(pr)-40 b(o)-9 b(gr)c(amming)318 -b(systems,)i(langua)-9 b(g)g(es,)318 b(and)f(applications)p -Fw(,)29087 59001 y(pages)221 b(944\226953,)g(Portland,)h(Ore)-13 -b(gon,)221 b(USA,)i(2006.)e(A)-35 b(CM.)p Black 27224 -60329 a([27])p Black 387 w(P)-98 b(.)316 b(W)-71 b(adler)-49 -b(.)622 b(Deforestation:)317 b(transforming)f(programs)g(to)g -(eliminate)g(trees.)29087 61326 y(In)218 b Fa(Pr)-40 -b(oceedings)220 b(of)f(the)g(Second)f(Eur)-40 b(opean)219 -b(Symposium)g(on)f(Pr)-40 b(o)-9 b(gr)c(amming)p Fw(,)29087 -62322 y(page)230 b(231\226248,)f(Amsterdam,)i(The)f(Netherlands,)i(The) -e(Netherlands,)h(1988.)29087 63318 y(North-Holland)221 -b(Publishing)h(Co.)p Black Black eop end +TeXDict begin 10 9 bop Black Black -150 66 a FA(8.)91 +b(Conclusion)21 b(and)h(Futur)n(e)g(W)-7 b(ork)-150 183 +y FC(In)28 b(this)f(paper)m(,)i(we)f(used)g(an)g(approach)i(based)f(on) +f(online)g(partial)g(e)n(v)n(al-)-150 266 y(uation)e(to)e(optimize)i(a) +o(w)o(ay)f(allocations)h(and)f(type)h(guards)f(in)g(the)g(traces)-150 +349 y(of)c(a)f(tracing)h(JIT)-6 b(.)21 b(In)f(this)h(conte)o(xt)g(a)g +(simple)f(approach)j(based)e(on)h(partial)-150 432 y(e)n(v)n(aluation)d +(gi)n(v)o(es)e(good)i(results.)e(This)g(is)g(due)h(to)f(the)g(f)o(act)h +(that)f(the)g(tracing)-150 515 y(JIT)h(itself)g(is)g(responsible)i(for) +e(all)g(control)h(issues,)g(which)f(are)h(usually)g(the)-150 +598 y(hardest)25 b(part)g(of)g(partial)f(e)n(v)n(aluation:)i(the)f +(tracing)g(JIT)f(selects)g(the)h(parts)-150 681 y(of)20 +b(the)g(program)g(that)g(are)g(w)o(orthwhile)g(to)f(optimize,)h(and)h +(e)o(xtracts)e(linear)-150 764 y(paths)h(through)g(them,)f(inlining)g +(functions)h(as)f(necessary)-5 b(.)20 b(What)e(is)h(left)f(to)-150 +847 y(optimize)h(are)g(only)h(those)f(linear)g(paths.)-50 +930 y(W)-6 b(e)24 b(e)o(xpect)h(a)f(similar)g(result)h(for)f(other)h +(optimizations)g(that)g(usually)-150 1013 y(require)18 +b(a)g(comple)o(x)h(analysis)f(phase)h(and)g(are)f(thus)g(normally)g +(too)g(slo)n(w)g(to)-150 1096 y(use)23 b(at)f(runtime.)g(A)g(tracing)h +(JIT)e(selects)i(interesting)f(linear)g(paths)h(by)g(it-)-150 +1179 y(self;)16 b(therefore,)g(a)h(nai)n(v)o(e)g(v)o(ersion)g(of)f(man) +o(y)h(optimizations)g(on)g(such)g(paths)-150 1262 y(should)g(gi)n(v)o +(e)f(mostly)g(the)g(same)g(results.)f(F)o(or)g(e)o(xample,)h(we)g(e)o +(xperimented)-150 1345 y(with)24 b(\(and)h(plan)g(to)f(write)g(about\)) +h(store-load)h(propagation)f(with)f(a)h(v)o(ery)-150 +1428 y(simple)19 b(alias)g(analysis.)-150 1619 y FA(Ackno)o +(wledgements)-150 1735 y FC(The)h(authors)g(w)o(ould)h(lik)o(e)f(to)g +(thank)g(Stef)o(an)g(Hallerstede,)g(Da)o(vid)f(Schnei-)-150 +1818 y(der)27 b(and)g(Thomas)g(Stiehl)f(for)g(fruitful)g(discussions)i +(and)f(detailed)g(feed-)-150 1901 y(back)22 b(during)h(the)e(writing)g +(of)h(the)f(paper)l(.)h(W)-6 b(e)21 b(thank)h(the)g(anon)o(ymous)i(re-) +-150 1984 y(vie)n(wers)19 b(for)g(the)g(v)n(aluable)h(comments.)-150 +2175 y FA(Refer)n(ences)p Black -117 2283 a Fw([1])p +Black 30 w(D.)14 b(Ancona,)j(M.)d(Ancona,)j(A.)d(Cuni,)i(and)g(N.)f(D.) +g(Matsakis.)22 b(RPython:)17 b(a)f(step)-10 2358 y(to)n(w)o(ards)24 +b(reconciling)i(dynamically)g(and)d(statically)j(typed)e(OO)e +(languages.)-10 2433 y(In)c Fa(Pr)m(oceedings)i(of)f(the)g(2007)g +(symposium)g(on)f(Dynamic)i(langua)o(g)o(es)p Fw(,)g(pages)-10 +2507 y(53\22664,)d(Montreal,)i(Quebec,)f(Canada,)g(2007.)f(A)m(CM.)p +Black -117 2607 a([2])p Black 30 w(V)-9 b(.)28 b(Bala,)h(E.)f(Duesterw) +o(ald,)j(and)e(S.)f(Banerjia.)63 b(Dynamo:)30 b(a)e(transparent)-10 +2682 y(dynamic)c(optimization)i(system.)43 b Fa(A)n(CM)23 +b(SIGPLAN)f(Notices)p Fw(,)j(35\(5\):1\22612,)-10 2756 +y(2000.)p Black -117 2856 a([3])p Black 30 w(M.)k(Bebenita,)k(F)-5 +b(.)29 b(Brandner)m(,)j(M.)d(F)o(ahndrich,)j(F)-5 b(.)29 +b(Logozzo,)i(W)-6 b(.)29 b(Schulte,)-10 2931 y(N.)23 +b(T)n(illmann,)i(and)g(H.)f(V)-7 b(enter)l(.)49 b(SPUR:)25 +b(a)g(trace-based)i(JIT)d(compiler)i(for)-10 3005 y(CIL.)55 +b(In)27 b Fa(Pr)m(oceedings)i(of)e(the)h(A)n(CM)e(international)31 +b(confer)n(ence)e(on)e(Ob-)-10 3080 y(ject)17 b(oriented)g(pr)m(o)o(gr) +o(amming)f(systems)g(langua)o(g)o(es)i(and)d(applications)p +Fw(,)j(pages)-10 3155 y(708\226725,)f(Reno/T)-5 b(ahoe,)19 +b(Ne)n(v)n(ada,)g(USA,)d(2010.)h(A)m(CM.)p Black -117 +3254 a([4])p Black 30 w(B.)24 b(Blanchet.)53 b(Escape)26 +b(analysis)h(for)e(Ja)o(v)n(a:)h(Theory)g(and)f(practice.)53 +b Fa(A)n(CM)-10 3329 y(T)l(r)o(ans.)16 b(Pr)m(o)o(gr)o(am.)h(Lang)o(.)f +(Syst.)p Fw(,)h(25\(6\):713\226775,)i(2003.)p Black -117 +3429 a([5])p Black 30 w(C.)g(F)-5 b(.)18 b(Bolz,)j(A.)e(Cuni,)h(M.)f +(Fija\007k)o(o)n(wski,)j(and)e(A.)f(Rigo.)33 b(T)n(racing)21 +b(the)f(meta-)-10 3504 y(le)n(v)o(el:)c(PyPy')l(s)f(tracing)h(JIT)d +(compiler)l(.)20 b(In)14 b Fa(Pr)m(oceedings)h(of)g(the)g(4th)f +(workshop)-10 3578 y(on)h(the)h(Implementation,)i(Compilation,)g +(Optimization)g(of)d(Object-Oriented)-10 3653 y(Langua)o(g)o(es)28 +b(and)e(Pr)m(o)o(gr)o(amming)h(Systems)p Fw(,)f(pages)h(18\22625,)f +(Geno)o(v)n(a,)h(Italy)l(,)-10 3728 y(2009.)17 b(A)m(CM.)p +Black -117 3827 a([6])p Black 30 w(C.)c(F)-5 b(.)13 b(Bolz,)h(A.)f(K)o +(uhn,)h(A.)f(Lienhard,)i(N.)e(Matsakis,)i(O.)e(Nierstrasz,)j(L.)c +(Reng-)-10 3902 y(gli,)i(A.)f(Rigo,)h(and)h(T)-5 b(.)12 +b(V)-7 b(erw)o(aest.)19 b(Back)c(to)f(the)h(future)g(in)f(one)h(week)f +(\227)g(imple-)-10 3977 y(menting)22 b(a)f(Smalltalk)i(VM)d(in)h(PyPy.) +37 b(In)20 b Fa(Self-Sustaining)25 b(Systems)p Fw(,)c(pages)-10 +4051 y(123\226139.)c(2008.)p Black -117 4151 a([7])p +Black 30 w(C.)25 b(F)-5 b(.)25 b(Bolz,)i(M.)f(Leuschel,)h(and)f(D.)g +(Schneider)l(.)55 b(T)-5 b(o)n(w)o(ards)27 b(a)f(jitting)j(VM)-10 +4226 y(for)21 b(Prolog)h(e)o(x)o(ecution.)41 b(In)21 +b Fa(Pr)m(oceedings)i(of)e(the)i(12th)f(international)i(A)n(CM)-10 +4301 y(SIGPLAN)d(symposium)i(on)g(Principles)h(and)e(pr)o(actice)j(of)d +(declar)o(ative)k(pr)m(o-)-10 4375 y(gr)o(amming)p Fw(,)17 +b(pages)h(99\226108,)f(Hagenber)o(g,)i(Austria,)e(2010.)g(A)m(CM.)p +Black -117 4475 a([8])p Black 30 w(C.)25 b(Bruni)i(and)f(T)-5 +b(.)25 b(V)-7 b(erw)o(aest.)53 b(PyGirl:)27 b(generating)i +(Whole-System)e(VMs)-10 4550 y(from)18 b(High-Le)n(v)o(el)j(prototypes) +f(using)f(PyPy.)29 b(In)18 b(W)-6 b(.)17 b(Aalst,)i(J.)f(Mylopoulos,) +-10 4624 y(N.)28 b(M.)g(Sadeh,)h(M.)f(J.)g(Sha)o(w)l(,)h(C.)f +(Szyperski,)i(M.)e(Oriol,)h(and)h(B.)e(Me)o(yer)m(,)-10 +4699 y(editors,)j Fa(Objects,)g(Components,)g(Models)g(and)f(P)-5 +b(atterns)p Fw(,)30 b(v)o(olume)g(33)g(of)-10 4774 y +Fa(Lectur)n(e)e(Notes)h(in)f(Business)f(Information)j(Pr)m(ocessing)p +Fw(,)e(pages)g(328\226347.)-10 4848 y(Springer)18 b(Berlin)h(Heidelber) +o(g,)g(2009.)24 b(10.1007/978-3-642-02571-6_19.)p Black +-117 4948 a([9])p Black 30 w(C.)c(Chambers,)i(D.)e(Ungar)m(,)h(and)h +(E.)e(Lee.)37 b(An)21 b(ef)n(\002cient)j(implementation)g(of)-10 +5023 y(SELF)c(a)i(dynamically-typed)k(object-oriented)g(language)e +(based)f(on)e(proto-)-10 5098 y(types.)j Fa(SIGPLAN)17 +b(Not.)p Fw(,)f(24\(10\):49\22670,)j(1989.)p Black Black +Black 2042 66 a([10])p Black 29 w(M.)e(Chang,)i(M.)e(Bebenita,)i(A.)e +(Y)-7 b(ermolo)o(vich,)20 b(A.)d(Gal,)g(and)i(M.)d(Franz.)27 +b(Ef)n(\002-)2181 141 y(cient)22 b(just-in-time)g(e)o(x)o(ecution)h(of) +c(dynamically)k(typed)e(languages)g(via)g(code)2181 216 +y(specialization)33 b(using)27 b(precise)i(runtime)g(type)f(inference.) +59 b(T)-5 b(echnical)30 b(Re-)2181 291 y(port)e(ICS-TR-07-10,)g(Donald) +g(Bren)g(School)g(of)f(Information)h(and)g(Com-)2181 +365 y(puter)19 b(Science,)f(Uni)n(v)o(ersity)i(of)d(California,)i +(Irvine,)f(2007.)p Black 2042 465 a([11])p Black 29 w(J.)d(Choi,)h(M.)e +(Gupta,)i(M.)f(Serrano,)h(V)-9 b(.)15 b(C.)g(Sreedhar)m(,)i(and)f(S.)e +(Midkif)n(f.)22 b(Escape)2181 540 y(analysis)d(for)e(Ja)o(v)n(a.)25 +b Fa(SIGPLAN)16 b(Not.)p Fw(,)h(34\(10\):1\22619,)h(1999.)p +Black 2042 639 a([12])p Black 29 w(A.)j(Cuni.)37 b Fa(High)21 +b(performance)i(implementation)h(of)e(Python)f(for)g(CLI/.NET)2181 +714 y(with)29 b(JIT)f(compiler)i(g)o(ener)o(ation)g(for)e(dynamic)i +(langua)o(g)o(es.)61 b Fw(PhD)27 b(thesis,)2181 789 y(Dipartimento)e +(di)d(Informatica)i(e)d(Scienze)j(dell'Informazione,)i(Uni)n(v)o +(ersity)2181 863 y(of)17 b(Geno)o(v)n(a,)i(2010.)24 b(T)-5 +b(echnical)20 b(Report)e(DISI-TH-2010-05.)p Black 2042 +963 a([13])p Black 29 w(A.)32 b(Gal,)h(B.)f(Eich,)g(M.)g(Sha)o(v)o(er)m +(,)h(D.)e(Anderson,)i(B.)f(Kaplan,)i(G.)d(Hoare,)2181 +1038 y(D.)k(Mandelin,)h(B.)f(Zbarsk)o(y)l(,)f(J.)g(Orendorf)n(f,)i(M.)e +(Bebenita,)j(M.)d(Chang,)2181 1112 y(M.)13 b(Franz,)h(E.)f(Smith,)g(R.) +g(Reitmaier)m(,)j(and)e(M.)f(Haghighat.)19 b(T)n(race-based)c(just-) +2181 1187 y(in-time)k(type)f(specialization)k(for)17 +b(dynamic)h(languages.)26 b(In)17 b Fa(PLDI)p Fw(,)f(2009.)p +Black 2042 1287 a([14])p Black 29 w(A.)28 b(Gal)h(and)g(M.)e(Franz.)61 +b(Incremental)31 b(dynamic)f(code)f(generation)i(with)2181 +1362 y(trace)23 b(trees.)36 b(T)-5 b(echnical)23 b(Report)f +(ICS-TR-06-16,)f(Donald)g(Bren)g(School)h(of)2181 1436 +y(Information)j(and)e(Computer)h(Science,)h(Uni)n(v)o(ersity)g(of)e +(California,)i(Irvine,)2181 1511 y(No)o(v)l(.)17 b(2006.)p +Black 2042 1611 a([15])p Black 29 w(A.)25 b(Gal,)g(C.)g(W)-6 +b(.)24 b(Probst,)h(and)g(M.)g(Franz.)50 b(HotpathVM:)28 +b(an)d(ef)n(fecti)n(v)o(e)k(JIT)2181 1685 y(compiler)k(for)f +(resource-constrained)k(de)n(vices.)72 b(In)31 b Fa(Pr)m(oceedings)i +(of)e(the)2181 1760 y(2nd)15 b(international)i(confer)n(ence)f(on)e(V) +-5 b(irtual)15 b(e)o(xecution)i(en)m(vir)m(onments)p +Fw(,)f(pages)2181 1835 y(144\226153,)i(Otta)o(w)o(a,)h(Ontario,)f +(Canada,)g(2006.)f(A)m(CM.)p Black 2042 1934 a([16])p +Black 29 w(A.)h(Geor)o(ges,)h(D.)e(Buytaert,)j(and)f(L.)e(Eeckhout.)29 +b(Statistically)23 b(rigorous)c(Ja)o(v)n(a)2181 2009 +y(performance)h(e)n(v)n(aluation.)28 b Fa(SIGPLAN)16 +b(Not.)p Fw(,)h(42\(10\):57\22676,)h(2007.)p Black 2042 +2109 a([17])p Black 29 w(A.)k(Gill,)g(J.)f(Launchb)o(ury)l(,)h(and)h +(S.)e(L.)g(P)-7 b(.)20 b(Jones.)40 b(A)22 b(short)g(cut)h(to)f +(deforesta-)2181 2183 y(tion.)47 b(In)23 b Fa(Pr)m(oceedings)j(of)e +(the)g(confer)n(ence)i(on)e(Functional)i(pr)m(o)o(gr)o(amming)2181 +2258 y(langua)o(g)o(es)19 b(and)d(computer)i(ar)n(c)o(hitectur)n(e)p +Fw(,)g(FPCA)e('93,)g(page)h(223\226232,)g(Ne)n(w)2181 +2333 y(Y)-7 b(ork,)17 b(NY)-9 b(,)16 b(USA,)g(1993.)i(A)m(CM.)p +Black 2042 2433 a([18])p Black 29 w(B.)e(Goldber)o(g)i(and)f(Y)-9 +b(.)16 b(G.)f(P)o(ark.)23 b(Higher)18 b(order)f(escape)h(analysis:)g +(optimizing)2181 2507 y(stack)k(allocation)i(in)c(functional)j(program) +e(implementations.)38 b(In)20 b Fa(Pr)m(oceed-)2181 2582 +y(ings)e(of)f(the)h(thir)n(d)g(Eur)m(opean)f(symposium)h(on)g(pr)m(o)o +(gr)o(amming)g(on)f(ESOP)f('90)p Fw(,)2181 2657 y(pages)26 +b(152\226160,)f(Copenhagen,)h(Denmark,)f(1990.)g(Springer)o(-V)-7 +b(erlag)27 b(Ne)n(w)2181 2731 y(Y)-7 b(ork,)17 b(Inc.)p +Black 2042 2831 a([19])p Black 29 w(F)-5 b(.)18 b(Henglein.)33 +b(Dynamic)21 b(typing:)g(syntax)f(and)f(proof)h(theory)l(.)32 +b Fa(Sci.)19 b(Comput.)2181 2906 y(Pr)m(o)o(gr)o(am.)p +Fw(,)d(22:197\226230,)j(June)e(1994.)p Black 2042 3005 +a([20])p Black 29 w(J.)30 b(J\370r)o(gensen.)66 b Fa(A)29 +b(Calculus)j(for)e(Boxing)h(Analysis)g(of)f(P)-5 b(olymorphically)2181 +3080 y(T)g(yped)17 b(Langua)o(g)o(es)p Fw(.)22 b(Ph.D.)15 +b(Thesis,)g(Uni)n(v)o(ersity)i(of)f(Copenhapen,)h(1996.)k(TR)2181 +3155 y(96/28.)p Black 2042 3254 a([21])p Black 29 w(J.)i(W)-6 +b(.)23 b(Llo)o(yd)h(and)h(J.)d(C.)i(Shepherdson.)47 b(P)o(artial)26 +b(e)n(v)n(aluation)h(in)d(logic)i(pro-)2181 3329 y(gramming.)f +Fa(J)n(.)17 b(Lo)o(g)o(.)g(Pr)m(o)o(gr)o(am.)p Fw(,)f +(11\(3-4\):217\226242,)j(1991.)p Black 2042 3429 a([22])p +Black 29 w(E.)13 b(Miranda.)18 b(Conte)o(xt)d(management)h(in)e(V)l +(isualW)-5 b(orks)14 b(5i.)j(T)-5 b(echnical)16 b(report,)2181 +3504 y(P)o(arcPlace)k(Di)n(vision,)e(CINCOM,)f(Inc.,)g(1999.)p +Black 2042 3603 a([23])p Black 29 w(T)-5 b(.)31 b(Mogensen.)71 +b(Constructor)33 b(specialization.)75 b(In)32 b Fa(Pr)m(oceedings)h(of) +e(the)2181 3678 y(1993)15 b(A)n(CM)f(SIGPLAN)f(symposium)i(on)f(P)-5 +b(artial)14 b(e)o(valuation)j(and)d(semantics-)2181 3753 +y(based)27 b(pr)m(o)o(gr)o(am)f(manipulation)p Fw(,)j(pages)d +(22\22632,)g(Copenhagen,)j(Denmark,)2181 3827 y(1993.)18 +b(A)m(CM.)p Black 2042 3927 a([24])p Black 29 w(Y)-9 +b(.)17 b(G.)g(P)o(ark)g(and)h(B.)f(Goldber)o(g.)26 b(Escape)18 +b(analysis)h(on)e(lists.)26 b Fa(SIGPLAN)16 b(Not.)p +Fw(,)2181 4002 y(27\(7\):116\226127,)j(1992.)p Black +2042 4101 a([25])p Black 29 w(A.)30 b(Rigo.)68 b(Representation-based) +36 b(just-in-time)e(specialization)h(and)c(the)2181 4176 +y(Psyco)24 b(prototype)g(for)f(Python.)43 b(In)23 b Fa(Pr)m(oceedings)i +(of)e(the)g(2004)g(A)n(CM)g(SIG-)2181 4251 y(PLAN)14 +b(symposium)i(on)e(P)-5 b(artial)16 b(e)o(valuation)h(and)e +(semantics-based)i(pr)m(o)o(gr)o(am)2181 4325 y(manipulation)p +Fw(,)j(pages)e(15\22626,)f(V)-7 b(erona,)17 b(Italy)l(,)h(2004.)f(A)m +(CM.)p Black 2042 4425 a([26])p Black 29 w(A.)26 b(Rigo)i(and)f(S.)e +(Pedroni.)56 b(PyPy')l(s)27 b(approach)i(to)d(virtual)j(machine)f(con-) +2181 4500 y(struction.)49 b(In)24 b Fa(Companion)i(to)e(the)h(21st)f(A) +n(CM)g(SIGPLAN)f(confer)n(ence)k(on)2181 4575 y(Object-oriented)i(pr)m +(o)o(gr)o(amming)c(systems,)g(langua)o(g)o(es,)h(and)e(applications)p +Fw(,)2181 4649 y(pages)18 b(944\226953,)g(Portland,)g(Ore)o(gon,)g +(USA,)e(2006.)h(A)m(CM.)p Black 2042 4749 a([27])p Black +29 w(P)-7 b(.)23 b(W)-5 b(adler)l(.)48 b(Deforestation:)28 +b(transforming)e(programs)e(to)g(eliminate)j(trees.)2181 +4824 y(In)17 b Fa(Pr)m(oceedings)i(of)e(the)g(Second)i(Eur)m(opean)e +(Symposium)h(on)f(Pr)m(o)o(gr)o(amming)p Fw(,)2181 4898 +y(page)i(231\226248,)f(Amsterdam,)g(The)g(Netherlands,)i(The)d +(Netherlands,)j(1988.)2181 4973 y(North-Holland)g(Publishing)f(Co.)p +Black Black eop end %%Trailer userdict /end-hook known{end-hook}if From afa at codespeak.net Fri Nov 26 15:07:14 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 26 Nov 2010 15:07:14 +0100 (CET) Subject: [pypy-svn] r79557 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101126140714.E3F0B282B9E@codespeak.net> Author: afa Date: Fri Nov 26 15:07:12 2010 New Revision: 79557 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py pypy/branch/fast-forward/lib_pypy/_ctypes/function.py Log: Fix the argument conversion: - first try "argtype.from_param" - then _as_parameter_ - then default guesses Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py Fri Nov 26 15:07:12 2010 @@ -46,15 +46,6 @@ else: return self.from_param(as_parameter) - def _CData_input(self, value): - """Used when data enters into ctypes from user code. 'value' is - some user-specified Python object, which is converted into a _rawffi - array of length 1 containing the same value according to the - type 'self'. - """ - cobj = self.from_param(value) - return cobj, cobj._get_buffer_for_param() - def _CData_value(self, value): cobj = self.from_param(value) # we don't care here if this stuff will live afterwards, as we're Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/function.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/function.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/function.py Fri Nov 26 15:07:12 2010 @@ -54,7 +54,15 @@ return self._argtypes_ def _setargtypes(self, argtypes): self._ptr = None - self._argtypes_ = argtypes + if argtypes is None: + self._argtypes_ = None + else: + for i, argtype in enumerate(argtypes): + if not hasattr(argtype, 'from_param'): + raise TypeError( + "item %d in _argtypes_ has no from_param method" % ( + i + 1,)) + self._argtypes_ = argtypes argtypes = property(_getargtypes, _setargtypes) def _getrestype(self): @@ -179,13 +187,15 @@ thisarg = None if argtypes is None: - argtypes = self._guess_argtypes(args) - argtypes, argsandobjs = self._wrap_args(argtypes, args) + argtypes = [] + args = self._convert_args(argtypes, args) + argtypes = [type(arg) for arg in args] restype = self._restype_ funcptr = self._getfuncptr(argtypes, restype, thisarg) - resbuffer = funcptr(*[arg._buffer for _, arg in argsandobjs]) - result = self._build_result(restype, resbuffer, argtypes, argsandobjs) + resbuffer = funcptr(*[arg._get_buffer_for_param()._buffer + for arg in args]) + result = self._build_result(restype, resbuffer, argtypes, args) # The 'errcheck' protocol if self._errcheck_: @@ -243,31 +253,33 @@ raise @staticmethod - def _guess_argtypes(args): + def _conv_param(argtype, arg, index): from ctypes import c_char_p, c_wchar_p, c_void_p, c_int - res = [] - for arg in args: - if hasattr(arg, '_as_parameter_'): - arg = arg._as_parameter_ - if isinstance(arg, str): - res.append(c_char_p) - elif isinstance(arg, unicode): - res.append(c_wchar_p) - elif isinstance(arg, _CData): - res.append(type(arg)) - elif arg is None: - res.append(c_void_p) - #elif arg == 0: - # res.append(c_void_p) - elif isinstance(arg, (int, long)): - res.append(c_int) - else: - raise TypeError("Don't know how to handle %s" % (arg,)) - return res + if argtype is not None: + arg = argtype.from_param(arg) + if hasattr(arg, '_as_parameter_'): + arg = arg._as_parameter_ + + if isinstance(arg, _CData): + # The usual case when argtype is defined + cobj = arg + elif isinstance(arg, str): + cobj = c_char_p(arg) + elif isinstance(arg, unicode): + cobj = c_wchar_p(arg) + elif arg is None: + cobj = c_void_p() + elif isinstance(arg, (int, long)): + cobj = c_int(arg) + else: + raise TypeError("Don't know how to handle %s" % (arg,)) + + return cobj - def _wrap_args(self, argtypes, args): + def _convert_args(self, argtypes, args): wrapped_args = [] consumed = 0 + for i, argtype in enumerate(argtypes): defaultvalue = None if i > 0 and self._paramflags is not None: @@ -294,7 +306,7 @@ val = defaultvalue if val is None: val = 0 - wrapped = argtype._CData_input(val) + wrapped = self._conv_param(argtype, val, consumed) wrapped_args.append(wrapped) continue else: @@ -309,23 +321,22 @@ raise TypeError("Not enough arguments") try: - wrapped = argtype._CData_input(arg) - except (UnicodeError, TypeError), e: + wrapped = self._conv_param(argtype, arg, consumed) + except (UnicodeError, TypeError, ValueError), e: raise ArgumentError(str(e)) wrapped_args.append(wrapped) consumed += 1 if len(wrapped_args) < len(args): extra = args[len(wrapped_args):] - extra_types = self._guess_argtypes(extra) - for arg, argtype in zip(extra, extra_types): + argtypes = list(argtypes) + for i, arg in enumerate(extra): try: - wrapped = argtype._CData_input(arg) - except (UnicodeError, TypeError), e: + wrapped = self._conv_param(None, arg, i) + except (UnicodeError, TypeError, ValueError), e: raise ArgumentError(str(e)) wrapped_args.append(wrapped) - argtypes = list(argtypes) + extra_types - return argtypes, wrapped_args + return wrapped_args def _build_result(self, restype, resbuffer, argtypes, argsandobjs): """Build the function result: @@ -338,7 +349,7 @@ if self._com_index: if resbuffer[0] & 0x80000000: raise get_com_error(resbuffer[0], - self._com_iid, argsandobjs[0][0]) + self._com_iid, argsandobjs[0]) else: retval = int(resbuffer[0]) elif restype is not None: @@ -357,8 +368,8 @@ results = [] if self._paramflags: - for argtype, (obj, _), paramflag in zip(argtypes[1:], argsandobjs[1:], - self._paramflags): + for argtype, obj, paramflag in zip(argtypes[1:], argsandobjs[1:], + self._paramflags): if len(paramflag) == 2: idlflag, name = paramflag elif len(paramflag) == 3: From afa at codespeak.net Fri Nov 26 15:25:50 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 26 Nov 2010 15:25:50 +0100 (CET) Subject: [pypy-svn] r79558 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101126142550.6CD05282BD6@codespeak.net> Author: afa Date: Fri Nov 26 15:25:48 2010 New Revision: 79558 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py Log: "_abstract_" Structure cannot be created Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/structure.py Fri Nov 26 15:25:48 2010 @@ -117,6 +117,8 @@ def __new__(self, name, cls, typedict): res = type.__new__(self, name, cls, typedict) + if "_abstract_" in typedict: + return res if isinstance(cls[0], StructOrUnionMeta): cls[0]._make_final() if '_fields_' in typedict: @@ -186,6 +188,8 @@ def __new__(cls, *args, **kwds): self = super(_CData, cls).__new__(cls, *args, **kwds) + if '_abstract_' in cls.__dict__: + raise TypeError("abstract class") if hasattr(cls, '_ffistruct'): self.__dict__['_buffer'] = self._ffistruct(autofree=True) return self From antocuni at codespeak.net Fri Nov 26 15:30:53 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Fri, 26 Nov 2010 15:30:53 +0100 (CET) Subject: [pypy-svn] r79559 - pypy/trunk/pypy/rlib/test Message-ID: <20101126143053.166A8282BEA@codespeak.net> Author: antocuni Date: Fri Nov 26 15:30:51 2010 New Revision: 79559 Modified: pypy/trunk/pypy/rlib/test/test_libffi.py Log: ops, fix the test for when we run it translated (and the exception we get is an LLException) Modified: pypy/trunk/pypy/rlib/test/test_libffi.py ============================================================================== --- pypy/trunk/pypy/rlib/test/test_libffi.py (original) +++ pypy/trunk/pypy/rlib/test/test_libffi.py Fri Nov 26 15:30:51 2010 @@ -264,7 +264,22 @@ assert res == initval+1 def test_wrong_number_of_arguments(self): + from pypy.rpython.llinterp import LLException libfoo = self.get_libfoo() func = (libfoo, 'sum_xy', [types.sint, types.double], types.sint) - py.test.raises(TypeError, "self.call(func, [38], rffi.LONG)") # one less - py.test.raises(TypeError, "self.call(func, [38, 12.3, 42], rffi.LONG)") # one more + + glob = globals() + loc = locals() + def my_raises(s): + try: + exec s in glob, loc + except TypeError: + pass + except LLException, e: + if str(e) != "": + raise + else: + assert False, 'Did not raise' + + my_raises("self.call(func, [38], rffi.LONG)") # one less + my_raises("self.call(func, [38, 12.3, 42], rffi.LONG)") # one more From arigo at codespeak.net Fri Nov 26 15:38:10 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 26 Nov 2010 15:38:10 +0100 (CET) Subject: [pypy-svn] r79560 - in pypy/release/1.4.x/pypy/translator: c/gcc platform Message-ID: <20101126143810.44F165080B@codespeak.net> Author: arigo Date: Fri Nov 26 15:38:08 2010 New Revision: 79560 Modified: pypy/release/1.4.x/pypy/translator/c/gcc/trackgcroot.py pypy/release/1.4.x/pypy/translator/platform/darwin.py Log: Merge trunk's r79536 and r79541 from antocuni: (mvt) fix os/x 64 bit translation (mvt) hopefully *really* fix translation :-) Modified: pypy/release/1.4.x/pypy/translator/c/gcc/trackgcroot.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/c/gcc/trackgcroot.py (original) +++ pypy/release/1.4.x/pypy/translator/c/gcc/trackgcroot.py Fri Nov 26 15:38:08 2010 @@ -1106,7 +1106,7 @@ format = 'darwin64' function_names_prefix = '_' - LABEL = ElfFunctionGcRootTracker32.LABEL + LABEL = ElfFunctionGcRootTracker64.LABEL r_jmptable_item = re.compile(r"\t.(?:long|quad)\t"+LABEL+"(-\"?[A-Za-z0-9$]+\"?)?\s*$") r_functionstart = re.compile(r"_(\w+):\s*$") Modified: pypy/release/1.4.x/pypy/translator/platform/darwin.py ============================================================================== --- pypy/release/1.4.x/pypy/translator/platform/darwin.py (original) +++ pypy/release/1.4.x/pypy/translator/platform/darwin.py Fri Nov 26 15:38:08 2010 @@ -12,12 +12,14 @@ so_ext = 'so' + default_cc = 'gcc' + def __init__(self, cc=None): if cc is None: try: cc = os.environ['CC'] except KeyError: - cc = 'gcc' + cc = self.default_cc self.cc = cc def _args_for_shared(self, args): @@ -85,3 +87,4 @@ link_flags = ('-arch', 'x86_64', '-mmacosx-version-min=10.4') cflags = ('-arch', 'x86_64', '-O3', '-fomit-frame-pointer', '-mmacosx-version-min=10.4') + default_cc = 'gcc-4.0' From arigo at codespeak.net Fri Nov 26 16:03:59 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Fri, 26 Nov 2010 16:03:59 +0100 (CET) Subject: [pypy-svn] r79561 - pypy/build/bot2/pypybuildbot Message-ID: <20101126150359.E935B282B9D@codespeak.net> Author: arigo Date: Fri Nov 26 16:03:57 2010 New Revision: 79561 Modified: pypy/build/bot2/pypybuildbot/master.py Log: Use the category name "osx64" explicitly. Modified: pypy/build/bot2/pypybuildbot/master.py ============================================================================== --- pypy/build/bot2/pypybuildbot/master.py (original) +++ pypy/build/bot2/pypybuildbot/master.py Fri Nov 26 16:03:57 2010 @@ -136,7 +136,16 @@ ) pypyJITTranslatedTestFactoryOSX = pypybuilds.Translated( - platform='osx', # works also for "OS/X 64" + platform='osx', + translationArgs=jit_translation_args, + targetArgs=[], + lib_python=True, + pypyjit=True, + app_tests=True, + ) + +pypyJITTranslatedTestFactoryOSX64 = pypybuilds.Translated( + platform='osx64', translationArgs=jit_translation_args, targetArgs=[], lib_python=True, @@ -295,7 +304,7 @@ {"name" : JITMACOSX64, "slavenames": ["macmini-mvt"], 'builddir' : JITMACOSX64, - 'factory' : pypyJITTranslatedTestFactoryOSX, + 'factory' : pypyJITTranslatedTestFactoryOSX64, 'category' : 'mac64', }, {"name" : JITWIN32, From fijal at codespeak.net Fri Nov 26 16:24:23 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 26 Nov 2010 16:24:23 +0100 (CET) Subject: [pypy-svn] r79562 - pypy/extradoc/pypy.org/source Message-ID: <20101126152423.10B59282BE3@codespeak.net> Author: fijal Date: Fri Nov 26 16:24:21 2010 New Revision: 79562 Modified: pypy/extradoc/pypy.org/source/features.txt pypy/extradoc/pypy.org/source/index.txt Log: In-progress website rebuild Modified: pypy/extradoc/pypy.org/source/features.txt ============================================================================== --- pypy/extradoc/pypy.org/source/features.txt (original) +++ pypy/extradoc/pypy.org/source/features.txt Fri Nov 26 16:24:21 2010 @@ -6,37 +6,30 @@ PyPy features =========================================================== -**PyPy 1.3** implements **Python 2.5.** It supports all of the core +**PyPy 1.4** implements **Python 2.5.** It supports all of the core language, passing the Python test suite (with minor modifications that were already accepted in the main python in newer versions). It supports most of the commonly used Python standard library modules. For known differences with CPython, see our `compatibility`_ page. If you are interested in helping with 2.6, 2.7 or 3.x features, `contact us`_! -PyPy 1.3 runs essentially only on Intel `x86 (IA-32)`_. On 64-bit platforms -you have to use the 32-bit compatibility mode, for now -- or `contact us`_ -to help! +PyPy 1.4 runs essentially only on Intel `x86 (IA-32)`_ and `x86_64`_ platforms, +with ARM being underway. +Feel free to contact us if you want to port it to any other platform! .. _`compatibility`: compat.html .. _`x86 (IA-32)`: http://en.wikipedia.org/wiki/IA-32 +.. _`x86_64`: http://en.wikipedia.org/wiki/X86_64 +Speed +----- -Speed or memory usage? -------------------------------- - -Our `main prototype`_ comes with a Just-in-Time compiler. It is +Our `main executable`_ comes with a Just-in-Time compiler. It is `really fast`_ in running most benchmarks. `Try it out!`_ -An issue with our JIT compiler is that it's hard to control -the exact amount of RAM that is used. If this is an important -issue to you, you can try our `baseline version`_, which does -not include a JIT compiler at all. - .. _`main prototype`: download.html#with-a-jit-compiler .. _`Try it out!`: download.html#with-a-jit-compiler .. _`really fast`: http://speed.pypy.org/ -.. _`baseline version`: download.html#with-no-jit-compiler - Sandboxing -------------------- Modified: pypy/extradoc/pypy.org/source/index.txt ============================================================================== --- pypy/extradoc/pypy.org/source/index.txt (original) +++ pypy/extradoc/pypy.org/source/index.txt Fri Nov 26 16:24:21 2010 @@ -22,7 +22,7 @@ .. class:: download -`Download and try out the PyPy release 1.3!`__ +`Download and try out the PyPy release 1.4!`__ .. __: download.html From afa at codespeak.net Fri Nov 26 16:31:07 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 26 Nov 2010 16:31:07 +0100 (CET) Subject: [pypy-svn] r79563 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101126153107.9A2B0282BE3@codespeak.net> Author: afa Date: Fri Nov 26 16:31:05 2010 New Revision: 79563 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py pypy/branch/fast-forward/lib_pypy/_ctypes/builtin.py pypy/branch/fast-forward/lib_pypy/_ctypes/pointer.py Log: Another common potential crasher: cast() should keep the original object alive, and make a copy of its keepalives. Remove _CData_value() this is a dangerous function when the value is a string: the buffer may be freed as soon as the function returns! Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/basics.py Fri Nov 26 16:31:05 2010 @@ -46,12 +46,6 @@ else: return self.from_param(as_parameter) - def _CData_value(self, value): - cobj = self.from_param(value) - # we don't care here if this stuff will live afterwards, as we're - # interested only in value anyway - return cobj._get_buffer_value() - def _CData_output(self, resbuffer, base=None, index=-1): #assert isinstance(resbuffer, _rawffi.ArrayInstance) """Used when data exits ctypes and goes into user code. Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/builtin.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/builtin.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/builtin.py Fri Nov 26 16:31:05 2010 @@ -11,7 +11,8 @@ def _string_at_addr(addr, lgt): # address here can be almost anything import ctypes - arg = ctypes.c_void_p._CData_value(addr) + cobj = ctypes.c_void_p.from_param(addr) + arg = cobj._get_buffer_value() return _rawffi.charp2rawstring(arg, lgt) def set_conversion_mode(encoding, errors): @@ -22,5 +23,6 @@ def _wstring_at_addr(addr, lgt): import ctypes - arg = ctypes.c_void_p._CData_value(addr) + cobj = ctypes.c_void_p.from_param(addr) + arg = cobj._get_buffer_value() return _rawffi.wcharp2rawunicode(arg, lgt) Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/pointer.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/pointer.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/pointer.py Fri Nov 26 16:31:05 2010 @@ -1,7 +1,8 @@ import _rawffi from _ctypes.basics import _CData, _CDataMeta, cdata_from_address -from _ctypes.basics import sizeof, byref, keepalive_key +from _ctypes.basics import keepalive_key, store_reference, ensure_objects +from _ctypes.basics import sizeof, byref from _ctypes.array import Array, array_get_slice_params, array_slice_getitem,\ array_slice_setitem @@ -102,7 +103,10 @@ return self._type_._CData_output(self._subarray(index), self, index) def __setitem__(self, index, value): - self._subarray(index)[0] = self._type_._CData_value(value) + cobj = self._type_.from_param(value) + if ensure_objects(cobj) is not None: + store_reference(self, index, cobj._objects) + self._subarray(index)[0] = cobj._get_buffer_value() def __nonzero__(self): return self._buffer[0] != 0 @@ -113,31 +117,34 @@ if not (isinstance(tp, _CDataMeta) and tp._is_pointer_like()): raise TypeError("cast() argument 2 must be a pointer type, not %s" % (tp,)) - if isinstance(obj, Array): - ptr = tp.__new__(tp) - ptr._buffer = tp._ffiarray(1, autofree=True) - ptr._buffer[0] = obj._buffer - return ptr if isinstance(obj, (int, long)): result = tp() result._buffer[0] = obj return result - if obj is None: + elif obj is None: result = tp() return result - if not (isinstance(obj, _CData) and type(obj)._is_pointer_like()): + elif isinstance(obj, Array): + ptr = tp.__new__(tp) + ptr._buffer = tp._ffiarray(1, autofree=True) + ptr._buffer[0] = obj._buffer + result = ptr + elif not (isinstance(obj, _CData) and type(obj)._is_pointer_like()): raise TypeError("cast() argument 1 must be a pointer, not %s" % (type(obj),)) - result = tp() + else: + result = tp() + result._buffer[0] = obj._buffer[0] # The casted objects '_objects' member: - # It must certainly contain the source objects one. + # From now on, both objects will use the same dictionary + # It must certainly contain the source objects # It must contain the source object itself. if obj._ensure_objects() is not None: - result._objects = {keepalive_key(0): obj._objects, - keepalive_key(1): obj} + result._objects = obj._objects + if isinstance(obj._objects, dict): + result._objects[id(obj)] = obj - result._buffer[0] = obj._buffer[0] return result def POINTER(cls): From afa at codespeak.net Fri Nov 26 16:38:16 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Fri, 26 Nov 2010 16:38:16 +0100 (CET) Subject: [pypy-svn] r79564 - pypy/branch/fast-forward/lib_pypy/_ctypes Message-ID: <20101126153816.38B18282B9D@codespeak.net> Author: afa Date: Fri Nov 26 16:38:14 2010 New Revision: 79564 Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py Log: A typo, fortunately spotted by cpython test suite Modified: pypy/branch/fast-forward/lib_pypy/_ctypes/array.py ============================================================================== --- pypy/branch/fast-forward/lib_pypy/_ctypes/array.py (original) +++ pypy/branch/fast-forward/lib_pypy/_ctypes/array.py Fri Nov 26 16:38:14 2010 @@ -134,7 +134,7 @@ if slicelength != len(value): raise ValueError("Can only assign slices of the same length") for i, j in enumerate(range(start, stop, step)): - self[j] = value[j] + self[j] = value[i] def array_slice_getitem(self, index): start, stop, step = self._get_slice_params(index) From fijal at codespeak.net Fri Nov 26 17:08:14 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 26 Nov 2010 17:08:14 +0100 (CET) Subject: [pypy-svn] r79565 - pypy/extradoc/pypy.org/source Message-ID: <20101126160814.59EB9282B9E@codespeak.net> Author: fijal Date: Fri Nov 26 17:08:12 2010 New Revision: 79565 Modified: pypy/extradoc/pypy.org/source/compat.txt pypy/extradoc/pypy.org/source/download.txt Log: update website Modified: pypy/extradoc/pypy.org/source/compat.txt ============================================================================== --- pypy/extradoc/pypy.org/source/compat.txt (original) +++ pypy/extradoc/pypy.org/source/compat.txt Fri Nov 26 17:08:12 2010 @@ -8,7 +8,7 @@ already accepted in the main python in newer versions). It supports most of the commonly used Python `standard library modules`_; details below. -PyPy has alpha-level support for the `CPython C API`_, however, as of 1.3 +PyPy has alpha-level support for the `CPython C API`_, however, as of 1.4 release this feature is not yet complete. Most libraries will require a bit of effort to work, but there are known success stories. Check out PyPy blog for updates. Modified: pypy/extradoc/pypy.org/source/download.txt ============================================================================== --- pypy/extradoc/pypy.org/source/download.txt (original) +++ pypy/extradoc/pypy.org/source/download.txt Fri Nov 26 17:08:12 2010 @@ -6,85 +6,45 @@ Download ============================================================ -Here are the various binaries of **PyPy 1.3** that we provide for x86 Linux, -Mac OS/X or Windows. This release improves over 1.2 in terms of stability -of the JIT. +Here are the various binaries of **PyPy 1.4** that we provide for x86 Linux, +Mac OS/X or Windows. This release improves over 1.3 in terms of speed, +memory usage and stability. .. class:: download_menu * Download - * `With a JIT Compiler`_ **(recommended!)** - * `With no JIT Compiler`_ + * `Default (with a JIT Compiler)`_ * `Sandboxed version`_ * `Stackless version`_ * `Installing`_ (optional) * `Building from source`_ -.. _`With a JIT Compiler`: +.. _`Default (with a JIT Compiler)`: "JIT Compiler" version ------------------------------- These binaries include a Just-in-Time compiler. They only work on -**32-bit** `x86 (IA-32)`_ CPUs that have the SSE2_ instruction set (most of -them do, nowadays), as well as on any `x86-64`_ machine in the 32-bit -compatibility mode. +CPUs that have the SSE2_ instruction set (most of +them do, nowadays). -* `Linux binary`__ -* `Mac OS/X binary`__ -* `Windows binary`__ - -.. __: http://pypy.org/download/pypy-1.3-linux.tar.bz2 -.. __: http://pypy.org/download/pypy-1.3-osx.tar.bz2 -.. __: http://pypy.org/download/pypy-1.3-win32.zip +* `Linux binary (32bit)`__ +* `Linux binary (64bit)`__ +* `Mac OS/X binary (32bit)`__ +* `Mac OS/X binary (64bit)`__ +* `Windows binary (32bit)`__ + +.. __: http://pypy.org/download/pypy-1.4-linux.tar.bz2 +.. __: http://pypy.org/download/pypy-1.4-linux64.tar.bz2 +.. __: http://pypy.org/download/pypy-1.4-osx.tar.bz2 +.. __: http://pypy.org/download/pypy-1.4-osx64.tar.bz2 +.. __: http://pypy.org/download/pypy-1.4-win32.zip If your CPU is really old, it may not have SSE2. In this case, you need to translate_ yourself with the option ``--jit-backend=x86-without-sse2``. -If your CPU is a 64-bit machine and you want to translate_ a 32-bit -version of PyPy yourself, `here are hints`_. - -If you want to help us with implementing the 64-bit JIT backend, -`contact us`_! - - -.. _`With no JIT Compiler`: -.. _`no jit version.`: - -"No JIT" version -------------------------------- - -**WARNING!** Unless you really want to try this out, we recommend the JIT -version. - -This version still has a few advantages over the JIT Compiler version. -Notably, for Python programs that require large amounts of memory (at -least a few hundred MBs), the following version of ``pypy-nojit`` runs them -by requiring generally 1.5x or 2x less memory than CPython. These -binaries work on 32-bit `x86 (IA-32)`_ CPUs as well as `x86-64`_ CPUs -in the 32-bit compatibility mode. - -* `Linux binary`__ -* `Mac OS/X binary`__ -* `Linux 64bit binary`__ -* `Windows binary`__ - -.. __: http://pypy.org/download/pypy-1.3-linux-nojit.tar.bz2 -.. __: http://pypy.org/download/pypy-1.3-osx-nojit.tar.bz2 -.. __: http://pypy.org/download/pypy-1.3-linux64-nojit.tar.bz2 -.. __: http://pypy.org/download/pypy-1.3-win32-nojit.zip - -If your CPU is a 64-bit machine and you want to translate_ a 32-bit -version of PyPy yourself, `here are hints`_. - -If you want to help us with finishing the implementation of the native -64-bit version (there are a few known issues, like missing code in -`ctypes`_ to implement calls to C functions), `contact us`_. In general -this version will more or less translate_ anywhere. - - .. _`Sandboxed version`: "Sandbox" version @@ -105,12 +65,7 @@ complicated, this reduce a bit the level of confidence we can put in the result. -If your CPU is a 64-bit machine and you want to translate_ a 32-bit -version of PyPy yourself, `here are hints`_. - -The Windows and the native 64-bit versions both need testing and careful -reviewing; `contact us`_! - +The Windows need testing and careful reviewing; `contact us`_! .. _`Stackless version`: @@ -124,8 +79,8 @@ * `Linux binary`__ * `Windows binary`__ -.. __: http://pypy.org/download/pypy-1.3-linux-stackless.tar.bz2 -.. __: http://pypy.org/download/pypy-1.3-win32-stackless.zip +.. __: http://pypy.org/download/pypy-1.4-linux-stackless.tar.bz2 +.. __: http://pypy.org/download/pypy-1.4-win32-stackless.zip It is not possible right now to combine Stackless features with the JIT. @@ -134,7 +89,6 @@ For the native 64-bit version, see the issues of the `no jit version.`_. - Installing ------------------------------- @@ -143,8 +97,8 @@ installed by manually moving the files:: /usr/bin/pypy # or pypy-nojit etc. - /usr/share/pypy-1.3/lib-python/* - /usr/share/pypy-1.3/pypy/* + /usr/share/pypy-1.4/lib-python/* + /usr/share/pypy-1.4/pypy/* You can also install it to ``/usr/local/bin`` and ``/usr/local/share``. @@ -157,11 +111,11 @@ 1. Get the source code. The following packages contain the source at the same revision as the above binaries (these are svn exports): - * `pypy-1.3-src.tar.bz2`__ (sources, Unix line endings) - * `pypy-1.3-src.zip`__ (sources, Windows line endings) + * `pypy-1.4-src.tar.bz2`__ (sources, Unix line endings) + * `pypy-1.4-src.zip`__ (sources, Windows line endings) - .. __: http://pypy.org/download/pypy-1.3-src.tar.bz2 - .. __: http://pypy.org/download/pypy-1.3-src.zip + .. __: http://pypy.org/download/pypy-1.4-src.tar.bz2 + .. __: http://pypy.org/download/pypy-1.4-src.zip Or you can checkout the current trunk using Subversion_ (the trunk usually works and is of course more up-to-date):: @@ -195,54 +149,6 @@ finish the translation, and 1.3 GB of RAM on a 32-bit system. (Do not start a translation on a machine with 1GB or less!) - -.. _`here are hints`: - -Note on building a 32-bit version on 64-bit systems -+++++++++++++++++++++++++++++++++++++++++++++++++++ - -To build 32-bit versions of ``pypy-c`` you need to run ``translate.py`` -in a 32-bit version of Python, and to make sure that the C compiler it -uses is also 32-bit. You can check the first condition with:: - - $ python - Python 2.6.2 (...) - >>> import sys - >>> sys.maxint - -This prints 9223372036854775807 in 64-bit versions and 2147483647 in -32-bit versions. Moreover, if your Python is 32-bit but your C compiler -is 64-bit, you will get ``AssertionError: Mixed configuration of the -word size of the machine`` when running ``translate.py.`` - -**On Linux,** the recommended way is to install and run in a fully -32-bit chroot (e.g. with the ``schroot`` Ubuntu package). The -alternative has issues about building some extension modules, but if you -want to try it, first compile yourself a 32-bit Python, e.g.:: - - # on Ubuntu Linux, you need at least: - apt-get install libc6-dev-i386 ia32-libs lib32z1-dev lib32ncurses5-dev - - cd Python-2.6.4 - CC="gcc -m32" LDFLAGS="-L/lib32 -L/usr/lib32 \ - -Wl,-rpath,/lib32 -Wl,-rpath,/usr/lib32" \ - ./configure - make - -And then run ``translate.py`` as follows:: - - cd pypy/translator/goal - CC="gcc -m32" /path/to/32bit/python translate.py ... - - -**On Mac OS/X,** the easiest is to systematically use Python 2.5 when -working with PyPy. Indeed, the standard Python 2.5 runs in 32-bit mode. -(XXX how to run the compiler in 32-bit mode??) - -**On Windows,** the only known way is to install a 32-bit Python -manually. (XXX how to run the compiler in 32-bit mode??) - - .. _`x86 (IA-32)`: http://en.wikipedia.org/wiki/IA-32 .. _`x86-64`: http://en.wikipedia.org/wiki/X86-64 .. _SSE2: http://en.wikipedia.org/wiki/SSE2 From fijal at codespeak.net Fri Nov 26 17:08:32 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 26 Nov 2010 17:08:32 +0100 (CET) Subject: [pypy-svn] r79566 - pypy/extradoc/pypy.org/js Message-ID: <20101126160832.B9DDB282BE3@codespeak.net> Author: fijal Date: Fri Nov 26 17:08:31 2010 New Revision: 79566 Modified: pypy/extradoc/pypy.org/js/detect.js Log: update this as well (how do we detect 64bit here???) Modified: pypy/extradoc/pypy.org/js/detect.js ============================================================================== --- pypy/extradoc/pypy.org/js/detect.js (original) +++ pypy/extradoc/pypy.org/js/detect.js Fri Nov 26 17:08:31 2010 @@ -2,13 +2,13 @@ $(document).ready(function() { var download_url, download_text; if (navigator.platform.indexOf('Linux') != -1) { - download_url = 'download/pypy-1.3-linux.tar.bz2'; + download_url = 'download/pypy-1.4-linux.tar.bz2'; download_text = 'Download linux x86 bin'; } else if (navigator.platform.indexOf('Win') != -1) { - download_url = 'download/pypy-1.3-win32.zip'; + download_url = 'download/pypy-1.4-win32.zip'; download_text = 'Download Windows x86 bin'; } else if (navigator.platform.indexOf('Mac') != 1) { - download_url = 'download/pypy-1.3-osx.tar.bz2'; + download_url = 'download/pypy-1.4-osx.tar.bz2'; downloat_text = 'Download Mac OS X 10.6 bin'; } else { download_url = "download.html"; From fijal at codespeak.net Fri Nov 26 17:10:29 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 26 Nov 2010 17:10:29 +0100 (CET) Subject: [pypy-svn] r79567 - in pypy/extradoc/pypy.org: . source Message-ID: <20101126161029.8694D282BEC@codespeak.net> Author: fijal Date: Fri Nov 26 17:10:27 2010 New Revision: 79567 Modified: pypy/extradoc/pypy.org/index.html pypy/extradoc/pypy.org/source/download.txt pypy/extradoc/pypy.org/source/features.txt Log: update html Modified: pypy/extradoc/pypy.org/index.html ============================================================================== --- pypy/extradoc/pypy.org/index.html (original) +++ pypy/extradoc/pypy.org/index.html Fri Nov 26 17:10:27 2010 @@ -124,4 +124,4 @@ - \ No newline at end of file + Modified: pypy/extradoc/pypy.org/source/download.txt ============================================================================== --- pypy/extradoc/pypy.org/source/download.txt (original) +++ pypy/extradoc/pypy.org/source/download.txt Fri Nov 26 17:10:27 2010 @@ -84,11 +84,6 @@ It is not possible right now to combine Stackless features with the JIT. -If your CPU is a 64-bit machine and you want to translate_ a 32-bit -version of PyPy yourself, `here are hints`_. - -For the native 64-bit version, see the issues of the `no jit version.`_. - Installing ------------------------------- Modified: pypy/extradoc/pypy.org/source/features.txt ============================================================================== --- pypy/extradoc/pypy.org/source/features.txt (original) +++ pypy/extradoc/pypy.org/source/features.txt Fri Nov 26 17:10:27 2010 @@ -27,7 +27,7 @@ Our `main executable`_ comes with a Just-in-Time compiler. It is `really fast`_ in running most benchmarks. `Try it out!`_ -.. _`main prototype`: download.html#with-a-jit-compiler +.. _`main executable`: download.html#with-a-jit-compiler .. _`Try it out!`: download.html#with-a-jit-compiler .. _`really fast`: http://speed.pypy.org/ From fijal at codespeak.net Fri Nov 26 17:11:41 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Fri, 26 Nov 2010 17:11:41 +0100 (CET) Subject: [pypy-svn] r79568 - pypy/extradoc/pypy.org Message-ID: <20101126161141.2FBD7282BEF@codespeak.net> Author: fijal Date: Fri Nov 26 17:11:39 2010 New Revision: 79568 Modified: pypy/extradoc/pypy.org/compat.html pypy/extradoc/pypy.org/download.html pypy/extradoc/pypy.org/features.html pypy/extradoc/pypy.org/index.html Log: update html for real Modified: pypy/extradoc/pypy.org/compat.html ============================================================================== --- pypy/extradoc/pypy.org/compat.html (original) +++ pypy/extradoc/pypy.org/compat.html Fri Nov 26 17:11:39 2010 @@ -51,7 +51,7 @@ language, passing Python test suite (with minor modifications that were already accepted in the main python in newer versions). It supports most of the commonly used Python standard library modules; details below.

    -

    PyPy has alpha-level support for the CPython C API, however, as of 1.3 +

    PyPy has alpha-level support for the CPython C API, however, as of 1.4 release this feature is not yet complete. Most libraries will require a bit of effort to work, but there are known success stories. Check out PyPy blog for updates.

    @@ -66,11 +66,11 @@ file if you want this to be enabled permanently.

    Standard library modules supported by PyPy, in alphabetical order:

      -
    • __builtin__ __pypy__ _codecs _lsprof _minimal_curses _random _rawffi _socket _sre _weakref bz2 cStringIO crypt errno exceptions fcntl gc itertools marshal math md5 mmap operator parser posix pyexpat select sha signal struct symbol sys termios thread time token unicodedata zipimport zlib
      • +
    • __builtin__ __pypy__ _ast _bisect _codecs _lsprof _minimal_curses _random _rawffi _ssl _socket _sre _weakref array bz2 cStringIO cpyext crypt errno exceptions fcntl  gc itertools marshal math md5 mmap operator parser posix pyexpat select sha signal struct symbol sys termios thread time token unicodedata zipimport zlib

    Supported, but written in pure-python:

      -
    • array binascii cPickle cmath collections ctypes datetime functools grp md5 pwd pyexpat sha sqlite3 syslog
      • +
    • binascii cPickle cmath collections ctypes datetime functools grp pwd sqlite3 syslog

    All modules that are pure python in CPython of course work.

    Python libraries known to work under PyPy (the list is not exhaustive):

    @@ -93,11 +93,11 @@

    or using the with keyword

    with open("filename", "w") as f:
        f.write("stuff")
    -

  • We don't support certain attributes that were decided to be -implementation-dependent. For example, gc.get_referrers does not exist. -Others may have different behavior; for example, gc.enable and -gc.disable are supported, but they don't enable and disable the GC, but -instead just enable and disable the running of finalizers.

    +

  • For the same reason, some functions and attributes of the gc module +behave in a slightly different way: for example, gc.enable and +gc.disable are supported, but instead of enabling and disabling the GC, +they just enable and disable the execution of finalizers. Also, +gc.garbage always returns an empty list.

  • You can't attach a __del__ method to a class after its creation.

    • Modified: pypy/extradoc/pypy.org/download.html ============================================================================== --- pypy/extradoc/pypy.org/download.html (original) +++ pypy/extradoc/pypy.org/download.html Fri Nov 26 17:11:39 2010 @@ -47,13 +47,12 @@

    Download and install

    -

    Here are the various binaries of PyPy 1.3 that we provide for x86 Linux, -Mac OS/X or Windows. This release improves over 1.2 in terms of stability -of the JIT.

    +

    Here are the various binaries of PyPy 1.4 that we provide for x86 Linux, +Mac OS/X or Windows. This release improves over 1.3 in terms of speed, +memory usage and stability.

    -

    “JIT Compiler” version

    +

    “JIT Compiler” version

    These binaries include a Just-in-Time compiler. They only work on -32-bit x86 (IA-32) CPUs that have the SSE2 instruction set (most of -them do, nowadays), as well as on any x86-64 machine in the 32-bit -compatibility mode.

    +CPUs that have the SSE2 instruction set (most of +them do, nowadays).

    If your CPU is really old, it may not have SSE2. In this case, you need to translate yourself with the option --jit-backend=x86-without-sse2.

    -

    If your CPU is a 64-bit machine and you want to translate a 32-bit -version of PyPy yourself, here are hints.

    -

    If you want to help us with implementing the 64-bit JIT backend, -contact us!

    -
    -
    -

    “No JIT” version

    -

    WARNING! Unless you really want to try this out, we recommend the JIT -version.

    -

    This version still has a few advantages over the JIT Compiler version. -Notably, for Python programs that require large amounts of memory (at -least a few hundred MBs), the following version of pypy-nojit runs them -by requiring generally 1.5x or 2x less memory than CPython. These -binaries work on 32-bit x86 (IA-32) CPUs as well as x86-64 CPUs -in the 32-bit compatibility mode.

    - -

    If your CPU is a 64-bit machine and you want to translate a 32-bit -version of PyPy yourself, here are hints.

    -

    If you want to help us with finishing the implementation of the native -64-bit version (there are a few known issues, like missing code in -ctypes to implement calls to C functions), contact us. In general -this version will more or less translate anywhere.

    “Sandbox” version

    @@ -115,24 +88,18 @@ sandboxing and the JIT compiler, although as the JIT is relatively complicated, this reduce a bit the level of confidence we can put in the result.

    -

    If your CPU is a 64-bit machine and you want to translate a 32-bit -version of PyPy yourself, here are hints.

    -

    The Windows and the native 64-bit versions both need testing and careful -reviewing; contact us!

    +

    The Windows need testing and careful reviewing; contact us!

    -
    +

    “Stackless” version

    Provides Stackless extensions, as well as greenlets. These binaries work on 32-bit x86 (IA-32) CPUs as well as x86-64 CPUs in the 32-bit compatibility mode.

    It is not possible right now to combine Stackless features with the JIT.

    -

    If your CPU is a 64-bit machine and you want to translate a 32-bit -version of PyPy yourself, here are hints.

    -

    For the native 64-bit version, see the issues of the no jit version..

    Installing

    @@ -141,8 +108,8 @@ installed by manually moving the files:

     /usr/bin/pypy     # or pypy-nojit etc.
-/usr/share/pypy-1.3/lib-python/*
-/usr/share/pypy-1.3/pypy/*
+/usr/share/pypy-1.4/lib-python/*
+/usr/share/pypy-1.4/pypy/*

    You can also install it to /usr/local/bin and /usr/local/share.

    @@ -152,8 +119,8 @@

  • Get the source code. The following packages contain the source at the same revision as the above binaries (these are svn exports):

    Or you can checkout the current trunk using Subversion (the trunk usually works and is of course more up-to-date):

    @@ -186,45 +153,6 @@ (Do not start a translation on a machine with 1GB or less!)

    • -
    -

    Note on building a 32-bit version on 64-bit systems

    -

    To build 32-bit versions of pypy-c you need to run translate.py -in a 32-bit version of Python, and to make sure that the C compiler it -uses is also 32-bit. You can check the first condition with:

    -
    -$ python
-Python 2.6.2 (...)
->>> import sys
->>> sys.maxint
-
    -

    This prints 9223372036854775807 in 64-bit versions and 2147483647 in -32-bit versions. Moreover, if your Python is 32-bit but your C compiler -is 64-bit, you will get AssertionError: Mixed configuration of the -word size of the machine when running translate.py.

    -

    On Linux, the recommended way is to install and run in a fully -32-bit chroot (e.g. with the schroot Ubuntu package). The -alternative has issues about building some extension modules, but if you -want to try it, first compile yourself a 32-bit Python, e.g.:

    -
    -# on Ubuntu Linux, you need at least:
-apt-get install libc6-dev-i386 ia32-libs lib32z1-dev lib32ncurses5-dev
-cd Python-2.6.4
-CC="gcc -m32" LDFLAGS="-L/lib32 -L/usr/lib32 \
-  -Wl,-rpath,/lib32 -Wl,-rpath,/usr/lib32" \
-  ./configure
-make
-
    -

    And then run translate.py as follows:

    -
    -cd pypy/translator/goal
-CC="gcc -m32" /path/to/32bit/python translate.py ...
-
    -

    On Mac OS/X, the easiest is to systematically use Python 2.5 when -working with PyPy. Indeed, the standard Python 2.5 runs in 32-bit mode. -(XXX how to run the compiler in 32-bit mode??)

    -

    On Windows, the only known way is to install a 32-bit Python -manually. (XXX how to run the compiler in 32-bit mode??)

    -

    Checksums

    Modified: pypy/extradoc/pypy.org/source/download.txt ============================================================================== --- pypy/extradoc/pypy.org/source/download.txt (original) +++ pypy/extradoc/pypy.org/source/download.txt Mon Nov 29 13:37:38 2010 @@ -113,22 +113,34 @@ cd pypy-trunk/pypy/translator/goal 4. Run the ``translate.py`` script. Here are the common combinations - of options:: + of options (works also with ``python`` instead of ``pypy``):: - python translate.py -Ojit # get the JIT version - python translate.py -O2 # get the no-jit version - python translate.py --sandbox # get the sandbox version - python translate.py --stackless # get the stackless version - python translate.py -Ojit --backend=cli # only for branch/cli-jit - -5. Enjoy Mandelbrot ``:-)`` It takes on the order of one hour to - finish the translation, and 1.2 GB of RAM on a 32-bit system - and 2.4 GB on 64-bit systems. (Do not start a translation on a + pypy translate.py -Ojit # get the JIT version + pypy translate.py -O2 # get the no-jit version + pypy translate.py --sandbox # get the sandbox version + pypy translate.py --stackless # get the stackless version + pypy translate.py -Ojit --backend=cli # only for branch/cli-jit + +5. Enjoy Mandelbrot ``:-)`` It takes on the order of half an hour to + finish the translation, and 1.7 GB of RAM on a 32-bit system + and 3.0 GB on 64-bit systems. (Do not start a translation on a machine with insufficient RAM! It will just swap forever.) -6. Once you have a PyPy, it is recommended to use it to do further - translations, instead of using CPython. It is twice as fast, - but uses a bit more RAM (1.7-2.0 GB on 32-bit, 3.0 GB on 64-bit). +Notes: + +* It is recommended to use PyPy to do translations, instead of using CPython, + because it is twice as fast. (Using CPython would lower the memory + requirement down to 1.2 GB on 32-bit, 2.4 GB on 64-bit.) + You should just start by downloading an official release of PyPy (with the + JIT). + +* In the final step, which is compiling the generated C files, the ``CFLAGS`` + are passed to the compiler. However, avoid passing too many custom + ``CFLAGS``, particularly when compiling with ``asmgcroot`` (needed by the + JIT). It will make ``trackgcroot.py`` unhappy. In case you want to try + it out anyway, after a CompilationError you can change your ``CFLAGS`` + and retry by typing ``cd /tmp/usession-$USER/testing_1; make``. + .. _`x86 (IA-32)`: http://en.wikipedia.org/wiki/IA-32 .. _`x86-64`: http://en.wikipedia.org/wiki/X86-64 From arigo at codespeak.net Mon Nov 29 13:46:10 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 29 Nov 2010 13:46:10 +0100 (CET) Subject: [pypy-svn] r79635 - pypy/trunk/pypy/rlib Message-ID: <20101129124610.56F67282BD6@codespeak.net> Author: arigo Date: Mon Nov 29 13:46:07 2010 New Revision: 79635 Modified: pypy/trunk/pypy/rlib/rarithmetic.py Log: Translation fix. Modified: pypy/trunk/pypy/rlib/rarithmetic.py ============================================================================== --- pypy/trunk/pypy/rlib/rarithmetic.py (original) +++ pypy/trunk/pypy/rlib/rarithmetic.py Mon Nov 29 13:46:07 2010 @@ -92,7 +92,8 @@ return False r_class = rffi.platform.numbertype_to_rclass[tp] assert issubclass(r_class, base_int) - return r_class.BITS < LONG_BIT + return r_class.BITS < LONG_BIT or ( + r_class.BITS == LONG_BIT and r_class.SIGNED) _should_widen_type._annspecialcase_ = 'specialize:memo' del _bits, _itest, _Ltest From arigo at codespeak.net Mon Nov 29 13:46:20 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 29 Nov 2010 13:46:20 +0100 (CET) Subject: [pypy-svn] r79636 - pypy/trunk/pypy/config Message-ID: <20101129124620.92CCC282BE3@codespeak.net> Author: arigo Date: Mon Nov 29 13:46:18 2010 New Revision: 79636 Modified: pypy/trunk/pypy/config/pypyoption.py Log: The binascii module translates. Modified: pypy/trunk/pypy/config/pypyoption.py ============================================================================== --- pypy/trunk/pypy/config/pypyoption.py (original) +++ pypy/trunk/pypy/config/pypyoption.py Mon Nov 29 13:46:18 2010 @@ -31,7 +31,7 @@ "crypt", "signal", "_rawffi", "termios", "zlib", "struct", "md5", "sha", "bz2", "_minimal_curses", "cStringIO", "thread", "itertools", "pyexpat", "_ssl", "cpyext", "array", - "_bisect"] + "_bisect", "binascii"] )) translation_modules = default_modules.copy() From arigo at codespeak.net Mon Nov 29 13:54:41 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 29 Nov 2010 13:54:41 +0100 (CET) Subject: [pypy-svn] r79637 - in pypy/extradoc/pypy.org: . source Message-ID: <20101129125441.DB9A2282BDA@codespeak.net> Author: arigo Date: Mon Nov 29 13:54:40 2010 New Revision: 79637 Modified: pypy/extradoc/pypy.org/download.html pypy/extradoc/pypy.org/source/download.txt Log: It is not recommended to use external tools like ``hg convert`` instead of ``svn``, because these are known to have issues with the PyPy repository. Modified: pypy/extradoc/pypy.org/download.html ============================================================================== --- pypy/extradoc/pypy.org/download.html (original) +++ pypy/extradoc/pypy.org/download.html Mon Nov 29 13:54:40 2010 @@ -117,6 +117,9 @@ 
     svn co http://codespeak.net/svn/pypy/trunk pypy-trunk
    +

    It is not recommended to use external tools like hg convert +instead of svn, because these are known to have issues with +the PyPy repository.

  • Make sure you installed the dependencies. See the list here.

    • Modified: pypy/extradoc/pypy.org/source/download.txt ============================================================================== --- pypy/extradoc/pypy.org/source/download.txt (original) +++ pypy/extradoc/pypy.org/source/download.txt Mon Nov 29 13:54:40 2010 @@ -104,6 +104,10 @@ svn co http://codespeak.net/svn/pypy/trunk pypy-trunk + It is not recommended to use external tools like ``hg convert`` + instead of ``svn``, because these are known to have issues with + the PyPy repository. + 2. Make sure you installed the dependencies. See the list here__. .. __: http://codespeak.net/pypy/dist/pypy/doc/getting-started-python.html#translating-the-pypy-python-interpreter From fijal at codespeak.net Mon Nov 29 14:14:39 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 29 Nov 2010 14:14:39 +0100 (CET) Subject: [pypy-svn] r79638 - in pypy/trunk/pypy/module/pyexpat: . test Message-ID: <20101129131439.A431C282BDD@codespeak.net> Author: fijal Date: Mon Nov 29 14:14:37 2010 New Revision: 79638 Modified: pypy/trunk/pypy/module/pyexpat/interp_pyexpat.py pypy/trunk/pypy/module/pyexpat/test/test_parser.py Log: (hodgestar) fix to pyexpat module issue578 resolved. Modified: pypy/trunk/pypy/module/pyexpat/interp_pyexpat.py ============================================================================== --- pypy/trunk/pypy/module/pyexpat/interp_pyexpat.py (original) +++ pypy/trunk/pypy/module/pyexpat/interp_pyexpat.py Mon Nov 29 14:14:37 2010 @@ -329,7 +329,7 @@ if self.returns_unicode: return space.call_function( space.getattr(space.wrap(s), space.wrap("decode")), - space.wrap(self.encoding), + space.wrap("utf-8"), space.wrap("strict")) else: return space.wrap(s) Modified: pypy/trunk/pypy/module/pyexpat/test/test_parser.py ============================================================================== --- pypy/trunk/pypy/module/pyexpat/test/test_parser.py (original) +++ pypy/trunk/pypy/module/pyexpat/test/test_parser.py Mon Nov 29 14:14:37 2010 @@ -15,3 +15,17 @@ assert res == 1 raises(pyexpat.ExpatError, p.Parse, "3") + + def test_encoding(self): + import pyexpat + for encoding_arg in (None, 'utf-8', 'iso-8859-1'): + for namespace_arg in (None, '{'): + print encoding_arg, namespace_arg + p = pyexpat.ParserCreate(encoding_arg, namespace_arg) + data = [] + p.CharacterDataHandler = lambda s: data.append(s) + encoding = encoding_arg is None and 'utf-8' or encoding_arg + + res = p.Parse(u"\u00f6".encode(encoding), isfinal=True) + assert res == 1 + assert data == [u"\u00f6"] From arigo at codespeak.net Mon Nov 29 14:19:33 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 29 Nov 2010 14:19:33 +0100 (CET) Subject: [pypy-svn] r79639 - pypy/trunk/lib_pypy Message-ID: <20101129131933.3FB0D282BD6@codespeak.net> Author: arigo Date: Mon Nov 29 14:19:31 2010 New Revision: 79639 Modified: pypy/trunk/lib_pypy/hashlib.py Log: Change this wrapper module to prefer the RPython versions of md5 and sha over the ctypes-calling-openssl. Modified: pypy/trunk/lib_pypy/hashlib.py ============================================================================== --- pypy/trunk/lib_pypy/hashlib.py (original) +++ pypy/trunk/lib_pypy/hashlib.py Mon Nov 29 14:19:31 2010 @@ -50,53 +50,30 @@ 'a4337bc45a8fc544c03f52dc550cd6e1e87021bc896588bd79e901e2' """ +import sys try: import _hashlib except ImportError: _hashlib = None -def __get_builtin_constructor(name): - if name in ('SHA1', 'sha1'): - import sha - return sha.new - elif name in ('MD5', 'md5'): - import md5 - return md5.new - elif name in ('SHA256', 'sha256'): - import _sha256 - return _sha256.sha256 - elif name in ('SHA224', 'sha224'): - import _sha256 - return _sha256.sha224 - elif name in ('SHA512', 'sha512'): - import _sha512 - return _sha512.sha512 - elif name in ('SHA384', 'sha384'): - import _sha512 - return _sha512.sha384 - raise ValueError, "unsupported hash type" - def __hash_new(name, string=''): """new(name, string='') - Return a new hashing object using the named algorithm; optionally initialized with a string. """ try: - if _hashlib: - return _hashlib.new(name, string) - except ValueError: - # If the _hashlib module (OpenSSL) doesn't support the named - # hash, try using our builtin implementations. - # This allows for SHA224/256 and SHA384/512 support even though - # the OpenSSL library prior to 0.9.8 doesn't provide them. - pass - - return __get_builtin_constructor(name)(string) + new = __byname[name] + except KeyError: + raise ValueError("unsupported hash type") + return new(string) new = __hash_new -def _setfuncs(): - # use the wrapper of the C implementation +# ____________________________________________________________ + +__byname = {} +def __use_openssl_funcs(): + # use the wrapper of the C implementation sslprefix = 'openssl_' for opensslfuncname, func in vars(_hashlib).items(): if not opensslfuncname.startswith(sslprefix): @@ -106,23 +83,41 @@ # try them all, some may not work due to the OpenSSL # version not supporting that algorithm. func() - # Use the C function directly (very fast) - globals()[funcname] = func + # Use the C function directly (very fast, but with ctypes overhead) + __byname[funcname] = func except ValueError: - try: - # Use the builtin implementation directly (fast) - globals()[funcname] = __get_builtin_constructor(funcname) - except ValueError: - # this one has no builtin implementation, don't define it - pass + pass + +def __use_builtin_funcs(): + # look up the built-in versions (written in Python or RPython), + # and use the fastest one: + # 1. the one in RPython + # 2. the one from openssl (slower due to ctypes calling overhead) + # 3. the one in pure Python + if 'sha1' not in __byname or 'sha' in sys.builtin_module_names: + import sha + __byname['sha1'] = sha.new + if 'md5' not in __byname or 'md5' in sys.builtin_module_names: + import md5 + __byname['md5'] = md5.new + if 'sha256' not in __byname: + import _sha256 + __byname['sha256'] = _sha256.sha256 + if 'sha224' not in __byname: + import _sha256 + __byname['sha224'] = _sha256.sha224 + if 'sha512' not in __byname: + import _sha512 + __byname['sha512'] = _sha512.sha512 + if 'sha384' not in __byname: + import _sha512 + __byname['sha384'] = _sha512.sha384 + +def __export_funcs(): + for key, value in __byname.items(): + globals()[key] = __byname[key.upper()] = value if _hashlib: - _setfuncs() -else: - # lookup the C function to use directly for the named constructors - md5 = __get_builtin_constructor('md5') - sha1 = __get_builtin_constructor('sha1') - sha224 = __get_builtin_constructor('sha224') - sha256 = __get_builtin_constructor('sha256') - sha384 = __get_builtin_constructor('sha384') - sha512 = __get_builtin_constructor('sha512') + __use_openssl_funcs() +__use_builtin_funcs() +__export_funcs() From arigo at codespeak.net Mon Nov 29 15:07:22 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 29 Nov 2010 15:07:22 +0100 (CET) Subject: [pypy-svn] r79640 - in pypy/extradoc/pypy.org: . source Message-ID: <20101129140722.62245282BDA@codespeak.net> Author: arigo Date: Mon Nov 29 15:07:15 2010 New Revision: 79640 Modified: pypy/extradoc/pypy.org/compat.html pypy/extradoc/pypy.org/source/compat.txt Log: Bold. Modified: pypy/extradoc/pypy.org/compat.html ============================================================================== --- pypy/extradoc/pypy.org/compat.html (original) +++ pypy/extradoc/pypy.org/compat.html Mon Nov 29 15:07:15 2010 @@ -51,7 +51,7 @@ language, passing Python test suite (with minor modifications that were already accepted in the main python in newer versions). It supports most of the commonly used Python standard library modules; details below.

    -

    PyPy has alpha-level support for the CPython C API, however, as of 1.4 +

    PyPy has alpha-level support for the CPython C API, however, as of 1.4 release this feature is not yet complete. Most libraries will require a bit of effort to work, but there are known success stories. Check out PyPy blog for updates.

    Modified: pypy/extradoc/pypy.org/source/compat.txt ============================================================================== --- pypy/extradoc/pypy.org/source/compat.txt (original) +++ pypy/extradoc/pypy.org/source/compat.txt Mon Nov 29 15:07:15 2010 @@ -8,7 +8,7 @@ already accepted in the main python in newer versions). It supports most of the commonly used Python `standard library modules`_; details below. -PyPy has alpha-level support for the `CPython C API`_, however, as of 1.4 +PyPy has **alpha-level** support for the `CPython C API`_, however, as of 1.4 release this feature is not yet complete. Most libraries will require a bit of effort to work, but there are known success stories. Check out PyPy blog for updates. From david at codespeak.net Mon Nov 29 15:36:39 2010 From: david at codespeak.net (david at codespeak.net) Date: Mon, 29 Nov 2010 15:36:39 +0100 (CET) Subject: [pypy-svn] r79641 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101129143639.2AB55282BFA@codespeak.net> Author: david Date: Mon Nov 29 15:36:38 2010 New Revision: 79641 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Log: Implement guard_class operations, currently only for boehm Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Mon Nov 29 15:36:38 2010 @@ -16,7 +16,7 @@ from pypy.jit.backend.llsupport.descr import BaseFieldDescr, BaseArrayDescr from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity, TempBox from pypy.jit.codewriter import heaptracker -from pypy.jit.metainterp.history import (Const, ConstInt, BoxInt, +from pypy.jit.metainterp.history import (Const, ConstInt, BoxInt, Box, BasicFailDescr, LoopToken, INT, REF) from pypy.jit.metainterp.resoperation import rop from pypy.rlib import rgc @@ -184,8 +184,8 @@ def _emit_guard(self, op, regalloc, fcond): descr = op.getdescr() assert isinstance(descr, BasicFailDescr) - if hasattr(op, 'getfailargs'): - print 'Failargs: ', op.getfailargs() + #if hasattr(op, 'getfailargs'): + # print 'Failargs: ', op.getfailargs() self.mc.ensure_can_fit(self.guard_size) self.mc.ADD_ri(r.pc.value, r.pc.value, self.guard_size, cond=fcond) descr._arm_guard_code = self.mc.curraddr() @@ -231,6 +231,52 @@ def emit_op_guard_overflow(self, op, regalloc, fcond): return self._emit_guard(op, regalloc, c.VS) + # from ../x86/assembler.py:1265 + def emit_op_guard_class(self, op, regalloc, fcond): + locs, boxes = self._prepare_guard_class(op, regalloc, fcond) + self._cmp_guard_class(op, locs, regalloc, fcond) + regalloc.possibly_free_vars(boxes) + return fcond + + def emit_op_guard_nonnull_class(self, op, regalloc, fcond): + locs, boxes = self._prepare_guard_class(op, regalloc, fcond) + self.mc.CMP_ri(locs[0].value, 0) + self._emit_guard(op, regalloc, c.NE) + self._cmp_guard_class(op, locs, regalloc, fcond) + regalloc.possibly_free_vars(boxes) + return fcond + + def _prepare_guard_class(self, op, regalloc, fcond): + assert isinstance(op.getarg(0), Box) + + x = regalloc.make_sure_var_in_reg(op.getarg(0), imm_fine=False) + y_temp = TempBox() + y = regalloc.force_allocate_reg(y_temp) + self.mc.gen_load_int(y.value, + self.cpu.cast_adr_to_int(op.getarg(1).getint())) + return [x, y], [op.getarg(0), y_temp] + + + def _cmp_guard_class(self, op, locs, regalloc, fcond): + offset = self.cpu.vtable_offset + x = locs[0] + y = locs[1] + if offset is not None: + t0 = TempBox() + l0 = regalloc.force_allocate_reg(t0) + assert offset == 0 + self.mc.LDR_ri(l0.value, x.value, offset) + self.mc.CMP_rr(l0.value, y.value) + regalloc.possibly_free_var(t0) + else: + raise NotImplentedError + # XXX port from x86 backend once gc support is in place + + regalloc.possibly_free_vars_for_op(op) + return self._emit_guard(op, regalloc, c.EQ) + + + class OpAssembler(object): _mixin_ = True Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Mon Nov 29 15:36:38 2010 @@ -43,8 +43,8 @@ pass def execute_token(self, executable_token): - i = [self.get_latest_value_int(x) for x in range(10)] - print 'Inputargs: %r for token %r' % (i, executable_token) + #i = [self.get_latest_value_int(x) for x in range(10)] + #print 'Inputargs: %r for token %r' % (i, executable_token) addr = executable_token._arm_bootstrap_code assert addr % 8 == 0 func = rffi.cast(lltype.Ptr(self.BOOTSTRAP_TP), addr) From arigo at codespeak.net Mon Nov 29 16:25:53 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 29 Nov 2010 16:25:53 +0100 (CET) Subject: [pypy-svn] r79642 - in pypy/extradoc/pypy.org: . source Message-ID: <20101129152553.04AC2282BEF@codespeak.net> Author: arigo Date: Mon Nov 29 16:25:50 2010 New Revision: 79642 Modified: pypy/extradoc/pypy.org/download.html pypy/extradoc/pypy.org/source/download.txt Log: Updates. Modified: pypy/extradoc/pypy.org/download.html ============================================================================== --- pypy/extradoc/pypy.org/download.html (original) +++ pypy/extradoc/pypy.org/download.html Mon Nov 29 16:25:50 2010 @@ -151,12 +151,16 @@ requirement down to 1.2 GB on 32-bit, 2.4 GB on 64-bit.) You should just start by downloading an official release of PyPy (with the JIT). -
  • In the final step, which is compiling the generated C files, the CFLAGS -are passed to the compiler. However, avoid passing too many custom -CFLAGS, particularly when compiling with asmgcroot (needed by the -JIT). It will make trackgcroot.py unhappy. In case you want to try -it out anyway, after a CompilationError you can change your CFLAGS -and retry by typing cd /tmp/usession-$USER/testing_1; make.
    • +
  • Because of asmgcroot, compiling the generated C files containing the +JIT is delicate. It requires using either MSVC or gcc with no particularly +fancy options. It does not work e.g. with clang, or if you pass uncommon +options with the CFLAGS environment variable. The generated C files +are left around in /tmp/usession-$USER/testing_1/, so if compilation +fails (i.e. you get a CompilationError) or if you want to build a +debug build, you can go there and retry running make. For example: +make CFLAGS="", or make lldebug. It is recommended to move this +directory away if you want to keep it; otherwise, further runs of +translate.py will remove it.
    • Modified: pypy/extradoc/pypy.org/source/download.txt ============================================================================== --- pypy/extradoc/pypy.org/source/download.txt (original) +++ pypy/extradoc/pypy.org/source/download.txt Mon Nov 29 16:25:50 2010 @@ -138,13 +138,16 @@ You should just start by downloading an official release of PyPy (with the JIT). -* In the final step, which is compiling the generated C files, the ``CFLAGS`` - are passed to the compiler. However, avoid passing too many custom - ``CFLAGS``, particularly when compiling with ``asmgcroot`` (needed by the - JIT). It will make ``trackgcroot.py`` unhappy. In case you want to try - it out anyway, after a CompilationError you can change your ``CFLAGS`` - and retry by typing ``cd /tmp/usession-$USER/testing_1; make``. - +* Because of ``asmgcroot``, compiling the generated C files containing the + JIT is delicate. It requires using either MSVC or gcc with no particularly + fancy options. It does not work e.g. with clang, or if you pass uncommon + options with the ``CFLAGS`` environment variable. The generated C files + are left around in ``/tmp/usession-$USER/testing_1/``, so if compilation + fails (i.e. you get a ``CompilationError``) or if you want to build a + debug build, you can go there and retry running ``make``. For example: + ``make CFLAGS=""``, or ``make lldebug``. It is recommended to move this + directory away if you want to keep it; otherwise, further runs of + ``translate.py`` will remove it. .. _`x86 (IA-32)`: http://en.wikipedia.org/wiki/IA-32 .. _`x86-64`: http://en.wikipedia.org/wiki/X86-64 From arigo at codespeak.net Mon Nov 29 16:31:15 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 29 Nov 2010 16:31:15 +0100 (CET) Subject: [pypy-svn] r79643 - in pypy/extradoc/pypy.org: . source Message-ID: <20101129153115.050F25080B@codespeak.net> Author: arigo Date: Mon Nov 29 16:31:14 2010 New Revision: 79643 Modified: pypy/extradoc/pypy.org/download.html pypy/extradoc/pypy.org/source/download.txt Log: Updates. Modified: pypy/extradoc/pypy.org/download.html ============================================================================== --- pypy/extradoc/pypy.org/download.html (original) +++ pypy/extradoc/pypy.org/download.html Mon Nov 29 16:31:14 2010 @@ -155,11 +155,12 @@ JIT is delicate. It requires using either MSVC or gcc with no particularly fancy options. It does not work e.g. with clang, or if you pass uncommon options with the CFLAGS environment variable. The generated C files -are left around in /tmp/usession-$USER/testing_1/, so if compilation +are left around in /tmp/usession-$USER/testing_1/ or +$TMPDIR/usession-$USER/testing_1, so if compilation fails (i.e. you get a CompilationError) or if you want to build a debug build, you can go there and retry running make. For example: -make CFLAGS="", or make lldebug. It is recommended to move this -directory away if you want to keep it; otherwise, further runs of +CFLAGS= make, or make lldebug. It is recommended to move this +directory away if you want to keep it; otherwise, future runs of translate.py will remove it.
    Modified: pypy/extradoc/pypy.org/source/download.txt ============================================================================== --- pypy/extradoc/pypy.org/source/download.txt (original) +++ pypy/extradoc/pypy.org/source/download.txt Mon Nov 29 16:31:14 2010 @@ -142,11 +142,12 @@ JIT is delicate. It requires using either MSVC or gcc with no particularly fancy options. It does not work e.g. with clang, or if you pass uncommon options with the ``CFLAGS`` environment variable. The generated C files - are left around in ``/tmp/usession-$USER/testing_1/``, so if compilation + are left around in ``/tmp/usession-$USER/testing_1/`` or + ``$TMPDIR/usession-$USER/testing_1``, so if compilation fails (i.e. you get a ``CompilationError``) or if you want to build a debug build, you can go there and retry running ``make``. For example: - ``make CFLAGS=""``, or ``make lldebug``. It is recommended to move this - directory away if you want to keep it; otherwise, further runs of + ``CFLAGS= make``, or ``make lldebug``. It is recommended to move this + directory away if you want to keep it; otherwise, future runs of ``translate.py`` will remove it. .. _`x86 (IA-32)`: http://en.wikipedia.org/wiki/IA-32 From antocuni at codespeak.net Mon Nov 29 16:35:28 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Mon, 29 Nov 2010 16:35:28 +0100 (CET) Subject: [pypy-svn] r79644 - in pypy/trunk: lib_pypy pypy/module/sys pypy/translator/goal pypy/translator/goal/test2 Message-ID: <20101129153528.B1CA0282BD6@codespeak.net> Author: antocuni Date: Mon Nov 29 16:35:27 2010 New Revision: 79644 Modified: pypy/trunk/lib_pypy/_pypy_interact.py pypy/trunk/pypy/module/sys/__init__.py pypy/trunk/pypy/translator/goal/app_main.py pypy/trunk/pypy/translator/goal/test2/test_app_main.py Log: set sys.{ps1,ps2} only when we enter the interactive prompt. There is code around which relies on it (e.g., my sitecustomize.py :-)) Modified: pypy/trunk/lib_pypy/_pypy_interact.py ============================================================================== --- pypy/trunk/lib_pypy/_pypy_interact.py (original) +++ pypy/trunk/lib_pypy/_pypy_interact.py Mon Nov 29 16:35:27 2010 @@ -4,6 +4,13 @@ def interactive_console(mainmodule=None): + # set sys.{ps1,ps2} just before invoking the interactive interpreter. This + # mimics what CPython does in pythonrun.c + if not hasattr(sys, 'ps1'): + sys.ps1 = '>>>> ' + if not hasattr(sys, 'ps1'): + sys.ps2 = '.... ' + # try: from _pypy_irc_topic import some_topic text = "And now for something completely different: ``%s''" % ( @@ -15,6 +22,7 @@ print text except ImportError: pass + # try: from pyrepl.simple_interact import check if not check(): Modified: pypy/trunk/pypy/module/sys/__init__.py ============================================================================== --- pypy/trunk/pypy/module/sys/__init__.py (original) +++ pypy/trunk/pypy/module/sys/__init__.py Mon Nov 29 16:35:27 2010 @@ -60,8 +60,6 @@ 'pypy_svn_url' : 'version.get_svn_url(space)', 'subversion' : 'version.get_subversion_info(space)', 'hexversion' : 'version.get_hexversion(space)', - 'ps1' : 'space.wrap(">>>> ")', - 'ps2' : 'space.wrap(".... ")', 'displayhook' : 'hook.displayhook', '__displayhook__' : 'hook.__displayhook__', Modified: pypy/trunk/pypy/translator/goal/app_main.py ============================================================================== --- pypy/trunk/pypy/translator/goal/app_main.py (original) +++ pypy/trunk/pypy/translator/goal/app_main.py Mon Nov 29 16:35:27 2010 @@ -495,13 +495,13 @@ sys.pypy_version_info = PYPY_VERSION sys.pypy_initial_path = pypy_initial_path os = nanos.os_module_for_testing - sys.ps1 = '>>>> ' - sys.ps2 = '.... ' try: sys.exit(int(entry_point(sys.argv[0], sys.argv[1:], os))) finally: - sys.ps1 = '>>> ' # restore the normal ones, in case - sys.ps2 = '... ' # we are dropping to CPython's prompt + # restore the normal prompt (which was changed by _pypy_interact), in + # case we are dropping to CPython's prompt + sys.ps1 = '>>> ' + sys.ps2 = '... ' import os; os.environ.update(reset) assert old_argv is sys.argv assert old_path is sys.path Modified: pypy/trunk/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/trunk/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/trunk/pypy/translator/goal/test2/test_app_main.py Mon Nov 29 16:35:27 2010 @@ -329,6 +329,10 @@ child = self.spawn(['-mpypy.translator.goal.test2.mymodule']) child.expect('mymodule running') + def test_ps1_only_if_interactive(self): + argv = ['-c', 'import sys; print hasattr(sys, "ps1")'] + child = self.spawn(argv) + child.expect('False') class TestNonInteractive: From antocuni at codespeak.net Mon Nov 29 16:39:25 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Mon, 29 Nov 2010 16:39:25 +0100 (CET) Subject: [pypy-svn] r79645 - pypy/trunk/lib_pypy Message-ID: <20101129153925.736A9282BD6@codespeak.net> Author: antocuni Date: Mon Nov 29 16:39:23 2010 New Revision: 79645 Modified: pypy/trunk/lib_pypy/_pypy_interact.py Log: (thanks arigo) obvious typo, it's a bit hard to write a test for it, though Modified: pypy/trunk/lib_pypy/_pypy_interact.py ============================================================================== --- pypy/trunk/lib_pypy/_pypy_interact.py (original) +++ pypy/trunk/lib_pypy/_pypy_interact.py Mon Nov 29 16:39:23 2010 @@ -8,7 +8,7 @@ # mimics what CPython does in pythonrun.c if not hasattr(sys, 'ps1'): sys.ps1 = '>>>> ' - if not hasattr(sys, 'ps1'): + if not hasattr(sys, 'ps2'): sys.ps2 = '.... ' # try: From cfbolz at codespeak.net Mon Nov 29 17:21:26 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 29 Nov 2010 17:21:26 +0100 (CET) Subject: [pypy-svn] r79646 - pypy/trunk/pypy/doc/statistic Message-ID: <20101129162126.88687282BF3@codespeak.net> Author: cfbolz Date: Mon Nov 29 17:21:23 2010 New Revision: 79646 Modified: pypy/trunk/pypy/doc/statistic/sprint_dates.dat Log: add some of the more recent sprints Modified: pypy/trunk/pypy/doc/statistic/sprint_dates.dat ============================================================================== --- pypy/trunk/pypy/doc/statistic/sprint_dates.dat (original) +++ pypy/trunk/pypy/doc/statistic/sprint_dates.dat Mon Nov 29 17:21:23 2010 @@ -1,26 +1,37 @@ PyPy sprints location, begin, end "Hildesheim",2003-02-17,2003-02-23 -"Gothenburg",2003-05-24,2003-05-31 -"LovainLaNeuve",2003-06-21,2003-06-24 +"G?teborg",2003-05-24,2003-05-31 +"Lovain-la-Neuve",2003-06-21,2003-06-24 "Berlin",2003-09-29,2003-10-04 "Amsterdam",2003-12-14,2003-12-21 -"Europython/Gothenburg",2004-06-01,2004-06-07 +"Europython/G?teborg",2004-06-01,2004-06-07 "Vilnius",2004-11-15,2004-11-23 "Leysin",2005-01-22,2005-01-29 "PyCon/Washington",2005-03-19,2005-03-22 -"Europython/Gothenburg",2005-07-01,2005-07-07 +"Europython/G?teborg",2005-07-01,2005-07-07 "Hildesheim",2005-07-25,2005-07-31 "Heidelberg",2005-08-22,2005-08-29 "Paris",2005-10-10,2005-10-16 -"Gothenburg",2005-12-05,2005-12-11 +"G?teborg",2005-12-05,2005-12-11 "Mallorca",2006-01-23,2006-01-29 "Pycon/Dallas",2006-02-27,2006-03-02 "Louvain-la-Neuve",2006-03-06,2006-03-10 "Japan",2006-04-23,2006-04-29 -"Duesseldorf",2006-06-02,2006-06-09 +"D?sseldorf",2006-06-02,2006-06-09 "Europython/Genf",2006-07-06,2006-07-09 "Limerick",2006-08-21,2006-08-27 -"Duesseldorf",2006-10-30,2006-11-05 +"D?sseldorf",2006-10-30,2006-11-05 "Leysin",2007-01-08,2007-01-14 "Hildesheim",2007-03-01,2007-03-05 +"G?teborg",2007-11-19,2007-11-25 +"Leysin",2008-01-12,2008-01-19 +"Berlin",2008-05-17,2008-05-22 +"EuroPython/Vilnius",2008-07-10,2008-07-12 +"D?sseldorf",2008-08-05,2008-08-13 +"Wroclaw",2009-02-07,2008-02-14 +"Leysin",2009-04-14,2009-04-21 +"G?teborg",2009-08-18,2009-08-25 +"D?sseldorf",2009-11-06,2009-11-13 +"CERN",2010-07-05,2010-07-09 +"D?sseldorf",2010-10-25,2010-10-31 From cfbolz at codespeak.net Mon Nov 29 17:28:29 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 29 Nov 2010 17:28:29 +0100 (CET) Subject: [pypy-svn] r79647 - pypy/trunk/pypy/doc/statistic Message-ID: <20101129162829.E366B282BF3@codespeak.net> Author: cfbolz Date: Mon Nov 29 17:28:28 2010 New Revision: 79647 Modified: pypy/trunk/pypy/doc/statistic/sprint_dates.dat Log: missed the bern sprint Modified: pypy/trunk/pypy/doc/statistic/sprint_dates.dat ============================================================================== --- pypy/trunk/pypy/doc/statistic/sprint_dates.dat (original) +++ pypy/trunk/pypy/doc/statistic/sprint_dates.dat Mon Nov 29 17:28:28 2010 @@ -24,6 +24,7 @@ "D?sseldorf",2006-10-30,2006-11-05 "Leysin",2007-01-08,2007-01-14 "Hildesheim",2007-03-01,2007-03-05 +"Bern",2007-10-22,2007-10-26 "G?teborg",2007-11-19,2007-11-25 "Leysin",2008-01-12,2008-01-19 "Berlin",2008-05-17,2008-05-22 From cfbolz at codespeak.net Mon Nov 29 17:32:22 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 29 Nov 2010 17:32:22 +0100 (CET) Subject: [pypy-svn] r79648 - pypy/trunk/pypy/doc/statistic Message-ID: <20101129163222.02AE8282BF3@codespeak.net> Author: cfbolz Date: Mon Nov 29 17:32:21 2010 New Revision: 79648 Modified: pypy/trunk/pypy/doc/statistic/release_dates.dat Log: release dates Modified: pypy/trunk/pypy/doc/statistic/release_dates.dat ============================================================================== --- pypy/trunk/pypy/doc/statistic/release_dates.dat (original) +++ pypy/trunk/pypy/doc/statistic/release_dates.dat Mon Nov 29 17:32:21 2010 @@ -7,3 +7,7 @@ 2006-06-25,"PyPy 0.9" 2007-02-17,"PyPy 0.99" 2007-03-27,"PyPy 1.0" +2009-04-28,"PyPy 1.1" +2010-03-12,"PyPy 1.2" +2010-06-26,"PyPy 1.3" +2010-11-26,"PyPy 1.4" From afa at codespeak.net Mon Nov 29 17:34:46 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 29 Nov 2010 17:34:46 +0100 (CET) Subject: [pypy-svn] r79649 - pypy/trunk/pypy/doc/statistic Message-ID: <20101129163446.811FC5080B@codespeak.net> Author: afa Date: Mon Nov 29 17:34:45 2010 New Revision: 79649 Modified: pypy/trunk/pypy/doc/statistic/sprint_dates.dat Log: Fix a typo Modified: pypy/trunk/pypy/doc/statistic/sprint_dates.dat ============================================================================== --- pypy/trunk/pypy/doc/statistic/sprint_dates.dat (original) +++ pypy/trunk/pypy/doc/statistic/sprint_dates.dat Mon Nov 29 17:34:45 2010 @@ -2,7 +2,7 @@ location, begin, end "Hildesheim",2003-02-17,2003-02-23 "G?teborg",2003-05-24,2003-05-31 -"Lovain-la-Neuve",2003-06-21,2003-06-24 +"Louvain-la-Neuve",2003-06-21,2003-06-24 "Berlin",2003-09-29,2003-10-04 "Amsterdam",2003-12-14,2003-12-21 "Europython/G?teborg",2004-06-01,2004-06-07 From cfbolz at codespeak.net Mon Nov 29 17:36:21 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 29 Nov 2010 17:36:21 +0100 (CET) Subject: [pypy-svn] r79650 - pypy/trunk/pypy/doc/statistic Message-ID: <20101129163621.17BF1282BFD@codespeak.net> Author: cfbolz Date: Mon Nov 29 17:36:19 2010 New Revision: 79650 Modified: pypy/trunk/pypy/doc/statistic/sprint_dates.dat Log: typo Modified: pypy/trunk/pypy/doc/statistic/sprint_dates.dat ============================================================================== --- pypy/trunk/pypy/doc/statistic/sprint_dates.dat (original) +++ pypy/trunk/pypy/doc/statistic/sprint_dates.dat Mon Nov 29 17:36:19 2010 @@ -30,7 +30,7 @@ "Berlin",2008-05-17,2008-05-22 "EuroPython/Vilnius",2008-07-10,2008-07-12 "D?sseldorf",2008-08-05,2008-08-13 -"Wroclaw",2009-02-07,2008-02-14 +"Wroclaw",2009-02-07,2009-02-14 "Leysin",2009-04-14,2009-04-21 "G?teborg",2009-08-18,2009-08-25 "D?sseldorf",2009-11-06,2009-11-13 From arigo at codespeak.net Mon Nov 29 17:42:46 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 29 Nov 2010 17:42:46 +0100 (CET) Subject: [pypy-svn] r79651 - pypy/trunk/pypy/module/binascii Message-ID: <20101129164246.2845C282BDD@codespeak.net> Author: arigo Date: Mon Nov 29 17:42:44 2010 New Revision: 79651 Modified: pypy/trunk/pypy/module/binascii/interp_crc32.py Log: Fix crc32 on 64 bits. Modified: pypy/trunk/pypy/module/binascii/interp_crc32.py ============================================================================== --- pypy/trunk/pypy/module/binascii/interp_crc32.py (original) +++ pypy/trunk/pypy/module/binascii/interp_crc32.py Mon Nov 29 17:42:44 2010 @@ -1,5 +1,6 @@ from pypy.interpreter.gateway import ObjSpace from pypy.rlib.rarithmetic import r_uint, intmask +from pypy.rpython.lltypesystem import rffi # ____________________________________________________________ @@ -57,19 +58,18 @@ 0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d ] -def _make_signed_32(v): - if v >= 0x80000000: - v -= 0x100000000 - return int(v) -crc_32_tab = map(_make_signed_32, crc_32_tab) +crc_32_tab = map(intmask, crc_32_tab) def crc32(space, data, oldcrc=0): "Compute the CRC-32 incrementally." - crc = ~oldcrc + + crc = r_uint(rffi.cast(rffi.UINT, ~oldcrc)) # signed => 32-bit unsigned + + # in the following loop, we have always 0 <= crc < 2**32 for c in data: - crc_shifted_by_8 = intmask(r_uint(crc) >> 8) - crc = crc_32_tab[(crc & 0xff) ^ ord(c)] ^ crc_shifted_by_8 - crc = ~crc + crc = crc_32_tab[(crc & 0xff) ^ ord(c)] ^ (crc >> 8) + + crc = ~intmask(rffi.cast(rffi.INT, crc)) # unsigned => 32-bit signed return space.wrap(crc) crc32.unwrap_spec = [ObjSpace, 'bufferstr', 'c_int'] From arigo at codespeak.net Mon Nov 29 17:48:30 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Mon, 29 Nov 2010 17:48:30 +0100 (CET) Subject: [pypy-svn] r79652 - pypy/trunk/pypy/module/binascii Message-ID: <20101129164830.8F29A282BDD@codespeak.net> Author: arigo Date: Mon Nov 29 17:48:28 2010 New Revision: 79652 Modified: pypy/trunk/pypy/module/binascii/interp_crc32.py Log: Fix this. It works anyway, but it's clearer. Modified: pypy/trunk/pypy/module/binascii/interp_crc32.py ============================================================================== --- pypy/trunk/pypy/module/binascii/interp_crc32.py (original) +++ pypy/trunk/pypy/module/binascii/interp_crc32.py Mon Nov 29 17:48:28 2010 @@ -58,7 +58,7 @@ 0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d ] -crc_32_tab = map(intmask, crc_32_tab) +crc_32_tab = map(r_uint, crc_32_tab) def crc32(space, data, oldcrc=0): From cfbolz at codespeak.net Mon Nov 29 17:54:45 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 29 Nov 2010 17:54:45 +0100 (CET) Subject: [pypy-svn] r79653 - pypy/trunk/pypy/doc/statistic Message-ID: <20101129165445.CC0ED5080B@codespeak.net> Author: cfbolz Date: Mon Nov 29 17:54:44 2010 New Revision: 79653 Modified: pypy/trunk/pypy/doc/statistic/sprint_dates.dat Log: add the report sprint too, just because Modified: pypy/trunk/pypy/doc/statistic/sprint_dates.dat ============================================================================== --- pypy/trunk/pypy/doc/statistic/sprint_dates.dat (original) +++ pypy/trunk/pypy/doc/statistic/sprint_dates.dat Mon Nov 29 17:54:44 2010 @@ -24,6 +24,7 @@ "D?sseldorf",2006-10-30,2006-11-05 "Leysin",2007-01-08,2007-01-14 "Hildesheim",2007-03-01,2007-03-05 +"Hildesheim",2007-03-18,2007-03-23 "Bern",2007-10-22,2007-10-26 "G?teborg",2007-11-19,2007-11-25 "Leysin",2008-01-12,2008-01-19 From cfbolz at codespeak.net Mon Nov 29 18:57:49 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Mon, 29 Nov 2010 18:57:49 +0100 (CET) Subject: [pypy-svn] r79654 - pypy/trunk/pypy/doc Message-ID: <20101129175749.563C6282BDD@codespeak.net> Author: cfbolz Date: Mon Nov 29 18:57:47 2010 New Revision: 79654 Modified: pypy/trunk/pypy/doc/sprint-reports.txt Log: update list of sprint reports, while I'm at it Modified: pypy/trunk/pypy/doc/sprint-reports.txt ============================================================================== --- pypy/trunk/pypy/doc/sprint-reports.txt (original) +++ pypy/trunk/pypy/doc/sprint-reports.txt Mon Nov 29 18:57:47 2010 @@ -30,6 +30,17 @@ * `D?sseldorf (October 2006)`_ * `Leysin (January 2007)`_ * `Hildesheim (Feb 2007)`_ (also `EU report writing sprint`_) + * `G?teborg (November 2007)`_ + * `Leysin (January 2008)`_ + * `Berlin (May 2008)`_ + * `Vilnius after EuroPython (July 2008)`_ + * `D?sseldorf (August 2008)`_ + * `Wroclaw (February 2009)`_ + * `Leysin (April 2009)`_ + * `G?teborg (August 2009)`_ + * `D?sseldorf (November 2009)`_ + * `CERN (July 2010)`_ + * `D?sseldorf (October 2010)`_ .. _Hildesheim (Feb 2003): http://codespeak.net/pypy/extradoc/sprintinfo/HildesheimReport.html .. _Gothenburg (May 2003): http://codespeak.net/pypy/extradoc/sprintinfo/gothenburg-2003-sprintreport.txt @@ -55,3 +66,15 @@ .. _Hildesheim (Feb 2007): http://codespeak.net/pypy/extradoc/sprintinfo/trillke-2007/sprint-report.txt .. _`EU report writing sprint`: http://codespeak.net/pypy/extradoc/sprintinfo/trillke-2007/eu-report-sprint-report.txt .. _`PyCon/Dallas (Feb 2006)`: http://codespeak.net/pypy/extradoc/sprintinfo/pycon06/sprint-report.txt + .. _`G?teborg (November 2007)`: http://morepypy.blogspot.com/2007_11_01_archive.html + .. _`Leysin (January 2008)`: http://morepypy.blogspot.com/2008/01/leysin-winter-sport-sprint-started.html + .. _`Berlin (May 2008)`: http://morepypy.blogspot.com/2008_05_01_archive.html + .. _`Vilnius after EuroPython (July 2008)`: http://morepypy.blogspot.com/2008/07/europython-2008-pypy-talks-and-sprint.html + .. _`D?sseldorf (August 2008)`: http://morepypy.blogspot.com/2008_10_01_archive.html + .. _`Wroclaw (February 2009)`: http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.html + .. _`Leysin (April 2009)`: http://morepypy.blogspot.com/2009/04/leysin-sprint-report.html + .. _`G?teborg (August 2009)`: http://morepypy.blogspot.com/2009/08/gothenburg-jit-sprint-report.html + .. _`D?sseldorf (November 2009)`: http://morepypy.blogspot.com/2009/11/dusseldorf-sprint-report.html + .. _`CERN (July 2010)`: http://morepypy.blogspot.com/2010/07/cern-sprint-report-wrapping-c-libraries.html + .. _`D?sseldorf (October 2010)`: http://morepypy.blogspot.com/2010/10/dusseldorf-sprint-report-2010.html + From afa at codespeak.net Mon Nov 29 19:13:45 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Mon, 29 Nov 2010 19:13:45 +0100 (CET) Subject: [pypy-svn] r79655 - in pypy/trunk/lib-python/modified-2.5.2/distutils: . tests Message-ID: <20101129181345.13532282BE9@codespeak.net> Author: afa Date: Mon Nov 29 19:13:43 2010 New Revision: 79655 Modified: pypy/trunk/lib-python/modified-2.5.2/distutils/msvccompiler.py pypy/trunk/lib-python/modified-2.5.2/distutils/tests/test_msvccompiler.py Log: distutils.msvccompiler.MSVCCompiler should be importable on Unix. Modified: pypy/trunk/lib-python/modified-2.5.2/distutils/msvccompiler.py ============================================================================== --- pypy/trunk/lib-python/modified-2.5.2/distutils/msvccompiler.py (original) +++ pypy/trunk/lib-python/modified-2.5.2/distutils/msvccompiler.py Mon Nov 29 19:13:43 2010 @@ -25,17 +25,20 @@ from distutils import log from distutils.util import get_platform -import _winreg - -RegOpenKeyEx = _winreg.OpenKeyEx -RegEnumKey = _winreg.EnumKey -RegEnumValue = _winreg.EnumValue -RegError = _winreg.error - -HKEYS = (_winreg.HKEY_USERS, - _winreg.HKEY_CURRENT_USER, - _winreg.HKEY_LOCAL_MACHINE, - _winreg.HKEY_CLASSES_ROOT) +try: + import _winreg +except ImportError: + pass +else: + RegOpenKeyEx = _winreg.OpenKeyEx + RegEnumKey = _winreg.EnumKey + RegEnumValue = _winreg.EnumValue + RegError = _winreg.error + + HKEYS = (_winreg.HKEY_USERS, + _winreg.HKEY_CURRENT_USER, + _winreg.HKEY_LOCAL_MACHINE, + _winreg.HKEY_CLASSES_ROOT) VS_BASE = r"Software\Microsoft\VisualStudio\%0.1f" VSEXPRESS_BASE = r"Software\Microsoft\VCExpress\%0.1f" Modified: pypy/trunk/lib-python/modified-2.5.2/distutils/tests/test_msvccompiler.py ============================================================================== --- pypy/trunk/lib-python/modified-2.5.2/distutils/tests/test_msvccompiler.py (original) +++ pypy/trunk/lib-python/modified-2.5.2/distutils/tests/test_msvccompiler.py Mon Nov 29 19:13:43 2010 @@ -11,11 +11,15 @@ else: self.fail("could not find a suitable manifest") +class MsvcCompilerSimplerTestCase(unittest.TestCase): + def test_import_module(self): + from distutils.msvccompiler import MSVCCompiler + def test_suite(): if sys.platform == 'win32': return unittest.makeSuite(MsvcCompilerTestCase) else: - return unittest.TestSuite([]) + return unittest.makeSuite(MsvcCompilerSimplerTestCase) if __name__ == "__main__": unittest.main(defaultTest="test_suite") From fijal at codespeak.net Mon Nov 29 20:46:58 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Mon, 29 Nov 2010 20:46:58 +0100 (CET) Subject: [pypy-svn] r79656 - in pypy/extradoc/talk/ctpug2010: . demo Message-ID: <20101129194658.DB495282BE0@codespeak.net> Author: fijal Date: Mon Nov 29 20:46:54 2010 New Revision: 79656 Added: pypy/extradoc/talk/ctpug2010/Makefile - copied unchanged from r79573, pypy/extradoc/talk/ep2010/talk/Makefile pypy/extradoc/talk/ctpug2010/author.latex - copied, changed from r79573, pypy/extradoc/talk/ep2010/talk/author.latex pypy/extradoc/talk/ctpug2010/beamerdefs.txt - copied unchanged from r79573, pypy/extradoc/talk/ep2010/talk/beamerdefs.txt pypy/extradoc/talk/ctpug2010/demo/ pypy/extradoc/talk/ctpug2010/demo/mapdict.py pypy/extradoc/talk/ctpug2010/demo/mapdict.py~ pypy/extradoc/talk/ctpug2010/demo/mapdictlog pypy/extradoc/talk/ctpug2010/demo/simple.py pypy/extradoc/talk/ctpug2010/demo/simple.pyc (contents, props changed) pypy/extradoc/talk/ctpug2010/demo/simplelog pypy/extradoc/talk/ctpug2010/demo/source.py pypy/extradoc/talk/ctpug2010/demo/source.py~ pypy/extradoc/talk/ctpug2010/demo/sourcelog pypy/extradoc/talk/ctpug2010/stylesheet.latex - copied unchanged from r79573, pypy/extradoc/talk/ep2010/talk/stylesheet.latex pypy/extradoc/talk/ctpug2010/talk.pdf (contents, props changed) pypy/extradoc/talk/ctpug2010/title.latex - copied, changed from r79573, pypy/extradoc/talk/ep2010/talk/title.latex pypy/extradoc/talk/ctpug2010/wxpython1.png - copied unchanged from r79573, pypy/extradoc/talk/ep2010/talk/wxpython1.png pypy/extradoc/talk/ctpug2010/wxpython2.png - copied unchanged from r79573, pypy/extradoc/talk/ep2010/talk/wxpython2.png Log: talk as it went Copied: pypy/extradoc/talk/ctpug2010/author.latex (from r79573, pypy/extradoc/talk/ep2010/talk/author.latex) ============================================================================== --- pypy/extradoc/talk/ep2010/talk/author.latex (original) +++ pypy/extradoc/talk/ctpug2010/author.latex Mon Nov 29 20:46:54 2010 @@ -1,8 +1,8 @@ \definecolor{rrblitbackground}{rgb}{0.0, 0.0, 0.0} -\title[PyPy 1.3]{PyPy \sout{1.2} 1.3: Status and News} -\author[amaury, antocuni, arigato] -{Amaury Forgeot d'Arc \\ Antonio Cuni \\ Armin Rigo} +\title[PyPy 1.4]{PyPy 1.4: Status and News} +\author[fijal] +{Maciej Fijalkowski} -\institute{EuroPython 2010} -\date{July 19 2010} +\institute{Cape Town PUG} +\date{November 27 2010 } Added: pypy/extradoc/talk/ctpug2010/demo/mapdict.py ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ctpug2010/demo/mapdict.py Mon Nov 29 20:46:54 2010 @@ -0,0 +1,13 @@ + +class A(object): + def __init__(self, i, next): + self.i = i + self.next = next + +def f(): + next = None + for i in range(2000): + next = A(i, next) + +if __name__ == '__main__': + f() Added: pypy/extradoc/talk/ctpug2010/demo/mapdict.py~ ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ctpug2010/demo/mapdict.py~ Mon Nov 29 20:46:54 2010 @@ -0,0 +1,5 @@ + +def f(): + l = [] + for i in range(2000): + l.append(A(i)) Added: pypy/extradoc/talk/ctpug2010/demo/mapdictlog ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ctpug2010/demo/mapdictlog Mon Nov 29 20:46:54 2010 @@ -0,0 +1,89 @@ +[753449cdb5df] {jit-log-opt-loop +# Loop 0 : loop with 80 ops +[p0, p1, p2, p3, i4, p5, p6, p7, p8, p9, p10] +debug_merge_point(' #19 FOR_ITER', 0) +guard_class(p5, 17375104, descr=) [p1, p0, p5, p2, p3, p6, p7, p8, p9, p10] +p12 = getfield_gc(p5, descr=) +guard_nonnull(p12, descr=) [p1, p0, p5, p12, p2, p3, p6, p7, p8, p9, p10] +i13 = getfield_gc(p5, descr=) +p14 = getfield_gc(p12, descr=) +guard_isnull(p14, descr=) [p1, p0, p5, i13, p12, p14, p2, p3, p6, p7, p8, p9, p10] +i15 = getfield_gc(p12, descr=) +i16 = int_ge(i13, i15) +guard_false(i16, descr=) [p1, p0, p5, i13, p12, p2, p3, p6, p7, p8, p9, p10] +i17 = getfield_gc(p12, descr=) +i18 = getfield_gc(p12, descr=) +i19 = int_mul(i13, i18) +i20 = int_add(i17, i19) +i22 = int_add(i13, 1) +debug_merge_point(' #22 STORE_FAST', 0) +debug_merge_point(' #25 LOAD_GLOBAL', 0) +p23 = getfield_gc(p0, descr=) +setfield_gc(p5, i22, descr=) +guard_value(p23, ConstPtr(ptr24), descr=) [p1, p0, p23, p2, p3, p5, p7, p8, p9, i20] +p25 = getfield_gc(p23, descr=) +guard_isnull(p25, descr=) [p1, p0, p25, p23, p2, p3, p5, p7, p8, p9, i20] +p27 = getfield_gc(ConstPtr(ptr26), descr=) +guard_value(p27, ConstPtr(ptr28), descr=) [p1, p0, p27, p2, p3, p5, p7, p8, p9, i20] +debug_merge_point(' #28 LOAD_FAST', 0) +debug_merge_point(' #31 LOAD_FAST', 0) +guard_nonnull(p9, descr=) [p1, p0, p9, p2, p3, p5, p27, p8, i20] +debug_merge_point(' #34 CALL_FUNCTION', 0) +i29 = getfield_gc(p0, descr=) +guard_false(i29, descr=) [p1, p0, p27, p2, p3, p5, p9, i20] +p30 = getfield_gc(ConstPtr(ptr28), descr=) +guard_value(p30, ConstPtr(ptr31), descr=) [p1, p0, p30, p2, p3, p5, p9, i20] +p33 = getfield_gc(ConstPtr(ptr32), descr=) +guard_value(p33, ConstPtr(ptr34), descr=) [p1, p0, p33, p2, p3, p5, p9, i20] +p35 = getfield_gc(ConstPtr(ptr32), descr=) +p36 = getfield_gc(ConstPtr(ptr32), descr=) +p37 = getfield_gc(ConstPtr(ptr32), descr=) +p38 = getfield_gc(ConstPtr(ptr32), descr=) +p40 = call(ConstClass(getexecutioncontext), descr=) +guard_no_exception(, descr=) [p1, p0, p40, p2, p3, p5, p9, p35, i20] +i41 = getfield_gc(p40, descr=) +i43 = getfield_gc(ConstPtr(ptr42), descr=) +i44 = int_gt(i41, i43) +guard_false(i44, descr=) [p1, p0, p40, p2, p3, p5, p9, p35, i20] +i46 = int_add(i41, 1) +p47 = getfield_gc(p40, descr=) +i48 = force_token() +p49 = getfield_gc(p40, descr=) +setfield_gc(p40, i46, descr=) +guard_isnull(p49, descr=) [p1, p0, p40, p49, p2, p3, p5, p9, p47, i48, p35, i20] +i50 = getfield_gc(p40, descr=) +i51 = int_is_zero(i50) +guard_true(i51, descr=) [p1, p0, p40, p2, p3, p5, p9, p47, i48, p35, i20] +debug_merge_point(' #0 LOAD_FAST', 1) +debug_merge_point(' #3 LOAD_FAST', 1) +debug_merge_point(' #6 STORE_ATTR', 1) +debug_merge_point(' #9 LOAD_FAST', 1) +debug_merge_point(' #12 LOAD_FAST', 1) +debug_merge_point(' #15 STORE_ATTR', 1) +debug_merge_point(' #18 LOAD_CONST', 1) +debug_merge_point(' #21 RETURN_VALUE', 1) +i52 = int_is_true(i50) +guard_false(i52, descr=) [p1, p0, p40, p2, p3, p5, p9, p47, i48, p35, i20] +debug_merge_point(' #37 STORE_FAST', 0) +debug_merge_point(' #40 JUMP_ABSOLUTE', 0) +i54 = getfield_raw(32212256, descr=) +i56 = int_sub(i54, 18) +setfield_raw(32212256, i56, descr=) +setfield_gc(p40, i41, descr=) +setfield_gc(p40, p47, descr=) +i58 = int_lt(i56, 0) +guard_false(i58, descr=) [p1, p0, p2, p3, p5, p9, None, None, None, i20] +debug_merge_point(' #19 FOR_ITER', 0) +p61 = new_with_vtable(17402248) +p63 = new_with_vtable(ConstClass(W_IntObject)) +setfield_gc(p63, i20, descr=) +setfield_gc(p61, p63, descr=) +setfield_gc(p61, p9, descr=) +setfield_gc(p61, ConstPtr(ptr64), descr=) +jump(p0, p1, p2, p3, 19, p5, ConstPtr(ptr66), ConstPtr(ptr67), ConstPtr(ptr68), p61, p63, descr=) +[753449d79c95] jit-log-opt-loop} +[753449d9bc25] {jit-backend-counts +[753449d9c5e5] jit-backend-counts} +[75344a0a1909] {jit-backend-counts +0:172 +[75344a0a39f7] jit-backend-counts} Added: pypy/extradoc/talk/ctpug2010/demo/simple.py ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ctpug2010/demo/simple.py Mon Nov 29 20:46:54 2010 @@ -0,0 +1,8 @@ + +def f(): + i = 0 + while i < 10000: + i += 1 + +if __name__ == '__main__': + f() Added: pypy/extradoc/talk/ctpug2010/demo/simple.pyc ============================================================================== Binary file. No diff available. Added: pypy/extradoc/talk/ctpug2010/demo/simplelog ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ctpug2010/demo/simplelog Mon Nov 29 20:46:54 2010 @@ -0,0 +1,29 @@ +[2f4327f16cb] {jit-log-opt-loop +# Loop 0 : loop with 20 ops +[p0, p1, p2, p3, i4, p5, p6, i7] +debug_merge_point(' #9 LOAD_FAST', 0) +debug_merge_point(' #12 LOAD_CONST', 0) +debug_merge_point(' #15 COMPARE_OP', 0) +i9 = int_lt(i7, 10000) +guard_true(i9, descr=) [p1, p0, p2, p3, i7] +debug_merge_point(' #18 JUMP_IF_FALSE', 0) +debug_merge_point(' #21 POP_TOP', 0) +debug_merge_point(' #22 LOAD_FAST', 0) +debug_merge_point(' #25 LOAD_CONST', 0) +debug_merge_point(' #28 INPLACE_ADD', 0) +i11 = int_add(i7, 1) +debug_merge_point(' #29 STORE_FAST', 0) +debug_merge_point(' #32 JUMP_ABSOLUTE', 0) +i13 = getfield_raw(32212256, descr=) +i15 = int_sub(i13, 1) +setfield_raw(32212256, i15, descr=) +i17 = int_lt(i15, 0) +guard_false(i17, descr=) [p1, p0, p2, p3, i11, None] +debug_merge_point(' #9 LOAD_FAST', 0) +jump(p0, p1, p2, p3, 9, ConstPtr(ptr20), ConstPtr(ptr21), i11, descr=) +[2f4328431d5] jit-log-opt-loop} +[2f432865e55] {jit-backend-counts +[2f4328667f7] jit-backend-counts} +[2f4328eaaf1] {jit-backend-counts +0:9000 +[2f4328ec522] jit-backend-counts} Added: pypy/extradoc/talk/ctpug2010/demo/source.py ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ctpug2010/demo/source.py Mon Nov 29 20:46:54 2010 @@ -0,0 +1,11 @@ + +def g(i): + return i + 1 + +def f(): + i = 0 + while i < 10000: + i = g(i) + +if __name__ == '__main__': + f() Added: pypy/extradoc/talk/ctpug2010/demo/source.py~ ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ctpug2010/demo/source.py~ Mon Nov 29 20:46:54 2010 @@ -0,0 +1,16 @@ + +def g(i): + raise KeyError + return i + 1 + +def f(): + i = 0 + while i < 10000: + try: + i = g(i) + 1 + except: + .. + ... + +if __name__ == '__main__': + f() Added: pypy/extradoc/talk/ctpug2010/demo/sourcelog ============================================================================== --- (empty file) +++ pypy/extradoc/talk/ctpug2010/demo/sourcelog Mon Nov 29 20:46:54 2010 @@ -0,0 +1,140 @@ +[757bff8d2abb] {jit-log-opt-loop +# Loop 0 : loop with 62 ops +[p0, p1, p2, p3, i4, p5, p6, p7] +debug_merge_point(' #9 LOAD_FAST', 0) +guard_nonnull_class(p7, ConstClass(W_IntObject), descr=) [p1, p0, p7, p2, p3, p5, p6] +debug_merge_point(' #12 LOAD_CONST', 0) +debug_merge_point(' #15 COMPARE_OP', 0) +i9 = getfield_gc_pure(p7, descr=) +i11 = int_lt(i9, 10000) +guard_true(i11, descr=) [p1, p0, p7, p2, p3] +debug_merge_point(' #18 JUMP_IF_FALSE', 0) +debug_merge_point(' #21 POP_TOP', 0) +debug_merge_point(' #22 LOAD_GLOBAL', 0) +p12 = getfield_gc(p0, descr=) +guard_value(p12, ConstPtr(ptr13), descr=) [p1, p0, p12, p2, p3, p7] +p14 = getfield_gc(p12, descr=) +guard_isnull(p14, descr=) [p1, p0, p14, p12, p2, p3, p7] +p16 = getfield_gc(ConstPtr(ptr15), descr=) +guard_nonnull_class(p16, ConstClass(Function), descr=) [p1, p0, p16, p2, p3, p7] +debug_merge_point(' #25 LOAD_FAST', 0) +debug_merge_point(' #28 CALL_FUNCTION', 0) +i18 = getfield_gc(p0, descr=) +guard_false(i18, descr=) [p1, p0, p16, p2, p3, p7] +p19 = getfield_gc(p16, descr=) +guard_value(p19, ConstPtr(ptr20), descr=) [p1, p0, p19, p16, p2, p3, p7] +p21 = getfield_gc(p16, descr=) +p22 = getfield_gc(p16, descr=) +p24 = call(ConstClass(getexecutioncontext), descr=) +guard_no_exception(, descr=) [p1, p0, p24, p2, p3, p16, p7, p21] +i25 = getfield_gc(p24, descr=) +i27 = getfield_gc(ConstPtr(ptr26), descr=) +i28 = int_gt(i25, i27) +guard_false(i28, descr=) [p1, p0, p24, p2, p3, p16, p7, p21] +i30 = int_add(i25, 1) +p31 = getfield_gc(p24, descr=) +i32 = force_token() +p33 = getfield_gc(p24, descr=) +setfield_gc(p24, i30, descr=) +guard_isnull(p33, descr=) [p1, p0, p24, p33, p2, p3, p16, p7, p31, i32, p21] +i34 = getfield_gc(p24, descr=) +i35 = int_is_zero(i34) +guard_true(i35, descr=) [p1, p0, p24, p2, p3, p16, p7, p31, i32, p21] +debug_merge_point(' #0 LOAD_FAST', 1) +debug_merge_point(' #3 LOAD_CONST', 1) +debug_merge_point(' #6 BINARY_ADD', 1) +i37 = int_add(i9, 1) +debug_merge_point(' #7 RETURN_VALUE', 1) +i38 = int_is_true(i34) +guard_false(i38, descr=) [p1, p0, p24, p2, p3, p16, p7, i37, p31, i32, p21] +debug_merge_point(' #31 LOAD_CONST', 0) +debug_merge_point(' #34 BINARY_ADD', 0) +i40 = int_add(i37, 1) +debug_merge_point(' #35 STORE_FAST', 0) +debug_merge_point(' #38 JUMP_ABSOLUTE', 0) +i42 = getfield_raw(32212256, descr=) +i44 = int_sub(i42, 9) +setfield_raw(32212256, i44, descr=) +setfield_gc(p24, i25, descr=) +setfield_gc(p24, p31, descr=) +i46 = int_lt(i44, 0) +guard_false(i46, descr=) [p1, p0, p2, p3, i40, None, None, None, None] +debug_merge_point(' #9 LOAD_FAST', 0) +p49 = new_with_vtable(ConstClass(W_IntObject)) +setfield_gc(p49, i40, descr=) +jump(p0, p1, p2, p3, 9, ConstPtr(ptr51), ConstPtr(ptr52), p49, descr=) +[757bffbe2de1] jit-log-opt-loop} +[757bffc0423f] {jit-backend-counts +[757bffc04bd5] jit-backend-counts} +[757bfff6cfb7] {jit-log-opt-loop +# Loop 1 : entry bridge with 64 ops +[p0, p1, p2, p3, i4, p5, i6, p7, p8, p9] +debug_merge_point(' #9 LOAD_FAST', 0) +guard_value(i4, 0, descr=) [i4, p1, p0, p2, p3, p5, i6, p7, p8, p9] +guard_nonnull_class(p9, ConstClass(W_IntObject), descr=) [p1, p0, p9, p2, p3, p5, p7, p8] +debug_merge_point(' #12 LOAD_CONST', 0) +guard_value(p2, ConstPtr(ptr12), descr=) [p1, p0, p2, p3, p5, p9, p8] +debug_merge_point(' #15 COMPARE_OP', 0) +i13 = getfield_gc_pure(p9, descr=) +i15 = int_lt(i13, 10000) +guard_true(i15, descr=) [p1, p0, p9, p3, p5] +debug_merge_point(' #18 JUMP_IF_FALSE', 0) +debug_merge_point(' #21 POP_TOP', 0) +debug_merge_point(' #22 LOAD_GLOBAL', 0) +p16 = getfield_gc(p0, descr=) +guard_value(p16, ConstPtr(ptr17), descr=) [p1, p0, p16, p3, p5, p9] +p18 = getfield_gc(p16, descr=) +guard_isnull(p18, descr=) [p1, p0, p18, p16, p3, p5, p9] +p20 = getfield_gc(ConstPtr(ptr19), descr=) +guard_nonnull_class(p20, ConstClass(Function), descr=) [p1, p0, p20, p3, p5, p9] +debug_merge_point(' #25 LOAD_FAST', 0) +debug_merge_point(' #28 CALL_FUNCTION', 0) +i22 = getfield_gc(p0, descr=) +guard_false(i22, descr=) [p1, p0, p20, p3, p5, p9] +p23 = getfield_gc(p20, descr=) +guard_value(p23, ConstPtr(ptr24), descr=) [p1, p0, p23, p20, p3, p5, p9] +p25 = getfield_gc(p20, descr=) +p26 = getfield_gc(p20, descr=) +p28 = call(ConstClass(getexecutioncontext), descr=) +guard_no_exception(, descr=) [p1, p0, p28, p3, p5, p20, p9, p25] +i29 = getfield_gc(p28, descr=) +i31 = getfield_gc(ConstPtr(ptr30), descr=) +i32 = int_gt(i29, i31) +guard_false(i32, descr=) [p1, p0, p28, p3, p5, p20, p9, p25] +i34 = int_add(i29, 1) +p35 = getfield_gc(p28, descr=) +i36 = force_token() +p37 = getfield_gc(p28, descr=) +setfield_gc(p28, i34, descr=) +guard_isnull(p37, descr=) [p1, p0, p28, p37, p3, p5, p20, p9, p35, i36, p25] +i38 = getfield_gc(p28, descr=) +i39 = int_is_zero(i38) +guard_true(i39, descr=) [p1, p0, p28, p3, p5, p20, p9, p35, i36, p25] +debug_merge_point(' #0 LOAD_FAST', 1) +debug_merge_point(' #3 LOAD_CONST', 1) +debug_merge_point(' #6 BINARY_ADD', 1) +i41 = int_add(i13, 1) +debug_merge_point(' #7 RETURN_VALUE', 1) +i42 = int_is_true(i38) +guard_false(i42, descr=) [p1, p0, p28, p3, p5, p20, p9, i41, p35, i36, p25] +debug_merge_point(' #31 LOAD_CONST', 0) +debug_merge_point(' #34 BINARY_ADD', 0) +i44 = int_add(i41, 1) +debug_merge_point(' #35 STORE_FAST', 0) +debug_merge_point(' #38 JUMP_ABSOLUTE', 0) +i46 = getfield_raw(32212256, descr=) +i48 = int_sub(i46, 9) +setfield_raw(32212256, i48, descr=) +setfield_gc(p28, i29, descr=) +setfield_gc(p28, p35, descr=) +i50 = int_lt(i48, 0) +guard_false(i50, descr=) [p1, p0, p3, p5, i44, None, None, None, None] +debug_merge_point(' #9 LOAD_FAST', 0) +p53 = new_with_vtable(ConstClass(W_IntObject)) +setfield_gc(p53, i44, descr=) +jump(p0, p1, p3, p5, 9, ConstPtr(ptr55), ConstPtr(ptr56), p53, descr=) +[757bfffa4e27] jit-log-opt-loop} +[757c00042ed9] {jit-backend-counts +0:2995 +1:2 +[757c00044cd5] jit-backend-counts} Added: pypy/extradoc/talk/ctpug2010/talk.pdf ============================================================================== Binary file. No diff available. Copied: pypy/extradoc/talk/ctpug2010/title.latex (from r79573, pypy/extradoc/talk/ep2010/talk/title.latex) ============================================================================== --- pypy/extradoc/talk/ep2010/talk/title.latex (original) +++ pypy/extradoc/talk/ctpug2010/title.latex Mon Nov 29 20:46:54 2010 @@ -1,8 +1,5 @@ \begin{titlepage} \begin{figure}[h] -\scalebox{0.8}{\includegraphics[width=80px]{../../img/py-web.png}} -\qquad -\qquad -\scalebox{0.8}{\includegraphics[width=64px,height=64px]{merlinux-logo.jpg}} +\scalebox{0.8}{\includegraphics[width=80px]{../img/py-web.png}} \end{figure} \end{titlepage} From hakanardo at codespeak.net Mon Nov 29 22:04:57 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Mon, 29 Nov 2010 22:04:57 +0100 (CET) Subject: [pypy-svn] r79657 - in pypy/branch/jit-unroll-loops/pypy/jit/metainterp: . optimizeopt test Message-ID: <20101129210457.9C0E9282BE0@codespeak.net> Author: hakanardo Date: Mon Nov 29 22:04:55 2010 New Revision: 79657 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeutil.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_greenfield.py Log: Allow for multiple short preambles and for bridges to try them one by one and not issue a retrace untill all of them fail Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/compile.py Mon Nov 29 22:04:55 2010 @@ -112,8 +112,8 @@ metainterp_sd.logger_ops.log_loop(loop.inputargs, loop.operations, n, type) short = loop.token.short_preamble if short: - metainterp_sd.logger_ops.log_short_preamble(short.inputargs, - short.operations) + metainterp_sd.logger_ops.log_short_preamble(short[-1].inputargs, + short[-1].operations) if not we_are_translated(): show_loop(metainterp_sd, loop) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/optimizer.py Mon Nov 29 22:04:55 2010 @@ -170,6 +170,12 @@ def emit_operation(self, op): self.next_optimization.propagate_forward(op) + def test_emittable(self, op): + return self.is_emittable(op) + + def is_emittable(self, op): + return self.next_optimization.test_emittable(op) + # FIXME: Move some of these here? def getvalue(self, box): return self.optimizer.getvalue(box) @@ -396,7 +402,9 @@ self.optimize_default(op) #print '\n'.join([str(o) for o in self.newoperations]) + '\n---\n' - + def test_emittable(self, op): + return True + def emit_operation(self, op): ###self.heap_op_optimizer.emitting_operation(op) self._emit_operation(op) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/rewrite.py Mon Nov 29 22:04:55 2010 @@ -25,6 +25,18 @@ break else: self.emit_operation(op) + + def test_emittable(self, op): + opnum = op.getopnum() + for value, func in optimize_guards: + if opnum == value: + try: + func(self, op, dryrun=True) + return self.is_emittable(op) + except InvalidLoop: + return False + return self.is_emittable(op) + def try_boolinvers(self, op, targs): oldop = self.optimizer.pure_operations.get(targs, None) @@ -145,7 +157,7 @@ self.emit_operation(ResOperation(rop.CALL, args, op.result, op.getdescr())) - def optimize_guard(self, op, constbox, emit_operation=True): + def optimize_guard(self, op, constbox, emit_operation=True, dryrun=False): value = self.getvalue(op.getarg(0)) if value.is_constant(): box = value.box @@ -153,33 +165,36 @@ if not box.same_constant(constbox): raise InvalidLoop return + if dryrun: return if emit_operation: self.emit_operation(op) value.make_constant(constbox) self.optimizer.turned_constant(value) - def optimize_GUARD_ISNULL(self, op): + def optimize_GUARD_ISNULL(self, op, dryrun=False): value = self.getvalue(op.getarg(0)) if value.is_null(): return elif value.is_nonnull(): raise InvalidLoop + if dryrun: return self.emit_operation(op) value.make_constant(self.optimizer.cpu.ts.CONST_NULL) - def optimize_GUARD_NONNULL(self, op): + def optimize_GUARD_NONNULL(self, op, dryrun=False): value = self.getvalue(op.getarg(0)) if value.is_nonnull(): return elif value.is_null(): raise InvalidLoop + if dryrun: return self.emit_operation(op) value.make_nonnull(len(self.optimizer.newoperations) - 1) - def optimize_GUARD_VALUE(self, op): + def optimize_GUARD_VALUE(self, op, dryrun=False): value = self.getvalue(op.getarg(0)) emit_operation = True - if value.last_guard_index != -1: + if not dryrun and value.last_guard_index != -1: # there already has been a guard_nonnull or guard_class or # guard_nonnull_class on this value, which is rather silly. # replace the original guard with a guard_value @@ -197,15 +212,15 @@ emit_operation = False constbox = op.getarg(1) assert isinstance(constbox, Const) - self.optimize_guard(op, constbox, emit_operation) + self.optimize_guard(op, constbox, emit_operation, dryrun) - def optimize_GUARD_TRUE(self, op): - self.optimize_guard(op, CONST_1) + def optimize_GUARD_TRUE(self, op, dryrun=False): + self.optimize_guard(op, CONST_1, dryrun=dryrun) - def optimize_GUARD_FALSE(self, op): - self.optimize_guard(op, CONST_0) + def optimize_GUARD_FALSE(self, op, dryrun=False): + self.optimize_guard(op, CONST_0, dryrun=dryrun) - def optimize_GUARD_CLASS(self, op): + def optimize_GUARD_CLASS(self, op, dryrun=False): value = self.getvalue(op.getarg(0)) expectedclassbox = op.getarg(1) assert isinstance(expectedclassbox, Const) @@ -214,6 +229,7 @@ if realclassbox.same_constant(expectedclassbox): return raise InvalidLoop + if dryrun: return emit_operation = True if value.last_guard_index != -1: # there already has been a guard_nonnull or guard_class or @@ -239,7 +255,8 @@ last_guard_index = value.last_guard_index value.make_constant_class(expectedclassbox, last_guard_index) - def optimize_GUARD_NO_EXCEPTION(self, op): + def optimize_GUARD_NO_EXCEPTION(self, op, dryrun=False): + if dryrun: return if not self.optimizer.exception_might_have_happened: return self.emit_operation(op) @@ -359,4 +376,4 @@ return False optimize_ops = _findall(OptRewrite, 'optimize_') - +optimize_guards = _findall(OptRewrite, 'optimize_', 'GUARD') Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Mon Nov 29 22:04:55 2010 @@ -72,7 +72,10 @@ short_loop.operations = short assert isinstance(loop.preamble.token, LoopToken) - loop.preamble.token.short_preamble = short_loop + if loop.preamble.token.short_preamble: + loop.preamble.token.short_preamble.append(short_loop) + else: + loop.preamble.token.short_preamble = [short_loop] @@ -332,15 +335,24 @@ if op.getopnum() == rop.JUMP: descr = op.getdescr() assert isinstance(descr, LoopToken) + # FIXME: Use a tree, similar to the tree formed by the full + # preamble and it's bridges, instead of a list to save time and + # memory short = descr.short_preamble if short: - self.inline(short.operations, short.inputargs, op.getarglist()) - return + for sh in short: + if self.inline(sh.operations, sh.inputargs, + op.getarglist(), dryrun=True): + self.inline(sh.operations, sh.inputargs, + op.getarglist()) + return + + raise RetraceLoop self.emit_operation(op) - def inline(self, loop_operations, loop_args, jump_args): + def inline(self, loop_operations, loop_args, jump_args, dryrun=False): self.argmap = argmap = {} assert len(loop_args) == len(jump_args) for i in range(len(loop_args)): @@ -360,11 +372,13 @@ newop.result = newop.result.clonebox() argmap[old_result] = newop.result - try: + if not dryrun: self.emit_operation(newop) - except InvalidLoop: - raise RetraceLoop - + else: + if not self.is_emittable(newop): + return False + + return True def inline_arg(self, arg): if isinstance(arg, Const): Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeutil.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeutil.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeutil.py Mon Nov 29 22:04:55 2010 @@ -19,7 +19,7 @@ # ____________________________________________________________ # Misc. utilities -def _findall(Class, name_prefix): +def _findall(Class, name_prefix, op_prefix=None): result = [] for name in dir(Class): if name.startswith(name_prefix): @@ -27,6 +27,8 @@ if opname.isupper(): assert hasattr(resoperation.rop, opname) for value, name in resoperation.opname.items(): + if op_prefix and not name.startswith(op_prefix): + continue if hasattr(Class, name_prefix + name): result.append((value, getattr(Class, name_prefix + name))) return unrolling_iterable(result) Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/pyjitpl.py Mon Nov 29 22:04:55 2010 @@ -1854,6 +1854,9 @@ if loop_token is None: return + if loop_token.short_preamble: + old_loop_tokens[0].short_preamble.extend(loop_token.short_preamble) + self.history.inputargs = original_inputargs self.history.operations = self.history.operations[:start] live_arg_boxes = bridge_arg_boxes Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_basic.py Mon Nov 29 22:04:55 2010 @@ -1840,6 +1840,45 @@ 'int_add': 1, 'int_mul': 1, 'int_sub': 2, 'int_gt': 2, 'jump': 2}) + def test_multiple_specialied_zigzag(self): + myjitdriver = JitDriver(greens = [], reds = ['y', 'x', 'res']) + class Base: + def __init__(self, val): + self.val = val + class A(Base): + def binop(self, other): + return A(self.val + other.val) + def switch(self): + return B(self.val) + class B(Base): + def binop(self, other): + return B(self.val * other.val) + def switch(self): + return A(self.val) + def f(x, y): + res = x + while y > 0: + myjitdriver.can_enter_jit(y=y, x=x, res=res) + myjitdriver.jit_merge_point(y=y, x=x, res=res) + if y % 4 == 0: + res = res.switch() + res = res.binop(x) + y -= 1 + return res + def g(x, y): + a1 = f(A(x), y) + a2 = f(A(x), y) + b1 = f(B(x), y) + b2 = f(B(x), y) + assert a1.val == a2.val + assert b1.val == b2.val + return a1.val + b1.val + res = self.meta_interp(g, [3, 23]) + assert res == 7068153 + self.check_loop_count(7) + self.check_loops(guard_true=4, guard_class=0, int_add=2, int_mul=2, + guard_false=2) + def test_current_trace_length(self): myjitdriver = JitDriver(greens = ['g'], reds = ['x']) @dont_look_inside Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_greenfield.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_greenfield.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/test/test_greenfield.py Mon Nov 29 22:04:55 2010 @@ -49,7 +49,7 @@ # res = self.meta_interp(g, [7]) assert res == -22 - self.check_loop_count(4) + self.check_loop_count(6) self.check_loops(guard_value=0) From wlav at codespeak.net Tue Nov 30 03:53:27 2010 From: wlav at codespeak.net (wlav at codespeak.net) Date: Tue, 30 Nov 2010 03:53:27 +0100 (CET) Subject: [pypy-svn] r79658 - pypy/branch/reflex-support/pypy/module/cppyy/test Message-ID: <20101130025327.A048436C231@codespeak.net> Author: wlav Date: Tue Nov 30 03:53:19 2010 New Revision: 79658 Modified: pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py Log: fix typo in comment Modified: pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/test/test_cppyy.py Tue Nov 30 03:53:19 2010 @@ -153,7 +153,7 @@ assert t.invoke("getCount") == 0 def testReturningOfAnObjectByPointer(self): - """Test returing of an instance as an argument.""" + """Test returning of an instance as an argument.""" t = self.example01 From wlav at codespeak.net Tue Nov 30 05:31:02 2010 From: wlav at codespeak.net (wlav at codespeak.net) Date: Tue, 30 Nov 2010 05:31:02 +0100 (CET) Subject: [pypy-svn] r79659 - pypy/branch/reflex-support/pypy/module/cppyy Message-ID: <20101130043102.4C2D1282BDD@codespeak.net> Author: wlav Date: Tue Nov 30 05:30:58 2010 New Revision: 79659 Modified: pypy/branch/reflex-support/pypy/module/cppyy/executor.py pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py pypy/branch/reflex-support/pypy/module/cppyy/pythonify.py Log: fix segfault when using pythonify Modified: pypy/branch/reflex-support/pypy/module/cppyy/executor.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/executor.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/executor.py Tue Nov 30 05:30:58 2010 @@ -10,8 +10,8 @@ _immutable_ = True libffitype = libffi.types.NULL - def __init__(self, space, cpptype): - pass + def __init__(self, space, name, cpptype): + self.name = name def execute(self, space, func, cppthis, num_args, args): raise NotImplementedError @@ -93,7 +93,8 @@ class InstancePtrExecutor(FunctionExecutor): _immutable_ = True - def __init__(self, space, cpptype): + def __init__(self, space, name, cpptype): + FunctionExecutor.__init__(self, space, name, cpptype) self.cpptype = cpptype def execute(self, space, func, cppthis, num_args, args): @@ -107,17 +108,18 @@ from pypy.module.cppyy import interp_cppyy try: - return _executors[name](space, None) + return _executors[name](space, "", None) except KeyError: pass compound = helper.compound(name) - cpptype = interp_cppyy.type_byname(space, helper.clean_type(name)) + clean_name = helper.clean_type(name) + cpptype = interp_cppyy.type_byname(space, clean_name) if compound == "*": - return InstancePtrExecutor(space, cpptype) + return InstancePtrExecutor(space, cpptype.name, cpptype) # currently used until proper lazy instantiation available in interp_cppyy - return FunctionExecutor(space, None) + return FunctionExecutor(space, "", None) # raise TypeError("no clue what %s is" % name) Modified: pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/interp_cppyy.py Tue Nov 30 05:30:58 2010 @@ -201,7 +201,7 @@ class W_CPPOverload(Wrappable): _immutable_ = True - _immutable_fields_ = ["functions[*]"] + _immutable_fields_ = ["func_name", "functions[*]"] def __init__(self, space, func_name, functions): self.space = space self.func_name = func_name @@ -211,10 +211,7 @@ return self.space.wrap(isinstance(self.functions[0], CPPFunction)) def get_returntype(self): - try: - return self.space.wrap(self.functions[0].executor.cpptype.name) - except AttributeError: - return None + return self.space.wrap(self.functions[0].executor.name) @jit.unroll_safe def call(self, cppthis, args_w): Modified: pypy/branch/reflex-support/pypy/module/cppyy/pythonify.py ============================================================================== --- pypy/branch/reflex-support/pypy/module/cppyy/pythonify.py (original) +++ pypy/branch/reflex-support/pypy/module/cppyy/pythonify.py Tue Nov 30 05:30:58 2010 @@ -20,10 +20,10 @@ return bound_obj def make_static_function(cpptype, name, rettype): - if rettype is None: + if not rettype: # return builtin type def method(*args): return cpptype.invoke(name, *args) - else: + else: # return instance cppclass = get_cppclass(rettype) def method(*args): return bind_object(cpptype.invoke(name, *args), cppclass) @@ -31,7 +31,7 @@ return staticmethod(method) def make_method(name, rettype): - if rettype is None: # return builtin type + if not rettype: # return builtin type def method(self, *args): return self._cppinstance.invoke(name, *args) else: # return instance From antocuni at codespeak.net Tue Nov 30 08:44:07 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 30 Nov 2010 08:44:07 +0100 (CET) Subject: [pypy-svn] r79660 - pypy/trunk/pypy/translator/platform Message-ID: <20101130074407.0D816282BE0@codespeak.net> Author: antocuni Date: Tue Nov 30 08:44:04 2010 New Revision: 79660 Added: pypy/trunk/pypy/translator/platform/freebsd.py (contents, props changed) Removed: pypy/trunk/pypy/translator/platform/freebsd7.py Log: apply last part of the patch issue551, I forgot to svn add the file Added: pypy/trunk/pypy/translator/platform/freebsd.py ============================================================================== --- (empty file) +++ pypy/trunk/pypy/translator/platform/freebsd.py Tue Nov 30 08:44:04 2010 @@ -0,0 +1,52 @@ + +import py, os +from pypy.translator.platform import posix + +def get_env(key, default): + if key in os.environ: + return os.environ[key] + else: + return default + +def get_env_vector(key, default): + string = get_env(key, default) + print key, string, default + # XXX: handle quotes + return string.split() + +class Freebsd(posix.BasePosix): + name = "freebsd" + + link_flags = get_env_vector("LDFLAGS", '-pthread') + cflags = get_env_vector("CFLAGS", "-O3 -pthread -fomit-frame-pointer") + standalone_only = [] + shared_only = [] + so_ext = 'so' + make_cmd = 'gmake' + + def __init__(self, cc=None): + if cc is None: + cc = get_env("CC", "gcc") + super(Freebsd, self).__init__(cc) + + def _args_for_shared(self, args): + return ['-shared'] + args + + def _preprocess_include_dirs(self, include_dirs): + res_incl_dirs = list(include_dirs) + res_incl_dirs.append(os.path.join(get_env("LOCALBASE", "/usr/local"), "include")) + return res_incl_dirs + + def _preprocess_library_dirs(self, library_dirs): + res_lib_dirs = list(library_dirs) + res_lib_dirs.append(os.path.join(get_env("LOCALBASE", "/usr/local"), "lib")) + return res_lib_dirs + + def include_dirs_for_libffi(self): + return [os.path.join(get_env("LOCALBASE", "/usr/local"), "include")] + + def library_dirs_for_libffi(self): + return [os.path.join(get_env("LOCALBASE", "/usr/local"), "lib")] + +class Freebsd_64(Freebsd): + shared_only = ('-fPIC',) From arigo at codespeak.net Tue Nov 30 10:57:18 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 10:57:18 +0100 (CET) Subject: [pypy-svn] r79661 - in pypy/trunk/pypy/translator/goal: . test2 Message-ID: <20101130095718.CF0E4282BD4@codespeak.net> Author: arigo Date: Tue Nov 30 10:57:15 2010 New Revision: 79661 Modified: pypy/trunk/pypy/translator/goal/app_main.py pypy/trunk/pypy/translator/goal/test2/test_app_main.py Log: Implement -E, and test it. (The branch/fast-forward added -E, but as far as I can tell it completely ignores it) Modified: pypy/trunk/pypy/translator/goal/app_main.py ============================================================================== --- pypy/trunk/pypy/translator/goal/app_main.py (original) +++ pypy/trunk/pypy/translator/goal/app_main.py Tue Nov 30 10:57:15 2010 @@ -11,6 +11,7 @@ -h, --help show this help message and exit -m library module to be run as a script (terminates option list) -W arg warning control (arg is action:message:category:module:lineno) + -E ignore environment variables (such as PYTHONPATH) --version print the PyPy version --info print translation information about this PyPy executable """ @@ -199,7 +200,7 @@ break # found! return newpath -def setup_initial_paths(executable, nanos): +def setup_initial_paths(executable, nanos, readenv=True, **extra): # a substituted os if we are translated global os os = nanos @@ -220,7 +221,7 @@ sys.executable = os.path.abspath(executable) newpath = get_library_path(executable) - path = os.getenv('PYTHONPATH') + path = readenv and os.getenv('PYTHONPATH') if path: newpath = path.split(os.pathsep) + newpath # remove duplicates @@ -230,7 +231,6 @@ if dir not in _seen: sys.path.append(dir) _seen[dir] = True - return executable def parse_command_line(argv): @@ -242,6 +242,7 @@ run_stdin = False warnoptions = [] unbuffered = False + readenv = True while i < len(argv): arg = argv[i] if not arg.startswith('-'): @@ -253,6 +254,8 @@ argv[i] = '-c' run_command = True break + elif arg == '-E': + readenv = False elif arg == '-u': unbuffered = True elif arg == '-O' or arg == '-OO': @@ -305,6 +308,7 @@ run_stdin, warnoptions, unbuffered, + readenv, cmd=None, **ignored): # with PyPy in top of CPython we can only have around 100 @@ -355,7 +359,7 @@ # * PYTHONINSPECT is set and stdin is a tty. # return (go_interactive or - (os.getenv('PYTHONINSPECT') and sys.stdin.isatty())) + (readenv and os.getenv('PYTHONINSPECT') and sys.stdin.isatty())) success = True @@ -388,7 +392,7 @@ # If stdin is a tty or if "-i" is specified, we print # a banner and run $PYTHONSTARTUP. print_banner() - python_startup = os.getenv('PYTHONSTARTUP') + python_startup = readenv and os.getenv('PYTHONSTARTUP') if python_startup: try: f = open(python_startup) @@ -451,7 +455,6 @@ '"license" for more information.') def entry_point(executable, argv, nanos): - executable = setup_initial_paths(executable, nanos) try: cmdline = parse_command_line(argv) except CommandLineError, e: @@ -459,8 +462,8 @@ return 2 if cmdline is None: return 0 - else: - return run_command_line(**cmdline) + setup_initial_paths(executable, nanos, **cmdline) + return run_command_line(**cmdline) if __name__ == '__main__': Modified: pypy/trunk/pypy/translator/goal/test2/test_app_main.py ============================================================================== --- pypy/trunk/pypy/translator/goal/test2/test_app_main.py (original) +++ pypy/trunk/pypy/translator/goal/test2/test_app_main.py Tue Nov 30 10:57:15 2010 @@ -217,6 +217,38 @@ finally: os.environ['PYTHONSTARTUP'] = old + def test_ignore_python_startup(self): + old = os.environ.get('PYTHONSTARTUP', '') + try: + os.environ['PYTHONSTARTUP'] = crashing_demo_script + child = self.spawn(['-E']) + child.expect(re.escape(banner)) + index = child.expect(['Traceback', '>>> ']) + assert index == 1 # no traceback + finally: + os.environ['PYTHONSTARTUP'] = old + + def test_ignore_python_inspect(self): + os.environ['PYTHONINSPECT_'] = '1' + try: + child = self.spawn(['-E', '-c', 'pass']) + from pexpect import EOF + index = child.expect(['>>> ', EOF]) + assert index == 1 # no prompt + finally: + del os.environ['PYTHONINSPECT_'] + + def test_ignore_python_path(self): + old = os.environ.get('PYTHONPATH', '') + try: + os.environ['PYTHONPATH'] = 'foobarbaz' + child = self.spawn(['-E', '-c', 'import sys; print sys.path']) + from pexpect import EOF + index = child.expect(['foobarbaz', EOF]) + assert index == 1 # no foobarbaz + finally: + os.environ['PYTHONPATH'] = old + def test_unbuffered(self): line = 'import os,sys;sys.stdout.write(str(789));os.read(0,1)' child = self.spawn(['-u', '-c', line]) From arigo at codespeak.net Tue Nov 30 12:39:23 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 12:39:23 +0100 (CET) Subject: [pypy-svn] r79663 - pypy/branch/fast-forward/lib-python Message-ID: <20101130113923.91265282BD4@codespeak.net> Author: arigo Date: Tue Nov 30 12:39:21 2010 New Revision: 79663 Modified: pypy/branch/fast-forward/lib-python/TODO Log: Remove DONE items. Modified: pypy/branch/fast-forward/lib-python/TODO ============================================================================== --- pypy/branch/fast-forward/lib-python/TODO (original) +++ pypy/branch/fast-forward/lib-python/TODO Tue Nov 30 12:39:21 2010 @@ -34,8 +34,6 @@ - missing posix.(confstr|pathconf|fpathconf)(_names)? -- some set() methods now accept a variable number of arguments DONE - - remove code duplication: bit_length() and _count_bits() in rlib/rbigint.py, objspace/std/longobject.py and objspace/std/longtype.py. @@ -48,8 +46,6 @@ - missing builtin: memoryview DONE (but not very much tested) -- signal.set_wakeup_fd() DONE - - Dictionary views: dict.viewkeys(), dict.viewvalues(), dict.viewitems() - add 'unicode' in ObjSpace.MethodTable + probably a default implementation that @@ -68,10 +64,6 @@ - Enable -3 option to run test_py3kwarn. -- Change the way the unicodedata module stores its database: in - unicodedb_5_2_0.py, the "_composition" map contains values > 2**32 which - causes the translation to fail. DONE - - "Shorter float representation": copy dtoa.c from CPython and use it to format/parse floats. Enable this with a translation option. From arigo at codespeak.net Tue Nov 30 13:01:34 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 13:01:34 +0100 (CET) Subject: [pypy-svn] r79664 - in pypy/trunk/pypy: interpreter module/__pypy__ module/__pypy__/test Message-ID: <20101130120134.C2422282BDC@codespeak.net> Author: arigo Date: Tue Nov 30 13:01:29 2010 New Revision: 79664 Modified: pypy/trunk/pypy/interpreter/function.py pypy/trunk/pypy/module/__pypy__/__init__.py pypy/trunk/pypy/module/__pypy__/interp_magic.py pypy/trunk/pypy/module/__pypy__/test/test_special.py Log: Remove the app-level constructor 'builtin_function_type(f)', and replace it with a custom helper in the __pypy__ module. Modified: pypy/trunk/pypy/interpreter/function.py ============================================================================== --- pypy/trunk/pypy/interpreter/function.py (original) +++ pypy/trunk/pypy/interpreter/function.py Tue Nov 30 13:01:29 2010 @@ -612,11 +612,9 @@ self.w_func_dict = func.w_func_dict self.w_module = func.w_module - def descr_builtinfunction__new__(space, w_subtype, w_func): - func = space.interp_w(Function, w_func) - bltin = space.allocate_instance(BuiltinFunction, w_subtype) - BuiltinFunction.__init__(bltin, func) - return space.wrap(bltin) + def descr_builtinfunction__new__(space, w_subtype): + raise OperationError(space.w_TypeError, + space.wrap("cannot create 'builtin_function' instances")) def descr_function_repr(self): return self.space.wrap('' % (self.name,)) Modified: pypy/trunk/pypy/module/__pypy__/__init__.py ============================================================================== --- pypy/trunk/pypy/module/__pypy__/__init__.py (original) +++ pypy/trunk/pypy/module/__pypy__/__init__.py Tue Nov 30 13:01:29 2010 @@ -15,6 +15,7 @@ 'debug_print' : 'interp_debug.debug_print', 'debug_stop' : 'interp_debug.debug_stop', 'debug_print_once' : 'interp_debug.debug_print_once', + 'builtinify' : 'interp_magic.builtinify', } def setup_after_space_initialization(self): Modified: pypy/trunk/pypy/module/__pypy__/interp_magic.py ============================================================================== --- pypy/trunk/pypy/module/__pypy__/interp_magic.py (original) +++ pypy/trunk/pypy/module/__pypy__/interp_magic.py Tue Nov 30 13:01:29 2010 @@ -51,3 +51,9 @@ return space.newtuple([space.newint(cache.hits.get(name, 0)), space.newint(cache.misses.get(name, 0))]) mapdict_cache_counter.unwrap_spec = [ObjSpace, str] + +def builtinify(space, w_func): + from pypy.interpreter.function import Function, BuiltinFunction + func = space.interp_w(Function, w_func) + bltn = BuiltinFunction(func) + return space.wrap(bltn) Modified: pypy/trunk/pypy/module/__pypy__/test/test_special.py ============================================================================== --- pypy/trunk/pypy/module/__pypy__/test/test_special.py (original) +++ pypy/trunk/pypy/module/__pypy__/test/test_special.py Tue Nov 30 13:01:29 2010 @@ -21,3 +21,18 @@ def test_cpumodel(self): import __pypy__ assert hasattr(__pypy__, 'cpumodel') + + def test_builtinify(self): + import __pypy__ + class A(object): + a = lambda *args: args + b = __pypy__.builtinify(a) + my = A() + assert my.a() == (my,) + assert my.b() == () + assert A.a(my) == (my,) + assert A.b(my) == (my,) + assert A.a.im_func(my) == (my,) + assert not hasattr(A.b, 'im_func') + assert A.a is not A.__dict__['a'] + assert A.b is A.__dict__['b'] From arigo at codespeak.net Tue Nov 30 13:03:58 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 13:03:58 +0100 (CET) Subject: [pypy-svn] r79665 - pypy/trunk/lib_pypy Message-ID: <20101130120358.478F436C222@codespeak.net> Author: arigo Date: Tue Nov 30 13:03:57 2010 New Revision: 79665 Modified: pypy/trunk/lib_pypy/_hashlib.py Log: Add 'builtinify' as a decorator. Modified: pypy/trunk/lib_pypy/_hashlib.py ============================================================================== --- pypy/trunk/lib_pypy/_hashlib.py (original) +++ pypy/trunk/lib_pypy/_hashlib.py Tue Nov 30 13:03:57 2010 @@ -148,21 +148,29 @@ return hash(ctx, name) # shortcut functions +from __pypy__ import builtinify + + at builtinify def openssl_md5(string=''): return new('md5', string) + at builtinify def openssl_sha1(string=''): return new('sha1', string) + at builtinify def openssl_sha224(string=''): return new('sha224', string) + at builtinify def openssl_sha256(string=''): return new('sha256', string) + at builtinify def openssl_sha384(string=''): return new('sha384', string) + at builtinify def openssl_sha512(string=''): return new('sha512', string) From antocuni at codespeak.net Tue Nov 30 13:27:23 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 30 Nov 2010 13:27:23 +0100 (CET) Subject: [pypy-svn] r79666 - pypy/extradoc/planning/hg-migration Message-ID: <20101130122723.77E205080B@codespeak.net> Author: antocuni Date: Tue Nov 30 13:27:19 2010 New Revision: 79666 Added: pypy/extradoc/planning/hg-migration/merge-howto.txt (contents, props changed) Log: instructions to merge existing svn branches once we migrate to hg Added: pypy/extradoc/planning/hg-migration/merge-howto.txt ============================================================================== --- (empty file) +++ pypy/extradoc/planning/hg-migration/merge-howto.txt Tue Nov 30 13:27:19 2010 @@ -0,0 +1,69 @@ +.. -*- mode: rst -*- + +How to merge existing SVN branches in mercurial +================================================ + +This document applies to branches that: + + 1. already existed in svn before the mercurial migration + + 2. contains one or more "merge from trunk" done in svn + +If the branch does not contain any "merge from trunk", the merging should be +straightforward (i.e., you just get "real" conflicts). + +The problem with "merge from trunk" is that mercurial does not know that we +have already applied a set of changes in our branch, so it tries to redo them +and we get false conflicts. + +To solve, we need to explicitly tell mercurial that trunk has already been +merged into our branch up to the right revision: + + 1. first of all, we need to find the SVN revision which contains the last + merge from trunk: when doing the actual merge, mercurial will apply only + the changes that happened on trunk from this point on. Let's call it + ``SR`` (Svn Revision). + + 2. we need to find the mercurial node that corresponds to ``SR``: see the + next section for how to do it. Let's call it ``HR`` (Hg Revision) + + 3. switch to the branch in mercurial:: + $ hg up -r my-branch + + 4. perform a "dummy merge": this just tells mercurial that all the changes + up to revision ``HR`` has been already merged into the branch:: + $ hg --config ui.merge=internal:local merge HR + + 5. update to ``default`` (i.e., the equivalent of svn trunk) and merge the branch:: + $ hg up -r default + $ hg merge my-branch + $ hg ci -m 'merged my-branch' + +How to find the ``HR`` corresponding to ``SR`` +---------------------------------------------- + +If we want to do it manually, we can use this command:: + + $ hg log --template '{node|short} {extras}\n' | less + +This will print a line for each changeset, showing the mercurial hash and a +string which among other things contains the SVN revision. We want to pick the +changeset with the highest svn revision number which is less than the target +``SR``. + +Alternatively, we can use the svnup_ mercurial extension. We need to put these +lines into ``~/.hgrc``:: + + [extensions] + svnup = /path/to/svnup.py + +Then, we can just use the ``svnfind`` command to find the changeset we are looking for:: + + $ hg svnfind -r 79389 + changeset: 39199:f96e54f36e15 + user: Maciej Fijalkowski + date: Tue Nov 23 07:02:43 2010 +0000 + summary: Comment out non-jit builds to not overload the server + + +.. http://codespeak.net/svn/user/antocuni/hg-conversion/svnup.py From antocuni at codespeak.net Tue Nov 30 13:32:29 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 30 Nov 2010 13:32:29 +0100 (CET) Subject: [pypy-svn] r79667 - pypy/extradoc/planning/hg-migration Message-ID: <20101130123229.A9DB0282BDC@codespeak.net> Author: antocuni Date: Tue Nov 30 13:32:28 2010 New Revision: 79667 Modified: pypy/extradoc/planning/hg-migration/merge-howto.txt Log: clarify a bit Modified: pypy/extradoc/planning/hg-migration/merge-howto.txt ============================================================================== --- pypy/extradoc/planning/hg-migration/merge-howto.txt (original) +++ pypy/extradoc/planning/hg-migration/merge-howto.txt Tue Nov 30 13:32:28 2010 @@ -14,7 +14,8 @@ The problem with "merge from trunk" is that mercurial does not know that we have already applied a set of changes in our branch, so it tries to redo them -and we get false conflicts. +(plus all the other changes that were done on trunk) and we get false +conflicts. To solve, we need to explicitly tell mercurial that trunk has already been merged into our branch up to the right revision: From arigo at codespeak.net Tue Nov 30 14:09:13 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 14:09:13 +0100 (CET) Subject: [pypy-svn] r79668 - in pypy/trunk/lib_pypy: . pypy_test Message-ID: <20101130130913.C14BB5080B@codespeak.net> Author: arigo Date: Tue Nov 30 14:09:09 2010 New Revision: 79668 Added: pypy/trunk/lib_pypy/pypy_test/hack___pypy__.py Modified: pypy/trunk/lib_pypy/_hashlib.py pypy/trunk/lib_pypy/_locale.py pypy/trunk/lib_pypy/_marshal.py pypy/trunk/lib_pypy/_minimal_curses.py pypy/trunk/lib_pypy/cPickle.py pypy/trunk/lib_pypy/cmath.py pypy/trunk/lib_pypy/grp.py pypy/trunk/lib_pypy/itertools.py pypy/trunk/lib_pypy/msvcrt.py pypy/trunk/lib_pypy/pwd.py pypy/trunk/lib_pypy/pyexpat.py pypy/trunk/lib_pypy/pypy_test/test_hashlib.py pypy/trunk/lib_pypy/pypy_test/test_structseq.py pypy/trunk/lib_pypy/readline.py pypy/trunk/lib_pypy/resource.py pypy/trunk/lib_pypy/syslog.py Log: A first round of adding @__pypy__.builtinify a bit everywhere... :-( Modified: pypy/trunk/lib_pypy/_hashlib.py ============================================================================== --- pypy/trunk/lib_pypy/_hashlib.py (original) +++ pypy/trunk/lib_pypy/_hashlib.py Tue Nov 30 14:09:09 2010 @@ -148,29 +148,29 @@ return hash(ctx, name) # shortcut functions -from __pypy__ import builtinify +import __pypy__ - at builtinify + at __pypy__.builtinify def openssl_md5(string=''): return new('md5', string) - at builtinify + at __pypy__.builtinify def openssl_sha1(string=''): return new('sha1', string) - at builtinify + at __pypy__.builtinify def openssl_sha224(string=''): return new('sha224', string) - at builtinify + at __pypy__.builtinify def openssl_sha256(string=''): return new('sha256', string) - at builtinify + at __pypy__.builtinify def openssl_sha384(string=''): return new('sha384', string) - at builtinify + at __pypy__.builtinify def openssl_sha512(string=''): return new('sha512', string) Modified: pypy/trunk/lib_pypy/_locale.py ============================================================================== --- pypy/trunk/lib_pypy/_locale.py (original) +++ pypy/trunk/lib_pypy/_locale.py Tue Nov 30 14:09:09 2010 @@ -11,6 +11,8 @@ # load the platform-specific cache made by running locale.ctc.py from ctypes_config_cache._locale_cache import * +import __pypy__ + # Ubuntu Gusty i386 structure class lconv(Structure): @@ -158,6 +160,7 @@ ul = ''.join(ul) string.letters = ul + at __pypy__.builtinify def setlocale(category, locale=None): "(integer,string=None) -> string. Activates/queries locale processing." if locale: @@ -182,6 +185,7 @@ groups.append(0) return groups + at __pypy__.builtinify def localeconv(): "() -> dict. Returns numeric and monetary locale-specific parameters." @@ -215,6 +219,7 @@ } return result + at __pypy__.builtinify def strcoll(s1, s2): "string,string -> int. Compares two strings according to the locale." @@ -233,6 +238,7 @@ # Collate the strings. return _wcscoll(s1, s2) + at __pypy__.builtinify def strxfrm(s): "string -> string. Returns a string that behaves for cmp locale-aware." @@ -246,6 +252,7 @@ _strxfrm(buf, s, n2) return buf.value + at __pypy__.builtinify def getdefaultlocale(): # TODO: Port code from CPython for Windows and Mac OS raise NotImplementedError() @@ -267,26 +274,31 @@ raise ValueError("unsupported langinfo constant") if HAS_LIBINTL: + @__pypy__.builtinify def gettext(msg): """gettext(msg) -> string Return translation of msg.""" return _gettext(msg) + @__pypy__.builtinify def dgettext(domain, msg): """dgettext(domain, msg) -> string Return translation of msg in domain.""" return _dgettext(domain, msg) + @__pypy__.builtinify def dcgettext(domain, msg, category): """dcgettext(domain, msg, category) -> string Return translation of msg in domain and category.""" return _dcgettext(domain, msg, category) + @__pypy__.builtinify def textdomain(domain): """textdomain(domain) -> string Set the C library's textdomain to domain, returning the new domain.""" return _textdomain(domain) + @__pypy__.builtinify def bindtextdomain(domain, dir): """bindtextdomain(domain, dir) -> string Bind the C library's domain to dir.""" @@ -297,6 +309,7 @@ return dirname if HAS_BIND_TEXTDOMAIN_CODESET: + @__pypy__.builtinify def bind_textdomain_codeset(domain, codeset): """bind_textdomain_codeset(domain, codeset) -> string Bind the C library's domain to codeset.""" Modified: pypy/trunk/lib_pypy/_marshal.py ============================================================================== --- pypy/trunk/lib_pypy/_marshal.py (original) +++ pypy/trunk/lib_pypy/_marshal.py Tue Nov 30 14:09:09 2010 @@ -3,7 +3,7 @@ This module contains functions that can read and write Python values in a binary format. The format is specific to Python, but independent of machine architecture issues (e.g., you can write a Python value to a file on a PC, transport the file to a Sun, and read it back there). Details of the format may change between Python versions. """ -import types +import types, __pypy__ from _codecs import utf_8_decode, utf_8_encode TYPE_NULL = '0' @@ -645,15 +645,18 @@ version = 1 + at __pypy__.builtinify def dump(x, f, version=version): # XXX 'version' is ignored, we always dump in a version-0-compatible format m = _Marshaller(f.write) m.dump(x) + at __pypy__.builtinify def load(f): um = _Unmarshaller(f.read) return um.load() + at __pypy__.builtinify def dumps(x, version=version): # XXX 'version' is ignored, we always dump in a version-0-compatible format buffer = [] @@ -661,6 +664,7 @@ m.dump(x) return ''.join(buffer) + at __pypy__.builtinify def loads(s): um = _FastUnmarshaller(s) return um.load() Modified: pypy/trunk/lib_pypy/_minimal_curses.py ============================================================================== --- pypy/trunk/lib_pypy/_minimal_curses.py (original) +++ pypy/trunk/lib_pypy/_minimal_curses.py Tue Nov 30 14:09:09 2010 @@ -35,18 +35,23 @@ # ____________________________________________________________ +import __pypy__ + + at __pypy__.builtinify def setupterm(termstr, fd): err = ctypes.c_int(0) result = clib.setupterm(termstr, fd, ctypes.byref(err)) if result == ERR: raise error("setupterm() failed (err=%d)" % err.value) + at __pypy__.builtinify def tigetstr(cap): result = clib.tigetstr(cap) if ctypes.cast(result, ctypes.c_void_p).value == ERR: return None return ctypes.cast(result, ctypes.c_char_p).value + at __pypy__.builtinify def tparm(str, i1=0, i2=0, i3=0, i4=0, i5=0, i6=0, i7=0, i8=0, i9=0): result = clib.tparm(str, i1, i2, i3, i4, i5, i6, i7, i8, i9) if result is None: Modified: pypy/trunk/lib_pypy/cPickle.py ============================================================================== --- pypy/trunk/lib_pypy/cPickle.py (original) +++ pypy/trunk/lib_pypy/cPickle.py Tue Nov 30 14:09:09 2010 @@ -4,6 +4,7 @@ from pickle import * from pickle import __doc__, __version__, format_version, compatible_formats +import __pypy__ BadPickleGet = KeyError UnpickleableError = PicklingError @@ -31,9 +32,11 @@ def getvalue(self): return self.__f and self.__f.getvalue() + at __pypy__.builtinify def dump(obj, file, protocol=None): Pickler(file, protocol).dump(obj) + at __pypy__.builtinify def dumps(obj, protocol=None): file = StringIO() Pickler(file, protocol).dump(obj) Modified: pypy/trunk/lib_pypy/cmath.py ============================================================================== --- pypy/trunk/lib_pypy/cmath.py (original) +++ pypy/trunk/lib_pypy/cmath.py Tue Nov 30 14:09:09 2010 @@ -5,9 +5,9 @@ # much code borrowed from mathmodule.c -import math +import math, __pypy__ from math import e, pi - + # constants _one = complex(1., 0.) @@ -24,6 +24,7 @@ return complex(real, imag) + at __pypy__.builtinify def acos(x): """acos(x) @@ -32,6 +33,7 @@ return -(_prodi(log((x+(_i*sqrt((_one-(x*x)))))))) + at __pypy__.builtinify def acosh(x): """acosh(x) @@ -41,6 +43,7 @@ return z+z + at __pypy__.builtinify def asin(x): """asin(x) @@ -52,6 +55,7 @@ return -(_prodi(log((sqrt_1_minus_x_sq+_prodi(x))))) + at __pypy__.builtinify def asinh(x): """asinh(x) @@ -61,6 +65,7 @@ return z+z + at __pypy__.builtinify def atan(x): """atan(x) @@ -69,6 +74,7 @@ return _halfi*log(((_i+x)/(_i-x))) + at __pypy__.builtinify def atanh(x): """atanh(x) @@ -77,6 +83,7 @@ return _half*log((_one+x)/(_one-x)) + at __pypy__.builtinify def cos(x): """cos(x) @@ -88,6 +95,7 @@ return complex(real, imag) + at __pypy__.builtinify def cosh(x): """cosh(x) @@ -99,6 +107,7 @@ return complex(real, imag) + at __pypy__.builtinify def exp(x): """exp(x) @@ -111,6 +120,7 @@ return complex(real, imag) + at __pypy__.builtinify def log(x, base=None): """log(x) @@ -125,6 +135,7 @@ return complex(real, imag) + at __pypy__.builtinify def log10(x): """log10(x) @@ -137,6 +148,7 @@ return complex(real, imag) + at __pypy__.builtinify def sin(x): """sin(x) @@ -148,6 +160,7 @@ return complex(real, imag) + at __pypy__.builtinify def sinh(x): """sinh(x) @@ -159,6 +172,7 @@ return complex(real, imag) + at __pypy__.builtinify def sqrt(x): """sqrt(x) @@ -184,6 +198,7 @@ _sqrt_half = sqrt(_half) + at __pypy__.builtinify def tan(x): """tan(x) @@ -204,6 +219,7 @@ return complex(real, imag) + at __pypy__.builtinify def tanh(x): """tanh(x) Modified: pypy/trunk/lib_pypy/grp.py ============================================================================== --- pypy/trunk/lib_pypy/grp.py (original) +++ pypy/trunk/lib_pypy/grp.py Tue Nov 30 14:09:09 2010 @@ -2,7 +2,7 @@ """ This module provides ctypes version of cpython's grp module """ -import sys +import sys, __pypy__ if sys.platform == 'win32': raise ImportError("No grp module on Windows") @@ -64,6 +64,7 @@ return Group(res.contents.gr_name, res.contents.gr_passwd, res.contents.gr_gid, mem) + at __pypy__.builtinify def getgrgid(gid): res = libc.getgrgid(gid) if not res: @@ -71,6 +72,7 @@ raise KeyError(gid) return _group_from_gstruct(res) + at __pypy__.builtinify def getgrnam(name): if not isinstance(name, str): raise TypeError("expected string") @@ -79,6 +81,7 @@ raise KeyError(name) return _group_from_gstruct(res) + at __pypy__.builtinify def getgrall(): libc.setgrent() lst = [] Modified: pypy/trunk/lib_pypy/itertools.py ============================================================================== --- pypy/trunk/lib_pypy/itertools.py (original) +++ pypy/trunk/lib_pypy/itertools.py Tue Nov 30 14:09:09 2010 @@ -27,6 +27,8 @@ 'ifilterfalse', 'imap', 'islice', 'izip', 'repeat', 'starmap', 'takewhile', 'tee'] +import __pypy__ + class chain(object): """Make an iterator that returns elements from the first iterable @@ -565,7 +567,8 @@ def __iter__(self): return self - + + at __pypy__.builtinify def tee(iterable, n=2): """Return n independent iterators from a single iterable. Note : once tee() has made a split, the original iterable Modified: pypy/trunk/lib_pypy/msvcrt.py ============================================================================== --- pypy/trunk/lib_pypy/msvcrt.py (original) +++ pypy/trunk/lib_pypy/msvcrt.py Tue Nov 30 14:09:09 2010 @@ -11,6 +11,7 @@ from ctypes_support import standard_c_lib as _c from ctypes_support import get_errno import errno +import __pypy__ try: open_osfhandle = _c._open_osfhandle @@ -34,6 +35,7 @@ _locking.argtypes = [ctypes.c_int, ctypes.c_int, ctypes.c_int] _locking.restype = ctypes.c_int + at __pypy__.builtinify def locking(fd, mode, nbytes): '''lock or unlock a number of bytes in a file.''' rv = _locking(fd, mode, nbytes) Modified: pypy/trunk/lib_pypy/pwd.py ============================================================================== --- pypy/trunk/lib_pypy/pwd.py (original) +++ pypy/trunk/lib_pypy/pwd.py Tue Nov 30 14:09:09 2010 @@ -10,7 +10,7 @@ exception is raised if the entry asked for cannot be found. """ -import sys +import sys, __pypy__ if sys.platform == 'win32': raise ImportError("No pwd module on Windows") @@ -79,10 +79,12 @@ _endpwent.argtypes = None _endpwent.restype = None + at __pypy__.builtinify def mkpwent(pw): pw = pw.contents return struct_passwd(pw) + at __pypy__.builtinify def getpwuid(uid): """ getpwuid(uid) -> (pw_name,pw_passwd,pw_uid, @@ -95,6 +97,7 @@ raise KeyError("getpwuid(): uid not found: %s" % uid) return mkpwent(pw) + at __pypy__.builtinify def getpwnam(name): """ getpwnam(name) -> (pw_name,pw_passwd,pw_uid, @@ -109,9 +112,10 @@ raise KeyError("getpwname(): name not found: %s" % name) return mkpwent(pw) + at __pypy__.builtinify def getpwall(): """ - "getpwall() -> list_of_entries + getpwall() -> list_of_entries Return a list of all available password database entries, in arbitrary order. See pwd.__doc__ for more on password database entries. """ Modified: pypy/trunk/lib_pypy/pyexpat.py ============================================================================== --- pypy/trunk/lib_pypy/pyexpat.py (original) +++ pypy/trunk/lib_pypy/pyexpat.py Tue Nov 30 14:09:09 2010 @@ -2,7 +2,7 @@ import ctypes import ctypes.util from ctypes import c_char_p, c_int, c_void_p, POINTER, c_char, c_wchar_p -import sys +import sys, __pypy__ # load the platform-specific cache made by running pyexpat.ctc.py from ctypes_config_cache._pyexpat_cache import * @@ -425,9 +425,11 @@ new_parser._set_unknown_encoding_handler() return new_parser + at __pypy__.builtinify def ErrorString(errno): return XML_ErrorString(errno)[:200] + at __pypy__.builtinify def ParserCreate(encoding=None, namespace_separator=None, intern=None): if (not isinstance(encoding, str) and not encoding is None): Added: pypy/trunk/lib_pypy/pypy_test/hack___pypy__.py ============================================================================== --- (empty file) +++ pypy/trunk/lib_pypy/pypy_test/hack___pypy__.py Tue Nov 30 14:09:09 2010 @@ -0,0 +1,9 @@ +# here only to make test runs work even if not running on top of PyPy +import sys, new + +def builtinify(f): + return f + +pypy = new.module('__pypy__') +pypy.builtinify = builtinify +sys.modules.setdefault('__pypy__', pypy) Modified: pypy/trunk/lib_pypy/pypy_test/test_hashlib.py ============================================================================== --- pypy/trunk/lib_pypy/pypy_test/test_hashlib.py (original) +++ pypy/trunk/lib_pypy/pypy_test/test_hashlib.py Tue Nov 30 14:09:09 2010 @@ -2,6 +2,7 @@ from ..ctypes_config_cache import rebuild rebuild.rebuild_one('hashlib.ctc.py') +from . import hack___pypy__ from .. import hashlib, _hashlib def test_unicode(): Modified: pypy/trunk/lib_pypy/pypy_test/test_structseq.py ============================================================================== --- pypy/trunk/lib_pypy/pypy_test/test_structseq.py (original) +++ pypy/trunk/lib_pypy/pypy_test/test_structseq.py Tue Nov 30 14:09:09 2010 @@ -1,6 +1,6 @@ from __future__ import absolute_import import py -from .._structseq import * +from .._structseq import structseqfield, structseqtype class mydata: Modified: pypy/trunk/lib_pypy/readline.py ============================================================================== --- pypy/trunk/lib_pypy/readline.py (original) +++ pypy/trunk/lib_pypy/readline.py Tue Nov 30 14:09:09 2010 @@ -6,8 +6,10 @@ are only stubs at the moment. """ -# Note that PyPy contains also a built-in module 'readline' which will hide -# this one if compiled in. However the built-in module is incomplete; -# don't use it. +import __pypy__ -from pyrepl.readline import * +import pyrepl.readline +__all__ = pyrepl.readline.__all__ + +for _name in __all__: + globals()[_name] = __pypy__.builtinify(getattr(pyrepl.readline, _name)) Modified: pypy/trunk/lib_pypy/resource.py ============================================================================== --- pypy/trunk/lib_pypy/resource.py (original) +++ pypy/trunk/lib_pypy/resource.py Tue Nov 30 14:09:09 2010 @@ -1,4 +1,4 @@ -import sys +import sys, __pypy__ if sys.platform == 'win32': raise ImportError('resource module not available for win32') @@ -77,6 +77,7 @@ ru_nvcsw = _structseq.structseqfield(14) ru_nivcsw = _structseq.structseqfield(15) + at __pypy__.builtinify def rlimit_check_bounds(rlim_cur, rlim_max): if rlim_cur > rlim_t_max: raise ValueError("%d does not fit into rlim_t" % rlim_cur) @@ -89,6 +90,7 @@ ("rlim_max", rlim_t), ) + at __pypy__.builtinify def getrusage(who): ru = _struct_rusage() ret = _getrusage(who, byref(ru)) @@ -116,6 +118,7 @@ ru.ru_nivcsw, )) + at __pypy__.builtinify def getrlimit(resource): if not(0 <= resource < RLIM_NLIMITS): return ValueError("invalid resource specified") @@ -127,6 +130,7 @@ raise error(errno) return (rlim.rlim_cur, rlim.rlim_max) + at __pypy__.builtinify def setrlimit(resource, rlim): if not(0 <= resource < RLIM_NLIMITS): return ValueError("invalid resource specified") @@ -143,6 +147,7 @@ else: raise error(errno) + at __pypy__.builtinify def getpagesize(): pagesize = 0 if _getpagesize: Modified: pypy/trunk/lib_pypy/syslog.py ============================================================================== --- pypy/trunk/lib_pypy/syslog.py (original) +++ pypy/trunk/lib_pypy/syslog.py Tue Nov 30 14:09:09 2010 @@ -5,7 +5,7 @@ syslog facility. """ -import sys +import sys, __pypy__ if sys.platform == 'win32': raise ImportError("No syslog on Windows") @@ -34,9 +34,11 @@ _setlogmask.argtypes = (c_int,) _setlogmask.restype = c_int + at __pypy__.builtinify def openlog(ident, option, facility): _openlog(ident, option, facility) + at __pypy__.builtinify def syslog(arg1, arg2=None): if arg2 is not None: priority, message = arg1, arg2 @@ -44,15 +46,19 @@ priority, message = LOG_INFO, arg1 _syslog(priority, "%s", message) + at __pypy__.builtinify def closelog(): _closelog() + at __pypy__.builtinify def setlogmask(mask): return _setlogmask(mask) + at __pypy__.builtinify def LOG_MASK(pri): return (1 << pri) + at __pypy__.builtinify def LOG_UPTO(pri): return (1 << (pri + 1)) - 1 From fijal at codespeak.net Tue Nov 30 14:10:18 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 30 Nov 2010 14:10:18 +0100 (CET) Subject: [pypy-svn] r79669 - in pypy/branch/out-of-line-guards/pypy/rpython: . lltypesystem test Message-ID: <20101130131018.098A9282BE0@codespeak.net> Author: fijal Date: Tue Nov 30 14:10:17 2010 New Revision: 79669 Modified: pypy/branch/out-of-line-guards/pypy/rpython/lltypesystem/rclass.py pypy/branch/out-of-line-guards/pypy/rpython/rclass.py pypy/branch/out-of-line-guards/pypy/rpython/test/test_rclass.py Log: An attempt to store necessary information on instances Modified: pypy/branch/out-of-line-guards/pypy/rpython/lltypesystem/rclass.py ============================================================================== --- pypy/branch/out-of-line-guards/pypy/rpython/lltypesystem/rclass.py (original) +++ pypy/branch/out-of-line-guards/pypy/rpython/lltypesystem/rclass.py Tue Nov 30 14:10:17 2010 @@ -15,7 +15,7 @@ RuntimeTypeInfo, getRuntimeTypeInfo, typeOf, \ Array, Char, Void, \ FuncType, Bool, Signed, functionptr, PyObject -from pypy.rpython.lltypesystem import lltype +from pypy.rpython.lltypesystem import lltype, llmemory from pypy.rpython.robject import PyObjRepr, pyobj_repr from pypy.rpython.extregistry import ExtRegistryEntry from pypy.annotation import model as annmodel @@ -84,6 +84,12 @@ 'stack': 'raw', } +# a linked-list of assembler codes to invalidate in case jit_invariant_fields +# are modified +ASMCODE = lltype.GcForwardReference() +ASMCODE.become(GcStruct('asmcode', ('address', llmemory.Address), + ('next', lltype.Ptr(ASMCODE)))) + def cast_vtable_to_typeptr(vtable): while typeOf(vtable).TO != OBJECT_VTABLE: vtable = vtable.super @@ -328,10 +334,16 @@ if self.classdef is None: fields['__class__'] = 'typeptr', get_type_repr(self.rtyper) else: + myllfields = [] + invariant_fields = self.classdef.classdesc.classdict.get( + '_jit_invariant_fields_') + if invariant_fields is not None: + for field in invariant_fields.value: + myllfields.append(('asmcodes_' + field, + lltype.Ptr(ASMCODE))) # instance attributes attrs = self.classdef.attrs.items() attrs.sort() - myllfields = [] for name, attrdef in attrs: if not attrdef.readonly: r = self.rtyper.getrepr(attrdef.s_value) Modified: pypy/branch/out-of-line-guards/pypy/rpython/rclass.py ============================================================================== --- pypy/branch/out-of-line-guards/pypy/rpython/rclass.py (original) +++ pypy/branch/out-of-line-guards/pypy/rpython/rclass.py Tue Nov 30 14:10:17 2010 @@ -209,9 +209,9 @@ while rbase.classdef is not None: fields += getattr(rbase, varname) rbase = rbase.rbase - self._parse_field_list(fields, accessor) + self._parse_field_list(fields, accessor, varname) - def _parse_field_list(self, fields, accessor): + def _parse_field_list(self, fields, accessor, varname): with_suffix = {} for name in fields: if name.endswith('[*]'): @@ -223,7 +223,10 @@ mangled_name, r = self._get_field(name) except KeyError: continue - with_suffix[mangled_name] = suffix + if varname == 'jit_invariant_fields': + with_suffix[mangled_name] = 'asmcodes_' + name + else: + with_suffix[mangled_name] = suffix accessor.initialize(self.object_type, with_suffix) return with_suffix Modified: pypy/branch/out-of-line-guards/pypy/rpython/test/test_rclass.py ============================================================================== --- pypy/branch/out-of-line-guards/pypy/rpython/test/test_rclass.py (original) +++ pypy/branch/out-of-line-guards/pypy/rpython/test/test_rclass.py Tue Nov 30 14:10:17 2010 @@ -763,8 +763,10 @@ t, typer, graph = self.gengraph(f, []) A_TYPE = deref(graph.getreturnvar().concretetype) accessor = A_TYPE._hints["jit_invariant_fields"] - assert accessor.fields == {'inst_x': ''} or \ - accessor.fields == {'ox': ''} + assert accessor.fields == {'inst_x': 'asmcodes_x'} or \ + accessor.fields == {'ox': 'asmcodes_x'} + if accessor.fields.keys() == ['inst_x']: + assert 'asmcodes_x' in A_TYPE._names def test_immutable_fields_subclass_1(self): class A(object): From fijal at codespeak.net Tue Nov 30 14:12:54 2010 From: fijal at codespeak.net (fijal at codespeak.net) Date: Tue, 30 Nov 2010 14:12:54 +0100 (CET) Subject: [pypy-svn] r79670 - pypy/branch/out-of-line-guards/pypy/rpython/test Message-ID: <20101130131254.101395080C@codespeak.net> Author: fijal Date: Tue Nov 30 14:12:53 2010 New Revision: 79670 Modified: pypy/branch/out-of-line-guards/pypy/rpython/test/test_rclass.py Log: simply skip this test on ootype Modified: pypy/branch/out-of-line-guards/pypy/rpython/test/test_rclass.py ============================================================================== --- pypy/branch/out-of-line-guards/pypy/rpython/test/test_rclass.py (original) +++ pypy/branch/out-of-line-guards/pypy/rpython/test/test_rclass.py Tue Nov 30 14:12:53 2010 @@ -763,10 +763,8 @@ t, typer, graph = self.gengraph(f, []) A_TYPE = deref(graph.getreturnvar().concretetype) accessor = A_TYPE._hints["jit_invariant_fields"] - assert accessor.fields == {'inst_x': 'asmcodes_x'} or \ - accessor.fields == {'ox': 'asmcodes_x'} - if accessor.fields.keys() == ['inst_x']: - assert 'asmcodes_x' in A_TYPE._names + assert accessor.fields == {'inst_x': 'asmcodes_x'} + assert 'asmcodes_x' in A_TYPE._names def test_immutable_fields_subclass_1(self): class A(object): @@ -1063,6 +1061,8 @@ class TestOOtype(BaseTestRclass, OORtypeMixin): + def test_jit_invariant_fields(self): + py.test.skip("no ootype support yet") def test__del__(self): class A(object): From arigo at codespeak.net Tue Nov 30 14:15:05 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 14:15:05 +0100 (CET) Subject: [pypy-svn] r79671 - in pypy/extradoc/pypy.org: . source Message-ID: <20101130131505.EA27B282BE7@codespeak.net> Author: arigo Date: Tue Nov 30 14:15:04 2010 New Revision: 79671 Modified: pypy/extradoc/pypy.org/download.html pypy/extradoc/pypy.org/source/download.txt Log: Add "Checksums". Modified: pypy/extradoc/pypy.org/download.html ============================================================================== --- pypy/extradoc/pypy.org/download.html (original) +++ pypy/extradoc/pypy.org/download.html Tue Nov 30 14:15:04 2010 @@ -58,6 +58,7 @@
  • Installing (optional)
  • Building from source
    • +
  • Checksums

    • “JIT Compiler” version

    Modified: pypy/extradoc/pypy.org/source/download.txt ============================================================================== --- pypy/extradoc/pypy.org/source/download.txt (original) +++ pypy/extradoc/pypy.org/source/download.txt Tue Nov 30 14:15:04 2010 @@ -19,6 +19,7 @@ * `Installing`_ (optional) * `Building from source`_ + * `Checksums`_ .. _`Default (with a JIT Compiler)`: From arigo at codespeak.net Tue Nov 30 14:17:27 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 14:17:27 +0100 (CET) Subject: [pypy-svn] r79672 - pypy/trunk/pypy/doc/config Message-ID: <20101130131727.6DF5A50810@codespeak.net> Author: arigo Date: Tue Nov 30 14:17:26 2010 New Revision: 79672 Added: pypy/trunk/pypy/doc/config/objspace.usemodules.binascii.txt Log: Add missing doc. Added: pypy/trunk/pypy/doc/config/objspace.usemodules.binascii.txt ============================================================================== --- (empty file) +++ pypy/trunk/pypy/doc/config/objspace.usemodules.binascii.txt Tue Nov 30 14:17:26 2010 @@ -0,0 +1 @@ +Use the RPython 'binascii' module. From arigo at codespeak.net Tue Nov 30 14:19:00 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 14:19:00 +0100 (CET) Subject: [pypy-svn] r79673 - pypy/trunk/lib_pypy/pypy_test Message-ID: <20101130131900.DC788282BE3@codespeak.net> Author: arigo Date: Tue Nov 30 14:18:59 2010 New Revision: 79673 Modified: pypy/trunk/lib_pypy/pypy_test/test_hashlib.py Log: Fix: sha1 might be the RPython-specific version instead of the one from _hashlib. Modified: pypy/trunk/lib_pypy/pypy_test/test_hashlib.py ============================================================================== --- pypy/trunk/lib_pypy/pypy_test/test_hashlib.py (original) +++ pypy/trunk/lib_pypy/pypy_test/test_hashlib.py Tue Nov 30 14:18:59 2010 @@ -6,7 +6,7 @@ from .. import hashlib, _hashlib def test_unicode(): - assert isinstance(hashlib.new('sha1', u'xxx'), _hashlib.hash) + assert isinstance(hashlib.new('sha256', u'xxx'), _hashlib.hash) def test_attributes(): for name, expected_size in {'md5': 16, From arigo at codespeak.net Tue Nov 30 14:53:47 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 14:53:47 +0100 (CET) Subject: [pypy-svn] r79674 - pypy/branch/jit-free-asm2 Message-ID: <20101130135347.DCF10282BD4@codespeak.net> Author: arigo Date: Tue Nov 30 14:53:38 2010 New Revision: 79674 Added: pypy/branch/jit-free-asm2/ - copied from r79673, pypy/trunk/ Log: Re-branching to finish jit-free-asm. From arigo at codespeak.net Tue Nov 30 14:56:30 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 14:56:30 +0100 (CET) Subject: [pypy-svn] r79675 - in pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport: . test Message-ID: <20101130135630.EF904282BDC@codespeak.net> Author: arigo Date: Tue Nov 30 14:56:22 2010 New Revision: 79675 Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/asmmemmgr.py pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/test/test_asmmemmgr.py Log: Add the class MachineDataBlockWrapper. Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/asmmemmgr.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/asmmemmgr.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/asmmemmgr.py Tue Nov 30 14:56:22 2010 @@ -43,6 +43,22 @@ self.total_mallocs -= (stop - start) self._add_free_block(start, stop) + def open_malloc(self, minsize): + """Allocate at least minsize bytes. Returns (start, stop).""" + result = self._allocate_block(minsize) + (start, stop) = result + self.total_mallocs += stop - start + return result + + def open_free(self, middle, stop): + """Used for freeing the end of an open-allocated block of memory.""" + if stop - middle >= self.min_fragment: + self.total_mallocs -= (stop - middle) + self._add_free_block(middle, stop) + return True + else: + return False # too small to record + def _allocate_large_block(self, minsize): # Compute 'size' from 'minsize': it must be rounded up to # 'large_alloc_size'. Additionally, we use the following line @@ -140,6 +156,40 @@ self._allocated = None +class MachineDataBlockWrapper(object): + def __init__(self, asmmemmgr, allblocks): + self.asmmemmgr = asmmemmgr + self.allblocks = allblocks + self.rawstart = 0 + self.rawposition = 0 + self.rawstop = 0 + + def done(self): + if self.rawstart != 0: + if self.asmmemmgr.open_free(self.rawposition, self.rawstop): + self.rawstop = self.rawposition + self.allblocks.append((self.rawstart, self.rawstop)) + self.rawstart = 0 + self.rawposition = 0 + self.rawstop = 0 + + def _allocate_next_block(self, minsize): + self.done() + self.rawstart, self.rawstop = self.asmmemmgr.open_malloc(minsize) + self.rawposition = self.rawstart + + def malloc_aligned(self, size, alignment): + p = self.rawposition + p = (p + alignment - 1) & (-alignment) + if p + size > self.rawstop: + self._allocate_next_block(size + alignment - 1) + p = self.rawposition + p = (p + alignment - 1) & (-alignment) + assert p + size <= self.rawstop + self.rawposition = p + size + return p + + class BlockBuilderMixin(object): _mixin_ = True # A base class to generate assembler. It is equivalent to just a list Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/test/test_asmmemmgr.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/test/test_asmmemmgr.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/test/test_asmmemmgr.py Tue Nov 30 14:56:22 2010 @@ -1,5 +1,6 @@ import random from pypy.jit.backend.llsupport.asmmemmgr import AsmMemoryManager +from pypy.jit.backend.llsupport.asmmemmgr import MachineDataBlockWrapper from pypy.jit.backend.llsupport.asmmemmgr import BlockBuilderMixin from pypy.rpython.lltypesystem import lltype, rffi @@ -215,3 +216,41 @@ def test_blockbuildermixin2(): test_blockbuildermixin(translated=False) + +def test_machinedatablock(): + ops = [] + class FakeMemMgr: + _addr = 1597 + def open_malloc(self, minsize): + result = (self._addr, self._addr + 100) + ops.append(('malloc', minsize) + result) + self._addr += 200 + return result + def open_free(self, frm, to): + ops.append(('free', frm, to)) + return to - frm >= 8 + # + allblocks = [] + md = MachineDataBlockWrapper(FakeMemMgr(), allblocks) + p = md.malloc_aligned(26, 16) + assert p == 1600 + assert ops == [('malloc', 26 + 15, 1597, 1697)] + del ops[:] + # + p = md.malloc_aligned(26, 16) + assert p == 1632 + p = md.malloc_aligned(26, 16) + assert p == 1664 + assert allblocks == [] + assert ops == [] + # + p = md.malloc_aligned(27, 16) + assert p == 1808 + assert allblocks == [(1597, 1697)] + assert ops == [('free', 1690, 1697), + ('malloc', 27 + 15, 1797, 1897)] + del ops[:] + # + md.done() + assert allblocks == [(1597, 1697), (1797, 1835)] + assert ops == [('free', 1835, 1897)] From arigo at codespeak.net Tue Nov 30 15:10:02 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 15:10:02 +0100 (CET) Subject: [pypy-svn] r79676 - in pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport: . test Message-ID: <20101130141002.3D1C8282BE9@codespeak.net> Author: arigo Date: Tue Nov 30 15:09:48 2010 New Revision: 79676 Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/gc.py pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/test/test_gc.py Log: Re-simplify the interface in the llsupport/gc module: compress the call shapes one by one again. Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/gc.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/gc.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/gc.py Tue Nov 30 15:09:48 2010 @@ -222,7 +222,7 @@ LOC_EBP_MINUS = 3 GCMAP_ARRAY = rffi.CArray(lltype.Signed) - CALLSHAPE_ARRAY = rffi.CArray(rffi.UCHAR) + CALLSHAPE_ARRAY_PTR = rffi.CArrayPtr(rffi.UCHAR) def __init__(self): # '_gcmap' is an array of length '_gcmap_maxlength' of addresses. @@ -264,8 +264,7 @@ self._gcmap_sorted = True return sorted - @rgc.no_collect - def _put(self, retaddr, callshapeaddr): + def put(self, retaddr, callshapeaddr): """'retaddr' is the address just after the CALL. 'callshapeaddr' is the address of the raw 'shape' marker. Both addresses are actually integers here.""" @@ -308,37 +307,6 @@ lltype.free(oldgcmap, flavor='raw', track_allocation=False) return j - def add_raw_gcroot_markers(self, asmmemmgr, allblocks, - markers, total_size, rawstart): - """The interface is a bit custom, but this routine writes the - shapes of gcroots (for the GC to use) into raw memory.""" - # xxx so far, we never try to share them. But right now - # the amount of potential sharing would not be too large. - dst = 1 - stop = 0 - for relpos, shape in markers: - # - if dst + len(shape) > stop: - # No more space in the previous raw block, - # allocate a raw block of memory big enough to fit - # as many of the remaining 'shapes' as possible - start, stop = asmmemmgr.malloc(len(shape), total_size) - # add the raw block to 'compiled_loop_token.asmmemmgr_blocks' - allblocks.append((start, stop)) - dst = start - # - # add the entry 'pos_after_call -> dst' to the table - self._put(rawstart + relpos, dst) - # Copy 'shape' into the raw memory, reversing the order - # of the bytes. Similar to compress_callshape() in - # trackgcroot.py. - total_size -= len(shape) - src = len(shape) - 1 - while src >= 0: - rffi.cast(rffi.CCHARP, dst)[0] = shape[src] - dst += 1 - src -= 1 - @rgc.no_collect def freeing_block(self, start, stop): # if [start:stop] is a raw block of assembler, then look up the @@ -409,6 +377,16 @@ assert reg_index > 0 shape.append(chr(self.LOC_REG | (reg_index << 2))) + def compress_callshape(self, shape, datablockwrapper): + # Similar to compress_callshape() in trackgcroot.py. + # Returns an address to raw memory (as an integer). + length = len(shape) + rawaddr = datablockwrapper.malloc_aligned(length, 1) + p = rffi.cast(self.CALLSHAPE_ARRAY_PTR, rawaddr) + for i in range(length): + p[length-1-i] = rffi.cast(rffi.UCHAR, shape[i]) + return rawaddr + class WriteBarrierDescr(AbstractDescr): def __init__(self, gc_ll_descr): Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/test/test_gc.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/test/test_gc.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/test/test_gc.py Tue Nov 30 15:09:48 2010 @@ -91,11 +91,27 @@ assert shape == map(chr, [6, 7, 11, 15, 2, 0, num1a, num2b, num2a, 4, 8, 12, 16]) + def test_compress_callshape(self): + class FakeDataBlockWrapper: + def malloc_aligned(self, size, alignment): + assert alignment == 1 # here + assert size == 4 + return rffi.cast(lltype.Signed, p) + datablockwrapper = FakeDataBlockWrapper() + p = lltype.malloc(rffi.CArray(lltype.Char), 4, immortal=True) + gcrootmap = GcRootMap_asmgcc() + shape = ['a', 'b', 'c', 'd'] + gcrootmap.compress_callshape(shape, datablockwrapper) + assert p[0] == 'd' + assert p[1] == 'c' + assert p[2] == 'b' + assert p[3] == 'a' + def test_put_basic(self): gcrootmap = GcRootMap_asmgcc() retaddr = 1234567890 shapeaddr = 51627384 - gcrootmap._put(retaddr, shapeaddr) + gcrootmap.put(retaddr, shapeaddr) assert gcrootmap._gcmap[0] == retaddr assert gcrootmap._gcmap[1] == shapeaddr p = rffi.cast(rffi.LONGP, gcrootmap.gcmapstart()) @@ -109,7 +125,7 @@ for i in range(700): shapeaddr = i * 100 + 1 retaddr = 123456789 + i - gcrootmap._put(retaddr, shapeaddr) + gcrootmap.put(retaddr, shapeaddr) for i in range(700): assert gcrootmap._gcmap[i*2+0] == 123456789 + i assert gcrootmap._gcmap[i*2+1] == i * 100 + 1 @@ -126,7 +142,7 @@ for i in range(700): shapeaddr = i * 100 # 0 if i == 0 retaddr = 123456789 + i - gcrootmap._put(retaddr, shapeaddr) + gcrootmap.put(retaddr, shapeaddr) if shapeaddr != 0: expected.append((retaddr, shapeaddr)) # at the first resize, the 0 should be removed @@ -142,72 +158,11 @@ # check that we can again insert 350 entries without a resize oldgcmap = gcrootmap._gcmap for i in range(0, 699, 2): - gcrootmap._put(515151 + i + repeat, 626262 + i) + gcrootmap.put(515151 + i + repeat, 626262 + i) expected.append((515151 + i + repeat, 626262 + i)) assert gcrootmap._gcmap == oldgcmap check() - def test_add_raw_gcroot_markers_maxalloc(self): - class FakeAsmMemMgr: - def malloc(self, minsize, maxsize): - assert minsize == 4 - assert maxsize == 7 - return (prawstart, prawstart + 8) - put = [] - def fakeput(a, b): - put.append((a, b)) - gcrootmap = GcRootMap_asmgcc() - gcrootmap._put = fakeput - memmgr = FakeAsmMemMgr() - allblocks = [] - p = lltype.malloc(rffi.CArray(lltype.Char), 7, immortal=True) - prawstart = rffi.cast(lltype.Signed, p) - gcrootmap.add_raw_gcroot_markers(memmgr, allblocks, - [(2, ['a', 'b', 'c', 'd']), - (4, ['e', 'f', 'g'])], - 4 + 3, 1200000) - assert allblocks == [(prawstart, prawstart + 8)] - assert ''.join([p[i] for i in range(7)]) == 'dcbagfe' - assert put == [(1200002, prawstart), - (1200004, prawstart + 4)] - - def test_add_raw_gcroot_markers_minalloc(self): - class FakeAsmMemMgr: - callnum = 0 - def malloc(self, minsize, maxsize): - self.callnum += 1 - if self.callnum == 1: - assert minsize == 4 - assert maxsize == 7 - return (prawstart, prawstart + 6) - elif self.callnum == 2: - assert minsize == 3 - assert maxsize == 3 - return (qrawstart, qrawstart + 5) - else: - raise AssertionError - put = [] - def fakeput(a, b): - put.append((a, b)) - gcrootmap = GcRootMap_asmgcc() - gcrootmap._put = fakeput - memmgr = FakeAsmMemMgr() - allblocks = [] - p = lltype.malloc(rffi.CArray(lltype.Char), 6, immortal=True) - prawstart = rffi.cast(lltype.Signed, p) - q = lltype.malloc(rffi.CArray(lltype.Char), 5, immortal=True) - qrawstart = rffi.cast(lltype.Signed, q) - gcrootmap.add_raw_gcroot_markers(memmgr, allblocks, - [(2, ['a', 'b', 'c', 'd']), - (4, ['e', 'f', 'g'])], - 4 + 3, 1200000) - assert allblocks == [(prawstart, prawstart + 6), - (qrawstart, qrawstart + 5)] - assert ''.join([p[i] for i in range(4)]) == 'dcba' - assert ''.join([q[i] for i in range(3)]) == 'gfe' - assert put == [(1200002, prawstart), - (1200004, qrawstart)] - def test_freeing_block(self): from pypy.jit.backend.llsupport import gc class Asmgcroot: From arigo at codespeak.net Tue Nov 30 15:28:27 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 15:28:27 +0100 (CET) Subject: [pypy-svn] r79677 - pypy/branch/rlist-jit/pypy/jit/tl/spli Message-ID: <20101130142827.4A11C282BEB@codespeak.net> Author: arigo Date: Tue Nov 30 15:28:24 2010 New Revision: 79677 Modified: pypy/branch/rlist-jit/pypy/jit/tl/spli/interpreter.py Log: Assert non-negative in this demo too. Modified: pypy/branch/rlist-jit/pypy/jit/tl/spli/interpreter.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/tl/spli/interpreter.py (original) +++ pypy/branch/rlist-jit/pypy/jit/tl/spli/interpreter.py Tue Nov 30 15:28:24 2010 @@ -105,26 +105,32 @@ self.stack_depth += 1 def pop(self): - self.stack_depth -= 1 - val = self.value_stack[self.stack_depth] - self.value_stack[self.stack_depth] = None + sd = self.stack_depth - 1 + assert sd >= 0 + self.stack_depth = sd + val = self.value_stack[sd] + self.value_stack[sd] = None return val def pop_many(self, n): return [self.pop() for i in range(n)] def peek(self): - return self.value_stack[self.stack_depth - 1] + sd = self.stack_depth - 1 + assert sd >= 0 + return self.value_stack[sd] def POP_TOP(self, _, next_instr, code): self.pop() return next_instr def LOAD_FAST(self, name_index, next_instr, code): + assert name_index >= 0 self.push(self.locals[name_index]) return next_instr def STORE_FAST(self, name_index, next_instr, code): + assert name_index >= 0 self.locals[name_index] = self.pop() return next_instr From arigo at codespeak.net Tue Nov 30 15:33:31 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 15:33:31 +0100 (CET) Subject: [pypy-svn] r79678 - in pypy/branch/jit-free-asm2/pypy/jit/backend: llsupport x86 x86/test Message-ID: <20101130143331.E5B39282BEC@codespeak.net> Author: arigo Date: Tue Nov 30 15:33:30 2010 New Revision: 79678 Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/asmmemmgr.py pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py pypy/branch/jit-free-asm2/pypy/jit/backend/x86/regalloc.py pypy/branch/jit-free-asm2/pypy/jit/backend/x86/test/test_gc_integration.py Log: Work in progress. Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/asmmemmgr.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/asmmemmgr.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/asmmemmgr.py Tue Nov 30 15:33:30 2010 @@ -206,7 +206,6 @@ SUBBLOCK_PTR.TO.become(SUBBLOCK) gcroot_markers = None - gcroot_markers_total_size = 0 def __init__(self, translated=None): if translated is None: @@ -274,11 +273,8 @@ self.copy_to_raw_memory(rawstart) if self.gcroot_markers is not None: assert gcrootmap is not None - gcrootmap.add_raw_gcroot_markers(asmmemmgr, - allblocks, - self.gcroot_markers, - self.gcroot_markers_total_size, - rawstart) + for pos, mark in self.gcroot_markers: + gcrootmap.put(rawstart + pos, mark) return rawstart def _become_a_plain_block_builder(self): @@ -297,4 +293,3 @@ if self.gcroot_markers is None: self.gcroot_markers = [] self.gcroot_markers.append((self.get_relative_pos(), mark)) - self.gcroot_markers_total_size += len(mark) Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py Tue Nov 30 15:33:30 2010 @@ -1,5 +1,6 @@ import sys, os from pypy.jit.backend.llsupport import symbolic +from pypy.jit.backend.llsupport.asmmemmgr import MachineDataBlockWrapper from pypy.jit.metainterp.history import Const, Box, BoxInt, BoxPtr, BoxFloat from pypy.jit.metainterp.history import (AbstractFailDescr, INT, REF, FLOAT, LoopToken) @@ -83,6 +84,7 @@ self.debug_counter_descr = cpu.fielddescrof(DEBUG_COUNTER, 'i') self.fail_boxes_count = 0 self._current_depths_cache = (0, 0) + self.datablockwrapper = None self.teardown() def leave_jitted_hook(self): @@ -125,10 +127,14 @@ self.set_debug(have_debug_prints()) debug_stop('jit-backend-counts') - def setup(self): + def setup(self, looptoken): assert self.memcpy_addr != 0, "setup_once() not called?" self.pending_guard_tokens = [] self.mc = codebuf.MachineCodeBlockWrapper() + if self.datablockwrapper is None: + allblocks = looptoken.compiled_loop_token.asmmemmgr_blocks + self.datablockwrapper = MachineDataBlockWrapper(self.cpu.asmmemmgr, + allblocks) def teardown(self): self.pending_guard_tokens = None @@ -209,7 +215,7 @@ # Arguments should be unique assert len(set(inputargs)) == len(inputargs) - self.setup() + self.setup(looptoken) self.currently_compiling_loop = looptoken funcname = self._find_debug_merge_point(operations) if log: @@ -235,7 +241,7 @@ self.write_pending_failure_recoveries() fullsize = self.mc.get_relative_pos() # - rawstart = self.materialize(looptoken) + rawstart = self.materialize_loop(looptoken) debug_print("Loop #%d (%s) has address %x to %x" % ( looptoken.number, funcname, rawstart + self.looppos, @@ -268,7 +274,7 @@ "was already compiled!") return - self.setup() + self.setup(original_loop_token) funcname = self._find_debug_merge_point(operations) if log: self._register_counter() @@ -289,7 +295,7 @@ self.write_pending_failure_recoveries() fullsize = self.mc.get_relative_pos() # - rawstart = self.materialize(original_loop_token) + rawstart = self.materialize_loop(original_loop_token) debug_print("Bridge out of guard %d (%s) has address %x to %x" % (descr_number, funcname, rawstart, rawstart + codeendpos)) @@ -328,7 +334,9 @@ p = rffi.cast(rffi.INTP, addr) p[0] = rffi.cast(rffi.INT, relative_target) - def materialize(self, looptoken): + def materialize_loop(self, looptoken): + self.datablockwrapper.done() # finish using cpu.asmmemmgr + self.datablockwrapper = None clt = looptoken.compiled_loop_token if clt.asmmemmgr_blocks is None: clt.asmmemmgr_blocks = [] Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/x86/regalloc.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/x86/regalloc.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/x86/regalloc.py Tue Nov 30 15:33:30 2010 @@ -1108,7 +1108,8 @@ if (isinstance(v, BoxPtr) and self.rm.stays_alive(v)): assert reg in self.rm.REGLOC_TO_GCROOTMAP_REG_INDEX gcrootmap.add_callee_save_reg(shape, self.rm.REGLOC_TO_GCROOTMAP_REG_INDEX[reg]) - return shape + return gcrootmap.compress_callshape(shape, + self.assembler.datablockwrapper) def consider_force_token(self, op): loc = self.rm.force_allocate_reg(op.result) Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/x86/test/test_gc_integration.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/x86/test/test_gc_integration.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/x86/test/test_gc_integration.py Tue Nov 30 15:33:30 2010 @@ -33,6 +33,10 @@ def add_callee_save_reg(self, shape, reg_index): index_to_name = { 1: 'ebx', 2: 'esi', 3: 'edi' } shape.append(index_to_name[reg_index]) + def compress_callshape(self, shape, datablockwrapper): + assert datablockwrapper == 'fakedatablockwrapper' + assert shape[0] == 'shape' + return ['compressed'] + shape[1:] class MockGcDescr(GcCache): def get_funcptr_for_new(self): @@ -57,6 +61,7 @@ cpu = CPU(None, None) cpu.setup_once() regalloc = RegAlloc(MockAssembler(cpu, MockGcDescr(False))) + regalloc.assembler.datablockwrapper = 'fakedatablockwrapper' boxes = [BoxPtr() for i in range(len(X86RegisterManager.all_regs))] longevity = {} for box in boxes: @@ -79,7 +84,7 @@ assert len(regalloc.assembler.movs) == 3 # mark = regalloc.get_mark_gc_roots(cpu.gc_ll_descr.gcrootmap) - assert mark[0] == 'shape' + assert mark[0] == 'compressed' base = -WORD * FRAME_FIXED_SIZE expected = ['ebx', 'esi', 'edi', base, base-WORD, base-WORD*2] assert dict.fromkeys(mark[1:]) == dict.fromkeys(expected) From arigo at codespeak.net Tue Nov 30 15:34:48 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 15:34:48 +0100 (CET) Subject: [pypy-svn] r79679 - pypy/branch/rlist-jit/pypy/jit/codewriter Message-ID: <20101130143448.6FFAD282BE0@codespeak.net> Author: arigo Date: Tue Nov 30 15:34:47 2010 New Revision: 79679 Modified: pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py Log: Fix a bit randomly. I *think* that the case can show up. Doesn't hurt in any case... Modified: pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py (original) +++ pypy/branch/rlist-jit/pypy/jit/codewriter/jtransform.py Tue Nov 30 15:34:47 2010 @@ -197,8 +197,8 @@ if op.args[0] in self.vable_array_vars: self.vable_array_vars[op.result]= self.vable_array_vars[op.args[0]] - def rewrite_op_cast_pointer(self, op): pass - def rewrite_op_cast_opaque_ptr(self, op): pass # rlib.rerased + rewrite_op_cast_pointer = rewrite_op_same_as + rewrite_op_cast_pointer = rewrite_op_same_as # rlib.rerased def rewrite_op_cast_primitive(self, op): pass def rewrite_op_cast_bool_to_int(self, op): pass def rewrite_op_cast_bool_to_uint(self, op): pass From arigo at codespeak.net Tue Nov 30 15:35:01 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 15:35:01 +0100 (CET) Subject: [pypy-svn] r79680 - pypy/branch/rlist-jit/pypy/rpython/test Message-ID: <20101130143501.238535080C@codespeak.net> Author: arigo Date: Tue Nov 30 15:34:59 2010 New Revision: 79680 Modified: pypy/branch/rlist-jit/pypy/rpython/test/test_rlist.py Log: Fix the test, for pypy.translator.oosupport.test.test_treebuilder. Modified: pypy/branch/rlist-jit/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/branch/rlist-jit/pypy/rpython/test/test_rlist.py (original) +++ pypy/branch/rlist-jit/pypy/rpython/test/test_rlist.py Tue Nov 30 15:34:59 2010 @@ -1077,7 +1077,7 @@ res = self.interpret(f, [0]) assert res == 1 - if self.rlist is ll_rlist: + if self.type_system == 'lltype': # on lltype we always get an AssertionError py.test.raises(AssertionError, self.interpret, f, [1]) else: @@ -1128,7 +1128,7 @@ res = self.interpret(f, [0]) assert res == 1 - if self.rlist is ll_rlist: + if self.type_system == 'lltype': # on lltype we always get an AssertionError py.test.raises(AssertionError, self.interpret, f, [1]) else: @@ -1171,7 +1171,7 @@ res = self.interpret(f, [0]) assert res == 1 - if self.rlist is ll_rlist: + if self.type_system == 'lltype': # on lltype we always get an AssertionError py.test.raises(AssertionError, self.interpret, f, [1]) else: @@ -1361,6 +1361,7 @@ ("y[*]" in immutable_fields) class TestLLtype(BaseTestRlist, LLRtypeMixin): + type_system = 'lltype' rlist = ll_rlist def test_memoryerror(self): @@ -1464,3 +1465,4 @@ class TestOOtype(BaseTestRlist, OORtypeMixin): rlist = oo_rlist + type_system = 'ootype' From arigo at codespeak.net Tue Nov 30 15:41:25 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 15:41:25 +0100 (CET) Subject: [pypy-svn] r79681 - pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/test Message-ID: <20101130144125.DD559282BE0@codespeak.net> Author: arigo Date: Tue Nov 30 15:41:23 2010 New Revision: 79681 Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/test/test_asmmemmgr.py Log: Fix the test. Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/test/test_asmmemmgr.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/test/test_asmmemmgr.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/llsupport/test/test_asmmemmgr.py Tue Nov 30 15:41:23 2010 @@ -152,14 +152,10 @@ prev_total = new_total def test_insert_gcroot_marker(self): + puts = [] class FakeGcRootMap: - def add_raw_gcroot_markers(self, asmmemmgr, allblocks, markers, - total_size, rawstart): - self.asmmemmgr = asmmemmgr - self.allblocks = allblocks - self.markers = markers - self.total_size = total_size - self.rawstart = rawstart + def put(self, retaddr, mark): + puts.append((retaddr, mark)) # mc = BlockBuilderMixin() mc.writechar('X') @@ -182,10 +178,8 @@ assert p[4] == 'Z' assert p[5] == 'z' assert allblocks == [(rawstart, rawstart + 6)] - assert gcrootmap.markers == [(2, ['a', 'b', 'c', 'd']), - (4, ['e', 'f', 'g'])] - assert gcrootmap.total_size == 4 + 3 - assert gcrootmap.rawstart == rawstart + assert puts == [(rawstart + 2, ['a', 'b', 'c', 'd']), + (rawstart + 4, ['e', 'f', 'g'])] def test_blockbuildermixin(translated=True): From david at codespeak.net Tue Nov 30 15:46:14 2010 From: david at codespeak.net (david at codespeak.net) Date: Tue, 30 Nov 2010 15:46:14 +0100 (CET) Subject: [pypy-svn] r79682 - pypy/branch/arm-backend/pypy/jit/backend/arm/test Message-ID: <20101130144614.38F7950810@codespeak.net> Author: david Date: Tue Nov 30 15:46:12 2010 New Revision: 79682 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py Log: Skip test_cond_call_gc_wb for now Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/test/test_runner.py Tue Nov 30 15:46:12 2010 @@ -1,3 +1,4 @@ +import py from pypy.jit.backend.arm.runner import ArmCPU from pypy.jit.backend.test.runner_test import LLtypeBackendTest from pypy.jit.backend.arm.test.support import skip_unless_arm @@ -53,3 +54,6 @@ output = [self.cpu.get_latest_value_int(i-1) for i in range(1, 15)] expected = [3, 7, 11, 15, 19, 23, 27, 3, 7, 11, 15, 19, 23, 27] assert output == expected + + def test_cond_call_gc_wb(self, *args): + py.test.skip('needs gc support') From arigo at codespeak.net Tue Nov 30 15:46:41 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 15:46:41 +0100 (CET) Subject: [pypy-svn] r79683 - pypy/branch/jit-free-asm2/pypy/jit/backend/x86 Message-ID: <20101130144641.B6BAA282BE9@codespeak.net> Author: arigo Date: Tue Nov 30 15:46:40 2010 New Revision: 79683 Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py Log: Fix, and add an XXX comment. Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py Tue Nov 30 15:46:40 2010 @@ -132,7 +132,7 @@ self.pending_guard_tokens = [] self.mc = codebuf.MachineCodeBlockWrapper() if self.datablockwrapper is None: - allblocks = looptoken.compiled_loop_token.asmmemmgr_blocks + allblocks = self.get_asmmemmgr_blocks(looptoken) self.datablockwrapper = MachineDataBlockWrapper(self.cpu.asmmemmgr, allblocks) @@ -209,6 +209,11 @@ _x86_arglocs _x86_debug_checksum ''' + # XXX this function is too longish and contains some code + # duplication with assemble_bridge(). Also, we should think + # about not storing on 'self' attributes that will live only + # for the duration of compiling one loop or a one bridge. + clt = CompiledLoopToken(self.cpu, looptoken.number) looptoken.compiled_loop_token = clt if not we_are_translated(): @@ -334,14 +339,17 @@ p = rffi.cast(rffi.INTP, addr) p[0] = rffi.cast(rffi.INT, relative_target) - def materialize_loop(self, looptoken): - self.datablockwrapper.done() # finish using cpu.asmmemmgr - self.datablockwrapper = None + def get_asmmemmgr_blocks(self, looptoken): clt = looptoken.compiled_loop_token if clt.asmmemmgr_blocks is None: clt.asmmemmgr_blocks = [] - return self.mc.materialize(self.cpu.asmmemmgr, - clt.asmmemmgr_blocks, + return clt.asmmemmgr_blocks + + def materialize_loop(self, looptoken): + self.datablockwrapper.done() # finish using cpu.asmmemmgr + self.datablockwrapper = None + allblocks = self.get_asmmemmgr_blocks(looptoken) + return self.mc.materialize(self.cpu.asmmemmgr, allblocks, self.cpu.gc_ll_descr.gcrootmap) def _find_debug_merge_point(self, operations): From david at codespeak.net Tue Nov 30 15:47:13 2010 From: david at codespeak.net (david at codespeak.net) Date: Tue, 30 Nov 2010 15:47:13 +0100 (CET) Subject: [pypy-svn] r79684 - pypy/branch/arm-backend/pypy/jit/backend/arm Message-ID: <20101130144713.5B9B850811@codespeak.net> Author: david Date: Tue Nov 30 15:47:11 2010 New Revision: 79684 Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Log: Exception and string copy operations Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/assembler.py Tue Nov 30 15:47:11 2010 @@ -5,7 +5,7 @@ from pypy.jit.backend.arm.codebuilder import ARMv7Builder, ARMv7InMemoryBuilder from pypy.jit.backend.arm.regalloc import ARMRegisterManager, ARMFrameManager from pypy.jit.backend.llsupport.regalloc import compute_vars_longevity, TempBox -from pypy.jit.metainterp.history import (ConstInt, BoxInt, BasicFailDescr, +from pypy.jit.metainterp.history import (Const, ConstInt, BoxInt, BasicFailDescr, INT, REF, FLOAT) from pypy.jit.metainterp.resoperation import rop from pypy.rlib import rgc @@ -16,6 +16,9 @@ # XXX Move to llsupport from pypy.jit.backend.x86.support import values_array +memcpy_fn = rffi.llexternal('memcpy', [llmemory.Address, llmemory.Address, + rffi.SIZE_T], lltype.Void, + sandboxsafe=True, _nowrapper=True) class AssemblerARM(ResOpAssembler): @@ -29,6 +32,7 @@ self.malloc_array_func_addr = 0 self.malloc_str_func_addr = 0 self.malloc_unicode_func_addr = 0 + self.memcpy_addr = 0 def setup(self): if self.mc is None: @@ -55,6 +59,7 @@ ll_new_unicode = gc_ll_descr.get_funcptr_for_newunicode() self.malloc_unicode_func_addr = rffi.cast(lltype.Signed, ll_new_unicode) + self.memcpy_addr = self.cpu.cast_ptr_to_int(memcpy_fn) def setup_failure_recovery(self): @@ -439,6 +444,20 @@ return arg.getint() <= size and lower_bound return False + def _ensure_value_is_boxed(self, thing, regalloc): + box = None + loc = None + if isinstance(thing, Const): + box = TempBox() + loc = regalloc.force_allocate_reg(box) + imm = regalloc.convert_to_imm(thing) + self.mc.gen_load_int(loc.value, imm.getint()) + else: + loc = regalloc.make_sure_var_in_reg(thing, imm_fine=False) + box = thing + return loc, box + + def _ensure_result_bit_extension(self, resloc, size, signed, regalloc): if size == 4: Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/opassembler.py Tue Nov 30 15:47:11 2010 @@ -180,15 +180,22 @@ _mixin_ = True - guard_size = ARMv7Builder.size_of_gen_load_int + 2*WORD - def _emit_guard(self, op, regalloc, fcond): + guard_size = ARMv7Builder.size_of_gen_load_int + 6*WORD + def _emit_guard(self, op, regalloc, fcond, save_exc=False): descr = op.getdescr() assert isinstance(descr, BasicFailDescr) #if hasattr(op, 'getfailargs'): # print 'Failargs: ', op.getfailargs() + self.mc.ensure_can_fit(self.guard_size) self.mc.ADD_ri(r.pc.value, r.pc.value, self.guard_size, cond=fcond) descr._arm_guard_code = self.mc.curraddr() + + self.mc.PUSH([reg.value for reg in r.caller_resp]) + addr = self.cpu.get_on_leave_jitted_int(save_exception=save_exc) + self.mc.BL(addr) + self.mc.POP([reg.value for reg in r.caller_resp]) + memaddr = self._gen_path_to_exit_path(op, op.getfailargs(), regalloc) descr._failure_recovery_code = memaddr regalloc.possibly_free_vars_for_op(op) @@ -358,6 +365,47 @@ regalloc.possibly_free_var(op.result) return fcond + def emit_op_cond_call_gc_wb(self, op, regalloc, fcond): + #XXX implement once gc support is in place + return fcond + + def emit_op_guard_no_exception(self, op, regalloc, fcond): + t = TempBox() + loc = regalloc.force_allocate_reg(t) + self.mc.gen_load_int(loc.value, self.cpu.pos_exception(), fcond) + self.mc.LDR_ri(loc.value, loc.value) + self.mc.CMP_ri(loc.value, 0) + self._emit_guard(op, regalloc, c.EQ, save_exc=True) + regalloc.possibly_free_var(t) + + def emit_op_guard_exception(self, op, regalloc, fcond): + args = op.getarglist() + t = TempBox() + t1 = TempBox() + loc = regalloc.force_allocate_reg(t, args) + loc1 = regalloc.force_allocate_reg(t1, args + [t]) + self.mc.gen_load_int(loc.value, + self.cpu.cast_adr_to_int( + op.getarg(0).getint()), fcond) + + self.mc.gen_load_int(loc1.value, self.cpu.pos_exception(), fcond) + self.mc.LDR_ri(loc1.value, loc1.value) + + self.mc.CMP_rr(loc1.value, loc.value) + self._emit_guard(op, regalloc, c.EQ, save_exc=True) + self.mc.gen_load_int(loc1.value, self.cpu.pos_exc_value(), fcond) + if op.result in regalloc.longevity: + resloc = regalloc.force_allocate_reg(op.result, args + [t, t1]) + self.mc.LDR_ri(resloc.value, loc1.value) + regalloc.possibly_free_var(resloc) + self.mc.gen_load_int(loc.value, self.cpu.pos_exception(), fcond) + self.mc.MOV_ri(r.ip.value, 0) + self.mc.STR_ri(r.ip.value, loc.value) + self.mc.STR_ri(r.ip.value, loc1.value) + regalloc.possibly_free_var(t) + regalloc.possibly_free_var(t1) + return fcond + class FieldOpAssembler(object): _mixin_ = True @@ -573,6 +621,106 @@ self.mc.STRB_ri(value_loc.value, temp.value, basesize, cond=fcond) return fcond + #from ../x86/regalloc.py:928 ff. + def emit_op_copystrcontent(self, op, regalloc, fcond): + self._emit_copystrcontent(op, regalloc, fcond, is_unicode=False) + + def emit_op_copyunicodecontent(self, op, regalloc, fcond): + self._emit_copystrcontent(op, regalloc, fcond, is_unicode=True) + + def _emit_copystrcontent(self, op, regalloc, fcond, is_unicode): + # compute the source address + args = op.getarglist() + boxes = [] + base_loc, box = self._ensure_value_is_boxed(args[0], regalloc) + boxes.append(box) + ofs_loc, box = self._ensure_value_is_boxed(args[2], regalloc) + boxes.append(box) + assert args[0] is not args[1] # forbidden case of aliasing + + srcaddr_box = TempBox() + forbidden_vars = [args[1], args[3], args[4], srcaddr_box] + srcaddr_loc = regalloc.force_allocate_reg(srcaddr_box, forbidden_vars) + self._gen_address_inside_string(base_loc, ofs_loc, srcaddr_loc, + is_unicode=is_unicode) + regalloc.possibly_free_var(boxes[0]) + if args[3] is not args[2] is not args[4]: # MESS MESS MESS: don't free + regalloc.possibly_free_var(boxes[1]) # it if ==args[3] or args[4] + + # compute the destination address + base_loc, box = self._ensure_value_is_boxed(args[1], regalloc) + boxes.append(box) + ofs_loc, box = self._ensure_value_is_boxed(args[3], regalloc) + boxes.append(box) + assert base_loc.is_reg() + assert ofs_loc.is_reg() + forbidden_vars = [args[4], srcaddr_box] + dstaddr_box = TempBox() + dstaddr_loc = regalloc.force_allocate_reg(dstaddr_box, + forbidden_vars) + self._gen_address_inside_string(base_loc, ofs_loc, dstaddr_loc, + is_unicode=is_unicode) + regalloc.possibly_free_var(boxes[2]) + if args[3] is not args[4]: # more of the MESS described above + regalloc.possibly_free_var(boxes[3]) + + # compute the length in bytes + length_loc, length_box = self._ensure_value_is_boxed(args[4], regalloc) + boxes.append(length_box) + if is_unicode: + forbidden_vars = [srcaddr_box, dstaddr_box] + bytes_box = TempBox() + bytes_loc = regalloc.force_allocate_reg(bytes_box, forbidden_vars) + scale = self._get_unicode_item_scale() + self.mc.MOV_rr(bytes_loc.value, length_loc.value) + self._load_address(length_loc, 0, scale, bytes_loc) + length_box = bytes_box + length_loc = bytes_loc + # call memcpy() + self._emit_call(self.memcpy_addr, [dstaddr_box, srcaddr_box, length_box], regalloc) + + regalloc.possibly_free_var(length_box) + regalloc.possibly_free_var(dstaddr_box) + regalloc.possibly_free_var(srcaddr_box) + regalloc.possibly_free_vars_for_op(op) + regalloc.possibly_free_vars(boxes) + + def _load_address(self, sizereg, baseofs, scale, result, baseloc=None): + if baseloc is not None: + assert baseloc.is_reg() + self.mc.MOV_rr(result.value, baseloc.value) + else: + self.mc.MOV_ri(result.value, 0) + assert sizereg.is_reg() + if scale > 0: + self.mc.LSL_ri(r.ip.value, sizereg.value, scale) + else: + self.mc.MOV_rr(r.ip.value, sizereg.value) + self.mc.ADD_rr(result.value, result.value, r.ip.value) + self.mc.ADD_ri(result.value, result.value, baseofs) + + def _gen_address_inside_string(self, baseloc, ofsloc, resloc, is_unicode): + cpu = self.cpu + if is_unicode: + ofs_items, _, _ = symbolic.get_array_token(rstr.UNICODE, + self.cpu.translate_support_code) + scale = self._get_unicode_item_scale() + else: + ofs_items, itemsize, _ = symbolic.get_array_token(rstr.STR, + self.cpu.translate_support_code) + assert itemsize == 1 + scale = 0 + self._load_address(ofsloc, ofs_items, scale, resloc, baseloc) + + def _get_unicode_item_scale(self): + _, itemsize, _ = symbolic.get_array_token(rstr.UNICODE, + self.cpu.translate_support_code) + if itemsize == 4: + return 2 + elif itemsize == 2: + return 1 + else: + raise AssertionError("bad unicode item size") class UnicodeOpAssembler(object): Modified: pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py ============================================================================== --- pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py (original) +++ pypy/branch/arm-backend/pypy/jit/backend/arm/runner.py Tue Nov 30 15:47:11 2010 @@ -52,17 +52,16 @@ return self.get_fail_descr_from_number(fail_index) def _execute_call(self, func): - #prev_interpreter = LLInterpreter.current_interpreter - #LLInterpreter.current_interpreter = self.debug_ll_interpreter + prev_interpreter = LLInterpreter.current_interpreter + LLInterpreter.current_interpreter = self.debug_ll_interpreter res = 0 - #try: - res = func() - #finally: - # LLInterpreter.current_interpreter = prev_interpreter + try: + res = func() + finally: + LLInterpreter.current_interpreter = prev_interpreter return res - @staticmethod - def cast_ptr_to_int(x): + def cast_ptr_to_int(self, x): adr = llmemory.cast_ptr_to_adr(x) return self.cast_adr_to_int(adr) From antocuni at codespeak.net Tue Nov 30 15:56:24 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 30 Nov 2010 15:56:24 +0100 (CET) Subject: [pypy-svn] r79685 - pypy/branch/jitypes2/pypy/module/pypyjit/test Message-ID: <20101130145624.8C34D282BE9@codespeak.net> Author: antocuni Date: Tue Nov 30 15:56:22 2010 New Revision: 79685 Modified: pypy/branch/jitypes2/pypy/module/pypyjit/test/test_pypy_c.py Log: write a test to check that a call through ctypes is fast, and also generally improve a bit the machinery in test_pypy_c Modified: pypy/branch/jitypes2/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/branch/jitypes2/pypy/module/pypyjit/test/test_pypy_c.py Tue Nov 30 15:56:22 2010 @@ -10,9 +10,9 @@ if op.getopname().startswith(prefix)] def __repr__(self): - return "%s%s" % (self.bytecode, list.__repr__(self)) + return "%s%s" % (self.opcode, list.__repr__(self)) -ZERO_OP_BYTECODES = [ +ZERO_OP_OPCODES = [ 'POP_TOP', 'ROT_TWO', 'ROT_THREE', @@ -82,11 +82,13 @@ def run_source(self, source, expected_max_ops, *testcases, **kwds): assert isinstance(expected_max_ops, int) threshold = kwds.pop('threshold', 3) + filter_loops = kwds.pop('filter_loops', False) # keep only the loops beginning from case%d.py if kwds: raise TypeError, 'Unsupported keyword arguments: %s' % kwds.keys() source = py.code.Source(source) filepath = self.tmpdir.join('case%d.py' % self.counter) logfilepath = filepath.new(ext='.log') + self.logfilepath = logfilepath self.__class__.counter += 1 f = filepath.open('w') print >> f, source @@ -125,7 +127,7 @@ if result.strip().startswith('SKIP:'): py.test.skip(result.strip()) assert result.splitlines()[-1].strip() == 'OK :-)' - self.parse_loops(logfilepath) + self.parse_loops(logfilepath, filepath, filter_loops) self.print_loops() print logfilepath if self.total_ops > expected_max_ops: @@ -133,7 +135,7 @@ self.total_ops, expected_max_ops) return result - def parse_loops(self, opslogfile): + def parse_loops(self, opslogfile, filepath, filter_loops): from pypy.jit.tool.oparser import parse from pypy.tool import logparser assert opslogfile.check() @@ -143,27 +145,49 @@ if not from_entry_bridge(part, parts)] # skip entry bridges, they can contain random things self.loops = [parse(part, no_namespace=True) for part in self.rawloops] - self.sliced_loops = [] # contains all bytecodes of all loops + if filter_loops: + self.loops = self.filter_loops(filepath, self.loops) + self.all_bytecodes = [] # contains all bytecodes of all loops + self.bytecode_by_loop = {} # contains all bytecodes divided by loops self.total_ops = 0 for loop in self.loops: + loop_bytecodes = [] + self.bytecode_by_loop[loop] = loop_bytecodes self.total_ops += len(loop.operations) for op in loop.operations: if op.getopname() == "debug_merge_point": - sliced_loop = BytecodeTrace() - sliced_loop.bytecode = op.getarg(0)._get_str().rsplit(" ", 1)[1] - self.sliced_loops.append(sliced_loop) + bytecode = BytecodeTrace() + bytecode.opcode = op.getarg(0)._get_str().rsplit(" ", 1)[1] + bytecode.debug_merge_point = op + loop_bytecodes.append(bytecode) + self.all_bytecodes.append(bytecode) else: - sliced_loop.append(op) + bytecode.append(op) self.check_0_op_bytecodes() + def filter_loops(self, filepath, loops): + newloops = [] + for loop in loops: + op = loop.operations[0] + # if the first op is not debug_merge_point, it's a bridge: for + # now, we always include them + if (op.getopname() != 'debug_merge_point' or + str(filepath) in str(op.getarg(0))): + newloops.append(loop) + return newloops + def check_0_op_bytecodes(self): - for bytecodetrace in self.sliced_loops: - if bytecodetrace.bytecode not in ZERO_OP_BYTECODES: + for bytecodetrace in self.all_bytecodes: + if bytecodetrace.opcode not in ZERO_OP_OPCODES: continue assert not bytecodetrace - def get_by_bytecode(self, name): - return [ops for ops in self.sliced_loops if ops.bytecode == name] + def get_by_bytecode(self, name, loop=None): + if loop: + bytecodes = self.bytecode_by_loop[loop] + else: + bytecodes = self.all_bytecodes + return [ops for ops in bytecodes if ops.opcode == name] def print_loops(self): for rawloop in self.rawloops: @@ -1232,6 +1256,54 @@ assert call.getarg(1).value == 2.0 assert call.getarg(2).value == 3.0 + def test_ctypes_call(self): + from pypy.rlib.test.test_libffi import get_libm_name + libm_name = get_libm_name(sys.platform) + out = self.run_source(''' + def main(): + import ctypes + libm = ctypes.CDLL('%(libm_name)s') + fabs = libm.fabs + fabs.argtypes = [ctypes.c_double] + fabs.restype = ctypes.c_double + x = -4 + for i in range(2000): + x = x + 0 # convince the perfect spec. to make x virtual + x = fabs(x) + x = x - 100 + print fabs._ptr.getaddr() + return x + ''' % locals(), + 10000, ([], -4.0), + threshold=1000, + filter_loops=True) + fabs_addr = int(out.splitlines()[0]) + assert len(self.loops) == 2 # the first is the loop, the second is a bridge + loop = self.loops[0] + call_functions = self.get_by_bytecode('CALL_FUNCTION', loop) + assert len(call_functions) == 2 + # + # this is the call "fabs(x)" + call_main = call_functions[0] + assert 'code object main' in str(call_main.debug_merge_point) + assert call_main.get_opnames('call') == ['call'] # this is call(getexecutioncontext) + # + # this is the ffi call inside ctypes + call_ffi = call_functions[1] + last_ops = [op.getopname() for op in call_ffi[-6:]] + assert last_ops == ['force_token', + 'setfield_gc', # framestackdepth + 'setfield_gc', # vable_token + 'call_may_force', + 'guard_not_forced', + 'guard_no_exception'] + call = call_ffi[-3] + assert call.getarg(0).value == fabs_addr + # + # finally, check that we don't force anything + for op in loop.operations: + assert op.getopname() != 'new_with_vtable' + # test_circular class AppTestJIT(PyPyCJITTests): From cfbolz at codespeak.net Tue Nov 30 16:10:36 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 30 Nov 2010 16:10:36 +0100 (CET) Subject: [pypy-svn] r79686 - pypy/trunk/pypy/module/pypyjit/test Message-ID: <20101130151036.DD83B282BE0@codespeak.net> Author: cfbolz Date: Tue Nov 30 16:10:35 2010 New Revision: 79686 Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Log: test I would like to pass Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Tue Nov 30 16:10:35 2010 @@ -630,6 +630,28 @@ ''', 3000, ([0], 2000*3)) assert len(self.loops) == 1 + def test_getattr_with_dynamic_attribute(self): + py.test.skip("fix this test") + self.run_source(''' + class A(object): + pass + + l = ["x", "y"] + + def main(arg): + sum = 0 + a = A() + a.x = 1 + a.y = 2 + i = 0 + while i < 2000: + name = l[i % 2] + sum += getattr(a, name) + i += 1 + return sum + ''', 3000, ([0], 3000)) + assert len(self.loops) == 1 + def test_blockstack_virtualizable(self): self.run_source(''' from pypyjit import residual_call From afa at codespeak.net Tue Nov 30 16:18:00 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 30 Nov 2010 16:18:00 +0100 (CET) Subject: [pypy-svn] r79687 - in pypy/branch/fast-forward/pypy: module/_hashlib module/_multiprocessing rlib Message-ID: <20101130151800.2A250282BE9@codespeak.net> Author: afa Date: Tue Nov 30 16:17:59 2010 New Revision: 79687 Modified: pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py pypy/branch/fast-forward/pypy/rlib/rwinreg.py Log: Help compilation with cpython2.5 Modified: pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py (original) +++ pypy/branch/fast-forward/pypy/module/_hashlib/interp_hashlib.py Tue Nov 30 16:17:59 2010 @@ -1,3 +1,4 @@ +from __future__ import with_statement from pypy.interpreter.gateway import unwrap_spec, interp2app from pypy.interpreter.typedef import TypeDef, GetSetProperty from pypy.interpreter.error import OperationError Modified: pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py (original) +++ pypy/branch/fast-forward/pypy/module/_multiprocessing/interp_connection.py Tue Nov 30 16:17:59 2010 @@ -1,3 +1,4 @@ +from __future__ import with_statement from pypy.interpreter.baseobjspace import ObjSpace, Wrappable, W_Root from pypy.interpreter.typedef import TypeDef, GetSetProperty from pypy.interpreter.gateway import interp2app, unwrap_spec Modified: pypy/branch/fast-forward/pypy/rlib/rwinreg.py ============================================================================== --- pypy/branch/fast-forward/pypy/rlib/rwinreg.py (original) +++ pypy/branch/fast-forward/pypy/rlib/rwinreg.py Tue Nov 30 16:17:59 2010 @@ -1,3 +1,4 @@ +from __future__ import with_statement from pypy.rpython.lltypesystem import lltype, rffi from pypy.rpython.tool import rffi_platform as platform from pypy.translator.tool.cbuild import ExternalCompilationInfo From arigo at codespeak.net Tue Nov 30 17:02:30 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 17:02:30 +0100 (CET) Subject: [pypy-svn] r79688 - in pypy/trunk/pypy: annotation annotation/test interpreter jit/codewriter jit/codewriter/test jit/metainterp jit/metainterp/test jit/tl jit/tl/spli module/cpyext module/cpyext/include module/cpyext/test module/pypyjit/test objspace/std rlib rlib/rsre rlib/rsre/test rlib/test rpython rpython/test Message-ID: <20101130160230.1A0C3282BE3@codespeak.net> Author: arigo Date: Tue Nov 30 17:02:26 2010 New Revision: 79688 Modified: pypy/trunk/pypy/annotation/description.py pypy/trunk/pypy/annotation/listdef.py pypy/trunk/pypy/annotation/test/test_annrpython.py pypy/trunk/pypy/annotation/unaryop.py pypy/trunk/pypy/interpreter/pycode.py pypy/trunk/pypy/jit/codewriter/jtransform.py pypy/trunk/pypy/jit/codewriter/test/test_list.py pypy/trunk/pypy/jit/codewriter/test/test_void_list.py pypy/trunk/pypy/jit/metainterp/pyjitpl.py pypy/trunk/pypy/jit/metainterp/test/test_list.py pypy/trunk/pypy/jit/metainterp/test/test_recursive.py pypy/trunk/pypy/jit/metainterp/test/test_virtualizable.py pypy/trunk/pypy/jit/metainterp/warmspot.py pypy/trunk/pypy/jit/tl/spli/interpreter.py pypy/trunk/pypy/jit/tl/tl.py pypy/trunk/pypy/module/cpyext/api.py pypy/trunk/pypy/module/cpyext/include/tupleobject.h pypy/trunk/pypy/module/cpyext/slotdefs.py pypy/trunk/pypy/module/cpyext/test/test_cpyext.py pypy/trunk/pypy/module/cpyext/test/test_tupleobject.py pypy/trunk/pypy/module/cpyext/tupleobject.py pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py pypy/trunk/pypy/objspace/std/objspace.py pypy/trunk/pypy/rlib/debug.py pypy/trunk/pypy/rlib/rsre/rsre_core.py pypy/trunk/pypy/rlib/rsre/test/test_zjit.py pypy/trunk/pypy/rlib/test/test_debug.py pypy/trunk/pypy/rpython/rlist.py pypy/trunk/pypy/rpython/test/test_rlist.py Log: Merge branch/rlist-jit. Improve in jtransform.py the handling of constructs like 'frame.locals[n]'. This is now done with the help of the annotator: from the hint _immutable_fields_=['locals[*]'] found on the class, the annotator flags the list returned by 'frame.locals' as being an immutable list; then the rtyper translates 'frame.locals[n]' with a call to ll_getitem_foldable instead of ll_getitem. The goal is to make an extra test in test_list.py pass: handling IndexErrors out of RPython-level lists. This is done essentially by letting the JIT see the code "if n>=len: raise IndexError" in rlist.py. Other changes (related): * fixes left and right because _immutable_fields_=['lst[*]'] now implies that we only read from the list using a known- nonnegative index (checked in jtransform.py) * cpyext refactoring about tuples Modified: pypy/trunk/pypy/annotation/description.py ============================================================================== --- pypy/trunk/pypy/annotation/description.py (original) +++ pypy/trunk/pypy/annotation/description.py Tue Nov 30 17:02:26 2010 @@ -636,6 +636,24 @@ return self return None + def maybe_return_immutable_list(self, attr, s_result): + # hack: 'x.lst' where lst is listed in _immutable_fields_ as 'lst[*]' + # should really return an immutable list as a result. Implemented + # by changing the result's annotation (but not, of course, doing an + # actual copy in the rtyper). Tested in pypy.rpython.test.test_rlist, + # test_immutable_list_out_of_instance. + search = '%s[*]' % (attr,) + cdesc = self + while cdesc is not None: + if '_immutable_fields_' in cdesc.classdict: + if search in cdesc.classdict['_immutable_fields_'].value: + s_result.listdef.never_resize() + s_copy = s_result.listdef.offspring() + s_copy.listdef.mark_as_immutable() + return s_copy + cdesc = cdesc.basedesc + return s_result # common case + def consider_call_site(bookkeeper, family, descs, args, s_result): from pypy.annotation.model import SomeInstance, SomePBC, s_None if len(descs) == 1: Modified: pypy/trunk/pypy/annotation/listdef.py ============================================================================== --- pypy/trunk/pypy/annotation/listdef.py (original) +++ pypy/trunk/pypy/annotation/listdef.py Tue Nov 30 17:02:26 2010 @@ -6,11 +6,16 @@ class TooLateForChange(Exception): pass +class ListChangeUnallowed(Exception): + pass + class ListItem(object): mutated = False # True for lists mutated after creation resized = False # True for lists resized after creation range_step = None # the step -- only for lists only created by a range() dont_change_any_more = False # set to True when too late for changes + immutable = False # for getattr out of _immutable_fields_ = ['attr[*]'] + must_not_resize = False # make_sure_not_resized() # what to do if range_step is different in merge. # - if one is a list (range_step is None), unify to a list. @@ -26,7 +31,6 @@ self.bookkeeper = bookkeeper self.itemof = {} # set of all ListDefs using this ListItem self.read_locations = {} - self.dont_resize = False if bookkeeper is None: self.dont_change_any_more = True @@ -34,12 +38,15 @@ if not self.mutated: if self.dont_change_any_more: raise TooLateForChange + self.immutable = False self.mutated = True def resize(self): if not self.resized: - if self.dont_change_any_more or self.dont_resize: + if self.dont_change_any_more: raise TooLateForChange + if self.must_not_resize: + raise ListChangeUnallowed("resizing list") self.resized = True def setrangestep(self, step): @@ -63,11 +70,13 @@ # things more general self, other = other, self - if other.dont_resize: - if self.resized: - raise TooLateForChange() - self.dont_resize = True - if other.mutated: self.mutate() + self.immutable &= other.immutable + if other.must_not_resize: + if self.resized: + raise ListChangeUnallowed("list merge with a resized") + self.must_not_resize = True + if other.mutated: + self.mutate() if other.resized: self.resize() if other.range_step != self.range_step: @@ -176,9 +185,11 @@ self.listitem.generalize(s_value) def __repr__(self): - return '<[%r]%s%s>' % (self.listitem.s_value, + return '<[%r]%s%s%s%s>' % (self.listitem.s_value, self.listitem.mutated and 'm' or '', - self.listitem.resized and 'r' or '') + self.listitem.resized and 'r' or '', + self.listitem.immutable and 'I' or '', + self.listitem.must_not_resize and '!R' or '') def mutate(self): self.listitem.mutate() @@ -189,13 +200,20 @@ def never_resize(self): if self.listitem.resized: - raise TooLateForChange() - self.listitem.dont_resize = True + raise ListChangeUnallowed("list already resized") + self.listitem.must_not_resize = True - def never_mutate(self): - if self.listitem.resized or self.listitem.mutated: - raise TooLateForChange() - self.listitem.dont_change_any_more = True + def mark_as_immutable(self): + # Sets the 'immutable' flag. Note that unlike "never resized", + # the immutable flag is only a hint. It is cleared again e.g. + # when we merge with a "normal" list that doesn't have it. It + # is thus expected to live only shortly, mostly for the case + # of writing 'x.list[n]'. + self.never_resize() + if not self.listitem.mutated: + self.listitem.immutable = True + #else: it's fine, don't set immutable=True at all (see + # test_can_merge_immutable_list_with_regular_list) MOST_GENERAL_LISTDEF = ListDef(None, SomeObject()) Modified: pypy/trunk/pypy/annotation/test/test_annrpython.py ============================================================================== --- pypy/trunk/pypy/annotation/test/test_annrpython.py (original) +++ pypy/trunk/pypy/annotation/test/test_annrpython.py Tue Nov 30 17:02:26 2010 @@ -10,7 +10,7 @@ from pypy.translator.translator import graphof as tgraphof from pypy.annotation import policy from pypy.annotation import specialize -from pypy.annotation.listdef import ListDef, TooLateForChange +from pypy.annotation.listdef import ListDef, ListChangeUnallowed from pypy.annotation.dictdef import DictDef from pypy.objspace.flow.model import * from pypy.rlib.rarithmetic import r_uint, base_int, r_longlong, r_ulonglong @@ -3206,7 +3206,7 @@ l.append(4) a = self.RPythonAnnotator() - py.test.raises(TooLateForChange, a.build_types, g, []) + py.test.raises(ListChangeUnallowed, a.build_types, g, []) assert called def test_listitem_no_mutating2(self): @@ -3229,7 +3229,7 @@ a = self.RPythonAnnotator() a.translator.config.translation.list_comprehension_operations = True - py.test.raises(TooLateForChange, a.build_types, fn, [int]) + py.test.raises(ListChangeUnallowed, a.build_types, fn, [int]) def test_listitem_never_resize(self): from pypy.rlib.debug import check_annotation @@ -3243,7 +3243,7 @@ check_annotation(l, checker) a = self.RPythonAnnotator() - py.test.raises(TooLateForChange, a.build_types, f, []) + py.test.raises(ListChangeUnallowed, a.build_types, f, []) def test_len_of_empty_list(self): @@ -3357,6 +3357,87 @@ # not a constant: both __enter__ and __exit__ have been annotated assert not s.is_constant() + def test_make_sure_not_resized(self): + from pypy.rlib.debug import make_sure_not_resized + + def pycode(consts): + make_sure_not_resized(consts) + def build1(): + return pycode(consts=[1]) + def build2(): + return pycode(consts=[0]) + def fn(): + build1() + build2() + + a = self.RPythonAnnotator() + a.translator.config.translation.list_comprehension_operations = True + a.build_types(fn, []) + # assert did not raise ListChangeUnallowed + + def test_return_immutable_list(self): + class A: + _immutable_fields_ = 'lst[*]' + def f(n): + a = A() + l1 = [n, 0] + l1[1] = n+1 + a.lst = l1 + return a.lst + + a = self.RPythonAnnotator() + s = a.build_types(f, [int]) + assert s.listdef.listitem.immutable + + def test_immutable_list_is_actually_resized(self): + class A: + _immutable_fields_ = 'lst[*]' + def f(n): + a = A() + l1 = [n] + l1.append(n+1) + a.lst = l1 + return a.lst + + a = self.RPythonAnnotator() + py.test.raises(ListChangeUnallowed, a.build_types, f, [int]) + + def test_can_merge_immutable_list_with_regular_list(self): + class A: + _immutable_fields_ = 'lst[*]' + def foo(lst): + pass + + def f(n): + a = A() + l1 = [n, 0] + l1[1] = n+1 + a.lst = l1 + if n > 0: + foo(a.lst) + else: + lst = [0] + lst[0] = n + foo(lst) + + a = self.RPythonAnnotator() + a.build_types(f, [int]) + + def f(n): + a = A() + l1 = [n, 0] + l1[1] = n+1 + a.lst = l1 + if n > 0: + lst = [0] + lst[0] = n + foo(lst) + else: + foo(a.lst) + + a = self.RPythonAnnotator() + a.build_types(f, [int]) + def g(n): return [0,1,2,n] Modified: pypy/trunk/pypy/annotation/unaryop.py ============================================================================== --- pypy/trunk/pypy/annotation/unaryop.py (original) +++ pypy/trunk/pypy/annotation/unaryop.py Tue Nov 30 17:02:26 2010 @@ -615,6 +615,9 @@ if basedef.classdesc.all_enforced_attrs is not None: if attr in basedef.classdesc.all_enforced_attrs: raise HarmlesslyBlocked("get enforced attr") + elif isinstance(s_result, SomeList): + s_result = ins.classdef.classdesc.maybe_return_immutable_list( + attr, s_result) return s_result return SomeObject() getattr.can_only_throw = [] Modified: pypy/trunk/pypy/interpreter/pycode.py ============================================================================== --- pypy/trunk/pypy/interpreter/pycode.py (original) +++ pypy/trunk/pypy/interpreter/pycode.py Tue Nov 30 17:02:26 2010 @@ -15,7 +15,7 @@ CO_OPTIMIZED, CO_NEWLOCALS, CO_VARARGS, CO_VARKEYWORDS, CO_NESTED, CO_GENERATOR, CO_CONTAINSGLOBALS) from pypy.rlib.rarithmetic import intmask -from pypy.rlib.debug import make_sure_not_resized, make_sure_not_modified +from pypy.rlib.debug import make_sure_not_resized from pypy.rlib import jit from pypy.rlib.objectmodel import compute_hash from pypy.tool.stdlib_opcode import opcodedesc, HAVE_ARGUMENT @@ -69,7 +69,7 @@ self.co_stacksize = stacksize self.co_flags = flags self.co_code = code - self.co_consts_w = make_sure_not_modified(consts) + self.co_consts_w = consts self.co_names_w = [space.new_interned_str(aname) for aname in names] self.co_varnames = varnames self.co_freevars = freevars @@ -269,7 +269,7 @@ dis.dis(co) def fget_co_consts(space, self): - return space.newtuple(self.co_consts_w[:]) + return space.newtuple(self.co_consts_w) def fget_co_names(space, self): return space.newtuple(self.co_names_w) @@ -383,7 +383,7 @@ w(self.co_stacksize), w(self.co_flags), w(self.co_code), - space.newtuple(self.co_consts_w[:]), + space.newtuple(self.co_consts_w), space.newtuple(self.co_names_w), space.newtuple([w(v) for v in self.co_varnames]), w(self.co_filename), Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/jtransform.py (original) +++ pypy/trunk/pypy/jit/codewriter/jtransform.py Tue Nov 30 17:02:26 2010 @@ -24,6 +24,7 @@ class Transformer(object): + vable_array_vars = None def __init__(self, cpu=None, callcontrol=None, portal_jd=None): self.cpu = cpu @@ -38,7 +39,6 @@ def optimize_block(self, block): if block.operations == (): return - self.immutable_arrays = {} self.vable_array_vars = {} self.vable_flags = {} renamings = {} @@ -83,6 +83,7 @@ self.follow_constant_exit(block) self.optimize_goto_if_not(block) for link in block.exits: + self._check_no_vable_array(link.args) self._do_renaming_on_link(renamings, link) def _do_renaming(self, rename, op): @@ -100,6 +101,28 @@ op.args[i] = ListOfKind(v.kind, newlst) return op + def _check_no_vable_array(self, list): + if not self.vable_array_vars: + return + for v in list: + if v in self.vable_array_vars: + raise AssertionError( + "A virtualizable array is passed around; it should\n" + "only be used immediately after being read. Note\n" + "that a possible cause is indexing with an index not\n" + "known non-negative, or catching IndexError, or\n" + "not inlining at all (for tests: use listops=True).\n" + "Occurred in: %r" % self.graph) + # extra expanation: with the way things are organized in + # rpython/rlist.py, the ll_getitem becomes a function call + # that is typically meant to be inlined by the JIT, but + # this does not work with vable arrays because + # jtransform.py expects the getfield and the getarrayitem + # to be in the same basic block. It works a bit as a hack + # for simple cases where we performed the backendopt + # inlining before (even with a very low threshold, because + # there is _always_inline_ on the relevant functions). + def _do_renaming_on_link(self, rename, link): for i, v in enumerate(link.args): if isinstance(v, Variable): @@ -170,9 +193,12 @@ else: return rewrite(self, op) - def rewrite_op_same_as(self, op): pass - def rewrite_op_cast_pointer(self, op): pass - def rewrite_op_cast_opaque_ptr(self, op): pass # rlib.rerased + def rewrite_op_same_as(self, op): + if op.args[0] in self.vable_array_vars: + self.vable_array_vars[op.result]= self.vable_array_vars[op.args[0]] + + rewrite_op_cast_pointer = rewrite_op_same_as + rewrite_op_cast_pointer = rewrite_op_same_as # rlib.rerased def rewrite_op_cast_primitive(self, op): pass def rewrite_op_cast_bool_to_int(self, op): pass def rewrite_op_cast_bool_to_uint(self, op): pass @@ -257,6 +283,7 @@ The name is one of '{residual,direct}_call_{r,ir,irf}_{i,r,f,v}'.""" if args is None: args = op.args[1:] + self._check_no_vable_array(args) lst_i, lst_r, lst_f = self.make_three_lists(args) reskind = getkind(op.result.concretetype)[0] if lst_f or reskind == 'f': kinds = 'irf' @@ -525,17 +552,14 @@ arrayfielddescr, arraydescr) return [] - # check for deepfrozen structures that force constant-folding - immut = v_inst.concretetype.TO._immutable_field(c_fieldname.value) - if immut: + # check for _immutable_fields_ hints + if v_inst.concretetype.TO._immutable_field(c_fieldname.value): if (self.callcontrol is not None and self.callcontrol.could_be_green_field(v_inst.concretetype.TO, c_fieldname.value)): pure = '_greenfield' else: pure = '_pure' - if immut == "[*]": - self.immutable_arrays[op.result] = True else: pure = '' argname = getattr(v_inst.concretetype.TO, '_gckind', 'gc') @@ -932,21 +956,19 @@ # base hints on the name of the ll function, which is a bit xxx-ish # but which is safe for now assert func.func_name.startswith('ll_') + # check that we have carefully placed the oopspec in + # pypy/rpython/rlist.py. There should not be an oopspec on + # a ll_getitem or ll_setitem that expects a 'func' argument. + # The idea is that a ll_getitem/ll_setitem with dum_checkidx + # should get inlined by the JIT, so that we see the potential + # 'raise IndexError'. + assert 'func' not in func.func_code.co_varnames non_negative = '_nonneg' in func.func_name fast = '_fast' in func.func_name - if fast: - can_raise = False - non_negative = True - else: - tag = op.args[1].value - assert tag in (rlist.dum_nocheck, rlist.dum_checkidx) - can_raise = tag != rlist.dum_nocheck - return non_negative, can_raise + return non_negative or fast def _prepare_list_getset(self, op, descr, args, checkname): - non_negative, can_raise = self._get_list_nonneg_canraise_flags(op) - if can_raise: - raise NotSupported("list operation can raise") + non_negative = self._get_list_nonneg_canraise_flags(op) if non_negative: return args[1], [] else: @@ -958,9 +980,8 @@ return v_posindex, [op0, op1] def _prepare_void_list_getset(self, op): - non_negative, can_raise = self._get_list_nonneg_canraise_flags(op) - if can_raise: - raise NotSupported("list operation can raise") + # sanity check: + self._get_list_nonneg_canraise_flags(op) def _get_initial_newlist_length(self, op, args): # normalize number of arguments to the 'newlist' function @@ -1005,7 +1026,7 @@ v_index, extraop = self._prepare_list_getset(op, arraydescr, args, 'check_neg_index') extra = getkind(op.result.concretetype)[0] - if pure or args[0] in self.immutable_arrays: + if pure: extra = 'pure_' + extra op = SpaceOperation('getarrayitem_gc_%s' % extra, [args[0], arraydescr, v_index], op.result) Modified: pypy/trunk/pypy/jit/codewriter/test/test_list.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/test/test_list.py (original) +++ pypy/trunk/pypy/jit/codewriter/test/test_list.py Tue Nov 30 17:02:26 2010 @@ -56,9 +56,8 @@ if '/' in oopspec_name: oopspec_name, property = oopspec_name.split('/') def force_flags(op): - if property == 'NONNEG': return True, False - if property == 'NEG': return False, False - if property == 'CANRAISE': return False, True + if property == 'NONNEG': return True + if property == 'NEG': return False raise ValueError(property) tr._get_list_nonneg_canraise_flags = force_flags op = SpaceOperation('direct_call', @@ -122,9 +121,6 @@ check_neg_index %r0, , %i0 -> %i1 getarrayitem_gc_i %r0, , %i1 -> %i2 """) - builtin_test('list.getitem/CANRAISE', - [varoftype(FIXEDLIST), varoftype(lltype.Signed)], - lltype.Signed, NotSupported) def test_fixed_getitem_foldable(): builtin_test('list.getitem_foldable/NONNEG', @@ -139,9 +135,6 @@ check_neg_index %r0, , %i0 -> %i1 getarrayitem_gc_pure_i %r0, , %i1 -> %i2 """) - builtin_test('list.getitem_foldable/CANRAISE', - [varoftype(FIXEDLIST), varoftype(lltype.Signed)], - lltype.Signed, NotSupported) def test_fixed_setitem(): builtin_test('list.setitem/NONNEG', [varoftype(FIXEDLIST), @@ -158,10 +151,6 @@ check_neg_index %r0, , %i0 -> %i1 setarrayitem_gc_i %r0, , %i1, %i2 """) - builtin_test('list.setitem/CANRAISE', [varoftype(FIXEDLIST), - varoftype(lltype.Signed), - varoftype(lltype.Signed)], - lltype.Void, NotSupported) def test_fixed_len(): builtin_test('list.len', [varoftype(FIXEDLIST)], lltype.Signed, @@ -206,9 +195,6 @@ check_resizable_neg_index %r0, , %i0 -> %i1 getlistitem_gc_i %r0, , , %i1 -> %i2 """) - builtin_test('list.getitem/CANRAISE', - [varoftype(VARLIST), varoftype(lltype.Signed)], - lltype.Signed, NotSupported) def test_resizable_setitem(): builtin_test('list.setitem/NONNEG', [varoftype(VARLIST), @@ -225,10 +211,6 @@ check_resizable_neg_index %r0, , %i0 -> %i1 setlistitem_gc_i %r0, , , %i1, %i2 """) - builtin_test('list.setitem/CANRAISE', [varoftype(VARLIST), - varoftype(lltype.Signed), - varoftype(lltype.Signed)], - lltype.Void, NotSupported) def test_resizable_len(): builtin_test('list.len', [varoftype(VARLIST)], lltype.Signed, Modified: pypy/trunk/pypy/jit/codewriter/test/test_void_list.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/test/test_void_list.py (original) +++ pypy/trunk/pypy/jit/codewriter/test/test_void_list.py Tue Nov 30 17:02:26 2010 @@ -51,9 +51,6 @@ builtin_test('list.getitem/NEG', [varoftype(FIXEDLIST), varoftype(lltype.Signed)], lltype.Void, "") - builtin_test('list.getitem/CANRAISE', - [varoftype(FIXEDLIST), varoftype(lltype.Signed)], - lltype.Void, NotSupported) def test_fixed_getitem_foldable(): builtin_test('list.getitem_foldable/NONNEG', @@ -62,9 +59,6 @@ builtin_test('list.getitem_foldable/NEG', [varoftype(FIXEDLIST), varoftype(lltype.Signed)], lltype.Void, "") - builtin_test('list.getitem_foldable/CANRAISE', - [varoftype(FIXEDLIST), varoftype(lltype.Signed)], - lltype.Void, NotSupported) def test_fixed_setitem(): builtin_test('list.setitem/NONNEG', [varoftype(FIXEDLIST), @@ -75,10 +69,6 @@ varoftype(lltype.Signed), varoftype(lltype.Void)], lltype.Void, "") - builtin_test('list.setitem/CANRAISE', [varoftype(FIXEDLIST), - varoftype(lltype.Signed), - varoftype(lltype.Void)], - lltype.Void, NotSupported) def test_fixed_len(): builtin_test('list.len', [varoftype(FIXEDLIST)], lltype.Signed, @@ -115,9 +105,6 @@ builtin_test('list.getitem/NEG', [varoftype(VARLIST), varoftype(lltype.Signed)], lltype.Void, "") - builtin_test('list.getitem/CANRAISE', - [varoftype(VARLIST), varoftype(lltype.Signed)], - lltype.Void, NotSupported) def test_resizable_setitem(): builtin_test('list.setitem/NONNEG', [varoftype(VARLIST), @@ -128,10 +115,6 @@ varoftype(lltype.Signed), varoftype(lltype.Void)], lltype.Void, "") - builtin_test('list.setitem/CANRAISE', [varoftype(VARLIST), - varoftype(lltype.Signed), - varoftype(lltype.Void)], - lltype.Void, NotSupported) def test_resizable_len(): builtin_test('list.len', [varoftype(VARLIST)], lltype.Signed, Modified: pypy/trunk/pypy/jit/metainterp/pyjitpl.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/pyjitpl.py (original) +++ pypy/trunk/pypy/jit/metainterp/pyjitpl.py Tue Nov 30 17:02:26 2010 @@ -612,8 +612,10 @@ virtualizable = vinfo.unwrap_virtualizable_box(virtualizable_box) arrayindex = vinfo.array_field_by_descrs[arrayfielddescr] index = indexbox.getint() - if index < 0: - index += vinfo.get_array_length(virtualizable, arrayindex) + # Support for negative index: disabled + # (see codewriter/jtransform.py, _check_no_vable_array). + #if index < 0: + # index += vinfo.get_array_length(virtualizable, arrayindex) assert 0 <= index < vinfo.get_array_length(virtualizable, arrayindex) return vinfo.get_index_in_array(virtualizable, arrayindex, index) Modified: pypy/trunk/pypy/jit/metainterp/test/test_list.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_list.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_list.py Tue Nov 30 17:02:26 2010 @@ -188,6 +188,26 @@ assert res == f(4) self.check_loops(call=0, getfield_gc=0) + def test_fold_indexerror(self): + jitdriver = JitDriver(greens = [], reds = ['total', 'n', 'lst']) + def f(n): + lst = [] + total = 0 + while n > 0: + jitdriver.can_enter_jit(lst=lst, n=n, total=total) + jitdriver.jit_merge_point(lst=lst, n=n, total=total) + lst.append(n) + try: + total += lst[n] + except IndexError: + total += 1000 + n -= 1 + return total + + res = self.meta_interp(f, [15], listops=True) + assert res == f(15) + self.check_loops(guard_exception=0) + class TestOOtype(ListTests, OOJitMixin): pass Modified: pypy/trunk/pypy/jit/metainterp/test/test_recursive.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_recursive.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_recursive.py Tue Nov 30 17:02:26 2010 @@ -927,12 +927,16 @@ x=x) frame.s = hint(frame.s, promote=True) n -= 1 - x += frame.l[frame.s] + s = frame.s + assert s >= 0 + x += frame.l[s] frame.s += 1 if codeno == 0: subframe = Frame([n, n+1, n+2, n+3], 0) x += f(1, 10, 1, subframe) - x += frame.l[frame.s] + s = frame.s + assert s >= 0 + x += frame.l[s] x += len(frame.l) frame.s -= 1 return x Modified: pypy/trunk/pypy/jit/metainterp/test/test_virtualizable.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/test/test_virtualizable.py (original) +++ pypy/trunk/pypy/jit/metainterp/test/test_virtualizable.py Tue Nov 30 17:02:26 2010 @@ -480,9 +480,13 @@ myjitdriver.jit_merge_point(frame=frame, n=n, x=x) frame.s = hint(frame.s, promote=True) n -= 1 - x += frame.l[frame.s] + s = frame.s + assert s >= 0 + x += frame.l[s] frame.s += 1 - x += frame.l[frame.s] + s = frame.s + assert s >= 0 + x += frame.l[s] x += len(frame.l) frame.s -= 1 return x @@ -994,7 +998,9 @@ jitdriver.can_enter_jit(frame=frame, n=n) jitdriver.jit_merge_point(frame=frame, n=n) popped = frame.stack[frame.stackpos] - frame.stackpos -= 1 + sp = frame.stackpos - 1 + assert sp >= 0 + frame.stackpos = sp to_push = intmask(popped * 3) frame.stack[frame.stackpos] = to_push frame.stackpos += 1 Modified: pypy/trunk/pypy/jit/metainterp/warmspot.py ============================================================================== --- pypy/trunk/pypy/jit/metainterp/warmspot.py (original) +++ pypy/trunk/pypy/jit/metainterp/warmspot.py Tue Nov 30 17:02:26 2010 @@ -159,6 +159,8 @@ self.check_access_directly_sanity(graphs) if backendopt: self.prejit_optimizations(policy, graphs) + elif self.opt.listops: + self.prejit_optimizations_minimal_inline(policy, graphs) self.build_meta_interp(ProfilerClass) self.make_args_specifications() @@ -256,6 +258,10 @@ remove_asserts=True, really_remove_asserts=True) + def prejit_optimizations_minimal_inline(self, policy, graphs): + from pypy.translator.backendopt.inline import auto_inline_graphs + auto_inline_graphs(self.translator, graphs, 0.01) + def build_cpu(self, CPUClass, translate_support_code=False, no_stats=False, **kwds): assert CPUClass is not None Modified: pypy/trunk/pypy/jit/tl/spli/interpreter.py ============================================================================== --- pypy/trunk/pypy/jit/tl/spli/interpreter.py (original) +++ pypy/trunk/pypy/jit/tl/spli/interpreter.py Tue Nov 30 17:02:26 2010 @@ -105,26 +105,32 @@ self.stack_depth += 1 def pop(self): - self.stack_depth -= 1 - val = self.value_stack[self.stack_depth] - self.value_stack[self.stack_depth] = None + sd = self.stack_depth - 1 + assert sd >= 0 + self.stack_depth = sd + val = self.value_stack[sd] + self.value_stack[sd] = None return val def pop_many(self, n): return [self.pop() for i in range(n)] def peek(self): - return self.value_stack[self.stack_depth - 1] + sd = self.stack_depth - 1 + assert sd >= 0 + return self.value_stack[sd] def POP_TOP(self, _, next_instr, code): self.pop() return next_instr def LOAD_FAST(self, name_index, next_instr, code): + assert name_index >= 0 self.push(self.locals[name_index]) return next_instr def STORE_FAST(self, name_index, next_instr, code): + assert name_index >= 0 self.locals[name_index] = self.pop() return next_instr Modified: pypy/trunk/pypy/jit/tl/tl.py ============================================================================== --- pypy/trunk/pypy/jit/tl/tl.py (original) +++ pypy/trunk/pypy/jit/tl/tl.py Tue Nov 30 17:02:26 2010 @@ -16,32 +16,40 @@ def __init__(self, size): self = hint(self, access_directly=True, fresh_virtualizable=True) self.stack = [0] * size - self.stackpos = 0 + self.stackpos = 0 # always store a known-nonneg integer here def append(self, elem): self.stack[self.stackpos] = elem self.stackpos += 1 def pop(self): - self.stackpos -= 1 - if self.stackpos < 0: + stackpos = self.stackpos - 1 + if stackpos < 0: raise IndexError - return self.stack[self.stackpos] + self.stackpos = stackpos # always store a known-nonneg integer here + return self.stack[stackpos] def pick(self, i): - self.append(self.stack[self.stackpos - i - 1]) + n = self.stackpos - i - 1 + assert n >= 0 + self.append(self.stack[n]) def put(self, i): elem = self.pop() - self.stack[self.stackpos - i - 1] = elem + n = self.stackpos - i - 1 + assert n >= 0 + self.stack[n] = elem def roll(self, r): if r < -1: i = self.stackpos + r if i < 0: raise IndexError - elem = self.stack[self.stackpos - 1] + n = self.stackpos - 1 + assert n >= 0 + elem = self.stack[n] for j in range(self.stackpos - 2, i - 1, -1): + assert j >= 0 self.stack[j + 1] = self.stack[j] self.stack[i] = elem elif r > 1: @@ -51,7 +59,9 @@ elem = self.stack[i] for j in range(i, self.stackpos - 1): self.stack[j] = self.stack[j + 1] - self.stack[self.stackpos - 1] = elem + n = self.stackpos - 1 + assert n >= 0 + self.stack[n] = elem def make_interp(supports_call): Modified: pypy/trunk/pypy/module/cpyext/api.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/api.py (original) +++ pypy/trunk/pypy/module/cpyext/api.py Tue Nov 30 17:02:26 2010 @@ -369,7 +369,7 @@ }.items(): GLOBALS['Py%s_Type#' % (cpyname, )] = ('PyTypeObject*', pypyexpr) - for cpyname in 'Method List Int Long Dict Tuple Class'.split(): + for cpyname in 'Method List Int Long Dict Class'.split(): FORWARD_DECLS.append('typedef struct { PyObject_HEAD } ' 'Py%sObject' % (cpyname, )) build_exported_objects() Modified: pypy/trunk/pypy/module/cpyext/include/tupleobject.h ============================================================================== --- pypy/trunk/pypy/module/cpyext/include/tupleobject.h (original) +++ pypy/trunk/pypy/module/cpyext/include/tupleobject.h Tue Nov 30 17:02:26 2010 @@ -10,9 +10,19 @@ /* defined in varargswrapper.c */ PyObject * PyTuple_Pack(Py_ssize_t, ...); -#define PyTuple_SET_ITEM PyTuple_SetItem -#define PyTuple_GET_ITEM PyTuple_GetItem +typedef struct { + PyObject_HEAD + PyObject **items; + Py_ssize_t size; +} PyTupleObject; +#define PyTuple_GET_ITEM PyTuple_GetItem + +/* Macro, trading safety for speed */ +#define PyTuple_GET_SIZE(op) (((PyTupleObject *)(op))->size) + +/* Macro, *only* to be used to fill in brand new tuples */ +#define PyTuple_SET_ITEM(op, i, v) (((PyTupleObject *)(op))->items[i] = v) #ifdef __cplusplus } Modified: pypy/trunk/pypy/module/cpyext/slotdefs.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/slotdefs.py (original) +++ pypy/trunk/pypy/module/cpyext/slotdefs.py Tue Nov 30 17:02:26 2010 @@ -28,14 +28,14 @@ def check_num_args(space, ob, n): from pypy.module.cpyext.tupleobject import PyTuple_CheckExact, \ - PyTuple_GET_SIZE + _PyTuple_Size_Fast if not PyTuple_CheckExact(space, ob): raise OperationError(space.w_SystemError, space.wrap("PyArg_UnpackTuple() argument list is not a tuple")) - if n == PyTuple_GET_SIZE(space, ob): + if n == _PyTuple_Size_Fast(space, ob): return raise operationerrfmt(space.w_TypeError, - "expected %d arguments, got %d", n, PyTuple_GET_SIZE(space, ob)) + "expected %d arguments, got %d", n, _PyTuple_Size_Fast(space, ob)) def wrap_init(space, w_self, w_args, func, w_kwargs): func_init = rffi.cast(initproc, func) Modified: pypy/trunk/pypy/module/cpyext/test/test_cpyext.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/test/test_cpyext.py (original) +++ pypy/trunk/pypy/module/cpyext/test/test_cpyext.py Tue Nov 30 17:02:26 2010 @@ -545,16 +545,17 @@ PyObject *true = Py_True; PyObject *tup = NULL; int refcnt = true->ob_refcnt; - int refcnt_after; + int refcnt_middle, refcnt_after; tup = PyTuple_New(1); Py_INCREF(true); if (PyTuple_SetItem(tup, 0, true) < 0) return NULL; - refcnt_after = true->ob_refcnt; + refcnt_middle = true->ob_refcnt; Py_DECREF(tup); - fprintf(stderr, "REFCNT2 %i %i\\n", refcnt, refcnt_after); - return PyBool_FromLong(refcnt_after == refcnt); + refcnt_after = true->ob_refcnt; + fprintf(stderr, "REFCNT2 %i %i %i\\n", refcnt, refcnt_middle, refcnt_after); + return PyBool_FromLong(refcnt_after == refcnt && refcnt_middle == refcnt+1); } static PyMethodDef methods[] = { Modified: pypy/trunk/pypy/module/cpyext/test/test_tupleobject.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/test/test_tupleobject.py (original) +++ pypy/trunk/pypy/module/cpyext/test/test_tupleobject.py Tue Nov 30 17:02:26 2010 @@ -7,24 +7,34 @@ class TestTupleObject(BaseApiTest): def test_tupleobject(self, space, api): assert not api.PyTuple_Check(space.w_None) - assert api.PyTuple_SetItem(space.w_None, 0, space.w_None) == -1 + #assert api.PyTuple_SetItem(space.w_None, 0, space.w_None) == -1 XXX atuple = space.newtuple([0, 1, 'yay']) assert api.PyTuple_Size(atuple) == 3 - assert api.PyTuple_GET_SIZE(atuple) == 3 - raises(TypeError, api.PyTuple_Size(space.newlist([]))) + #raises(TypeError, api.PyTuple_Size(space.newlist([]))) XXX api.PyErr_Clear() def test_tuple_resize(self, space, api): - py_tuple = api.PyTuple_New(3) + ref_tup = api.PyTuple_New(3) ar = lltype.malloc(PyObjectP.TO, 1, flavor='raw') - ar[0] = rffi.cast(PyObject, make_ref(space, py_tuple)) + ar[0] = rffi.cast(PyObject, ref_tup) api._PyTuple_Resize(ar, 2) - py_tuple = from_ref(space, ar[0]) - assert len(py_tuple.wrappeditems) == 2 + assert ar[0] == rffi.cast(PyObject, ref_tup) + # ^^^ our _PyTuple_Resize does not actually need to change the ptr so far + assert api.PyTuple_Size(ar[0]) == 2 api._PyTuple_Resize(ar, 10) - py_tuple = from_ref(space, ar[0]) - assert len(py_tuple.wrappeditems) == 10 + assert api.PyTuple_Size(ar[0]) == 10 api.Py_DecRef(ar[0]) lltype.free(ar, flavor='raw') + + def test_tuple_setup(self, space, api): + ref_tup = api.PyTuple_New(2) + ref0 = make_ref(space, space.wrap(123)) + api.PyTuple_SetItem(ref_tup, 0, ref0) + ref1 = make_ref(space, space.wrap(456)) + api.PyTuple_SetItem(ref_tup, 1, ref1) + + w_tup = from_ref(space, ref_tup) + assert space.is_true(space.eq(w_tup, space.wrap((123, 456)))) + api.Py_DecRef(ref_tup) Modified: pypy/trunk/pypy/module/cpyext/tupleobject.py ============================================================================== --- pypy/trunk/pypy/module/cpyext/tupleobject.py (original) +++ pypy/trunk/pypy/module/cpyext/tupleobject.py Tue Nov 30 17:02:26 2010 @@ -1,55 +1,144 @@ from pypy.interpreter.error import OperationError from pypy.rpython.lltypesystem import rffi, lltype from pypy.module.cpyext.api import (cpython_api, Py_ssize_t, CANNOT_FAIL, - build_type_checkers) + build_type_checkers, PyObjectFields, + cpython_struct, bootstrap_function) from pypy.module.cpyext.pyobject import (PyObject, PyObjectP, Py_DecRef, - borrow_from, make_ref, from_ref) + borrow_from, make_ref, from_ref, make_typedescr, get_typedescr, Reference, + track_reference) from pypy.module.cpyext.pyerrors import PyErr_BadInternalCall -from pypy.objspace.std.tupleobject import W_TupleObject +## +## Implementation of PyTupleObject +## =============================== +## +## We have the same problem as PyStringObject: a PyTupleObject can be +## initially used in a read-write way with PyTuple_New(), PyTuple_SetItem() +## and _PyTuple_Resize(). +## +## The 'size' and 'items' fields of a PyTupleObject are always valid. +## Apart from that detail, see the big comment in stringobject.py for +## more information. +## + +ARRAY_OF_PYOBJ = rffi.CArrayPtr(PyObject) +PyTupleObjectStruct = lltype.ForwardReference() +PyTupleObject = lltype.Ptr(PyTupleObjectStruct) +PyTupleObjectFields = PyObjectFields + \ + (("items", ARRAY_OF_PYOBJ), ("size", Py_ssize_t)) +cpython_struct("PyTupleObject", PyTupleObjectFields, PyTupleObjectStruct) + + at bootstrap_function +def init_tupleobject(space): + "Type description of PyTupleObject" + make_typedescr(space.w_tuple.instancetypedef, + basestruct=PyTupleObject.TO, + attach=tuple_attach, + dealloc=tuple_dealloc, + realize=tuple_realize) PyTuple_Check, PyTuple_CheckExact = build_type_checkers("Tuple") +def new_empty_tuple(space, length): + """ + Allocate a PyTupleObject and its array, but without a corresponding + interpreter object. The array items may be mutated, until + tuple_realize() is called. + """ + typedescr = get_typedescr(space.w_tuple.instancetypedef) + py_obj = typedescr.allocate(space, space.w_tuple) + py_tup = rffi.cast(PyTupleObject, py_obj) + + py_tup.c_items = lltype.malloc(ARRAY_OF_PYOBJ.TO, length, + flavor='raw', zero=True) + py_tup.c_size = length + return py_tup + +def tuple_attach(space, py_obj, w_obj): + """ + Fills a newly allocated PyTupleObject with the given tuple object. + """ + items_w = space.fixedview(w_obj) + py_tup = rffi.cast(PyTupleObject, py_obj) + py_tup.c_items = lltype.nullptr(ARRAY_OF_PYOBJ.TO) + py_tup.c_size = len(items_w) + +def tuple_realize(space, py_obj): + """ + Creates the tuple in the interpreter. The PyTupleObject items array + must not be modified after this call. + """ + py_tup = rffi.cast(PyTupleObject, py_obj) + # If your CPython extension creates a self-referential tuple + # with PyTuple_SetItem(), you loose. + c_items = py_tup.c_items + items_w = [from_ref(space, c_items[i]) for i in range(py_tup.c_size)] + w_obj = space.newtuple(items_w) + track_reference(space, py_obj, w_obj) + return w_obj + + at cpython_api([PyObject], lltype.Void, external=False) +def tuple_dealloc(space, py_obj): + """Frees allocated PyTupleObject resources. + """ + py_tup = rffi.cast(PyTupleObject, py_obj) + if py_tup.c_items: + for i in range(py_tup.c_size): + Py_DecRef(space, py_tup.c_items[i]) + lltype.free(py_tup.c_items, flavor="raw") + from pypy.module.cpyext.object import PyObject_dealloc + PyObject_dealloc(space, py_obj) + +#_______________________________________________________________________ + @cpython_api([Py_ssize_t], PyObject) def PyTuple_New(space, size): - return space.newtuple([space.w_None] * size) + return rffi.cast(PyObject, new_empty_tuple(space, size)) @cpython_api([PyObject, Py_ssize_t, PyObject], rffi.INT_real, error=-1) -def PyTuple_SetItem(space, w_t, pos, w_obj): - if not PyTuple_Check(space, w_t): - # XXX this should also steal a reference, test it!!! - PyErr_BadInternalCall(space) - assert isinstance(w_t, W_TupleObject) - w_t.wrappeditems[pos] = w_obj - Py_DecRef(space, w_obj) # SetItem steals a reference! +def PyTuple_SetItem(space, ref, pos, ref_item): + # XXX steals a reference at the level of PyObjects. Don't try to + # XXX call this function with an interpreter object as ref_item! + + # XXX do PyTuple_Check, without forcing ref as an interpreter object + # XXX -- then if it fails it should also steal a reference, test it!!! + ref_tup = rffi.cast(PyTupleObject, ref) + if not ref_tup.c_items: + msg = "PyTuple_SetItem() called on an already-escaped tuple object" + raise OperationError(space.w_SystemError, space.wrap(msg)) + ref_old = ref_tup.c_items[pos] + ref_tup.c_items[pos] = ref_item # SetItem steals a reference! + Py_DecRef(space, ref_old) return 0 @cpython_api([PyObject, Py_ssize_t], PyObject) -def PyTuple_GetItem(space, w_t, pos): - if not PyTuple_Check(space, w_t): - PyErr_BadInternalCall(space) - assert isinstance(w_t, W_TupleObject) - w_obj = w_t.wrappeditems[pos] - return borrow_from(w_t, w_obj) - - at cpython_api([PyObject], Py_ssize_t, error=CANNOT_FAIL) -def PyTuple_GET_SIZE(space, w_t): - """Return the size of the tuple p, which must be non-NULL and point to a tuple; - no error checking is performed. """ - assert isinstance(w_t, W_TupleObject) - return len(w_t.wrappeditems) +def PyTuple_GetItem(space, ref, pos): + # XXX do PyTuple_Check, without forcing ref as an interpreter object + ref_tup = rffi.cast(PyTupleObject, ref) + if ref_tup.c_items: + return Reference(ref_tup.c_items[pos]) # borrowed reference + else: + w_t = from_ref(space, ref) + w_obj = space.getitem(w_t, space.wrap(pos)) + return borrow_from(w_t, w_obj) + + at cpython_api([PyObject], Py_ssize_t, error=-1) +def _PyTuple_Size_Fast(space, ref): + # custom version: it's not a macro, so it can be called from other .py + # files; but it doesn't include PyTuple_Check() + ref_tup = rffi.cast(PyTupleObject, ref) + return ref_tup.c_size @cpython_api([PyObject], Py_ssize_t, error=-1) def PyTuple_Size(space, ref): """Take a pointer to a tuple object, and return the size of that tuple.""" - if not PyTuple_Check(space, ref): - raise OperationError(space.w_TypeError, - space.wrap("expected tuple object")) - return PyTuple_GET_SIZE(space, ref) + # XXX do PyTuple_Check, without forcing ref as an interpreter object + ref_tup = rffi.cast(PyTupleObject, ref) + return ref_tup.c_size @cpython_api([PyObjectP, Py_ssize_t], rffi.INT_real, error=-1) -def _PyTuple_Resize(space, ref, newsize): +def _PyTuple_Resize(space, refp, newsize): """Can be used to resize a tuple. newsize will be the new length of the tuple. Because tuples are supposed to be immutable, this should only be used if there is only one reference to the object. Do not use this if the tuple may already @@ -60,18 +149,22 @@ this function. If the object referenced by *p is replaced, the original *p is destroyed. On failure, returns -1 and sets *p to NULL, and raises MemoryError or SystemError.""" - py_tuple = from_ref(space, ref[0]) - if not PyTuple_Check(space, py_tuple): - PyErr_BadInternalCall(space) - assert isinstance(py_tuple, W_TupleObject) - py_newtuple = PyTuple_New(space, newsize) - - to_cp = newsize - oldsize = len(py_tuple.wrappeditems) - if oldsize < newsize: - to_cp = oldsize - for i in range(to_cp): - py_newtuple.wrappeditems[i] = py_tuple.wrappeditems[i] - Py_DecRef(space, ref[0]) - ref[0] = make_ref(space, py_newtuple) + # XXX do PyTuple_Check, without forcing ref as an interpreter object + # XXX -- then if it fails it should reset refp[0] to null + ref_tup = rffi.cast(PyTupleObject, refp[0]) + c_newitems = lltype.malloc(ARRAY_OF_PYOBJ.TO, newsize, + flavor='raw', zero=True) + c_olditems = ref_tup.c_items + if not c_olditems: + msg = "_PyTuple_Resize() called on an already-escaped tuple object" + raise OperationError(space.w_SystemError, space.wrap(msg)) + oldsize = ref_tup.c_size + for i in range(min(oldsize, newsize)): + c_newitems[i] = c_olditems[i] + # decref items deleted by shrinkage + for i in range(newsize, oldsize): + Py_DecRef(space, c_olditems[i]) + ref_tup.c_items = c_newitems + ref_tup.c_size = newsize + lltype.free(c_olditems, flavor='raw') return 0 Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Tue Nov 30 17:02:26 2010 @@ -695,11 +695,9 @@ i = t2[3] del t2 return i - ''', 100, ([], 100)) + ''', 40, ([], 100)) bytecode, = self.get_by_bytecode('BINARY_SUBSCR') - assert len(bytecode.get_opnames('new_array')) == 1 - # XXX I would like here to say that it's 0, but unfortunately - # call that can raise is not exchanged into getarrayitem_gc + assert len(bytecode.get_opnames('new_array')) == 0 def test_overflow_checking(self): startvalue = sys.maxint - 2147483647 Modified: pypy/trunk/pypy/objspace/std/objspace.py ============================================================================== --- pypy/trunk/pypy/objspace/std/objspace.py (original) +++ pypy/trunk/pypy/objspace/std/objspace.py Tue Nov 30 17:02:26 2010 @@ -376,7 +376,7 @@ self, w_obj, expected_length)[:]) if expected_length != -1 and len(t) != expected_length: raise self._wrap_expected_length(expected_length, len(t)) - return t + return make_sure_not_resized(t) def fixedview_unroll(self, w_obj, expected_length=-1): return self.fixedview(w_obj, expected_length, unroll=True) Modified: pypy/trunk/pypy/rlib/debug.py ============================================================================== --- pypy/trunk/pypy/rlib/debug.py (original) +++ pypy/trunk/pypy/rlib/debug.py Tue Nov 30 17:02:26 2010 @@ -226,31 +226,6 @@ hop.exception_cannot_occur() return hop.inputarg(hop.args_r[0], arg=0) -def make_sure_not_modified(arg): - """ Function checking whether annotation of SomeList is never resized - and never modified, useful for debugging. Does nothing when run directly - """ - return arg - -class Entry(ExtRegistryEntry): - _about_ = make_sure_not_modified - - def compute_result_annotation(self, s_arg): - from pypy.annotation.model import SomeList - assert isinstance(s_arg, SomeList) - # the logic behind it is that we try not to propagate - # make_sure_not_resized, when list comprehension is not on - if self.bookkeeper.annotator.translator.config.translation.list_comprehension_operations: - s_arg.listdef.never_mutate() - else: - from pypy.annotation.annrpython import log - log.WARNING('make_sure_not_modified called, but has no effect since list_comprehension is off') - return s_arg - - def specialize_call(self, hop): - hop.exception_cannot_occur() - return hop.inputarg(hop.args_r[0], arg=0) - class IntegerCanBeNegative(Exception): pass Modified: pypy/trunk/pypy/rlib/rsre/rsre_core.py ============================================================================== --- pypy/trunk/pypy/rlib/rsre/rsre_core.py (original) +++ pypy/trunk/pypy/rlib/rsre/rsre_core.py Tue Nov 30 17:02:26 2010 @@ -1,5 +1,5 @@ import sys -from pypy.rlib.debug import check_nonneg, make_sure_not_modified +from pypy.rlib.debug import check_nonneg from pypy.rlib.unroll import unrolling_iterable from pypy.rlib.rsre import rsre_char from pypy.tool.sourcetools import func_with_new_name @@ -471,7 +471,6 @@ while True: op = ctx.pat(ppos) ppos += 1 - make_sure_not_modified(ctx.pattern) #jit.jit_debug("sre_match", op, ppos, ptr) # Modified: pypy/trunk/pypy/rlib/rsre/test/test_zjit.py ============================================================================== --- pypy/trunk/pypy/rlib/rsre/test/test_zjit.py (original) +++ pypy/trunk/pypy/rlib/rsre/test/test_zjit.py Tue Nov 30 17:02:26 2010 @@ -1,6 +1,5 @@ from pypy.jit.metainterp.test import test_basic from pypy.rlib.nonconst import NonConstant -from pypy.rlib.debug import make_sure_not_modified from pypy.rlib.rsre.test.test_match import get_code from pypy.rlib.rsre import rsre_core from pypy.rpython.lltypesystem import lltype Modified: pypy/trunk/pypy/rlib/test/test_debug.py ============================================================================== --- pypy/trunk/pypy/rlib/test/test_debug.py (original) +++ pypy/trunk/pypy/rlib/test/test_debug.py Tue Nov 30 17:02:26 2010 @@ -42,14 +42,14 @@ py.test.raises(IntegerCanBeNegative, interpret, g, [9]) def test_make_sure_not_resized(): - from pypy.annotation.listdef import TooLateForChange + from pypy.annotation.listdef import ListChangeUnallowed def f(): result = [1,2,3] make_sure_not_resized(result) result.append(4) return len(result) - py.test.raises(TooLateForChange, interpret, f, [], + py.test.raises(ListChangeUnallowed, interpret, f, [], list_comprehension_operations=True) Modified: pypy/trunk/pypy/rpython/rlist.py ============================================================================== --- pypy/trunk/pypy/rpython/rlist.py (original) +++ pypy/trunk/pypy/rpython/rlist.py Tue Nov 30 17:02:26 2010 @@ -9,7 +9,7 @@ from pypy.rpython import robject from pypy.rlib.objectmodel import malloc_zero_filled from pypy.rlib.debug import ll_assert -from pypy.rlib.rarithmetic import ovfcheck, widen +from pypy.rlib.rarithmetic import ovfcheck, widen, r_uint, intmask from pypy.rpython.annlowlevel import ADTInterface from pypy.rlib import rgc @@ -241,17 +241,22 @@ class __extend__(pairtype(AbstractBaseListRepr, IntegerRepr)): def rtype_getitem((r_lst, r_int), hop, checkidx=False): + v_lst, v_index = hop.inputargs(r_lst, Signed) if checkidx: - spec = dum_checkidx + hop.exception_is_here() else: - spec = dum_nocheck - v_func = hop.inputconst(Void, spec) - v_lst, v_index = hop.inputargs(r_lst, Signed) + hop.exception_cannot_occur() if hop.args_s[0].listdef.listitem.mutated or checkidx: if hop.args_s[1].nonneg: llfn = ll_getitem_nonneg else: llfn = ll_getitem + if checkidx: + spec = dum_checkidx + else: + spec = dum_nocheck + c_func_marker = hop.inputconst(Void, spec) + v_res = hop.gendirectcall(llfn, c_func_marker, v_lst, v_index) else: # this is the 'foldable' version, which is not used when # we check for IndexError @@ -259,11 +264,7 @@ llfn = ll_getitem_foldable_nonneg else: llfn = ll_getitem_foldable - if checkidx: - hop.exception_is_here() - else: - hop.exception_cannot_occur() - v_res = hop.gendirectcall(llfn, v_func, v_lst, v_index) + v_res = hop.gendirectcall(llfn, v_lst, v_index) return r_lst.recast(hop.llops, v_res) rtype_getitem_key = rtype_getitem @@ -538,12 +539,14 @@ dest.ll_setitem_fast(dest_start + i, item) i += 1 ll_arraycopy._annenforceargs_ = [None, None, int, int, int] +# no oopspec -- the function is inlined by the JIT def ll_copy(RESLIST, l): length = l.ll_length() new_lst = RESLIST.ll_newlist(length) ll_arraycopy(l, new_lst, 0, 0, length) return new_lst +# no oopspec -- the function is inlined by the JIT def ll_len(l): return l.ll_length() @@ -551,6 +554,7 @@ def ll_list_is_true(l): # check if a list is True, allowing for None return bool(l) and l.ll_length() != 0 +# no oopspec -- the function is inlined by the JIT def ll_len_foldable(l): return l.ll_length() @@ -558,6 +562,7 @@ def ll_list_is_true_foldable(l): return bool(l) and ll_len_foldable(l) != 0 +# no oopspec -- the function is inlined by the JIT def ll_append(l, newitem): length = l.ll_length() @@ -588,6 +593,7 @@ ll_arraycopy(l1, l, 0, 0, len1) ll_arraycopy(l2, l, 0, len1, len2) return l +# no oopspec -- the function is inlined by the JIT def ll_insert_nonneg(l, index, newitem): length = l.ll_length() @@ -674,60 +680,72 @@ l.ll_setitem_fast(length_1_i, tmp) i += 1 length_1_i -= 1 +ll_reverse.oopspec = 'list.reverse(l)' def ll_getitem_nonneg(func, l, index): ll_assert(index >= 0, "unexpectedly negative list getitem index") if func is dum_checkidx: if index >= l.ll_length(): raise IndexError - else: - ll_assert(index < l.ll_length(), "list getitem index out of bound") return l.ll_getitem_fast(index) -ll_getitem_nonneg.oopspec = 'list.getitem(l, index)' +ll_getitem_nonneg._always_inline_ = True +# no oopspec -- the function is inlined by the JIT def ll_getitem(func, l, index): - length = l.ll_length() - if index < 0: - index += length if func is dum_checkidx: - if index < 0 or index >= length: - raise IndexError + length = l.ll_length() # common case: 0 <= index < length + if r_uint(index) >= r_uint(length): + # Failed, so either (-length <= index < 0), or we have to raise + # IndexError. First add 'length' to get the final index, then + # check that we now have (0 <= index < length). + index = r_uint(index) + r_uint(length) + if index >= r_uint(length): + raise IndexError + index = intmask(index) else: - ll_assert(index >= 0, "negative list getitem index out of bound") - ll_assert(index < length, "list getitem index out of bound") + # We don't want checking, but still want to support index < 0. + # Only call ll_length() if needed. + if index < 0: + index += l.ll_length() + ll_assert(index >= 0, "negative list getitem index out of bound") return l.ll_getitem_fast(index) -ll_getitem.oopspec = 'list.getitem(l, index)' +# no oopspec -- the function is inlined by the JIT -def ll_getitem_foldable_nonneg(func, l, index): - return ll_getitem_nonneg(func, l, index) +def ll_getitem_foldable_nonneg(l, index): + ll_assert(index >= 0, "unexpectedly negative list getitem index") + return l.ll_getitem_fast(index) ll_getitem_foldable_nonneg.oopspec = 'list.getitem_foldable(l, index)' -def ll_getitem_foldable(func, l, index): - return ll_getitem(func, l, index) -ll_getitem_foldable.oopspec = 'list.getitem_foldable(l, index)' +def ll_getitem_foldable(l, index): + if index < 0: + index += l.ll_length() + return ll_getitem_foldable_nonneg(l, index) +ll_getitem_foldable._always_inline_ = True +# no oopspec -- the function is inlined by the JIT def ll_setitem_nonneg(func, l, index, newitem): ll_assert(index >= 0, "unexpectedly negative list setitem index") if func is dum_checkidx: if index >= l.ll_length(): raise IndexError - else: - ll_assert(index < l.ll_length(), "list setitem index out of bound") l.ll_setitem_fast(index, newitem) -ll_setitem_nonneg.oopspec = 'list.setitem(l, index, newitem)' +ll_setitem_nonneg._always_inline_ = True +# no oopspec -- the function is inlined by the JIT def ll_setitem(func, l, index, newitem): - length = l.ll_length() - if index < 0: - index += length if func is dum_checkidx: - if index < 0 or index >= length: - raise IndexError + length = l.ll_length() + if r_uint(index) >= r_uint(length): # see comments in ll_getitem(). + index = r_uint(index) + r_uint(length) + if index >= r_uint(length): + raise IndexError + index = intmask(index) else: - ll_assert(index >= 0, "negative list setitem index out of bound") - ll_assert(index < length, "list setitem index out of bound") + if index < 0: + index += l.ll_length() + ll_assert(index >= 0, "negative list setitem index out of bound") l.ll_setitem_fast(index, newitem) -ll_setitem.oopspec = 'list.setitem(l, index, newitem)' +# no oopspec -- the function is inlined by the JIT def ll_delitem_nonneg(func, l, index): ll_assert(index >= 0, "unexpectedly negative list delitem index") @@ -751,19 +769,20 @@ l._ll_resize_le(newlength) ll_delitem_nonneg.oopspec = 'list.delitem(l, index)' -def ll_delitem(func, l, i): - length = l.ll_length() - if i < 0: - i += length +def ll_delitem(func, l, index): if func is dum_checkidx: - if i < 0 or i >= length: - raise IndexError + length = l.ll_length() + if r_uint(index) >= r_uint(length): # see comments in ll_getitem(). + index = r_uint(index) + r_uint(length) + if index >= r_uint(length): + raise IndexError + index = intmask(index) else: - ll_assert(i >= 0, "negative list delitem index out of bound") - ll_assert(i < length, "list delitem index out of bound") - ll_delitem_nonneg(dum_nocheck, l, i) -ll_delitem.oopspec = 'list.delitem(l, i)' - + if index < 0: + index += l.ll_length() + ll_assert(index >= 0, "negative list delitem index out of bound") + ll_delitem_nonneg(dum_nocheck, l, index) +# no oopspec -- the function is inlined by the JIT def ll_extend(l1, l2): len1 = l1.ll_length() @@ -799,6 +818,7 @@ lst.ll_setitem_fast(j, c) i += 1 j += 1 +# not inlined by the JIT -- contains a loop def ll_extend_with_str_slice_startstop(lst, s, getstrlen, getstritem, start, stop): @@ -824,6 +844,7 @@ lst.ll_setitem_fast(j, c) i += 1 j += 1 +# not inlined by the JIT -- contains a loop def ll_extend_with_str_slice_minusone(lst, s, getstrlen, getstritem): len1 = lst.ll_length() @@ -843,6 +864,7 @@ lst.ll_setitem_fast(j, c) i += 1 j += 1 +# not inlined by the JIT -- contains a loop def ll_extend_with_char_count(lst, char, count): if count <= 0: @@ -859,6 +881,7 @@ while j < newlength: lst.ll_setitem_fast(j, char) j += 1 +# not inlined by the JIT -- contains a loop def ll_listslice_startonly(RESLIST, l1, start): len1 = l1.ll_length() @@ -869,6 +892,7 @@ ll_arraycopy(l1, l, start, 0, newlength) return l ll_listslice_startonly._annenforceargs_ = (None, None, int) +# no oopspec -- the function is inlined by the JIT def ll_listslice_startstop(RESLIST, l1, start, stop): length = l1.ll_length() @@ -881,6 +905,7 @@ l = RESLIST.ll_newlist(newlength) ll_arraycopy(l1, l, start, 0, newlength) return l +# no oopspec -- the function is inlined by the JIT def ll_listslice_minusone(RESLIST, l1): newlength = l1.ll_length() - 1 @@ -888,6 +913,7 @@ l = RESLIST.ll_newlist(newlength) ll_arraycopy(l1, l, 0, 0, newlength) return l +# no oopspec -- the function is inlined by the JIT def ll_listdelslice_startonly(l, start): ll_assert(start >= 0, "del l[start:] with unexpectedly negative start") @@ -958,6 +984,7 @@ return False j += 1 return True +# not inlined by the JIT -- contains a loop def ll_listcontains(lst, obj, eqfn): lng = lst.ll_length() @@ -971,6 +998,7 @@ return True j += 1 return False +# not inlined by the JIT -- contains a loop def ll_listindex(lst, obj, eqfn): lng = lst.ll_length() @@ -984,6 +1012,7 @@ return j j += 1 raise ValueError # can't say 'list.index(x): x not in list' +# not inlined by the JIT -- contains a loop def ll_listremove(lst, obj, eqfn): index = ll_listindex(lst, obj, eqfn) # raises ValueError if obj not in lst @@ -1030,3 +1059,4 @@ i += 1 j += length return res +# not inlined by the JIT -- contains a loop Modified: pypy/trunk/pypy/rpython/test/test_rlist.py ============================================================================== --- pypy/trunk/pypy/rpython/test/test_rlist.py (original) +++ pypy/trunk/pypy/rpython/test/test_rlist.py Tue Nov 30 17:02:26 2010 @@ -12,6 +12,7 @@ from pypy.rpython.rint import signed_repr from pypy.objspace.flow.model import Constant, Variable from pypy.rpython.test.tool import BaseRtypingTest, LLRtypeMixin, OORtypeMixin +from pypy.rlib.debug import ll_assert # undo the specialization parameter for n1 in 'get set del'.split(): @@ -1076,7 +1077,13 @@ res = self.interpret(f, [0]) assert res == 1 - py.test.raises(AssertionError, self.interpret, f, [1]) + if self.type_system == 'lltype': + # on lltype we always get an AssertionError + py.test.raises(AssertionError, self.interpret, f, [1]) + else: + # on ootype we happen to get through the ll_asserts and to + # hit the IndexError from ootype.py + self.interpret_raises(IndexError, f, [1]) def f(x): l = [1] @@ -1121,12 +1128,13 @@ res = self.interpret(f, [0]) assert res == 1 - try: - self.interpret_raises(IndexError, f, [1]) - except (AssertionError,), e: - pass + if self.type_system == 'lltype': + # on lltype we always get an AssertionError + py.test.raises(AssertionError, self.interpret, f, [1]) else: - assert False + # on ootype we happen to get through the ll_asserts and to + # hit the IndexError from ootype.py + self.interpret_raises(IndexError, f, [1]) def f(x): l = [1] @@ -1163,12 +1171,13 @@ res = self.interpret(f, [0]) assert res == 1 - try: - self.interpret_raises(IndexError, f, [1]) - except (AssertionError,), e: - pass + if self.type_system == 'lltype': + # on lltype we always get an AssertionError + py.test.raises(AssertionError, self.interpret, f, [1]) else: - assert False + # on ootype we happen to get through the ll_asserts and to + # hit the IndexError from ootype.py + self.interpret_raises(IndexError, f, [1]) def test_charlist_extension_1(self): def f(n): @@ -1327,8 +1336,32 @@ res = self.interpret(f, [2]) assert res == True + def test_immutable_list_out_of_instance(self): + from pypy.translator.simplify import get_funcobj + for immutable_fields in (["a", "b"], ["a", "b", "y[*]"]): + class A(object): + _immutable_fields_ = immutable_fields + class B(A): + pass + def f(i): + b = B() + lst = [i] + lst[0] += 1 + b.y = lst + ll_assert(b.y is lst, "copying when reading out the attr?") + return b.y[0] + res = self.interpret(f, [10]) + assert res == 11 + t, rtyper, graph = self.gengraph(f, [int]) + block = graph.startblock + op = block.operations[-1] + assert op.opname == 'direct_call' + func = get_funcobj(op.args[0].value)._callable + assert ('foldable' in func.func_name) == \ + ("y[*]" in immutable_fields) class TestLLtype(BaseTestRlist, LLRtypeMixin): + type_system = 'lltype' rlist = ll_rlist def test_memoryerror(self): @@ -1420,14 +1453,16 @@ lst2 = [i] lst2.append(42) # mutated list return lst1[i] + lst2[i] - _, _, graph = self.gengraph(f, [int]) + from pypy.annotation import model as annmodel + _, _, graph = self.gengraph(f, [annmodel.SomeInteger(nonneg=True)]) block = graph.startblock lst1_getitem_op = block.operations[-3] # XXX graph fishing lst2_getitem_op = block.operations[-2] func1 = lst1_getitem_op.args[0].value._obj._callable func2 = lst2_getitem_op.args[0].value._obj._callable assert func1.oopspec == 'list.getitem_foldable(l, index)' - assert func2.oopspec == 'list.getitem(l, index)' + assert not hasattr(func2, 'oopspec') class TestOOtype(BaseTestRlist, OORtypeMixin): rlist = oo_rlist + type_system = 'ootype' From arigo at codespeak.net Tue Nov 30 17:37:48 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 17:37:48 +0100 (CET) Subject: [pypy-svn] r79691 - pypy/branch/rlist-jit Message-ID: <20101130163748.B250D282BEB@codespeak.net> Author: arigo Date: Tue Nov 30 17:37:47 2010 New Revision: 79691 Removed: pypy/branch/rlist-jit/ Log: Remove merged branch. From afa at codespeak.net Tue Nov 30 17:49:06 2010 From: afa at codespeak.net (afa at codespeak.net) Date: Tue, 30 Nov 2010 17:49:06 +0100 (CET) Subject: [pypy-svn] r79692 - pypy/branch/fast-forward/pypy/module/_io Message-ID: <20101130164906.274535080C@codespeak.net> Author: afa Date: Tue Nov 30 17:49:04 2010 New Revision: 79692 Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Log: Fix a JIT translation issue. Members with default values defined at the class level should be "simple" types: no longlong, no float. Modified: pypy/branch/fast-forward/pypy/module/_io/interp_textio.py ============================================================================== --- pypy/branch/fast-forward/pypy/module/_io/interp_textio.py (original) +++ pypy/branch/fast-forward/pypy/module/_io/interp_textio.py Tue Nov 30 17:49:04 2010 @@ -172,7 +172,7 @@ def setstate_w(self, space, w_state): w_buffer, w_flag = space.unpackiterable(w_state, 2) flag = space.r_longlong_w(w_flag) - self.pendingcr = (flag & 1) + self.pendingcr = bool(flag & 1) flag >>= 1 if self.w_decoder and not space.is_w(self.w_decoder, space.w_None): From arigo at codespeak.net Tue Nov 30 17:58:21 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 17:58:21 +0100 (CET) Subject: [pypy-svn] r79693 - pypy/branch/jit-free-asm2/pypy/jit/backend/x86 Message-ID: <20101130165821.69BB6282BE7@codespeak.net> Author: arigo Date: Tue Nov 30 17:58:19 2010 New Revision: 79693 Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py Log: Now we really win: use MachineDataBlockWrapper to replace the hacks done at start-up to allocate the float constants with a 16-bytes alignment. Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py Tue Nov 30 17:58:19 2010 @@ -151,13 +151,9 @@ debug_stop('jit-backend-counts') def _build_float_constants(self): - # 44 bytes: 32 bytes for the data, and up to 12 bytes for alignment - addr = lltype.malloc(rffi.CArray(lltype.Char), 44, flavor='raw', - track_allocation=False) - if not we_are_translated(): - self._keepalive_malloced_float_consts = addr - float_constants = rffi.cast(lltype.Signed, addr) - float_constants = (float_constants + 15) & ~15 # align to 16 bytes + datablockwrapper = MachineDataBlockWrapper(self.cpu.asmmemmgr, []) + float_constants = datablockwrapper.malloc_aligned(32, alignment=16) + datablockwrapper.done() addr = rffi.cast(rffi.CArrayPtr(lltype.Char), float_constants) qword_padding = '\x00\x00\x00\x00\x00\x00\x00\x00' # 0x8000000000000000 From cfbolz at codespeak.net Tue Nov 30 18:10:51 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 30 Nov 2010 18:10:51 +0100 (CET) Subject: [pypy-svn] r79694 - in pypy/trunk/pypy: module/pypyjit/test objspace/std Message-ID: <20101130171051.D650950810@codespeak.net> Author: cfbolz Date: Tue Nov 30 18:10:50 2010 New Revision: 79694 Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py pypy/trunk/pypy/objspace/std/mapdict.py Log: fix the test, which should make getattr and setattr a bit more JIT-friendly Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py ============================================================================== --- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py (original) +++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py Tue Nov 30 18:10:50 2010 @@ -631,7 +631,6 @@ assert len(self.loops) == 1 def test_getattr_with_dynamic_attribute(self): - py.test.skip("fix this test") self.run_source(''' class A(object): pass @@ -641,6 +640,11 @@ def main(arg): sum = 0 a = A() + a.a1 = 0 + a.a2 = 0 + a.a3 = 0 + a.a4 = 0 + a.a5 = 0 # workaround, because the first five attributes need a promotion a.x = 1 a.y = 2 i = 0 Modified: pypy/trunk/pypy/objspace/std/mapdict.py ============================================================================== --- pypy/trunk/pypy/objspace/std/mapdict.py (original) +++ pypy/trunk/pypy/objspace/std/mapdict.py Tue Nov 30 18:10:50 2010 @@ -42,12 +42,21 @@ return None def index(self, selector): - if (self.space.config.objspace.std.withmethodcache and + if (self.space.config.objspace.std.withmethodcache and not jit.we_are_jitted()): return self._index_cache(selector) else: + # hack for the jit: + # the _index method is pure too, but its argument is never + # constant, because it is always a new tuple + if jit.we_are_jitted(): + return self._index_jit_pure(selector[0], selector[1]) return self._index(selector) + @jit.purefunction + def _index_jit_pure(self, name, index): + return self._index((name, index)) + @jit.dont_look_inside def _index_cache(self, selector): space = self.space @@ -498,10 +507,11 @@ def _mapdict_read_storage(self, index): assert index >= 0 - for i in rangenmin1: - if index == i: - erased = getattr(self, "_value%s" % i) - return rerased.unerase(erased, W_Root) + if index < nmin1: + for i in rangenmin1: + if index == i: + erased = getattr(self, "_value%s" % i) + return rerased.unerase(erased, W_Root) if self._has_storage_list(): return self._mapdict_get_storage_list()[index - nmin1] erased = getattr(self, "_value%s" % nmin1) From arigo at codespeak.net Tue Nov 30 18:15:41 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 18:15:41 +0100 (CET) Subject: [pypy-svn] r79695 - pypy/branch/jit-free-asm2/pypy/jit/backend/x86 Message-ID: <20101130171541.F0153282BEB@codespeak.net> Author: arigo Date: Tue Nov 30 18:15:40 2010 New Revision: 79695 Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py pypy/branch/jit-free-asm2/pypy/jit/backend/x86/regalloc.py pypy/branch/jit-free-asm2/pypy/jit/backend/x86/regloc.py Log: Kill code, using the datablockwrapper instead. Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/x86/assembler.py Tue Nov 30 18:15:40 2010 @@ -56,7 +56,6 @@ DEBUG_COUNTER = lltype.Struct('DEBUG_COUNTER', ('i', lltype.Signed)) class Assembler386(object): - _float_constants = None _regalloc = None _output_loop_log = None Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/x86/regalloc.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/x86/regalloc.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/x86/regalloc.py Tue Nov 30 18:15:40 2010 @@ -60,32 +60,6 @@ r15: 5, } -class FloatConstants(object): - BASE_CONSTANT_SIZE = 1000 - - def __init__(self): - self.cur_array_free = 0 - self.const_id = 0 - - def _get_new_array(self): - n = self.BASE_CONSTANT_SIZE - # known to leak - self.cur_array = lltype.malloc(rffi.CArray(lltype.Float), n, # YYY leak - flavor='raw', track_allocation=False) - self.cur_array_free = n - _get_new_array._dont_inline_ = True - - def record_float(self, floatval): - if self.cur_array_free == 0: - self._get_new_array() - arr = self.cur_array - n = self.cur_array_free - 1 - arr[n] = floatval - self.cur_array_free = n - self.const_id += 1 - return (self.const_id, rffi.cast(lltype.Signed, arr) + n * 8) - - class X86XMMRegisterManager(RegisterManager): box_types = [FLOAT] @@ -93,20 +67,11 @@ # we never need lower byte I hope save_around_call_regs = all_regs - def __init__(self, longevity, frame_manager=None, assembler=None): - RegisterManager.__init__(self, longevity, frame_manager=frame_manager, - assembler=assembler) - if assembler is None: - self.float_constants = FloatConstants() - else: - if assembler._float_constants is None: - assembler._float_constants = FloatConstants() - self.float_constants = assembler._float_constants - def convert_to_imm(self, c): - const_id, adr = self.float_constants.record_float(c.getfloat()) - return ConstFloatLoc(adr, const_id) - + adr = self.assembler.datablockwrapper.malloc_aligned(8, 8) + rffi.cast(rffi.CArrayPtr(rffi.DOUBLE), adr)[0] = c.getfloat() + return ConstFloatLoc(adr) + def after_call(self, v): # the result is stored in st0, but we don't have this around, # so genop_call will move it to some frame location immediately Modified: pypy/branch/jit-free-asm2/pypy/jit/backend/x86/regloc.py ============================================================================== --- pypy/branch/jit-free-asm2/pypy/jit/backend/x86/regloc.py (original) +++ pypy/branch/jit-free-asm2/pypy/jit/backend/x86/regloc.py Tue Nov 30 18:15:40 2010 @@ -177,24 +177,15 @@ class ConstFloatLoc(AssemblerLocation): # XXX: We have to use this class instead of just AddressLoc because - # AddressLoc is "untyped" and also we to have need some sort of unique - # identifier that we can use in _getregkey (for jump.py) - + # we want a width of 8 (... I think. Check this!) _immutable_ = True - width = 8 - def __init__(self, address, const_id): + def __init__(self, address): self.value = address - self.const_id = const_id def __repr__(self): - return '' % (self.value, self.const_id) - - def _getregkey(self): - # XXX: 1000 is kind of magic: We just don't want to be confused - # with any registers - return 1000 + self.const_id + return '' % (self.value,) def location_code(self): return 'j' From cfbolz at codespeak.net Tue Nov 30 18:23:02 2010 From: cfbolz at codespeak.net (cfbolz at codespeak.net) Date: Tue, 30 Nov 2010 18:23:02 +0100 (CET) Subject: [pypy-svn] r79696 - pypy/trunk/pypy/objspace/std Message-ID: <20101130172302.20E8E282BD4@codespeak.net> Author: cfbolz Date: Tue Nov 30 18:23:00 2010 New Revision: 79696 Modified: pypy/trunk/pypy/objspace/std/mapdict.py Log: Reshuffle a bit, should be even faster if you use getattr with the JIT because now the residual call to _index_jit_pure uses the caching version. Modified: pypy/trunk/pypy/objspace/std/mapdict.py ============================================================================== --- pypy/trunk/pypy/objspace/std/mapdict.py (original) +++ pypy/trunk/pypy/objspace/std/mapdict.py Tue Nov 30 18:23:00 2010 @@ -42,20 +42,23 @@ return None def index(self, selector): - if (self.space.config.objspace.std.withmethodcache and - not jit.we_are_jitted()): - return self._index_cache(selector) - else: + if jit.we_are_jitted(): # hack for the jit: # the _index method is pure too, but its argument is never # constant, because it is always a new tuple - if jit.we_are_jitted(): - return self._index_jit_pure(selector[0], selector[1]) - return self._index(selector) + return self._index_jit_pure(selector[0], selector[1]) + else: + return self._index_indirection(selector) @jit.purefunction def _index_jit_pure(self, name, index): - return self._index((name, index)) + return self._index_indirection((name, index)) + + @jit.dont_look_inside + def _index_indirection(self, selector): + if (self.space.config.objspace.std.withmethodcache): + return self._index_cache(selector) + return self._index(selector) @jit.dont_look_inside def _index_cache(self, selector): From antocuni at codespeak.net Tue Nov 30 18:39:09 2010 From: antocuni at codespeak.net (antocuni at codespeak.net) Date: Tue, 30 Nov 2010 18:39:09 +0100 (CET) Subject: [pypy-svn] r79697 - in pypy/branch/jitypes2/pypy/module/_ffi: . test Message-ID: <20101130173909.A5AAA282BD4@codespeak.net> Author: antocuni Date: Tue Nov 30 18:39:08 2010 New Revision: 79697 Modified: pypy/branch/jitypes2/pypy/module/_ffi/interp_ffi.py pypy/branch/jitypes2/pypy/module/_ffi/test/test__ffi.py Log: raise an applevel AttributeError when trying to get a pointer to a non existent function Modified: pypy/branch/jitypes2/pypy/module/_ffi/interp_ffi.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/_ffi/interp_ffi.py (original) +++ pypy/branch/jitypes2/pypy/module/_ffi/interp_ffi.py Tue Nov 30 18:39:08 2010 @@ -192,7 +192,12 @@ argtypes = [self.ffitype(w_argtype) for w_argtype in space.listview(w_argtypes)] restype = self.ffitype(w_restype, allow_void=True) - func = self.cdll.getpointer(name, argtypes, restype) + try: + func = self.cdll.getpointer(name, argtypes, restype) + except KeyError: + raise operationerrfmt(space.w_AttributeError, + "No symbol %s found in library %s", name, self.name) + return W_FuncPtr(func) Modified: pypy/branch/jitypes2/pypy/module/_ffi/test/test__ffi.py ============================================================================== --- pypy/branch/jitypes2/pypy/module/_ffi/test/test__ffi.py (original) +++ pypy/branch/jitypes2/pypy/module/_ffi/test/test__ffi.py Tue Nov 30 18:39:08 2010 @@ -142,3 +142,8 @@ def test_OSError_loading(self): from _ffi import CDLL, types raises(OSError, "CDLL('I do not exist')") + + def test_AttributeError_missing_function(self): + from _ffi import CDLL, types + libfoo = CDLL(self.libfoo_name) + raises(AttributeError, "libfoo.getfunc('I_do_not_exist', [], types.void)") From arigo at codespeak.net Tue Nov 30 18:41:43 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 18:41:43 +0100 (CET) Subject: [pypy-svn] r79698 - in pypy/trunk/pypy/jit/codewriter: . test Message-ID: <20101130174143.A046E282BD4@codespeak.net> Author: arigo Date: Tue Nov 30 18:41:42 2010 New Revision: 79698 Modified: pypy/trunk/pypy/jit/codewriter/call.py pypy/trunk/pypy/jit/codewriter/test/test_codewriter.py Log: Test and fix. Modified: pypy/trunk/pypy/jit/codewriter/call.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/call.py (original) +++ pypy/trunk/pypy/jit/codewriter/call.py Tue Nov 30 18:41:42 2010 @@ -95,15 +95,8 @@ else: v_obj = op.args[1].concretetype graphs = v_obj._lookup_graphs(op.args[0].value) - if graphs is not None: - result = [] - for graph in graphs: - if is_candidate(graph): - result.append(graph) - if result: - return result # common case: look inside these graphs, - # and ignore the others if there are any - else: + # + if graphs is None: # special case: handle the indirect call that goes to # the 'instantiate' methods. This check is a bit imprecise # but it's not too bad if we mistake a random indirect call @@ -112,7 +105,16 @@ CALLTYPE = op.args[0].concretetype if (op.opname == 'indirect_call' and len(op.args) == 2 and CALLTYPE == rclass.OBJECT_VTABLE.instantiate): - return list(self._graphs_of_all_instantiate()) + graphs = list(self._graphs_of_all_instantiate()) + # + if graphs is not None: + result = [] + for graph in graphs: + if is_candidate(graph): + result.append(graph) + if result: + return result # common case: look inside these graphs, + # and ignore the others if there are any # residual call case: we don't need to look into any graph return None Modified: pypy/trunk/pypy/jit/codewriter/test/test_codewriter.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/test/test_codewriter.py (original) +++ pypy/trunk/pypy/jit/codewriter/test/test_codewriter.py Tue Nov 30 18:41:42 2010 @@ -152,6 +152,42 @@ assert 'B2' in names assert 'dont_look' in names +def test_instantiate_with_unreasonable_attr(): + # It is possible to have in real code the instantiate() function for + # a class be dont-look-inside. This is caused by the code that + # initialize the instance attributes: if one attribute has a strange + # type, the whole function is disabled. Check that it still works. + class MyFakePolicy: + def look_inside_graph(self, graph): + name = graph.name + return not (name.startswith('instantiate_') and + name.endswith('A2')) + class A1: + pass + class A2(A1): + pass + def f(n): + if n > 5: + x = A1 + else: + x = A2 + x() + rtyper = support.annotate(f, [35]) + maingraph = rtyper.annotator.translator.graphs[0] + cw = CodeWriter(FakeCPU(rtyper), [FakeJitDriverSD(maingraph)]) + cw.find_all_graphs(MyFakePolicy()) + cw.make_jitcodes(verbose=True) + # + names = [jitcode.name for jitcode in cw.assembler.indirectcalltargets] + assert len(names) == 1 + assert names[0].startswith('instantiate_') and names[0].endswith('A1') + # + print cw.assembler.list_of_addr2name + names = dict.fromkeys([value + for key, value in cw.assembler.list_of_addr2name]) + assert 'A1' in names + assert 'A2' in names + def test_int_abs(): def f(n): return abs(n) From arigo at codespeak.net Tue Nov 30 18:44:15 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 18:44:15 +0100 (CET) Subject: [pypy-svn] r79699 - pypy/trunk/pypy/jit/codewriter Message-ID: <20101130174415.13BE0282BE0@codespeak.net> Author: arigo Date: Tue Nov 30 18:44:13 2010 New Revision: 79699 Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py Log: Oupssvn diff sorry Modified: pypy/trunk/pypy/jit/codewriter/jtransform.py ============================================================================== --- pypy/trunk/pypy/jit/codewriter/jtransform.py (original) +++ pypy/trunk/pypy/jit/codewriter/jtransform.py Tue Nov 30 18:44:13 2010 @@ -198,7 +198,7 @@ self.vable_array_vars[op.result]= self.vable_array_vars[op.args[0]] rewrite_op_cast_pointer = rewrite_op_same_as - rewrite_op_cast_pointer = rewrite_op_same_as # rlib.rerased + rewrite_op_cast_opaque_ptr = rewrite_op_same_as # rlib.rerased def rewrite_op_cast_primitive(self, op): pass def rewrite_op_cast_bool_to_int(self, op): pass def rewrite_op_cast_bool_to_uint(self, op): pass From arigo at codespeak.net Tue Nov 30 18:59:10 2010 From: arigo at codespeak.net (arigo at codespeak.net) Date: Tue, 30 Nov 2010 18:59:10 +0100 (CET) Subject: [pypy-svn] r79700 - pypy/trunk/lib_pypy Message-ID: <20101130175910.B442F282BE7@codespeak.net> Author: arigo Date: Tue Nov 30 18:59:08 2010 New Revision: 79700 Modified: pypy/trunk/lib_pypy/readline.py Log: Bah. Cancel this: it's all bound methods, not functions. Modified: pypy/trunk/lib_pypy/readline.py ============================================================================== --- pypy/trunk/lib_pypy/readline.py (original) +++ pypy/trunk/lib_pypy/readline.py Tue Nov 30 18:59:08 2010 @@ -6,10 +6,4 @@ are only stubs at the moment. """ -import __pypy__ - -import pyrepl.readline -__all__ = pyrepl.readline.__all__ - -for _name in __all__: - globals()[_name] = __pypy__.builtinify(getattr(pyrepl.readline, _name)) +from pyrepl.readline import * From hakanardo at codespeak.net Tue Nov 30 20:29:33 2010 From: hakanardo at codespeak.net (hakanardo at codespeak.net) Date: Tue, 30 Nov 2010 20:29:33 +0100 (CET) Subject: [pypy-svn] r79701 - pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt Message-ID: <20101130192933.22F64282BE0@codespeak.net> Author: hakanardo Date: Tue Nov 30 20:29:30 2010 New Revision: 79701 Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Log: cleanups Modified: pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py ============================================================================== --- pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py (original) +++ pypy/branch/jit-unroll-loops/pypy/jit/metainterp/optimizeopt/unroll.py Tue Nov 30 20:29:30 2010 @@ -12,6 +12,49 @@ opt = UnrollOptimizer(metainterp_sd, loop, optimizations) opt.propagate_all_forward() +class Inliner(object): + def __init__(self, inputargs, jump_args): + assert len(inputargs) == len(jump_args) + self.argmap = {} + for i in range(len(inputargs)): + self.argmap[inputargs[i]] = jump_args[i] + self.snapshot_map = {None: None} + + def inline_op(self, op): + newop = op #.clone() FIXME: move clone here + args = newop.getarglist() + newop.initarglist([self.inline_arg(a) for a in args]) + + if op.is_guard(): + args = newop.getfailargs() + if args: + newop.setfailargs([self.inline_arg(a) for a in args]) + + if newop.result: + old_result = newop.result + newop.result = newop.result.clonebox() + self.argmap[old_result] = newop.result + + descr = newop.getdescr() + if isinstance(descr, ResumeGuardDescr): + descr.rd_snapshot = self.inline_snapshot(descr.rd_snapshot) + + return newop + + def inline_arg(self, arg): + if isinstance(arg, Const): + return arg + return self.argmap[arg] + + def inline_snapshot(self, snapshot): + if snapshot in self.snapshot_map: + return self.snapshot_map[snapshot] + boxes = [self.inline_arg(a) for a in snapshot.boxes] + new_snapshot = Snapshot(self.inline_snapshot(snapshot.prev), boxes) + self.snapshot_map[snapshot] = new_snapshot + return new_snapshot + + class UnrollOptimizer(Optimization): """Unroll the loop into two iterations. The first one will become the preamble or entry bridge (don't think there is a @@ -54,12 +97,7 @@ loop.operations = self.optimizer.newoperations - short = self.create_short_preamble(loop.preamble.operations, - loop.preamble.inputargs, - loop.preamble.token, - loop.operations, - loop.inputargs, - loop.token) + short = self.create_short_preamble(loop.preamble, loop) if short: if False: # FIXME: This should save some memory but requires @@ -114,15 +152,11 @@ def inline(self, loop_operations, loop_args, jump_args): - self.argmap = argmap = {} - assert len(loop_args) == len(jump_args) - for i in range(len(loop_args)): - argmap[loop_args[i]] = jump_args[i] + self.inliner = inliner = Inliner(loop_args, jump_args) for v in self.optimizer.values.values(): v.last_guard_index = -1 # FIXME: Are there any more indexes stored? - self.snapshot_map = {None: None} inputargs = [] seen_inputargs = {} for arg in jump_args: @@ -137,21 +171,9 @@ for newop in loop_operations: #print 'N:', newop if newop.getopnum() == rop.JUMP: - args = inputargs - else: - args = newop.getarglist() - newop.initarglist([self.inline_arg(a) for a in args]) + newop.initarglist(inputargs) + newop = inliner.inline_op(newop) - if newop.result: - old_result = newop.result - newop.result = newop.result.clonebox() - argmap[old_result] = newop.result - #print 'P:', newop - - descr = newop.getdescr() - if isinstance(descr, ResumeGuardDescr): - descr.rd_snapshot = self.inline_snapshot(descr.rd_snapshot) - self.optimizer.first_optimization.propagate_forward(newop) # Remove jump to make sure forced code are placed before it @@ -175,7 +197,7 @@ if not isinstance(a, Const) and not a in boxes_created_this_iteration: if a not in inputargs: inputargs.append(a) - box = self.inline_arg(a) + box = inliner.inline_arg(a) if box in self.optimizer.values: box = self.optimizer.values[box].force_box() jumpargs.append(box) @@ -184,19 +206,6 @@ self.optimizer.newoperations.append(jmp) return inputargs - def inline_arg(self, arg): - if isinstance(arg, Const): - return arg - return self.argmap[arg] - - def inline_snapshot(self, snapshot): - if snapshot in self.snapshot_map: - return self.snapshot_map[snapshot] - boxes = [self.inline_arg(a) for a in snapshot.boxes] - new_snapshot = Snapshot(self.inline_snapshot(snapshot.prev), boxes) - self.snapshot_map[snapshot] = new_snapshot - return new_snapshot - def sameop(self, preambleop, loopop): #if preambleop.getopnum() != loopop.getopnum(): # return False @@ -209,17 +218,19 @@ except KeyError: return False - def create_short_preamble(self, preamble, preambleargs, preamble_token, - loop, inputargs, token): + def create_short_preamble(self, preamble, loop): #return None # Dissable + preamble_ops = preamble.operations + loop_ops = loop.operations + state = ExeState() short_preamble = [] loop_i = preamble_i = 0 - while preamble_i < len(preamble)-1: - op = preamble[preamble_i] - if self.sameop(op, loop[loop_i]) \ - and loop_i < len(loop)-1: + while preamble_i < len(preamble_ops)-1: + op = preamble_ops[preamble_i] + if self.sameop(op, loop_ops[loop_i]) \ + and loop_i < len(loop_ops)-1: loop_i += 1 else: if not state.safe_to_move(op): @@ -231,23 +242,22 @@ state.update(op) preamble_i += 1 - - if loop_i < len(loop)-1: + if loop_i < len(loop_ops)-1: debug_print("create_short_preamble failed due to", "loop contaning ops not in preamble" "at position", loop_i) return None - jumpargs = [None] * len(inputargs) - allboxes = preambleargs[:] + jumpargs = [None] * len(loop.inputargs) + allboxes = preamble.inputargs[:] for op in short_preamble: if op.result: allboxes.append(op.result) for result in allboxes: - box = self.inline_arg(result) - for i in range(len(inputargs)): - b = inputargs[i] + box = self.inliner.inline_arg(result) + for i in range(len(loop.inputargs)): + b = loop.inputargs[i] if self.optimizer.getvalue(box) is self.optimizer.getvalue(b): jumpargs[i] = result break @@ -259,12 +269,12 @@ return None jmp = ResOperation(rop.JUMP, jumpargs[:], None) - jmp.setdescr(token) + jmp.setdescr(loop.token) short_preamble.append(jmp) # Check that boxes used as arguemts are produced. seen = {} - for box in preambleargs: + for box in preamble.inputargs: seen[box] = True for op in short_preamble: for box in op.getarglist(): @@ -282,8 +292,8 @@ op = short_preamble[i] if op.is_guard(): op = op.clone() - op.setfailargs(preambleargs) - op.setjumptarget(preamble_token) + op.setfailargs(preamble.inputargs) + op.setjumptarget(preamble.token) short_preamble[i] = op return short_preamble @@ -353,25 +363,11 @@ def inline(self, loop_operations, loop_args, jump_args, dryrun=False): - self.argmap = argmap = {} - assert len(loop_args) == len(jump_args) - for i in range(len(loop_args)): - argmap[loop_args[i]] = jump_args[i] + inliner = Inliner(loop_args, jump_args) for op in loop_operations: - newop = op.clone() - args = newop.getarglist() - newop.initarglist([self.inline_arg(a) for a in args]) - - if op.is_guard(): - args = newop.getfailargs() - newop.setfailargs([self.inline_arg(a) for a in args]) + newop = inliner.inline_op(op.clone()) - if newop.result: - old_result = newop.result - newop.result = newop.result.clonebox() - argmap[old_result] = newop.result - if not dryrun: self.emit_operation(newop) else:
  • +
    Download PyPy